Discreet Cosine Transform

A few thoughts on ruby, video and other things

Marshal Ruby Objects to Riak and Back

Thought I would share a little bit of work I have been messing around with to marshal ruby objects to the Riak NoSQL store and back. There is Ripple, a library for exactly this purpose, but it implements the ActiveRecord model and that is not exactly what I wanted.

riak-client

Riak client supports automatic serialization based on what you set as an object’s content-type. So for example, the follow code would automatically store this Ruby hash as JSON in Riak:

1
2
3
4
5
6
client = Riak::Client.new(:protocol => "pbc")
bucket  = client.bucket("my_bucket")
new_one = Riak::RObject.new(bucket, "hash_object.json")
new_one.content_type = "application/json"
new_one.data = one
new_one.store

The difference here is in the use of #data and not #raw_data. There is a way to define your own serializers for different content-types, but that seems to be undocumented by the riak-client team for now. By default it supports both YAML and JSON.

That will work fine for the typical JSON data types such as Hash and Array. But if you want to store a more complex object, and get it back, you will need a little more.

Using the JSON Gem

riak-client uses multi_json as you might expect. So, if you have the JSON gem installed or loaded, you can use their method of storing a ruby object and marshaling it back from JSON.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
class ComplexObject
  attr_accessor :place,:time,:characters

  def initialize(stuff={})
    @place = stuff[:place]
    @time = stuff[:time]
    @characters = stuff[:characters]

  end

  #we have to define our own #to_json and self.json_create methods
  def to_json(*a)
    {'json_class'=>self.class.name,'place'=>@place,'time'=>@time,'characters'=>@characters}.to_json(*a)
  end

  def self.json_create(data)
    new(:place=>data['place'],:time=>data['time'],:characters=>data['characters'])
  end
end


one = ComplexObject.new(:place=>'woods',:time=>'tomorrow',:characters=>'burt')


client = Riak::Client.new(:protocol => "pbc")
bucket = client.bucket("my_bucket")


#in case I have other JSON libraries loaded that MultiJson would favor
MultiJson.use :json_gem

#this is a key, as it does not default to true 
#and without it the JSON gem won't call json_create
MultiJson.load_options = {:create_additions=>true}


new_one = Riak::RObject.new(bucket, "complex_object.json")
new_one.content_type = "application/json"
new_one.data = one
new_one.store

#viola, you get back a new object, not just a Hash
puts client['my_bucket']['complex_object.json'].data

Using Oj

It appears Oj has even more innate support for serializing and deserializing ruby objects to JSON, but I can’t say I am deeply familiar with Oj yet so take that with a grain of salt.

Here is the example above using Oj:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
class ComplexObject
  attr_accessor :place,:time,:characters

  def initialize(stuff={})
    @place = stuff[:place]
    @time = stuff[:time]
    @characters = stuff[:characters]

  end

  #with oj, no need for our own serialize and deserialize methods

end

one = ComplexObject.new(:place=>'woods',:time=>'tomorrow',:characters=>'burt')

client = Riak::Client.new(:protocol => "pbc")
bucket = client.bucket("my_bucket")

#MultiJson would default to oj if present, just being explicit here
MultiJson.use = :oj

#we want oj's object mode to do its automatic serialization
MultiJson.load_options = {:mode=>:object}
MultiJson.dump_options = {:mode=>:object}

#Store a ruby hash as json in riak

new_one = Riak::RObject.new(bucket, "complex_object.json")
new_one.content_type = "application/json"
new_one.data = one
new_one.store

#viola, again, the full ruby object
puts client['my_bucket']['complex_object.json'].data

Comments