Utility classes to define optimized and evolutive serializers.
Paquito provides utility classes to define optimized and evolutive serializers.
Add this line to your application’s Gemfile:
gem 'paquito'
And then execute:
$ bundle install
Or install it yourself as:
$ gem install paquito
Paquito being a serialization library, takes extra care in always being able to deserialize payloads serialized from previous versions.
However the inverse may not always be true, so when upgrading Paquito it is essential to first upgrade the gem without any applicative code change, so
that all Ruby processes in production are able to read the new format.
Additionally format changes can be controlled through Paquito.format_version that directly maps with the gem major version.
For example, paquito 0.10.0 introduce a new serialization format for Time and DateTime objects, but retain Paquito.format_version = 0.
The upgrade process is as follows:
paquito ~> 0.10.Paquito.format_version = 1 or upgrade to paquito ~> 1.0.Generally speaking it’s heavily recommended to carefully read the CHANGELOG and not to skip intermediary versions.
chainPaquito::CoderChain combines two or more serializers into one.
Example:
compressed_yaml_coder = Paquito.chain(YAML, Zlib)
payload = compressed_yaml_coder.dump({ foo: 42 }) # YAML compressed with gzip
compressed_yaml_coder.load(payload) # => { foo: 42 }
ConditionalCompressorPaquito::ConditionalCompressor compresses payloads if they are over a defined size.
Example:
coder = Paquito::ConditionalCompressor.new(Zlib, 256)
coder.dump("foo") # => "\x00foo"
coder.dump("foo" * 500) # => "\x01<compressed-data....>"
SingleBytePrefixVersionPaquito::SingleBytePrefixVersion prepends a version prefix to the payloads, which allows you to seamlessly transition
between different serialization methods.
The first argument is the current version used for newly generated payloads.
Example:
coder = Paquito::SingleBytePrefixVersion.new(1,
0 => YAML,
1 => JSON,
2 => MessagePack,
)
coder.dump([1]) # => "\x01[1]"
coder.load("\x00---\n:foo: 42") # => { foo: 42 }
SingleBytePrefixVersionWithStringBypassWorks like Paquito::SingleBytePrefixVersion except that versions 253, 254 and 255 are reserved for serializing strings
in an optimized way.
When the object to serialize is an UTF-8, ASCII or BINARY string, rather than invoking the underlying serializer, it simply
prepends a single byte to the string which indicates the encoding.
Additionally, you can pass a distinct serializer for strings only:
Example:
coder = Paquito::SingleBytePrefixVersionWithStringBypass.new(
1,
{ 0 => YAML, 1 => JSON },
Paquito::ConditionalCompressor.new(Zlib, 1024), # Large strings will be compressed but not serialized in JSON.
)
The larger the string the larger the speed gain is, e.g. for a 1MB string, it’s over 500x faster than going through MessagePack or Marshal.
CommentPrefixVersionSimilar to the single byte prefix, but meant to be human readable and to allow for migrating unversioned payloads.
Payloads without a version prefix are assumed to be version 0.
The first argument is the current version used for newly generated payloads.
Example:
coder = Paquito::CommentPrefixVersion.new(1,
0 => YAML,
1 => JSON,
)
coder.load("---\n:foo: 42") # => { foo: 42 }
coder.dump([1]) # => "#☠1☢\n[1]"
allow_nilIn some situations where you’d rather not serialize nil, you can use the Paquito.allow_nil shorthand:
coder = Paquito.allow_nil(Marshal)
coder.dump(nil) # => nil
coder.load(nil) # => nil
TranslateErrorsIf you do need to handle serialization or deserialization errors, for instance to fallback to acting like a cache miss,
Paquito::TranslateErrors translates all underlying exceptions into Paquito::Error descendants.
Example:
coder = Paquito::TranslateErrors.new(Paquito::CoderChain.new(YAML, Zlib))
coder.load("\x00") # => Paquito::UnpackError (buffer error)
CodecFactoryPaquito::CodecFactory is a utility facade to create advanced MessagePack factories with support for common Ruby
and Rails types.
Example
coder = Paquito::CodecFactory.build([Symbol, Set])
coder.load(coder.dump(%i(foo bar).to_set)) # => #<Set: {:foo, :bar}>
TypedStructPaquito::TypedStruct is a opt-in Sorbet runtime plugin that allows T::Struct classes to be serializable. You need
to explicitly include the module in the T::Struct classes that you will be serializing. Also, Paquito must be added after Sorbet in your Gemfile for this constant to be available.
Example
class MyStruct < T::Struct
include Paquito::TypedStruct
prop :foo, String
prop :bar, Integer
end
my_struct = MyStruct.new(foo: "foo", bar: 1)
my_struct.as_pack # => [26450, "foo", 1]
MyStruct.from_pack([26450, "foo", 1]) # => <MyStruct bar=1, foo="foo">
paquito doesn’t depend on rails or any of its components, however it does provide some optional utilities.
CacheEntryCoderPaquito::CacheEntryCoder turns an ActiveSupport::Cache::Entry instance into a simple Array instance. This allows
you to implement custom coders for ActiveSupport::Cache.
Example:
ActiveSupport::Cache::FileStore.new("tmp/cache", coder: Paquito.chain(Paquito::CacheEntryCoder, JSON))
FlatCacheEntryCoderPaquito::FlatCacheEntryCoder is a variation of Paquito::CacheEntryCoder. Instead of encoding ActiveSupport::Cache::Entry
into an Array of three members, it serializes the entry metadata itself and adds it as a prefix to the serialized payload.
This allows to leverage Paquito::SingleBytePrefixVersionWithStringBypass effectively.
Example:
ActiveSupport::Cache::FileStore.new(
"tmp/cache",
coder: Paquito::FlatCacheEntryCoder.new(
Paquito::SingleBytePrefixVersionWithStringBypass.new(
1,
0 => Marshal,
1 => JSON,
)
)
)
SerializedColumnPaquito::SerializedColumn allows you to decorate any encoder to behave like Rails’s builtin YAMLColumn
Example:
class Shop < ActiveRecord::Base
serialize :settings, Paquito::SerializedColumn.new(
Paquito::CommentPrefixVersion.new(
1,
0 => YAML,
1 => Paquito::CodecFactory.build([Symbol]),
),
Hash,
attribute_name: :settings,
)
end
Shop.new.settings # => {}
After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/Shopify/paquito.
The gem is available as open source under the terms of the MIT License.