Serialization

msgspec works as a faster JSON or MessagePack library on its own. Still, its docs recommend it for the full serialization and validation workflow: define your schemas with type annotations, encode them, then decode them back with validation. Structs are the preferred way to define those types. Pass the type when you decode, and msgspec validates while decoding at no added runtime cost: msgspec.json.decode(data, type=User). A consistent interface covers MessagePack, YAML, and TOML too.

Moving from the standard json module to orjson? The largest difference is that orjson.dumps returns bytes, not str. orjson serializes dataclasses, datetimes, and UUIDs natively. Decoding gives you only dicts, lists, and other builtins, and orjson leaves schemas to validation libraries a level above.

msgpack reads and writes MessagePack, a binary format that lets you exchange data among languages like JSON, but faster and smaller. Call packb and unpackb for one-shot use, and read many objects from one stream with an Unpacker. When the data comes from an untrusted source, set max_buffer_size to limit the buffer.

marshmallow makes no assumption about your web framework or database layer, so its schemas work with just about any ORM, or none. Declare a schema as a class that maps attribute names to fields. Its dump method turns your objects into primitive Python types, and load validates and deserializes incoming data, raising ValidationError on invalid input. To get objects back instead of dicts, decorate a schema method with post_load.

cloudpickle is built for cluster computing, where Python code is shipped over the network to run on remote hosts. It pickles lambdas, plus functions and classes you define in a script, a shell, or a Jupyter notebook, and the result loads back with the standard pickle.loads. Use it only between the exact same version of Python, never for long-term storage.

dill is a drop-in replacement for pickle: write import dill as pickle, and nested functions, lambdas, and more exotic types pickle too. Call dill.dump_module() to save your interpreter session to a file, and dill.load_module() to restore it later. dill isn't meant to be secure against maliciously constructed data, and no pickle is. Never load one from a source you don't trust: use a data format instead.

orjson, msgspec, and msgpack all pitch speed, but msgspec's own benchmark page encourages you to write your own benchmarks before deciding.