Brussels / 4 & 5 February 2023


Accelerating object serialization by using constraints

How we achieved 3x-100x faster data serialization to a binary format or to JSON using low-level Cython and Python C API.

The talk will demonstrate how doing less can result in doing more: how domain-specific data constraints may lead to dramatic speedups in serialization. There will be two examples: binary caching of numpy object arrays (Pandas) and generating json web responses from @dataclass-es. We will also cover the revolution of using C++17 in Cython, custom arena memory allocators based on mimalloc, and gory internals of CPython.


Photo of Vadim Markovtsev Vadim Markovtsev