Buffer pool performance improvements
How the LRU replacement and log checkpoints were made faster in MariaDB 10.5
- Track: MariaDB devroom
- Room: D.mariadb
- Day: Saturday
- Start: 17:30
- End: 18:00
- Video with Q&A: D.mariadb
- Video only: D.mariadb
- Chat: Join the conversation!
In any block-oriented, update-in-place database with log-based recovery, two kinds of page flushing are needed. Eviction flushing is necessary when all least recently used blocks are dirty (modifications exist in the buffer pool). Checkpoint flushing (writing out the oldest modified page first) allows the log checkpoint to advance, shortening the potential recovery time by logically discarding the start of the log. Any write of a persistent page must be preceded by a corresponding log write and optional doublewrite, to guarantee that crash recovery works.
We will cover some improvements in this area in MariaDB Server's version of InnoDB storage engine. We will also discuss how a database could be configured to minimize write amplification.
MariaDB 10.5 simplified page flushing and fixed bottlenecks that had originally been worked around by introducing multiple buffer pool instances and multi-threaded flushing. It turns out that a single buffer pool and page flushing thread (assisted by a log flushing thread) can still saturate most contemporary I/O subsystems. Data structures were simplified, some synchronous writes replaced with asynchronous ones, and mutex contention reduced through more use of atomic memory operations.
Speakers
| Marko Mäkelä |
Attachments
Links
- refactoring the LRU block replacement algorithm
- refactoring the log checkpointing
- simplifying buffer pool data structures
- cache-friendlier buf_pool.page_hash table with custom rw-lock
- avoid writing freed pages
- defer writes to temporary tablespace
- buffer pool implementation
- Video recording (WebM/VP9)
- Video recording (mp4)
- Submit feedback