In the first part of the talk we'll look at what a search engine like Algolia or Elasticsearch does, how tantivy works, what things we need to do in order to make it suitable for user-facing search and how we implement it into our code to build our own application.

In the second part of the talk I talk about the biggest issue you run into when building a search engine or any other system designed to store data... Fault tolerance and replication, together we'll go through the challenges of implementing this for a search engine using tantivy, keeping the performance competitive, reliable and the tradeoffs that are made to make it work. We'll look at some existing implementations like Raft and take a dive into the world of eventual consistency!

With any luck, you'll have a better understanding of how we can build services like search engine using libraries like tantivy along with solving technical challenges like consensus and replication for distributed systems in Rust.