Data streaming is gaining popularity, as it offers decreased latency, a radically simplified data infrastructure architecture, and the ability to cope with new data that is generated continuously. Apache Flink is a full-featured stream processing framework with:

• Easy to use Java- and Scala-embedded APIs that make stream analytics easy, yet provide powerful tools to deal with time and uncertainty
• Throughput of million of events per second per core
• Latencies as low as the millisecond range
• Exactly-once consistency guarantees, and the ability to realize distributed transactional data movement between systems (e.g., between Kafka and HDFS)
• Ease of configuration and separation between application logic and fault tolerance via a novel asynchronous checkpointing algorithm
• No single point of failure
• Integration with popular open source infrastructure (e.g., Hadoop, HBase, Kafka, Cascading, Elasticsearch, …)
• Support for event time and out of order arrivals with flexible windows, watermarks, and triggers
• Batch processing as a special case of stream processing, including dedicated libraries for machine learning and graph processing, managed memory on-, and off-heap, and query optimization

Flink is used in several companies, including at ResearchGate, Bouygues Telecom, the Otto Group, and Capital One, and has a large and active developer community of well over 140 contributors. In this talk, we provide an overview of the system and its streaming-first philosophy, as well as the project roadmap and vision: fully unifying the, now separate, worlds of “batch” and “streaming” analytics.