Help I need a stream processor - learning how to chose between Spark, Flink, Samza, and Storm

06/07/2016 - 16:30 to 17:10
Frannz Club
long talk (40 min)

Session abstract: 

Today if a byte of data were a gallon of water, in only 10 seconds there would be enough data to fill an average home, in 2020 it will only take 2 seconds. With this explosive growth comes the demand from consumers and businesses to leverage and act on what is happening right now. Without stream processing these demands will never be met, and there will be no big data and no Internet of Things. It is only a matter of time before you will be faced with building a real-time streaming pipeline. As soon as you embark on this journey, you will be faced with a myriad of questions. A major key decision you will need to quickly answer is which stream-processing framework should you use? When you survey the landscape you will find many contenders. In this session we will focus on the most popular open source frameworks, in particular: Apache Spark Streaming, Apache Storm, Apache Flink, and Apache Samza. We will dive into each of these tools and tease out all of the essential pieces you need to consider, compare and contrast them and end up with an understanding of how to evaluate each as well as future products.