Event Sourcing brings the promise of highly-scalable and loosely-coupled systems that are performant, reliable, and maintainable. It looked like a perfect solution for Yammer's reliability and performance challenges, but nothing comes for free!
Only slightly over a year ago, Yammer's entire system was either based on synchronous calls or Ruby workers and RabbitMQ. For a while, we have been moving performance-critical components out of the Ruby on Rails monolith toward Dropwizard-based services. This has served us well, but with increased reliability requirements, the pressure to simplify and decouple our system's architecture also increased.
Over the course of the last year, we first created a prototype implementation of Event Sourcing and, after validating the idea, have been moving it to a managed Azure Event Hubs based solution.
We are going to cover not only how to migrate to a new technology, but also look at how to change organizational thinking from a synchronous world to one of event streams. We will tell the war stories of our migration from a self-hosted Kafka cluster to a solution based in the cloud.