Using Xbox to get started with Kafka Connect & Kafka Streams

It’s easy for developers who aren’t immersed in all-things-Kafka to assume that “Apache Kafka” just means an event backbone: something that hosts topics (and perhaps the client libraries to produce and consume messages using those topics). But Kafka is more than that. It is an ecosystem of tools that enables a complete event-streaming application.

That was the premise of this talk, recorded at Devoxx UK, which I gave to a room of Java developers. I introduced them to two other bits of Kafka: Kafka Connect (for getting data in and out of Kafka topics from external systems) and Kafka Streams (for developing stream processing applications).

Because they were Java developers, I thought the best way to give them a flavour of these tools was to show them the APIs, and walk through an example solution made using the APIs.

The example solution used Kafka tools to process data from Xbox – mostly because I’m a gamer and it made for a fun, if silly, demo.

recording of the talk at


Source code:

Note: I gave a similar-but-different talk at Kafka Summit London today.

That was a very different audience (attendees at Kafka Summit know what Connect and Streams are, whereas for Devoxx I was introducing Java developers to parts of the Kafka ecosystem that they weren’t familiar with). Although I used some of the same demo, the talk was fairly different as a result. If you want to see a version of the talk aimed at a Kafka audience, the recording and the slides for that are here.

Tags: , , ,

Comments are closed.