In this post, I introduce a few core Flink SQL functions using worked examples of processing a stream of click tracking events from a retail website.
I find that a practical, real-world (ish) example can help to explain how to use Flink SQL in a way that abstract descriptions, such as processing coloured blocks sometimes doesn’t quite achieve.
I’ll use this post to give examples of my most-used Flink SQL functions, in the context of a retail scenario: a stream of events from customers on the website for a clothing retailer.
Note: I used Event Processing to create the flows, as the assistants in the canvas helped me create examples quickly. Everything I’ve created is standard Apache Flink SQL, so you don’t need to have Event Processing to try these examples.
- The examples:
- 0 Consuming Avro – bring click tracking events into Flink
- 1 Transforming – deriving new properties
- 2 Joining – correlating with related event streams
- 3 Aggregating (tumble) – counting in a tumble window
- 4 Aggregating (session) – counting in a session window
- 5 Aggregating (session) – collecting in a session window
- Data – the events I’m processing in these examples
- Setup – how to recreate this if you want to try this for yourself