In this post, I’ll show how to bring posts from open social media networks (Bluesky and Mastodon) into Kafka using Kafka Connect source connectors.
My goal is to be able to populate a Kafka topic with status updates posted to social media.
Rather than to try and do this with the full firehose of all status updates, this is done with status updates that match a search term or hashtag.
For example, the screenshot above is a Kafka topic with posts from Bluesky that mention the term “xbox”.
In this post, I’ll demonstrate how Event Processing can use parameters from an external source (such as a rules engine) in event processing flows.
To illustrate the idea, I created a simple demo event processing flow. The flow takes a stream of order events, filters it to keep only orders for high value items, and then modifies the description property in some of the events:
The filter node is comparing the price with “40”, so only order events for items with a value above $40 are kept.
The transform node is modifying the description property of order events – any description that contains the string “Cargo Jeans” is replaced with “Combat Trousers”.
What if you wanted to modify the threshold for the filter, to change that $40 minimum value for an order to be considered “large”?
Or what if you wanted to modify the transformation, so that different strings would be used in the regular expression replacement?
With the values hard-coded in the flow as shown above, you would need to:
This is a workable approach, although it does require a little downtime and some administrative effort.
The aim for this post is to highlight an alternative approach.
MATCH_RECOGNIZEI play a lot of video games. That includes a lot of modern games, but I also still love going back to the retro games of my childhood. There are a lot of fun things from that era of video games that I love.
For example, cheat codes. You’d press a specific sequence of buttons on the game controller at a specific time to unlock some “secret” bit of content – like special abilities, special resources, or levels.
Some of these are so ingrained in me now that my fingers just know how to enter them without thinking. The level select cheat for Sonic the Hedgehog is the best example of this: press UP, DOWN, LEFT, RIGHT, START + A during the title screen to access a level select mode that would let you jump immediately to any part of the game.
level select cheat code for Sonic the Hedgehog
With this in the back of my head, it’s perhaps no surprise that when I needed to explain pattern recognition in Apache Flink, the metaphor I thought of first was how games of yesteryear could recognize certain button press sequences.
If you think of each button press on the game controller as an event, then recognizing a cheat code is just a pattern of events to recognize.
And once I thought of the metaphor – I had to build it. 🙂
There is more detail on how I built this in the git repository, but this is the overall idea for what I’ve made.
Event Processing (one of the new capabilities of IBM Event Automation) makes it easy to perform stateful processing of streams of events from Kafka topics. In this post, I want to drill down a little into how windowed processing behaves.
I’ve enjoyed the chance to see the solutions that people have started to build with the tool. As part of this, I’ve been helping several people to understand the results produced by the event processing flows they’ve made.
These often started with different questions, such as:
However, these are often symptoms of a single common question: how windowed processing operations behave.
I’ve tried to come up with simple ways to demonstrate how it works, so in this post I want to share how I’m currently explaining it.
This is IBM Event Automation : a new product we released last month to help our clients create event driven solutions.
I’ve written a 200-word summary of what IBM Event Automation is, but in this post I wanted to dive a little bit deeper and show what it can do.
A summary of IBM Event Automation in under 200 words.
Last week was the first release of IBM Event Automation. I’ve been asked what it’s all about, so I thought it’d be helpful to have a brief summary to point people at.
IBM Event Automation is a collection of three components, designed to help companies be productive in creating event-driven solutions.