In this post, I’ll share how you can get your own Event Endpoint Management demo instance with just seven minutes of work.
(Seven minutes of hands-on-keyboard time… there is a lot of waiting-for-stuff-to-run time, but it doesn’t sound so impressive if I include waiting time!)
In this post, I want to suggest some approaches for introducing event-driven architecture patterns into your existing application environment. I’ll demonstrate how you can incrementally adopt Apache Kafka without needing to immediately build new applications or rebuild your existing applications, and show how this can be delivered in Red Hat OpenShift.
As part of AsyncAPI Conference this week, I ran a session on how to describe Kafka security in AsyncAPI.
The aim of the session was to quickly show how to describe the security configuration of a Kafka cluster in an AsyncAPI document.
And, in reverse, if you’ve been given an AsyncAPI document, to show how to use that to configure a Kafka client or application to connect to the cluster, using the details in the AsyncAPI spec.
The recording and the slides I used are below.
IBM App Connect Enterprise (ACE) is a broker for developing and hosting high-throughput, high-scale integrations between a large number of applications and systems, including Apache Kafka.
In this post, I’ll describe how to use App Connect Enterprise to process Kafka messages that were serialized to a stream of bytes using Apache Avro schemas.
Best practice when using Apache Kafka is to define Apache Avro schemas with a definition of the structure of your Kafka messages.
(For more detail about this, see my last post on From bytes to objects: describing Kafka events, or the intro to Avro that I wrote a couple of years ago.)
In this post, I’m assuming that you have embraced Avro, and you have Kafka topics with messages that were serialized using Avro schemas.
Perhaps you used a Java producer with an Avro SerDe that handled the serialization automatically for you.
Or your messages are coming from a Kafka Connect source connector, with an Avro converter that is handling the serialization for you.
Or you are doing the serialization yourself, such as if you’re producing Avro-serialized messages from a Python app.
Now you want to use IBM App Connect Enterprise to develop and host integrations for processing those Kafka messages. But you need App Connect to know how to:
The recording of the talk that Kate Stanley and I gave at Kafka Summit Americas is now available.
Events stored in Kafka are just bytes, this is one of the reasons Kafka is so flexible. But when developing a producer or consumer you want objects, not bytes. Documenting and defining events provides a common way to discuss and agree on an approach to using Kafka. It also informs developers how to consume events without needing access to the developers responsible for producing events.
In our talk, we introduced the most popular formats for documenting events that flow through Kafka, such as AsyncAPI, Avro, CloudEvents, JSON schemas, and Protobuf.
We discussed the differences between the approaches and how to decide on a documentation strategy. Alongside the formats, we also touched on the tooling available for the different approaches. Tools for testing and code generation can make a big difference to your day-to-day developer experience.
The talk was aimed at developers who maybe aren’t already documenting their Kafka events or who wanted to see other approaches.
I’ve added a pitch extraction machine learning model to Machine Learning for Kids today. In this post, I want to describe the model a little, and suggest a few ways that students could use it.
I started adding pretrained machine learning models to Machine Learning for Kids last year. Although my main focus is still allowing students to create their own machine learning models and make things with them, there are some fun projects that can be made using models that are too complex for students to train by themselves.
imagenet (that I added last Christmas), and the question-answering model (that I added in April) are both good examples of that!
I hope this one will be similarly welcomed!
The new model is a pitch estimation model. Given some audio as input, you can use it to recognize the dominant pitch in sung audio (even if there is background music and noise).
I was lucky enough to be invited onto a couple of podcasts this month to talk about Machine Learning for Kids.
Read the rest of this entry »
I’ve made a Kafka Connect source connector for sending “real-time” (not really) stock price events to a Kafka topic.
Choose your stock (e.g. “IBM“) and run the Connector pointed at your topic, and every minute a JSON event will be produced like:
{"open":137.9,"high":137.9,"low":137.9,"close":137.9,"volume":500,"timestamp":1629421200,"datetime":"2021-08-19 20:00:00"}