Archive for March, 2021

A Kafka Developer’s Guide to AsyncAPI

Tuesday, March 30th, 2021

How Kafka developers can use the AsyncAPI specification to describe how their applications are using Kafka topics.

In my post “Why should you document your Kafka topics?” last week, I wrote about the benefits of documenting your Kafka event sources, and mentioned a few of the problems that this can help with.

In this post, I want to show you how you can document the API for your Kafka event sources by creating AsyncAPI documents.

You don’t necessarily have to learn the AsyncAPI specification – tools such as the new Event Endpoint Management capability that I work on in Cloud Pak for Integration make it easy to document APIs with user-friendly forms that generate AsyncAPI documents for you. However, some developers will want to know more about what is happening under the covers, so here is an introduction.


youtu.be/Ni5tCY9r0TY

(more…)

Migrating your Apache Kafka cluster using MirrorMaker 2

Wednesday, March 24th, 2021

You have a Kafka cluster that you have been using for a while. Your cluster has many topics, and the topics have many messages.

Now you’ve decided to move and start using a new, different Kafka cluster somewhere else.

How can you take your topics with you?

Huge thanks to Andrew Borley for co-writing this with me. Useful insights in here probably came from him, the mistakes from me.

(more…)

AsyncAPI plugin for Node-RED

Sunday, March 21st, 2021

screenshot

I’ve been tinkering with a new AsyncAPI plugin for Node-RED as a side project over the last couple of weeks. Time to share what I’ve got working so far.

Node-RED is an open-source visual programming tool. You assemble flows on a canvas from a palette of nodes, that you customize and then wire together. That makes it ideal for quick prototyping.

There are nodes for different types of servers and devices, which makes it great for quick integration projects. This includes nodes for sending and receiving messages using protocols like Kafka and MQTT.

Normally, this means having to choosing a node from the palette, dragging it onto the canvas, clicking into it, and customizing it: filling in the connection details for the broker and topic you want to use.

The idea of this plugin is to do all of that for you, if you have an AsyncAPI specification for your topics.

The plugin can generate and customize the nodes for you, based on your spec – making rapid prototyping based on the spec even quicker.

(more…)