Using Mirror Maker 2 with IBM Event Streams to share topics across multiple regions

April 5th, 2024

This is the third in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters, but sometimes I feel like this can be forgotten if we only talk about it in the context of disaster recovery.

In these posts, I want to inspire you to think about other ways you could use Mirror Maker 2. The best way to learn about what is possible is to play with it for yourself, so with these posts I’ll include a script to create a demonstration of the scenario.

For this third post, I’ll look at using Mirror Maker to create logical topics shared across multiple regions.

Read the rest of this entry »

How ML for Kids handles numbers models now

April 4th, 2024

I’m on vacation for the Easter holidays this week. Apparently I don’t know how vacations work, so I’ve spent a lot of the last six days working on a major rewrite of a big chunk of Machine Learning for Kids. In this post, I want to describe what I’ve been doing and why.

Read the rest of this entry »

Using Mirror Maker 2 with IBM Event Streams to broadcast events to multiple regions

April 2nd, 2024

This is the second in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters, but sometimes I feel like this can be forgotten if we only talk about it in the context of disaster recovery.

In these posts, I want to inspire you to think about other ways you could use Mirror Maker 2. The best way to learn about what is possible is to play with it for yourself, so with these posts I’ll include a script to create a demonstration of the scenario.

For this second post, I’ll look at using Mirror Maker to broadcast events to clusters in multiple regions.

Where the last post described a “fan in” scenario, this is effectively the opposite use case: a “fan out” scenario.

Read the rest of this entry »

Using Mirror Maker 2 with IBM Event Streams to aggregate events across regions

March 30th, 2024

This is the first in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters, but sometimes I feel like this can be forgotten if we only talk about it in the context of disaster recovery.

In these posts, I want to inspire you to think about other ways you could use Mirror Maker 2. The best way to learn about what is possible is to play with it for yourself, so with these posts I’ll include a script to create a demonstration of the scenario.

For this first post, I’ll look at using Mirror Maker to aggregate events across clusters in multiple regions.

Read the rest of this entry »

Accelerating your event-driven projects with AsyncAPI 3

March 21st, 2024

Demonstrating some of the first tools to support AsyncAPI v3, and how they can help accelerate your event-driven projects.

AsyncAPI version 3 was released in December 2023. In this post, I want to share some of the things that you can do with AsyncAPI v3 documents, with a demonstration of:

  • Event Endpoint Management
    Download AsyncAPI documents for Kafka topics shared for reuse
    AsyncAPI v3 support added in version 11.1.1 (Dec 2023)
  • AsyncAPI java-template
    Generate complete Java projects from AsyncAPI documents
    AsyncAPI v3 support added in version 0.3.0 (Mar 2024)
  • Microcks
    Generate Kafka topics with mock data from AsyncAPI documents
    AsyncAPI v3 support added in version 1.9.0 (Mar 2024)

Read the rest of this entry »

Getting demo events onto IBM Event Streams topics in a hurry

March 13th, 2024

Sharing a couple of tips for quick-and-dirty demo setups.

I often need to put together demos of IBM Event Automation without much notice. The starting point is almost always needing to get a bunch of interesting events onto a Kafka topic.

What I need is a jumping-off point to illustrate the benefit of sharing streams of events in Event Endpoint Management, or the types of processing you can do in Event Processing. And to do that, I need a topic with events on them that will look interesting or relevant to who I’m demo’ing to.

If I’ve got time to do this properly, I’ll setup a generator that will give me a continuous stream of randonly-generated events (example). But if I’m in a hurry, I’ll use the REST Producer API and do something like this instead.

Read the rest of this entry »

Explaining regression in Scratch

February 15th, 2024

In this post, I want to share a preview of a new feature I’m adding to Machine Learning for Kids, to ask for feedback and ideas for projects that it could be used to make.

I’ll start by contrasting this new feature with what I’ve done with Machine Learning for Kids before, then I’ll share screenshots of the new feature, and finally I’ve got a ten-minute video showing the sort of school lesson that I think it could be used for.

Read the rest of this entry »

How to create a new Kafka Connect connector

January 28th, 2024

I’m helping with a hackathon this week to get developers to create their first Kafka Connect connectors. To help get them going, I’ll be starting the day with a quick crash course.

I’ve created a simple skeleton connector project, and I’ll be walking through how they can use that to skip the boilerplate stuff and jump straight into trying out their ideas for new connectors.

Read the rest of this entry »