Archive for the ‘code’ Category

« Older Entries
Newer Entries »

Using books data in Scratch

Sunday, May 19th, 2024

In this post, I want to share a Scratch extension that I’ve been working on today: enabling access to books data from the OpenLibrary API through new Scratch blocks.

Most of the work I do on Machine Learning for Kids involves adding machine learning models into Scratch. To enable students to create interesting projects, it also helps to make it easier to get external data into Scratch that they can use for training and classifying. A few examples of where I’ve done this in the past include creating Scratch blocks to access weather data, data from Spotify, and data from Wikipedia.

New blocks

The new blocks I’ve worked on today use the OpenLibrary API to enable access to information about books.

(more…)

Tags:,
Posted in code | Comments Closed

Processing XML with Kafka Connect

Wednesday, May 15th, 2024

I spoke at Devoxx UK last week, about how to process XML data using a Kafka Connect pipeline.

This was based on some work I did last year, but it was good to get a chance to share it with a new audience.


youtu.be/NfYHE2i0-es

Tags:,
Posted in code | Comments Closed

Processing Apache Avro-serialized messages from Kafka using IBM App Connect Enterprise

Monday, May 13th, 2024

IBM App Connect Enterprise (ACE) is a broker for developing and hosting high-throughput, high-scale integrations between a large number of applications and systems, including Apache Kafka.

In this post, I’ll describe how to use App Connect Enterprise to process Kafka messages that were serialized using Apache Avro schemas.

screenshot

This is an update to an earlier version of this post, reflecting updates to the sample code.

Background

Best practice when using Apache Kafka is to define Apache Avro schemas with a definition of the structure of your Kafka messages, and to store those schemas in a central registry that client applications can access at runtime.

If you want to use IBM App Connect Enterprise to develop and host integrations for processing those Kafka messages, you need App Connect to know how to:

  • retrieve the Avro schemas it needs using schema registry REST APIs
  • use the schemas to turn the binary stream of bytes on your Kafka topics into structured objects that ACE can manipulate and process

(more…)

Tags:, , , , ,
Posted in code | Comments Closed

Using Mirror Maker 2 with IBM Event Streams to migrate to a new cluster

Thursday, April 18th, 2024

This is the sixth in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters.

For this sixth post, I’ll look at using Mirror Maker to migrate your Kafka cluster to a new region.

I’ve broken this down into multiple stages. For each stage, I’ll explain the intent and share a demo script I’ve created to let you try this for yourself.

(more…)

Tags:, ,
Posted in code | Comments Closed

Using Mirror Maker 2 with IBM Event Streams to restore from a backup cluster

Friday, April 12th, 2024

This is the fifth in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters.

For this fifth post, I’ll look at using Mirror Maker to maintain a backup of your Kafka events, and to be able to restore from that backup.

This is more complex than the previous posts as there are multiple stages involved. For each stage, I’ll explain the intent and share the demo script I’ve created to let you try this for yourself.

(more…)

Tags:, ,
Posted in code | Comments Closed

Using Mirror Maker 2 with IBM Event Streams to create a failover cluster

Monday, April 8th, 2024

This is the fourth in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters.

For this fourth post, I’ll look at using Mirror Maker to create an active/passive topology with a backup cluster ready to failover to.

(more…)

Tags:, ,
Posted in code | Comments Closed

Using Mirror Maker 2 with IBM Event Streams to share topics across multiple regions

Friday, April 5th, 2024

This is the third in a series of blog posts sharing examples of ways to use Mirror Maker 2 with IBM Event Streams.

Mirror Maker 2 is a powerful and flexible tool for moving Kafka events between Kafka clusters, but sometimes I feel like this can be forgotten if we only talk about it in the context of disaster recovery.

In these posts, I want to inspire you to think about other ways you could use Mirror Maker 2. The best way to learn about what is possible is to play with it for yourself, so with these posts I’ll include a script to create a demonstration of the scenario.

For this third post, I’ll look at using Mirror Maker to create logical topics shared across multiple regions.

(more…)

Tags:, ,
Posted in code | Comments Closed

How ML for Kids handles numbers models now

Thursday, April 4th, 2024

I’m on vacation for the Easter holidays this week. Apparently I don’t know how vacations work, so I’ve spent a lot of the last six days working on a major rewrite of a big chunk of Machine Learning for Kids. In this post, I want to describe what I’ve been doing and why.

(more…)

Tags:
Posted in code | Comments Closed

« Older Entries
Newer Entries »