Archive for the ‘ibm’ Category

« Older Entries
Newer Entries »

Understanding windows in Event Processing

Wednesday, October 11th, 2023

Event Processing (one of the new capabilities of IBM Event Automation) makes it easy to perform stateful processing of streams of events from Kafka topics. In this post, I want to drill down a little into how windowed processing behaves.

I’ve enjoyed the chance to see the solutions that people have started to build with the tool. As part of this, I’ve been helping several people to understand the results produced by the event processing flows they’ve made.

These often started with different questions, such as:

  • Why hasn’t my flow produced any results?
  • Why isn’t my one-minute window producing one result every minute?
  • Why did the last one event on my Kafka topic cause results to be produced for several different windows?
  • etc.

However, these are often symptoms of a single common question: how windowed processing operations behave.

I’ve tried to come up with simple ways to demonstrate how it works, so in this post I want to share how I’m currently explaining it.


youtu.be/x_r6GNZmsd4

Tags:, , ,
Posted in ibm | Comments Closed

What can you do with IBM Event Automation?

Sunday, July 9th, 2023

This is IBM Event Automation : a new product we released last month to help our clients create event driven solutions.

I’ve written a 200-word summary of what IBM Event Automation is, but in this post I wanted to dive a little bit deeper and show what it can do.

(more…)

Tags:, , ,
Posted in ibm | Comments Closed

What is IBM Event Automation?

Tuesday, July 4th, 2023

A summary of IBM Event Automation in under 200 words.

Last week was the first release of IBM Event Automation. I’ve been asked what it’s all about, so I thought it’d be helpful to have a brief summary to point people at.

IBM Event Automation is a collection of three components, designed to help companies be productive in creating event-driven solutions.

(more…)

Tags:, , , ,
Posted in ibm | Comments Closed

Using client quotas with IBM Event Streams

Sunday, February 26th, 2023

In this post, I want to highlight a feature that I often see under-used in IBM Event Streams, and show how you can easily give it a try.

Kafka can enforce quotas to limit the impact that client applications can have on your cluster. To quote the Kafka documentation:

It is possible for producers and consumers to produce/consume very high volumes of data or generate requests at a very high rate and thus monopolize broker resources, cause network saturation and generally DOS other clients and the brokers themselves.

Having quotas protects against these issues and is all the more important in large multi-tenant clusters where a small set of badly behaved clients can degrade user experience for the well behaved ones.

In fact, when running Kafka as a service this even makes it possible to enforce API limits according to an agreed upon contract.

(more…)

Tags:, , ,
Posted in code, ibm | Comments Closed

What is IBM Client Engineering?

Saturday, December 17th, 2022

I’ve been working in Client Engineering since last summer.

This video is a great description of the aspiration of the team, and the vision of the sort of work we want to do with our customers. (Plus you get a few glimpses of our office in York Road where I worked this year).

Posted in ibm | Comments Closed

Setting up the Event Streams UI for developer-only use

Friday, December 9th, 2022

A quick tip for how to give a developer access to the IBM Event Streams UI only for the Kafka topics used by their application, and not everything else.

Imagine I’m a Kafka cluster admin. I’m running a cluster with a variety of topics on it.

Only viewing their own topics

One of my developers is responsible for the flight tracking app, and wants to use the Event Streams UI. But I don’t want them to be able to access the other sensitive topics for other applications.

I can create them their own login for the UI, that only lets them see their own topics.

The permissions I want to give them are:

- operation: Read
  resource:
    name: FLIGHT.
    patternType: prefix
    type: topic

(Remember, managing my Kafka cluster through Kubernetes resources is a good fit with a CI/CD workflow.)

(more…)

Tags:, , ,
Posted in code, ibm | Comments Closed

Using IBM DataStage to process JSON events on Apache Kafka topics

Monday, November 28th, 2022

In this post, I share a step-by-step guide for how to use IBM DataStage to merge JSON messages from multiple different Apache Kafka topics, into a single joined-up stream of events.

screenshot

(more…)

Tags:, , ,
Posted in ibm | Comments Closed

Geo-steering with IBM Code Engine and Cloud Internet Services

Saturday, September 3rd, 2022

In this post, I want to share a small tip from how I run Machine Learning for Kids: how I run instances of the site in different regions, and use geo-steering so that users are directed to the instance of the site nearest to them.

(more…)

Tags:
Posted in code, ibm | Comments Closed

« Older Entries
Newer Entries »