Posts Tagged ‘ibmeventstreams’

« Older Entries
Newer Entries »

Setting up the Event Streams UI for developer-only use

Friday, December 9th, 2022

A quick tip for how to give a developer access to the IBM Event Streams UI only for the Kafka topics used by their application, and not everything else.

Imagine I’m a Kafka cluster admin. I’m running a cluster with a variety of topics on it.

Only viewing their own topics

One of my developers is responsible for the flight tracking app, and wants to use the Event Streams UI. But I don’t want them to be able to access the other sensitive topics for other applications.

I can create them their own login for the UI, that only lets them see their own topics.

The permissions I want to give them are:

- operation: Read
  resource:
    name: FLIGHT.
    patternType: prefix
    type: topic

(Remember, managing my Kafka cluster through Kubernetes resources is a good fit with a CI/CD workflow.)

(more…)

Tags:, , ,
Posted in code, ibm | Comments Closed

Using IBM DataStage to process JSON events on Apache Kafka topics

Monday, November 28th, 2022

In this post, I share a step-by-step guide for how to use IBM DataStage to merge JSON messages from multiple different Apache Kafka topics, into a single joined-up stream of events.

screenshot

(more…)

Tags:, , ,
Posted in ibm | Comments Closed

Setting up trusted SSL for IBM Event Streams

Thursday, October 27th, 2022

A quick how-to for setting up Event Streams with trusted certificates when running a development project.

Problem

You’re working on a project using IBM Event Streams. It’s just a development project, so you’re not using an SSL certificate signed by your real, trusted, corporate signer.

Everything works, but…

You get errors like these every time you access the web tooling – which you have to click through.

And you get errors like these from your Kafka client applications – which you have to configure with a custom truststore to avoid (although, if you do need to do that, I have a guide to help!)

[2021-06-27 23:19:06,048] ERROR [Consumer clientId=consumer-dalegrp-1, groupId=dalegrp] Connection to node -1 (dale-kafka-saslscram-bootstrap-strimzi.apps.eem-test-fest-6.cp.fyre.ibm.com/9.46.199.58:443) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2021-06-27 23:19:06,049] WARN [Consumer clientId=consumer-dalegrp-1, groupId=dalegrp] Bootstrap broker dale-kafka-saslscram-bootstrap-strimzi.apps.eem-test-fest-6.cp.fyre.ibm.com:443 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-06-27 23:19:06,069] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:269)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1339)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1214)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1157)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:770)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)

(more…)

Tags:, , , ,
Posted in code | Comments Closed

Event Endpoint Management “demo in a box”

Friday, December 17th, 2021

In this post, I’ll share how you can get your own Event Endpoint Management demo instance with just seven minutes of work.

(Seven minutes of hands-on-keyboard time… there is a lot of waiting-for-stuff-to-run time, but it doesn’t sound so impressive if I include waiting time!)


(more…)

Tags:, , ,
Posted in code | Comments Closed

Talking about IBM Event Streams

Wednesday, September 9th, 2020

We’ve been running a virtual event this week to explain the capabilities of IBM’s Cloud Pak for Integration.

One of these is Event Streams, so I gave an overview of the Event Streams Operator.

But what it really reminded me is that I miss going to conferences and tech events. I don’t want to sound ungrateful for what I’m sure has been a huge amount of work for event organisers in the pivot to online events. It’s great that we can still do events at all, and that organisers are still trying out ways to make it interactive, to enable panels and Q&A sessions.

(more…)

Tags:,
Posted in ibm | Comments Closed

Supporting CI/CD with Kubernetes Operators

Thursday, August 20th, 2020

Operators bring a lot of benefits as a way of managing complex software systems in a Kubernetes cluster. In this post, I want to illustrate one in particular: the way that custom resources (and declarative approaches to managing systems in general) enable easy integration with source control and a CI/CD pipeline.

I’ll be using IBM Event Streams as my example here, but the same principles will be true for many Kubernetes Operators, in particular, the open-source Strimzi Kafka Operator that Event Streams is based on.

(more…)

Tags:, , , ,
Posted in code | Comments Closed

Installing IBM Event Streams using the kubectl-operator plugin

Thursday, August 13th, 2020

Installing operators in Red Hat OpenShift from the CLI is much easier with the new kubectl-operator plugin. Here’s an example of how you can use it to install the Event Streams Operator.

Installing operators in OpenShift from the CLI is a little fiddly. It’s possible, but you have to create a bunch of custom resources that aren’t entirely intuitive, like Subscriptions and OperatorGroups.

It’s easy if you’re using the OpenShift Console web UI, as it does this all for you so you don’t need to worry about it. But sometimes you want to do things from the command line. And the new kubectl-operator plugin looks like it’ll make that much simpler.

I had a quick play with it this evening, and it let me get the Event Streams operator running with three commands. (Compare this with the OpenShift Console web UI equivalent in my Event Streams demo video).

(more…)

Tags:,
Posted in ibm | Comments Closed

Using MirrorMaker 2

Wednesday, July 15th, 2020

I’ve been talking about MirrorMaker 2 this week – the Apache Kafka tool for replicating data across two Kafka clusters. You can use it to make a copy of messages on your Kafka cluster to a remote Kafka cluster running on a different data centre, and keep that copy up to date in the background.

For the discussion we had, I needed to give examples of how you might use MirrorMaker 2, which essentially meant I spent an afternoon drawing pictures. As some of them were a little pretty, I thought I’d tidy them up and share them here.

We went through several different use cases, but I’ll just describe two examples here.
(more…)

Tags:, , ,
Posted in tech | 4 Comments »

« Older Entries
Newer Entries »