Posts Tagged ‘appconnect’

Processing Apache Avro-serialized messages from Kafka using IBM App Connect Enterprise

Monday, May 13th, 2024

IBM App Connect Enterprise (ACE) is a broker for developing and hosting high-throughput, high-scale integrations between a large number of applications and systems, including Apache Kafka.

In this post, I’ll describe how to use App Connect Enterprise to process Kafka messages that were serialized using Apache Avro schemas.

screenshot

This is an update to an earlier version of this post, reflecting updates to the sample code.

Background

Best practice when using Apache Kafka is to define Apache Avro schemas with a definition of the structure of your Kafka messages, and to store those schemas in a central registry that client applications can access at runtime.

If you want to use IBM App Connect Enterprise to develop and host integrations for processing those Kafka messages, you need App Connect to know how to:

  • retrieve the Avro schemas it needs using schema registry REST APIs
  • use the schemas to turn the binary stream of bytes on your Kafka topics into structured objects that ACE can manipulate and process

(more…)

Tags:, , , , ,
Posted in code | No Comments »

Connecting App Connect Enterprise to Event Endpoint Management

Friday, October 20th, 2023

Configuring IBM App Connect Enterprise to consume messages from Kafka topics in IBM Event Endpoint Management requires careful configuration. In this post, I’ll share the steps I use that help me to avoid missing any required values.

If this sounds familiar, it might be because I wrote a post like this about using App Connect Enterprise to work with topics from Event Streams. People seem to have found that post useful, so I thought I’d do something similar for topics in Event Endpoint Management this time.

To illustrate this, I’ll create a simple App Connect flow that consumes messages from a Kafka topic and publish them to an MQTT topic.

The key to getting this to work correctly first time is to make sure that values are accurately copied from Event Endpoint Management to App Connect.

To help with this, I use a grid like the one below.

The instructions in this post start with Event Endpoint Management, and explain how to populate the grid with the information you need.

Then the instructions will switch to App Connect, and explain how to use the values in the grid to set up your App Connect flow.

What this is Values you will see in my screenshots Your value
A Topic name
DEMO.ACE
B Bootstrap address
my-eem-gateway-ibm-egw-rt-event-automation.apps.dalelane.cp.fyre.ibm.com:443
C SASL mechanism
PLAIN
D SASL config
org.apache.kafka.common.security.plain.PlainLoginModule required;
E Security protocol
SASL_SSL
F Certificate
eem-cert.jks
G Certificate password
STOREPASSW0RD
H Username
eem-9c8fc5d9-fddd-48dd-ab41-e062214166e5
I Password
dd08a1fc-99be-4931-8059-70aef88c1f0c
J Policy project name
demo-policies
K Policy name
demo-eem-policy
L Security identity name
eem-credentials
M Truststore identity name
eem-truststore

(more…)

Tags:, ,
Posted in ibm | Comments Closed

Deploying App Connect Enterprise applications from a CI/CD pipeline

Saturday, October 22nd, 2022

Sharing an example Tekton pipeline for deploying an IBM App Connect Enterprise application to Red Hat OpenShift.

This post is about a repository I’ve shared on github at dalelane/app-connect-tekton-pipeline. It contains an example of how to use Tekton to create a CI/CD pipeline that builds and deploys an App Connect Enterprise application to Red Hat OpenShift.

The pipeline uses the IBM App Connect Operator to easily build, deploy and manage your applications in containers. The pipeline runs on OpenShift to allow it to easily be integrated into an automated continuous delivery workflow without needing to build anything locally from a developer’s workstation.

For background information about the Operator, and the different types of Kubernetes resources that this pipeline will create (e.g. IntegrationServer and Configuration), see these blog posts:

(more…)

Tags:, , , ,
Posted in code | Comments Closed

Connecting App Connect Enterprise to Event Streams

Sunday, June 19th, 2022

Configuring IBM App Connect Enterprise to produce or consume messages from Kafka topics in IBM Event Streams requires careful configuration. In this post, I’ll share the steps I use that help me to avoid missing any required values.

To illustrate this, I’ll create a simple App Connect flow that implements a REST API, where any data I POST to the REST API is sent to a Kafka topic.

The key to getting this to work correctly first time is to make sure that values are accurately copied from Event Streams to App Connect.

To help with this, I use a grid like the one below.

The instructions in this post start with Event Streams, and explain how to populate the grid with the information you need.

Then the instructions will switch to App Connect, and explain how to use the values in the grid to set up your App Connect flow.

What this is Values you will see in my screenshots Your value
A Topic name
THIS.IS.MY.TOPIC
B Bootstrap address
kafkadev_kafka_bootstrap_demo.itzroks_120000f8p4_f9nd74_6ccd7f378ae819553d37d5f2ee142bd6_0000.eu_gb.containers.appdomain.cloud:443

kafkadev_kafka_bootstrap.demo.svc:9093

kafkadev_kafka_bootstrap.demo.svc:9092
C SASL mechanism
SCRAM-SHA-512
D SASL config
org.apache.kafka.common.security.scram.ScramLoginModule required;
E Security protocol
SASL_SSL

SASL_PLAINTEXT

SSL

PLAINTEXT
F Certificate
es-cert.jks
G Certificate password
wo05RndLJQgI
H Username
app-connect-enterprise
I Password
AIYJjrM2bSic
J Policy project name
demo-policies
K Policy name
demo-eventstreams-policy
L Security identity name
kafka-credentials
M Truststore identity name
kafka-truststore

(more…)

Tags:, , , ,
Posted in ibm | Comments Closed