{"id":4573,"date":"2022-06-19T01:11:07","date_gmt":"2022-06-19T01:11:07","guid":{"rendered":"https:\/\/dalelane.co.uk\/blog\/?p=4573"},"modified":"2022-06-19T01:11:07","modified_gmt":"2022-06-19T01:11:07","slug":"connecting-app-connect-enterprise-to-event-streams","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=4573","title":{"rendered":"Connecting App Connect Enterprise to Event Streams"},"content":{"rendered":"<p><strong>Configuring IBM App Connect Enterprise to produce or consume messages from Kafka topics in IBM Event Streams requires careful configuration. In this post, I&#8217;ll share the steps I use that help me to avoid missing any required values.<\/strong><\/p>\n<p>To illustrate this, I&#8217;ll create a simple App Connect flow that implements a REST API, where any data I POST to the REST API is sent to a Kafka topic.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-01-overview.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-01-overview.png\" width=\"450\"\/><\/a><\/p>\n<p>The key to getting this to work correctly first time is to make sure that values are accurately copied from Event Streams to App Connect.<\/p>\n<p>To help with this, I use a grid like the one below.<\/p>\n<p>The instructions in this post start with Event Streams, and explain how to populate the grid with the information you need.<\/p>\n<p>Then the instructions will switch to App Connect, and explain how to use the values in the grid to set up your App Connect flow.<\/p>\n<style>table.aceesgrid { border-collapse: collapse; table-layout: fixed; width: 450px; } table.aceesgrid, .aceesgrid th, .aceesgrid td { font-size: 9.5px; text-align: center; border: thin black solid; } .aceesgrid th { background-color: #f1f1f1; } .aceesgrid th, .aceesgrid td { padding: 3px; } .aceesgrid th.examples { width: 190px; } .aceesgrid td.examples { max-width: 190px; width: 100%; text-align: left; overflow: auto; } .aceesgrid td.examples > div { hyphens: none; -webkit-hyphens: none; margin: 0; padding: 0; max-width: 190px; width: 100%; text-align: left; font-size: 9px; font-family: 'Courier New', Courier, monospace; } .aceesgrid td.label { padding: 5px 8px; width: 60px; } <\/style>\n<table class=\"aceesgrid\">\n<thead>\n<tr>\n<th style=\"width: 18px;\"><\/th>\n<th style=\"width: 60px;\">What this is<\/th>\n<th class=\"examples\">Values you will see in my screenshots<\/th>\n<th>Your value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>A<\/strong><\/td>\n<td class=\"label\">Topic name<\/td>\n<td class=\"examples\">\n<div>THIS.IS.MY.TOPIC<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>B<\/strong><\/td>\n<td class=\"label\">Bootstrap address<\/td>\n<td class=\"examples\">\n<div>kafkadev_kafka_bootstrap_demo.itzroks_120000f8p4_f9nd74_6ccd7f378ae819553d37d5f2ee142bd6_0000.eu_gb.containers.appdomain.cloud:443<\/p>\n<p>kafkadev_kafka_bootstrap.demo.svc:9093<\/p>\n<p>kafkadev_kafka_bootstrap.demo.svc:9092<\/p>\n<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>C<\/strong><\/td>\n<td class=\"label\">SASL mechanism<\/td>\n<td class=\"examples\">\n<div>SCRAM-SHA-512<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>D<\/strong><\/td>\n<td class=\"label\">SASL config<\/td>\n<td class=\"examples\">\n<div>org.apache.kafka.common.security.scram.ScramLoginModule&nbsp;required;<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>E<\/strong><\/td>\n<td class=\"label\">Security protocol<\/td>\n<td class=\"examples\">\n<div>SASL_SSL<\/p>\n<p>SASL_PLAINTEXT<\/p>\n<p>SSL<\/p>\n<p>PLAINTEXT<\/p>\n<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>F<\/strong><\/td>\n<td class=\"label\">Certificate<\/td>\n<td class=\"examples\">\n<div>es-cert.jks<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>G<\/strong><\/td>\n<td class=\"label\">Certificate password<\/td>\n<td class=\"examples\">\n<div>wo05RndLJQgI<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>H<\/strong><\/td>\n<td>Username<\/td>\n<td class=\"examples\">\n<div>app-connect-enterprise<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>I<\/strong><\/td>\n<td>Password<\/td>\n<td class=\"examples\">\n<div>AIYJjrM2bSic<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>J<\/strong><\/td>\n<td class=\"label\">Policy project name<\/td>\n<td class=\"examples\">\n<div>demo-policies<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>K<\/strong><\/td>\n<td class=\"label\">Policy name<\/td>\n<td class=\"examples\">\n<div>demo-eventstreams-policy<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>L<\/strong><\/td>\n<td class=\"label\">Security identity name<\/td>\n<td class=\"examples\">\n<div>kafka-credentials<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><strong>M<\/strong><\/td>\n<td class=\"label\">Truststore identity name<\/td>\n<td class=\"examples\">\n<div>kafka-truststore<\/div>\n<\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><!--more--><\/p>\n<p><em>Note: To see screenshots in more detail, you can click on them to open a higher-resolution version.<\/em><\/p>\n<h3>Pre-requisites<\/h3>\n<p>There are some things you will need before you can start following the instructions. This post is already long enough, so I will skip any detail on these steps.<\/p>\n<p>You need an OpenShift cluster.<br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/01-prereqs-00-openshift.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/01-prereqs-00-openshift.png\" width=\"450\"\/><\/a><\/p>\n<p>You need to install the Operators for App Connect and Event Streams.<br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/01-prereqs-01-operators.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/01-prereqs-01-operators.png\" width=\"450\"\/><\/a><\/p>\n<p>You need to create a secret with your key for the Entitled Registry.<br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/01-prereqs-02-entitlementkey.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/01-prereqs-02-entitlementkey.png\" width=\"450\"\/><\/a><\/p>\n<p>You need an Event Streams cluster.<br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/01-prereqs-03-eventstreams.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/01-prereqs-03-eventstreams.png\" width=\"450\"\/><\/a><\/p>\n<p>You need a topic that you want App Connect to send messages to.<br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/01-prereqs-04-kafkatopic.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/01-prereqs-04-kafkatopic.png\" width=\"450\"\/><\/a><\/p>\n<h3>Step 1<br \/>\nCollecting values you need from Event Streams<\/h3>\n<h4>A &#8211; the topic name<\/h4>\n<p>Get the name of the topic you want Event Streams to send messages to.<\/p>\n<p>Fill this in the grid as <strong>value A<\/strong>.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-01-topicname.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-01-topicname.png\" width=\"450\"\/><\/a><\/p>\n<h4>B &#8211; the bootstrap address<\/h4>\n<p>Get the bootstrap address that you want App Connect to use to connect to the Kafka cluster.<\/p>\n<p>The addresses you have to choose from will depend on how you have configured your Event Streams instance.<\/p>\n<p>If you have enabled external listeners, you could choose an external address.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-02-bootstrap-external.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-02-bootstrap-external.png\" width=\"450\"\/><\/a><\/p>\n<p>If you have enabled internal listeners, you could choose an internal address.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-03-bootstrap-internal.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-03-bootstrap-internal.png\" width=\"450\"\/><\/a><\/p>\n<p>Whichever address you choose, fill it in the grid as <strong>value B<\/strong>.<\/p>\n<h4>C &#8211; security mechanism<\/h4>\n<p>Look next to the bootstrap address that you chose for value B.<\/p>\n<p>If you see a reference to &#8220;SCRAM&#8221;, that means the Kafka listener you have chosen to use for App Connect is configured to require authentication, with credentials provided using the SASL\/SCRAM mechanism.<\/p>\n<p>If that is the case, fill <code>SCRAM-SHA-512<\/code> in the grid as <strong>value C<\/strong>.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-04-saslmechanism-scram.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-04-saslmechanism-scram.png\" width=\"450\"\/><\/a><\/p>\n<p>If you see a reference that credentials aren&#8217;t required, that means the Kafka listener you have chosen to use for App Connect is configured to not require authentication.<\/p>\n<p>If that is the case, leave <strong>value C<\/strong> empty.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-05-saslmechanism-plain.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-05-saslmechanism-plain.png\" width=\"450\"\/><\/a><\/p>\n<h4>D &#8211; security config<\/h4>\n<p>If value C is <code>SCRAM-SHA-512<\/code>, then set <strong>value D<\/strong> in the grid to<\/p>\n<pre>org.apache.kafka.common.security.scram.ScramLoginModule required;<\/pre>\n<p>Otherwise, leave <strong>value D<\/strong> empty.<\/p>\n<h4>E &#8211; security protocol<\/h4>\n<p>You need to identify whether the Kafka listener is configured to require encryption.<\/p>\n<p>If you chose an external listener, this is simple &#8211; encryption is always required for external listeners.<\/p>\n<p>If you chose an internal listener, you will need to find the spec for the listener. One way to do this is to look at the spec for your Event Streams cluster in the OpenShift Console.<\/p>\n<p>Find the listener that you chose in <code>.status.kafkaListeners<\/code>.<\/p>\n<p>If the <code>type<\/code> of the listener you chose is <code>tls<\/code> or <code>external<\/code> then encryption is required.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-06-securityprotocol-tls.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-06-securityprotocol-tls.png\" width=\"450\"\/><\/a><\/p>\n<p>Otherwise (if the <code>type<\/code> is <code>plain<\/code>), then encryption is not required.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-07-securityprotocol-plain.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-07-securityprotocol-plain.png\" width=\"450\"\/><\/a><\/p>\n<p>Use the table below to work out what you need to fill in the grid as <strong>value E<\/strong>.<\/p>\n<table class=\"aceesgrid\">\n<thead>\n<tr>\n<th>if value C is&#8230;?<\/th>\n<th>is encryption required?<\/th>\n<th>then set value E to&#8230;<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code>SCRAM-SHA-512<\/code><\/td>\n<td>yes<\/td>\n<td><code>SASL_SSL<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>SCRAM-SHA-512<\/code><\/td>\n<td>no<\/td>\n<td><code>SASL_PLAINTEXT<\/code><\/td>\n<\/tr>\n<tr>\n<td><em>(empty)<\/em><\/td>\n<td>yes<\/td>\n<td><code>SSL<\/code><\/td>\n<\/tr>\n<tr>\n<td><em>(empty)<\/em><\/td>\n<td>no<\/td>\n<td><code>PLAINTEXT<\/code><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h4>F &#8211; SSL certificate<\/h4>\n<p>If the Kafka listener is configured to require encryption, you need to download the CA certificate for the listener.<\/p>\n<p>You can download the PKCS12 certificate from Event Streams.<\/p>\n<p>Keep this file safe, and make a note of the file name in the grid as <strong>value F<\/strong>.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-08-certificate.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-08-certificate.png\" width=\"450\"\/><\/a><\/p>\n<h4>G &#8211; SSL certificate password<\/h4>\n<p>When you download the PKCS12 certificate, the password will be displayed.<\/p>\n<p>Fill it in the grid as <strong>value G<\/strong>.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-09-certificate-password.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-09-certificate-password.png\" width=\"450\"\/><\/a><\/p>\n<h4>H\/I &#8211; username \/ password<\/h4>\n<p>If the Kafka listener is configured to require SCRAM credentials, you need to create a username and password for App Connect to use.<\/p>\n<p>(If no credentials are required, you can skip this step and leave values H and I empty.)<\/p>\n<p>Choose a username. Enter it into the grid as <strong>value H<\/strong>.<\/p>\n<p>Click on the Generate SCRAM credentials button.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-04-saslmechanism-scram.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-04-saslmechanism-scram.png\" width=\"450\"\/><\/a><\/p>\n<p>Use the name from <strong>value H<\/strong> as the name for your credentials.<\/p>\n<p>Make sure you include the permission that App Connect will need (&#8220;consume&#8221; if you want App Connect to be able to receive messages, &#8220;produce&#8221; if you want App Connect to be able to send messages).<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-10-username.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-10-username.png\" width=\"450\"\/><\/a><\/p>\n<p>Use the topic name from value A when specifying the permissions for the credentials.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-11-credentials-topic.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-11-credentials-topic.png\" width=\"450\"\/><\/a><\/p>\n<p>Enter the generated password into the grid as <strong>value I<\/strong>.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/02-collectinginfo-12-password.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/02-collectinginfo-12-password.png\" width=\"450\"\/><\/a><\/p>\n<h3>Step 1(b) &#8211; Workaround step<\/h3>\n<p>At the time of writing, if you have a p12 file in value E, you will need to convert it to a JKS file because of the issue described in the <a href=\"https:\/\/ibm.github.io\/event-streams\/troubleshooting\/pkcs12-keystore-java-client\">Event Streams support docs<\/a>. Follow the instructions on that page to create the JKS file.<\/p>\n<p>If you do this, update <strong>value F<\/strong> in the grid with the new jks file name.<\/p>\n<h3>Step 2 &#8211; Choose some App Connect names<\/h3>\n<p>You will create some resources in App Connect.<\/p>\n<p>Choose a name for your App Connect policy project.<br \/>\nEnter it into the grid as <strong>value J<\/strong>.<\/p>\n<p>Choose a name for your App Connect policy.<br \/>\nEnter it into the grid as <strong>value K<\/strong>.<\/p>\n<p>If you have a SCRAM username\/password, choose a name for the security identity.<br \/>\nEnter it into the grid as <strong>value L<\/strong>.<\/p>\n<p>If you have a truststore file (p12 file, or jks file, depending on whether the workaround is still required), choose a name for the truststore identity.<br \/>\nEnter it into the grid as <strong>value M<\/strong>.<\/p>\n<h3>Step 3 &#8211; Creating your App Connect policy<\/h3>\n<p>Use the App Connect Enterprise toolkit to create a new Policy project.<\/p>\n<p>Use the name from <strong>value J<\/strong> in the grid for the name.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-01-project.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-01-project.png\" width=\"450\"\/><\/a><\/p>\n<p>Create a policy in your new project. Use <strong>value K<\/strong> from the grid as the file name.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-02-policy.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-02-policy.png\" width=\"450\"\/><\/a><\/p>\n<p>Make sure that the policy name matches <strong>value K<\/strong> from the grid.<\/p>\n<p>Set the policy Type and Template both to &#8220;Kafka&#8221;.<\/p>\n<p>Fill in the rest of the policy using values from the grid.<\/p>\n<p>Set Bootstrap servers (<code>&lt;bootstrapServers&gt;<\/code>) to <strong>value B<\/strong> from the grid.<\/p>\n<p>Set Security protocol (<code>&lt;securityProtocol&gt;<\/code>) to <strong>value E<\/strong> from the grid.<\/p>\n<p>Set SASL Mechanism (<code>&lt;saslMechanism&gt;<\/code>) to <strong>value C<\/strong> from the grid.<\/p>\n<p>Set Security identity (<code>&lt;securityIdentity&gt;<\/code>) to <strong>value L<\/strong> from the grid.<\/p>\n<p>Set SASL config (<code>&lt;saslConfig&gt;<\/code>) to <strong>value D<\/strong> from the grid.<\/p>\n<p>If you have a filename in <strong>value F<\/strong> in the grid, set SSL truststore location (<code>&lt;sslTruststoreLocation&gt;<\/code>) to <code>\/home\/aceuser\/truststores\/<\/code> followed by your filename.<br \/>\n(e.g. <code>\/home\/aceuser\/truststores\/es-cert.jks<\/code>).<br \/>\nOtherwise, leave SSL truststore location blank.<\/p>\n<p>If you have a filename in <strong>value F<\/strong> in the grid, set SSL truststore type (<code>&lt;sslTruststoreType&gt;<\/code>) to <code>JKS<\/code> (if you have a jks file) or <code>PKCS12<\/code> (if you have a p12 file).<\/p>\n<p>Set SSL truststore security identity<br \/>\n(<code>&lt;sslTruststoreSecurityIdentity&gt;<\/code>) to <strong>value M<\/strong> from the grid.<\/p>\n<p>Set SSL certificate hostname checking<br \/>\n(<code>&lt;sslEnableCertificateHostnameChecking&gt;<\/code>) to <code>false<\/code>.<\/p>\n<p>A few examples of how this could look, depending on some of the choices you could have made&#8230;<br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-03-policy-values.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-03-policy-values.png\" width=\"450\"\/><\/a><br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-04-policy-values.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-04-policy-values.png\" width=\"450\"\/><\/a><br \/>\n<a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-05-policy-values.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-05-policy-values.png\" width=\"450\"\/><\/a><\/p>\n<p>Export the policy project to a zip file.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-06-export-wizard.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-06-export-wizard.png\" width=\"450\"\/><\/a><\/p>\n<p>The file name you use isn&#8217;t significant, so choose any name that you like.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/03-acepolicy-07-export-wizard.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/03-acepolicy-07-export-wizard.png\" width=\"450\"\/><\/a><\/p>\n<h3>Step 4 &#8211; Creating your App Connect flow<\/h3>\n<p>To illustrate how to use the grid, I&#8217;ll create a flow that sends data received over HTTP to the Kafka topic. You could configure a Kafka consumer node in a similar way.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-01-overview.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-01-overview.png\" width=\"450\"\/><\/a><\/p>\n<p>I&#8217;m using an HTTP input node with a path of <code>\/demo<\/code><\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-02-http-input.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-02-http-input.png\" width=\"450\"\/><\/a><\/p>\n<p>Configure the Kafka node, starting with the &#8220;Basic&#8221; tab.<\/p>\n<p>Set Topic name to <strong>value A<\/strong> from the grid.<\/p>\n<p>The Bootstrap servers value won&#8217;t be used, but it&#8217;s a required value, so put any value in there. I use &#8220;not-used&#8221; for this to avoid confusion.<\/p>\n<p>Set Client ID to something that can be used to identify App Connect in Event Streams monitoring.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-03-kafkaproducer-basic.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-03-kafkaproducer-basic.png\" width=\"450\"\/><\/a><\/p>\n<p>Next, fill in the &#8220;Security&#8221; tab.<\/p>\n<p>Set Security identity to <strong>value L<\/strong> from the grid.<\/p>\n<p>Set Security protocol to <strong>value E<\/strong> from the grid.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-04-kafkaproducer-security.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-04-kafkaproducer-security.png\" width=\"450\"\/><\/a><\/p>\n<p>Finally, fill in the &#8220;Policy&#8221; tab.<\/p>\n<p>Set Policy using <strong>value J<\/strong> and <strong>value K<\/strong> from the grid. It should look like <code>{valueJ}:valueK<\/code>.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-05-kafkaproducer-policy.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-05-kafkaproducer-policy.png\" width=\"450\"\/><\/a><\/p>\n<p>Export the app with your flow to a BAR file.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/04-aceflow-06-barfile.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/04-aceflow-06-barfile.png\" width=\"450\"\/><\/a><\/p>\n<h3>Step 5 &#8211; Set up App Connect<\/h3>\n<h4>Create an App Connect dashboard<\/h4>\n<p>This will make it easier to deploy your App Connect flow.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-01-create-dashboard.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-01-create-dashboard.png\" width=\"450\"\/><\/a><\/p>\n<h4>Add the Configurations to the dashboard<\/h4>\n<p>Use the Dashboard to create Configurations.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-02-create-configuration.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-02-create-configuration.png\" width=\"450\"\/><\/a><\/p>\n<p>If you have a filename in <strong>value F<\/strong> in the grid, create a new Configuration.<br \/>\nSet the Type to &#8220;Truststore&#8221; and upload your truststore file.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-03-truststore.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-03-truststore.png\" width=\"450\"\/><\/a><\/p>\n<p>Create another new Configuration.<br \/>\nSet the Type to &#8220;Policy project&#8221; and upload your exported policy project zip file.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-04-policy.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-04-policy.png\" width=\"450\"\/><\/a><\/p>\n<p>If you have values for any of <strong>value H<\/strong>, <strong>value I<\/strong>, <strong>value F<\/strong> in the grid, create another new Configuration.<br \/>\nSet the Type to &#8220;setdbparms.txt&#8221;.<\/p>\n<p>If you have values for <strong>value H<\/strong> and <strong>value I<\/strong>, add a line with <code>kafka::<\/code> followed by the security identity name, then a space, then the username and password.<\/p>\n<pre>kafka::valueL valueH valueI<\/pre>\n<p>If you have a filename in <strong>value F<\/strong> in the grid, add a line with <code>truststore::<\/code> followed by the truststore identity name, then a space, then an unused placeholder value, then the truststore password.<\/p>\n<pre>truststore::valueM notused valueG<\/pre>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-05-setdbparms.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-05-setdbparms.png\" width=\"450\"\/><\/a><\/p>\n<p>Depending on how your Event Streams listener is configured, you should now have between one and three configurations.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-06-configurations.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-06-configurations.png\" width=\"450\"\/><\/a><\/p>\n<h4>Upload the BAR file to the dashboard<\/h4>\n<p>Use the Dashboard to upload a bar file.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-07-bar.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-07-bar.png\" width=\"450\"\/><\/a><\/p>\n<p>Import the bar file that you created with the Kafka message flow.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/05-appconnect-08-bar.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/05-appconnect-08-bar.png\" width=\"450\"\/><\/a><\/p>\n<h3>Step 6 &#8211; Deploy the message flow<\/h3>\n<p>Use the Dashboard to create a new integration server.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/06-integrationserver-01-dashboard.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/06-integrationserver-01-dashboard.png\" width=\"450\"\/><\/a><\/p>\n<p>If you are using a truststore, note that creating an integration server with CPU and memory limits that are too small can result in SSL handshake errors when connecting to Kafka. If this happens, try different CPU and memory limits. For example, setting CPU limit to at least 1 and memory to at least 512Mi may help.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/06-integrationserver-02-createserver.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/06-integrationserver-02-createserver.png\" width=\"450\"\/><\/a><\/p>\n<p>Choose your BAR file<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/06-integrationserver-03-bar.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/06-integrationserver-03-bar.png\" width=\"450\"\/><\/a><\/p>\n<p>Enable all of the Configurations that you created<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/06-integrationserver-04-configurations.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/06-integrationserver-04-configurations.png\" width=\"450\"\/><\/a><\/p>\n<p>Name the integration and click Create.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/06-integrationserver-05-create.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/06-integrationserver-05-create.png\" width=\"450\"\/><\/a><\/p>\n<h4>Step 6(b) &#8211; a gotcha<\/h4>\n<p>If you are using an internal Kafka listener, and your App Connect integration server is running in a different namespace to the Event Streams cluster, then you may need to create a NetworkPolicy to give the integration server permission to make the connection to a different namespace.<\/p>\n<h3>Step 7 &#8211; Try it out!<\/h3>\n<p>Try using <code>curl<\/code> to send some text to the message flow.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/07-test-01-curl.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/07-test-01-curl.png\" width=\"450\"\/><\/a><\/p>\n<p>You should see the text appear in a message on your Kafka topic.<\/p>\n<p><a target=\"screenshot\" href=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/large\/07-test-02-topic.png\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/github.com\/dalelane\/ace-es-intro\/raw\/master\/small\/07-test-02-topic.png\" width=\"450\"\/><\/a><\/p>\n<p>Finished! Using a grid like the one detailed here should hopefully ensure that you have it working first time, without a lot of time-consuming errors and debugging.<\/p>\n<style>code, pre { color: #770000; font-weight: 600; background-color: #ffffc0 } <\/style>\n","protected":false},"excerpt":{"rendered":"<p>Configuring IBM App Connect Enterprise to produce or consume messages from Kafka topics in IBM Event Streams requires careful configuration. In this post, I&#8217;ll share the steps I use that help me to avoid missing any required values. To illustrate this, I&#8217;ll create a simple App Connect flow that implements a REST API, where any [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[605,593,604,606,584],"class_list":["post-4573","post","type-post","status-publish","format-standard","hentry","category-ibm","tag-ace","tag-apachekafka","tag-appconnect","tag-cp4i","tag-kafka"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/4573","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4573"}],"version-history":[{"count":0,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/4573\/revisions"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4573"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4573"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4573"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}