{"id":4303,"date":"2021-03-24T23:19:49","date_gmt":"2021-03-24T23:19:49","guid":{"rendered":"https:\/\/dalelane.co.uk\/blog\/?p=4303"},"modified":"2021-03-25T00:20:07","modified_gmt":"2021-03-25T00:20:07","slug":"migrating-your-apache-kafka-cluster-using-mirrormaker-2","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=4303","title":{"rendered":"Migrating your Apache Kafka cluster using MirrorMaker 2"},"content":{"rendered":"<p><strong>You have a Kafka cluster that you have been using for a while. Your cluster has many topics, and the topics have many messages. <\/strong><\/p>\n<p><strong>Now you&#8217;ve decided to move and start using a new, different Kafka cluster somewhere else. <\/strong><\/p>\n<p><strong>How can you take your topics with you?<\/strong><\/p>\n<p><em>Huge thanks to <a href=\"https:\/\/github.com\/ajborley\">Andrew Borley<\/a> for co-writing this with me. Useful insights in here probably came from him, the mistakes from me.<\/em><\/p>\n<p><!--more--><\/p>\n<ul>\n<li><a href=\"#terminology\">Terminology<\/a><\/li>\n<li><a href=\"#overview\">Overview<\/a><\/li>\n<li><a href=\"#prerequisites\">Prerequisites<\/a><\/li>\n<li><a href=\"#start-mm2\">Start MirrorMaker 2<\/a><\/li>\n<li><a href=\"#stop-mm2\">Stop MirrorMaker 2 and clean up<\/a><\/li>\n<li><a href=\"#update-topic-replication-factors\">Update your topic replication factors<\/a><\/li>\n<li><a href=\"#resume-client-applications\">Resume your client applications<\/a><\/li>\n<\/ul>\n<h2 id=\"terminology\">Terminology<\/h2>\n<p>For the purposes of this post, we&#8217;ll refer to the two Kafka clusters as:<\/p>\n<ul>\n<li><strong>&#8220;origin&#8221;<\/strong> &#8211; your existing Kafka cluster that you are migrating from<\/li>\n<li><strong>&#8220;target&#8221;<\/strong> &#8211; your new Kafka cluster that you are migrating to<\/li>\n<\/ul>\n<p>The instructions here use the <a href=\"https:\/\/strimzi.io\">Strimzi Operator<\/a> as a convenient way to configure and run MirrorMaker 2, but neither the &#8220;origin&#8221; or &#8220;target&#8221; Kafka clusters need to be managed by the Strimzi Operator for this to work.<\/p>\n<p>We&#8217;ll be running MirrorMaker 2 in a namespace called &#8220;migration&#8221;.<\/p>\n<h2 id=\"overview\">Overview<\/h2>\n<p>This post is a brief introduction to how you can set up and run MirrorMaker 2 for data migration using the Strimzi Operator.<\/p>\n<p>MirrorMaker 2 can be run in a variety of different ways, for a variety of use cases &#8211; for example, it can be used for <a href=\"https:\/\/dalelane.co.uk\/blog\/?p=4074\">disaster-recovery use-cases where it is run as a continuous background mirroring process<\/a>. However, for the purposes of this post, the requirement is a one-off data migration, so the description here will describe how to do that.<\/p>\n<h2 id=\"prerequisites\">Prerequisites<\/h2>\n<p>Before you start, you need to obtain credentials and TLS certificates for both of your Kafka clusters:<\/p>\n<ul>\n<li><a href=\"#origin-cluster-credentials\">Credentials from the &#8220;origin&#8221; cluster<\/a><\/li>\n<li><a href=\"#origin-cluster-ca-cert\">TLS certificate from the &#8220;origin&#8221; cluster<\/a><\/li>\n<li><a href=\"#target-cluster-credentials\">Credentials from the &#8220;target&#8221; cluster<\/a><\/li>\n<li><a href=\"#target-cluster-ca-cert\">TLS certificate from the &#8220;target&#8221; cluster<\/a><\/li>\n<\/ul>\n<h3 id=\"origin-cluster-credentials\">Credentials from the &#8220;origin&#8221; cluster<\/h3>\n<p>If your &#8220;origin&#8221; cluster requires authentication, you will need to create credentials for MirrorMaker 2 to use.<\/p>\n<p>If your &#8220;origin&#8221; cluster is managed by Strimzi, you could do this by creating a <code>KafkaUser<\/code> resource something like this (below).<\/p>\n<pre style=\"max-height: 107px; padding: 10px; overflow-y: scroll; border: thin black solid; font-size: smaller;\">apiVersion: kafka.strimzi.io\/v1beta1\nkind: KafkaUser\nmetadata:\n  name: mm2-credentials\n  labels:\n    strimzi.io\/cluster: origin\n  namespace: origin-ns\nspec:\n  authentication:\n    type: scram-sha-512\n  authorization:\n    acls:\n      - host: '*'\n        operation: Read\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Describe\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: DescribeConfigs\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Create\n        resource:\n          type: cluster\n      - host: '*'\n        operation: Read\n        resource:\n          type: cluster\n      - host: '*'\n        operation: Describe\n        resource:\n          type: cluster\n      - host: '*'\n        operation: Write\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Describe\n        resource:\n          name: '*'\n          patternType: literal\n          type: group\n      - host: '*'\n        operation: Read\n        resource:\n          name: '*'\n          patternType: literal\n          type: group\n    type: simple<\/pre>\n<p>This will create a new secret with your credentials in the namespace where your &#8220;origin&#8221; Kafka cluster is running. Copy these into a secret in a namespace on the Kubernetes cluster where you want to run MirrorMaker 2.<\/p>\n<pre style=\"max-height: 107px; padding: 10px; overflow-y: scroll; border: thin black solid; font-size: smaller;\">apiVersion: v1\nkind: Secret\nmetadata:\n  name: origin-cluster-credentials\n  namespace: migration\ndata:\n  password: YzZPSHZNUHhlWTZm<\/pre>\n<p>These credentials will allow MirrorMaker to find and consume from all of the topics on the &#8220;origin&#8221; cluster. You will need to do something similar using the type of Kafka cluster you are running. However you create your credentials, you should create a <code>Secret<\/code> similar to the one above.<\/p>\n<h3 id=\"origin-cluster-ca-cert\">TLS certificate from the &#8220;origin&#8221; cluster<\/h3>\n<p>If your &#8220;origin&#8221; cluster requires TLS, you may need to obtain a CA cert for MirrorMaker 2 to use.<\/p>\n<p>If your &#8220;origin&#8221; cluster is managed by Strimzi, you will be able to find this in a <code>Secret<\/code> called something like <code>origin-cluster-ca-cert<\/code> in the namespace where your Kafka cluster is running.<\/p>\n<p>Copy this into a secret in a namespace on the Kubernetes cluster where you want to run MirrorMaker 2.<\/p>\n<pre style=\"max-height: 107px; padding: 10px; overflow: scroll; border: thin black solid; font-size: smaller;\">apiVersion: v1\nkind: Secret\nmetadata:\n  name: origin-cluster-ca-cert\n  namespace: migration\ndata:\n  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURPekNDQWlPZ0F3SUJBZ0lVYldZKy9TaFVYM1NvRFdwWHpQa3ZUK2F6dDlJd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0xURVRNQkVHQTFVRUNnd0thVzh1YzNSeWFXMTZhVEVXTUJRR0ExVUVBd3dOWTJ4MWMzUmxjaTFqWVNCMgpNREFlRncweU1UQXpNRFF4TmpVMk16bGFGdzB5TWpBek1EUXhOalUyTXpsYU1DMHhFekFSQmdOVkJBb01DbWx2CkxuTjBjbWx0ZW1reEZqQVVCZ05WQkFNTURXTnNkWE4wWlhJdFkyRWdkakF3Z2dFaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUNwd1VnVEdnTkZkTTN5MUsrWnhVMlV5R21FN1J4bW5YWDBYaEVrNFVzeApmZVZCc01aS0NSWXlHeXN2QXVSc3Z4UUdJRDhvcnB5WDVZLzRMQzlScTZqTHd1LzhUNHBJOFk0VUxWR3VqVHprCi9OK3RGd2tYTUhuWkpGVFExN0tRM2lTVEFqcjg3TFVQSjdpd0dveTJiSEtCeXg1eGVSYWdpdHVJZTBUWGlwOTgKRzNCRVpkelBHeFhjalk2bFdlM0h2eXNiUHlrWnl4c3djSXZha0N0MThpakNCNXpiUXFCTGV2ZlpIa3NlWTJNMApPcjBPa0FFaUFtVkVML3dMK2JyYVd6YWw4UEJPd0VENXVKZ1FnYmZqQk1lYzJXNi9CMllscnBJWG1kYTdMb0VDClVnZGZKcTBnMUVZaFEyNEJBVjlyODFocFdQaHA5L0hwZE5FMXlGdWplZnhsQWdNQkFBR2pVekJSTUIwR0ExVWQKRGdRV0JCUUNrQnVKRENabndCUmttdlRVdFJUVCtWYXAvakFmQmdOVkhTTUVHREFXZ0JRQ2tCdUpEQ1pud0JSawptdlRVdFJUVCtWYXAvakFQQmdOVkhSTUJBZjhFQlRBREFRSC9NQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUFhClZtdlhZakM2aVNIQ2tldHB6dlFINzNBc1R4NUd5V2tmYmJjcHMvNWV1MWljUGhkWWl2TUttTjZMRUhzVjN0c1gKQXpEblRSOU9wOUxhSkRKZHhWYTVZcGx5V0o5bUtTcTRoaGdWZ0J6aW0yc3h6UW1DNFVNM3dUYUFJTXJJRU82cApUangyU0EydVF2Q3hWYnFVU2hLK3VkT3dxZG1wUUFMSDF5emtvTlNuQ0JzZ05jSE9WMzVTRHZBS2d0cVdnZjlYCjFrcWNacmRDMzhndkFKQXBhcmI5QW1sbit0a0dvNWZJV3FoN0dQb0J1TjJBWnpkUG5WbXE1RytqckxPdWlYUHYKSHdUUGdnTkx0aDdxTndtQkZpRHliSXlZMUQ1WHNLWjdLWWNSdi9uWlhyLy9NRlRWZVgxVmxoaGZnK3I2SGRXQQphVUVoTHBZNHJkUUxVQVJEdklIMwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==<\/pre>\n<h3 id=\"target-cluster-credentials\">Credentials from the &#8220;target&#8221; cluster<\/h3>\n<p>If your &#8220;target&#8221; cluster requires authentication, you will need to create credentials for MirrorMaker 2 to use.<\/p>\n<p>If your &#8220;target&#8221; cluster is managed by Strimzi, you could do this by creating a <code>KafkaUser<\/code> resource something like this (below).<\/p>\n<p><strong>Note:<\/strong> This is <strong>not<\/strong> the same as before, as the credentials for the &#8220;target&#8221; Kafka cluster need to allow creating topics.<\/p>\n<pre style=\"max-height: 107px; padding: 10px; overflow-y: scroll; border: thin black solid; font-size: smaller;\">apiVersion: kafka.strimzi.io\/v1beta1\nkind: KafkaUser\nmetadata:\n  name: mm2-credentials\n  labels:\n    strimzi.io\/cluster: target\n  namespace: target-ns\nspec:\n  authentication:\n    type: scram-sha-512\n  authorization:\n    acls:\n      - host: '*'\n        operation: Read\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Write\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Describe\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: DescribeConfigs\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: AlterConfigs\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Alter\n        resource:\n          name: '*'\n          patternType: literal\n          type: topic\n      - host: '*'\n        operation: Create\n        resource:\n          type: cluster\n      - host: '*'\n        operation: Alter\n        resource:\n          type: cluster\n      - host: '*'\n        operation: DescribeConfigs\n        resource:\n          type: cluster\n      - host: '*'\n        operation: Read\n        resource:\n          name: '*'\n          patternType: literal\n          type: group\n      - host: '*'\n        operation: Describe\n        resource:\n          name: '*'\n          patternType: literal\n          type: group\n    type: simple<\/pre>\n<p>This will create a new secret with your credentials in the namespace where your &#8220;target&#8221; Kafka cluster is running. Copy these into a secret in a namespace on the Kubernetes cluster where you want to run MirrorMaker 2.<\/p>\n<pre style=\"max-height: 107px; padding: 10px; overflow-y: scroll; border: thin black solid; font-size: smaller;\">apiVersion: v1\nkind: Secret\nmetadata:\n  name: target-cluster-credentials\n  namespace: migration\ndata:\n  password: TDQzaEoxVVlSMUds<\/pre>\n<p>These credentials will allow MirrorMaker to create and produce to topics on the &#8220;target&#8221; cluster. You will need to do something similar using the type of Kafka cluster you are running. However you create your credentials, you should create a <code>Secret<\/code> similar to the one above.<\/p>\n<h3 id=\"target-cluster-ca-cert\">TLS certificate from the &#8220;target&#8221; cluster<\/h3>\n<p>If your &#8220;target&#8221; cluster requires TLS, you may need to obtain a CA cert for MirrorMaker 2 to use.<\/p>\n<p>If your &#8220;target&#8221; cluster is managed by Strimzi, you will be able to find this in a <code>Secret<\/code> called something like <code>target-cluster-ca-cert<\/code> in the namespace where your Kafka cluster is running.<\/p>\n<p>Copy this into a secret in a namespace on the Kubernetes cluster where you want to run MirrorMaker 2.<\/p>\n<pre style=\"max-height: 107px; padding: 10px; overflow: scroll; border: thin black solid; font-size: smaller;\">apiVersion: v1\nkind: Secret\nmetadata:\n  name: target-cluster-ca-cert\n  namespace: migration\ndata:\n  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURMVENDQWhXZ0F3SUJBZ0lKQVAzUVdiZGs4UGFPTUEwR0NTcUdTSWIzRFFFQkN3VUFNQzB4RXpBUkJnTlYKQkFvTUNtbHZMbk4wY21sdGVta3hGakFVQmdOVkJBTU1EV05zZFhOMFpYSXRZMkVnZGpBd0hoY05NakV3TXpJMApNVEV4TkRFMldoY05Nakl3TXpJME1URXhOREUyV2pBdE1STXdFUVlEVlFRS0RBcHBieTV6ZEhKcGJYcHBNUll3CkZBWURWUVFEREExamJIVnpkR1Z5TFdOaElIWXdNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUIKQ2dLQ0FRRUF3T2NQbVl1d0dzbkcrRmFkR0syYmF4azJQVFFVL282SDU3Z0xobU90a2tLc1hISVVXVmxpMnFJeQpwTCtGZ3FrV215RG9Ia1JicmlHUzZObm9QMnN1RlEvMThNUnBkU0ltSHZvQ1E3Y1plcVpqSDdlaFBnWVA0YlQ5Cmp3WUtxRVU2NXFmTFc3eHZnanp3bWl3T0RmbGpKSDF6TTN4aCthRjFFbEpBWlBzQ2YzTVY4TkVOamw2ZGpHdEEKcWFmZkczYk5tZ1FzcmZBVGhWcU5ibDMrOFpSOUJUcThuazBDN2JCUEJhb3Ewa3BuTXNHdHdTcWhPaGVLcTdreApkTXhQNFV3N2wzY3ZsSHZKellEZXhSRk1JMUViZkZIQXJmcGxjcCt5ZjJRR3JRU1BpVzVUdHgxNXhLWTJYRlNmCnRNSWxjNVRUeEdZUndtaWtGNnU2cEtxN3RYTW9KUUlEQVFBQm8xQXdUakFkQmdOVkhRNEVGZ1FVYjZ0RUdVcnkKbkRuR2VZUXJEODl3WUR4Wjg0MHdId1lEVlIwakJCZ3dGb0FVYjZ0RUdVcnluRG5HZVlRckQ4OXdZRHhaODQwdwpEQVlEVlIwVEJBVXdBd0VCL3pBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQU1FTVhwem9PUWlIdVc4Z3hJd3dXClY1b1Z3YlFOa3V6c1FYeVVRbmFQRGhhQ1pTRGo0TG1TRUdQa3NlRW5JeStoQXBPY1RCSGRNY203MTJIZUlNRmIKME44cGc3VW1XeHRPM0h3YXlnRzUwTEJCVHlDVkZOd2tMWDcrdU1zMFA5d3QzdTNSNzNGVVAyOUZRS0NNMExGeQpLcFJFek0zY004dGlSdnhRc2Q0QVdQQmZYZW1nQTZEcFM5Q2tmTzBrcU9vWWZmMWpqNUp5Z2JZM09BL3huNStXCll5RVZIYkcxaThMUFI0VkpKREVyMWh3MmNmQnN6d291elMrUUtzVWsxVzM3YkZhWjIxWHZiVzRmOVdKYUFzRUIKdEQ2Y3VVL3l3a3lRTTZWZHB4ZlFxSEdGQnFoUzh0VS9qTTVMTURLRXgvZ0xzWTQ1b1VlRHYvZkhyOHZ5T0loMQp5UT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K<\/pre>\n<h2 id=\"start-mm2\">Start MirrorMaker 2<\/h2>\n<p>Create the following <code>KafkaMirrorMaker2<\/code> resource so that the Strimzi Operator can configure MirrorMaker 2 to start the data migration.<\/p>\n<p>The configuration options below are commented in the definition to explain where you may want to customize the values for your own use.<\/p>\n<pre id=\"mm2-config\" style=\"padding: 10px; overflow-y: scroll; border: thin black solid; font-size: smaller;\">apiVersion: kafka.strimzi.io\/v1alpha1\nkind: KafkaMirrorMaker2\nmetadata:\n  name: data-migration\n  namespace: migration\nspec:\n  <span style=\"color: darkgreen;\"># How many instances of MirrorMaker 2 do you want to run in parallel?\n  #\n  # If you have a large \"origin\" cluster with a lot of data to migrate\n  #  then you can increase this value.<\/span>\n  replicas: 1\n\n\n  clusters:\n\n  # connection details for your \"origin\" cluster\n  - alias: \"origin-cluster\"\n    <span style=\"color: darkgreen;\"># change this to provide the bootstrap servers address for your \"origin\" cluster<\/span>\n    bootstrapServers: \"origin-kafka-bootstrap-origin-ns.apps.my-origin-cluster-name.cp.fyre.ibm.com:443\"\n    <span style=\"color: darkgreen;\"># remove this section if your \"origin\" cluster does not require authentication<\/span>\n    authentication:\n      <span style=\"color: darkgreen;\"># update this to match the authentication method used by your \"origin\" cluster<\/span>\n      type: scram-sha-512\n      username: mm2-credentials\n      passwordSecret:\n        <span style=\"color: darkgreen;\"># name of the secret where you stored the\n        #  credentials for your \"origin\" cluster<\/span>\n        secretName: <a href=\"#origin-cluster-credentials\">origin-cluster-credentials<\/a>\n        <span style=\"color: darkgreen;\"># name of the key in the secret that contains\n        #  the password<\/span>\n        password: password\n    <span style=\"color: darkgreen;\"># remove this section if your \"origin\" cluster does not require a custom TLS CA certificate<\/span>\n    tls:\n      trustedCertificates:\n        <span style=\"color: darkgreen;\"># name of the secret where you stored the\n        #  TLS certificate for your \"origin\" cluster<\/span>\n        - secretName: <a href=\"#origin-cluster-ca-cert\">origin-cluster-ca-cert<\/a>\n          <span style=\"color: darkgreen;\"># name of the key in the secret that contains\n          #  the certificate<\/span>\n          certificate: ca.crt\n\n  # connection details for your \"target\" cluster\n  - alias: \"target-cluster\"\n    <span style=\"color: darkgreen;\"># change this to provide the bootstrap servers address for your \"target\" cluster<\/span>\n    bootstrapServers: \"target-kafka-bootstrap-target-ns.apps.my-target-cluster-name.cp.fyre.ibm.com:443\"\n    <span style=\"color: darkgreen;\"># remove this section if your \"target\" cluster does not require authentication<\/span>\n    authentication:\n      <span style=\"color: darkgreen;\"># update this to match the authentication method used by your \"target\" cluster<\/span>\n      type: scram-sha-512\n      username: mm2-credentials\n      passwordSecret:\n        <span style=\"color: darkgreen;\"># name of the secret where you stored the\n        #  credentials for your \"target\" cluster<\/span>\n        secretName: <a href=\"#target-cluster-credentials\">target-cluster-credentials<\/a>\n        <span style=\"color: darkgreen;\"># name of the key in the secret that contains\n        #  the password<\/span>\n        password: password\n    <span style=\"color: darkgreen;\"># remove this section if your \"target\" cluster does not require a custom TLS CA certificate<\/span>\n    tls:\n      trustedCertificates:\n        <span style=\"color: darkgreen;\"># name of the secret where you stored the\n        #  TLS certificate for your \"target\" cluster<\/span>\n        - secretName: <a href=\"#target-cluster-ca-cert\">target-cluster-ca-cert<\/a>\n          <span style=\"color: darkgreen;\"># name of the key in the secret that contains\n          #  the certificate<\/span>\n          certificate: ca.crt\n    config:\n      <span name=\"mm2-state-topics\" style=\"color: darkgreen;\"># These topics will be created on the \"target\" Kafka\n      #  cluster for MirrorMaker 2 to store it's state.\n      # Make sure that these names don't match the names of any\n      #  of your existing topics.\n      # We will delete these topics once MirrorMaker 2 has finished.<\/span>\n      offset.storage.topic: migration-connect-cluster-offsets\n      config.storage.topic: migration-connect-cluster-configs\n      status.storage.topic: migration-connect-cluster-status\n\n  connectCluster: \"target-cluster\"\n\n  mirrors:\n    - sourceCluster: \"origin-cluster\"\n      targetCluster: \"target-cluster\"\n      sourceConnector:\n        config:\n          <span id=\"mm2-replication-factor\" style=\"color: darkgreen;\"># the replication factor that will be used for\n          #  all topics created on the \"target\" Kafka cluster<\/span>\n          replication.factor: 1\n\n          # don't try to copy permissions across from the \"origin\"\n          #  cluster to the \"target\" cluster\n          sync.topic.acls.enabled: \"false\"\n          # create topics on the \"target\" cluster with names that\n          #  match the names of the topics on the \"origin\" cluster\n          replication.policy.class: \"io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy\"\n          replication.policy.separator: \"\"\n          # syncing offsets\n          offset-syncs.topic.replication.factor: 1\n\n      checkpointConnector:\n        config:\n          checkpoints.topic.replication.factor: 1\n          refresh.groups.interval.seconds: 600\n          # migrates the consumer group offsets\n          emit.checkpoints.enabled: true\n          sync.group.offsets.enabled: true\n          sync.group.offsets.interval.seconds: 60\n          emit.checkpoints.interval.seconds: 60\n          # ensures that consumer group offsets on the \"target\" cluster\n          #  are correctly mapped to consumer groups on the \"origin\" cluster\n          replication.policy.class: \"io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy\"\n          replication.policy.separator: \"\"\n\n      <span style=\"color: darkgreen;\"># Which topics should be migrated from the\n      #  \"origin\" cluster to the \"target\" cluster ?\n      # If you don't want to migrate all of your topics, modify this pattern to\n      #  match only the topics you want. <\/span>\n      topicsPattern: \".*\"\n\n      <span style=\"color: darkgreen;\"># Which consumer groups should be migrated from the\n      #  \"origin\" cluster to the \"target\" cluster ?\n      # If you don't want to migrate all of your groups, modify this pattern to\n      #  match only the groups you want. <\/span>\n      groupsPattern: \".*\"\n<\/pre>\n<h2 id=\"stop-mm2\">Stop MirrorMaker 2 and clean up<\/h2>\n<p>Once MirrorMaker 2 has finished migrating all of your topics, you can delete it and the resources that it created while running.<\/p>\n<ul>\n<li>Delete the <code>data-migration<\/code> KafkaMirrorMaker2 resource<\/li>\n<li>Delete the three <a href=\"#mm2-state-topics\">topics that MirrorMaker 2 creates<\/a> to store it&#8217;s state (in the config above, these start with the name <code>migration-connect-cluster-<\/code>)<\/li>\n<\/ul>\n<p>Don&#8217;t delete the <code>origin-cluster.checkpoints.internal<\/code> topic yet, until you&#8217;ve verified your consumer group offsets in the final step.<\/p>\n<h2 id=\"update-topic-replication-factors\">Update your topic replication factors<\/h2>\n<p>MirrorMaker 2 does not migrate the replication factor for topics it creates on the &#8220;target&#8221; cluster to match the &#8220;origin&#8221; cluster. All topics that it creates on the &#8220;target&#8221; cluster will have the replication factor you <a href=\"#mm2-replication-factor\">specify in the MirrorMaker 2 config<\/a>.<\/p>\n<p>You should update the replication factor on the topics on your &#8220;target&#8221; cluster to match your requirements before continuing.<\/p>\n<h2 id=\"resume-client-applications\">Resume your client applications<\/h2>\n<p>Kafka client applications that were using the &#8220;origin&#8221; Kafka cluster can now start using the &#8220;target&#8221; Kafka cluster.<\/p>\n<p>MirrorMaker 2&#8217;s <code>MirrorCheckpointConnector<\/code> automatically store consumer group offset checkpoints for consumer groups on the &#8220;origin&#8221; Kafka cluster. Each checkpoint maps the last committed offset for each consumer group in the &#8220;origin&#8221; cluster to the equivalent offset in the &#8220;target&#8221; cluster.<\/p>\n<p>These checkpoints are stored in the <code>origin-cluster.checkpoints.internal<\/code> topic on the &#8220;target&#8221; cluster.<\/p>\n<p>Since Kafka 2.7.0, MirrorMaker 2 has been able to translate the consumer group offsets for the &#8220;target&#8221; cluster, so Kafka consumers that start consuming from the same topic on the &#8220;target&#8221; cluster will be able to resume receiving messages from the last offset they committed on the &#8220;origin&#8221; cluster.<\/p>\n<p>Once you have confirmed that your Kafka consumers are able to resume from their previously stored offsets, and no longer need the offset checkpoints that MirrorMaker 2 has stored, you can delete the <code>origin-cluster.checkpoints.internal<\/code> topic.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>You have a Kafka cluster that you have been using for a while. Your cluster has many topics, and the topics have many messages. Now you&#8217;ve decided to move and start using a new, different Kafka cluster somewhere else. How can you take your topics with you? Huge thanks to Andrew Borley for co-writing this [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[593,584,601],"class_list":["post-4303","post","type-post","status-publish","format-standard","hentry","category-code","tag-apachekafka","tag-kafka","tag-strimzi"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/4303","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4303"}],"version-history":[{"count":0,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/4303\/revisions"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4303"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4303"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4303"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}