{"id":5594,"date":"2025-06-12T22:59:25","date_gmt":"2025-06-12T22:59:25","guid":{"rendered":"https:\/\/dalelane.co.uk\/blog\/?p=5594"},"modified":"2026-03-14T21:24:52","modified_gmt":"2026-03-14T21:24:52","slug":"how-to-use-kafka-console-consumer-sh-to-view-the-contents-of-apache-avro-encoded-events","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=5594","title":{"rendered":"How to use kafka-console-consumer.sh to view the contents of Apache Avro-encoded events"},"content":{"rendered":"

kafka-console-consumer.sh<\/code> is one of the most useful tools in the Kafka user’s toolkit. But if your topic has Avro-encoded events, the output can be a bit hard to read. <\/p>\n

<\/p>\n

You don\u2019t have to put up with that, as the tool has a formatter plugin framework<\/a>. With the right plugin, you can get nicely formatted output from your Avro-encoded events. <\/p>\n

<\/p>\n

With this in mind, I\u2019ve written a new Avro formatter for a few common Avro situations. You can find it at:<\/p>\n

github.com\/IBM\/kafka-avro-formatters<\/a><\/strong><\/p>\n

The README<\/a> includes instructions on how to add it to your Kafka console command, and configure it with how to find your schema. <\/p>\n

I won’t duplicate those instructions here, but I do want to briefly call out a few ways you can use it, depending on where your Avro schema is.<\/p>\n

\n

Schemas in Apicurio Schema Registry<\/h3>\n

If you keep your schema in an Apicurio schema registry<\/a> the formatter can dynamically fetch the schemas it needs from the registry (just like your regular deserializer would).<\/p>\n

<\/p>\n

\n

Schemas in local files<\/h3>\n

If you keep your schema together with your application source code, and have it available in a local file, then the formatter can just read that local file directly. <\/p>\n

<\/p>\n

\n

Schemas in Event Endpoint Management<\/h3>\n

If you\u2019re consuming events from someone else\u2019s topic that you found in Event Endpoint Management<\/a>, then the formatter can fetch any schemas it needs from the Catalog.<\/p>\n

<\/p>\n

\n

I hope you\u2019ll find this useful. If you have any ideas for what else it could do, please feel free to create an issue in the Github repository<\/a>.<\/p>\n

Extra tips<\/h3>\n

Finally, I want to highlight a couple of quick tips for how to get more<\/a> out of the formatter:<\/p>\n

Use a properties file and a shell alias<\/h4>\n

Save yourself some typing! <\/p>\n

Instead of typing your schema registry URL and credentials in --property<\/code> options every time you use kafka-console-consumer.sh, leave them all in a properties file and create an Avro-themed alias for kafka-console-consumer that points at your properties file. <\/p>\n

(See an example of how to do this<\/a>)<\/p>\n

Pair it with jq<\/h4>\n

The formatter outputs the parsed Avro-encoded events as a JSON string. This means that you can pipe it to something like jq<\/a> for colour formatting and advanced parsing and processing. <\/p>\n

This is very useful with large and complex event data, where you just want to view specific fields. <\/p>\n

(See an example of how to do this<\/a>)<\/p>\n","protected":false},"excerpt":{"rendered":"

kafka-console-consumer.sh is one of the most useful tools in the Kafka user’s toolkit. But if your topic has Avro-encoded events, the output can be a bit hard to read. You don\u2019t have to put up with that, as the tool has a formatter plugin framework. With the right plugin, you can get nicely formatted output […]<\/p>\n","protected":false},"author":1,"featured_media":5600,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[595,593,594,583,584],"class_list":["post-5594","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-code","tag-apacheavro","tag-apachekafka","tag-avro","tag-ibmeventstreams","tag-kafka"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5594","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5594"}],"version-history":[{"count":1,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5594\/revisions"}],"predecessor-version":[{"id":5897,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5594\/revisions\/5897"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/media\/5600"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5594"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5594"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5594"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}