Archive for the ‘code’ Category

« Older Entries
Newer Entries »

The Scratch coordinate system

Tuesday, July 23rd, 2019

In Scratch 3, the stage in the top right where your sprites live is implemented as an HTML canvas. Unfortunately the internal coordinate system used by Scratch logically to maintain state, and the coordinate system used by HTML canvases both work very differently.

For some of the Scratch blocks I’ve written for Machine Learning for Kids, I need to be able to convert between coordinates and sizes between the two different coordinate systems.

For example, my ML blocks can let a student use an image classifier they’ve trained to recognise what is on the background behind a certain Sprite in their project. To do that, the backdrop image block needs to:

  1. get the location of the Sprite (which will be returned using the Scratch coordinate system)
  2. get the image data of what is rendered on the canvas at that location (using HTML canvas APIs – using the HTML coordinate system)

I couldn’t find a way to convert between the two documented anywhere, and it was a tiny bit fiddly, so I’m documenting it here for the next time I need it!

(more…)

Tags:,
Posted in code | Comments Closed

How to write your first Avro schema

Saturday, July 20th, 2019

Any time there is more than one developer using a Kafka topic, they will need a way to agree on the shape of the data that will go into messages. The most common way to document the schema of messages in Kafka is to use the Apache Avro serialization system.

This post is a beginner’s guide to writing your first Avro schema, and a few tips for how to use it in your Kafka apps.

(more…)

Tags:, , , ,
Posted in code | 2 Comments »

An introduction to serverless and OpenWhisk for Kafka users

Saturday, July 13th, 2019

I gave a talk at Kafka Summit London this year about Apache OpenWhisk. It was aimed at Kafka users who want to know what the serverless hype is all about.

I covered:

  • a simple introduction of what serverless is for
  • an introduction to some of the serverless platforms available
  • a quick crash course in how to get started with Apache OpenWhisk

I also had a quick tangent looking into how Apache OpenWhisk itself uses Kafka internally, because I thought that was interesting!

My slides are on SlideShare if you’d like to see a higher-res version of any of them.

If this convinces you to give OpenWhisk a try, I have a post on how to get started with OpenWhisk that has all the commands you need to copy/paste to get yourself a working OpenWhisk environment connected to a Kafka source of events.

(more…)

Tags:, , ,
Posted in code | Comments Closed

Getting started with OpenWhisk and Kafka

Saturday, July 6th, 2019

Apache OpenWhisk (and serverless platforms in general) are a great way to host and manage code that you want to run in response to events.
Apache Kafka topics are a great source of events.

In this post, I’ll run through a super simple beginner’s guide to writing code for OpenWhisk that processes events on your Kafka topics.

(more…)

Tags:, ,
Posted in code | Comments Closed

Using Node-RED with IBM Event Streams

Friday, June 28th, 2019


Click to enlarge

IBM Event Streams is the distributed streaming real-time data platform Apache Kafka, from IBM.

Node-RED is a visual flow-based development tool, with nodes that you drag and drop onto a canvas and wire together. It’s useful for loads of tasks, such as quick and flexible prototyping.

In this post, I’ll show how Event Streams and Node-RED work well together. You can use Node-RED to quickly and easily create flows that consume messages from Kafka topics, or that process events from different sources and produce the output to Kafka topics.

(more…)

Tags:, , ,
Posted in code, ibm | Comments Closed

Using kafkacat and kaf with IBM Event Streams

Sunday, June 9th, 2019

IBM Event Streams is IBM’s Kafka offering. Naturally it comes with it’s own UI and CLI tools, but one of the great things about Apache Kafka is that it’s not just a single thing from a single company – rather it is an active and diverse ecosystem, which means you’ve got a variety of tools to choose from.

I thought I’d try a couple of open source CLI tools, and share how to connect them and what they can do.

First up, kafkacat.

(more…)

Tags:, ,
Posted in code, ibm | Comments Closed

Using nginx to enable custom domains in Auth0

Sunday, December 2nd, 2018

In this post, I’m sharing the nginx config needed to set up a reverse proxy for enabling custom domains in Auth0.

I’ll start by explaining why I needed this, but if you don’t care about that, you can just skip straight to the code.

I’ve mentioned before that I use Auth0 for authentication and authorization in Machine Learning for Kids. (And I’ve mentioned before that the developer experience using Auth0 is generally fantastic).

But I’ve had one auth-related complaint from schools since launching the site. And it’s been increasing in the last couple of months:

Users who have their web browsers set to block third-party cookies weren’t able to log in.

This is because the cookies were coming from an auth0.com domain, and not machinelearningforkids.co.uk.

(more…)

Tags:, ,
Posted in code | Comments Closed

How to create a Twitter API proxy using nginx in Cloud Foundry

Sunday, November 18th, 2018

In this post, I’ll describe how to run nginx in Cloud Foundry to provide a Twitter API proxy that includes authentication and caching.

First, I want to talk a bit about why I wanted this, but if you don’t care about any of that, you can just skip to the code at the end of the post. 🙂

I’ve wanted for a while to enable projects in Machine Learning for Kids that use tweets. Using live tweets is a great way to make text analytics real for students, and a good example of how natural language processing is used in the real world.

The question was how to enable this from Scratch in a way that would be easy to use by schools.

The title of this post gives away the answer I ended up with, but I’ll describe why.

(more…)

Tags:, ,
Posted in code | Comments Closed

« Older Entries
Newer Entries »