I’ve started adding pretrained machine learning models to Machine Learning for Kids. In this post, I wanted to describe what I’m doing.
Archive for the ‘code’ Category
Pretrained models in Machine Learning for Kids
Monday, May 25th, 2020
Geo-spatial data in Scratch
Wednesday, November 13th, 2019In this post, I want to share a random thing I made in Scratch this week, and ask for suggestions of what I could do with it.
I get a lot of emails from teachers and coding groups asking for help with Scratch projects. They’re normally small or specific questions – asking for help figuring out a bug in a Scratch project or how to get something working.
But this week I got a more challenging email. It asked for a way to show a map in Scratch, and use a Scratch script to plot points on the map, given coordinates in latitude and longitude.
I agreed to give it a try. (Details for how to access it below.)
Using TensorFlow with IBM Event Streams
(Kafka + Machine Learning = Awesome)
Thursday, October 31st, 2019
In this post, I want to explain how to get started creating machine learning applications using the data you have on Kafka topics.
I’ve written a sample app, with examples of how you can use Kafka topics as:
- a source of training data for creating machine learning models
- a source of test data for evaluating machine learning models
- an ongoing stream of events to make predictions about using machine learning models
I’ll use this post to explain how it works, and how you can use it as the basis of writing your first ML pipeline using the data on your own Kafka topics.
Using Avro schemas from Python apps with IBM Event Streams
Thursday, October 17th, 2019I’ve written before about how to write a schema for your developers using Kafka. The examples I used before were all in Java, but someone asked me yesterday if I could share some Python equivalents.
The principles are described in the Event Streams documentation, but in short, your Kafka producers use Apache Avro to serialize the message data that you send, and identify the schema that you’ve used in the Kafka message header. In your Kafka consumers, you look at the headers of the messages that you receive to know which schema to retrieve, and use that to deserialize message data.
SQL queries on Kafka topics using Apache Hive
Tuesday, August 6th, 2019Apache Hive is open source data warehouse software built on top of Hadoop. It gives you an SQL-like interface to a wide variety of databases, filesystems, and other systems.
One of the Hive storage handlers is a Kafka storage handler, which lets you create a Hive “external table” based on a Kafka topic.
And once you’ve created a Hive table based on a Kafka topic, you can run SQL queries based on attributes of the messages on that topic.
I was having a play with Hive this evening, as a way of running SQL queries against messages on my Kafka topics. In this post, I’ll share a few queries that I tried.
Using OpenWhisk in Machine Learning for Kids
Sunday, July 28th, 2019I’ve moved a couple of bits of Machine Learning for Kids into OpenWhisk functions. In this post, I’ll describe what I’m trying to solve by doing this, and what I’ve done.
Background
I’ve talked before how I implemented Machine Learning for Kids, but the short version is that most of it is a Node.js app, hosted in Cloud Foundry so I can easily run multiple instances of it.
The most computationally expensive thing the site has to do is for projects that train a machine learning model to recognize images.
In particular, the expensive bit is when a student clicks on the Train new machine learning model button for a project to train the computer to recognize images.
The Scratch coordinate system
Tuesday, July 23rd, 2019In Scratch 3, the stage in the top right where your sprites live is implemented as an HTML canvas. Unfortunately the internal coordinate system used by Scratch logically to maintain state, and the coordinate system used by HTML canvases both work very differently.
For some of the Scratch blocks I’ve written for Machine Learning for Kids, I need to be able to convert between coordinates and sizes between the two different coordinate systems.
For example, my ML blocks can let a student use an image classifier they’ve trained to recognise what is on the background behind a certain Sprite in their project. To do that, the backdrop image block needs to:
- get the location of the Sprite (which will be returned using the Scratch coordinate system)
- get the image data of what is rendered on the canvas at that location (using HTML canvas APIs – using the HTML coordinate system)
I couldn’t find a way to convert between the two documented anywhere, and it was a tiny bit fiddly, so I’m documenting it here for the next time I need it!
How to write your first Avro schema
Saturday, July 20th, 2019
Any time there is more than one developer using a Kafka topic, they will need a way to agree on the shape of the data that will go into messages. The most common way to document the schema of messages in Kafka is to use the Apache Avro serialization system.
This post is a beginner’s guide to writing your first Avro schema, and a few tips for how to use it in your Kafka apps.