I-Spy (using Watson services from Scratch projects)

It’s half-term week, so that means more time for geekiness with the kids.

This is something Grace made this week: a game of “I spy” built using Scratch, that uses the Watson Vision Recognition API to let the game dynamically pick objects that it recognises in photos, so you can then make guesses.

Apart from being a fun game to make in it’s own right, I wanted to share why I particularly think it’s useful to be able to use Watson API’s from Scratch projects.

Screen Shot 2017-02-22 at 13.43.53

Machine Learning is an important field in computing, and I’ve felt for a long while now that it’s becoming increasingly important that we explain and introduce it to school kids. This is partly because we need to support the next generation of engineers and developers who will build the future. It’s perhaps more importantly because everyone should understand how the world around them works and the implications that our increasing use of AI brings.

I’ve done a bunch of projects to try and find ways to introduce kids to machine learning – lots with my own kids, but also a few with some local schools. They all aimed to explain ML by giving kids a chance to use it to make something.

For example, training an ML system to recognise a picture of hands to be able to play Rock, Paper, Scissors.
Or training an ML system to be able to play Top Trumps.
Or training an ML system to recognise attributes of faces in photos and understand questions, to be able to play Guess Who.
And so on.

The other thing they had in common was that they were fairly prescriptive. They started from a premise of “We’re going to train the computer to play <game X>”, and then that’s what we did.

I’ve felt for a while that the next step needs to be to let kids have a first-hand chance playing with ML techniques and technologies, but starting from a blank canvas that allows them to be creative, without telling them what they should do with it.

Scratch is a great model for what I mean with this. I’ve talked about Scratch before, but it’s widely used to introduce kids to programming by giving them a drag and drop canvas where they can snap together blocks representing code.

Grace’s I-Spy game is an early example of what some of this might look like. This is interesting because it shows that if you can give kids an easy way to be able to train ML models (which is something for a later post) then thanks to Scratch Extensions it’s pretty easy to let them use that from inside Scratch.

Screen Shot 2017-02-22 at 13.43.53-zoomed
This was the Watson-specific bit of the script – sending the URL of the image to the Watson API, getting back a list of objects recognised in the image.

Tags: , ,

Leave a Reply