dale lane

I gave a short presentation today to explain how you can use TensorFlow machine learning models in the student block-based coding platform, Scratch.

This post has the recording of my presentation, and I’ve put some notes (all the stuff I meant to say but forgot!) and screenshots below.

recording at https://youtu.be/qHKwtefn21w

I demonstrated three things:

1. Using your own TensorFlow models in Scratch
2. Using pretrained models in Scratch
3. Creating TensorFlow models in Teachable Machine and using them in Scratch

If you know how to create your own models…

If you know some Python, you can build your own model. You can create simple tiny models. Or big complex models that take hours to train.

Either way, once you have a model, you can use TensorFlow.js converter to convert your model to a form that can be used in a web browser.

Once converted, you need to host it somewhere online that Scratch will be able to access (which means a server that won’t complain about CORS, if you know what that means).

The Machine Learning for Kids website has a tool for launching Scratch with TensorFlow models – you just need to provide the URL for where your model is hosted.

I demonstrated it with a random imagenet model I found – using the machine learning model from Scratch to recognize the object in photos.

It actually did better than I thought at the time. It classified a photo of a cat as a “tiger cat” which I thought was a mistake while I was demo’ing, but apparently that’s a way of describing some types of tabby cat.

The point is, if you know how to create custom, bespoke machine learning models in TensorFlow, it’s simple to make them available for students to play with in Scratch.

If you can find pretrained models…

I didn’t actually train the imagenet model that I demonstrated myself. I got it from TensorFlow Hub.

That was the second point in my talk – that if you don’t mind poking around sites like GitHub and TensorFlow Hub, then there are loads of interesting models around to try.

Not all TF models will work with the Scratch plugin in Machine Learning for Kids. It’s very much an early, experimental feature. I’ve started with support for some types of image models, but I’m working on broadening it out to support more model types which should be available soon.

If you can make TensorFlow models in Teachable Machine…

This was the final demo, to make sure that everyone had at least something they could try.

I used Teachable Machine to train an image classifier to recognize a few Doctor Who minifigs I have on my desk.

My plan was to train it to recognize all of them, but that took a lot longer than I expected so I ended up doing just a few to show the idea.

The nice thing about Teachable Machine is that not only will it export TensorFlow models for you, but that Google will host them for you.

This is something that Node-RED has had support for for ages, which is partly why I wanted to try doing something similar in Scratch (which is sort of like Node-RED but for kids!).

I did a quick bit of live-coding demo to show how you could use this to get a Scratch project to recognize the Doctor Who minifig visible in the webcam.

Amazingly I managed to get through the whole demo without noticing that I’d left the webcam flipped, so my “clever” attempt at adding a relevant background was a little bit spoiled. Woops.

I got a bit distracted playing with trying to get my webcam to focus on the tiny minifigs, so I probably spent longer on that demo than I needed to. Just getting a Scratch script to show what is in the webcam is fairly boring – as it’s just recreating what is already shown in the Teachable Machine web page.

The more interesting opportunities are in making projects that respond in some way to what the machine learning model has recognized.

For example, while I was playing with the demo, I thought of adding blocks that could play different sound effects for each Doctor (if I’d planned ahead and thought of that ahead of the demo, I could’ve got some sound files ready!), or adding a TARDIS sprite with costumes for each design of TARDIS, that changes to match the Doctor that is visible.

(I really like that idea, but I expect I’ll get myself into copyright trouble if I try and write a worksheet based on it… shame – I might still make the project anyway!)

I gave a short presentation on using #TensorFlow models in #Scratch. The recording includes the live demo of how I made this Doctor Who themed projecthttps://t.co/VFxaubxcLH pic.twitter.com/RnyqPzgtGL

— Dale Lane (@dalelane) November 19, 2020

Tags: machine learning, mlforkids-tech, scratch, tensorflow

This entry was posted on Thursday, November 19th, 2020 at 5:39 pm and is filed under tech. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.