Using TensorFlow.js for training image classifiers

Machine Learning for Kids lets students train their own machine learning models in a simplified child-friendly training tool. A variety of project types are supported (such as classifying text, images, numeric data, sound recordings, etc.). Under the covers, machine learning models they train are created and hosted using IBM Watson cloud services, such as Watson Assistant and Watson Visual Recognition

I’m currently investigating image projects being created and hosted in the browser, without using Watson cloud API calls.

I’ve got something working using TensorFlow.js but I’m restricting access to a small group of beta testers for now until I’ve validated that this is a good idea or not.

Is this a good idea?

The main argument I can think of in favour is speed.

The Visual Recognition cloud service sometimes queues up training requests when the service is busy, so you could have to wait for five minutes for the model to be ready to use. In normal “workplace” usage, this isn’t a major problem – a lot of developers I know don’t need much of an excuse to go make a coffee. But in a classroom setting, it’s a pain – five minutes waiting for a model to train is disruptive. (I try to mitigate this by providing quiz questions while they wait, and encouraging students to work on their Scratch projects while they wait… but even so – it’s a pain). In contrast, the simple models I’m training in the browser should be ready in seconds.

I can think of a few arguments against.

I’m nervous about the hardware requirements. Doing all the ML training and prediction computation in the cloud means students can use very old, very slow, very low-powered computers. For example, I know that many code clubs use the site with laptops based on the Raspberry Pi, without any problems. Will the ML implementation I’ve done in JavaScript work okay on those sorts of systems? I don’t know – so far I’ve only tried it on my MacBook Pro, which is hardly representative.

I’m nervous about the sorts of results it will give. I’m talking about going from a cloud service running on systems with access to fast GPUs and large compute resources, to a tiny lightweight simplified implementation running on a student machine. Will it give results that are good enough for the sorts of projects that students are making, with the tiny training sets that they typically collect?

It means I’ll have to implement this a few separate times. ML for Kids has integrations with several different tools, such as Scratch 3, ScratchX, App Inventor, and, with more on the way such as EduBlocks. And each of these is implemented in a different language (Flash, JavaScript, Java, Python).

I’ve started by implementing support for training image classifiers in JavaScript using TensorFlow.js for use in Scratch 3. For App Inventor projects, I’ll have to do this again but in Java. And then again in Python for use in And so on. The benefit of doing everything in a cloud service means I have one implementation that can be used in several integrations. Doing everything natively means I’ll have to write and maintain a lot more code to support the same range of integrations. That’s a lot. Maybe too much.

All of this means I’m moving forwards cautiously with this idea. Before I go further, I want to see how it behaves with the range of projects that students make on the site, and on more types of computers.

If you’re interested in joining the beta test for this, please let me know!


One Response to “Using TensorFlow.js for training image classifiers”

  1. Mash says:

    Hi Dale,
    I am interested to help/test