{"id":3808,"date":"2019-07-28T12:07:32","date_gmt":"2019-07-28T12:07:32","guid":{"rendered":"https:\/\/dalelane.co.uk\/blog\/?p=3808"},"modified":"2019-07-28T12:44:58","modified_gmt":"2019-07-28T12:44:58","slug":"using-openwhisk-in-machine-learning-for-kids","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=3808","title":{"rendered":"Using OpenWhisk in Machine Learning for Kids"},"content":{"rendered":"

I’ve moved a couple of bits of Machine Learning for Kids<\/a> into OpenWhisk<\/a> functions. In this post, I’ll describe what I’m trying to solve by doing this, and what I’ve done.<\/strong><\/p>\n

Background<\/h3>\n

I’ve talked before how I implemented Machine Learning for Kids<\/a>, but the short version is that most of it is a Node.js app, hosted in Cloud Foundry so I can easily run multiple instances of it.<\/p>\n

The most computationally expensive thing the site has to do is for projects that train a machine learning model to recognize images.<\/p>\n

<\/p>\n

In particular, the expensive bit is when a student clicks on the Train new machine learning model<\/strong> button for a project to train the computer to recognize images.<\/p>\n

When they do that, at a high level, this is what has to happen:<\/p>\n

    \n
  1. All of their images will be downloaded<\/strong>\n
      \n
    1. Images they created themselves (e.g. images they\u2019ve drawn, photos they\u2019ve taken with the webcam, etc.) will be retrieved from Cloud Object Storage<\/li>\n
    2. Images they found on the web (which I only store as URLs) are downloaded from their original websites<\/li>\n<\/ol>\n<\/li>\n
    3. Downloaded images will be resized<\/strong> to a size suitable for training (images created and stored in Cloud Object Storage will already be the right size<\/em>)<\/li>\n
    4. A new zip file is created<\/strong> with all of the images, ready for uploading to the Watson Visual Recognition service (sometimes more than once using different Watson credentials<\/em>)<\/li>\n<\/ol>\n

      <\/p>\n

      The problems<\/h3>\n

      Compared with everything else that the site has to do, resizing large images and creating large zip files are both fairly expensive in terms of memory and CPU.<\/p>\n

      This is compounded by the fact that it tends to happen in bursts.<\/p>\n

      If I’m unlucky, a class of thirty students all click their Train ML<\/strong> button at roughly the same time.<\/p>\n

      When I\u2019m really unlucky, a STEM event of a hundred students all click Train ML<\/strong> button at the same time!<\/p>\n

      (Teachers often get their classes to go through projects together, so they all move on to the next step at the same time. It is more common than you’d expect for dozens of training requests to hit my site at once).<\/p>\n

      This means I had two problems:<\/p>\n

      Problem 1 : The site was horribly over-provisioned<\/h4>\n

      The rest of the site server is efficient and light-weight, using hardly any CPU, memory or local disk.<\/p>\n

      But I had to provision it to cope with the worst possible spikes without crashing with out of memory errors. So that means I scaled up the memory footprint of the site instances to a level that has been far more than the site has needed most of the time.<\/p>\n

      I’m running the site on a shoestring budget, so I can’t afford to run it with far more resource than it needs!<\/p>\n

      Problem 2 : I had to throttle it to be slower than it could be<\/h4>\n

      To cope with the worst spikes without blowing up with out-of-memory errors, I was fairly aggressive about avoiding parallelising the task described above.<\/p>\n

      For example, instead of processing all of a student’s images at once in parallel, I’d limit it to do a couple at a time<\/a>. That wasn’t necessary if only one or two students had clicked the Train<\/strong> button – it would’ve worked okay to just do lots, if not all, of their images in parallel. But to protect against a possible spike of loads of students, I was doing it slower for everyone.<\/p>\n

      Training image models is already annoyingly slow. Making it even slower isn’t great for a tool used by classes of impatient children!<\/p>\n

      Both of these problems were only getting worse.<\/h4>\n

      The site continues to get busier, and I’m getting more and more schools and code clubs using the site. I was rapidly approaching a point where I’d have to scale up the site app instances even more to avoid the risk of out-of-memory errors.<\/p>\n

      <\/p>\n

      The solution<\/h3>\n

      You can probably guess from the fact that I’ve been talking about OpenWhisk recently<\/a> (or just the title and intro to this post!) that I think these problems are well suited to serverless.<\/p>\n