{"id":6116,"date":"2026-07-10T01:45:53","date_gmt":"2026-07-10T01:45:53","guid":{"rendered":"https:\/\/dalelane.co.uk\/blog\/?p=6116"},"modified":"2026-07-11T00:13:39","modified_gmt":"2026-07-11T00:13:39","slug":"generative-ai-with-tool-calling-in-scratch","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=6116","title":{"rendered":"Generative AI with tool calling in Scratch"},"content":{"rendered":"<p><strong>In this post, I want to share a new feature in the generative AI support in <a href=\"https:\/\/machinelearningforkids.co.uk\/\">Machine Learning for Kids<\/a>: tool calling.<\/strong><\/p>\n<p>I wrote last year about how I <a href=\"https:\/\/dalelane.co.uk\/blog\/?p=5538\">introduced Generative AI in Machine Learning for Kids by adding support for projects using small language models<\/a>. And earlier this year, I walked through <a href=\"https:\/\/dalelane.co.uk\/blog\/?p=5847\">my six favourite projects for explaining different aspects of Gen AI<\/a>.<\/p>\n<p>This week I&#8217;ve been working on extending the language model support in the site, by adding a new model that is capable of tool calling. <\/p>\n<p><iframe loading=\"lazy\" width=\"450\" height=\"245\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/HdcTseNvjhU?si=0CaQDLyArO7YIx8E\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><br \/>\n<small>demo video at <a href=\"https:\/\/youtu.be\/HdcTseNvjhU\">youtu.be\/HdcTseNvjhU<\/a><\/small><\/p>\n<h3>Why is tool calling useful?<\/h3>\n<p>Try asking a model what the weather is like in New York right now.<\/p>\n<p>If the temperature and top-p is high enough, many of the models will likely hallucinate an answer. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-01.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>If the temperature and top-p is low enough, the models can just respond  that they cannot answer that.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-02.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>(<em>If you don&#8217;t know why &#8220;temperature&#8221; makes that difference, I&#8217;ve <a href=\"https:\/\/dalelane.co.uk\/blog\/?p=5538\">written about that before<\/a> and have a <a href=\"https:\/\/dalelane.co.uk\/blog\/?p=5847\">student worksheet that focuses on this<\/a>.<\/em>)<\/p>\n<p>This second answer is more accurate at least, but it is still not helpful. The point is, if you ask the model for something that can&#8217;t possibly be represented in the knowledge used to create it, the model cannot give a helpful answer by itself. <\/p>\n<p><strong>Tool calling helps in these situations by making tools available to the model. It can call these tools to help respond to prompts that can&#8217;t be answered by the model alone.<\/strong> <\/p>\n<p><!--more--><\/p>\n<h3>How am I adding this to Machine Learning for Kids?<\/h3>\n<p>I&#8217;ve added a new model to the choice available on the site: a model that is capable of tool calling.<\/p>\n<p>It is the smallest one I could find, but it is big. It&#8217;s so big, it doesn&#8217;t fit on the graphs I added to help compare the model choices. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-03.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>Specifically, I&#8217;ve gone with <a href=\"https:\/\/huggingface.co\/NousResearch\/Hermes-3-Llama-3.1-8B\">Hermes 3<\/a> from Nous Research, which has been <a href=\"https:\/\/huggingface.co\/NousResearch\/Hermes-3-Llama-3.1-8B#prompt-format-for-function-calling\">trained to do tool calling<\/a>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-04.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>To let students experiment with this, I start them off with three simple tools. <\/p>\n<p>Two of them return information that a language model cannot natively contain:<\/p>\n<ul>\n<li>get the current date and time<\/li>\n<li>get the current weather at a location<\/li>\n<\/ul>\n<p>One of them computes information that language models are typically unreliable at generating:<\/p>\n<ul>\n<li>multiply two numbers<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-05.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>Students can try out the tools on the page to see what they output when they are run. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-06.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<h3>Testing out a model with and without tools<\/h3>\n<p>If I ask the same question as before (&#8220;What is the weather right now in New York?&#8221;) with tool calling enabled, I can see that the model selected one of the three tools, and used its own knowledge to generate the input values to give to the tool. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-07.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>The Enable \/ Disable toggle shows the difference that it makes to the model output when you enable tool calling. <\/p>\n<p>For example, if you ask the model for today&#8217;s date, without access to tools, you might get told that isn&#8217;t possible, or you might get a hallucination. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-08.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>The same prompt, with access to tools, will return the correct result. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-09.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<h3>Creating your own tools<\/h3>\n<p>That is enough to introduce the basic idea, but the more interesting bit is for students to create their own tools. For that, they take the model into Scratch.<\/p>\n<p>Students create tools by coding in Scratch. As long as they give them useful names and descriptions, the language model should be able to choose the appropriate tool it needs. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-10.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>For example, I made these tools to do sums:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-11.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>Obviously I didn&#8217;t need to do most of that to calculate the results, I could have just done the last bits. My tools do things like change costume, move the sprite to a different location, and turn it to face the middle of the Stage. And then it returns the calculation. <\/p>\n<p>What I&#8217;m showing here is that students can make tools as interesting or complex as they like. Anything they can do in Scratch can be included in a tool. <\/p>\n<p>If you ask the model to do a complex sum, <strong>without<\/strong> giving it access to tools:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-12.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>You get the wrong answer. It&#8217;s close, but not right. (<em>I&#8217;ve <a href=\"https:\/\/dalelane.co.uk\/blog\/?p=5669\">written about this sort of error before<\/a>.<\/em>)<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-13.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>But if you do the same thing and let the model choose from the tools that you have created in Scratch:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-14.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>You get the right answer. From a wizard. In a random location. Because that&#8217;s what the tool was coded to do. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-15.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>The model not only selected which tool to use, it also had to generate the parameters to submit to the tool (for example, from the word &#8220;treble&#8221; it chose the number &#8220;3&#8221; as one of the input parameters to submit).<\/p>\n<h3>What could tools in Scratch do?<\/h3>\n<p>Lots!<\/p>\n<p>They could check if one sprite is touching another sprite. Or how close it is to something. Or what way a sprite is facing. <\/p>\n<p>Tools don&#8217;t have to retrieve information, they can take actions. A tool could just be used to change the appearance of a sprite, or to move it to a location &#8211; as I did above. <\/p>\n<p>The point is, anything the language model can&#8217;t do by itself, if you can describe it in Scratch coding, now the model can do it by calling on a tool. <\/p>\n<h3>How is this used in real life?<\/h3>\n<p>Tool calling has had a huge impact on how we use LLMs. It is what turned ChatGPT from a chatbot, limited to what it could access in the chat window, to an assistant that can search the web, access information, and interact with other systems. <\/p>\n<p>For an intro to this, I&#8217;d recommend the <a href=\"https:\/\/www.ibm.com\/think\/podcasts\/techsplainers\">Techsplainers podcast<\/a> episode on tool calling. You can find it on podcast players, such as <a href=\"https:\/\/overcast.fm\/+BTXinUjxws\">Overcast<\/a>, <a href=\"https:\/\/open.spotify.com\/episode\/5s1KSQO9Eb1tveIKx7DWzG?si=kXd4uY9ISaKxIybL3I3ygg\">Spotify<\/a>, or <a href=\"https:\/\/podcasts.apple.com\/us\/podcast\/what-is-tool-calling\/id1850811611?i=1000747166143\">Apple Podcasts<\/a>.<\/p>\n<p>Alternatively, if you&#8217;d prefer to read something, there is a good intro at <a href=\"https:\/\/www.ibm.com\/think\/topics\/tool-calling\">ibm.com\/think\/topics\/tool-calling<\/a><\/p>\n<h3>How great is this feature?<\/h3>\n<p>I am very pleased with this new feature, and I do think it opens up some exciting project opportunities. But&#8230; take another look at that graph I showed earlier. <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/images.dalelane.co.uk\/2026-07-09-tool-calling\/screenshot-04.png\" style=\"border: thin black solid; width: 100%; max-width: 450px;\"\/><\/p>\n<p>This model is huge. It&#8217;s a 4.2gb download. It&#8217;s <a href=\"https:\/\/huggingface.co\/NousResearch\/Hermes-2-Pro-Llama-3-8B\">an 8 billion parameter model<\/a>.<\/p>\n<p>It&#8217;s reasonably fast on my five-year old M1 Macbook Pro, but on the sort of older Windows computers I see in schools, I think this is going to be very slow to return results. Maybe impractically slow.<\/p>\n<p>I chose tiny models like Smol and Qwen when I started for a reason. They seem to be roughly at the level that a lot of school computers can support. <\/p>\n<p>So&#8230; I suspect this feature will have a limited audience. That is a shame, but &#8220;limited&#8221; is still better than nothing, so I&#8217;m giving it a try. <\/p>\n<h3>Thanks to<\/h3>\n<p>Finally, I should say a big thanks to <a href=\"https:\/\/www.linkedin.com\/in\/zayan-siddiqui-raja-785b94385\/\">Zayan Siddiqui-Raja<\/a> for helping put this together. The idea for adding this feature came from a discussion with him, and he contributed the first iteration of the support in Scratch for building and running tools.  <\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post, I want to share a new feature in the generative AI support in Machine Learning for Kids: tool calling. I wrote last year about how I introduced Generative AI in Machine Learning for Kids by adding support for projects using small language models. And earlier this year, I walked through my six [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[587,536],"class_list":["post-6116","post","type-post","status-publish","format-standard","hentry","category-code","tag-mlforkids-tech","tag-scratch"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/6116","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6116"}],"version-history":[{"count":3,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/6116\/revisions"}],"predecessor-version":[{"id":6120,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/6116\/revisions\/6120"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6116"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6116"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}