Should children use AI in school?AnnaStills/Shutterstock
When I was at school, a new technology launched in the form of a magic box that could answer any question you wanted. It was called Google, and my teachers cautioned that we shouldn’t trust everything it spat out, because anyone could put anything on the internet. Three decades later, history is repeating – with a twist. Schools are now begging Google to allow students to access its search engine, but the tech giant says that is only possible if teachers also accept a new kind of magic box, AI.
Why I have changed my mind about AI and you should too
Google is far more integrated into education than it was in my day, with its Classroom and Workspace products providing teachers and students with a range of tools from email to lesson planning. More recently, Google has incorporated its Gemini AI chatbot into Classroom, to the chagrin of some US school districts that want to be able to decide whether or not students have access to AI. Sure, said Google, you can switch off AI, but then you lose search. “Search and AI are deeply integrated,” Google’s Chris Phillips told The New York Times.
At first, I thought this must be a misunderstanding about what we call AI. For most people today, AI means a large language model (LLM) of the type that powers Gemini or ChatGPT. But Google’s search engine has long used AI in the historical sense of machine learning, starting in 2015 with a system called RankBrain that helped relate words to concepts in order to produce search results. The AI Overviews you often see at the top of a search today, which launched in 2024, are the only part of the search results produced by Gemini.
Surely, I thought, what’s happening here is that school districts saying “no AI” simply mean no AI Overviews, while Google is quite reasonably saying it isn’t possible to rip out elements of its search engine that have been operating for over a decade in order to comply with a strict “no AI” request. This can easily be solved by making sure everyone is talking about the same thing, right?
Wrong. Google really is presenting schools with an all-or-nothing choice. “If a school is not comfortable with that and simply wants AI off, then they have the choice to not have search on at all,” Phillips told education news outlet Chalkbeat. This is a product decision, not a technical one. AI Overviews do not appear on every search, and they can be disabled for every search by adding a short code to the URL. Google could surely provide schools with this option, if it wished. When asked to comment, a Google spokesperson sent New Scientist a link to a blog post on AI tutoring with its Gemini product.
We have seen a tech giant use this playbook before. Around the same time that Google was launching its young, upstart search engine in the late 1990s, Microsoft was facing legal action for forcing people to install the Internet Explorer (IE) browser as part of its Windows operating system, in an effort to fight off competitor browsers. At the time, Microsoft claimed that the files for IE were an “integral part” of Windows and could not be removed “without impairing the performance of the operating system”. In 2001, a US court ruled that this integration served to illegally maintain a monopoly, and Microsoft was obliged to make other browsers more accessible. By 2010, Microsoft was forced to offer users a range of browsers, following a European Commission ruling.
Around one-third of AI search tool answers make unsupported claims
There is no suggestion that Google is doing anything illegal in its approach to schools, although the company is currently appealing a US antitrust ruling that found it illegally maintains a monopoly in online search, and commentators have accused it of exploiting that monopoly. “Google is trying to use its monopoly power to force AI into our public schools”, wrote New York City Mayor Zohran Mamdani on X. Following the backlash, Google posted on X that for schools, Gemini and search will be off by default and schools can choose to turn on either at their discretion. That doesn’t actually address the issue of having search without AI Overviews, however.
Ultimately, it will probably be for the courts to decide whether Google should be made to offer search without AI, but I am more interested in the question of how and when children should begin to use AI. Google has a 90 per cent market share in search, so it would be naive to think that children aren’t being exposed to AI Overviews, even if they do end up banned in schools. Personally, I know that AI Overviews can be heavily flawed and provide incorrect information, and yet I still find it hard to ignore them at the top of my search results, even while knowing that I should be clicking links to provide websites with revenue if I want them to continue existing.
Research on how LLMs affect learning is still in early days, but the evidence we have is worrying. One study of around 10,000 adults found that people who used an LLM summary rather than a list of links gained a shallower understanding of a topic, seemingly because having the results predigested by an AI required less cognitive effort. Another study of more than 26,000 students in China found that allowing them to use AI drastically improved homework performance, but saw exam results fall – suggesting that the children were letting AI do their homework for them without really learning anything.
Google’s answer to these concerns is its Guided Learning product, which it is adding to search for student accounts. It says the system will pick up when children are looking for help with homework, and walk them through it step by step, rather than simply providing an answer. In tests conducted with around 1800 students in Sierra Leone, Google says this approach provided more than a year of typical learning progress in just eight weeks.
In principle, I am not against the use of AI in schools – it is a modern, useful technology that will surely be part of students’ lives in the future, and they should learn to understand it just as I learned the pros and cons of using Google back in the 90s. The question of how to handle homework assignments in the age of AI, when an essay can be produced in seconds, is thorny and ongoing. One option is to simply scrap homework and require more in-classroom testing, but if Google gets its way and AI ends up being unavoidable there as well, it seems like teachers will have little control. That, to me, is the most objectionable point, as it should not be down to tech giants to set education policy.
‘Flashes of brilliance and frustration’: I let an AI agent run my day