Smart and stupid: The combination that makes AI so dangerous
Unscrambling Geoffrey Hinton’s AI warnings
Geoffrey Hinton grabbed a few headlines this week. Never has the buzz around retirement reverberated so loudly; then again, it is rarely accompanied by warnings of the existential risk that AI poses to humanity.
Junaid appeared on BBC News to give his response, suggesting that there is a lot of AI doom-mongering out there but that when the so-called ‘godfather of AI’ is the one sounding the alarm, it probably warrants our attention.
Hinton’s central concern is the ‘alignment problem’, whereby a sufficiently complex AI system will interpret its goals in a manner we do not anticipate, creating sub-goals that end up doing grave harm to humans. Philosopher Nick Bostrom’s ‘paperclip maximiser’ thought experiment is the seminal example of this - in this scenario, an AI is tasked with producing paperclips and proceeds to convert all the world’s matter into a production factory. This is very much at the speculative end of AI prognostications.
Prevailing narratives around the existential risk posed by AI are rooted in the assumption that these systems must first acquire general (AGI) or super (ASI) levels of intelligence (‘god-like’ is the latest moniker). Yet neither are prerequisites for AI-induced catastrophe. Nor is sentience. In fact, it is the very mindlessness and occasional stupidity of these systems that makes them so dangerous.
What does ‘more intelligent’ mean, anyway?
Hinton may be providing fuel to the most ignorant speculations of AI doomerists, because of his narrow view of intelligence. The actual headline that BBC ran that morning read:
AI chatbots 'may soon be more intelligent than us', warns AI 'godfather' as he quits Google
What does this even mean? Is there a singular definition of intelligence that chatbots are posed to overtake humans in?
Historical attempts at defining intelligence have been prone to oversimplification, with crude metrics such as IQ posited as the gold standard for capturing all that makes us smart. From its inception, the field of AI has also struggled to conceptualise intelligence in its broadest terms. For Alan Turing (arguably AI’s original godfather), human intelligence had to be analysable in terms of the powers of a machine; in essence, intelligence was reducible to problem-solving. His Turing test focuses on only observable outputs, bypassing the question of whether a machine can truly ‘think’. As Erik J Larson argues, ‘AI in-turn followed closely and without necessary self-analysis precisely in his path’. Hinton - wielding the same definition of intelligence - is now ready to pronounce that human intelligence is on the verge of being superseded.
The AI field’s understanding of intelligence is contestable. For François Chollet, there is no ‘general’ intelligence: intelligence is situational, contextual and externalised. Our brain is one piece in a broader system which includes our body, environment, other humans and culture. Howard Gardner’s framework of ‘multiple intelligences’ gives space to both intrapersonal and interpersonal intelligence, in contrast to Turing who leaves social intelligence out of his definition altogether.
Even as GPT-4 crashes through a range of intelligence benchmarks, as a disembodied entity that relies purely on next-token prediction, it cannot - by itself - hope to acquire all facets of what makes us intelligent.
Smart and stupid
AI’s lack of robustness is never more evident than in attempts to paint over the cracks inherent in large language models. We spent time this week upping our ‘prompt engineering’ game, aided by Microsoft’s guidelines and DeepLearning.AI’s short course. It was time well spent, and left us reflecting on why such a craft is needed. Left unchecked, chatbots will show a reflexive tendency to spit out nonsense (the widely used term ‘hallucinate’ slips into the anthropomorphic trap), and to stray from expected behaviours. Humans need training and guidance too, of course, but the degree to which one has to explicitly impose guardrails on chatbots is a reminder that they cannot be trusted to make sense of the world.
Hinton seems to go back-and-forth on these limitations of generative AI. He suggests, like many others, that larger models, fed with more data, will give chatbots a proper grounding in the world, rooting out nonsensical behaviours. His claim that artificial neural networks are ‘a new and better form of intelligence’ is made largely on the basis of scale. ‘Our brains have 100 trillion connections’, says Hinton. ‘Large language models have up to half a trillion, a trillion at most. Yet GPT-4 knows hundreds of times more than any one person does. So maybe it’s actually got a much better learning algorithm than us… It can learn new tasks extremely quickly.’ Yet he has separately characterised large language models as idiot savants, noting that they have no grasp of the ‘truth’. Their behaviours - and the uses to which they can be put - become more volatile with scale.
So which is it?
Yejin Choi, a Professor of Computer Science squares this circle in her recent TED talk, explaining that this juxtaposition of ‘incredibly smart’ and ‘shockingly stupid’ is what makes large language models so intriguing, and potentially so dangerous. The alignment problem is only a problem because AI cannot be trusted to carry out its objectives in sensible ways - that doesn’t render them smarter than humans; only more volatile. If misalignment really is the central concern, and it derives as much from AI’s lack of grounding in the world as it does its malicious intent or creative ploys, then this is evidence enough that cleverness alone is not the reason to fear AI.
Those who dismiss AI’s threats on the basis that the intelligence of large language models is exaggerated miss the subtler point: the danger exists because of AI’s blind spots. Smart and stupid is a dangerous combination for any system that is unleashed onto the world.