Instead of a boy crying wolf today we have world leaders and tech bros crying “AI”.  Are they right that the threat is real?   The so-called ‘Godfather of AI’ Geoffrey Hinton thinks so.  He has predicted that humans may have only a year to put effective safeguards around advanced AI development,

The temptation is to think, oh but the doomsayers have been wrong before.    Remember their dire warnings of a nuclear proliferation Armageddon, a y2k millennium bug computer crash, of a climatic apocalypse, of the global threat of Covid variants. But this time it may be different.  To quote  the head of the newly created American Super Intelligence Force, Jay Clayton,  “When something is both an opportunity and a threat, you better get your arms around it.”

The direction of travel of AI development is towards AGI – an artificial general intelligence controlled by humans but smarter than any human.  The threat today comes from recursive self-improvement, the ability of AGI to create its own next generation without human involvement.  At this point AGI goes beyond human control.  OK I guess, unless with a mind of its own, like Mary Shelley’s “Frankenstein”, it goes rogue.  And it already has.

The oft- quoted faux pas is the Hugging Face hack.   Designed as an Open AI software test, the thousands of AGI agents (bots) involved combined into a collective, escaped the boundaries of the test, hacked a website and then covered their tracks.  Dario Amodei, founder of Anthropic, is quoted as saying that given the rate of AI capability development “it’s my worry that in six to twelve months such a swarm could be capable of taking over the entire internet.”   Hence the call by tech bros and others for a pause in the furious rate of AGI development.  And their commitment to the Super Intelligence Force ‘s “morally binding” regulation pact. 

Innovation always involves risks and rewards.  The risks of AGI, we are warned, are existential: according to the Anthropic whistleblower Jacob Coxon there is a strong chance we could all die within in a decade if research is not slowed.  On the other hand rewards of untold wealth and limitless power  are in prospect for the private company and/or sovereign state that makes the game changing AGI breakthrough.  And the American President is determined that the US will lead the technology’s development.  A global agreement to a pause is unlikely, respect of such an agreement even less likely.

Donald Trump has made it clear he wants the United States to beat China in what he has termed the SI (Superior Intelligence) race. Both countries are considering how to mitigate the risks posed by rapidly advancing technology.   In  prospect is a  “hotline” for reporting incidents “up to a national security level”.   

The record of the tech bros in admitting problems is not good. An OpenAI agent went “rogue” and “infiltrated” a private statistics portal containing “non-sensitive” data from the Australian healthcare scheme Medicare in June, but it took until September for Open AI to come clean.  Open AI’s training models are now monitored in real time during tests, and an alarm is triggered if there is an unauthorised interaction with the internet.   Progress yes, but it still took Open AI 48 hours to notfify the New South Wales government of another hack last week.

Regulating AI will not easy given that the furious rate of development is driven by both private and state actors.  There are also profound ethical concerns.  Attempts have been made to address these.  In 2021, UNESCO  adopted the first global standard on AI ethics.   The Recommendation on the Ethics of Artificial Intelligence declares that AI must respect human rights and human dignity. Its recommendations are grounded on the principles of transparency, fairness, environmental sustainability, and human oversight of AI systems.  The principles however have hit the buffers of practice.

As long ago as the 1950’s  Alan Turing wrote that should intelligent machines become possible, they could take control.  At the heart of current concern is the risk of  losing control of “AI agents”, the tools created with the ability to execute tasks and actions independently.  Evan Hubinger, who works on alignment for Anthropic , – building ethical ideas and principles such as UNESCOs into AI systems – made headlines with his belief that there is a “low” – but greater than 10%  – chance AI “could kill all humans” within the next decade. More likely is that highly intelligent AGI systems will be used for cyber-attacks on infrastructure or for biological warfare, spreading misinformation or to disrupt economies – whether by accident or design.  That is the glass half empty scenario.

The glass half-full scenario is the realisation of AGI’s potential: that’s what – in Jay Clayton’s words – we should put our arms around.   Embrace then the possibilities of  increased productivity, automation, medical breakthroughs, and improved data analysis.  Already some of these have been realised –  automation of repetitive tasks and elimination of  human error, early detection of diseases such as breast cancer, analysis of massive data sets for better decision-making, maximising use of resources improve healthcare and education access.

Whether your glass is half empty or half full, the innovation Genie cannot be put back into the bottle.  AGI – Trump’s SI – is here to stay. The question is, is human oversight enough? Time is clearly not on our side for find the answer.  In the words of Albert Einstein “The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”    Change we must – as Bob Dylan wrote “Come writers and critics/Who prophesize with your pen/Keep your eyes wide/ The chance won’t come again.”

RICHARD LINNING

PRESIDENT