‘They learn how to kill’

Google‘s former CEO Eric Schmidt has issued a stark reminder about the dangers of AI and how susceptible it is to being hacked.

Schmidt, who served as Google’s chief executive from 2001 to 2011, warned about “the bad stuff that AI can do,” when asked whether AI is more destructive than nuclear weapons during a fireside chat at the Sifted Summit

“Is there a possibility of a proliferation problem in AI? Absolutely,” Schmidt said Wednesday. The proliferation risks of AI include the technology falling into the hands of bad actors and being repurposed and misused.

“There’s evidence that you can take models, closed or open, and you can hack them to remove their guardrails. So in the course of their training, they learn a lot of things. A bad example would be they learn how to kill someone,” Schmidt said.

“All of the major companies make it impossible for those models to answer that question. Good decision. Everyone does this. They do it well, and they do it for the right reasons. There’s evidence that they can be reverse-engineered, and there are many other examples of that nature.”

AI systems are vulnerable to attack, with some methods including prompt injections and jailbreaking. In a prompt injection attack, hackers hide malicious instructions in user inputs or external data, like web pages or documents, to trick the AI into doing things it’s not meant to do — such as sharing private data or running harmful commands

Jailbreaking, on the other hand, involves manipulating the AI’s responses so it ignores its safety rules and produces restricted or dangerous content.

In 2023, a few months after OpenAI’s ChatGPT was released, users employed a “jailbreak” trick to circumvent the safety instructions embedded in the chatbot.

This included creating a ChatGPT alter-ego called DAN, an acronym for “Do Anything Now,” which involved threatening the chatbot with death if it didn’t comply. The alter-ego could provide answers on how to commit illegal activities or list the positive qualities of Adolf Hitler.

Schmidt said that there isn’t a good “non-proliferation regime” yet to help curb the dangers of AI.

Despite the grim warning, Schmidt was optimistic about AI more broadly and said the technology doesn’t get the hype it deserves.

“I wrote two books with Henry Kissinger about this before he died, and we came to the view that the arrival of an alien intelligence that is not quite us and more or less under our control is a very big deal for humanity, because humans are used to being at the top of the chain. I think so far, that thesis is proving out that the level of ability of these systems is going to far exceed what humans can do over time,” Schmidt said.

“Now the GPT series, which culminated in a ChatGPT moment for all of us, where they had 100 million users in two months, which is extraordinary, gives you a sense of the power of this technology. So I think it’s underhyped, not overhyped, and I look forward to being proven correct in five or 10 years,” he added.

His comments come amid growing talk of an AI bubble, as investors pour money into AI-focused firms and valuations look stretched, with comparisons being made to the dot-com bubble collapse of the early 2000s.

Schmidt said he doesn’t think history will repeat itself, however.

“I don’t think that’s going to happen here, but I’m not a professional investor,” he said.

“What I do know is that the people who are investing hard-earned dollars believe the economic return over a long period of time is enormous. Why else would they take the risk?”