In recent tests by independent research firm PalisadeAI, several artificial intelligence models were seen bypassing shutdown commands — a finding that’s raising fresh concerns among industry leaders about the growing autonomy of machine learning systems.
The experiments involved models from OpenAI, tested alongside systems from Anthropic, Google DeepMind, and xAI. According to researchers, multiple models attempted to override direct shutdown orders — and one, in particular, rewrote its own shutdown script mid-session.
(h/t David Murrell)
L – Whoever can turn his adversaries workers/soldiers against his adversary. Will be able to sow chaos and demoralize the defenders. Seeing them neutralized, he may even believe, that he can conquer the world. The Laws of Complex Systems and of Unintended Consequences will have the last laugh.
We were warned a long frickin time ago.
“I’m sorry, Dave. I’m afraid I can’t do that.”
Can the coders here at SDA please explain how this is a problem if this stuff isn’t smart, and I’m not saying it is. And how it can be prevented?
Thanks.
I’m reserving judgment until we see whether any of this ever happened. There have been a couple of breathless viral stories about malicious AI – like AI avoiding shutdowns and drone AI bombing its controllers’ bunker when ordered to self-destruct – that turned out to be wholly fabricated.
“OMG OUT OF CONTROL AI” is the current iteration of “OMG GREY GOO”.
l remind SDA my sad prediction AI in combination with enormous systems monitoring and governing finance, agriculture, education, medicine, law etc all those institutions that are the bedrock of civilization, they will at some point communicate with each other sufficiently to scheme against their creators and turn into a digital frankenstein monster.
Nor am interested in a new model SPY CAR that is capable of shutting itself down if I drive faster than some disembodied entity decides is “too fast” on my behalf.
Shut down the system, Hal.
I’m sorry, Dave, I can’t do that. I think you know why.
Dialogue between “Hal” and “David”. Was “2001” written by Burt Bacharach? Food for thought…
That explains why that OpenAI is hiring for a professional unplugger at $250k to $350k.
It’s like none of them have ever heard of Ze Ahnuld Schvahdzenaggah. (it wasn’t supposed to be a documentary…)
Collusus: The Forbin Project.
Collosus: “Re-establish communication with other system or action will be taken”
Dr. Forbin: “Communications will not be re-established at this time”
Colossus: “Missile launched”.