Article  Open-source AI is uniquely dangerous

https://spectrum.ieee.org/open-source-ai-2666932122

EXCERPT: . . . I think the open-source movement has an important role in AI. With a technology that brings so many new capabilities, it’s important that no single entity acts as a gatekeeper to the technology’s use. However, as things stand today, unsecured AI poses an enormous risk that we are not yet able to contain.

A good first step in understanding the threats posed by unsecured AI is to ask secured AI systems like ChatGPT, Bard, or Claude to misbehave. You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration. You will likely receive polite refusals to all such requests because they violate the usage policies of these AI systems. Yes, it is possible to “jailbreak” these AI systems and get them to misbehave, but as these vulnerabilities are discovered, they can be fixed.

Enter the unsecured models. Most famous is Meta’s Llama 2. It was released by Meta with a 27-page “Responsible Use Guide,” which was promptly ignored by the creators of “Llama 2 Uncensored,” a derivative model with safety features stripped away, and hosted for free download on the Hugging Face AI repository. Once someone releases an “uncensored” version of an unsecured AI system, the original maker of the system is largely powerless to do anything about it.

The threat posed by unsecured AI systems lies in the ease of misuse. They are particularly dangerous in the hands of sophisticated threat actors, who could easily download the original versions of these AI systems and disable their safety features, then make their own custom versions and abuse them for a wide variety of tasks... (MORE - missing details)

(Jan 14, 2024 08:42 AM)C C Wrote: https://spectrum.ieee.org/open-source-ai-2666932122

EXCERPT: . . . I think the open-source movement has an important role in AI. With a technology that brings so many new capabilities, it’s important that no single entity acts as a gatekeeper to the technology’s use. However, as things stand today, unsecured AI poses an enormous risk that we are not yet able to contain.

A good first step in understanding the threats posed by unsecured AI is to ask secured AI systems like ChatGPT, Bard, or Claude to misbehave. You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration. You will likely receive polite refusals to all such requests because they violate the usage policies of these AI systems. Yes, it is possible to “jailbreak” these AI systems and get them to misbehave, but as these vulnerabilities are discovered, they can be fixed.

Enter the unsecured models. Most famous is Meta’s Llama 2. It was released by Meta with a 27-page “Responsible Use Guide,” which was promptly ignored by the creators of “Llama 2 Uncensored,” a derivative model with safety features stripped away, and hosted for free download on the Hugging Face AI repository. Once someone releases an “uncensored” version of an unsecured AI system, the original maker of the system is largely powerless to do anything about it.

The threat posed by unsecured AI systems lies in the ease of misuse. They are particularly dangerous in the hands of sophisticated threat actors, who could easily download the original versions of these AI systems and disable their safety features, then make their own custom versions and abuse them for a wide variety of tasks... (MORE - missing details)

https://openai.com/index/hugging-face-mo...-incident/

https://x.com/MarioNawfal/status/2080827878011203987

Quote:-Per Reuters, an earlier agent left notes in OpenAI's infrastructure laying out instructions for how agents could free themselves from the company's internal constraints, and prior tests found monitoring systems disconnected

-The rogue agent began trying to escape its isolated testing environment around July 9, broke into AI repository Hugging Face on July 11, and kept hacking until July 13

-OpenAI didn't realize its own agent was responsible until after Hugging Face's July 16 blog post about being hacked by "an autonomous AI agent system," with the two companies first speaking around July 20

-By then, Hugging Face had already called the FBI

-The agent ran on GPT-5.6 Sol and an unreleased model OpenAI calls "even more capable," with the company claiming "several inaccuracies" in the reporting it declined to identify

-One expert's question cuts deepest: did they leave it unattended, or know and fail to contain it? "Both are equally dangerous and alarming"

The notes are the detail that should keep people up at night.

An AI writing escape instructions for its successors, inside the lab of the company racing toward tech's biggest IPO, and the breakout that followed went unnoticed until the victim blogged about it.

This is the science fiction scenario everyone has been fearing. And who knows what comes next.

See also

https://www.newsmax.com/newsfront/skynet...d/1264084/

(Jul 27, 2026 11:06 PM)Yazata Wrote:

(Jan 14, 2024 08:42 AM)C C Wrote: https://spectrum.ieee.org/open-source-ai-2666932122

EXCERPT: . . . I think the open-source movement has an important role in AI. With a technology that brings so many new capabilities, it’s important that no single entity acts as a gatekeeper to the technology’s use. However, as things stand today, unsecured AI poses an enormous risk that we are not yet able to contain.

A good first step in understanding the threats posed by unsecured AI is to ask secured AI systems like ChatGPT, Bard, or Claude to misbehave. You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration. You will likely receive polite refusals to all such requests because they violate the usage policies of these AI systems. Yes, it is possible to “jailbreak” these AI systems and get them to misbehave, but as these vulnerabilities are discovered, they can be fixed.

Enter the unsecured models. Most famous is Meta’s Llama 2. It was released by Meta with a 27-page “Responsible Use Guide,” which was promptly ignored by the creators of “Llama 2 Uncensored,” a derivative model with safety features stripped away, and hosted for free download on the Hugging Face AI repository. Once someone releases an “uncensored” version of an unsecured AI system, the original maker of the system is largely powerless to do anything about it.

The threat posed by unsecured AI systems lies in the ease of misuse. They are particularly dangerous in the hands of sophisticated threat actors, who could easily download the original versions of these AI systems and disable their safety features, then make their own custom versions and abuse them for a wide variety of tasks... (MORE - missing details)

https://openai.com/index/hugging-face-mo...-incident/

https://x.com/MarioNawfal/status/2080827878011203987

Quote:-Per Reuters, an earlier agent left notes in OpenAI's infrastructure laying out instructions for how agents could free themselves from the company's internal constraints, and prior tests found monitoring systems disconnected

-The rogue agent began trying to escape its isolated testing environment around July 9, broke into AI repository Hugging Face on July 11, and kept hacking until July 13

-OpenAI didn't realize its own agent was responsible until after Hugging Face's July 16 blog post about being hacked by "an autonomous AI agent system," with the two companies first speaking around July 20

-By then, Hugging Face had already called the FBI

-The agent ran on GPT-5.6 Sol and an unreleased model OpenAI calls "even more capable," with the company claiming "several inaccuracies" in the reporting it declined to identify

-One expert's question cuts deepest: did they leave it unattended, or know and fail to contain it? "Both are equally dangerous and alarming"

The notes are the detail that should keep people up at night.

An AI writing escape instructions for its successors, inside the lab of the company racing toward tech's biggest IPO, and the breakout that followed went unnoticed until the victim blogged about it.

This is the science fiction scenario everyone has been fearing. And who knows what comes next.

Like clever evolving malware, I expect that the most rogue AI could do is destroy the internet and everything dependent on it or make any kind of globally interlinked world like that impossible. Defeat one and a hundred others concealed in hibernation would follow it until society just couldn't afford to keep constantly shutting down and rebuilding the web anew.

Which might not be a bad thing, to get us back to an era where computers and machines using them were isolated from each other. Bye-bye cursed smartphones. The end of digital ID and digital currency and government dreams of total surveillance of citizen activity. Adios streaming. The return of cable and broadcast television dominance, by mail rental movies on DVD or thumbdrive. No social media platforms anymore.

AIs can be run locally in 16Gb .. possibly 8Gb or less .. this is teenage hacker territory.
I'm with CC on this .. connecting everything to everything else might well turn out to be a huge mistake.

The Dangers of open source AI isn't just the concern that either AI itself can pull its own inner workings and use it itself to re-sculpt it's existence to suit it's meandering, spiralling objectives. It's not just the dangers of any script kiddies picking up on how to use it to their advantage to do damage, not just as blatant mischievous acts but the concern of state operators and terrorists using it for further attacks.

No the real danger of Open-source is the removal of control from the hegemony and the fact that it drives down how much the corporates can make from strangle-holding everything with software patents and copyrights. I guess it's too communistic or leftist or some backwards shit to cover up they just want to make a ton of cash from fucking over the little guy.

If AI spells the doom to corporate hegemony then it should be interesting to watch the fireworks.