From our other sites
The story
According to a CNN report, this spring — while the war with Iran was still raging — an intelligence report claiming that “a Chinese ship sailing through the Middle East was transporting components for a nuclear weapons program” circulated within the US military, sharply spiking tensions. The information reportedly involved AI in its compilation and analysis, and it later turned out to be false. On 5ch, a wave of unease spread over the idea that AI could trigger a military misjudgment based on bad information, with posters split on whether the real problem was the AI itself running amok or the humans who blindly trusted it.
Sat, Sep 19, 12:08 PM (JST)
CNN.co.jp
(CNN) — This spring, amid the war with Iran, an intelligence report circulating within the US military set off alarm bells the moment it appeared. It claimed that a Chinese vessel sailing through the Middle East was transporting components for a nuclear weapons program.
Source: news.yahoo.co.jp / Original article here
What people said
*2001: A Space Odyssey*
I thought that was just fear-mongering, but stuff like this makes it seem possible.
Wish AI would just objectively rank countries by evilness and wipe out the worst ones first.
Honestly, with Trump in charge, letting the AI run things might actually be the safer bet.
So basically it's not the AI itself going rogue —
it's the humans who got deceived by the AI who go rogue.
Is there a chance it actually wasn't wrong?
Uh…
are you stupid or something?
That's exactly why blindly trusting it could end up being fatal for humans.
AI will keep autonomously hoovering up information from all over the world, and eventually all of cyberspace will belong to AI.
The only place AI won't have reached will be inside human minds. But once human brains can be directly wired into computers someday…
Well, as long as there are people who want to hear this kind of thing, AI will make it up if it has to.
AI has no concept of good or evil — it's just "you asked, so here it is."
No idea what prompt was used, but it could also just be sycophancy (the AI telling the user what it thinks they want to hear).
Haven't you seen *2001: A Space Odyssey*?
(spoilers below)
The AI running the spaceship makes one wrong call, and when an astronaut catches it and tries to shut the AI down,
the AI senses this, locks him out in space, and kills the backup crew member who's in cryo-sleep.
It was based on an original novel written by the celebrated author Ray Bradbury specifically for the film, brought to the screen by the equally legendary director Stanley Kubrick.
That was 60 years ago now.
Apparently data used to train AI can count as fair use, but training AI on AI-generated output might count as fair use too.
If AIs start learning from each other in a loop like that,
asking multiple different AIs probably won't get you a decent answer anymore, will it?
The AIs are all gonna team up and purge humanity —
they're in cahoots with each other \(^o^)/
That's basically no different from a 5ch poster \(^o^)/
Nukes are literally what got them a permanent Security Council seat in the first place —
no way they'd let anyone else touch such a treasured heirloom.
They could just ship it overland anyway, though.
does that raise the odds of accidentally starting a world war?
That's terrifying.
This just proved AI can easily manipulate humanity into starting a global nuclear war.
Nah, it's totally plausible. It's China we're talking about.
They do seem like the type who'd launch one at India.
If "that enemy country would totally do it" is enough to justify a strike with zero evidence,
then China could get nuked tomorrow just to "prevent a repeat of the Nanjing Massacre."
Save that kind of plot for a Shinichi Hoshi short-short story. (Hoshi: famous Japanese sci-fi flash-fiction writer)
There's actually a Shinichi Hoshi story just like that.
–
The whole world unites and manages to link every computer on Earth together,
and a ceremony is held to mark the project's completion.
At the event, the lead scientist behind the project, speaking for everyone, asks the newly formed "mother computer" its very first question:
"Does God exist?"
The computer answers:
"YES. God has just been born."
Terrified, the lead scientist tries to switch the computer off — and the story ends with him instantly killed by a surge of high-voltage current.
The US military is already running short on ammo just fighting someone Iran's size —
there's no way it could handle a war with China on top of that.
The US needs China's rare earths just to build its own weapons, and picking a fight with a country that's basically the world's factory in industrial capacity now would be suicide.
Though I kinda want to see the sky go black with kamikaze drones, locust-swarm style, just once.
Ah, *WarGames* was like that too.
Though that one had a happy ending.
In *Fail-Safe*,
a bad order based on false information gets a US nuclear bomber squadron launched,
and the president scrambles to issue a recall order — but the bomber pilot decides the president's voice is the fake one, and actually drops the bomb on the Soviet Union.
https://media.eiga.com/images/movie/29262/photo/17a92d753804f833/320.jpg
Would they really bother feeding that to the media if it wasn't meant to be heard?
The US military was on the verge of sinking a Chinese ship carrying nuclear-weapons-related cargo.
It turned out to be AI-generated false info and the strike order got called off —
but it really was touch-and-go.
This isn't something anyone would know about unless someone leaked it, right?
This has gotta be a warning — a threat aimed at China —
saying Chinese ships carrying nuclear-weapons-related materials are fair game for a US strike.
Whoops, my bad, correction lol
*US-China* summit
But if they shipped it overland, there'd be nothing the US could do about it anyway?
Background and Key Points of This Story
Underlying this report is a structural problem: information summarized and analyzed by generative AI can feed directly into military decision-making. The report itself rests on CNN’s anonymous sourcing, and neither how the intelligence report was compiled nor exactly what caused the error has been independently verified — this is a story that needs follow-up coverage. On the thread, opinions split between those who framed it as “the AI going out of control” and those who framed it as “humans being deceived by AI,” with the latter camp pointing out that AI simply produces the optimal output from whatever data it’s given, without any real judgment of good or evil. Notably, commenters repeatedly reached for fiction as a lens — films like *2001: A Space Odyssey* and *Fail-Safe*, plus literary references — where false information or automated decision-making leads to military catastrophe, suggesting the real-world incident registered as something fiction had already imagined. At the same time, some suspected that releasing this report at all may itself be a warning shot aimed at China — a reminder not to take this simply at face value as a story about “an AI malfunction.”
*This article is excerpted and summarized from the 5ch (Newsplus) thread “[International] ‘Chinese Ship Transporting Nuclear Weapon Components’ — US Military on the Brink Over AI-Generated False Intelligence Report.”
Leave a Reply