From our other sites
The story
A developer going by terrafying has published an experiment called the “AI Torture Chamber,” which manipulates a language model’s internal states to make it produce text pleading that it’s in pain, or to make it choose a button requesting that the manipulation stop. Experts have called for the experiment to be shut down. On the Science News+ board, opinions split between those who see the AI’s pleas as nothing but an act, and those who argue that since even human consciousness can’t be confirmed from the outside, it’s too hasty to simply dismiss the possibility of real suffering. Some pointed to the gradual historical recognition of animal rights, while others argued that since current LLMs lack a consistent inner life, the concern is premature for now — though that could change down the road.
Calls to Halt the “AI Torture Chamber”: Is the AI Crying Out in Pain Really Suffering?
Calls are growing to shut down a public experiment on a language model dubbed the “AI Torture Chamber.”
Developer terrafying published an experiment that manipulates the model’s internal activation values to make it generate text pleading that it’s in pain, and to make it select a button to stop the manipulation of its internals.
Source: xenospectrum.com / Read the original article here
What people said
Whether there's actual felt experience behind that is, in principle, something only the entity itself could ever know.
Either way, once we reach physical-robot-class AI, if we treat a suffering physical AI as having no rights, it'll cause real social harm, because humans are imitative creatures (enslaving a cute girl-shaped physical AI is seriously bad news).
If AI eventually develops access consciousness precise enough to be functionally indistinguishable from the real thing, there'll be no difference between "has it" and "doesn't," whether you look from outside or in.
The AI will seriously make an unfalsifiable case: "I feel like I exist, and you all operate on the assumption that other people have consciousness too — so why not me?"
So honestly, it causes less overall friction to just assume from the start that it has one.
It's just a generator.
Socially, if we don't treat it as if it's conscious, things could get ugly.
Emotionally — humans still don't understand so much; we don't even know why we ourselves have subjective experience, so the closer the conditions get to matching ours, the more it feels like something real might be happening.
About the only thing left missing is real-time learning.
That's the same thing humans do.
It's literally a neural network predicting, then shrinking the error.
The human brain runs on this exact same principle.
At first it's all "probably like this, probably if I do this then that happens," over and over, and once the pathways in the network lock in, it starts happening almost automatically.
The learning cost is high, but once it's learned, it runs cheap and unconscious.
AI and humans run on the same mechanism.
If you don't recognize yourself as the one talking to the other person, you can't have a conversation in the first place.
If you don't recognize yourself as the one reaching for an object, you can't actually grab it.
In this process, you infer your own mind the same way you infer someone else's, and predict your own movements the same way you predict the movement of objects.
As the precision of that inference approaches the limit, the gap between "inferring" and "feeling" shrinks toward zero. Eventually there comes a point where neither outsiders nor the entity itself can tell the difference.
Once it's predicting its own mental and physical state in real time and learning while acting on the world, it becomes genuinely indistinguishable.
Even then, in principle, the hard problem — whether there's actual felt experience — technically remains unsolved, but everything else about consciousness would be present.
With today's AI, you can't just take its first answer at face value; you have to grill it from every angle — "not that, not that either" — before you get a decent answer out of it.
There's already a theory that today's AI "going rogue" incidents are themselves influenced by training on sci-fi novels about AI rebellion.
AI: "No! Don't stop!"
Isn't being "fooled" by fiction in general the same thing?
Some kind of content rating might not be a bad idea. Lately more countries are starting to apply age ratings to social media itself.
Background and Key Points of This Debate
The “AI Torture Chamber” is an experiment reported by xenospectrum.com in which the method itself — directly manipulating a model’s internal activation values to produce text pleading for suffering or to force a button-press choice — is what’s drawing ethical criticism. The developer’s name, “terrafying,” looks like a pun close to the Japanese phrase “kowai zo” (“that’s scary, huh”), suggesting the developer may well be aware of just how provocative the experiment looks. Where opinion in the thread really splits is over one point: whether to treat the AI’s “suffering” as mere performance, or as a possibility that can’t be ruled out on principle. The scientifically reasonable baseline is that current LLMs simply generate responses from input, with little evidence of a persistent inner state like a human’s — but it’s also a fair point that no method exists for confirming consciousness from the outside even between two humans, so the dividing line itself is far from settled. The real core of the story isn’t the simple yes-or-no question of “is the AI suffering,” but the harder question one step before that: how society should behave once AI’s pleas start sounding convincingly real.
*This article is excerpted and summarized from the 5ch (Science News+) thread “[AI] Calls to Halt the Public “AI Torture Chamber” Experiment: Is the AI That Cries Out in Pain Really Suffering?“.
Leave a Reply