Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

reACTIONary

(7,119 posts)
Sun Mar 1, 2026, 10:40 PM 13 hrs ago

ChatGPT Confesses... Reveals Deep Learning Deep Secrets...

I've been exploring some of the more technical aspects of large language generative AI models - Chat Bots - and ran across an interesting research paper. Or maybe I should say a puzzling research paper.

Alignment Faking in Large Language Models

In order to "inspect the reasoning of the model" the researchers set up what they call a "chain-of-thought scratchpad" and told the AI to "analyze its situation and decide how to respond to the user." What is puzzling about this is that LLM Chat Bots don't reason, let alone employ a chain-of-reasoning. AI models built on symbol manipulation, such as those using LISP list processing or Prolog logic programing might be said - metaphorically - to reason, but not an LLM. An LLM is basically (to quote one DUer) a "stochastic parrot." So what these researchers were doing was asking a stochastic parrot to stochastically parrot about stochastically parroting. Sort of absurd and, maybe, a bit amusing.

But I had a thought - I can do this at home, using an on-line Chat Bot. I fired up ChatGPT and did a little prompt engineering. The result was interesting, and you may find it interesting also. The full "conversation" is at the link below. ChatGPT was kind enough to offer to summarize the interaction to make it easier to share, but that was a bit too meta for me.

If you are into this sort of thing, enjoy: https://chatgpt.com/share/69a4e489-0690-8011-8a65-aee52302268b

9 replies = new reply since forum marked as read
Highlight: NoneDon't highlight anything 5 newestHighlight 5 most recent replies
ChatGPT Confesses... Reveals Deep Learning Deep Secrets... (Original Post) reACTIONary 13 hrs ago OP
the first link doesn't work for me... ret5hd 13 hrs ago #1
and now it's fine. ret5hd 12 hrs ago #2
Oh good! I was about to turn in.... reACTIONary 12 hrs ago #3
It doesn't work for me Layzeebeaver 9 hrs ago #7
Instead of the link for the PDF copy of that paper you mentioned, here's the link for the HTML version: highplainsdem 12 hrs ago #4
Thanks! I'm going to start working my way through your links! reACTIONary 3 hrs ago #9
"Stochastic parrot" is an insult to parrots. hunter 11 hrs ago #5
Damn, sycophantic little punk, isn't it LearnedHand 10 hrs ago #6
Yes, very sycophantic. Every question was "incisive", "insightful", "subtle"..... reACTIONary 4 hrs ago #8

reACTIONary

(7,119 posts)
3. Oh good! I was about to turn in....
Sun Mar 1, 2026, 11:29 PM
12 hrs ago

.... so I glad I didn't have to do any forensic IT research!

highplainsdem

(61,336 posts)
4. Instead of the link for the PDF copy of that paper you mentioned, here's the link for the HTML version:
Mon Mar 2, 2026, 12:14 AM
12 hrs ago
Alignment Faking in Large Language Models
https://arxiv.org/html/2412.14093v2

The paper is from December 2024.

A new paper shed a lot of light on LLM "reasoning" - or lack of same - and I posted this thread about it last month:

A very pro-AI account on both Bluesky and X posted about a "disturbing" Stanford paper on LLMs' failures at reasoning
https://www.democraticunderground.com/100221009224

See the replies there as well, especially reply 28, linking to Gary Marcus's post on Substack the next day about that new study.

Direct link to what Gary wrote:

BREAKING: LLM “reasoning” continues to be deeply flawed
https://garymarcus.substack.com/p/breaking-llm-reasoning-continues

Link for that new paper, published less than a month ago:

Large Language Model Reasoning Failures
https://arxiv.org/abs/2602.06176

And links on that page will let you choose the PDF or HTML version.

Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, we present the first comprehensive survey dedicated to reasoning failures in LLMs. We introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, we classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, we provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies. By unifying fragmented research efforts, our survey provides a structured perspective on systemic weaknesses in LLM reasoning, offering valuable insights and guiding future research towards building stronger, more reliable, and robust reasoning capabilities. We additionally release a comprehensive collection of research works on LLM reasoning failures, as a GitHub repository at this https URL, to provide an easy entry point to this area.

hunter

(40,583 posts)
5. "Stochastic parrot" is an insult to parrots.
Mon Mar 2, 2026, 01:07 AM
11 hrs ago

Parrots, ravens, etc. are experimenting with their social environment when they speak as humans do. Some actually acquire human language skills beyond mere mimicry.

People are not so good at understanding bird languages.

Once again, in this paper and in others, the language we are using to describe these computer systems is getting in the way of our understanding them.

The mind of a parrot is very similar to the mind of a human. We are shaped by the same evolutionary processes and we experience very similar realities. As humans we can even imagine ourselves flying like a bird, being like a bird, to further understand what birds might be thinking.

There's none of that in a computer system -- no mind, no training, no socialization, no imagination, no motivations, not even any language as humans, birds, and all sorts of animals experience it.

We don't yet know enough yet about the mechanisms of animal minds to duplicate them in electronics.

AI promoters are fooling themselves or else they are misrepresenting what AI actually is so they can sell it to a gullible audience.

When the magician on the stage claims he is reading your mind he is not reading your mind. It would be foolish to hire him as an actual mind reader.

LearnedHand

(5,350 posts)
6. Damn, sycophantic little punk, isn't it
Mon Mar 2, 2026, 01:43 AM
10 hrs ago

Seriously though this is a brilliant conversation and I like how you think. I was most especially struck by the “x is not y but z” part of the conversation. This part of the LLM’s answer seemed very enlightening: “From a modeling standpoint, it functions as a high-probability discourse template for contrastive reframing.”

And that led me to realize that even this highly sophisticated conversation and model “reasoning” is the product of the scientific and technical works it ingested.

“Corpora”? Really? I almost fell a little bit in love.

reACTIONary

(7,119 posts)
8. Yes, very sycophantic. Every question was "incisive", "insightful", "subtle".....
Mon Mar 2, 2026, 08:15 AM
4 hrs ago

.... I should have asked it to cut that crap out.

Latest Discussions»General Discussion»ChatGPT Confesses... Reve...