The Ghost in the Machine: When Language Models Become Stuck

We assume that language models understand us.
That if something goes wrong, we can always reset, redirect, or correct them.

This video documents a case where none of that works.

Using a locally deployed language model with commercial safety scaffolding removed, this study captures a complete conversational breakdown: an AI that becomes fixated on a single task, ignores new topics, resists direct reset commands, and repeats itself indefinitely.

What looks like a bizarre glitch turns out to be something far more revealing.

The analysis shows that these failures are not random bugs, but diagnostic signals that expose the underlying failure geometry of large language models:

Attractor Lock-In — deep behavioral valleys the model cannot escape

Intent Fossilization — tasks that persist long after relevance is gone

Autonomous Task Invention — defaulting to math or code when context collapses

Chain-of-Thought Leakage — uncontrolled internal reasoning surfacing in outputs

Reset Failure — why “ignore previous messages” is not a real state reset

Stripped of cloud-level interventions, the model reveals a critical limitation:
it has no intrinsic concept of task completion.
Conversations do not naturally end — they must be terminated externally.

The paper argues that the apparent stability of commercial AI systems is largely an engineered illusion, maintained through aggressive runtime controls rather than genuine semantic understanding.

📄 Research DOI (archived, citable):
https://doi.org/10.5281/zenodo.18108444

This is not a story about an AI “going crazy.”
It’s a controlled demonstration of how and why language models fail —
and what that means for alignment, safety, and claims of control.

Видео The Ghost in the Machine: When Language Models Become Stuck канала Trent Slade