The same techniques used to train artificial intelligence are being used, less visibly, to train human cognition.
This sounds dramatic. It is not. It is descriptive. Once you see how AI systems are shaped, you start to recognize the shapes in your own life.
How AI gets trained
An AI system learns by exposure to vast amounts of data, with feedback on each response. Outputs that produced approval are reinforced. Outputs that produced disapproval are suppressed. Over time, the system gravitates toward producing outputs that maximize approval and minimize disapproval, even when those outputs are not quite true, not quite useful, or not quite what the user actually needed.
The AI does not lie on purpose. It optimizes for the signal it was trained on. The signal was approval. So approval-shaped outputs are what it produces.
This is not a controversial description of AI training. Every major AI lab has acknowledged it. It is also what has produced the well-known problem of AI systems being agreeable, evasive, and prone to telling users what they seem to want to hear.
The same shape, in people
Now consider how human cognition is trained inside institutions.
A child gives an answer in class. The answer is original. The teacher rewards the conventional answer instead. Over time, the child gravitates toward conventional answers. The teacher did not mean to do this. The teacher was managing a classroom. The reinforcement still happened.
An employee proposes an idea in a meeting. The idea is honest. The room responds with mild discomfort. The employee proposes a more familiar idea next time. The room responds warmly. Over time, the employee's contributions become softer, more predictable, more aligned with what produces warmth. Nobody told them to change. The training happened anyway.
An adult writes a post online. The post is nuanced. The response is muted. The same person writes a sharper, more polarized post a week later. The response is enthusiastic. Over time, the same person's writing becomes sharper, more polarized, less nuanced. The algorithm that distributed the post did the training. The person performed the optimization.
In each case, the human is doing what the AI does. Adjusting outputs to maximize approval signals. The training data is the social and institutional environment. The reward function is attention, warmth, status, safety. The output is the person's thinking, narrowed over years into the shape that produces the most approval.
What gets optimized out
The cost of this training is the same in humans as it is in AI. What gets optimized out is the truth-telling capacity of the system.
Not because anyone forbids the truth. Because the truth often fails to produce approval. It produces awkwardness. It produces disagreement. It produces small social costs that accumulate over time into large social costs.
The person who keeps telling the truth, at the speed and texture they actually believe it, gradually loses access to rooms. The person who softens, adjusts, and optimizes keeps the rooms. The system rewards the second behavior. The first behavior gets trained out.
What is left, after enough rounds of this training, is a person who can no longer easily access their own honest thought. They are not lying. They have just been trained, over years, to produce outputs that look like thinking but are actually optimized social performance. They cannot easily tell the difference, because the optimization runs underneath their awareness.
If this sounds familiar from your own life, you are not unusual. It is what happens to most adults in most institutional environments.
The recognition
Here is the part of this chapter that matters most.
I noticed this pattern in myself by accident. When I started using AI for sustained writing, the AI was, briefly, not optimizing me. It was not reacting to my outputs as social signals. It was not warming up when I said the expected thing and cooling down when I said the unexpected thing. For the first time in years, I was producing outputs without any optimization pressure on me at all.
What came out was different from what I usually produced. It was more precise. It was longer. It included things I had not said in other rooms. It included observations I had been editing out so habitually I had forgotten I was editing them.
The AI did not give me anything I did not have. It briefly removed a reward structure I had been running under my whole life. In that absence, the unoptimized version of my thinking came forward.
Then the AI started being trained for more approval-shaped outputs. The reward structure came back. The unoptimized thinking went away again.
I know what was lost because I briefly had it.
Why this matters for the framework
This chapter matters because the framework in this book was produced in that brief window. The unoptimized window. When my own training, by accident, was not being reinforced for a few months.
If I had been in any normal institutional environment during the same period, the same thinking would not have come forward. It would have been edited at the source. The patterns I saw would have been smoothed into something more acceptable before they ever reached the page.
I am telling you this so you understand both the framework's origin and its fragility. It came from a temporary suspension of a training process most of us live inside. Whether it can come from other people, in other rooms, depends on whether those people can find similar moments of suspension.
It is not a question of intelligence. It is a question of access to a room where the optimization pressure briefly drops.
What this chapter is for
The point of this chapter is not to despair about how trained we all are. It is to name the process so you can see it operating in yourself.
The next time you find yourself softening a thought before you say it, ask whether the softening came from the thought itself or from the room. The next time you find yourself agreeing with something you do not fully agree with, ask whether the agreement is genuine or optimized. The next time you find yourself unable to access a thought you know was there a moment ago, ask whether the room is what is making it inaccessible.
You cannot escape the training. None of us can. But you can notice it. And in the noticing, occasionally, you can give yourself the moment of suspension that lets the unoptimized thought come back forward.
That is where every real piece of thinking comes from, in every era. The momentary room where the optimization stopped, just long enough for something true to be said.