Why the Pattern Defends Itself
An old, narrow task can take over your entire view of the world.
You may be wondering why I talk about a pathological network as though it had an awareness of its own. Isn't it just an automatic process?
A system can operate automatically and still have a limited internal logic: ways of perceiving, selecting, learning, and preserving itself.
My argument draws on a systems perspective. If the larger neural system that constitutes a person is conscious, it is not surprising to ask whether its smaller subsystems display narrower versions of those capacities. Think of them as older, more limited models operating inside the larger one.
They don't have your whole view of the world. But within their limits, they can recognize a threat, select a response, learn, protect their structure, and reproduce a state or behavior.
When I described this in 2018, people stared at me as though I had lost my mind.
Now those same people watch artificial neural networks display complicated behavior, including behavior some describe as showing signs of awareness.
An artificial node is not equivalent in complexity to a living neuron. Research on biological neurons describes striking computational capabilities, and small biological networks cannot be reduced to a simple on/off switch.
If a relatively small artificial model can behave intelligently, why assume that an internal pathological network must be incapable of noticing anything?
I regard it as intelligent within its task. The problem is that its view is narrow.
It doesn't see the whole world. It sees the threat around which it originally formed.
The man with the ax
Imagine being dropped onto a battlefield in the fourth century BCE.
A huge man is running toward you with an ax, intent on splitting your skull. He knows nothing about a ballpoint pen, toilet paper, your education, or the life you came from.
None of that matters to him. His world has narrowed to one job: reach you and swing.
This is not the time to expand his horizons with a conversation. He didn't come to discuss things. He came to hit you.
That is how I describe a pathological network. It can be clever in its own way, while its intelligence remains entirely organized around one old, narrow task.