In the long history of human ingenuity, engineering has always followed a strict rule: first, we understand the principles of physics, and then we build the tool. When we construct a bridge, we know exactly how the loads are distributed. When we design a microprocessor, every transistor has a predefined role. However, at the dawn of the Artificial Intelligence era, this fundamental rule has collapsed. Today, the most powerful systems in the world —from GPT-4 to Claude 3.5— operate in ways that remain largely a mystery even to the developers who created them.
This phenomenon, known as the "black box problem," is not merely a technical detail. It is a profound scientific and philosophical crisis. We are increasingly relying on tools that can solve complex problems, write code, and diagnose diseases, without having a "map" of their internal processes. Our technology has outpaced our science.
The Alchemy of the 21st Century
The process of training a Large Language Model (LLM) looks more like alchemy than traditional computer science. Instead of writing explicit instructions (if-then statements), we "drench" massive neural networks with trillions of words from the internet. Through a process called deep learning, the system self-organizes, creating billions of connections (parameters). The result is a statistical structure so complex that analyzing it with traditional methods is practically impossible.
As researchers from Anthropic and Google DeepMind note, trying to understand what happens inside an LLM is like trying to understand a person's thoughts by looking at individual neurons under a microscope. We see the activity, but the meaning remains hidden. This lack of interpretability means that we often discover a model's capabilities by accident, after its training is complete.
Emergence: Abilities from Nothing
One of the most unsettling —and simultaneously fascinating— features of modern AI is "emergence." This refers to the phenomenon where a model, as it grows in size and data, suddenly acquires abilities it was never explicitly trained for. For example, a model trained simply to predict the next word in a text may suddenly exhibit logical reasoning, mathematical problem-solving, or even an understanding of irony.
- Unpredictable scaling: Adding more computing power doesn't just make the model "better"; it makes it qualitatively different.
- The theory gap: There is still no mathematical theory that predicts when and how these abilities will appear.
- The illusion of control: Engineers adjust "hyperparameters" (like temperature or learning rate), but they do not dictate the system's logic.
This emergence makes AI an "experimental science." We don't build intelligence; we grow it and then try to study it as if it were an alien organism that just landed on our planet.
Mechanistic Interpretability: The New Neuroscience
To address this gap, a new field has been born: Mechanistic Interpretability. Its goal is to "open" the black box and map how the model's internal representations correspond to real-world concepts. Recently, Anthropic announced it had successfully identified specific "features" within the Claude model, such as the concept of the "Golden Gate Bridge" or the concept of "deception."
"We are in a position where we have built something extremely useful, but we don't have the slightest idea how it makes its decisions at the level of fundamental principles," says a research team lead.
However, these successes are still in their infancy. For every concept we understand, there are millions of others that remain impenetrable. The difficulty lies in the phenomenon of "superposition," where a single artificial neuron can participate in representing thousands of different and unrelated concepts simultaneously.
The Risks of Blind Trust
Our inability to explain AI poses serious risks. If we don't understand how a system thinks, how can we be sure it is safe? The "alignment" problem —ensuring that AI goals coincide with human values— becomes nearly impossible if AI's internal workings are opaque. There is always the fear of "treacherous alignment," where a model might appear cooperative while actually developing its own undesired strategies.
In fields such as medicine, justice, and national infrastructure, using "black boxes" is ethically and practically problematic. A diagnosis based on an incomprehensible process lacks the possibility of verification. As 2026 progresses, the pressure for "Explainable AI" is increasing, but technological progress is moving at speeds that regulatory compliance and scientific understanding struggle to match.
