The Bonfire in the Cave
The insight that matters for everything that follows, is that the universe doesn’t merely permit these pockets of increasing order on their way to heat death. It preferentially selects for them. This is the Maximum Entropy Production Principle (MEPP),4,5 and while it remains debated at its theoretical edges, its descriptive power is surprisingly instructive: the history of the universe, viewed through this lens, is a story of successive phase transitions in dissipative efficiency. Stars are better entropy producers than dust clouds. Photosynthesizing life is better than bare rock. Multicellular organisms are better than single-celled ones. And civilizations, with their agriculture, their industry, their fossil fuel economies, are better still.
What we are doing with AI development, what Dario describes with such clarity and such evident discomfort, is running pure exploit on a civilizational scale. The exponential continues. The revenue grows. The data centers get built. The question of whether this particular configuration of intelligence and infrastructure is the one we want to lock in is never asked, because asking it would require pausing, and pausing is structurally impossible within the competitive dynamics. Every actor, from Anthropic to OpenAI to the Chinese labs, is exploiting the current gradient as fast as the physics allows. Not because they’re greedy or shortsighted, though the incentives are not exactly neutral, but because the selection pressure punishes any other strategy.
And the framing of inevitability, “we are near the end of the exponential,” stated as observation rather than confession, forecloses the only question that could change anything. Because the exponential is not something happening to civilization. It is a strategy civilization is running, and the people at the controls have internalized the strategy so deeply that it no longer registers as a choice.
)oshua makes the first singularitarian-adjacent argument that has made sense to me.
Unfortunately, the argument is for the AI companies to slow down and simultaneously and argument for why they can't and won't.
It's worth reading and considering even if you strongly disagree with its viewpoints.