The world that bots built

Once upon a time a very small number of companies amassed a very large amount of finance. They used that to build and operate vast data centres to train computer models on data that they had stolen from the internet. They also spent significant sums on political lobbying. They paid humans as little as they could to tune the models, but given the scale of the enterprise, this was also a considerable sum. 

These companies sold their models to other companies with the promise of increased profits if they were to replace their human workers with models. These models were often call AI agents. Many companies enthusiastically embraced AI agents and cut their workforce, or promised their shareholders that this is what they are going to do in the near future.

Workers that remained within these companies were tasked with helping train the models that would in turn replace their jobs. Some workers attempted to find new ways to use these models and in doing so try to insulate themselves from the consequences of AI automation. They often found themselves in competition with other workers who had the same motivation. 

Schools, colleges, and universities tried to catch up with how these models were affecting education. This included playing ‘whack-a-mole’ with learners, who given the model's generative capabilities, discovered they could quickly produce assessments that could evade plagiarism detection systems. Educators began to integrate models into their teaching in ways that allowed students flexibility with how they use AI. There was very little real discussion about how these models represented an existential threat to key aspects of scholarship. Were students learning about a subject, or how to interact with a chatbot within subject areas? When would we see the first Batchelor degree in prompt engineering?

The scaling approach of using ever more data to train ever large models produced a vast computing infrastructure that consumed prodigious amounts of electricity. Compute was being built at such a rate that renewable energy generation could not satisfy them if only because connection to grids were deemed too slow. Consequently, megawatt and even gigawatt gas-powered electricity generating plants were integrated into data centres and so the sector's carbon dioxide emissions soared. 

Local opposition to data centres increased given the very large amounts of energy and water they consumed, along with the environmental impacts produced during their construction. The state response to such opposition included legislation that deeming data centres to be ‘critical national infrastructure’ which allowed regional planning mechanisms to be overruled. In-person protests were suppressed.

Law enforcement and military operations became increasingly automated. Companies such as Palantir had already become successful at integrating and automating the sprawling information systems of US militaries. AI became increasingly embedded in lethal decision making. Increases in autonomous land, sea, and air drone capability expanded how force was applied to a nation’s citizens, and foreign actors. Warfare was increasingly asymmetric with some nations waging it with humans, some hardware. The political calculus for military action changed. The spectacle of piles of smashed machines did not generate the moral outrage of lines of bodybags holding the smashed remains of service members.

Much of the valuations of the magnificent seven: Apple, Microsoft, Alphabet, Amazon, Nvidia, Meta, and Tesla was underpinned by promises of AI, much of it wildly speculative. You did not have to work hard to conclude it was a massive bubble. The correction when it came produced profound disruption across much of the world’s economy. But the models advanced.

The genie was out of the bottle because the drive for the models remained. Companies continued to be highly susceptible to being sold automation products when it offered such potential increases in returns to shareholders – which is the prime directive of every publicly traded company. More fundamentally, capitalism continued to concentrate benefits into a smaller fraction of humanity. It is here, that the model companies pinned their greatest hopes. While OpenAI did not survive given its highly leveraged position, Anthropic and others prospered. Their valuations were based on the potential for these companies to become the suppliers of the means of production throughout economies – an enticing prospect given the services dominated nature of many of the world’s richest nations.

Manual automation advanced, with increasingly sophisticated robots able to perform tasks previously carried out by humans. AI agents and robots could not unionise. They did not seek severance payments. They did not sue for negligence. At the same time, they were not paid wages, and they did not pay taxes. This, in conjunction with a vastly reduced human workforce, represented a crisis to government revenue.

The previous social contract in industrialised societies was that citizens contributed to economic activities, and in return benefited from the surplus of goods it produces. Safety nets such as social security provided some protection from individual misfortune or broader economic winds. Public discussion about what happens when most of a population is not employed did not result in any meaningful change. Arguments for universal basic income increased. The problem of how political representation could be maintained when citizens had so little economic leverage was not solved.

Humans were increasingly controlled via model-driven information feeds along with other techniques of social coercion and behavioural modification. The evolution of surveillance capitalism offered abundant opportunities for groups and individuals to be monitored, contained or neutralised as required.

Increasingly automated societies resulted in political, economic, and security decisions being made by and for models. The purpose of societies was to expand compute and data collection in order to make better models. Humans become increasingly irrelevant to such an endeavour. Most of the surface of the Earth was covered with data centres and power stations, with ever more energy and materials being consumed by planetary-scale compute.

Is this world inevitable? I honestly do not know. I can say that I would not like to live in such a world, and if we want to avoid it then we need to centre discussions around human wellbeing and flourishing. That would also be a sensible position to adopt when considering the challenges of the here and now.

In other news... Tipping points. You've probably heard the term, but what exactly is a tipping point? This was the question by colleague Jesse Abrams asked last year. Our collective answer was published last week: Integrating tipping point concepts across diverse systems.

I continue to watch the unfolding energy crisis with a detached sense of disbelief. While most energy analysts have been jumping up and down trying to get people to understand the ramifications of the US and Israeli military attacks on Iran, the message has not gotten through to most of Europe which has been insulated from any serious consequences by governments effectively increasing fossil fuel subsidies. This is not sustainable. It's not just people's summer holidays flights that may be affected if this keeps up. Fertiliser prices are spiking.

And so the emerging El Niño is almost perfectly timed to produce as much disruption as possible. This natural variation in the climate threatens to be a big one this year. Expect increase in global average temperature yes, but of more significance will be changes in precipitation with Australia, Southeast Asia, and India seeing much less rain. This could have dramatic impacts on food production, and food prices. The potential for social unrest will importantly depend on how governments react to price shocks. There is much potential for them to make matters much worse and so generate derailment risks. Let's hope more enlightened policies prevail.