Karen Hao on the Empire of AI, AGI evangelists, and the cost of belief
Image Credits:Karen Hao
At the center of every empire is an ideology, a belief system that propels the system forward and justifies expansion – even if the cost of that expansion directly defies the ideology’s stated mission.
For European colonial powers, it was Christianity and the promise of saving souls while extracting resources. For today’s AI empire, it’s artificial general intelligence to “benefit all humanity.” And OpenAI is its chief evangelist, spreading zeal across the industry in a way that has reframed how AI is built.
“I was interviewing people whose voices were shaking from the fervor of their beliefs in AGI,” Karen Hao, journalist and bestselling author of “Empire of AI,” told TechCrunch on a recent episode of Equity.
In her book, Hao likens the AI industry in general, and OpenAI in particular, to an empire.
“The only way to really understand the scope and scale of OpenAI’s behavior…is actually to recognize that they’ve already grown more powerful than pretty much any nation state in the world, and they’ve consolidated an extraordinary amount of not just economic power, but also political power,” Hao said. “They’re terraforming the Earth. They’re rewiring our geopolitics, all of our lives. And so you can only describe it as an empire.”
OpenAI has described AGI as “a highly autonomous system that outperforms humans at most economically valuable work,” one that will somehow “elevate humanity by increasing abundance, turbocharging the economy, and aiding in the discovery of new scientific knowledge that changes the limits of possibility.”
These nebulous promises have fueled the industry’s exponential growth — its massive resource demands, oceans of scraped data, strained energy grids, and willingness to release untested systems into the world. All in service of a future that many experts say may never arrive.
Hao says this path wasn’t inevitable, and that scaling isn’t the only way to get more advances in AI.
“You can also develop new techniques in algorithms,” she said. “You can improve the existing algorithms to reduce the amount of data and compute that they need to use.”
But that tactic would have meant sacrificing speed.
“When you define the quest to build beneficial AGI as one where the victor takes all — which is what OpenAI did — then the most important thing is speed over anything else,” Hao said. “Speed over efficiency, speed over safety, speed over exploratory research.”
For OpenAI, she said, the best way to guarantee speed was to take existing techniques and “just do the intellectually cheap thing, which is to pump more data, more supercomputers, into those existing techniques.”
OpenAI set the stage, and rather than fall behind, other tech companies decided to fall in line.
“And because the AI industry has successfully captured most of the top AI researchers in the world, and those researchers no longer exist in academia, then you have an entire discipline now being shaped by the agenda of these companies, rather than by real scientific exploration,” Hao said.
The spend has been, and will be, astronomical. Last week, OpenAI said it expects to burn through $115 billion in cash by 2029. Meta said in July that it would spend up to $72 billion on building AI infrastructure this year. Google expects to hit up to $85 billion in capital expenditures for 2025, most of which will be spent on expanding AI and cloud infrastructure.
Meanwhile, the goal posts keep moving, and the loftiest “benefits to humanity” haven’t yet materialized, even as the harms mount. Harms like job loss, concentration of wealth, and AI chatbots that fuel delusions and psychosis. In her book, Hao also documents workers in developing countries like Kenya and Venezuela who were exposed to disturbing content, including child sexual abuse material, and were paid very low wages — around $1 to $2 an hour — in roles like content moderation and data labeling.
Hao said it’s a false tradeoff to pit AI progress against present harms, especially when other forms of AI offer real benefits.
She pointed to Google DeepMind’s Nobel Prize-winning AlphaFold, which is trained on amino acid sequence data and complex protein folding structures, and can now accurately predict the 3D structure of proteins from their amino acids — profoundly useful for drug discovery and understanding disease.
“Those are the types of AI systems that we need,” Hao said. “AlphaFold does not create mental health crises in people. AlphaFold does not lead to colossal environmental harms … because it’s trained on substantially less infrastructure. It does not create content moderation harms because [the datasets don’t have] all of the toxic crap that you hoovered up when you were scraping the internet.”
Alongside the quasi-religious commitment to AGI has been a narrative about the importance of racing to beat China in the AI race, so that Silicon Valley can have a liberalizing effect on the world.
“Literally, the opposite has happened,” Hao said. “The gap has continued to close between the U.S. and China, and Silicon Valley has had an illiberalizing effect on the world … and the only actor that has come out of it unscathed, you could argue, is Silicon Valley itself.”
Of course, many will argue that OpenAI and other AI companies have benefitted humanity by releasing ChatGPT and other large language models, which promise huge gains in productivity by automating tasks like coding, writing, research, customer support, and other knowledge-work tasks.
But the way OpenAI is structured — part non-profit, part for-profit — complicates how it defines and measures its impact on humanity. And that’s further complicated by the news this week that OpenAI reached an agreement with Microsoft that brings it closer to eventually going public.
Two former OpenAI safety researchers told TechCrunch that they fear the AI lab has begun to confuse its for-profit and non-profit missions — that because people enjoy using ChatGPT and other products built on LLMs, this ticks the box of benefiting humanity.
Hao echoed these concerns, describing the dangers of being so consumed by the mission that reality is ignored.
“Even as the evidence accumulates that what they’re building is actually harming significant amounts of people, the mission continues to paper all of that over,” Hao said. “There’s something really dangerous and dark about that, of [being] so wrapped up in a belief system you constructed that you lose touch with reality.”
Leave a Comment
Your email address will not be published. Required fields are marked *
Leave a Comment
Your email address will not be published. Required fields are marked *