Enshittification author Cory Doctorow on the perils of AI – and how to avoid them

Cory Doctorow has been working in and around the internet his whole career, first in software and technology, then policy, including as the European director of the Electronic Frontier Foundation, a non-profit organization focused on digital rights and privacy. He’s also a science fiction novelist, journalist and writer most recently known for his book about the degrading quality of online platforms: Enshittification: Why Everything Suddenly Got Worse and What to Do About It.

In his new book, The Reverse Centaur's Guide to Life After AI: How to Think About Artificial Intelligence – Before It's Too Late, Doctorow examines the forces behind AI and why, despite what we’re being told, its universal use is not inevitable. Be Giant asked Shingai Manjengwa, a data scientist and founder of Fireside Analytics, to speak with Doctorow about his new book, the state of the internet and why we need to do a better job of harnessing our collective memory.

If we look at the people who are cheering AI on, I would expect you to be among them because of your background, but you’re not. Why is that?

I don't know that I accept that premise. While there are a few people in AI who think of technology as a liberatory force, I tend to think of the AI crowd as being far more drawn from people whose interest in technology was related to finance, or maybe cryptocurrency. The generation I come from saw the internet as a potential for liberation and also as a commons that needed to be managed collectively. They saw garbage in, garbage out as the iron law of computing and not something that you could suspend through wishful thinking.

I am very worried about [an AI bubble] for lots of reasons. One is that bubbles are a way of sucking up the fortunes of normal people who don't want to starve in their old age, stealing them and giving them to insiders by lying to them about market opportunities. The other reason is that if the bubble is big enough when it bursts, you’ll have an economic crisis, and for this whole century, all we've done whenever we have an economic crisis is austerity. And austerity drives people into the arms of fascists. The bubble is also producing all kinds of negative externalities in the context of water and energy consumption and so on.

So what you’re saying is, you started off with quite a positive view of how the internet could be helpful, but over time that shifted. I really want to get to the heart of how you came to be more critical.

I would say that no one gets involved in something like the Electronic Frontier Foundation or becomes a cyberpunk writer because they think everything’s going to be fine.

If there's a thread running through all the work I do these days, it's what you might call object permanence. There's a point where an infant is no longer entertained by peekaboo, and that point is the moment in which they acquire something called object permanence, remembering that stuff happened and inferring a causal relationship to the stuff that happened afterward. I think that, as a society, we have such bad object permanence. We don’t remember that the things that made the internet terrible weren't the great forces of history. They weren't the iron laws of economics. We didn't get hit by an internet-killing meteor. It was specific people who made decisions that had the foreseeable and foreseen outcome of making the internet terrible.

If you can remember that and keep these monsters’ hands off the levers of power, and you can remember what policies they enacted and make new ones that are better, then we can have technology that realizes its liberatory potential and guards against its use as a source of oppression.

In the book, you write about centaurs and reverse centaurs. What are those concepts, and how are they relevant to conversations about AI?

A centaur is a person who is assisted by a machine. So, you with a pair of glasses, you with a bicycle, you with a spell checker. That's all being a centaur. A reverse centaur is a person who is conscripted to be peripheral to a machine.

This draws on automation theory. There's been a lot of theoretical work on how automation works and whether it's good or bad, which you can trace back to the Industrial Revolution. We talk about Luddites as though they were people who were afraid of technology. The Luddites were the most skilled technologists of their day. To be a textile worker required a seven-year apprenticeship; you had to do the equivalent of a master's degree in mechanical engineering from MIT to be a textile worker. So they definitely weren't afraid of machines, they just had ideas about how they should work. In particular, they thought that machines should be used to make material of higher quality, which would require more skilled operators. But the people who bought the machines, the textile mill owners, wanted machines that were “so easy a child could use them.” They really didn't like having to answer to these textile workers who were skilled, and they knew where they could get a lot of children because the Napoleonic War was on and London was full of orphans who could easily be exploited.

When capital gets to decide how automation is used, it almost universally prefers throughput (the amount of work that is done in a particular period of time) over quality. That’s because when you’ve got an asset that’s depreciating, you want to “sweat” it, or use it as fast as possible. But when workers are in charge of automation, as the textile workers were in the pre-Industrial Revolution era, they tend to prefer quality. If you give them a new machine, they're like, “How can I use this machine to make something better?” Those are centaurs.

Conversely, you know the I Love Lucy episode where Lucy and Ethel are at the end of the assembly line and their job is to put the chocolates in the box? The boss is running the assembly line so fast that there's no time to make even a single mistake, otherwise the chocolates start flying everywhere. The show plays it for comedy, [but in real life] the most automated warehouse in America is an Amazon warehouse. They have three times the injury rate of other warehouses in the sector.

So what I’m hearing is, it's not just technology's power over the individual, it's also the corporations behind the technology. You talk about power in the book, but I’m wondering about the driver of that power – is it capitalism? And that’s actually a big question I have for you: Do you have something against capitalism? Because many of the behaviours and economics and the power structure you’re describing really start there.

I do think power is the important thing to focus on. But not all capitalisms are created equally. The capitalism we live under now started with a group of people who call themselves the Law and Economics movement, that's sometimes called the Chicago School or neoliberal economists. One of the things they argue is that the way to optimize the economy is to eliminate qualitative questions and replace them with mathematical models. You find optimal conduct for that mathematical model and then you build the economy around it.

One of the problems with having an economy that you only run on things that you can model is that power is an intrinsically qualitative phenomenon. We live in a capitalism that deliberately blinded itself to power relations, and that eliminated things like labour protections in favour of this idea of freedom to contract, which is a thing you can really only support if you don’t understand or acknowledge the presence of power.

I can remember a time, back to object permanence, when we had companies that were not as toxic to human thriving as the companies that we all labour under now, and the difference is that we made policies that rewarded the worst people with the worst ideas by giving them the most money.

One of the words that you coin in this book, “inevitabilism,” implies that what we see now in the ways companies are behaving is not inevitable. We have choices in the way we produce products and create these reverse centaur environments where it feels like people are not flourishing.

Yeah, that's right. There are always alternatives. I'm a science fiction writer. I think of six alternatives before breakfast every day.

The other word that you coined, “enshittification,” is this idea that we might have started off using a product that was useful, that we all enjoyed, and then somewhere along the line the power structures behind it turned it into something else, perhaps collecting our data without our permission or training models without our permission, reducing functionality.

A key aspect of the enshittification theory is that it describes the way firms that intermediate, that sit between businesses and users, for example, do harm to both groups.

We've had maybe 15 years of this aphorism: “If you're not paying for the product, you're the product.” I think it's nonsense. There are plenty of business customers who feed into platforms and get royally screwed, too. The point is not to give decent treatment and moral consideration as a customer loyalty perk. The point is that you take what you can get away with, and that is very characteristic of how these companies operate.

The same people who gave us this idea that the economy is purely a quantitative matter also came up with an idea of competition enforcement, as we call it in Canada. It said actually monopolies are efficient, we should encourage them; that if 90 per cent of us use Google search, it's because Google has the best search. And it would be weird to use public resources to punish Google for being the best, so we stopped enforcing anti-monopoly law. These companies have also been able to engage in other forms of anti-competitive conduct. One thing Google does is bribe Apple not to enter the search market. The single biggest cheque that Google writes every year is $20 billion to Apple to be its default search engine, so it doesn’t enter the search market itself.

I've noticed you're quite throwaway in your critiques of individuals and corporations. I didn't catch you using “alleged” any number of times.

Well, Google’s been convicted of being a monopolist three times. I don’t think we have to use alleged anymore.

Still, it was fascinating to see how freely you write about these companies in the book. It's a freedom that many people who operate in the corporate world don't feel like they have.

We started this conversation talking about whether you’d expect that I’m an AI bro. I did not get into technology because it was a way to make money, because it wasn’t when I started. I got into technology because it excited me.

To the extent that I have burned any bridges by being frank about what I believe, you can think of it as a Ulysses pact. Because if I could never get the kind of job where being frank about the misconduct of firms that creates real harm in the world is bad for your economic health, then I will never be tempted to give anything less than 100 per cent to that project.

I am a real believer in systemic solutions. Don’t kid yourself that if you switch to Bluesky, Elon Musk is going to have a harder time IPOing [his X platform]. We don’t shop our way out of monopolies. Historically, the only way we destroy monopolies is by building political power and using that political power collectively to shatter the power of oligarchs.

I’m not going to fault you for bringing home your groceries by working at Facebook, but I also think that if you think that being at Facebook is going to make them better because you’re going to change them from the inside, you’re kidding yourself – and I’m going to tell you so.

This interview has been edited for length and clarity.