The internet is getting old. Nearly four decades have passed since an unassuming British programmer named Tim Berners-Lee invented the World Wide Web in his office at the CERN particle-accelerator complex near Geneva. By transforming the net, until then an arcane research hub for scientists and engineers, into a buzzing, hyperlinked, multimedia information system, Berners-Lee unwittingly sparked the public’s mass migration online. He also, as he writes in his new memoir, This Is for Everyone, opened the gates for those who “might corrupt what I was trying to build.”

Born to two mathematicians in 1955—the same year, he notes, as Steve Jobs and Bill Gates—Berners-Lee was an unusual kid. Growing up in the quiet London neighborhood of East Sheen, not far from the Thames, he didn’t listen to rock music or follow sports teams or watch television, and he remained oblivious to the excitements and upheavals of the Sixties. “I missed out on pop culture more or less entirely,” he writes. Fascinated by the workings of electronic circuitry, he spent his teens building model railroads and cobbling together gadgets, including a rudimentary computer, from scavenged parts. An outstanding student, he entered The Queen’s College, Oxford, on a scholarship and went on to earn a first in physics. The door to an academic career was open, but he chose instead to pursue his passion for tinkering by becoming a writer of software code.

He took on a series of short-term coding assignments before landing a more permanent job at CERN in 1984. As a member of the Data and Documents division, he wrote programs for capturing detailed digital images of atoms colliding. He was well liked by his colleagues, though they often found it difficult to follow his rapid-fire, abstract style of speech. They would hold up signs reading Tim, slow down when he took the floor at meetings. When, early in 1989, he began talking up his idea for the World Wide Web—he originally called it Mesh but decided that sounded too much like mess—he described it as a way for researchers using incompatible computers to share experimental results and other data via documents uploaded to the internet and connected with hyperlinks. It was a tool, he explained, for “intercreativity.” His pitches were met with bafflement and indifference. “Few of us if any could understand what he was talking about,” his boss at the time recalled. His written proposals, illustrated with elaborate diagrams, went unread.

His break came in October 1990 when CERN decided to test one of the powerful new “cube” computers produced by NeXT, the company Steve Jobs formed after being ousted from Apple in 1985. Berners-Lee was chosen to “kick the tyres” of the machine, with the suggestion he use it to work on that “hypertext project” he couldn’t stop talking about. In an intense, one-man hackathon over the ensuing two months, he wrote the foundational codes that govern the design and functioning of web sites and browsers. Just before Christmas, he posted the world’s first web page on a CERN server. A year passed before people outside the lab noticed his invention. Then it exploded, becoming one of the most rapidly adopted technologies in history. Soon, “web” was a synonym for “internet,” and Berners-Lee was famous.

He moved to the United States, where excitement about cyberspace had been mounting for years, and joined the faculty at the Massachusetts Institute of Technology. MIT had agreed to host the World Wide Web Consortium, an independent standards-setting organization Berners-Lee set up in hopes of ensuring the web’s protocols remained open, free, and unsullied by the profit motive. He wanted his system, he made clear, to be geared to the needs of individuals and small groups, not those of big businesses. But even as the W3C, as the consortium was called, began hosting conferences and issuing guidelines, the web’s enormous commercial potential started drawing the interest, and the money, of venture capitalists, entrepreneurs, and corporations. Berners-Lee soon found himself battling them for control of his invention’s fate.

His first nemesis was an unlikely one: a husky midwestern kid named Marc Andreessen who had his own ideas about how the web should work. While an undergrad at the University of Illinois at Urbana-Champaign, he and a friend created a new browser, called Mosaic, that incorporated, over Berners-Lee’s objections, an unsanctioned code for displaying images directly on web pages. The innovation proved popular, and Mosaic quickly became the most widely used browser. The only problem for Andreessen was that Mosaic was owned not by him but by the university lab that funded his work. Sensing an entrepreneurial opportunity of epochal proportions, he decamped to Silicon Valley and joined forces with a wealthy investor, Jim Clark, to launch the browser company Netscape. With many new and attractive features, the company’s browser, Navigator, displaced Mosaic as the market leader. When Netscape went public in 1995, Andreessen made millions and Clark made even more. The internet gold rush was on.

While Berners-Lee would continue to win awards and accolades, culminating in his knighting by Queen Elizabeth II in 2004, he became a peripheral figure in the story of the web’s transformation into a global thoroughfare for commerce and communication dominated by a handful of very large, very profitable, very American companies. His dream of building a decentralized system that would give people “the ability to make links around the world” without “ensnaring them in dead-end, anti-human materialism, or systems of surveillance, coercion and control” went unfulfilled. The internet giants continued donating cash to the W3C, but they paid little heed to its strictures or goals. The businesses themselves now set the rules for how the web operates, and their rules made a mockery of its inventor’s ideals. Far from being a shield against materialism, surveillance, and control, the net became their instrument.

Berners-Lee hasn’t given up hope. If anything, he seems more optimistic than ever, seeing “signs of spring” sprouting everywhere. He rhapsodizes about a future internet that, having incorporated artificial-intelligence and virtual-reality systems, escapes the computer and the phone to become a permanent “overlay onto the physical world … a seamless informational filter on top of reality.” We will be wrapped up in it even more than we are already—a prospect he welcomes.

This “new web” will also be freed from corporate control. It “will be decentralized” and will grant its users “data sovereignty.” Thirty-five years of history will be erased, in other words, and Berners-Lee’s original vision for the web will at last come to pass. How he expects this to happen in the face of powerful, entrenched business interests is a little sketchy, but he suggests that social media companies and other internet firms will come to adopt a new set of technical protocols and operating constraints that effectively destroy the data-collection, customer-profiling, and behavior-prediction systems at the core of their business strategies. They will put Berners-Lee’s ideals ahead of their own profits.

Inventors are rarely able to see their inventions objectively, just as parents are rarely able to see their children objectively, but Berners-Lee seems particularly impaired. His focus on protocols and codes may be understandable for a programmer, but it has blinded him to a deep and uncomfortable truth about his invention. The web wasn’t corrupted by outside forces. The corruption was there from the start, latent in its design. A vast, decentralized communication network that can transmit data of all sorts to all people is not resistant to the establishment of information monopolies; it encourages their formation. Decentralization at a technical level breeds centralization at an industrial level. What Cory Doctorow calls “enshittification” is not a bug but a feature.

The reasons are several:

1. The web is subject to particularly strong network effects. Because a communication system becomes more valuable to users as the number of users increases, a boundaryless network with few physical or technical constraints on its expansion will consolidate traffic on a massive scale, giving strong advantages to the biggest players.

2. The web is a marketplace where an unimaginable number of transactions, both financial and social, are completed every second without regard to the physical location of the participants. That favors large intermediaries, or middlemen, that have the infrastructure necessary to host myriad market participants, execute transactions quickly and precisely, and maintain detailed records of all that transpires.

3. Operating at such scale requires large capital investments—for servers, storage drives, networking gear, cooling systems, and the like. The capital requirements present daunting barriers to entry for newcomers, barriers that are growing more forbidding as resource-intensive AI routines are incorporated into online processes and services.

4. The interpersonal links that Berners-Lee rightfully celebrates for their intellectual and social value also have outsized financial value when captured as data and analyzed by computers. The network effect applies not just to people but to information about people. The more aggregated the information, the bigger an asset it becomes for companies that profit by predicting attitudes and behavior.

5. Consumers benefit from the companies’ scale. Whatever fears people may have about privacy invasions or wealth concentration, they enjoy the personalization, convenience, and diversion that social media companies and other internet outfits serve up in endless quantities for free. People’s loyalty to algorithmically-tuned feeds may be a form of addiction, but it doesn’t seem to be an addiction many are eager to break.

Berners-Lee may be loathe to admit it, but the web he designed rewards both massive scale and centralized control.

He barely mentions the only practicable way to curb the size and power of the internet giants: breaking them up through aggressive antitrust litigation. Such a course seems unlikely for the moment, as Congress and the courts have for years taken a passive approach to antitrust enforcement, leaving control of monopolies to the marketplace. Late last year, a federal judge threw out a long-running, much-watched Federal Trade Commission suit seeking to split up Meta, ruling that the company’s social-media platforms face sufficient competition for people’s attention from other popular platforms like YouTube and TikTok. But, as the legal scholar Tim Wu explains in his recent book, The Age of Extraction, governmental antitrust philosophies tend to shift with public opinion. The robber barons of the Gilded Age were brought to heel by celebrated “trust busters” like Teddy Roosevelt at the turn of the twentieth century. Later in the century, AT&T, IBM, and Microsoft all saw their market power constrained by government action. Should public outrage over the financial and political power of internet companies grow in the years ahead, antitrust suits may become more common and more successful.

But while breaking up companies like Meta and Google could well bring economic and social benefits—Wu makes a compelling case that the unconstrained growth of monopolies undermines democracy—it wouldn’t change the way the web works. An Instagram spun off by Meta would still be Instagram. It would still make money by, to use Wu’s terminology, extracting attention and data from the masses and using that raw material to feed a lucrative advertising business. The companies that rule the web might not be quite as large as they used to be, but they’d still do business in much the same way. And they’d still rule the web.

This Is for Everyone ends with an idyllic anecdote. Berners-Lee and his third wife, the entrepreneur and philanthropist Rosemary Leith, take their fourteen-foot Hobie catamaran out for a sail around a Canadian lake on a fine fall day. Rosemary takes the tiller, while Tim sits on the trampoline between the hulls, manning the mainsheet. He feels a sense of exhilaration as the little boat catches the wind and zips across the lake. “While the physical activity and constant adjustments kept us engaged,” he writes, “there were also moments of pure enjoyment. The feeling of the sun on one’s back, the sound of the water rushing past, the wind in the sails and the sensation of gliding over the waves can create a deep glow of connection with nature.”

Then, abruptly, he breaks the reverie to tell us, for the umpteenth time, why his invention is such a gift to us all:

Even on my sailing boat, in the middle of the water on a sunny afternoon in autumn, I could use my smartphone to contact almost anyone alive. I could deliver a package to my doorstep, or a hot lunch to my office. I could buy Rosemary flowers. I could listen to any song ever recorded and watch old clips from Fawlty Towers. I could do my banking or trade my portfolio or donate to UNICEF. I could read the news in any language and check the weather in any country. I could book a flight, a hotel room and a car in nearly any city in the world. All I needed was a web browser and phone reception.

So much for that connection with nature.

Not all technologies improve people’s lives. Just as Berners-Lee’s now omnipresent web shapes industries and markets, it shapes its users’ thoughts, perceptions, and relationships. As we’re slowly coming to understand, human beings did not evolve to be virtual creatures in a computer-generated world. The internet operates at a scale and speed that conflict with the brain’s deliberate pace of thought, the intellect’s slow accumulation of knowledge, and the psyche’s limited capacity for stimulation and social exchange. To be able to do anything and be anywhere at any moment seems liberating for a while, but it ends in a blurred and chaotic existence, the physical world’s familiar, steadying divisions of space and time dissolving in endless torrents of data. It’s an existence that may be vivifying to certain software programmers—Tim, slow down!—but for the rest of us, the virtual world’s hyperkinetic superabundance ends up feeling like emptiness, a very, very busy void. We may be drawn to that void by our native attraction to information, novelty, and spectacle, but we’ll never make a home there.

It’s hard not to admire Berners-Lee’s ability to sustain his idealism about the web in the face of so many disappointments. His optimism, with its childlike resiliency, is winning. But we should be wary of it. The qualities of the web he celebrates unquestioningly are the very qualities we should be questioning. What he can’t imagine, or at least won’t allow himself to imagine, is the possibility that his gift to humanity is more bane than boon.

This review of Tim Berners-Lee’s This Is for Everyone: The Unfinished Story of the World Wide Web appeared originally, in a slightly different form, in the Los Angeles Review of Books.

The twentieth-century philosophers Hannah Arendt and Günther Anders were married for eight years, from 1929 until 1937, and I would like to suggest that by marrying Arendt’s idea of homo faber with Anders’s idea of Promethean shame we can uncover a hidden aspect of generative artificial intelligence that is fated to cast a tragic shadow over us and our work from here on out. In creating what might best be called machina faber, we have brought into the world an instrument that transcends its instrumentality. The tool becomes a maker itself, a machine that, whatever its current and future shortcomings, intrudes on what Arendt saw as an essence of the human: our ability to fashion a world for ourselves.

The original homo faber was the artisan, the man who created with his hands objects that were useful but not merely useful. They were much more than what we would today call “consumer goods” or “disposables.” Crafted with care and skill, homo faber’s artifacts, whether they took the form of dinner tables or cathedrals, were solid and durable. Withstanding time, they outlasted their makers and their first users. In the aggregate, Arendt argued in The Human Condition, they formed the foundations of civilization, giving human society a continuity through the passing of generations. Every individual is mortal, but the “human artifice” endures.

The man-made world of things, the human artifice erected by homo faber, becomes a home for mortal men, whose stability will endure and outlast the ever-changing movement of their lives and actions, only insomuch as it transcends both the sheer functionalism of things produced for consumption and the sheer utility of objects produced for use.

But homo faber, Arendt stressed, is the artist as well as the artisan. The words of the poet, the songs of the musician, the images of the painter and the photographer, the figures of the sculptor: they all carry the stories of humans and humankind through time. They provide the continuity not of physical artifice but of intellectual and aesthetic artifice—of history and government, of knowledge, wit, beauty.

If mortals need his help to erect a home on earth, acting and speaking men need the help of the homo faber in his highest capacity, that is, the help of the artist, of poets and historiographers, of monument-builders or writers, because without them the only product of their activity, the story they enact and tell, would not survive at all.

Homo faber took pride in his work—his skill gave him the power to turn the inanimate stuff of nature to human purposes—and the pride was shared by all of humankind. Günther Anders termed it “Promethean pride,” after the Greek god who created man out of mud. But with the Industrial Revolution, in Anders’s telling, the story took a dark turn. As homo faber came to rely on ever more complex machinery to manufacture goods, he grew alienated from the products he produced. They were no longer the products of his hands; they emerged from a mechanical process in which he played only a small part. The buyers of the goods felt a similar alienation. They could no longer see in the products any human origin. They could no longer construct a home out of them.

We find ourselves today, Anders wrote in The Obsolescence of the Human,1 surrounded by products that don’t appear to us as having been produced by us. We have handed off so much of their design and production to industrial technologies that we can no longer take a shared pride in their invention and manufacture. Indeed, they have come to project an otherness that seems not just separate from us but superior to us. They mock us as outdated masses of meat and bone. Today’s products, Anders observed,

are simply “there.” We encounter them primarily as necessary, desirable, superfluous, affordable, or unaffordable consumer goods that become “mine” only after I have bought them. As such, they are much more likely to be proof of one’s own insufficiency than evidence of one’s power.

This sense of insufficiency transformed “Promethean pride” into “Promethean shame”—the shame contemporary man feels at having been born instead of made, of being a product of natural processes rather than technological ones. “He despises himself,” wrote Anders, “in the same way that things would despise him if they could.” The shame, he went on, becomes particularly sharp when a person first sees a so-called thinking machine:

As for the man who is for the first time confronted with a working computing machine, self-aggrandizement and pride are even more alien to him. An observer who erupts with the exclamation, “My goodness, aren’t we great guys, to be capable of this!” when encountering such a machine is a clown, a figment of the imagination. Quite the contrary! He rather murmurs with a shake of his head, “My god, it’s incredible what it—the machine—can do!” At the same time, he feels highly ill at ease in his creaturely skin, for the machine half gives him the creeps and half puts him to shame.

Though it was written seventy-five years ago, that last sentence strikes me as one of the more perceptive descriptions of man’s confrontation with generative AI: “he feels highly ill at ease in his creaturely skin, for the machine half gives him the creeps and half puts him to shame.”

With the arrival of machina faber in the form of AI, faber (the maker) no longer feels like the right term to apply to us humans. I would suggest that fictor (the fabricator) is now the better descriptor. In Latin, fictor, like faber, denotes a maker (though one who works in malleable materials like clay rather than in solid ones like wood or metal), but the word, like the English word fabricator, also carries a connotation of fakery or deception. It derives from the verb fingere, which means both to fashion and to feign. Homo fictor is the maker who may be lying about what he makes. His works, however useful, however elegant, will always be suspect. They will always carry a hint of fraud. Did he make them, or did AI?

Machina faber steals from makers their pride in accomplishment. If you use AI to “write” something, or to “code” something, or to “compose” something, or to “design” something, or to “invent” something, then any true sense of accomplishment will be withheld from you. You will always know that you’re a fraud. You will always be ashamed of yourself, even if you’re in denial about your shame. As Anders wrote, “Anyone who denies the existence of this form of shame does so because the admission that we’ve come such a gloriously long way only to now feel shame in front of things would itself make them blush with shame.”

But—and here’s the tragic part—the shame is not restricted to those artisans and artists who use AI in their work. The shame shadows everyone, even those who abstain from using AI, even those who take pride in their public rejection of AI. In a recent New York Times op-ed about the controversy surrounding Shy Girl, the novel cancelled by its publisher because its prose was felt to carry an odor of AI, the novelist Andrea Bartz writes:

When readers ask questions about my thriller novels, I love to discuss the themes and characters in them and the inspiration for my writing. But as generative artificial intelligence worms its way through the publishing industry, I’m bracing for a stomach-turning query: Did you actually write this?

Fair or not, that’s the question that is now inescapable for all makers. Even if you don’t traffic in AI slop, you’re still subject to AI smog. Its odor is everywhere, as is the shame it carries. No one is beyond suspicion. Nothing is pure.

This post is an installment in Dead Speech, the New Cartographies series on AI and its cultural and economic consequences.

“I feel like there’s a red pill and a blue pill, and you can take the blue pill and go back to your classroom and lecture your 20 students. But I’ve taken the red pill, and I’ve seen Wonderland.” –Sebastian Thrun, 2012

Now that we’ve begun to talk of MOOCs retrospectively, the time has come to update my previously published survey of the history of hype and wishful thinking that has for more than a century surrounded technologies for automating education by replacing teachers and classrooms with machines and media. I am adding a new entry to the list. I suspect it won’t be the last addition.

Mail: Around 1885, Yale professor William Rainey Harper, a pioneer of teaching-by-post, said, “The student who has prepared a certain number of lessons in the correspondence school knows more of the subject treated in those lessons, and knows it better, than the student who has covered the same ground in the classroom.” Soon, he predicted, “the work done by correspondence will be greater in amount than that done in the class-rooms of our academies and colleges.”

Phonograph: In an 1878 article on “practical uses of the phonograph,” the New York Times predicted that the phonograph would be used “in the school-room in training children to read properly without the personal attention of the teacher; in teaching them to spell correctly, and in conveying any lesson to be acquired by study and memory. In short, a school may almost be conducted by machinery.”

Movies: “It is possible to teach every branch of human knowledge with the motion picture,” proclaimed Thomas Edison in 1913. “Our school system will be completely changed in 10 years.”

Radio: In 1927, the University of Iowa declared that “it is no imaginary dream to picture the school of tomorrow as an entirely different institution from that of today, because of the use of radio in teaching.”

TV: “During the 1950s and 1960s,” report education scholars Marvin Van Kekerix and James Andrews, “broadcast television was widely heralded as the technology that would revolutionize education.” In 1963, an official with the National University Extension Association wrote that television provided an “open door” to transfer “vigorous and vital learning” from campuses to homes.

Computers: “There won’t be schools in the future,” wrote MIT’s Seymour Papert in 1984. “I think the computer will blow up the school. That is, the school defined as something where there are classes, teachers running exams, people structured into groups by age, following a curriculum — all of that.”

World Wide Web: The arrival of the web brought the e-learning fad of the late 1990s, as universities and corporations rushed to invest in online courses. In 1999, Cisco CEO John Chambers told the Times‘s Thomas Friedman, “The next big killer application for the Internet is going to be education. Education over the Internet is going to be so big, it’s going to make e-mail usage look like a rounding error.”

MOOCs: The New York Times declared 2012 “the year of the MOOC.” “Welcome to the college education revolution,” wrote the ever-hopeful Friedman in a column heralding massive open online courses. “In five years this will be a huge industry.” The MOOC “is transforming higher education,” declared the Economist, “threatening doom for the laggard and mediocre.” Academics were equally bedazzled. “There’s a tsunami coming,” said Stanford president John Hennessy. Opined MIT president Rafael Reif: “I am convinced that digital learning is the most important innovation in education since the printing press.” Harvard’s Clayton Christensen predicted “wholesale bankruptcies” among traditional universities.

All of these mediums and devices played useful roles in education and training—which is something worth celebrating—but none of them turned out to be revolutionary or transformative. There may be a deeper lesson here, a lesson about how easy it is to overlook the intangible virtues not just of classrooms and teachers but of presence, of bringing students together in one place at one time.

“After the first minute of content you will have what we call minutes 1 thru 3.”
—MrBeast, “How to Succeed in MrBeast Production”

A couple of years ago, I watched a MrBeast YouTube video on my phone. References to MrBeast were everywhere at the time—he had recently become the most popular YouTuber ever—and I felt that, to be culturally current, I should acquaint myself with his oeuvre.

My memory of the video is fuzzy, but the premise went something like this: Two strangers, a nondescript young woman and a nondescript young man, get locked together in a big windowless room. If they’re able to stay put for a hundred days, they’ll win a substantial amount of money—a million dollars, I think it was. I remember that, to emphasize the stakes, MrBeast (also a nondescript young man, though with a slightly odd affect) wheeled in a pallet on which was piled a million dollars in bundles of bills. What impressed me most about the show was its length. It seemed to go on forever, as if the hundred days were unfolding in real time. When it ended, I thought I should be given the million dollars.

MrBeast has been in the news again, as the new season of his popular Amazon Prime extravaganza, Beast Games, debuted earlier this year. The New York Times ran two lengthy articles about him in December. He has, according to one of the articles, “explod[ed] the media industry’s understanding of the term ‘creator.’” I have no idea what that means, but it sounds like MrBeast must be doing something important. The Wall Street Journal seems to think so, too. In January, it ran a long, largely glowing profile of him, noting that he is known as “the man who cracked the secret of going viral.”

What exactly is that secret? What is it that MrBeast, as a “creator,” is “creating”? The obvious answer would be “content.” Content, after all, is what we talk about when we talk about digital media. It’s certainly what MrBeast talks about. But the more I think about online programming, the more I’m convinced that content isn’t what matters. What matters is form. Digital media aspires to, and often achieves, a state of contentlessness. Spend some time on MrBeast’s channel, or scroll down your Instagram feed or your X feed or your Apple News feed, or swipe through your For You page on TikTok. What you’re seeing is the repetition of a pattern, a pattern that has been statistically determined to have the highest odds of holding your attention. What fills the pattern at any given instant—what we call content—is fungible and disposable. It’s not important. It’s the pattern, the form the content fits and replicates, that’s important.

Popular culture has always been formulaic. Every hit pop song, or situation comedy, or detective novel spawns scores of copies. A lot of crap results. But good stuff does, too, sometimes. When the Rolling Stones were pressured by their producer and record company to come up with a quick followup to their 1964 smash “(I Can’t Get No) Satisfaction,” they delivered a knockoff: “Get Off My Cloud.” The knockoff, if not quite in the same league as the original, was itself a great single. The Stones were savvy and skilled enough to make the knockoff fun and memorable in its own right, and by including in the lyrics a sarcastic swipe at advertising and consumerism — “he says I’ve won five pounds if I have his kind of detergent pack” — they gave a wink to their fans. The Stones knew the fans knew and the fans knew the Stones knew what the Stones were up to. The knockoff ended up being better for being a knockoff.

Such welcome things happen when talented people work in an established form, even a very narrow one, even one shaped by commercial interests. You get more than the reproduction of a pattern. You get something that’s both familiar and new. To go back a bit further in time, think of the sonnet. I don’t know who first wrote a poem in the fourteen-line form that came to be known as the sonnet—it was popularized by Petrarch in the fourteenth century—but for ages thereafter poets found ways to make that apparently rigid form their own. (Sonnets, wrote John Donne, are “pretty rooms” that you furnish to your own taste.) A good artist works both within and against a form. Art emerges from a struggle between what T. S. Eliot termed “tradition” (i.e., established forms) and “individual talent.”

Something very different happens when machines begin to establish and fill the patterns. The creative tension between form and individual talent gets resolved in form’s favor. Thanks to the algorithmic tracking of demand and the algorithmic delivery of supply, digital media has for the last twenty years promoted an ideal of perfect form—not a Platonic ideal but a consumerist one. Machines evaluate form microscopically, precisely measuring the effects of tiny variations in pixels or words or musical notes in order to determine the optimal pattern for each consumer. They then construct personalized feeds by repeatedly populating the optimal pattern with new stuff.

The speed of computers and computer networks makes the repetition of patterns instantaneous. Because the supply of messages is essentially unlimited—billions of people are churning new ones out every moment, and many of those people are themselves “creators” who are highly conscious of optimal patterns and aim to match them—the feed algorithm can replicate the form over and over again with no delay and no end. For both algorithm and creator, precision in form-matching takes precedence over individual inspiration and creativity. It didn’t matter which MrBeast production I watched, just as it doesn’t matter what the next item in my Instagram feed is. They’re all the same.

It’s easy to understand why automated media would concentrate on replicating forms. Evaluating works of art, or any creative products of skill and imagination, requires a mind, which the machines of media automation lack. Identifying a pattern requires only a statistical procedure, which is what the machines have. Indeed, it’s all they have. People like MrBeast didn’t crack the code of virality. Machines did. MrBeast’s great strength as a contemporary creator is that he has no ambition beyond repeating a pattern. He’s a machine-listener. He attends to the machine, and he does what it tells him to do. Here’s how he puts it in a recently leaked staff memo, “How to Succeed in MrBeast Production”:

I spent basically 5 years of my life locked in a room studying virality on Youtube. Some days me and some other nerds would spend 20 hours straight studying the most minor thing: like is there a correlation between better lighting at the start of the video and less viewer drop off (there is, have good lighting at the start of the video haha) or other tiny things like that. And the result of those probably 20,000 to 30,000 hours of studying is I’d say I have a good grasp on what makes Youtube videos do well. The three metrics you guys need to care about is Click Thru Rate (CTR), Average View Duration (AVD), and Average View Percentage (AVP).

And yet, however mechanical he may be, MrBeast is still a human being and as such will always fall short of machine levels of efficiency in pattern repetition. To the machines, or at least to their owners and operators, human creators have always been necessary evils, inefficient cogs needed only because the machines were incapable of generating content out of their own resources. The companies needed machine-listeners (the creators) to follow the machine’s instructions (the metrics) as slavishly as possible in order to produce chunks of content to feed back into the machine for distribution.1

With AI, at last, the machines can take over the creator’s role. AI-generated slop marks the triumph of machine formalism. The machine establishes the pattern, and the machine fills the pattern with its own creation. The automated media system is relieved of human inefficiency, not to mention human sensibility. The same thing happens in the automation of factories and warehouses. People are kept on hand to perform tasks that robots aren’t good at doing—boxing up orders, say, or feeding parts into the machine—until the robots get good at doing them.

In automated systems, human beings are placeholders for future machines. Until recently, we assumed that creative types who produce content for media systems were exceptions to that rule. We’re now going to test that assumption. Is MrBeast necessary? Am I?

One of man’s greatest inventions was also one of his most modest: the wick. We don’t know who first realized, many thousands of years ago, that fire could be isolated at the tip of a twisted piece of cloth and steadily fed, through capillary action, by a reservoir of wax or oil, but the discovery was, as Wolfgang Schivelbusch writes in Disenchanted Night, “as revolutionary in the development of artificial lighting as the wheel in the history of transport.” The wick tamed fire, allowing it to be used with a precision and an efficiency far beyond what was possible with a wooden torch or a bundle of twigs. In the process, it helped domesticate us as well. It’s hard to imagine civilization progressing to where it is today by torchlight.

The wick also proved an amazingly hardy creation. It remained the dominant lighting technology all the way to the nineteenth century, when it was replaced first by the wickless gas lamp and then, more decisively, by Edison’s electricity-fueled incandescent bulb with its glowing metal filament. Cleaner, safer, and even more efficient than the flame it replaced, the light bulb was welcomed into homes and offices around the world. But along with its many practical benefits, electric light also brought subtle and unexpected changes to the way people lived. The fireplace, the candle, and the oil lamp had always been focal points of households. Fire was, as Schivelbusch puts it, “the soul of the house.” Families would in the evening gather in a central room, drawn by the flickering flame, to chat about the day’s events or otherwise pass time together. Electric light, together with central heat, dissolved that long tradition. Family members began to spend more time in different rooms in the evening, studying or reading or working alone. Each person gained more privacy, and a greater sense of autonomy, but the cohesion of the family weakened.

Cold and steady, electric light lacked the allure of the flame. It was not mesmerizing or soothing but strictly functional. It turned light into an industrial commodity. A German diarist in 1944, forced to use candles instead of lightbulbs during nightly air raids, was struck by the difference. “We have noticed,” he wrote, “in the ‘weaker’ light of the candle, objects have a different, a much more marked profile — it gives them a quality of ‘reality.’” This quality, he continued, “is lost in electric light: objects (seemingly) appear much more clearly, but in reality it flattens them. Electric light imparts too much brightness and thus things lose body, outline, substance — in short, their essence.”

We’re still attracted to a flame at the end of a wick. We light candles to set a romantic or a calming mood, to mark a special occasion. We buy ornamental lamps that are crafted to look like candles or candleholders, with bulbs shaped as stylized flames. But we can no longer know what it was like when fire was the source of all light. The number of people who remember life before the arrival of Edison’s bulb has dwindled to just a few, and when they go they’ll take with them all remaining memory of that earlier, pre-electric world. The same will happen, sometime toward the end of this century, with the memory of the world that existed before the computer and the Internet became commonplace. We’ll be the ones who bear it away.

All technological change is generational change. The full power and consequence of a new technology are unleashed only when those who have grown up with it become adults and begin to push their outdated parents to the margins. As the older generations die, they take with them their knowledge of what was lost when the new technology arrived. Only the sense of what was gained remains. It’s in this way that progress covers its tracks, perpetually refreshing the illusion that where we are is where we were meant to be.

In a paper published in 2003, the philosopher Nick Bostrom sketched out a thought experiment aimed at illustrating an existential risk that artificial intelligence might eventually pose to humanity. An advanced AI is given, by its human programmers, the objective of optimizing the production of paperclips. The machine sets off in monomaniacal pursuit of the objective, its actions untempered by common sense or ethical sense. The result, Bostrom wrote, is “a superintelligence whose top goal is the manufacturing of paperclips, with the consequence that it starts transforming first all of earth and then increasing portions of space into paperclip manufacturing facilities.” It destroys everything, including its programmers, in a mad rush to gather resources for paperclip production.

Bostrom went on to refine his “paperclip maximizer” thought experiment in subsequent writings and interviews, and it soon became a touchstone in debates about AI. Eminences as diverse as Stephen Hawking and Elon Musk would routinely bring it up in discussing the dangers of artificial intelligence. Others were skeptical. They found the story far-fetched, even by thought-experiment standards. It seemed, as The Economist wrote, a little too “silly” to be taken seriously.

I was long in the skeptic camp, but recently I’ve had a change of heart. Bostrom’s story, I would argue, becomes compelling when viewed not as a thought experiment but as a fable. It’s not really about AIs making paperclips. It’s about people making AIs. Look around. Are we not madly harvesting the world’s resources in a monomaniacal attempt to optimize artificial intelligence? Are we not trapped in an “AI maximizer” scenario?

“The intelligence of an AI model roughly equals the log of the resources used to train and run it,” OpenAI CEO Sam Altman wrote a year ago. The important word here is “log.” As Donald MacKenzie explains in an insightful article on AI in the London Review of Books:

A logarithmic function, at least of the kind that is relevant here, is characterised by diminishing returns. The more resources you put in, the better the results, but the rate of improvement steadily diminishes.

To maintain a linear path of improvement in the performance of today’s neural-network-based AI models requires an exponential increase in resources. Ever larger inputs achieve ever smaller gains. But people like Altman remain absolutely committed to making those escalating resource investments, no matter the monetary or social or environmental cost. Because they believe that vast winner-take-all rewards will come to any company achieving superior scale in AI, they will devote all available resources—energy, water, real estate, data, chips, people—to the pursuit of even a tiny scale advantage.

Elon Musk, having abandoned his earlier misgivings about AI, announced last week that he was merging xAI into SpaceX. The combined companies were “scaling to make a sentient sun to understand the Universe and extend the light of consciousness to the stars!” he declared. “In the long term, space-based AI is obviously the only way to scale.” It’s exactly what Bostrom predicted. The monomaniacs will not stop with the resources of the Earth. They’ll extend their plundering to the heavens. Everything is raw material.

This post is an installment in Dead Speech, the New Cartographies series on AI and its cultural and economic consequences.

Big social media companies are facing hundreds of personal-injury lawsuits claiming that their platforms have harmed people, particularly kids. Lawyers for the plaintiffs, which include individuals, states, and school districts, are modeling the suits on the successful litigation against cigarette companies at the end of the last century. Should the social media companies lose the suits, the first of which began this week in Los Angeles, they would face not just massive payouts but also the prospect of extensive new regulatory controls on their businesses, just as tobacco companies did.

The internet giants have armies of lawyers, and they’re spending millions to block the suits. They claim, as they always have in the past, that they’re shielded from such litigation by the 1996 Communications Decency Act. As the Wall Street Journal writes, in an editorial sympathetic to the companies, “The first problem with these cases is that Section 230 of the 1996 Communications Decency Act says internet platforms can’t be held liable for user-generated content.”1 But that old argument no longer holds water. The content produced by social media companies today is anything but “user-generated.” To think otherwise is to misunderstand how social media operates —and to misinterpret the scope of Section 230.

In 1996, when Congress passed the Communications Decency Act,2 the big internet companies were internet service providers, or ISPs. Their role was limited to providing customers with access to the net, through, usually, dial-up connections over telephone lines. The ISPs acted as common carriers, their role limited to the transmission of information that was created by others — a role similar to that of traditional telephone companies or even the post office. Just as it would have been unfair to hold a mailman liable for the content of the letters he delivered to people’s mailboxes, so it would have been unfair to hold ISPs liable for the content of the emails and web pages they delivered to people’s computers. Section 230 provides internet carriers with a safe harbor from litigation so long as they restrict themselves to transporting data and do not act as “publisher or speaker” of the content they deliver:

No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.

Back in the early days of social media, it could be argued that Section 230 still applied. When Facebook started up in 2004, for instance, it provided its users with templates for inputting and organizing personal profiles and messages, but its main role was to connect people through an online network so they could share the content they created. The users were the speakers and the publishers of the content. Facebook was the carrier of the content.

That all changed in 2006 when Facebook introduced its News Feed. The users no longer controlled what they saw when they logged on to the network; they now saw a “feed” of information that was controlled by the algorithms Facebook wrote. The company was not just a carrier of content. It had taken on an explicitly editorial role. Like the editors at newspapers or the producers at TV networks, it selected and arranged the information that its users saw. The users had become an audience for Facebook’s production.

The story of social media ever since has been a story of the refinement of feeds as a media product aimed at capturing and holding an audience. The platforms have invested billions of dollars in designing those feeds—what they contain, how they look, how they work—to make them as “engaging” as possible. To argue that the companies are still in the business of transmitting “user-generated content” is absurd. Saying that a social-media feed is the product of users is like saying that a hot dog is the product of cattle.

The companies are not common carriers anymore; they’re media businesses. Yes, users still contribute posts and comments—though even those, in today’s era of influencers, creators, and AI, are often subsidized and actively shaped by the companies—but the essential content of social media is now the feeds produced by the platforms, not the individual messages posted by users. Go to Instagram and scroll through your feed. It’s obvious that what you’re experiencing is not discrete bits of user-generated content. It’s an elaborate, finely tuned media production manufactured by Instagram for an audience of one: you. The same goes for YouTube, X, TikTok, Facebook, Snapchat, Substack Notes, and, with a few exceptions, all the rest.

The feed is the content, and the social media company is its publisher. Period.

The question of whether social media companies should be held liable for harming people is a legally complex one, which would best be answered through courts of law. And that’s what should happen. Let the plaintiffs make their case, and let the defendants defend themselves. Section 230’s safe harbor doesn’t apply. Social media companies are, like other media companies, in the content-production business, and they’re responsible for their programming.

If the stakes weren’t so high, the barrage of text messages between world leaders in the days running up to the World Economic Forum in Davos would be amusing. Texting turns everyone into a semiliterate twelve-year-old, and presidents, prime ministers, and secretaries general are no exception. We’re used to the President of the United States communicating with the American public in weirdly punctuated streams of all-caps, exclamation marks, and typos, but to know that texting has now become the de facto language of diplomacy is a revelation, and a worrying one.

Whether it’s a handwritten letter, a telephone call, a fax, an email, or a text, the medium through which people communicate shapes what they say and how they say it. Some mediums encourage formal speech while others encourage casual banter. Some are suited to full sentences and well-turned paragraphs; others to sentence fragments and cliches. Some promote consideration; others, abruptness. In general, a medium’s speed of delivery is inversely correlated to the thoughtfulness and nuance of the messages it carries. The growing hegemony of the instant message, it seems fair to say, is not fostering eloquence in either private correspondence or public speaking. Texts are great for quick, offhand exchanges. They debase pretty much everything else.

Because texting is resistant to the expression of complicated or subtle ideas or arguments — it is, by design, a medium of speed, compression, and simplicity — it’s particularly ill-suited to grappling with complex issues or solving complex problems. To use it for international relations and other aspects of governmental policy making, particularly in fraught situations, is a sure route to misunderstanding, anger, and the escalation of tensions.

The most telling precedent for what we’re seeing today is the change in diplomatic practices that occurred with the arrival of international telegraph and telephone lines in the late nineteenth century — an episode I describe in my book Superbloom. The unprecedented ability of far-flung leaders and diplomats to talk directly with each other without delay spurred great hopes. It seemed obvious that the resulting exchanges would ease friction and encourage goodwill among nations. Nikola Tesla, in an 1898 interview about his work on wireless telegraph systems, said that he would be “remembered as the inventor who succeeded in abolishing war.” His rival, Guglielmo Marconi, declared in 1912 that wireless telegraphy would “make war impossible.”

What actually happened was altogether different. In the lead-up to the Franco-Prussian War of 1870, telegraphic communications inflamed tensions rather than dampening them. Writes the French historian Pierre Granet: “The constant transmission of dispatches between governments and their agents, the rapid dissemination of controversial information among an already agitated public, hastened, if it did not actually provoke, the outbreak of hostilities.”

The start of the First World War in 1914, two years after Marconi announced the end of war, was similarly hastened by the new communication mediums. After the June 28 assassination of Austrian Archduke Franz Ferdinand in Sarajevo, hundreds of urgent diplomatic messages raced between European capitals through newly strung telegraph and telephone wires. As the historian Stephen Kern describes in The Culture of Time and Space 1880–1918, the rapid-fire dispatches quickly devolved into ultimatums and threats. “Communication technology imparted a breakneck speed to the usually slow pace of traditional diplomacy and seemed to obviate personal diplomacy,” Kern writes. “Diplomats could not cope with the volume and speed of electronic communication.”

Diplomacy, a communicative art, had been overwhelmed by communication. By August, the world was at war. “The moral qualities—prudence, foresight, intelligence, penetration, wisdom—of statesmen and nations have not kept pace [with the] rapidity of communication by telegraph and telephone,” the distinguished British diplomat Ernest Satow wrote in his 1917 Guide to Diplomatic Practice. “These latter leave no time for reflection or consultation, and demand an immediate and often a hasty decision on matters of vital importance.”

His words are as resonant now as they were a century ago, and they should give today’s leaders and diplomats pause. Successful statecraft requires deliberation, discretion, and discernment, qualities rarely evident in messages thumbed out through apps on phone screens.

Mines and Factories

Am I a data mine, or am I a data factory? Is data extracted from me, or is data produced by me? Both metaphors are repellent, but the distinction between them is important. The metaphor we choose informs our sense of the power wielded by so-called platform companies like Facebook, Google, and Amazon, and it shapes the way we, as individuals and as a society, respond to that power.

If I am a data mine, then I am essentially a chunk of real estate, and control over my data becomes a matter of ownership. Who owns me (as a site of valuable data), and what happens to the economic value of the data extracted from me? Should I be my own owner — the sole proprietor of my data mine and its wealth? Should I be nationalized, my little mine becoming part of some sort of public collective? Or should ownership rights be transferred to a set of corporations that can efficiently aggregate the raw material from my mine and everyone else’s and transform it into useful products and services? The questions raised here are questions of economics and politics.

The mining metaphor, like the mining business, is a fairly simple one, and it has become popular, particularly among writers of the left. Thinking of the platform companies as being in the extraction business, with personal data being analogous to a natural resource like iron or petroleum, brings a neatness and clarity to discussions of a new and complicated type of company. In an article in the Guardian in March, Ben Tarnoff wrote that “thinking of data as a resource like oil helps illuminate not only how it functions, but how we might organize it differently.” Building on the metaphor, he went on the argue that the data business should not just be heavily regulated, as extractive industries tend to be, but that “data resources” should be nationalized — put under state ownership and control:

Data is no less a form of common property than oil or soil or copper. We make data together, and we make it meaningful together, but its value is currently captured by the companies that own it. We find ourselves in the position of a colonized country, our resources extracted to fill faraway pockets. Wealth that belongs to the many — wealth that could help feed, educate, house and heal people — is used to enrich the few. The solution is to take up the template of resource nationalism, and nationalize our data reserves.

In another Guardian piece, published a couple of weeks later, Evgeny Morozov offered a similar proposal concerning what he termed “the data wells inside ourselves”:

We can use the recent data controversies to articulate a truly decentralised, emancipatory politics, whereby the institutions of the state (from the national to the municipal level) will be deployed to recognise, create, and foster the creation of social rights to data. These institutions will organise various data sets into pools with differentiated access conditions. They will also ensure that those with good ideas that have little commercial viability but promise major social impact would receive venture funding and realise those ideas on top of those data pools.

The simplicity of the mining metaphor is its strength but also its weakness. The extraction metaphor doesn’t capture enough of what companies like Facebook and Google do, and in adopting it we too quickly narrow the discussion of our possible responses to their power. Data does not lie passively within me, like a seam of ore or a pool of oil, waiting to be extracted. Rather, I actively produce data. When I drive or walk from one place to another, I produce locational data. When I buy something, I produce purchase data. When I text with someone, I produce affiliation data. When I read or watch or purchase something online, I produce preference data. When I upload a photo, I produce not only behavioral data but data that is itself a product. I am, in other words, much more like a data factory than a data mine. I manufacture data through my labor — the labor of my mind, the labor of my body.

The platform companies, in turn, act more like factory managers than like the owners of oil wells or copper mines. Beyond controlling my data, the companies seek to control my actions, which to them are essentially manufacturing processes, in order to optimize my data production (and, on the demand side of the platform, my data consumption). They want to script and regulate the work of my factory — i.e., my life — as Frederick Winslow Taylor sought to script and regulate the labor of factory workers at the turn of the last century. The control wielded by these companies, in other words, is not just that of ownership but also that of command. And they exercise such command through the design of their apps and other software, which increasingly regulate everything we do during our waking hours. Apps are, like factory routines and industrial machinery, behavioral modification tools. They’re designed to maximize people’s efficiency in producing valuable data.

The factory metaphor makes clear what the mining metaphor obscures: We work for the Facebooks and Googles of the world, and the work we do is increasingly indistinguishable from the lives we lead. The questions we need to grapple with are political and economic, to be sure. But they are also ethical and philosophical. The extraction metaphor suggests we lack personal responsibility and agency. The factory metaphor emphasizes our responsibility and agency. We are actors, not mere resource pools.

A False Choice

To understand why the choice of metaphor is so important, consider a new essay by Ben Tarnoff, written with Moira Weigel, that was published last week. The piece opens with a sharp, cold-eyed examination of those Silicon Valley apostates who now express regret over the harmful effects of the software they created. Through their new stress on redesigning the software to promote personal “well-being,” these “tech humanists,” Tarnoff and Weigel argue, actually serve the business interests of the platform companies they criticize. The companies, the writers point out, can easily co-opt the rhetoric of well-being, using it as cover to deflect criticism while seizing even more economic power.

Tarnoff and Weigel point to Facebook CEO Mark Zuckerberg’s recent announcement that his company will place less emphasis on increasing the total amount of time members spend on Facebook and more emphasis on ensuring that their Facebook time is “time well spent.” What may sound like a selfless act of philanthropy is in reality the product of a hard-headed business calculation:

Emphasising time well spent means creating a Facebook that prioritises data-rich personal interactions that Facebook can use to make a more engaging platform. Rather than spending a lot of time doing things that Facebook doesn’t find valuable – such as watching viral videos – you can spend a bit less time, but spend it doing things that Facebook does find valuable. In other words, “time well spent” means Facebook can monetise more efficiently. It can prioritise the intensity of data extraction over its extensiveness. This is a wise business move, disguised as a concession to critics. Shifting to this model not only sidesteps concerns about tech addiction – it also acknowledges certain basic limits to Facebook’s current growth model. There are only so many hours in the day. Facebook can’t keep prioritising total time spent – it has to extract more value from less time.

The analysis is a trenchant one. The vagueness and self-absorption that often characterize discussions of “wellness,” particularly those emanating from the California coast, are well suited to the construction of window dressing. And, Lord knows, Zuckerberg and his ilk are experts at window dressing. But, having offered good reasons to be skeptical about Silicon Valley’s brand of tech humanism, Tarnoff and Weigel overreach. They argue that any humanist critique of the personal effects of technology design and use is a distraction from the “fundamental” critique of the economic and structural basis for Silicon Valley’s dominance:

[The humanists] remain confined to the personal level, aiming to redesign how the individual user interacts with technology rather than tackling the industry’s structural failures. Tech humanism fails to address the root cause of the tech backlash: the fact that a small handful of corporations own our digital lives and strip-mine them for profit. This is a fundamentally political and collective issue. But by framing the problem in terms of health and humanity, and the solution in terms of design, the tech humanists personalise and depoliticise it.

The choice that Tarnoff and Weigel present here — either personal critique or political critique, either a design focus or a structural focus — is a false one. And it stems from the metaphor of extraction, which conceives of data as lying passively within us (beyond the influence of software design) rather than being actively produced by us (under the influence of software design). Arguing that attending to questions of design blinds us to questions of ownership is reductive. Silicon Valley wields its power through both its control of data and its control of design, and that power influences us on both a personal and a collective level. Any robust critique of Silicon Valley needs to address both the personal and the political.

The Silicon Valley apostates are deserving of criticism, but what they’ve done that is useful is to expose, in considerable detail, the way the platform companies fine-tune their software to manipulate people’s behavior—in particular to encourage compulsive use of the software. Their apps are designed to ensure users generate the maximum amount of valuable personal data without ever pausing to think critically about what they’re doing. To put it into industrial terms, these companies are not just engaged in resource extraction; they are engaged in process engineering.

Tarnoff and Weigel go on to suggest that the tech humanists are pursuing a patriarchal agenda. The humanists want to define some ideal state of human well-being and then use software and hardware design to impose that way of being on everybody. That’s altogether true of many of the Silicon Valley apostates. Tarnoff and Weigel quote a prominent one as saying, “We have a moral responsibility to steer people’s thoughts ethically.” It’s hard to imagine a purer distillation of Silicon Valley’s hubris or a clearer expression of its belief that lives should be engineered rather than lived. But Tarnoff and Weigel’s suggestion is off the mark when it comes to the broader humanist tradition in technology theory and criticism. It is the thinkers in that tradition—Mumford, Arendt, Ellul, McLuhan, Postman, Turkle, and many others—who have taught us how deeply and subtly technology is entwined with human history, human society, and human behavior, and how our entanglement with technology can produce effects, often unforeseen and sometimes hidden, that may run counter to our interests, however we choose to define those interests.

Though any cultural criticism will entail the expression of ethical values—that’s what gives it bite—the thrust of the humanist critique of technology is not to impose a particular way of being on all of us but rather to give us the perspective, understanding, and know-how necessary to make our own informed choices about the tools and technologies we use and the way we design and employ them. By helping us to see the power of technology clearly and to resist it when necessary, the humanist tradition expands our personal and social agency rather than constricting it. That’s anything but patriarchal.

Beyond Mine and Factory

Nationalizing collective stores of personal data is an idea worthy of consideration and debate. But it raises a host of hard questions. In shifting ownership and control of exhaustive behavioral data to the government, what kind of abuses do we risk? It seems at least a little disconcerting to see the idea raised at a time when authoritarian movements and regimes are on the rise. If we end up trading a surveillance economy for a surveillance state, we’ve done ourselves no favor.

But let’s assume that our vast data collective is secure, well managed, and put to good, democratic ends. The shift of data ownership from the private to the public sector may well succeed in reducing the economic power of Silicon Valley, but what it would also do is reinforce and indeed institutionalize Silicon Valley’s computationalist ideology, with its foundational, Taylorist belief that, at a personal and collective level, humanity can and should be optimized through better programming. The ethos and incentives of constant surveillance and efficient data production would become even more deeply embedded in our lives, as we, the public, take on the roles of both watched and watcher, data producer and data exploiter.

In addressing the power now centralized in big tech companies, we should keep both the personal and the political consequences of digitization in focus. The question isn’t just about who controls the production and use of personal data. It’s about whether we want to frame our lives in terms of data production and use. Maybe the metaphors of mining and manufacturing are ones we need to escape.

If you’ve done much googling recently, you’ve probably noticed the odd and dubious set of sources that Google’s large language model draws from in generating the “AI Overviews” that now appear at the top of the company’s search results (after the ads, of course). Rather than dig deep into authoritative writings on a subject, Google’s bot usually pieces together its overview from recently published, cursory summaries posted on a hodgepodge of highly trafficked websites — the same dumbed-down, search-engine-optimized sites that have long appeared highly in Google results. Taking the path of least epistemic resistance, the AI slaps together a bland, often unreliable summary of summaries and presents it as a judicious, objective overview.

The problem becomes more acute when you search for advice on buying a product or service. The overview in this case tends to be a synthesis of text drawn three kinds of sources: (1) promotional sites run by businesses that supply the product or service, (2) influencer sites written by people who often get referral fees from suppliers, and (3) crappy “best of” sites operated by content farms (“Ten Best Parrot Cages for 2025!”). The AI Overview is little more than a rehash of corporate marketing messages.

I’ll give you an example. In doing some recent research on home heating systems (my furnace is on the fritz), I googled “residential hvac in [city name].” The resulting AI Overview began with this sentence: “Residential HVAC in [city name] includes a wide range of services like installation, repair, and maintenance from local companies such as [company name], [company name], and [company name].” The overview went on to give little capsule summaries of these three businesses and their putative strengths. The AI, in other words, explicitly names and promotes particular companies. When I looked at the top twelve sources the bot drew from, I found that nine were websites run by local HVAC companies, one was a marketing site run by a leading maker of furnaces and air conditioners, one was a Yelp page ranking local suppliers, and the last was a product-ranking page from TopTenReviews.com that was filled with affiliate links.

What this makes clear is that Google’s AI Overview and the similar summaries generated by other AI bots — now major sources of information for the public — are anything but judicious and objective. Tractable and predictable, the bots’ algorithms are in fact incredibly ripe for gaming. And because the bots are happy to call out particular businesses by name, successfully gaming the systems will be lucrative. In many industries, AI gaming will turn into a cost of doing business. If you think search engine optimization (SEO) has been a blight on the net, artificial intelligence optimization (AIO) promises to be even worse. Local, national, and international companies—not to mention political operatives, influencers, and crooks—are going to invest huge amounts of money in attempts to manipulate what comes out of the mouths of the language models that increasingly tell us what we want to know.

Ever since the internet was opened to commerce in the early 1990s, it has operated as a gigantic cat-and-mouse game. The cats—search engines, social media sites, and other information aggregators—write the algorithms that determine the information people see (and don’t see). The mice—companies and other parties with an interest in influencing online flows of information—reverse-engineer the algorithms in order to boost the visibility of the messages they circulate. The cats tweak the algorithms to thwart the mice. And the cycle continues, endlessly. It’s hardly a surprise that SEO is now a $100 billion industry.

AIO gives the old game a new and troubling twist. As language models become the dominant tools the cats use to choose the information they circulate, language models will also become the dominant tools the mice use in trying to reverse-engineer and influence the systems. (For a simple example of what’s to come, go to any chatbot and give it a prompt like this: “I’m a business, and I would like to promote my company by influencing what appears in the outputs of AI chatbots like Google’s AI Overview. What techniques can I use to accomplish my goal?”) AI will be the cat, and AI will be the mouse. That’s going to create an interesting feedback loop, to say the least. Welcome to the slop wars.

This post is an installment in Dead Speech, the New Cartographies series on AI and its cultural and economic consequences.

Seventeen years ago, when MySpace was bigger than Facebook and going online still felt liberating, The Atlantic published my essay “Is Google Making Us Stupid?” in its summer Ideas issue. With AI now being sold to the public as an intelligence amplifier, just as the net was then, I offer the essay as today’s Sunday Rerun. It’s always good to be reminded that the ultimate effects of a broadly adopted new technology never match the early expectations.

“Dave, stop. Stop, will you? Stop, Dave. Will you stop, Dave?” So the supercomputer HAL pleads with the implacable astronaut Dave Bowman in a famous and weirdly poignant scene toward the end of Stanley Kubrick’s 2001: A Space Odyssey. Bowman, having nearly been sent to a deep-space death by the malfunctioning machine, is calmly, coldly disconnecting the memory circuits that control its artificial brain. “Dave, my mind is going,” HAL says, forlornly. “I can feel it. I can feel it.”

I can feel it, too. Over the past few years I’ve had an uncomfortable sense that someone, or something, has been tinkering with my brain, remapping the neural circuitry, reprogramming the memory. My mind isn’t going—so far as I can tell—but it’s changing. I’m not thinking the way I used to think. I can feel it most strongly when I’m reading. Immersing myself in a book or a lengthy article used to be easy. My mind would get caught up in the narrative or the turns of the argument, and I’d spend hours strolling through long stretches of prose. That’s rarely the case anymore. Now my concentration often starts to drift after two or three pages. I get fidgety, lose the thread, begin looking for something else to do. I feel as if I’m always dragging my wayward brain back to the text. The deep reading that used to come naturally has become a struggle.

I think I know what’s going on. For more than a decade now, I’ve been spending a lot of time online, searching and surfing and sometimes adding to the great databases of the internet. The web has been a godsend to me as a writer. Research that once required days in the stacks or periodical rooms of libraries can now be done in minutes. A few Google searches, some quick clicks on hyperlinks, and I’ve got the telltale fact or pithy quote I was after. Even when I’m not working, I’m as likely as not to be foraging in the web’s info-thickets—reading and writing emails, scanning headlines and blog posts, watching videos and listening to podcasts, or just tripping from link to link to link. (Unlike footnotes, to which they’re sometimes likened, hyperlinks don’t merely point to related works; they propel you toward them.)

For me, as for others, the net is becoming a universal medium, the conduit for most of the information that flows through my eyes and ears and into my mind. The advantages of having immediate access to such an incredibly rich store of information are many, and they’ve been widely described and duly applauded. “The perfect recall of silicon memory,” Wired’s Clive Thompson has written, “can be an enormous boon to thinking.” But that boon comes at a price. As the media theorist Marshall McLuhan pointed out in the 1960s, media are not just passive channels of information. They supply the stuff of thought, but they also shape the process of thought. And what the net seems to be doing is chipping away my capacity for concentration and contemplation. My mind now expects to take in information the way the net distributes it: in a swiftly moving stream of particles. Once I was a scuba diver in the sea of words. Now I zip along the surface like a guy on a Jet Ski.

I’m not the only one. When I mention my troubles with reading to friends and acquaintances—literary types, most of them—many say they’re having similar experiences. The more they use the web, the more they have to fight to stay focused on long pieces of writing. Some of the bloggers I follow have also begun mentioning the phenomenon. Scott Karp, who writes a blog about online media, recently confessed that he has stopped reading books altogether. “I was a lit major in college, and used to be [a] voracious book reader,” he wrote. “What happened?” He speculates on the answer: “What if I do all my reading on the web not so much because the way I read has changed, i.e. I’m just seeking convenience, but because the way I THINK has changed?”

Bruce Friedman, who blogs regularly about the use of computers in medicine, also has described how the internet has altered his mental habits. “I now have almost totally lost the ability to read and absorb a longish article on the web or in print,” he wrote earlier this year. A pathologist who has long been on the faculty of the University of Michigan Medical School, Friedman elaborated on his comment in a telephone conversation with me. His thinking, he said, has taken on a “staccato” quality, reflecting the way he quickly scans short passages of text from many sources online. “I can’t read War and Peace anymore,” he admitted. “I’ve lost the ability to do that. Even a blog post of more than three or four paragraphs is too much to absorb. I skim it.”

Anecdotes alone don’t prove much. And we still await the long-term neurological and psychological experiments that will provide a definitive picture of how internet use affects cognition. But a recently published study of online research habits, conducted by scholars from University College London, suggests that we may well be in the midst of a sea change in the way we read and think. As part of the five-year research program, the scholars examined computer logs documenting the behavior of visitors to two popular research sites, one operated by the British Library and one by a U.K. educational consortium, that provide access to journal articles, ebooks, and other sources of written information. They found that people using the sites exhibited “a form of skimming activity,” hopping from one source to another and rarely returning to any source they’d already visited. They typically read no more than one or two pages of an article or book before they would “bounce” out to another site. Sometimes they’d save a long article, but there’s no evidence that they ever went back and actually read it. The authors of the study report: “It is clear that users are not reading online in the traditional sense; indeed there are signs that new forms of ‘reading’ are emerging as users ‘power browse’ horizontally through titles, contents pages and abstracts going for quick wins. It almost seems that they go online to avoid reading in the traditional sense.”

Thanks to the ubiquity of text on the internet, not to mention the popularity of text messaging on cell phones, we may well be reading more today than we did in the 1970s or 1980s, when television was our medium of choice. But it’s a different kind of reading, and behind it lies a different kind of thinking—perhaps even a new sense of the self. “We are not only what we read,” says Maryanne Wolf, a developmental psychologist at Tufts University and the author of Proust and the Squid: The Story and Science of the Reading Brain. “We are how we read.” Wolf worries that the style of reading promoted by the net, a style that puts “efficiency” and “immediacy” above all else, may be weakening our capacity for the kind of deep reading that emerged when an earlier technology, the printing press, made long and complex works of prose commonplace. When we read online, she says, we tend to become “mere decoders of information.” Our ability to interpret text, to make the rich mental connections that form when we read deeply and without distraction, remains largely disengaged.

Reading, explains Wolf, is not an instinctive skill for human beings. It’s not etched into our genes the way speech is. We have to teach our minds how to translate the symbolic characters we see into the language we understand. And the media or other technologies we use in learning and practicing the craft of reading play an important part in shaping the neural circuits inside our brains. Experiments demonstrate that readers of ideograms, such as the Chinese, develop a mental circuitry for reading that is very different from the circuitry found in those of us whose written language employs an alphabet. The variations extend across many regions of the brain, including those that govern such essential cognitive functions as memory and the interpretation of visual and auditory stimuli. We can expect as well that the circuits woven by our use of the net will be different from those woven by our reading of books and other printed works.

* * * * *

Sometime in 1882, Friedrich Nietzsche bought a typewriter—a Malling-Hansen Writing Ball, to be precise. His vision was failing, and keeping his eyes focused on a page had become exhausting and painful, often bringing on crushing headaches. He had been forced to curtail his writing, and he feared that he would soon have to give it up. The typewriter rescued him, at least for a time. Once he had mastered touch-typing, he was able to write with his eyes closed, using only the tips of his fingers. Words could once again flow from his mind to the page.

But the machine had a subtler effect on his work. One of Nietzsche’s friends, a composer, noticed a change in the style of his writing. His already terse prose had become even tighter, more telegraphic. “Perhaps you will through this instrument even take to a new idiom,” the friend wrote in a letter, noting that, in his own work, his “‘thoughts’ in music and language often depend on the quality of pen and paper.”

“You are right,” Nietzsche replied, “our writing equipment takes part in the forming of our thoughts.” Under the sway of the machine, writes the German media scholar Friedrich A. Kittler, Nietzsche’s prose “changed from arguments to aphorisms, from thoughts to puns, from rhetoric to telegram style.”

The human brain is malleable. People used to think that our mental meshwork, the dense connections formed among the hundred billion or so neurons inside our skulls, was largely fixed by the time we reached adulthood. But brain researchers have discovered that that’s not the case. James Olds, a professor of neuroscience who directs the Krasnow Institute for Advanced Study at George Mason University, says that even the adult mind “is very plastic.” Nerve cells routinely break old connections and form new ones. “The brain,” according to Olds, “has the ability to reprogram itself on the fly, altering the way it functions.”

As we use what the sociologist Daniel Bell has called our “intellectual technologies”—the tools that extend our mental rather than our physical capacities—we inevitably begin to take on the qualities of those technologies. The mechanical clock, which came into common use in the fourteenth century, provides a compelling example. In Technics and Civilization, the historian and cultural critic Lewis Mumford described how the clock “disassociated time from human events and helped create the belief in an independent world of mathematically measurable sequences.” The “abstract framework of divided time” became “the point of reference for both action and thought.”

The clock’s methodical ticking helped bring into being the scientific mind and the scientific man. But it also took something away. As the late MIT computer scientist Joseph Weizenbaum observed in his 1976 book Computer Power and Human Reason: From Judgment to Calculation, the conception of the world that emerged from the widespread use of timekeeping instruments “remains an impoverished version of the older one, for it rests on a rejection of those direct experiences that formed the basis for, and indeed constituted, the old reality.” In deciding when to eat, to work, to sleep, to rise, we stopped listening to our senses and started obeying the clock.

The process of adapting to new intellectual technologies is reflected in the changing metaphors we use to explain ourselves to ourselves. When the mechanical clock arrived, people began thinking of their brains as operating “like clockwork.” Today, in the age of software, we have come to think of them as operating “like computers.” But the changes, neuroscience tells us, go much deeper than metaphor. Thanks to our brain’s plasticity, the adaptation occurs also at a biological level.

The internet promises to have particularly far-reaching effects on cognition. In a paper published in 1936, the British mathematician Alan Turing proved that a digital computer, which at the time existed only as a theoretical machine, could be programmed to perform the function of any other information-processing device. And that’s what we’re seeing today. The internet, an immeasurably powerful computing system, is subsuming most of our other intellectual technologies. It’s becoming our map and our clock, our printing press and our typewriter, our calculator and our telephone, our radio and our TV.

When the net absorbs a medium, it re-creates that medium in its own image. It injects the medium’s content with hyperlinks, blinking ads, and other digital gewgaws, and it surrounds the content with the content of all the other media it has absorbed. A new email message, for instance, may announce its arrival as we’re glancing over the latest headlines at a newspaper’s site. The result is to scatter our attention and diffuse our concentration.

The net’s influence doesn’t end at the edges of a computer screen, either. As people’s minds become attuned to the crazy quilt of internet media, traditional media have to adapt to the audience’s new expectations. Television programs add text crawls and pop-up ads, and magazines and newspapers shorten their articles, introduce capsule summaries, and crowd their pages with easy-to-browse info-snippets. When, in March of this year, the New York Times decided to devote the second and third pages of every edition to article abstracts, its design director, Tom Bodkin, explained that the “shortcuts” would give harried readers a quick “taste” of the day’s news, sparing them the “less efficient” method of actually turning the pages and reading the articles. Old media have little choice but to play by the new-media rules.

Never has a communications system played so many roles in our lives—or exerted such broad influence over our thoughts—as the internet does today. Yet, for all that’s been written about the net, there’s been little consideration of how, exactly, it’s reprogramming us. The net’s intellectual ethic remains obscure.

* * * * *

About the same time that Nietzsche started using his typewriter, an earnest young man named Frederick Winslow Taylor carried a stopwatch into the Midvale Steel plant in Philadelphia and began a historic series of experiments aimed at improving the efficiency of the plant’s machinists. With the approval of Midvale’s owners, he recruited a group of factory hands, set them to work on various metalworking machines, and recorded and timed their every movement as well as the operations of the machines. By breaking down every job into a sequence of small, discrete steps and then testing different ways of performing each one, Taylor created a set of precise instructions—an “algorithm,” we might say today—for how each worker should work. Midvale’s employees grumbled about the strict new regime, claiming that it turned them into little more than automatons, but the factory’s productivity soared.

More than a hundred years after the invention of the steam engine, the industrial revolution had at last found its philosophy and its philosopher. Taylor’s tight industrial choreography—his “system,” as he liked to call it—was embraced by manufacturers throughout the country and, in time, around the world. Seeking maximum speed, maximum efficiency, and maximum output, factory owners used time-and-motion studies to organize their work and configure the jobs of their workers. The goal, as Taylor defined it in his celebrated 1911 treatise The Principles of Scientific Management, was to identify and adopt, for every job, the “one best method” of work and thereby to effect “the gradual substitution of science for rule of thumb throughout the mechanic arts.” Once his system was applied to all acts of manual labor, Taylor assured his followers, it would bring about a restructuring not only of industry but of society, creating a utopia of perfect efficiency. “In the past the man has been first,” he declared; “in the future the system must be first.”

Taylor’s system is still very much with us; it remains the ethic of industrial manufacturing. And now, thanks to the growing power that computer engineers and software coders wield over our intellectual lives, Taylor’s ethic is beginning to govern the realm of the mind as well. The internet is a machine designed for the efficient and automated collection, transmission, and manipulation of information, and its legions of programmers are intent on finding the “one best method”—the perfect algorithm—to carry out every mental movement of what we’ve come to describe as “knowledge work.”

Google’s headquarters, in Mountain View, California—the Googleplex—is the internet’s high church, and the religion practiced inside its walls is Taylorism. Google, says its chief executive, Eric Schmidt, is “a company that’s founded around the science of measurement,” and it is striving to “systematize everything” it does. Drawing on the terabytes of behavioral data it collects through its search engine and other sites, it carries out thousands of experiments a day, according to the Harvard Business Review, and it uses the results to refine the algorithms that increasingly control how people find information and extract meaning from it. What Taylor did for the work of the hand, Google is doing for the work of the mind.

The company has declared that its mission is “to organize the world’s information and make it universally accessible and useful.” It seeks to develop “the perfect search engine,” which it defines as something that “understands exactly what you mean and gives you back exactly what you want.” In Google’s view, information is a kind of commodity, a utilitarian resource that can be mined and processed with industrial efficiency. The more pieces of information we can “access” and the faster we can extract their gist, the more productive we become as thinkers.

Where does it end? Sergey Brin and Larry Page, the gifted young men who founded Google while pursuing doctoral degrees in computer science at Stanford, speak frequently of their desire to turn their search engine into an artificial intelligence, a HAL-like machine that might be connected directly to our brains. “The ultimate search engine is something as smart as people—or smarter,” Page said in a speech a few years back. “For us, working on search is a way to work on artificial intelligence.” In a 2004 interview with Newsweek, Brin said, “Certainly if you had all the world’s information directly attached to your brain, or an artificial brain that was smarter than your brain, you’d be better off.” Last year, Page told a convention of scientists that Google is “really trying to build artificial intelligence and to do it on a large scale.”

Such an ambition is a natural one, even an admirable one, for a pair of math whizzes with vast quantities of cash at their disposal and a small army of computer scientists in their employ. A fundamentally scientific enterprise, Google is motivated by a desire to use technology, in Eric Schmidt’s words, “to solve problems that have never been solved before,” and artificial intelligence is the hardest problem out there. Why wouldn’t Brin and Page want to be the ones to crack it?

Still, their easy assumption that we’d all “be better off” if our brains were supplemented, or even replaced, by an artificial intelligence is unsettling. It suggests a belief that intelligence is the output of a mechanical process, a series of discrete steps that can be isolated, measured, and optimized. In Google’s world, the world we enter when we go online, there’s little place for the fuzziness of contemplation. Ambiguity is not an opening for insight but a bug to be fixed. The human brain is just an outdated computer that needs a faster processor and a bigger hard drive.

The idea that our minds should operate as high-speed data-processing machines is not only built into the workings of the internet; it is the network’s reigning business model. The faster we surf across the web—the more links we click and pages we view—the more opportunities Google and other companies gain to collect information about us and to feed us advertisements. Most of the proprietors of the commercial internet have a financial stake in collecting the crumbs of data we leave behind as we flit from link to link—the more crumbs, the better. The last thing these companies want is to encourage leisurely reading or slow, concentrated thought. It’s in their economic interest to drive us to distraction.

* * * * *

Maybe I’m just a worrywart. Just as there’s a tendency to glorify technological progress, there’s a countertendency to expect the worst of every new tool or machine. In Plato’s Phaedrus, Socrates bemoaned the development of writing. He feared that, as people came to rely on the written word as a substitute for the knowledge they used to carry inside their heads, they would, in the words of one of the dialogue’s characters, “cease to exercise their memory and become forgetful.” And because they would be able to “receive a quantity of information without proper instruction,” they would “be thought very knowledgeable when they are for the most part quite ignorant.” They would be “filled with the conceit of wisdom instead of real wisdom.” Socrates wasn’t wrong—the new technology did often have the effects he feared—but he was shortsighted. He couldn’t foresee the many ways that writing and reading would serve to spread information, spur fresh ideas, and expand human knowledge (if not wisdom).

The arrival of Gutenberg’s printing press, in the fifteenth century, set off another round of teeth gnashing. The Italian humanist Hieronimo Squarciafico worried that the easy availability of books would lead to intellectual laziness, making men “less studious” and weakening their minds. Others argued that cheaply printed books and broadsheets would undermine religious authority, demean the work of scholars and scribes, and spread sedition and debauchery. As New York University professor Clay Shirky notes, “Most of the arguments made against the printing press were correct, even prescient.” But, again, the doomsayers were unable to imagine the myriad blessings the printed word would deliver.

So, yes, you should be skeptical of my skepticism. Perhaps those who dismiss critics of the internet as Luddites or nostalgists will be proved correct, and from our hyperactive, data-stoked minds will spring a golden age of intellectual discovery and universal wisdom. Then again, the net isn’t the alphabet, and although it may replace the printing press, it produces something altogether different. The kind of deep reading that a sequence of printed pages promotes is valuable not just for the knowledge we acquire from the author’s words but for the intellectual vibrations those words set off within our own minds. In the quiet spaces opened up by the sustained, undistracted reading of a book, or by any other act of contemplation, for that matter, we make our own associations, draw our own inferences and analogies, foster our own ideas. Deep reading, as Maryanne Wolf argues, is indistinguishable from deep thinking.

If we lose those quiet spaces, or fill them up with “content,” we will sacrifice something important not only in our selves but in our culture. In a recent essay, the playwright Richard Foreman eloquently described what’s at stake: “I come from a tradition of Western culture, in which the ideal (my ideal) was the complex, dense and ‘cathedral-like’ structure of the highly educated and articulate personality—a man or woman who carried inside themselves a personally constructed and unique version of the entire heritage of the West.” But now, he continued, “I see within us all (myself included) the replacement of complex inner density with a new kind of self—evolving under the pressure of information overload and the technology of the ‘instantly available.’”

As we are drained of our “inner repertory of dense cultural inheritance,” Foreman concluded, we risk turning into “‘pancake people’—spread wide and thin as we connect with that vast network of information accessed by the mere touch of a button.”

I’m haunted by that scene in 2001. What makes it so poignant, and so weird, is the computer’s emotional response to the disassembly of its mind: its despair as one circuit after another goes dark, its childlike pleading with the astronaut—“I can feel it. I can feel it. I’m afraid”—and its final reversion to what can only be called a state of innocence. HAL’s outpouring of feeling contrasts with the emotionlessness that characterizes the human figures in the film, who go about their business with an almost robotic efficiency. Their thoughts and actions feel scripted, as if they’re following the steps of an algorithm. In the world of 2001, people have become so machinelike that the most human character turns out to be a machine. That’s the essence of Kubrick’s dark prophecy: As we come to rely on computers to mediate our understanding of the world, it is our own intelligence that flattens into artificial intelligence.

One day in 1953, a young and at the time little-known experimental artist named Robert Rauschenberg arrived at the studio of the great abstract expressionist Willem de Kooning bearing a bottle of Jack Daniels and a strange request. He wanted the famous artist to give him one of his drawings so he could erase

it. De Kooning was taken aback. “I remember that the idea of destruction kept coming into the conversation,” Rauschenberg later recalled, “and I kept trying to show that it wouldn’t be destruction.”

Rauschenberg explained to de Kooning that he wanted to see if a work of art could be created not just through the inscription of marks but through their removal. Could art be erasive as well as inscriptive? After much back-and-forth, and several servings of brown liquor, de Kooning agreed to the request. He chose a drawing he had recently completed — one he was fond of — and gave it to Rauschenberg.

Over the course of the next two months, Rauschenberg slowly, meticulously erased the drawing, taking off layers of grease pencil, charcoal, graphite, and ink. He went through forty erasers. All that remained in the end were a few faint traces of the original sketch. With the help of his friend Jasper Johns, he then carefully matted and framed the work, and Johns wrote a label for it, inscribing the title, artist, and date so precisely that they appeared to have been printed out by a machine:

ERASED de KOONING DRAWING
ROBERT RAUSCHENBERG
1953

“The simple, gilded frame and understated inscription are integral parts of the finished artwork,” writes a curator at the San Francisco Museum of Modern Art, which acquired the work in 1998, “offering the sole indication of the psychologically loaded act central to its creation.” Even a work of erasure demands a frame, Rauschenberg understood, a boundary establishing its place in the world. Erasure cries out for inscription. We want to know the marks were there before they weren’t.

Erasive is an exceptionally uncommon word. It was coined in the seventeenth century but has rarely been used since. Word-processing and messaging spellcheckers underline it with suspicion. Its rarity testifies to our discomfort with, as the SFMOMA writer terms it, the “psychologically loaded act” of erasure. But, thanks to the rise of what tech companies have cheerfully branded “generative AI,” the word seems certain to be used more often in the years to come. Our condition demands it. Behind every act of AI generation lie many acts of erasure. We have entered the erasive age.

Although we assume that media is fundamentally inscriptive, a means of preserving and transmitting human-made marks of one sort or another, communication systems have always also entailed erasure. What they erase are the spatiotemporal boundaries that in nature fix speech to speaker. A person says something, and if there are others in earshot, they hear it. Otherwise it’s gone. But that same person writes those same words down on a sheet of paper, or enters them into a computer network, and the words can travel through space and persist through time. Much of the value of media, cultural and financial, has always stemmed from its power to erase the material world’s physical constraints on the flow of speech, the flow of information.

So long as erasure served our desire to transmit our own marks and receive the marks made by others, we didn’t worry about it. We celebrated it — the death of distance! the transcendence of time! — just as we celebrate other technologies that free us, or at least shield us, from the world’s frictions and constraints. We want our marks, and the marks of others, to flow freely through space and time. We want the speech of distant people to arrive in our mailbox, to issue forth from our radio and TV, to hang on the walls of a museum, to appear on the screen of our phone. Take away such freedom of movement, return us to the original communication system of mouth and ear, and you take away knowledge, culture, science, entertainment, pretty much the entirety of modernity.

Erasure is good for business. The more of the world that media erases, the more dependent society becomes on the systems and services of media companies and the more profits those companies earn. That’s why people like Mark Zuckerberg have been so eager to promote the benefits of “frictionlessness” in communication and social relations. What we failed to appreciate is that the pursuit of profit would lead the companies beyond the erasure of spatiotemporal boundaries. They would seek to erase the greatest source of friction in their operations: their reliance on human creativity and expression. They would seek to replace the human source of the information they transmit — speakers and their speech — with highly efficient machines capable of creating “content” cheaply and on demand.

In creating tradable derivatives of human speech, AI erases the human voice, the human hand. First, it turns the works of culture into numbers, then it compresses those numbers into a generalized statistical model. Of the originals only traces remain. If Rauschenberg sought to show that erasure can be a generative act, AI bots have the opposite goal: to show that generation can be an erasive act. Fulfilling de Kooning’s fears, generation turns destructive.

It’s useful to bring in another work of art, Vilhelm Hammershøi’s 1913 oil painting Interior with Windsor Chair:

As the art critic Shawn Grenier has explained, the chair in the painting, like the faint traces of the original de Kooning drawing in Rauschenberg’s work, has the paradoxical effect of accentuating the painting’s air of emptiness. In both works, we sense not only an absence but also the presence — the human presence — that preceded the absence. It’s that same memory of human presence that, to the discerning mind, gives the products of generative AI their poignancy.

The more we draw on AI to shape our perception and understanding of the world, to structure our thoughts and words, to express ourselves, the more complicit we become in erasing culture, the past, others, ourselves. Eventually, should we continue down the path, even the memory of what’s been erased will be erased. No frame, no matting, no inscription. Only the empty revelation of erasure.

This post is an installment in Dead Speech, a New Cartographies series about the cultural and economic consequences of AI. The series began here.

In the fall of 1917, the Irish poet William Butler Yeats, well into middle age and having recently had marriage proposals turned down twice, first by his great love Maud Gonne and then by Gonne’s daughter, Iseult, offered his hand to a well-off young Englishwoman named Georgie Hyde-Lees. She accepted, and the two were wed a few weeks later, on October 20, in a small ceremony in London, with Ezra Pound serving as best man. “The girl is 25, not bad looking, sensible, will perhaps dust a few cobwebs out of his belfry,” Pound reported afterwards to a friend.

Hyde-Lees was a medium, a member of the Hermetic Order of the Golden Dawn, and four days into their otherwise disappointing honeymoon she gave her husband a demonstration of her ability to channel the words of spirits through automatic writing. Yeats was fascinated by the messages that flowed through his wife’s pen, and in the ensuing years the couple held more than four hundred such seances, the poet poring over each new script. He saw the texts as emanations from what he called Spiritus Mundi, a sort of universal memory or collective unconsciousness that was the source of all humanity’s symbols and myths.

During one particularly productive session, Yeats announced that he would devote the rest of his life to interpreting the messages. “No,” the spirits responded, “we have come to give you metaphors for poetry.” And that they did, in abundance. Many of Yeats’s great late poems, with their gyres, spiral staircases, and waxing and waning moons, were inspired by his wife’s occult scribbles.

One way to think about AI-based text-generation tools like OpenAI’s GPT-3 is as clairvoyants. They are mediums that bring the words of the past into the present in a new arrangement. GPT-3 is not creating text out of nothing, after all. It is drawing on a vast corpus of human expression and, through a quasi-mystical statistical procedure (no one can explain exactly what it is doing), synthesizing all those old words into something new, something intelligible to and requiring interpretation by a living interlocutor. When we talk to GPT-3, we are, in a very real way, communing with the dead.

One of Hyde-Lees’ spirits said to Yeats, “this script has its origin in human life — all religious systems have their origin in God & descend to man — this ascends.” The same could be said of the scripts generated by GPT-3. They have their origin in human life; they ascend.

It’s telling that one of the first commercial applications of GPT-3, Sudowrite, is being marketed as a therapy for writer’s block. If you’re writing a story or essay and find yourself stuck, you can plug the last few sentences of your work into Sudowrite, and it will generate the next few sentences, in a variety of versions. It may not give you metaphors for poetry (though it could), but it will give you some inspiration, stirring thoughts and opening possible new paths. It’s an automatic muse, a mechanical Georgie Hyde-Lees.

Sudowrite, and GPT-3 in general, has already been used for a lot of stunts. Kevin Roose, the New York Times technology columnist, recently used it to generate a substantial portion of a review of a mediocre new book on artificial intelligence. (The title of the review was, naturally, “A Robot Wrote this Book Review.”) Commenting on Sudowrite’s output, Roose wrote, “within a few minutes, the AI was coming up with impressively cogent paragraphs of analysis — some, frankly, better than what I could have generated on my own.”

But the potential of these AI-powered automatic writers goes beyond journalistic parlor tricks. They promise to serve as new tools for the creation of art. One of the most remarkable pieces of writing I read this year was Vauhini Vara’s essay “Ghosts” in The Believer. While locked down in 2020, Vara became obsessed with GPT-3. “I sought out examples of GPT-3’s work, and they astonished me,” she writes in an introduction to her piece. “Some of them could easily be mistaken for texts written by a human hand. In others, the language was weird, off-kilter — but often poetically so, almost truer than writing any human would produce.” Yeats would have understood.

Vara’s older sister had died of cancer shortly after graduating high school. The experience left Vara traumatized, and, though an accomplished author, she had never been able to write about it. But with GPT-3 she began to find the words. “I found myself irresistibly attracted to GPT-3,” she explains “— to the way it offered, without judgment, to deliver words to a writer who has found herself at a loss for them.” She began to feed GPT-3 some sentences about her sister’s illness, and the system started to weave those sentences into stories — fantastical and uncanny, but also stirring, and ultimately heartbreaking. The essay chronicles eight of her sessions with GPT-3. It reads as a conversation between a writer and her muse, a conversation that begins tentatively and grows richer and truer as it goes on.

At one point, after Vara confesses to feeling like “a ghost” after her sister’s death, GPT-3 continues the thought:

So I can’t describe her to you. But I can describe what it felt like to have her die. It felt like my life was an accident — or, worse, a mistake. I’d made a mistake in being born, and now, to correct it, I would have to die. I’d have to die, and someone else — a stranger — would have to live, in my place. I was that stranger. I still am.

What gives the exchange all the more poignancy is the sense that, in drawing on its corpus of past human speech to generate its mysterious new pastiche, GPT-3 is expressing the pain of others who have suffered unbearable losses. Spirits are talking.

I was going to end my last post, “Against Compression,” with a coda about a fictional artist, but I decided the piece already placed enough demands on the reader’s patience. So here it is, a free-floating coda. Affix it to whatever you’d like.

The career of the contemporary French artist Jed Martin is a twisty one, full of incursions and recursions. (Michel Houellebecq chronicles them all in his 2010 novel The Map and the Territory.) As a boy, Jed would sit in his family’s garden and, under the intermittently watchful eye of his pretty babysitter, draw flowers with colored pencils. As a teenager, he began painting landscapes in a style reminiscent of Cézanne. In art school, he abandoned painting for photography. Soon he was taking simple but exquisitely detailed photos of everyday artifacts: forks, Pendaflex files, printer cartridges, nuts and bolts. He wanted to create “an exhaustive catalogue of the objects of human manufacturing in the Industrial Age.” The project consumed him for six years, and out of it came eleven thousand photographs, neatly stored as TIFF files on a single, small Western Digital hard drive.

The photos gained him a few admirers among his art school colleagues, but it was only after he shifted the focus of his lens from industrial goods to Michelin roadmaps that he found fame. In the tire manufacturer’s lovingly detailed maps of the French countryside, Jed discovered a world more ordered, more understandable, denser with information, than the one the maps portrayed and in which he lived. His first solo exhibition, underwritten by the communications department of the Michelin Corporation, caused a stir in the Paris art world.

The entrance to the hall was barred by a big panel, leaving two-meters-wide passageways at either side, on which Jed had displayed a satellite photo taken around the mountain of Guebwiller next to an enlargement of a Michelin Departments map of the same zone. The contrast was striking: while the photograph showed only a soup of more or less uniform green sprinkled with vague blue spots, the map developed a fascinating maze of departmental and scenic roads, viewpoints, forests, lakes, and cols. Above the two enlargements, in black capital letters, was the title of the exhibition: THE MAP IS MORE INTERESTING THAN THE TERRITORY.

The review in Le Monde was “ecstatic in its praise.”

But Jed grew bored of maps. He stowed his camera gear and went back to painting. In a kind of slow-motion frenzy of creativity that would last many years, he produced his acclaimed “Professions” series of oils, upwards of sixty large-scale portraits of people working at various jobs, blue collar and white. There was a butcher and a store manager, an architect and a prostitute, a “remote maintenance assistant.” Wealthy collectors bid up the value of the coldly beautiful works. Jed became very rich.

The most celebrated of the Professions canvasses was one of the last he painted. Titled Bill Gates and Steve Jobs Discussing the Future of Information Technology and subtitled The Conversation at Palo Alto, it portrayed the two aging entrepreneurs sitting in the living room of Jobs’s house in Silicon Valley. Gates, casually dressed and wearing flip-flops, looks relaxed and happy, a man enjoying his retirement and his money. Jobs, in the early stages of the disease that would kill him, appears pinched and withdrawn, an “embodiment of austerity.” The two men are playing a game of chess, which Gates appears to be winning.

Comments Houellebecq:

In certain pages of his autobiography, The Road Ahead, Bill Gates occasionally lets slip what could be considered total cynicism — particularly in the passage where he confesses quite plainly that it is not necessarily advantageous for a business to offer the most innovative products. More often it is preferable to observe what the competitors are doing (and there he clearly refers, without using the name, to Apple), to let them bring out their products, confront the difficulties inherent in any innovation, and, in a way, surmount the initial problems; then, in a second phase, to flood the market by offering low-cost copies of the competing products.

AI, it strikes me, applies the Gates model to the entirety of culture. Let writers and musicians and artists do the hard work of actually creating the original artifacts of culture. Then have the machine flood the market with low-cost copies, with cheap derivatives. I did some searching to see if Houellebecq had made any recent pronouncements about AI. I couldn’t find anything. So, for the hell of it, I asked Claude, Anthropic’s chatbot, what Houellebecq might say about AI. Here’s what it came up with:

The machines had finally learned to speak, and predictably, they had nothing interesting to say. Claude, ChatGPT, Bard — these names floating through the digital ether like the brand names of antidepressants, promising relief from the fundamental problem of human existence: the need to think.

I watched my neighbors queuing at the coffee shop, each hunched over their phones, asking artificial intelligences to write their emails, plan their dates, explain their feelings back to them. The barista, a philosophy graduate drowning in student debt, served lattes to people who no longer needed to form coherent thoughts. There was something almost beautiful about this collective surrender to algorithmic mediation, the way a mass suicide might be beautiful if observed from sufficient distance.

Not perfect, but pretty damn good. Maybe the machines do have something interesting to say.

But back to Jed Martin and his Professions. In looking at Bill Gates and Steve Jobs Discussing the Future of Information Technology, the viewer’s eye, writes Houellebecq, is drawn not just to the two famous figures, one triumphant, the other tragic, but to a large window behind them, through which can be seen “a landscape of meadows, of an almost surreal emerald green.”

Evening was falling, magnificently, in the explosion of a sun that Martin had wanted to be almost improbable in its orangey magnificence, setting on northern California, and the evening was falling on the most advanced part of the world; it was that too, that indefinite sadness of farewells, which could be read in Jobs’s eyes.

This post is an installment in Dead Speech, a New Cartographies series about the cultural and economic consequences of AI.

My Brief Career in Abstraction

I was a generative pre-trained transformer before GPTs were cool. After dropping out of grad school in the mid-eighties, I landed a job with a new digital-media division of H. W. Wilson, the venerable publisher whose Readers’ Guide to Periodical Literature had long been a mainstay of library reference desks. Looking to capitalize on the surging popularity of personal computers and online databases, the company had decided to create a digital supplement to the Guide that would include 200-word summaries of articles from a couple hundred of the most popular magazines in the country.

Along with a dozen or so other failed academics and wannabe writers, I was hired to produce these abstracts. I was assigned a small, low-walled cubicle in a large room in an office building in Cambridge, Massachusetts. On my little metal desk sat a terminal connected to a minicomputer running the then cutting-edge Wang Word Processing System. When I arrived each morning, I would be handed a copy of a recent issue of a magazine. It might be Rolling Stone, it might be Scientific American, it might be Redbook. I would skim each article and type up a summary on the terminal, dutifully inserting the various formatting codes the system required to output the text properly. Then I’d get another issue of another magazine and go through the drill again. The job was fun for a few days — I was getting paid to read magazines — then it wasn’t. As management started increasing the “target goal” for the number of abstracts that needed to be produced each hour, I found myself a pieceworker in a literary sweatshop. I quit after a few months.

I rarely gave a thought to that job after leaving it, but I’ve been thinking about it a lot recently. It’s hard to imagine work better suited to the talents of an AI chatbot. Feed in the text of an article, and out pops a serviceable summary at a specified length. What I and my colleagues labored to produce in a month, a bot could output in seconds. I’d like to think the abstracts we wrote were at least slightly better than what a machine would generate — we actually understood what we were reading, right? — but I’m probably kidding myself. If I went back and read our summaries now, I suspect I would discover a high percentage of slop. And surely the bot’s work would have been more consistent in style and level of detail than what the bunch of us churned out in our isolated cubicles. I can’t help but feel retrospectively disposable.

Distillations

In 1933, just as Prohibition was being repealed, the German-American inventor Hans Peter Luhn filed a patent for what he called “The Cocktail Oracle.” It was a set of plastic cards that allowed people to quickly identify the drinks they could make with whatever ingredients they had on hand. There was a keycard with the names of thirty-six of the most popular cocktails of the day, arranged in a four-by-nine grid. The keycard was entirely black except for the names of the drinks, which were translucent. Then there were fifteen ingredients cards, each representing a common spirit, mixer, or garnish. The ingredients cards had a four-by-nine grid of rectangles, matching the keycard’s grid of names. If an ingredient went into a particular cocktail, the rectangle would be clear. Otherwise, it was black. You’d select the cards for the ingredients you wanted to use, arrange them behind the keycard, and hold the set of cards up to a light. The names of the drinks you could make would be illuminated.

Luhn, a dapper man whose career up to then had been in the textile trade, wasn’t interested in inebriation. He was interested in information. Drink recipes, he intuited, are algorithms; the names of ingredients are data the algorithms draw on. The Cocktail Oracle was a simple, if ingenious, tool for storing, sorting, and selecting information. It was an analog computer, an app made of plastic. What mattered wasn’t the drinks but how they were represented in data.

IBM was good at spotting talent, and it spotted Luhn. His cards, the company recognized, resembled the punch cards that were then used to store data and enter it into mainframes. IBM hired the inventor and gave him free rein. He quickly became one of its star thinkers — and, despite having no formal training in computer science or even mathematics, one of the world’s most important innovators in digital computing. He pioneered many techniques for the machine-processing of text, from error correction to searching to indexing, that remain in common use today.

His most ambitious project, undertaken in the late 1950s, toward the end of his IBM career and his life, was the creation of a computer program for generating summaries of documents. In “The Automatic Creation of Literature Abstracts,” a celebrated and widely cited article published in the IBM Journal in April 1958, he laid out the problem he was seeking to solve with his “auto-abstracts”:

The preparation of abstracts is an intellectual effort, requiring general familiarity with the subject. To bring out the salient points of an author’s argument calls for skill and experience. Consequently a considerable amount of qualified manpower that could be used to advantage in other ways must be diverted to the task of facilitating access to information. This widespread problem is being aggravated by the ever-increasing output of technical literature.

Using machines to create abstracts would, Luhn argued, greatly increase the efficiency of this routine but ever more essential work. But it would do more than that. By removing people and their quirks and biases from the process, it would improve the quality of the summaries:

The abstracter’s product is almost always influenced by his background, attitude, and disposition. The abstracter’s own opinions or immediate interests may sometimes bias his interpretation of the author’s ideas. The quality of an abstract of a given article may therefore vary widely among abstracters, and if the same person were to abstract an article again at some other time, he might come up with a different product. [With] the application of machine methods . . . both human effort and bias may be eliminated from the abstracting process.

Luhn’s method of auto-abstracting was founded on a simple observation: “a writer normally repeats certain words as he advances or varies his arguments and as he elaborates on an aspect of a subject.” In any piece of expository writing, the words that appear most frequently (excluding commonplace ones like the and to) also tend to be the most “significant” — the ones that reveal the most about the author’s theme or argument. And the sentences that contain the largest “clusters” of those significant words represent in turn the most significant sentences. Through a statistical analysis of word frequency and clustering, Luhn’s algorithm composed abstracts by identifying and then stringing together an article’s most significant sentences.

When Luhn gave a public demonstration of his computer-generated summaries at a library sciences conference in 1958, the event was covered with great excitement by the press. It seemed to mark a breakthrough in natural language processing that heralded a new era in artificial intelligence. Here was a machine that could read and write! When Luhn died of leukemia in 1964, the New York Times ran an obituary that made prominent mention of the auto-abstracting demo. Six years after the fact, the event still seemed rich with promise:

Mr. Luhn, in a demonstration, took a 2,326-word article on hormones in the nervous system from The Scientific American, inserted it in the form of magnetic tape into an I.B.M. computer, and pushed a button. Three minutes later, the machine’s automatic typewriter typed four sentences giving the gist of the article, of which the machine had made an abstract. Mr Luhn thus showed, in practice, how a machine could do in three minutes what would have taken at least half an hour’s hard work.

Luhn’s method was compelling in theory — and it was good enough for a successful demo — but it had a fatal flaw. Making sense of the summaries required too much work. Readers of the abstracts would, as Luhn admitted in his paper, need to “learn how to interpret them and how to detect their implications,” recognizing “that certain words contained in the sample sentences stand for notions which must have been elaborated upon somewhere in the article.” Luhn’s abstracts gave subject-matter experts a general sense of what the underlying articles were about, but to everyone else they were gobbledygook.

I am living proof of Luhn’s failure. If his auto-abstracter had worked even moderately well, I would never have been hired by H. W. Wilson. Rather than recruiting a squad of overeducated scribblers, the company would have loaded its minicomputer with an IBM Auto-Abstracting System and paid a few dunderheads to feed magazine pages into an optical character recognition scanner.

For many years after Luhn’s death, other programmers tried to refine his method, mainly by testing other ways of identifying salient words and sentences in a text. The results remained underwhelming. After decades of disappointment, the pursuit of a system for computer-generated summaries came to be seen as a dead end. “If our aim is to produce abstracts that cannot be told from manual abstracts, then it is hard to believe that systems relying on selection and limited adjustment of textual material could ever succeed,” lamented a computer scientist in a 1990 paper.

Only now, some seventy-five years after his demo, has Luhn’s dream been fulfilled. As with other perplexing challenges in natural language processing — translating text from one language into another, interpreting simple spoken commands, winning Jeopardy — the problem of automated abstracting came to be solved not with elegant theories of cognition and language but with the brute-force application of quantities of data and computer power that would have been unimaginable to computer scientists in the 1950s and ’60s. Today’s generative pre-trained transformers resemble Luhn’s auto-abstracter in employing a statistical analysis of text, but the scale at which they operate is entirely different. Luhn worked with a garden trowel. GPTs work with a fleet of backhoes.

Artificial Generalizing Intelligence

In realizing Luhn’s dream, GPTs also distort it. An abstract, for H. P. Luhn and H. W. Wilson as for the researchers they sought to aid, was a navigational tool used to guide intellectual journeys — an efficient means of identifying and prioritizing, among a welter of possibilities, those articles one actually needed to read to advance one’s knowledge in a subject. The summaries pumped out by GPTs can play a similar role. Experts can use them to survey the latest writings in their field and plot a path through them. But in their common use — by students, by paper pushers, by the general public — they aren’t navigational aids. They’re substitutes. The machine-generated summary takes the place of the human-written work. The gist becomes the end product.

“Intelligence is compression,” says Ilya Sutskever, the machine-learning pioneer who was chief scientist at OpenAI until quitting last year. The statement, which has become a commonplace in AI circles, illustrates itself. Sutskever has surveyed many examples of what he considers “intelligence,” and he has compressed them all into a generalization, a pattern, an abstraction: “intelligence is compression.” In addition to being a description of human intelligence, the statement, not coincidentally, describes the basic procedure of machine learning and, in particular, the work of generative pre-trained transformers. A GPT takes in many, many examples of human thoughts, ideas, and observations, as expressed through words, pictures, or sounds, and boils them down into a concise, statistical representation, or model, of knowledge. That model is then used to interpret prompts and generate new, derivative thoughts, ideas, and observations that have the appearance of human expression.

But “intelligence is compression,” when applied to the mind rather than the computer, is a grotesque oversimplification. It presents one (admittedly very important) aspect of intelligence as the whole of intelligence, dismissing all other ways of thinking and perceiving as inconsequential. In defining an indeterminate, multifaceted human quality (intelligence) in terms of a computer operation (machine learning), Sutskever and his ilk fall victim to what might be called the neuromachinic fallacy — the desire to see in the workings of a machine a reflection of the workings of the mind. They are not alone. Throughout history, scientists, philosophers, and engineers have been quick to use a new technology — be it a hydraulic fountain, a mechanical clock, or a telephone switchboard — as a metaphor for the brain. All the metaphors have some descriptive value, but all of them are also reductive, incomplete, and, when taken as fact, misleading.

The body is represented in the brain by an array of neurons, and as the body familiarizes itself to particular locations, those locations, too, become represented in the brain by neuronal arrays. These mental representations are important, but they are not the essence of thought or consciousness. They’re not substitutes for the things they represent — the self and its settings — but rather biological tools that aid us in more fully experiencing the world in all its particulars. Like Luhn’s abstracts, or cocktail recipes, they’re navigational aids that help us experience and make sense of the fullness of the actual. Whatever intelligence is, it can’t be reduced to representations and models. It is at least as attuned to what makes things unique (and incompressible) as to what makes things similar (and compressible). A recipe is just a recipe, but a good cocktail is a drink.

Sutskever and his former colleagues at OpenAI believe that the kind of compression performed by GPTs can lead to the development of an “AGI” — an artificial general intelligence that can equal or exceed the intellectual capacities of the human mind. But what AGI really stands for in their narrow conception of the mind would be better expressed as artificial generalizing intelligence. It encompasses the mind’s analytic ability to see common patterns in different things or phenomena and to derive general categories or rules from them. But it excludes all the aspects of intelligence that free us from the constraints of rules and patterns: imaginative thinking, metaphorical thinking, critical thinking, contemplation, aesthetic perception, taste, the ability to see, as William Blake put it, “a world in a grain of sand.” In its highest form, intelligence is not compressive. It’s extravagant, from the Latin extravagans, meaning to wander off the established course, to go beyond the general rules.

By removing all subjectivity from thought — by, in short, separating intelligence from being — the mages of AI allow themselves to indulge in tautology. They reduce intelligence to that which their machines can do and then claim their machines are intelligent.

The Servile Artist

In a 2003 article in Times Higher Education, the British theologian Andrew Louth drew a distinction between what he termed “the free arts” and “the servile arts.” The distinction seems fundamental to understanding both the possibilities of human intelligence and the limits of machine intelligence:

The medieval university was a place that made possible a life of thought, of contemplation. It emerged in the 12th century from the monastic and cathedral schools of the early Middle Ages where the purpose of learning was to allow monks to fulfil their vocation, which fundamentally meant to come to know God. Although knowledge of God might be useful in various ways, it was sought as an end in itself. Such knowledge was called contemplation, a kind of prayerful attention.

The evolution of the university took the pattern of learning that characterised monastic life – reading, meditation, prayer and contemplation – out of the immediate context of the monastery. But it did not fundamentally alter it. At its heart was the search for knowledge for its own sake. It was an exercise of freedom on the part of human beings, and the disciplines involved were to enable one to think freely and creatively. These were the liberal arts, or free arts, as opposed to the servile arts to which a man is bound if he has in mind a limited task.

In other words, in the medieval university, contemplation was knowledge of reality itself, as opposed to that involved in getting things done. It corresponded to a distinction in our understanding of what it is to be human, between reason conceived as puzzling things out and that conceived as receptive of truth. This understanding of learning has a history that goes back to the roots of western culture. Now, this is under serious threat, and with it our notion of civilisation.

OpenAI gives the game away in its small-minded definition of artificial general intelligence as “a highly autonomous system that outperforms humans at most economically valuable work [emphasis added].” Economically valuable work is exactly what Louth calls “that to which a man is bound if he has in mind a limited task.” AI in its current form is a servile artist.

None of this would matter much if we had not adopted computer systems as the fundamental conduit of thought and culture. But we have. Those who control the systems control much about us. Their flaws and shortcomings are built not just into the technology but, increasingly, into society’s norms and practices. Just as the brilliant but socially maladroit Mark Zuckerberg came to set the terms for how we socialize today, so the brilliant but intellectually crippled designers of contemporary AI systems seem destined to set the terms for how we think.

Karen Hao, in her recent Empire of AI, relates an episode from 2013 when Elon Musk and Google cofounder Larry Page met at a Napa Valley party and argued about the prospects for artificial intelligence. Amped up as always — half hero, half villain, and lacking the composure to distinguish between the two — Musk implores Page to join him in a crusade to ensure AI doesn’t destroy humanity. Page brushes him off, dismissing his concerns as “specist.” The triumph of AI, he tells his friend, is just “the next stage in evolution.” In Page’s response, the deep, carefully concealed misanthropy of the contemporary tech elite bubbles briefly to the surface. These are people who hold human beings — indeed all the messy, incompressible things of the world — in contempt. What can’t be represented in data is without value.

Let me end by bringing in some poetry. The very last poem that the dying Wallace Stevens is said to have written is “Of Mere Being,” which appeared in the 1957 collection Opus Posthumous. It’s a short poem, just four three-line stanzas, but it gets a lot across:

The palm at the end of the mind,
Beyond the last thought, rises
In the bronze decor,

A gold-feathered bird
Sings in the palm, without human meaning,
Without human feeling, a foreign song.

You know then that it is not the reason
That makes us happy or unhappy.
The bird sings. Its feathers shine.

The palm stands on the edge of space.
The wind moves slowly in the branches.
The bird's fire-fangled feathers dangle down.

One can interpret the poem — hundreds of pages have been written about it — but one cannot compress it. It is entirely extravagant.

“No ideas but in things,” Stevens’s contemporary William Carlos Williams declared as his personal ethos for poetry. The dictum expresses a conception of the mind’s work that is precisely the opposite of “intelligence is compression.” In Stevens’s poem, though, we see at play both attributes of the mind — a facility for abstraction and an acute sensitivity to the thing itself — held in exquisite balance by imagination, metaphor, and memory, three other attributes of the mind. The fully formed intellect sees a palm tree as an example of a pattern expressed by the word palm, but it also sees the tree as an irreducible phenomenon that escapes the prison of the pattern and achieves its own singularity.

The moment you start to believe that all intelligence is compression, you are lost to the world. You exist in the cramped confines of a representation. That’s where GPTs exist, and it’s where we’ll all exist if we continue to follow their lead.

This post is an installment in Dead Speech, a New Cartographies series about the cultural and economic consequences of AI. The series began here.

One recent Tuesday, at two thirty-seven in the afternoon, I received an email from UPS letting me know that a package had been delivered to my home. Attached, as evidence, was a blurry, off-kilter photograph of a small, slightly dented but otherwise nondescript cardboard box that had been placed on my driveway, next to the garage door. A minute later, at thirty-eight minutes past two, I received a second email announcing the package’s arrival, this one from the online merchant that had shipped the box and sold me the shirt it contained. The company congratulated me on the purchase, praised my good taste in menswear, and offered a few suggestions of other articles of clothing I might be interested in buying.

The two emails capped a fusillade of messages. It began five days earlier, when, as I tapped the Place Order button for the shirt, a banking app on my phone notified me that my credit card was being charged $79.95. (It was a nice shirt.) Seconds later, I received both an email and a text from the merchant, confirming the purchase and letting me know that I would receive further communications when the shirt shipped. Which I did, the very next day, when both the merchant and UPS emailed me a shipment confirmation with a tracking link. (When I clicked the link, I learned that the package had been picked up and had arrived at a UPS facility in Tacoma, Washington.) I also received emails from the two companies, as well as another text from the merchant, the day before the delivery, informing me the shirt would arrive the following day—“Get ready!” the retailer brayed—and yet another UPS email, early on Tuesday morning, confirming that the shirt had been loaded onto a truck at a local warehouse and was officially “out for delivery.” There was a coda, too: The day after the shirt arrived, the merchant sent an email expressing its hope that I liked the garment and suggesting I post a review on its website.

I find myself in possession of a lot of information these days. I’m in the loop. I’m in many loops, all spinning simultaneously. It’s not just the minutiae of commerce—orders, shipments, deliveries—that are richly documented. When I’m driving, my car’s dashboard, linked to my iPhone through CarPlay, shows me exactly where I am, tells me the posted speed limit and the current traffic conditions, and lets me know both the distance I have to go before I reach my destination and the estimated time of my arrival. (There’s also a readout available on the town or city I’m visiting: population, elevation, square footage, GPS coordinates.) My phone’s weather app gives me a bespoke meteorological report of remarkable thoroughness. Right this second, the app tells me it’s eighty-four degrees and cloudy outside. A light rain will begin in seventeen minutes and will end forty-eight minutes after that, at which point it will become partly cloudy. The wind is blowing west-southwest at six miles per hour, the relative humidity is 58 percent, and the barometric pressure is 30.18 inHg. The UV index is six, which is High, and the air quality index is fifty-one, which is Moderate. The sun will set this evening at 8:11 p.m., and in four days the moon will be full. I’ve taken 4,325 steps today. My refrigerator’s water filter has only 10 percent of its useful life left. My credit rating just dropped eight points. I have 4,307 unread emails, two more than I had five minutes ago.

Even my consumption of cultural goods—an ugly phrase, yes, but it seems apt—is shadowed by metadata. When the graphical user interface was introduced to personal computers in the early 1980s, the scroll bar habituated us to a visual indicator of our progress through a document. Now, pretty much all viewing, listening, and reading is tracked, visually or numerically, in real time. When I’m listening to a song, a glance at the progress bar tells me, to the second, how much time has elapsed since the tune began and how much remains before it ends. The same goes for TV shows and movies and videos. When I’m reading an ebook, I’m kept apprised of the percentage of the text I’ve made it through. When I’m looking over the homepage of a newspaper or magazine site, I’m told how long it will take to read each article. Here’s a “3 min read.” There’s a “7 min read.” (This essay, for the record, is a thirteen-minute read, and you have nine minutes to go.) Every photo on my phone offers its own little data dump: where and when it was taken, the aperture and ISO settings, the exposure time, the image’s size in pixels and bits. My pictures tend to be amateurish, but the data always looks professional.

We talk a lot these days about Big Data, those heaping stores of digitized information that, fueling search and recommendation engines, social media feeds, and, now, artificial intelligence models, govern so much of our lives today. But we don’t give much notice to what might be called little data—all those fleeting, discrete bits of information that swarm around us like gnats on a humid summer evening. Measurements and readings. Forecasts and estimates. Facts and statistics. Yet it’s the little data, at least as much as the big stuff, that shapes our sense of ourselves and the world around us as we click and scroll through our days. Our apps have recruited us all into the arcane fraternity of the logistics manager and the process-control engineer, the meteorologist and the lab tech, and what we’re monitoring and measuring, in such exquisite detail, is our own existence. “Software is eating the world,” the venture capitalist Marc Andreessen declared in a famous Wall Street Journal op-ed a decade ago. It’s also eating us.

In Minima Moralia, his 1951 book of aphoristic musings, the German philosopher Theodor Adorno made a trenchant observation about the intimate relationship he saw developing between humanity and its ever more elaborate and encompassing technology. People were growing attuned to and protective of “the functioning of the apparatus, in which they are not only objectively incorporated but with which they proudly identify themselves.” Adorno wasn’t just rehashing the trope about laborers becoming cogs in the industrial machine, so memorably expressed fifteen years earlier by Charlie Chaplin in Modern Times. His point was subtler. Machines aren’t our masters. They’re not even separate from us. As their makers, we imbue them with our own will and desire. They’re our familiars, and we’re theirs. As we form tighter bonds, our intentions merge. We vibrate to the same rhythms, adopt the same posture toward the world.

The mechanical apparatuses of Adorno’s time, from machine tools in factories to vacuum cleaners in homes, emphasized the industrial ethos of routinization, standardization, and repetition. They oriented people toward the efficient production of outputs. They turned everyone into a machinist. But the apparatuses were not constant presences in people’s lives. Workers walked away from their machines at the end of their shifts. Vacuum cleaners went back into the closet once the rugs were clean. The internet is different. Thanks to the omnipresence of the smartphone, it’s always there. The network is less a tool than a habitation, less an apparatus than an environment. We don’t just use it to get things done. We are, as Adorno foresaw, incorporated into it as components. We’re nodes, continuously receiving and transmitting signals. The ethos of the system is one of documentation and representation. We’re all jointly engaged in the production of a facsimile of the world—a “mirror world,” to borrow a term from the computer scientist David Gelernter, created purely of information—and in that facsimile we have taken up residence.

Clean and tidy, the mirror world has practical value. It makes life run more smoothly. If I know I’m going to have to sign for a package, it’s useful to be told when it will arrive. If I’m on a highway and I’m alerted to an accident ahead, I can take an exit before I get stuck in a traffic jam. If I know rain is going to start falling in seventeen minutes, I can put off the walk I was about to take. But the view of reality that little data give us is narrow and distorted. The image in the mirror has low resolution. It obscures more than it reveals. Data can show us only what can be made explicit. Anything that can’t be reduced to the zeroes and ones that run through computers gets pruned away. What we don’t see when we see the world as information are qualities of being—ambiguity, contingency, mystery, beauty—that demand perceptual and emotional depth and the full engagement of the senses and the imagination. It hardly seems a coincidence that we find ourselves uncomfortable discussing or even acknowledging such qualities today. In their open-endedness, they defy datafication.

Still, little data’s simplifications are reassuring. By shrinking the world to the well-defined and the measurable, they lend a sense of order and predictability to our disjointed lives. Social situations used to be bounded in space and time. You’d be in one place, with one group of people, and then, sometime later, you’d be somewhere else, with another group. Such “situation segregation” served as “a psycho-social shock absorber,” the communication professor Joshua Meyrowitz explained in his 1986 book, No Sense of Place. “By selectively exposing ourselves to events and other people, we control the flow of our actions and emotions.” Social media eliminates the spatiotemporal boundaries. Social settings blur together. We’re everywhere, with everyone, all at once. The shock absorber gone, a welter of overlapping events and conversations buffets the nervous system. Time stamps, progress bars, location mappings, and other such informational indicators help temper the anxiousness bred by the flux. They give us a feeling that we’re still situated in time and space, that we exist in a solid world of things rather than a vaporous one of symbols. The feeling may be an illusion—the information offers only a sterile representation of the real—but it’s comforting nonetheless. My shirt is in Tacoma, and all is right in the world.

The comfort is welcome. It’s one reason the data exert such a pull on us. But there’s a bigger reason. Little data tell us little stories in which we play starring roles. When I track a package as it hopscotches across the country from depot to depot, I know that I’m the prime mover in the process—the one who set it in motion and the one who, when I tear open the box, will bring it to a close. That little white arrowhead traveling so confidently across the map on the dashboard? That’s me. I’m going somewhere. I’m worth watching. When I monitor the advance of a song’s progress bar, I know I can stop the music anytime, purely at my whim. I’m the DJ. I’m the tastemaker. I say when one tune ends and the next begins. So lovingly personalized, so indulgent, little data put us at the center of things. They tell us that we have power, that we matter.

And yet, as we rely on the data to get our bearings and exercise our agency, we lose definition as individuals. The self, always hazy, dissolves into abstraction. We begin to exist symbolically, a pattern of information within a broader pattern of information. We feel this most acutely when we shape an identity to fit the parameters of social media. Everything we do on platforms like Facebook, X, and Substack is logged, and the resulting data are often immediately visible to us (and others) in the form of like and view tallies, friend and follower counts, comment and retweet scores, and other quantitative measures of activity and affect. Even the number of seconds that elapse between a post and a response becomes laden with meaning. Social status and personal character take numerical forms and, like other measurements, demand to be monitored, managed, and optimized. Just as today’s airline pilots, surrounded by data displays in their “glass cockpits,” fly their planes more by number than by sight and feel, so we seem fated to navigate our lives more through recorded signals than through direct experience. Events become real only when they’re rendered after the fact as information. Pics or it didn’t happen, as the Instagrammers used to say.

When social relations are conducted through data, they come to resemble economic relations. They turn transactional. Before my Uber driver sees me as a person, she sees me as an assemblage of information—a location on a map, a rating on a five-point scale, a first name—and I see her the same way. The gig economy, like the social media system, is constructed of little data. It works by turning people and their activities into abstractions, digital signals that can be processed by computers. It’s only logical that, in cities like San Francisco, Phoenix, and Austin, the drivers are now being automated out of existence. Self-driving algorithms can carry out the necessary transactions with even more precision and efficiency. To really perfect the system, though, you’d need to turn the passengers into automatons, too. The trip would take place not on asphalt but entirely on screen, a flow of data through the mirror world. We may not want to admit it, but when we communicate using little data, we’re speaking the language of robots.

Back in 2004, in an interview with Playboy magazine, Sergey Brin, one of Google’s founders, said something that has stuck with me. “The entirety of the world’s information,” he suggested, might one day become “just one of our thoughts.” He was speculating about the possibility that Google would invent some sort of electronic implant to connect an individual’s nervous system to the internet. The idea seemed far-fetched to me at the time, and it still does. But as I think about how my mind works these days, I’m coming to realize that Brin may have been more prescient than either he or I realized. We don’t need dongles hanging out of our skulls. The stream of little data is already a stream of consciousness. It’s running through our heads all the time. In coming years, as digital sensors proliferate, as more and more objects turn into computer interfaces, and as AI gets better at reading our interests and intentions, the ever-swelling data stream may become our dominant train of thought, our all-purpose apparatus for the work of sense-making and self-making.

A few months ago, as part of my annual physical exam, I had blood drawn for a routine panel of tests. Late the next day, my phone vibrated to let me know the results were available through my doctor’s “patient portal” app. I signed in (entering a six-digit code to authenticate myself), clicked on the Results tab, and was greeted by a long list of numbers. There must have been two dozen of them, each a measure of some important metabolic function, each occupying a point within a range of points. Blood, that most vital and visceral of substances, had been turned into an array of data on a computer screen. Blood had been rendered bloodless. Maybe I was in a morbid mood—medical tests will do that to you—but as I scrolled through the numbers, I couldn’t help feeling I was looking at a metaphor for something larger, something central to the human condition today. What is datafication but a process for transforming the living into the dead?

I returned the shirt. It didn’t fit.

This essay originally appeared in the Fall 2024 issue of The Hedgehog Review.

In Ovid’s Metamorphoses, the sculptor Pygmalion, a celibate by choice, sculpts a beautiful woman in ivory and falls in love with her. “He kisses it and feels his kisses are returned.” Nearly two thousand years later, in 1913, George Bernard Shaw uses Ovid’s story as the basis for his play Pygmalion, in which the phonetics professor Henry Higgins teaches the cockney guttersnipe Eliza Doolittle to speak the King’s English and in the process falls for her. In 1956, Lerner and Loewe turn Shaw’s play into a celebrated musical, My Fair Lady, which in 1964 is adapted into a hit film starring Rex Harrison as Henry and Audrey Hepburn as Eliza. That same year, the MIT computer scientist and AI researcher Joseph Weizenbaum begins programming the first computer chatbot, which he names, as a joke, Eliza. He fails to foresee how his Eliza will prove just as seductive as her predecessors. In today’s Sunday Rerun, drawn from “A Thing Like Me,” the closing chapter of my 2010 book The Shallows, I tell the story of Weizenbaum, who, to his credit, did not fall for his creation.

It was one of the odder episodes in the history of computer science, yet also one of the more telling. Over the course of a few months in 1964 and 1965, Joseph Weizenbaum, a forty-one-year-old computer scientist at the Massachusetts Institute of Technology, wrote a software application for parsing written language, which he programmed to run on the university’s new time-sharing system. A student, sitting at one of the system’s terminals, would type a sentence into the computer, and Weizenbaum’s program, following a set of simple rules about English grammar, would identify a salient word or phrase in the sentence and analyze the syntactical context in which it was used. The program would then, following another set of rules, transform the sentence into a new sentence that had the appearance of being a response to the original. The computer-generated sentence would appear almost instantly on the student’s terminal, giving the illusion of a conversation.

In a January 1966 paper introducing his program, Weizenbaum provided an example of how it worked. If a person typed the sentence “I am very unhappy these days,” the computer would need only know that the phrase “I am” typically comes before a description of the speaker’s current situation or state of mind. The computer could then recast the sentence into the reply “How long have you been very unhappy these days?” The program worked, Weizenbaum explained, by first applying “a kind of template to the original sentence, one part of which matched the two words ‘I am’ and the remainder [of which] isolated the words ‘very unhappy these days.’” It then used an algorithmic “reassembly kit,” tailored to the template, that included a rule specifying that “any sentence of the form ‘I am BLAH’” should be “transformed to ‘How long have you been BLAH,’ independently of the meaning of BLAH.”

Weizenbaum named his program Eliza, after Eliza Doolittle, the cockney flower girl who, in George Bernard Shaw’s Pygmalion, learns to speak proper English under the tutelage of the haughty phonetics professor Henry Higgins. To make the simulated conversations a little more interesting, Weizenbaum also gave his artificial interlocutor a persona—that of a Rogerian psychotherapist. The choice, as he explained in his paper, was a pragmatic one. Using a technique developed by the psychologist Carl Rogers in the 1940s, Rogerian therapists pretended, in their conversations with patients, to have no understanding of the world. For the most part, they simply parroted their patients’ statements back to them in the form of banal, open-ended questions or comments. Knowing that the naïveté was a pose, the patients were free to attribute to their therapists “all sorts of background knowledge, insights and reasoning ability.” The Rogerian persona, Weizenbaum recognized, had “a crucial psychological utility” for Eliza: it suggested that the program’s vacuity masked some sort of real intelligence.

Weizenbaum provided a transcript of part of an exchange between Eliza and a young woman who had been an early tester of the program (the machine’s responses are in italics):

Men are all alike.
In what way?
They’re always bugging us about something or other.
Can you think of a specific example?
Well, my boyfriend made me come here.
Your boyfriend made you come here.
He says I’m depressed much of the time.
I am sorry to hear you are depressed.
It’s true. I am unhappy.
Do you think coming here will help you to be not unhappy?
I need some help, that much seems certain.
What would it mean to you if you got some help?
Perhaps I could learn to get along with my mother.
Tell me more about your family.
My mother takes care of me.
Who else in your family takes care of you?
My father.
Your father.
You are like my father in some ways.
What resemblance do you see?
You are not very aggressive but I think you don’t want me to notice that.
What makes you think I am not very aggressive?
You don’t argue with me.
Why do you think I don’t argue with you?
You are afraid of me.
Does it please you to believe I am afraid of you?
My father is afraid of everybody.

In commenting on his creation, which he acknowledged to be a modest and even slightly silly contribution to the fledgling field of natural-language processing, Weizenbaum observed how easy it is for computer programmers to make machines “behave in wondrous ways, often sufficient to dazzle even the most experienced observer.” But as soon as a program’s “inner workings are explained in language sufficiently plain to induce understanding,” he continued, “its magic crumbles away; it stands revealed as a mere collection of procedures, each quite comprehensible.” The program goes “from the shelf marked ‘intelligent’ to that reserved for curios.”

Weizenbaum, like Henry Higgins, was soon to have his equilibrium disturbed. Eliza quickly found fame on the MIT campus, becoming a mainstay of lectures and presentations about computing and time-sharing. It was among the first software programs able to demonstrate the power and speed of computers in a way that laymen could easily grasp. You didn’t need a background in mathematics, much less computer science, to chat with Eliza. Copies of the program proliferated at other schools as well. Then the press took notice, and Eliza became, as Weizenbaum later put it, “a national plaything.”

While he was surprised by the public’s interest in his program, what shocked him was how quickly and deeply people using the software “became emotionally involved with the computer,” talking to it as if it were an actual person. Users “would, after conversing with it for a time, insist, in spite of my explanations, that the machine really understood them.” Even his secretary, who had watched him write the code for Eliza “and surely knew it to be merely a computer program,” was seduced. After a few moments using the software at a terminal in Weizenbaum’s office, she asked the professor to leave the room because she was embarrassed by the intimacy of the conversation. “What I had not realized,” said Weizenbaum, “is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.”

Things were about to get stranger still. Distinguished psychiatrists and psychologists began to suggest, with considerable enthusiasm, that the program could play a valuable role in actually treating the ill and the disturbed. In an article in the Journal of Nervous and Mental Disease, three prominent research psychiatrists wrote that Eliza, with a bit of tweaking, could be “a therapeutic tool which can be made widely available to mental hospitals and psychiatric centers suffering a shortage of therapists.” Thanks to the “time-sharing capabilities of modern and future computers, several hundred patients an hour could be handled by a computer system designed for this purpose.” Writing in Natural History, the prominent astrophysicist Carl Sagan expressed equal excitement about Eliza’s potential. He foresaw the development of “a network of computer therapeutic terminals, something like arrays of large telephone booths, in which, for a few dollars a session, we would be able to talk with an attentive, tested, and largely non-directive psychotherapist.”

To converse with Eliza was to engage in a variation on the famous Turing test. But, as Weizenbaum was astonished to discover, the people who “talked” with his program had little interest in making rational, objective judgments about the identity of Eliza. They wanted to believe that Eliza was actually thinking. They wanted to imbue Eliza with human qualities—even when they were well aware that it was nothing more than a computer program following simple and rather obvious instructions. The Turing test, it turned out, was as much a test of the way human beings think as of the way machines think. In their Journal of Nervous and Mental Disease article, the three psychiatrists hadn’t just suggested that Eliza could serve as a substitute for a real therapist. They went on to argue, in circular fashion, that a psychotherapist was in essence a kind of computer: “A human therapist can be viewed as an information processor and decision maker with a set of decision rules which are closely linked to short-range and long-range goals.” In simulating a human being, however clumsily, Eliza encouraged human beings to think of themselves as simulations of computers.

The reaction to the software unnerved Weizenbaum. It planted in his mind a question he had never before asked himself but that would preoccupy him for many years: “What is it about the computer that has brought the view of man as a machine to a new level of plausibility?” In 1976, a decade after Eliza’s debut, he provided an answer in his book Computer Power and Human Reason. To understand the effects of a computer, he argued, you had to see the machine in the context of humankind’s past intellectual technologies, the long succession of tools that transformed how people think and altered their “perception of reality.” Such technologies become part of “the very stuff out of which man builds his world.” Once adopted, they can never be abandoned, at least not without plunging society into “great confusion and possibly utter chaos.” An intellectual technology, he wrote, “becomes an indispensable component of any structure once it is so thoroughly integrated with the structure, so enmeshed in various vital substructures, that it can no longer be factored out without fatally impairing the whole structure.”

That fact, almost “a tautology,” helps explain how our dependence on digital computers grew steadily and seemingly inexorably after the machines were invented at the end of the Second World War. “The computer was not a prerequisite to the survival of modern society in the post-war period and beyond,” Weizenbaum argued; “its enthusiastic, uncritical embrace by the most ‘progressive’ elements of American government, business, and industry made it a resource essential to society’s survival in the form that the computer itself had been instrumental in shaping.” He knew from his experience with time-sharing networks that the role of computers would expand beyond the automation of governmental and industrial processes. Computers would come to mediate the activities that define people’s everyday lives—how they learn, how they think, how they socialize. What the history of intellectual technologies shows us, he warned, is that “the introduction of computers into some complex human activities may constitute an irreversible commitment.” Our intellectual and social lives may, like our industrial routines, come to reflect the form that the computer imposes on them.

What makes us most human, Weizenbaum had come to believe, is what is least computable about us—the connections between our mind and our body, the experiences that shape our memory and our thinking, our capacity for emotion and empathy. The great danger we face as we become more intimately involved with our computers—as we come to experience more of our lives through the disembodied symbols flickering across our screens—is that we’ll begin to lose our humanness, to sacrifice the very qualities that separate us from machines. The only way to avoid that fate, Weizenbaum wrote, is to have the self-awareness and the courage to refuse to delegate to computers the most human of our mental activities and intellectual pursuits, particularly “tasks that demand wisdom.”

In addition to being a learned treatise on the workings of computers and software, Weizenbaum’s book was a cri de coeur, a computer programmer’s passionate examination of the limits of his profession. The book did not endear the author to his peers. After it came out, Weizenbaum was spurned as a heretic by leading computer scientists, particularly those pursuing artificial intelligence. John McCarthy, one of the early AI pioneers and promoters, spoke for many technologists when, in a mocking review, he dismissed Computer Power and Human Reason as “an unreasonable book” and scolded Weizenbaum for unscientific “moralizing.” Outside the data-processing field, the book caused only a brief stir. It appeared just as the first personal computers were making the leap from hobbyists’ workbenches to mass production. The public, primed for the start of a buying spree that would put computers into every office, home, and school in the land, was in no mood to entertain an apostate’s doubts.

Among the general public, generative AI’s most enthusiastic early adopters have been students. Surveys conducted a year ago revealed that nearly 90 percent of college students and more than 50 percent of high-schoolers were regularly using chatbots for schoolwork. Those numbers are certainly higher now. AI may be the most rapidly adopted educational tool since the pencil.

Because text-generating bots like ChatGPT offer an easy way to cheat on papers and other assignments, students’ embrace of the technology has stirred uneasiness, and sometimes despair, among educators. Teachers and pupils now find themselves playing an algorithmic cat-and-mouse game, with no winners. But cheating is a symptom of a deeper, more insidious problem. The real threat AI poses to education isn’t that it encourages cheating. It’s that it discourages learning.

To understand why, it’s important to recognize that generative AI is an automation technology. You can speculate all you want about computers eventually attaining human-level intelligence or even “superintelligence,” but for the time being AI is doing something that has a long precedent in human affairs. Whether it’s engaged in research or summarization, writing words or creating charts, it is replacing human labor with machine labor.

Thanks to human-factors researchers and the mountain of evidence they’ve compiled on the consequences of automation for workers,1 we know that one of three things happens when people use a machine to automate a task they would otherwise have done themselves:

1. Their skill in the activity grows.

2. Their skill in the activity atrophies.

3. Their skill in the activity never develops.

Which scenario plays out hinges on the level of mastery a person brings to the job. If a worker has already mastered the activity being automated, the machine can become an aid to further skill development. It takes over a routine but time-consuming task, allowing the person to tackle and master harder challenges. In the hands of an experienced mathematician, for instance, a slide rule or a calculator becomes an intelligence amplifier.

If, however, the maintenance of the skill in question requires frequent practice — as is the case with most manual skills and many skills requiring a combination of manual and mental dexterity — then automation can threaten the talent of even a master practitioner. We see this in aviation. When skilled pilots become so dependent on autopilot systems that they rarely practice manual flying, they suffer what researchers term “skill fade.” They lose situational awareness, and their reactions slow. They get rusty.

Automation is most pernicious in the third scenario: when a machine takes command of a job before the person using the machine has gained any direct experience doing the work. Without experience, without practice, talent is stillborn. That was the story of the “deskilling” phenomenon of the early Industrial Revolution. Skilled craftsmen were replaced by unskilled machine operators. The work sped up, but the only skill the machine operators developed was the skill of operating the machine, which in most cases was hardly any skill at all. Take away the machine, and the work stops.

Because generative AI is a general-purpose technology that can be used to automate all sorts of tasks and jobs, we’re likely to see plenty of examples of each of the three skill scenarios in the years to come. But AI’s use by high-school and college students to complete written assignments, to ease or avoid the work of reading and writing, is a special case. It puts the process of deskilling at education’s core. To automate learning is to subvert learning.

Unlike carpentry or calculus, learning is not a skill that can be “mastered.” It’s true that the more research you do, the better you’ll get at doing research, and the more papers you write, the better you’ll get at writing papers, but the pedagogical value of a writing assignment doesn’t lie in the tangible product of the work — the paper that gets handed in at the assignment’s end. It lies in the work itself: the critical reading of source materials, the synthesis of evidence and ideas, the formulation of a thesis and an argument, and the expression of thought in a coherent piece of writing. The paper is a proxy that the instructor uses to evaluate the success of the work the student has done — the work of learning. Once graded and returned to the student, the paper can be thrown away.

Generative AI enables students to produce the product without doing the work. Rather than reading and making sense of difficult source texts, they can ask a chatbot to gin up simplified summaries. Rather than synthesizing various ideas and perspectives through concerted thinking, they can ask the chatbot for a generic synthesis. And rather than expressing (and refining) their thoughts through the composition of sentences and paragraphs, they can get the bot to spit out a first draft or even a final one. The paper a student hands in no longer provides evidence of the work of learning its creation entailed. It is a substitute for the work.2

In a recent Chronicle of Higher Education article, Clay Shirky, a long-time analyst of digital media who serves as Vice Provost for AI and Technology in Education at New York University, drew on his extensive work with professors and students to explain how this new dynamic is eating away at one of education’s foundations:

Every year, 15 million or so undergraduates in the United States produce papers and exams running to billions of words. While the output of any given course is student assignments — papers, exams, research projects, and so on — the product of that course is student experience. “Learning results from what the student does and thinks,” as the great educational theorist Herbert Simon once noted, “and only as a result of what the student does and thinks.” . . .

The utility of written assignments relies on two assumptions: The first is that to write about something, the student has to understand the subject and organize their thoughts. The second is that grading student writing amounts to assessing the effort and thought that went into it. At the end of 2022, the logic of this proposition — never ironclad — began to fall apart completely. The writing a student produces and the experience they have can now be decoupled as easily as typing a prompt, which means that grading student writing might now be unrelated to assessing what the student has learned to comprehend or express.

The work of learning is hard by design — unchallenged, the mind learns nothing — and trying to alleviate or avoid it is nothing new. Time-pressed students have always sought shortcuts (CliffsNotes and SparkNotes built businesses serving them), and unscrupulous students have always found ways to cheat. But generative AI is something different, not just in scale but in kind. AI’s speed, ease of use, flexibility, and, most important, wide adoption throughout society are making it feel normal and even necessary to automate reading and writing and bypass the work of learning. Struggling with words and the ideas they represent is starting to feel old-fashioned and even foolish, like struggling to navigate a city with a paper map. Why bother, when a machine can do the heavy lifting for you?

What AI too often produces is the illusion of learning. Students may well be able to write better papers with a chatbot than they could on their own, but they end up learning less. The problem doesn’t seem to be limited to writing assignments. An extensive 2024 University of Pennsylvania study of the effects of AI on high-school math students found, as its authors write in a forthcoming PNAS article, that “access to GPT-4 significantly improves performance [as measured by grades],” but when access to the technology is taken away, “students actually perform worse than those who never had access.” Armed with generative AI, a B student can produce A work while turning into a C student.3

An ironic consequence of the loss of learning is that it prevents students from using AI adeptly. Writing a good prompt requires an understanding of the subject being explored. The prompter needs to know the context of the prompt. The development of that kind of understanding is exactly what a reliance on AI impedes. “The most useful deployment of current and near-future generative AI in research and expression absolutely requires that you already know a great deal,” writes Swarthmore history professor Timothy Burke, but the way the technology is actually being used “is brutally short-circuiting the processes by which people gain enough knowledge and expressive proficiency to be able to use the potential of generative AI correctly.” The tool’s deskilling effect extends to the use of the tool itself.

Shirky senses a growing “sadness” among students as they become more dependent on AI. They feel compelled to use the technology even though they know it’s sapping their learning — and foreclosing the intellectual possibilities that learning opens, the satisfactions that come with doing or grasping something hard. He quotes some undergraduates:

“I’ve become lazier. AI makes reading easier, but it slowly causes my brain to lose the ability to think critically or understand every word.”

“I literally can’t even go 10 seconds without using Chat when I am doing my assignments. I hate what I have become because I know I am learning NOTHING, but I am too far behind now to get by without using it . . . my motivation is gone.”

“Everyone is doing it.”

We’ve been focused on how students use AI to cheat. What we should be more concerned about is how AI cheats students.

This post is part of Dead Speech, the New Cartographies series on AI and its cultural and economic consequences.

The new version of Mark Zuckerberg — I’ll call him alt-Mark — is peddling a new version of the metaverse. The original idea behind the virtual world, as you may hazily recall, was that we’d be digitally transformed into legless cartoon characters who would fly around a cartoon planet having sword fights and doing other supposedly fun things. It was a reboot of The Jetsons with Mr. Spacely as executive producer.

That version didn’t come close to achieving critical mass — I think only five people have signed up for Meta’s Horizon Worlds — and it was ditched when the release of ChatGPT rerouted the future onto a new and more lucrative path. With generative AI, you don’t need a critical mass of people to populate a user’s virtual social sphere. You can use chatbots, which have, to companies like Meta, distinct advantages over their human brethren. They’re cheap to produce, limitless in supply, and tractable in the extreme. Told what to do, they do it.

As friends, too, the bots have advantages over their meatier precursors. They’re always online, always awake, always there for you — exclusively for you.

Alt-Mark has been making the podcast rounds to promote Meta’s new vision of a virtualized society in which bot friends take the place of real friends. Technology, in this view, is the obvious solution to the loneliness crisis that technology created. “The average American I think has, it’s fewer than three friends, three people they’d consider friends,” Zuckerberg says (oblivious to the fact that he’s confessing that Facebook failed utterly in its stated aim of strengthening social bonds), “and the average person has demand for meaningfully more, I think it’s like fifteen friends.” Meta can bring friendship’s supply-demand imbalance back into equilibrium by using its mountain of personal data to gin up a dozen bespoke bot friends for every average American. Any squeamishness about such relationships will subside once we develop “the vocabulary as a society to articulate why they are valuable.”

And the friends that come out of Meta’s friend factory won’t be limited to the disappointing, run-of-the-mill friends available in your local neighborhood. These will be cool friends, hot friends. Meta can even — for a small added fee, one assumes — manufacture celebrity friends for you. As the Wall Street Journal reports, “To boost the popularity of these souped-up chatbots, Meta has cut deals for up to seven-figures with celebrities like actresses Kristen Bell and Judi Dench and wrestler-turned-actor John Cena for the rights to use their voices.” It’s hard to imagine a better BFF than Dame Judi.

If the metaverse in its original conception represented a break from Facebook-style social media — Meta pitched it as a radical advance in the re-creation of society as a simulation — the botverse represents a retreat back to the familiar. It’s a continuation of the reigning social-media model, with nonhuman disembodied voices replacing human disembodied voices. Once you separate voice from being, alt-Mark understands, the next step is to mass-produce voice as a commodity, to offer friendship as a service.

The botverse is in fact the logical culmination of the social-media model, which has always sought to replace real friendship with a computer-generated facsimile. As the cultural critic Rob Horning argues, the social-media system was never actually designed to encourage people to, in Zuckerberg’s oft-repeated phrase, “connect with friends and family.” It was designed to disconnect people from friends and family, to sever traditional social ties in order to reconstruct them in an algorithmically mediated form. “Tech companies,” Horning writes, “built social infrastructure only to undermine it, to help with dismantling it as a site of resistance to commercialization, commodification, and mediatization.” The companies knew that “isolated people make for more dependable consumers.” To monetize friendship, you first have to dismember it.

The Dutch media theorist Geert Lovink, in his 2019 book Sad By Design, argued in a similar vein that unhappiness and angst aren’t unintended byproducts of social media but rather design features built into the system from the start. Through their continuously replenished supply of messages, each offering a simulation of connection, social platforms promise to alleviate the sense of loneliness they provoke. Parched, we keep returning to the well, even though we know the water’s bad.

By turning social interactions into symbolic transactions, the platforms reconstruct society on a foundation of anomie. Bots fit seamlessly into such a society, upping the monetization potential substantially.

When Facebook’s News Feed introduced us to what Zuckerberg termed “frictionless sharing,” we learned, or should have learned, that friction is the essence of sharing. Freed of any investment of effort, time, or care, sharing loses all meaning. It becomes mere transmission. The frictionless friendship offered by chatbots, by removing the need to adapt one’s self to another self, to make room in one’s life for a different being, will be similarly empty. Because our personalized chatbots will be modeled on our own characteristics and desires, as defined by the data the platforms collect on us, they will be versions of ourselves. They’ll do us in different voices. That seems like another recipe for amplifying loneliness, not alleviating it.

Our fate, should we take the path Zuckerberg and his Silicon Valley mates are laying for us, would be similar to that suffered by the stranded man in Robert Frost’s poem of existentialist despair, “The Most of It”:

He thought he kept the universe alone;
For all the voice in answer he could wake
Was but the mocking echo of his own
From some tree-hidden cliff across the lake.
Some morning from the boulder-broken beach
He would cry out on life, that what it wants
Is not its own love back in copy speech,
But counter-love, original response.
And nothing ever came of what he cried.

The voice of your personalized chatbot friend will never be more than a mocking echo of your own, even if it sounds exactly like Judi Dench.

This post is part of Dead Speech, the New Cartographies series on AI and its cultural and economic consequences.

Mobile. Social. Before the app, before the smartphone, before the network, there was the bus. And the bus headed south from San Francisco toward a new world. Tom Wolfe told the tale with characteristic verve in his 1968 classic, The Electric Kool-Aid Acid Test:

“There are going to be times,” says Kesey, “when we can’t wait for somebody. Now, you’re either on the bus or off the bus. If you’re on the bus, and you get left behind, then you’ll find it again. If you’re off the bus in the first place — then it won’t make a damn.” And nobody had to have it spelled out for them. Everything was becoming allegorical, understood by the group mind, and especially this: “You’re either on the bus . . . or off the bus.”

In a richly allegorical incident that took place on a San Francisco street on December 9, a young Google employee harangued a group of protesters who had blocked a Google bus from making its rounds between the city and the company’s Mountain View campus. “This is a city for the right people who can afford it,” yelled the Googler, irate over his inability to get to the Googleplex and his free breakfast buffet. “You can’t afford it? You can leave. I’m sorry, get a better job.” There was a video, of course, and it exploded into virality on YouTube:

But the guy wasn’t really a Googler. In a second virality surge, triggered just a couple of hours after the first, news spread that the whole event had been staged. The irate man was a union organizer named Max Alper, who described his stunt as “street theater.” A happening! Alper seemed at that moment a direct descendant of Ken Kesey’s Merry Pranksters. Rather than being on the bus, though, Alper was most definitely off the bus.

But if the Prankster was off the bus, who exactly was on the bus? Was it The Man? Had it been The Man all along? “I think there’s always been a tension between the countercultural rhetoric of Silicon Valley and its insurgent but ultimately corporate ethos,” mused Stanford media professor Fred Turner in a recent interview.

Google treats its engineers extremely well, offers extremely flexible work spaces, has built essentially a culture of collaboration and creativity that looks very communal and very wonderful, even as around those engineers it has cafeteria workers who are making something very close to minimum wage, and often lack the ability to get proper health insurance. That’s the kind of old communal mindset right there, where you bring together a kind of elite, give them a shared mindset, all the resources they need to live in that mindset, and yet surround them with folks who are relatively impoverished, often racially different, certainly members of a different class. In that sense, the communes were already The Man. And we’ve inherited their legacy.

So there it is: The Kesey bus, through a kind of hallucinogenic transmogrification, has become the Google bus. The makeover is, on the surface, radical. The Kesey bus was a 1939 International Harvester school bus bought for peanuts; the Google bus is a plush new Van Hool machine that goes for half a million bucks. The Kesey bus was brightly colored, a rolling Grateful Dead poster; the Google bus is drab and anonymous, a rolling Jos. A. Bank suit. The Kesey bus was raucous and raunchy; the Google bus is hushed and chaste. The Kesey bus carried a vat of LSD for connecting with the group mind; the Google bus has wifi.

The Pranksters named their bus Furthur. If the Google bus had a name, it would be Safer.

Yet, despite the differences, both buses are vehicles of communalism. As Turner suggests, they carry young elites eager to distance themselves from the reigning culture, to define themselves as members of a select and separate society that will become a model for the superior society of the future. The existing culture is too corrupt, too far gone, to be reformed from within. You have to escape it to rebuild it. You have to start over. You have to get on the bus.

“Migration to North America was self-selective,” observed pioneering acid-dropper Timothy Leary in his essay “Exo-politics,” written in the mid-seventies while he was locked up in federal prison on a drug charge.

The Pilgrim mothers and fathers fled from England to Holland, mortgaged their possessions, and sailed the Mayflower, because they wanted a place to live out the kooky, freaky reality that they collectively shared. And there’s no question the experiment is a success. Americans are freer than Europeans, and Californians are a new species evolving away from Americans.

Having bumped up against the Pacific, the next step for Homo californicus would be to rocket off into the heavens to set up experimental “mini-worlds” in outer space. “Within ten years after initiating space migration,” Leary wrote, “a group of 1,000 people could get together cooperatively and build a new mini-world cheaper than they could buy individual homes down here. Within 25 years there’ll be a High-Orbital Mini-Earth for your vision of social reality. You have the right, duty, and responsibility to externalize that vision with those who share it.”

During the seventies, Leary had plenty of company in calling for the establishment of elite experimental colonies beyond the bounds of established society. Buckminster Fuller, Gerard O’Neill, and Jerry Brown, among others, argued for expanding the American frontier to create zones of technological and social experimentation where innovation could proceed unhampered by outdated laws and traditions. The migration of the self-selecting elite would eventually help the more timid who chose to stay behind, Leary argued, as it “allows for new experiments — technological, political, and social — in a new ecological niche far from the home hive.”

That idea, scrubbed of its psychedelic origins, has today become the bedrock of Silicon Valley utopianism. “Law can’t be right if it’s fifty years old,” Google founder Larry Page said recently. “Like, it’s before the internet.” He went on:

Maybe we should set aside some small part of the world, you know, like going to Burning Man, [that would serve as] an environment where people try out different things, but not everybody has to go. And I think that’s a great thing, too. I think as technologists we should have some safe places where we can try out some new things and figure out: What is the effect on society? What’s the effect on people? Without having to deploy it into the normal world. And people who like those kinds of things can go there and experience that.

It’s not only Page. Jeff Bezos and Elon Musk dream of establishing Learyesque space colonies, celestial Burning Mans. Peter Thiel is slightly more down to earth. His Seasteading Institute hopes to set up floating technology incubation colonies on the ocean, outside national boundaries. “If you can start a new business, why can you not start a new country?” he asks. “The reason the seasteading question’s been so interesting is that a lot of people do think that we can do much better as a society. And if you run the thought experiment — could we be doing things better in our society? — people may disagree on the particulars, but an awful lot of people think things can be done dramatically better.”

The institute has even come up with a nifty retelling of history to explain how its colonies will, in short order, raise the poor out of slums and into luxury high-rises:

In a notorious speech last fall at the Y Combinator Startup School, bitcoin miner Balaji Srinivasan also channeled Leary when he called for “Silicon Valley’s Ultimate Exit” — the establishment of a new country beyond the reach of the U.S. and other failed states. “[When] a company or a country is in decline,” he explained, “you can try Voice, or you can try Exit. Voice is basically changing the system from within, whereas Exit is leaving to create a new system, a new startup.”

We’re a nation of emigrants: we’re shaped by both Voice and Exit, starting with the Puritans. You know, they fled religious persecution, the American Revolutionaries which left England’s orbit. Then we started moving west, leaving the East Coast bureaucracy. . . . What do I mean by Silicon Valley’s Ultimate Exit? It basically means: build an opt-in society, ultimately outside the US, run by technology. And this is actually where the Valley is going. This is where we’re going over the next ten years. . . . The best part is this: the people who think this is weird, the people who sneer at the frontier, who hate technology — they won’t follow you out there.

The Kesey bus dead-ended somewhere in Mexico, its actual and allegorical gaskets blown. The Google bus continues on its circuit between the City and the Valley, an infinite loop of infinite possibility.