There is No "Human-Centered 'AI'"

Two recently-published reports use the same phrase – “human-centered AI” – urging schools to adopt automated and predictive technologies that, as The 74’s Greg Toppo reports, “serve human-centered learning [and] that doesn’t simply push for more efficiency. To do anything else risks creating a generation of young people ill-equipped for the future.”

“Human-centered ‘AI’.” What the hell does that even mean?

As you might guess, whenever these sorts of reports are released -- one sponsored by an AI training group and one by an education reform think-tank – it means “get out your wallet.” It means “AI literacy” and “AI training” and the pretense of a kinder and friendly ed-tech future, one bursting with “innovation” and “redesign” and (of course) “workforce readiness.” We can’t let “big tech” dictate the shape of “AI” in schools, these reports suggest; we can’t let the cult of efficiency drive decision-making – schools’ or students’. But I’m not sure we can take their assertions all that seriously because in the end, they insist that “AI” is inevitable, and schools and students have little choice but to submit. There's no space for refusal.

“Human-centered.” It's a powerful adjectival phrase, cleverly wielded here; and much like “personalized” in “personalized learning”, it appeals to people who haven’t looked all that closely at the fine print. One can readily append these terms – "personalized," "human-centered" – in front of a technology, in front of a product and obscure that what’s happening is actually anything but.

“Human-centered ‘AI’.” Which humans are we talking about here? Surely not the “ghost workers” tasked with labeling and training these systems -- who perform critical, skilled work with low pay, no benefits, no job security and who are exposed daily to violent, harmful content. Surely not the communities who live near data centers and their power plants, who suffer from soaring energy costs and environmental pollution. Surely not the people who are victims of AI-generated CSAM -- a growing problem among children and at school. Surely not the families who are being targeted by ICE’s adoption of AI surveillance tools. Surely not those who’ve seen loved ones triggered by chatbots into destructive, delusional thinking, into suicide. Surely not the workers being told they're being replaced. Surely not those designated as targets by the military’s use of AI, including, yes, Anthropic’s Claude (“Anthropic has much more in common with the Department of War than we have differences,” Anthropic’s CEO wrote this week.)

Which “humans” are going to receive some sort of “human-centered ‘AI’”? I dare say none of us.

“But what about the good uses of computers?” People ask me this question all the time, often complaining that I don't write enough nice things about ed-tech. There must be good, they plead. Please let there be something good.

It’s the wrong question and perhaps even the wrong impulse – all predicated on the ideology of the computer as a neutral object, as a piece of malleable clay that can be shaped and reshaped, bent towards the desires of the users and away from the designs of industry, away from its original mission: a weapon of war.

Since giving my talk on Wednesday to a group of retired teachers, I’m still thinking about the stories we tell about computers and “AI” and the ways in which these almost inevitably diminish our belief in our own human capacities. Of course, that’s a crucial part of the marketing for “AI” – it is a technology of ranking machine over human in no small part because “intelligence” is entangled with eugenics, with ranking certain humans over others. These stories don’t just occur in science fiction; they’re part of policy initiatives and policy rationales too. They’re core to the neoliberal project, so well-documented in Daniel Greene’s book The Promise of Access, that has come to dominate how we think about public institutions like schools and libraries: no need to fund these, no need to staff these, as everyone can just use the computer and the Internet instead.

We could structure society differently. We could have different funding priorities, different staffing priorities. We could have smaller classrooms. We could have more certified teachers and translators and aides in each of them. We could have more librarians and more nurses. Every child could have their own tutor, their own human tutor. Why the hell not?

Because we don't believe we can. We can, but we're told repeatedly that the best we can hope for is "human-centered 'AI'" or some such expensive, inferior substitute.

It always strikes me as such an utter failure of the imagination when people dust off some Cold War-era science fiction fantasy about the future. These are old stories. These are old visions. They're not that great! And none of the gadgetry supposedly inspired by these stories is all that great either. None of this “AI” stuff really works reliably. (Apps, WiFi, phones, servers, websites, laptops, printers – they're all janky AF.) And yet the people who wave their hands and talk about some magical “AI” future insist they're the realists; and the ones who want to fund schools and not the military, who want to hire teachers not buy tech gadgets, who want to build a future that cares for people not profits – we’re the dreamers; we’re the crazy ones.

Elsewhere in anti-education technologies:

• “Why Suno Won't Produce the Next Charlie Parker” by Tim Requarth

In a better world, individual creation and creative culture would reinforce each other — more people making music would mean more people immersed in music, which would mean richer scenes, which would produce more extraordinary musicians. But that’s not the world Suno is entering. The democratization pitch works because it’s a bit of rhetorical legerdemain, treating individual content creation and creative culture as synonyms, as if what Gong does with a prompt and what Parker did in New York City are just different points on the same spectrum. I don’t think they are. And in the ecosystem that actually exists, I think a very real concern is that one is going to eat the other.

• “Grammarly is using our identities without permission,” writes Stevie Bonifield for The Verge. The plagiarism machine just keeps spinning. Good thing no one is pushing for Grammarly in schools.
• “‘Creepy’ AI tool now uses dead teachers to review your work” by Connor Bennett
• Via Cal Matters: “California colleges spend millions on faulty AI systems: ‘The chatbot is outdated’”
• “MrBeast Is Getting Into Financial Services. Parents Should Pay Attention,” Tara Siegel Bernard and Ron Lieber report for The NYT
• The more you watch short-form videos, the poorer your cognition and the shorter your attention. (Good thing no one has been pushing for ever shorter video content in education. Phew.) So says some new research – “Feeds, Feelings, and Focus: A Systematic Review and Meta-Analysis Examining the Cognitive and Mental Health Correlates of Short-Form Video Use” – from Lan Nguyen, Jared Walters, Siddharth Paul, Shay Monreal Ijurco, Georgia E. Rainey, Nupur Parekh, Gabriel Blair, and Miranda Darrah.
• Via The 74: “Study Links Increased Broadband Access to Suicide Risk Among Teens”
• “Schools are using AI counselors to track students’ mental health. Is it safe?” asks Edsurge’s Daniel Mollencamp in The Guardian. We don’t need Betteridge’s Law of Headlines to tell us the answer, do we?
• Dave Karpf also asks the obvious: “Can AI Replace Social Science Researchers?”
• “I Don’t Believe This Finding That AI Is Saving Teachers Six Weeks per Year,” writes Dan Meyer. Six weeks?! LOL. I don’t even need to math anything to tell that’s some bullshit.

Today’s bird is the northern bald ibis. Once widespread across the Middle East, northern Africa, and southern Europe, the bird has nearly vanished. The first decrees to protect the bird were issued in the sixteenth century; but today almost the entire breeding population resides at Souss-Massa National Park in Morocco. According to Wikipedia, “Religious traditions helped this species to survive in one Turkish colony long after the species had disappeared from Europe, since it was believed that the ibis migrated each year to guide Hajj pilgrims to Mecca.”

Thanks for reading Second Breakfast. Please consider becoming a paid subscriber as your support makes this work possible.