The Final Boss

This morning I attended one of the new NYC Chancellor's public "conversations," his administration's initiative to "engage directly with communities to reflect on what safety, academic rigor, and true integration look like in practice." There were about one hundred folks in attendance, including members of the AI Moratorium for NYC schools, who were there to leaflet beforehand (and were vastly outnumbered, I should note, by the NYPD).

As the aforementioned name suggests, this coalition of local organizations is asking for a two-year moratorium on AI in the city's schools, pointing to the growing opposition to AI and (in their words) "to evidence that it represents substantial risk to student privacy, cognitive development and skills, critical thinking, creativity, mental health, and the environment." I'd add that it represents substantial risk more broadly: to labor (teachers', librarians', translators', social workers') and to democracy itself.

And really, what's the rush?! I mean, other than the desperate need of the tech sector to prove that the trillions of dollars invested in this endeavor will soon show some profit and that – unlike crypto and Web 3.0 – this isn't just some giant fraud being perpetrated so executives can buy more private islands.

I've said repeatedly (but didn't articulate into any open mic at the meeting because I still very much feel like a new New Yorker), this recent push for "AI" is yet another grandiose and grotesque experiment on children – one that no one asked for and few want. Another grandiose and grotesque experiment on all of us.

We have lived through decades and decades now of repeated digital promises -- we'll be better, faster, stronger, more connected, what have you -- and none of the computational fantasies have really come to fruition, certainly not for everyone. We are not more productive (despite now being asked to work so much more, clicking away on our devices at all hours of every day); we are not smarter; and most importantly, we are not better. (A tiny group of men are, on the other hand, now richer than any other humans have ever been in all of history. So there's that.) Our public institutions are crumbling, in no small part because these men are fully and openly committed to the failure of democracy, having positioned themselves to profit mightily from years of neoliberalism. "AI" marks the further (and they hope, final) consolidation of their power – not just the privatization and monopolization of all information under their control, but the automation of the dissemination and replication of knowledge. These men are more than happy to sell a story, a system that trains all of us, but particularly young people, to become entirely dependent on and subservient to computational machinery; they are more than happy for us to sacrifice our cognitive capabilities, our creativity, our agency, our decision-making, our morality, to solidify their crude oligarchal dreams of total efficiency, total financialization, total domination.

Jennifer Berkshire writes about the back history to the growing backlash against not just "AI" but a lot of ed-tech and what she calls “the curious case of collective amnesia" (invoking one of Hack Education's enduring contributions to "the discourse: "The 100 Worst Ed-Tech Debacles of the Decade" as well as Teaching Machines).

We should know by now that this stuff is almost entirely wretched – we do, right? I mean, at this stage, I'd be deeply embarrassed if I was out there, trying to argue that this stuff is any damn good. And yet here comes Silicon Valley and education reform, hand-in-hand once again, trying to peddle disruption and innovation and their long war on "one size fits all education," armed with their algorithmic bullshit and billionaire board members.

It doesn't help, I think, that there are several prominent technology journalists who keep falling for / perpetuating this stuff, who loudly insist in caps-lock-on prose that "THERE IS NO EVIDENCE!!!111" that devices are bad for children. (The irony, of course, is after they repeat this claim -- and with such certainty -- they turn around and point to dozens of stories of the most batshit-crazy news about the horrors of digital culture.)

And maybe part of the problem too is just that: we are so steeped in the insanity of techno-capitalism, the insanity of techno-capitalists that some folks are losing track of what aberrant behavior really is. Cory Doctorow writes a bit about this this week, offering "three more AI Psychoses" -- a response, in part, to Samantha Cole's excellent piece in 404 Media, “How to Talk to Someone Experiencing 'AI Psychosis’."

I wonder if it isn't simply that "AI" delusions are ubiquitous (at this stage, I'm thinking these delusions are experienced by almost everyone, not just a tiny fraction of "AI" users); it's that many of these delusions are unrecognizable as such because they reflect precisely the sort of sociopathy long embraced by Silicon Valley's Ayn-Randian, libertarian set. "Here's to the crazy ones" indeed.

[A] great embarrassing fact… haunts all attempts to represent the market as the highest form of human freedom: that historically, impersonal, commercial markets originate in theft. – David Graeber, Debt

Via Wired: “Grammarly Is Facing a Class Action Lawsuit Over Its AI ‘Expert Review’ Feature.” ("Vindicated At Last In My Years-Long Loathing Of Grammarly," writes Sabrina Imbler.)

If plagiarism is wrong and bad and theft is wrong and bad and schools are duty-bound to help instill these values in students, how can they justify adoption of a technology that is, at its core, built on stolen work and whose purpose is the extrusion of text to be passed off as one's own thinking and writing?

Max Read on what those "Which is AI?" quizzes actually tell us:

Quizzes like the Times’ are games that L.L.M.s are designed to excel at. A.I. writing is literally optimized to be the writing most people prefer in A/B preference tests; the main thing an L.L.M. chatbot “wants” when replying is to be generating the text that its users would choose as a good answer to the prompt over all other possibilities.

In a world of techno-authoritarianism, everyone wants to be banal. No one wants to be noticed; no one can afford to be different; no one dares challenge; no one dares be challenging.

The Hour of Code is now the Hour of AI. Of course it is.

Every time Code.org stuff crosses my desk, I always check to see if its leader, Hadi Parvoti, has severed ties with Axon, the police technology / surveillance company where he has been on the Board of Directors.

And no. No, he has not.

Via The New Yorker: “Shot by Border Patrol, Then Called a ‘Domestic Terrorist’” -- the story of preschool teacher and US citizen Marimar Martinez.

This is what "AI" in education looks like. And to be clear, it doesn't just look like cop shit. This isn't a metaphor. It literally is cop shit.

Via the BBC: "AI toys for children misread emotions and respond inappropriately, researchers warn." And AI toys that are labeled as ed-tech?

Please stop it with all the talk about "AI guard rails” ensuring your school's implementation is "safe." This phrase – "guard rails" – is mostly marketing language, promoted by companies hoping to avoid scrutiny and regulation.

LLMs are trained on violent content. They are trained on child pornography. They are trained on racism, misogyny, homophobia, ableism, nationalism, anti-Semitism – they are trained on the Internet. The well is poisoned, and while they've erected a sign to say "warning!" and maybe even a little fence (mostly so they can't be sued), that doesn't change that the waters remain poisonous.

Via The Guardian: “‘Happy (and safe) shooting!’: chatbots helped researchers plot deadly attacks”

Today’s bird is the bohemian waxwing, whose red-tipped wings supposedly look like sealing wax. These birds are big fans of berries, particularly rowan berries. “They can metabolise alcohol produced in fermenting fruit,” according to Wikipedia, “but can still become intoxicated, sometimes fatally.” Despite the overlap in their habitat, bohemian waxwings are not brood parasitized (is that the right way to write that?) by birds like the cuckoo and cowbird because the latters’ young cannot handle the diet heavy in fruit. Eggs of other species placed in bohemian waxwings’ nests are always rejected, with the adults crying out in a rousing chorus, “So you think you can stone me and spit in my eye? // So you think you can love me and leave me to die?” Or something like that...

Thanks for reading Second Breakfast. Please consider becoming a paid subscriber. Your support makes this work possible.