The House that TurnItIn Built

I gave this talk this afternoon at Concordia University, as part of the SkepTech series questioning education technology.

Today's bird, which does not appear anywhere in the talk, is the house sparrow, which along with the pigeon is one of the most widely distributed wild birds, thanks in no small part to human's own colonization efforts. It was first introduced to North America in the middle of the nineteenth century – the pigeons have been here longer – in order to curb an outbreak of moths. Sparrows have been treated in some places as invasive species and pests. They are opportunistic feeders, consuming whatever is available in their vicinity. A bit like ed-tech, I suppose.

In my work, I talk a lot about how old ideas get repackaged and sold as new ideas – old ideas, old “solutions,” and even old technologies, often peddled by the same old men, the same people who repeatedly make the same arguments, the same promises about innovation, transformation, change.

Artificial intelligence is one of the most obvious examples of this -- somehow both a brand new “revolution” and also a field of research that is about seventy years old. Seventy years old, if you date its origins to the research proposal that John McCarthy penned in 1956 to attain funding for a gathering at Dartmouth of a handful of mathematicians and computer sciences; even older if you go back to Alan Turing’s 1950 paper “Computing Machinery and Intelligence”; older still, perhaps, if you trace the history through centuries of attempts to understand the mechanisms (and then, of course, to replicate), in historian Jessica Riskin’s words, “what makes living things tick.”

Despite all this time, all this work in the field, despite generations of researchers and thousands of dissertations and millions of lines of code, today we’re sold a story that “AI” has suddenly appeared – almost out of nowhere – and we now find ourselves in some transformational age of “AI.” We have no choice, or so we’re told, but to surrender our thinking, our learning, our institutions, indeed our very being, to this machinery because the future is inevitable; it has already been written. (Indeed, if we recognize the long history, we might recognize too that the future that many of these “AI” visionaries are drawing from is itself quite ancient).

Here is another ancient tale, one that I also refer to quite often to explain the mythology surrounding the tech industry. This is an amphora from ancient Greece -- some 2500 years ago -- that depicts the birth of Athena, goddess of wisdom. She emerges, fully grown, shield and spear in hand, from her father Zeus’s head. (For those who unfamiliar with Greek mythology: Zeus had ingested Metis, also a goddess of wisdom, when she was pregnant with Athena. A prophesy had decreed that while Zeus’s first born child with Metis would be a daughter, his second would be a son who would overthrow him, much as Zeus had done to his father and his father and done to his father before that. So Zeus swallowed the pregnant goddess to avert the prophesy -- perhaps one of the only times in which such a thing actually worked. But doing so gave him terrible pains and Hephaestus had to split the god’s head open with an axe. And out jumped Athena. Full grown. Ready for battle.)

To be clear, I do not make this classical allusion to suggest that any technology, even “AI”, is a god or goddess of intelligence, let alone wisdom. Good grief, no. Generative “AI” and large language models are, to borrow another classic observation, stochastic parrots, as Emily Bender, Margaret Mitchell, Timnit Gebru, and Angelina McMillan-Major famously (infamously) argued. LLMs are probabilistic text extruding machines. They are not intelligent and they are certainly not wise. Nor will they ever be.

But this image of a god appearing violently, suddenly, fully grown, and ready for war surely aligns with the ways in which the tech industry’s owners and oligarchs think of themselves and speak of their products. “All of a sudden” and from out of nowhere, the product emerged -- complete and powerful and perfect. Revolutionary. Violent. Because with these products, “everything changes.” It must, for they are the new gods.

Another aside: because Athena emerged from her father's head shouldn't be seen as symbolic of her association with wisdom and intellect. The Ancient Greeks -- indeed like most cultures throughout most of history -- associated thinking and thought with the heart. This is why we "learn by heart," for example. Emotions and ideas resided in our chest, not in our head. I say this to remind us that our understanding of human thinking and learning, our understanding of the mind, the brain, remains incredibly limited. But also because we have always had two meanings for being thoughtful: that we are thoughtful intellectually, but that we are thoughtful compassionately. Maybe that's another story, or maybe that's very much this story too...

There’s always a story. There's always a history. There’s always some background; there’s always context. And it’s worth remembering how crucial context-lessness and ahistoricism are to the tech industry's, to Silicon Valley’s mythos: how, for example, we’re supposed to talk about “AI” as something brand new when not only has it seventy years since the coining of the phrase "artificial intelligence," it's been almost five years since the release of the first stable version of ChatGPT, it’s been over a decade since the publication of “Attention is All You Need,” the academic paper that laid out the technical foundation for transformer technology and for what has become known as “generative AI.” Neither did ChatGPT emerge, fully formed, from Sam Altman’s head, nor did it appear -- as in this amphora -- with no background, no back-drop. 2022 -- lest we forget -- was the end of the pandemic (or something like that); and many many people had spent the previous few years at home alone. Isolated. Online.

This was the backdrop for education, no doubt – one of the few areas in which generative “AI” has, unfortunately, found a powerful use-case. That is, for students to autocomplete their homework assignments. For students to cheat.

But panic about cheating, and panic about cheating with computing technology, also has a history – a very very very long history, one dates back at least to the earliest academic practices. There was widespread cheating documented in the civil service exams of the Sui Dynasty in China some 1500 years ago, for example, which as James Lang argues in his book Cheating Lessons should remind us that “the more pressure you load onto any single performance, the more you are inducing human beings to cheat.” But clearly it’s not only on high stakes tests where we see dishonesty and deception today.

When ChatGPT "suddenly appeared" (as the myth would have it) in 2022, it arrived in academic institutions that already had thousands of years of received wisdom, policies, practices, cultural expectations, and technologies that dictated how students and scholars were supposed to work, think, write (and of course thousands of years of subterfuge and rule-bending and yes cheating as well).

I want to talk a little bit about one of these products today -- one a little more recent than ancient Chinese testing practices but still fairly ancient (older by several decades than today’s undergraduates) -- because I hope that, by doing so, we can crack open some of these very old “problems” that technologies create and (claim to) solve. We can re-examine some of the cultural, institutional assumptions surrounding computing -- educational and otherwise -- and recognize how computers and the Internet have both challenged and reinforced our expectations of academic and intellectual labor -- students' as well as teachers' -- for a good long while now, the ways in which the ideology of “the digital” has already seeped into the culture of school, reinforcing some rather reactionary educational practices. (Not all revolutions, particularly those embraced by tech oligarchs, as I think we are increasingly recognizing, are remotedly progressive.)

And that technology is TurnItIn.com, which based on absolutely no scientific data, I will declare as one of the most reviled pieces of education technology – alongside the learning management system, a product from which it can, quite literally, not be separated (historically, and now, in the interface itself). "Learning management" and "cheating management" (i.e. surveillance) are intertwined in a product that teachers and students use every day.