One of postmodern theory’s most counterintuitive but important claims is that classification systems do not merely describe the world, they produce it. Categories are not discovered; they are constructed. And the act of construction has consequences.

This claim can be illustrated by a simple historical example. Before the nineteenth century, homosexuality did not exist as a category of person. Sexual acts between people of the same sex occurred, of course, but there was no identity built around those acts, no type of person defined by them. In the late nineteenth century, psychiatry and law created the category of the homosexual, a type of person defined by their sexual orientation, and this creation had profound consequences. It made people legible to the state in new ways, enabled new forms of persecution and pathologisation, and simultaneously created new possibilities for identity, community, and resistance.

Ian Hacking, the philosopher of science, calls this looping effects of human kinds: when a human category is created, the people so classified begin to understand themselves in terms of the category, change their behaviour in response to it, and thereby change what the category describes. Categories of people are not stable natural kinds. They are socially produced and historically changing.

AI classification systems create human kinds at scale, at speed, and with the authority of algorithmic objectivity. When a credit scoring system classifies people as creditworthy or not, it is not describing a natural property. It is constructing a social category, with real consequences for people’s access to housing, education, and economic participation. When a predictive policing algorithm classifies a neighbourhood as high-risk, it is not describing a natural property of the neighbourhood. It is producing a category that will shape how the neighbourhood is treated, which will shape what happens in it, which will shape the data the algorithm learns from.

The looping effect is central to understanding algorithmic harm. An algorithm that classifies people as high-risk for loan default will cause those people to be denied credit, which will affect their financial situation, which will affect their actual default risk, in ways that may confirm the algorithm’s prediction while producing the very outcome it was supposedly predicting. The algorithm does not describe the world passively. It acts on it.

This has immediate implications for AI governance. Impact assessments that treat AI as a passive describer of social reality miss the ways in which AI actively constructs that reality. Governance frameworks that focus only on accuracy miss the ways in which an accurate description of an unjust social order can perpetuate that injustice.

Reflection question: Think of an AI classification system you know. What category does it create? What are the consequences of being placed in that category? How might those consequences change the thing being classified?