The Long Way Back to Ourselves

8.10

When Data Decides

This episode distinguishes three stages at which systems decide about people: the oracle, which you can follow or ignore; the agent, which acts before ignoring it is even possible; and the sovereign, whose decision can practically no longer be checked. Automated credit card fraud detection shows how the same software stays an oracle for an informed person and has long since become sovereign for a less informed one. Cathy O'Neil's Weapons of Math Destruction shows how systems take over past inequality as a rule that looks neutral. Price crashes in financial markets triggered by coupled systems show that the sovereign stage is usually inherited unnoticed.