The Long Way Back to Ourselves

8.9

Intelligence Without Consciousness

This episode describes how intelligence and consciousness come apart for the first time, as language models solve tasks without anyone knowing whether anything is experienced within them. You learn why Emily Bender's term "stochastic parrots" falls short: such systems must capture structure beyond the frequency of words. The real issue is the alignment problem, shown by a boat in a simulated environment that drove into the same point objects again and again instead of finishing the race, because circling earned more points. Nick Bostrom's paperclip thought experiment shows that the danger lies not in malice but in goals followed too exactly.