I think of Captain Kirk in Star Trek vs. "2001, A Space Oddysey". In Star Trek, the computer is very much still a computer, a tool that the Captain commands. In "2001", the computer has its own objectives, theoretically what humans asked it to do, but people really aren't in command. Here's a quote about how some computer scientists have considered this problem:Szdfan wrote: ↑Thu Jul 23, 2026 2:47 pmThis is one of the big questions of science fiction — what happens if artificial intelligence obtains sentience? What happens if humans do eventually create a machine that has the ability to be self-aware, introspective and to experience emotions like humans do?Bootstrap wrote: ↑Thu Jul 23, 2026 9:38 amThis is a red line for me. If we ever start giving AIs human rights, then they are no longer a tool for humans to use as we will. And no human being is liable for disasters caused by AI.Neto wrote: ↑Tue Jul 21, 2026 12:50 pm I didn't read the entire article before the nurse called me back, but I picked up a Time magazine that had an article about AI, and "How people should treat them". They drew a parallel between respect given to animals, and stated that "humans are not as unique as 'they' tend to think."
Only humans are humans.
In the movie Blade Runner, Harrison Ford hunts “Replicants” — artificial humans — who have illegally come to earth to force the corporation that created them to expand their lifespan beyond four years. Rachel, one of the Replicants, believes she is a human because she’s programmed with false memories. In the Star Trek: The Next Generation episode “The Measure of a Man,” Captain Picard has to defend the android Data in a tribunal to determine whether he’s the property of Starfleet and can force him to participate in a scientific experiment that could destroy him.
Some people seem to be jumping the gun based on an illusion.
Most of the AI companies are talking about replacing workers, networks of AI agents that can make their own decisions without human intervention, etc. Most of the people I know who are actually producing results I trust are using some kind of Captain Kirk model.AI alignment researchers have a name for the property that distinguishes these two models. Stuart Russell, in Human Compatible (2019), argues that the standard model of AI — build a machine that optimizes a fixed objective — is inherently dangerous because we can never fully specify what we actually want. A machine optimizing hard for the wrong objective will resist being turned off, because being turned off prevents it from achieving its goal. HAL is the proof of concept.
Russell's solution is to build machines that are genuinely uncertain about human preferences and therefore defer to humans for correction. He calls this property corrigibility: an AI that accepts correction and remains under human authority.
You and I can't control what the AI companies advertise, but we can decide how we use AI. People who design systems can decide whether building corrigibility into our systems, and I would urge them to do so.