The Turing test: how the imitation game works and the main objections
Alan Turing proposed replacing the question 'can machines think?' with an imitation game in which an interrogator tries to tell a human from a machine in conversation. The test has been widely debated, with objections ranging from mathematical limits to concerns about genuine understanding. Modern experiments show that large language models can sometimes fool judges, but the test remains controversial.
The imitation game
Alan Turing's 1950 paper described an imitation game involving a person, a machine, and an interrogator. As explained in the Stanford Encyclopedia's Turing test entry, the interrogator is isolated in a separate room, knows the other two only as X and Y, and must determine which is the person and which is the machine by asking questions. Turing presented this game as a way to give a precise meaning to the question of whether machines can think, which he considered too vague for direct discussion.
The game is purely conversational, focusing on outward performance rather than on internal mental states. The machine's goal is to imitate human responses convincingly, while the interrogator tries to uncover the machine through probing questions.
Turing's motivation and legacy
Alan Turing's 1950 paper 'Computing Machinery and Intelligence' is one of the most frequently cited works in modern philosophy, according to Stanford Encyclopedia's Turing biography. Turing related the traditional mind-body problem to the mathematical concept of computability that he had introduced earlier, and his work is regarded as the foundation of computer science and the artificial intelligence program.
Turing suggested that the question 'Can machines think?' is too meaningless to deserve discussion. Instead, he focused on the more precise question of whether a digital computer could perform well in the imitation game, and he predicted that computers would eventually do so.
Main objections
Various objections to the idea that machines can think are outlined in Internet Encyclopedia's AI entry. These include the mathematical objection, which refers to the limits of formal systems, and the argument from behavioral disabilities, which points to the lack of feelings and other subjective experiences. The article also discusses Lady Lovelace's objection, which claims that machines only do what they are programmed to do, and Searle's Chinese Room argument, which holds that syntactic symbol manipulation cannot produce genuine understanding.
Many of these objections challenge the assumption that passing a conversational test demonstrates real intelligence. They argue that behavior alone is insufficient for thinking, and that internal mental states matter. The Turing test is purely behavioral, so it cannot rule out a machine that simulates conversation without understanding.
Modern evidence from large language models
A study by Cameron R. Jones and Benjamin K. Bergen evaluated GPT-4 in a public online Turing test. Their study, reported in the arXiv paper by Jones and Bergen, found that the most effective GPT-4 prompt succeeded in 49.7% of games, while ELIZA and GPT-3.5 achieved 22% and 20% respectively, and human participants set a baseline of 66%. The authors found that participants' decisions were based mainly on linguistic style and socioemotional traits, supporting the idea that intelligence alone is not sufficient to pass the test.
The study also suggests that participants who were knowledgeable about large language models and played more games were better at detecting the AI, indicating that learning and practice can help spot deception. The authors argue that the Turing test remains relevant as an assessment of naturalistic communication and deception, even though it is not a complete test of intelligence.
Historical and ongoing relevance
The quest to build machines that behave intelligently has a long history. The Computer History Museum's exhibit on artificial intelligence notes that researchers have drawn on philosophy, biology, and engineering, and it quotes John McCarthy's definition of AI as the effort to make machines act in ways that would be judged intelligent if a human did them. The museum also highlights milestones such as IBM's Deep Blue defeating world chess champion Garry Kasparov, which showed that specific intellectual tasks can be performed by machines. See the Computer History Museum's exhibit on artificial intelligence.
The Turing test continues to influence debates about machine intelligence, even as alternative tests have been proposed. The museum's exhibit illustrates that the field has expanded far beyond the imitation game, with achievements in game playing, robotics, and language processing. However, the philosophical questions about whether machines truly think remain unresolved.
| Participant or model | Pass rate |
|---|---|
| Best GPT-4 prompt | 49.7% |
| ELIZA | 22% |
| GPT-3.5 | 20% |
| Human participants | 66% |
Frequently asked questions
What is the Turing test?
It is a test proposed by Alan Turing in 1950, based on an imitation game where an interrogator tries to distinguish a human from a machine through conversation.
Why did Turing propose the test?
He thought the question 'Can machines think?' was too meaningless and wanted a precise question about whether a computer could perform well in a conversational game.
What are the main objections?
Objections include the mathematical objection, arguments from consciousness, Lady Lovelace's objection, and Searle's Chinese Room argument, which claim that behavior alone is insufficient for genuine thinking.
Has any AI passed the Turing test?
In a recent experiment, a version of GPT-4 passed in 49.7% of games, but this was below the human baseline of 66%, and no machine has conclusively passed the test to everyone's satisfaction.
Is the Turing test still relevant?
It remains relevant as a measure of naturalistic communication and deception, though it is not a complete test of intelligence.