Toying with Turing's test

Alan Turing specified an interesting test with which to attempt to identify intelligence in a computer program. Let a panel of judges, in one place, communicate with, on the one hand, a Real Human Being and, on the other, the putative artificial intelligence. The judges don't know which channel of communication is connected to which party. They just ask questions and get two streams of answers. If the judges don't work out which is which, believe in the intelligence of the machine.

Of course, you'd want to try to bury cultural factors if you wanted a suitably clean experiment. I think this involves seven or eight parties. There's your

The ritual of concealment happens early in proceedings: someone choses between the putative AI and the Real Human being, records its choice, tosses a coin in the presence of witnesses and labels the chosen answerer either heads or tails according to the fall of the coin: the other label is used for the other answerer. Each answer the panel recieves is marked with the appropriate answerer's label: but the panel is not told which label corresponds to which answerer.

Questions passed by the question censor are presented to all three answerers, each of whom gives an answer to the referee. Answerers don't have to answer questions in the order in which they're asked: while they're trying to answer a question the panel may get another question past the question censor. When the referee has recieved answers to a given question from all three answerers, the question and its answers are passed to the answer censor. If the answer censor rejects any of the answers, that question is aborted, but the panel is not told: the panel cannot distinguish between aborted questions and ones which haven't yet been answered by all answerers. When the answer censor accepts the answers, the referee passes on to the panel the answers of the putative AI and the Real Human Being, labelled in accordance with the ritual of concealment.

The putative AI, or someone having access to it, issues its challenge and invites a panel of skeptics to be judge. This panel of judges choses a referee and question censor. The referee choses an answer censor, who should be familiar with the cultural perspective of the panel of judges (ideally a committee of their peers). The question censor choses a Real Human Being who is ideally more culturally kindred to the putative AI than to the panel of judges: failing that, as culturally unfamiliar (alien, if you like) to the panel as is recognised as intelligent, human, whatever. The answer censor choses a control, I guess culturally more kindred to the panel and the putative AI than to the RHB.

The question censor establishes the Real-World connections to the Real Human Being and the putative AI and keeps quiet about which is which. The question censor should do the ritual of concealment. The referee only knows the two channels as left and right (or heads and tails). Both censors see all questions and all answers, including the ones the other censor rejects: but they know nothing of the panel's debates in chosing the questions. The referee only sees answers and accepted questions, passing the latter on to all three answerers and passing on to the panel each accepted answer from each of left and right, along with a reminder of the question to which the two relate.

I believe it would be constructive to describe a model of the above using mailing lists, some of them moderated, and an anonymous re-mailer (which performs the left/right naming). The censors can sensibly be in communication with the internet at large, with the panel, putative AI and Real Human Being kept isolated. The control may well vanish, to be replaced by a body of folk sending e-mail to the answer-censors' discussion list but not getting to read that list, only the accepted questions. All mailing lists and newsgroups in what follows are archived by someone who can read them: when the experiment is over, these archives are made public. Within each committee (censors, panel and probably referee) there may be arbitrary discussion unseen to anyone else. All parties are at liberty to write to newsgroups and mailing lists at all times, or to broadcast some or all of their internal discussions. The censors and referee can read anything that's made public. The only restrictions are on what Left, Right and the panel may read.


Valid CSSValid HTML 4.01 Written by Eddy.