People like to shit on the Turing test, but if you step back from the subjective judgement angle, and instead imagine that the person performing the Turing test is a scientist trying to collect evidence that the agent that it is communicating with is _NOT_ intelligent/human, it is actually still very relevant. Tools like statistical analysis of output and responses to jailbreak prompts and recursive/self referential prompts designed to confuse machines and generate emotional responses from humans could be used to generate probability of human/not human in a much more rigorous way.
The actual Turing Test is a party game like Werewolf. If the humans are skilled then they should be able to authenticate by picking a subject that the AI can’t compete on. This would be a very difficult game to build a computer opponent for and nobody really tries.
Yes, but people underestimate what that involves. Serious players would study previous games for weaknesses, looking for subject areas that include unpublished knowledge that isn’t in the training set.
Casual talk about “passing the Turing Test” sets a much lower bar.
