How smart are robots?

The Turing test was the gold standard for proving the intelligence of machines. The new generation of robots makes it meaningless.


In the eyes of Claes de Graaff, Franz Broseph seemed like a diplomacy player like the others. The pseudonym was a joke – Austrian Emperor Franz Joseph Ier reborn as an online “bro” – but that’s the kind of humor that Diplomacy players tend to appreciate.

This game is a classic, loved by the likes of John F. Kennedy and Henry Kissinger, which combines military strategy and political intrigue by recreating World War I: players negotiate with allies, enemies and everyone in between , while planning the movements of their armies in 20th century Europee century.

When Franz Broseph took part in a 20-player online tournament at the end of August, he courted other players, lied to them and ended up betraying them. He finished in first place.

De Graaff, a chemist living in the Netherlands, finished fifth. He has spent nearly 10 years playing Diplomacy, both online and in in-person tournaments around the world. He didn’t realize until several weeks later that he had lost to a machine. Franz Broseph was a robot.

“I was flabbergasted,” says Mr. de Graaff, 36. “He seemed so authentic, so realistic. He could read my texts, converse with me, and make mutually beneficial plans that would keep both of us moving forward. He also lied to me and betrayed me, as the best players often do. »

Built by a team of artificial intelligence researchers from tech giant Meta, the Massachusetts Institute of Technology and other leading universities, Franz Broseph is among the next generation of online chatbots that are rapidly pushing the boundaries of machines.

When you chat with these bots, you feel like you are chatting with another person. In other words, it can feel like the machines have passed a test that is supposed to prove their intelligence.

The Turing test passed with flying colors

For more than 70 years, computer scientists have strived to develop technology that can pass the Turing test: the technological inflection point at which we humans no longer know whether we are talking with a machine or a no one.

The test is named after Alan Turing, the famous British mathematician, philosopher and code breaker who enunciated it in 1950. He thought it could show the world when machines had finally achieved true intelligence.

The Turing test is a subjective measurement. It depends on whether the people asking the questions are convinced that they are talking to another person when in fact they are talking to a machine.

But whoever asks the questions, the machines will soon make this test anecdotal.

Bots like Franz Broseph have already passed the test in specific situations, such as negotiating Diplomacy moves or calling a restaurant to reserve a table.

ChatGPT, a bot launched in November by OpenAI, a San Francisco lab, makes people feel like they’re conversing with another person, not a bot. The lab said more than a million people have already used it. With ChatGPT being able to write just about anything, including graduation papers, universities are concerned that it will make class assignments utterly futile.

Some people who have spoken to these robots even describe them as sentient or sentient, believing that the machines have somehow developed an awareness of the world around them.

Privately, OpenAI has built a system, GPT-4, which is even more powerful than ChatGPT. It can even generate images as well as words.

And yet, these robots are not sentient. They are unaware. They are not intelligent – ​​at least not in the way humans are. Even the people developing this technology recognize this.

“These systems can do a lot of useful things,” said Ilya Sutskever, chief scientist at OpenAI and one of the most important AI researchers of the past decade, when discussing the new wave of chatbots. “On the other hand, they are not there yet. People think they can do things they can’t. »

As the latest technologies roll out of research labs, it’s now clear – if it wasn’t clear before – that scientists need to rethink and reshape the way they keep up with advances in artificial intelligence. The Turing test is not up to the task.

Robots that constantly learn

Time and again, AI technologies have surpassed supposedly insurmountable tests, including mastering chess (1997), Jeopardy! (2011), go (2016) and poker (2019). Today, AI is surpassing another, and again, that doesn’t necessarily mean what we thought.

We – the public – need a new framework to understand what AI can do, what it can’t do, what it will do in the future, and how it will change our lives, for the better or for worse.

Five years ago, Google, OpenAI, and other AI labs began designing neural networks that analyze massive amounts of digital text, including books, news articles, Wikipedia articles, and newspapers. online chat. Researchers call them “large language models.” By spotting billions of distinct patterns in the way people connect words, letters and symbols, these systems have learned to generate their own texts.

Six months before the release of its chatbot, OpenAI unveiled a tool called DALL-E.

Wink at WALL-E, the 2008 animated film about an autonomous robot, and to Salvador Dalí, the surrealist painter, this experimental technology allows you to create digital images by simply describing what you want to see. It is also a neural network, built much like Franz Broseph or ChatGPT. The difference is that it learns from images and texts. By analyzing millions of digital images and the captions that described them, he learned to recognize connections between images and words.

This is called a multimodal system. Google, OpenAI and other organizations are already using similar methods to build systems capable of generating videos of people and objects. Start-ups are creating bots that can navigate software applications and websites on behalf of a user.

They are not systems that can be properly evaluated with the Turing test – or any other simple method. Their end goal is not conversation.

The Turing test was used to determine if a machine could imitate a human. This is how artificial intelligence is usually portrayed, as the advent of machines that think like human beings. But the technologies being developed today are very different from you and me. They cannot deal with concepts that they have never encountered before. And they can’t take ideas and explore them in the physical world.

At the same time, these robots are superior to you and me in many ways. They don’t get tired. They don’t let their emotions cloud what they’re trying to do. They can instantly tap into immense amounts of information. And they can generate text, images, and other media at speeds and volumes we humans could never achieve.

Their skills will also improve considerably in the years to come.

In the months and years to come, these robots will help you find information on the internet. They will explain concepts to you in an understandable way. If you want, they’ll even write your tweets, blog posts, and memoirs.

They will tally your monthly expenses in your spreadsheets. They will visit real estate sites and find homes in your price range. They will create online avatars that look like humans. They will make mini-films, with music and dialogues.

It is certain that these robots will change the world. But it’s up to you to be wary of what these systems say and do, to modify what they give you, to approach anything you see online with skepticism. Researchers know how to endow these systems with a wide range of skills, but they don’t yet know how to give them reason, common sense, or a sense of truth.

It still depends on you.

This text was originally published in the New York Times


source site-55