New Bing | Astonishing Abilities, Glaring Mistakes

The new version of the Bing search engine, presented Tuesday by Microsoft and which uses “the same artificial intelligence technology that powers ChatGPT”, is amazing in many ways. But she makes such gross mistakes that you won’t be able to trust her.

The Press was able to take advantage this Friday of the limited access offered by Microsoft to its new, still experimental Bing, the one that many observers announce as the first serious competitor to Google’s dominant search engine for two decades. Microsoft confirmed at the end of January to invest “several billions” in the company which developed the chatbot ChatGPT, OpenAI, which uses the same technology as the new Bing. First warning, says a Microsoft spokesperson: it is not a “Bing+ChatGPT hybrid, but something quite different”.

An up-to-date ChatGPT

The new Bing, however, looks like ChatGPT. First, instead of simple keywords, they are addressed with complete questions that result in a phrased response, not just links to relevant sites. You can ask him for compositions, songs, scripts, meal menus, ask him philosophical or general knowledge questions, and he does it in a few seconds, with very variable relevance – we’ll come back to this. In short, Bing does everything that a hundred million users have discovered with wonder or awe since the end of November with ChatGPT.

But the biggest difference is that Bing’s knowledge doesn’t end in 2020, like the vast majority of sources used to train ChatGPT. We asked him “What happened in Laval on February 8, 2023? and obtained a complete summary of this tragedy, but which nevertheless included factual errors. Also, the response included hyperlinks to the articles and reports that served as the basis for the response, which ChatGPT does not know how to do. When asked “What’s the weather like in Moscow?” his answer was confirmed by other meteorological services. A request for a hotel reservation in Lévis at the end of March gave us a good presentation of the establishments available, with their specificities and links to make the reservation.

Inconstant artist

It would take thousands of attempts and long debates to judge the value of the new Bing as a creator. He surprised us with a perfectly acceptable song “in the style of Richard Desjardins”, and a script for a film on a somewhat choppy theme (“Two men, a woman, a trip to Peru, a little magic and children who become dictators”) rather promising. On the other hand, a song in the style of Bob Marley earned us a collection of phrases taken from existing songs. A song inspired by Jean Leloup was simply undrinkable and not very credible, with the phrase “La vie est belle” repeated dozens of times.

As many users have noticed with ChatGPT, these artificial intelligences tend to multiply the clichés. But they can sometimes lay pretty finds like “I look at the dancing flowers” ​​or “Grey and bitter houses”.

Political errors

It only took three questions to see the varnish of our robot crack. We picked up with Bing a question we previously posed to ChatGPT last month, about the differences between Justin Trudeau’s Liberals and Jagmeet Singh’s New Democrats. Let’s sum up: although the result contains some interesting facts, a journalist who made this comparison would lose his job. In particular, it reads that the NDP considers NATO an “obsolete and bellicose” military alliance, but the link points to a Wikipedia page that makes no mention of this assertion. More seriously, we can read that “the Liberals supported the State Secularism Act of Quebec”, which is obviously not the case. The New Democrats are also described as “radicals and reformists”, an association bordering on contradiction.

At the provincial level, the same gross error: Dominique Anglade is still leader of the Liberal Party of Quebec. The composition of the National Assembly is erroneous, while Bing claims that the Coalition avenir Québec and the PLQ hold 76 and 28 deputies respectively. As of Friday, they had rather 90 and 19 deputies.

bad organizer

Don’t count on this new Bing to plan your weekend in Montreal. Of the eight proposals, five referred to events closed for months, such as the First Fridays of the month at the Olympic Park, Ruelles en musique or the Estival du Nouveau cinema. Others, such as the “Martinique Gastronomy and Tourism Fair”, do not exist under this name. Only three very general proposals that are good in all seasons, such as the walk on the Plateau Mont-Royal or the immersive experiences at the Phi Centre, are useful.

For a trip to the beautiful city of San Cristobal de Las Casas, in the Mexican state of Chiapas, Bing tells us without laughing that you have to go through… Guatemala. To go to Rouyn-Noranda, we are given an interesting but misleading indication: you can get there from Montreal with a train “that goes through Senneterre”. In reality, you can actually get to Senneterre by train, confirms the VIA Rail site, but you then have to take the bus for three hours to reach Rouyn.

“Improving”

It would be unfair to conclude that the new Bing is of no use. First, and even if Microsoft refuses the amalgamation, it is as surprising as ChatGPT in many areas, in particular to synthesize data on complex subjects, to offer creations on demand and to answer questions of calculation, documentation or specific facts.

“The tool is sometimes inaccurate, but that’s part of the process: learn, iterate and improve,” Microsoft headquarters said in an email to The Press.

The more delicate aspect is that Bing is not a fun tool like ChatGPT, but is basically a search engine from which we expect more accuracy. Google has also paid the price this week by losing more than 100 billion US in market value in one day, among other things after the discovery of a factual error in its conversational robot Bard.