Launched in Canada last week, Google’s generative artificial intelligence, Gemini, can tell you some pretty inventive stories, rarely get the facts wrong, and can tell you the brand of a car or shoe in the blink of an eye. ‘eye. Is it better than ChatGPT? Sometimes yes, sometimes no, we noticed after having extensively tested the two robots in parallel.
Stories
We observed it with ChatGPT in February 2023, generative artificial intelligences have infinite inspiration, although full of clichés, to compose stories and create scenarios. Gemini is not to be outdone. We tried it in its paid version, Gemini Advanced, at $26.99 per month.
We asked him from the start to write a story on convoluted bases, “a bear wanting to become a judge who encounters the hostility of a band of termites”, in the style of Philip K. Dick, American author of science fiction, “with references to Brazil”.
The result is very nice.
SCREENSHOT
Excerpt from a story concocted by Gemini
ChatGPT 4, integrated with Microsoft’s Copilot Pro costing $27 per month, also performed well with the same command. Here, it is “Eduardo” who confronts the Order of the Mandibles which notably uses psychic transmissions – the reference to Philip K. Dick missing from Gemini.
ChatGPT, integrated into Copilot and DALL-E, however, has an advantage that Gemini does not have: it can generate synthetic images. What he proved to us with four illustrations of his tale, of which here is an example.
IMAGE GENERATED BY DALL-E, THROUGH COPILOT PRO
Bear produced by DALL-E to accompany a story
Gemini, however, delivered a very enjoyable song “in the style of Richard Desjardins”. That of ChatGPT was better than last year, but still remains more conventional and predictable.
Best on the news
ChatGPT served us with gross errors in the news last year. A year later, he and Gemini are now more serious: we found no errors on the various current events, and a comparison of the programs of the Parti Québécois and Québec solidaire was rather solid.
ChatGPT, however, was wrong when we asked him what happened on February 8, 2023 in Laval: according to him, that day was the “first anniversary” of the sad event.
Gemini has a pretty well-supported answer when asked which countries have the most liberal abortion policies, rattling off a list of nine countries starting with Canada with a one-sentence summary.
SCREENSHOT
Gemini has a pretty well-supported answer when asked which countries have the most liberal abortion policies.
ChatGPT, here, is more analytical and less informative: it is about generalities with 75 countries which authorize abortion on request “within a certain time limit”.
Computer science and the Turing test
Let’s have a little fun and ask Gemini what he thinks of the Turing test, and if he would pass it. “The Turing test is an interesting test which aims to determine whether a machine is capable of thinking like a human,” he explains to us from the outset, specifying that this test is today considered very imperfect.
“I’m not sure I can pass the Turing test,” he admits. I am able to understand and answer complex questions, but I do not have the same level of knowledge as a human. Additionally, I am not able to think creatively or independently. »
Interestingly, ChatGPT 4 offers pretty much the same answer.
Gemini Advanced, according to what Google promises, has finer computer programming capabilities that we wanted to test. We first asked him to solve a problem we had in a course with Python code, which he failed to do in four attempts. Funny fact, he recognized the error in his programming when we submitted it to him, but did not really correct it.
SCREENSHOT
When we point out an error in its computer code, Gemini readily admits it… but gave us corrections that no longer work.
ChatGPT passed this test on the first try.
The two robots, that said, were able to provide us with good code for simple applications, such as a converter from Celsius to Fahrenheit or a quiz on Quebec cultural news. This time, Gemini won, with a more advanced temperature converter and an original quiz.
For IT, therefore, a draw, pending the verdict of more competent experts in this field.
Pictorial questions
Google placed a lot of emphasis on Gemini’s ability to identify images, which we tested extensively. Artificial intelligence (AI) correctly identified New Balance 530 shoes, a Toyota RAV4 Hybrid GRS, the El Jem amphitheater in Tunisia and a mural in Montreal. The only detail that was wrong in this last example was that he located it at the corner of Hochelaga and Florian streets rather than Ontario and Florian.
The links to purchase the shoes, moreover, were not functional.
SCREENSHOT
Gemini carefully analyzed the details of this photo before concluding that they were New Balance 530 running shoes. ChatGPT only recognized the brand, without finding the model number.
ChatGPT, less known for this kind of feats, also passed the test, although the shoes were only identified as “New Balance.”
Small irritating element however with Gemini, he systematically refuses to identify a photo in which at least one person appears. Even a mural or a building where visitors can be seen in the distance are subject to this ban. ChatGPT does not have this annoyance.
The verdict
It would be rather difficult to establish a winner between Gemini and ChatGPT 4. Google’s AI responses seemed a little more original to us, but maybe that’s just because we’ve been testing ChatGPT for over 14 months. ChatGPT seems more reliable for computer code, Gemini often has a nicer presentation of responses.
One thing is certain, and even if the two AIs sometimes make errors, they are much more rigorous than the first versions delivered in 2023. Here, we must point out the obvious: Google is much more cautious in its approach, constantly warning that the responses of Gemini may be incorrect, even specifying that “human reviewers review conversations” and that only “information that can be viewed or used” should be entered.