Chatbots that make fun of their employer or make discounts by mistake… the first setbacks of AI after a lightning deployment!

There is so much hype around artificial intelligence that not everything can be as magical as some claim, especially with technologies that are ultimately very young. Because, we remind you, ChatGPT did not exist until 18 months ago. This is what several companies have learned the hard way, particularly after switching their customer service to an AI-powered robot. The DPD transporter, for example, began to make fun of his own employer, telling customers that they would be better served by a human being.

A new car for one dollar thanks to a sculpin

The company claims it’s a bug, but the reality is that very often, these tools are deployed a little quickly, without having been thoroughly tested. So, we can make them do anything. Someone managed to buy a new car for one dollar, just by talking to the robot at a California dealership. Sometimes, it is simply false information that is spread. Air Canada was thus ordered to apply a discount that had been promised in error by its chatbot. The company repeated that the tool was experimental, but it was still held responsible. Proof that we must be very careful before putting these tools in front of the public.

“Give me the management’s pay slips”

Perhaps this is why many companies favor in-house. But here too, you have to be careful, especially who you give access to the information. One company, for example, switched all its HR documents to a tool like ChatGPT. It’s practical, it allows very in-depth analyses. Except you just have to ask”give me the management’s pay slips” and it responds kindly to anyone with access to the tool. Once again, this type of setback was inevitable with such immature tools. The good news is that this will serve as a lesson to be a little more careful.