Before conquering Hollywood, Denis Villeneuve had turned his back on cinema. He was given carte blanche by the Phi Center in Montreal in 2009 to direct the short film. Next Floor who revived it. Fifteen years later, this same carte blanche can be yours… thanks to the generative AIs Sora and Veo.
Tools for generating photos from text are outdated. In 2024, the generation of videos from texts is all the rage. The two protagonists of this film are Google and OpenAI.
OpenAI unveiled Sora in spectacular fashion last February. In fact, OpenAI has posted a handful of short videos — and the short text from which they were created — that illustrate the potential of its generative AI. It quickly caused a sensation on social networks.
This also sparked a reaction at Google, which also has the ambition of becoming the dominant player in generative AI. Last week, Google presented its own tool for creating videos from text. His name: Veo.
Sora
On paper, Sora’s technical abilities are relatively modest. After all, his videos have a maximum length of one minute. Their resolution is quite acceptable, at 1920 x 1080 pixels. It’s high definition, but cinema, on a big screen, uses a few more pixels than that. And the application is not currently available to the public.
That said, the videos released by hand-picked filmmakers who used it at OpenAI’s request are spectacular. Especially due to the fact that they were made without any intervention other than a computer keyboard.
For example: “A woman walks down a Tokyo street filled with colorful neon lights. She wears a long black leather raincoat, a red dress and black boots. She is holding a bag in her hand. She wears sunglasses and lipstick. She is confident and relaxed. The street is wet and reflects the colors of the city. We see other pedestrians. »
The resulting video is astonishing. We recognize in the background, but cannot identify it, a dense district of Tokyo. Up close, we see the details of the woman’s clothes, with Asian features, and a close-up shows the details of her skin and a few unruly strands. We begin to wonder if this is not an advertisement for the manufacturer of its smoked glasses… until a close-up reveals a logo that makes no sense: if Sora has a defect, it is this one. Generative AI is unable to produce stylized but realistic text in its videos.
This will be fixed soon, promises OpenAI. The videos will also be extended in the near future. And we can even add a soundtrack, with lyrics, if we wish.
Veo
The first thing about Veo that DeepMind, the Google subsidiary that developed it, says is that this video generator unveiled last week is capable of generating videos with a duration “greater than a minute “.
In other words: Veo 1, Sora 0.
Otherwise, Veo works the same as Sora. And more sophisticated. We enter a description of the video we wish to produce. It can take more than one sentence, to specify the desired atmosphere, set the scene, describe the subject. You can also ask him to add special effects. We can also impose a specific format, such as a hyper-accelerated video or an aerial view of the action.
That said, like Sora, Veo is not yet open to the public. Its use is currently reserved for a handful of hand-picked people. However, you may soon be one of those people because Google has created a waiting list for people eager to bring their cinematic ideas to life.
Ideas that can be relatively complex, we are not afraid to say at Google, which is not its first tool for generating visuals: Imagen Video, VideoPoet and Lumiere came before Veo and were merged into the new came.
Like Sora, Veo only needs a single short paragraph to generate a video which, as a base, lasts around a minute. For example: it can bring to life a life-size elephant which is actually a knitted piece of wool… The user can then press more than once on an icon which invites the AI to stretch the video a few more seconds, until ‘to reach the desired duration.
We do not yet know if tools like Sora and Veo will enter the palette of directors of short and feature films. We didn’t think that journalists would use AI to help them write their stories, but now two of the most recent Pulitzer Prize winners have declared in recent days that they have done so.
What we know, in any case, is that Veo is already used to produce music videos. American actor, rapper and director Donald Glover, aka Childish Gambino, had preview access and promises something spectacular coming soon.
“Everyone is going to want to be a director, and now everyone should be a director,” Glover sums up. At the end of the day, we just want to tell stories. »