Last December, the circus troupe Les 7 Fingers presented for the first time a show it had been working on for more than three years. The performance was delivered in a room in front of spectators, but also in virtual reality for a handpicked audience, who were treated to a technology-enhanced version. THE Duty was able to witness this prototype in multiple realities, Carry Me Home, in LAB7, the 7 fingers laboratory, in Montreal. Journey to the heart of the creation of a digital performance, where artistic needs and technical constraints mingle.
Wearing a virtual reality headset, two controllers in her hands, Marion Cossin has a broad smile on her lips. This researcher at the Center for Research, Innovation and Transfer in Circus Arts (CRITAC) works with Les 7 Fingers on solutions to integrate certain new technologies into the troupe’s shows. In the virtual world where she evolves, she finds herself on the shore of a lake, where two children are playing with a hoop. She can hear the wind blowing through the leaves of the trees, and almost feel the water lapping at her feet. The children perform acrobatics while the researcher leaps in their midst, turns on herself and does not hesitate to throw herself on the ground to observe them from every angle. She scrupulously notes the details that do not faithfully reproduce reality – a hand that does not close on a hoop, a foot that passes through the ground…
If these virtual children seem so strikingly realistic, it is no coincidence. On the floor below takes place, in front of an audience, the same performance as the one she attends, but with artists in the flesh. All around the stage are arranged infrared cameras, and small diodes have been cleverly hidden in the costumes of the acrobats. This ingenious system is invisible to the eyes of the spectators present in the room; but in the virtual world, it allows the characters of the two children to reproduce the slightest actions and gestures of the artists live, like digital puppets.
Beyond the artistic performance, this show like no other is at the heart of one of CRITAC’s research topics: making motion capture a technology better suited to the circus arts.
What is motion capture?
Perhaps you have already seen the footage of a film where actors are dressed in black jumpsuits dotted with white balls: this is one of the main methods of motion capture, which has made its way into the world of cinema and video games since the 1990s.
Unlike a character entirely animated by computer, this technology aims to reproduce in a virtual universe the actions of a very real actor. The dimensions of the digital puppets thus created can be modified — this is the case of the Na’vis in the films Avatar by James Cameron, or that of Gollum in the trilogy of Lord of the Rings. In these two examples, we recorded the gestures of the actors thanks to motion capture; the color of their skin, their height, the proportion of the different parts of their bodies and the features of their faces were then modified to create characters that were both fictional and strikingly realistic.
With this method, we could explore new artistic languages and reach new audiences, so that they live new cultural experiences in their hospital bed or from a remote area, for example.
The system is based on these famous white balls, called markers. About fifty are placed on the actor, in particular at the level of his joints. As he moves on stage, cameras placed throughout the space send infrared light, which is reflected by the markers. The actor is thus transformed into a real Christmas tree in the eyes of the cameras, which are particularly sensitive to infrared.
If a ball is spotted by three of them, software can then find its exact position. He then takes care of virtually reconstructing the body of the actor from all the balls that have been seen by the cameras – one corresponds to the left shoulder, another to the right foot, etc. Some more advanced techniques even make it possible to recognize facial expressions, using tiny markers placed on the face.
Creativity and technological constraints
In the case of the circus, the applications of such technology are numerous: a performance where a gesture triggers a sound or visual effect, costumes projected onto the bodies of acrobats, or even a show given in a virtual world — such as Carry Me Home.
This state-of-the-art technique opens up new creative avenues to the 7 fingers, according to their artistic director and co-founder, Samuel Tétreault. “We wanted to create here a universe that evokes childhood. We used an oil-paint aesthetic and downsized the acrobats to look like little boys, he explains. With this method, we could explore new artistic languages and reach new audiences, so that they live new cultural experiences in their hospital bed or from a remote area, for example. »
However, the use of motion capture is not easy in the context of performing arts. “Luckily we don’t give this kind of show every day! laughs Isabelle Domens. Headphones on one ear, eyes riveted on her screens, this project manager with 7 fingers made sure that everything worked for the best during the performance on the side of the spectators, but also upstream with the artists. And some problems had to be solved!
She first points out that the “real” acrobats move on stage without the audience noticing the white balls on which motion capture usually rests. “They could bother them or come off when they make sudden, rapid, powerful gestures on other objects or other bodies,” she underlines.
Rather than using markers that reflect infrared light, the 7 fingers have opted for small diodes that emit it themselves. Due to their size and the fact that they send light invisible to the eyes of the spectators, they can easily be concealed in costumes. This is what makes the strength of this performance in multiple realities: the people seated in the room let themselves be transported by the performance on stage. They don’t notice all the state-of-the-art equipment that allows the live broadcast of the performers’ movements to viewers in the virtual world.
A few concessions
Despite these changes, Samuel Tétreault had to make creative concessions. “From an acrobatic and choreographic point of view, it’s very limiting to have all these diodes on the body,” he admits. He had to choose figures and disciplines that minimize the artists’ discomfort and the risk of breaking diodes.
He would also have liked to be able to follow the movement of their hands thanks to connected gloves. Such equipment has, for example, made it possible to record the comings and goings of the Metallica guitarist’s fingers, to reproduce them faithfully in one of the video games in the series. Guitar Hero. But again, it’s not so easy to use them in the context of the circus.
“Often, acrobats need the grip of the palms of their hands and sometimes even their feet, like for the hoop, juggling or balance. This surface of their body is very important for their practice, we cannot put any clothes on them,” recalls Isabelle Domens.
Finally, this technology remains very expensive and time-consuming — each diode must be removed, recharged and replaced in the costumes before the performance… and with around fifty diodes per artist, this requires several hours of preparation before each performance!
In order to go beyond this prototype, the 7 fingers called on a team of CRITAC researchers, of which Marion Cossin is a member. Scientists are currently working on developing a new way to capture motion using a technique based on artificial intelligence, in order to reduce the number of markers to be placed on the body of acrobats. The latter could thus regain a certain freedom of movement and artistic creation. A new show prototype, produced with students from the National Circus School, should see the light of day in the fall.
A long version of this text received in 2023 the first prize of the Fernand-Seguin scholarship from the Association of Scientific Communicators of Quebec.