For the first time on Broadway human and drone performances fuse to create a new form of artistic expression. The magic happened in Cirque du Soleil’s first musical on Broadway: ‘Paramour’ at the Lyric Theatre. The show is themed on the Golden Age of Hollywood and follows the life of a poet who is forced to choose between love and art. The contributions of the technology firm Verity Studios include the choreography of the drone show segment, the frame and lighting design of the drone costumes, and all underlying drone technologies. The system was operated by the show’s automation team, with Verity Studios providing maintenance services twice per year. The dancing drones completed almost 400 shows, including more than 7,000 autonomous takeoffs, flights, and landings.
Over the past year, 398 audiences of up to 2,000 people witnessed an octet of colorful lampshades perform an airborne choreography during Cirque du Soleil’s Broadway show Paramour, which ran until April 20th. The work behind the design and choreography of the flying lampshades, which turn out to be self-piloted drones, bears the signature of the Swiss high-tech company Verity Studios.
But how novel is it really that robots have appeared in theater? Since Karel Capek’s science fiction play R.U.R. (short for Rossum’s Universal Robots) introduced the word “robot” to the English language and to science fiction almost 100 years ago, the technical challenges of incorporating robots into live performance and theater have been overwhelming. Before these Broadway drones, nearly all theater robots were remote-controlled puppets, relying on humans hidden off-scene to steer their movements and provide their intelligence. What holds for Broadway, also holds for the even bigger picture.
It is intriguing that the aerial revolution gained momentum on the ground first: Even though the complexity of most advanced industrial automation systems found in state-of-the art factories and warehouses hardly compares to the complexity of drone show systems that allow for the autonomous navigation of dozens, hundreds, or thousands of drones in a safety-critical environment, one single example stands out. The only comparable systems that can serve as a template for operating large numbers of autonomous mobile robots with high demands for reliability and safety are those developed by the technology company Kiva Systems (bought by Amazon for USD 775 million in 2012), where Verity Studios’ founder Professor Raffaello D’Andrea and his former colleagues developed a solution for automated storage and retrieval in warehouses. His team unleashed thousands of mobile robots in warehouses, which started the robotics gold rush in the Silicon Valley. Now, Professor D’Andrea’s team is pursuing an analogous, even more ambitious goal in the air.
Placing intelligent, autonomous flying machines in live theater presents a multi-faceted challenge: creating a compelling performance with safety, reliability, and ease of operation. While the compelling performance translates into shaping a convincing creative concept around the drones’ choreography, the latter chiefly points to designing computerized systems in lieu of making use of human pilots: Verity Studios’ drones are flying mobile robots and navigate autonomously, piloting themselves, only supervised by a human operator. To navigate autonomously, these drones require a reliable method for determining their position in space. Since GPS is usually not available indoors, Verity Studios has built on more than a decade of research and development at the Flying Machine Arena of Switzerland’s Federal Institute of Technology (ETH Zurich) to provide a novel localization method.
The first groups of autonomous robots are now meeting the live event industries’ high requirements: Initially, Verity Studios set the stage for widely recognized flying machines in entertainment in 2014, by collaborating with Cirque de Soleil on Sparked, a short film in which 10 siblings of the similarly dressed 8 quadcopters from Paramour dance, hover, and flicker around a stunned human actor. Sparked was named a Winner of the 2016 New York City Drone Film Festival. However, it is one thing to feature drones in a 4-minute video, where the makers have the luxury to shoot it 100 times and then choose the best shot. Live performances on Broadway running 8 times per week in front of up to 2,000 people every time, by contrast, require the drone shows to work every single time. Verity Studios has met these requirements in the best sense: During their one-year run, the drones on Broadway safely completed more than 7,000 autonomous take-offs, flights, and landings. How was that possible?
In contrast to the long list of safety incidents involving remote-controlled drones at events, performance and reliability are key to Paramour’s dancing drones which are custom-made and hand-crafted in Switzerland. Each of the eight flying machines featured in Paramour uses 80 sensors to fly and performs roughly 1.5 billion calculations per second. Each critical calculation is double-checked every time it is performed in case any of the concerned processors makes a mistake. In other words, accidents are prevented by self-test and monitoring. And even if something does happen, a back-up is in place. For example, on the show of 2017-02-08, the battery from one of the eight vehicles could not provide sufficient power during the takeoff phase. The quadrocopter detected the problem and returned to the ground on only two propellers.
In most cases, the drones withstand any point of failure, but where a single point of failure is unavoidable, a fail-safe or weakest link design is used. The failure of any single system component cannot result in an unmanaged loss of control, but must be handled appropriately. For instance, a drone should be able to land in a controlled fashion despite a failure of any one of its motors, any part of its electronics, any one of its cables or connectors, and any one of its batteries. Similarly, loss of communication between a drone and its ground station must not create a dangerous situation. This can be achieved by designing a fully redundant system, i.e., a system that can continue operation, or trigger a safe emergency behavior, if any one of its components fails. While such designs can be more expensive to create than designs without duplication, they provide much higher safety (and help reduce insurance costs). Their decades-long history in the manned aviation industry has shown this approach’s effectiveness and provides a treasure trove of experience and evidence for the emerging drone industry.
One way to design such a redundant system is to first design a simple flight-capable system, to then duplicate all its components (e.g. sensors, processors, actuators) to achieve the necessary redundancy, and to carefully design switch-over from a failed system to its backup. Importantly, the design of such a system cannot stop with the design of fully redundant drones, but needs to extend to all other critical components of the drone show system, including the positioning system, communications architecture, e-stop systems, and control stations. In the end, this uncompromising approach allows for a degree of safety and robustness that makes those intelligent and autonomous robots suitable for performance next to theater audiences. The result is a one-of-a-kind performance that extends the traditional palette of light, sound, stage effects, and interaction with human performers by translating an intimate character beat into an unexpected visual motif.
The enticing reality is that there are many more areas to explore, ranging from live concerts to active scenography, that offer dramatic possibilities for performing robots on a much broader scale. Once again, not only such stage flyers, but also Lucie micro-drones will make history, not only on Broadway or at Amazon, but all across the domain of performing arts revolutionizing the scene with a new realm of creative expression.
