Image source: Bitcraze
Yes, you heard that correctly: the goal is permanent airtime. Robotic flies roaming a room in RoboHouse with no human guidance – achieved within six months. In the future, 24/7 swarms like these may revolutionise aircraft inspection. Imagine a fighter jet enveloped by hundreds of nano drones that build-up a detailed picture in minutes. It’s a challenging mission, but not all challenges are equal. So we asked each Crazyflies team member: What is your favourite problem?
Okay, maybe permanent flying is exaggerating a bit, at some point batteries need recharging, but it remains the overall design essence. For team member Lennart, this is the main challenge: “We want to optimise the charging process so that you have as many drones in the air as possible with a minimum amount of charging pads.”
Each Crazyflie can buzz off for seven minutes before needing a 35 minute recharge. Through the use of wireless charging pads, human intervention is cancelled out, the alternative being manual battery replacement.
But challenges go way further than just battery strategy. Student Seppe identifies his favourite obstacle-to-overcome in collision avoidence: “This does not only include collisions between drones, but also with stationary objects,” Seppe tells us. “By deploying sensors and proper coding, these risks are minimised. Yet the strength of a robust system doesn’t lie in reducing risks, it lies in handling them when they happen.”
Servaas’s favourite challenge ties in with that of his colleague: round-trip latency. Or in English: the time it takes for the flying AI-insects to send their observations and receive commands in return. “Depending on how much time this transfer of information takes up, we could for instance let the drones react to more unpredictable objects such as humans.” Perhaps actual flies could also identify as such an object.
Floating away from technical aspects, Andreas defines solving real-world problems his goal: “Designing an autonomous, 24/7 flying drone swarm is cool, but we also want to have an actual impact through real-world application.” Andreas seeks to fulfil this wish by doing market research and identifying problems that yet remain devoid of a solution. One such application could be the inspection of large or difficult-to-access infrastructure like bridges or power lines.
Not coming from a robotic background, for fifth team member Andrea the challenge amounted to familiarising all this software involved. Luckily, Andrea managed to learn the tools of the trade, finding the AI-insects’ autonomy one of the next exciting challenges to be tackled.
But wait, this does not yet complete the team. There are a hundred other individuals, quite literally also team members. The students have included the Crazyflies in their team, deciding to name them ‘member 6 to 105’. These drones are going to inspect infrastructure all by themselves, only stopping occasionally to recharge their batteries.
If all goes well, the Crazyflies could become part of the Crazy Zoo robot exhibition on TU Delft Campus, an initiative by Chris Verhoeven, theme leader swarm robots at TU Delft. For now though, the students have a lot of work on their hands to realise their dreams and live up to the challenges. We have no doubt they will fly high.
The post Robotic flies to swarm 24/7 in RoboHouse appeared first on RoboHouse.
Nico, Emil, and Moritz founded ReRun with the mission of making powerful visualization tools free and easily accessible for roboticists. Nico and Emil talk about how these powerful tools help debug the complex problem scopes faced by roboticists. Tune in for more.
Nikolaus West
Co-Founder & CEO
Niko is a second-time founder and software engineer with a computer vision background from Stanford. He’s fanatic about bringing great computer vision and robotics products to the physical world.
Emil Ernerfeldt
Co-Founder & CTO
Emil fell in love with coding over 20 years ago and hasn’t looked back since. He’s the creator of egui, an easy-to-use immediate mode GUI in Rust, that we’re using to build Rerun. He brings a strong perspective from the gaming industry, with a focus on great and blazing fast tools.
Researchers created DribbleBot, a system for in-the-wild dribbling on diverse natural terrains including sand, gravel, mud, and snow using onboard sensing and computing. In addition to these football feats, such robots may someday aid humans in search-and-rescue missions. Photo: Mike Grimmett/MIT CSAIL
By Rachel Gordon | MIT CSAIL
If you’ve ever played soccer with a robot, it’s a familiar feeling. Sun glistens down on your face as the smell of grass permeates the air. You look around. A four-legged robot is hustling toward you, dribbling with determination.
While the bot doesn’t display a Lionel Messi-like level of ability, it’s an impressive in-the-wild dribbling system nonetheless. Researchers from MIT’s Improbable Artificial Intelligence Lab, part of the Computer Science and Artificial Intelligence Laboratory (CSAIL), have developed a legged robotic system that can dribble a soccer ball under the same conditions as humans. The bot used a mixture of onboard sensing and computing to traverse different natural terrains such as sand, gravel, mud, and snow, and adapt to their varied impact on the ball’s motion. Like every committed athlete, “DribbleBot” could get up and recover the ball after falling.
Programming robots to play soccer has been an active research area for some time. However, the team wanted to automatically learn how to actuate the legs during dribbling, to enable the discovery of hard-to-script skills for responding to diverse terrains like snow, gravel, sand, grass, and pavement. Enter, simulation.
A robot, ball, and terrain are inside the simulation — a digital twin of the natural world. You can load in the bot and other assets and set physics parameters, and then it handles the forward simulation of the dynamics from there. Four thousand versions of the robot are simulated in parallel in real time, enabling data collection 4,000 times faster than using just one robot. That’s a lot of data.
Video: MIT CSAIL
The robot starts without knowing how to dribble the ball — it just receives a reward when it does, or negative reinforcement when it messes up. So, it’s essentially trying to figure out what sequence of forces it should apply with its legs. “One aspect of this reinforcement learning approach is that we must design a good reward to facilitate the robot learning a successful dribbling behavior,” says MIT PhD student Gabe Margolis, who co-led the work along with Yandong Ji, research assistant in the Improbable AI Lab. “Once we’ve designed that reward, then it’s practice time for the robot: In real time, it’s a couple of days, and in the simulator, hundreds of days. Over time it learns to get better and better at manipulating the soccer ball to match the desired velocity.”
The bot could also navigate unfamiliar terrains and recover from falls due to a recovery controller the team built into its system. This controller lets the robot get back up after a fall and switch back to its dribbling controller to continue pursuing the ball, helping it handle out-of-distribution disruptions and terrains.
“If you look around today, most robots are wheeled. But imagine that there’s a disaster scenario, flooding, or an earthquake, and we want robots to aid humans in the search-and-rescue process. We need the machines to go over terrains that aren’t flat, and wheeled robots can’t traverse those landscapes,” says Pulkit Agrawal, MIT professor, CSAIL principal investigator, and director of Improbable AI Lab.” The whole point of studying legged robots is to go terrains outside the reach of current robotic systems,” he adds. “Our goal in developing algorithms for legged robots is to provide autonomy in challenging and complex terrains that are currently beyond the reach of robotic systems.”
The fascination with robot quadrupeds and soccer runs deep — Canadian professor Alan Mackworth first noted the idea in a paper entitled “On Seeing Robots,” presented at VI-92, 1992. Japanese researchers later organized a workshop on “Grand Challenges in Artificial Intelligence,” which led to discussions about using soccer to promote science and technology. The project was launched as the Robot J-League a year later, and global fervor quickly ensued. Shortly after that, “RoboCup” was born.
Compared to walking alone, dribbling a soccer ball imposes more constraints on DribbleBot’s motion and what terrains it can traverse. The robot must adapt its locomotion to apply forces to the ball to dribble. The interaction between the ball and the landscape could be different than the interaction between the robot and the landscape, such as thick grass or pavement. For example, a soccer ball will experience a drag force on grass that is not present on pavement, and an incline will apply an acceleration force, changing the ball’s typical path. However, the bot’s ability to traverse different terrains is often less affected by these differences in dynamics — as long as it doesn’t slip — so the soccer test can be sensitive to variations in terrain that locomotion alone isn’t.
“Past approaches simplify the dribbling problem, making a modeling assumption of flat, hard ground. The motion is also designed to be more static; the robot isn’t trying to run and manipulate the ball simultaneously,” says Ji. “That’s where more difficult dynamics enter the control problem. We tackled this by extending recent advances that have enabled better outdoor locomotion into this compound task which combines aspects of locomotion and dexterous manipulation together.”
On the hardware side, the robot has a set of sensors that let it perceive the environment, allowing it to feel where it is, “understand” its position, and “see” some of its surroundings. It has a set of actuators that lets it apply forces and move itself and objects. In between the sensors and actuators sits the computer, or “brain,” tasked with converting sensor data into actions, which it will apply through the motors. When the robot is running on snow, it doesn’t see the snow but can feel it through its motor sensors. But soccer is a trickier feat than walking — so the team leveraged cameras on the robot’s head and body for a new sensory modality of vision, in addition to the new motor skill. And then — we dribble.
“Our robot can go in the wild because it carries all its sensors, cameras, and compute on board. That required some innovations in terms of getting the whole controller to fit onto this onboard compute,” says Margolis. “That’s one area where learning helps because we can run a lightweight neural network and train it to process noisy sensor data observed by the moving robot. This is in stark contrast with most robots today: Typically a robot arm is mounted on a fixed base and sits on a workbench with a giant computer plugged right into it. Neither the computer nor the sensors are in the robotic arm! So, the whole thing is weighty, hard to move around.”
There’s still a long way to go in making these robots as agile as their counterparts in nature, and some terrains were challenging for DribbleBot. Currently, the controller is not trained in simulated environments that include slopes or stairs. The robot isn’t perceiving the geometry of the terrain; it’s only estimating its material contact properties, like friction. If there’s a step up, for example, the robot will get stuck — it won’t be able to lift the ball over the step, an area the team wants to explore in the future. The researchers are also excited to apply lessons learned during development of DribbleBot to other tasks that involve combined locomotion and object manipulation, quickly transporting diverse objects from place to place using the legs or arms.
The research is supported by the DARPA Machine Common Sense Program, the MIT-IBM Watson AI Lab, the National Science Foundation Institute of Artificial Intelligence and Fundamental Interactions, the U.S. Air Force Research Laboratory, and the U.S. Air Force Artificial Intelligence Accelerator. The paper will be presented at the 2023 IEEE International Conference on Robotics and Automation (ICRA).