News

New white paper investigates models and functions of international institutions that could help manage opportunities and mitigate risks of advanced AI.

The education of robot umpires has been complicated by an open secret in baseball for the past 150 years: The strike zone called on the field doesn't match the one mapped out in the rule book.

By Jennifer Michalowski | McGovern Institute for Brain Research

MIT scientists have developed tiny, soft-bodied robots that can be controlled with a weak magnet. The robots, formed from rubbery magnetic spirals, can be programmed to walk, crawl, swim — all in response to a simple, easy-to-apply magnetic field.

“This is the first time this has been done, to be able to control three-dimensional locomotion of robots with a one-dimensional magnetic field,” says Professor Polina Anikeeva, whose team published an open-access paper on the magnetic robots in the journal Advanced Materials. “And because they are predominantly composed of polymer and polymers are soft, you don’t need a very large magnetic field to activate them. It’s actually a really tiny magnetic field that drives these robots,” adds Anikeeva, who is a professor of materials science and engineering and brain and cognitive sciences at MIT, a McGovern Institute for Brain Research associate investigator, as well as the associate director of MIT’s Research Laboratory of Electronics and director of MIT’s K. Lisa Yang Brain-Body Center.

The new robots are well suited to transport cargo through confined spaces and their rubber bodies are gentle on fragile environments, opening the possibility that the technology could be developed for biomedical applications. Anikeeva and her team have made their robots millimeters long, but she says the same approach could be used to produce much smaller robots.

Magnetically actuated fiber-based soft robots

Engineering magnetic robots

Anikeeva says that until now, magnetic robots have moved in response to moving magnetic fields. She explains that for these models, “if you want your robot to walk, your magnet walks with it. If you want it to rotate, you rotate your magnet.” That limits the settings in which such robots might be deployed. “If you are trying to operate in a really constrained environment, a moving magnet may not be the safest solution. You want to be able to have a stationary instrument that just applies magnetic field to the whole sample,” she explains.

Youngbin Lee PhD ’22, a former graduate student in Anikeeva’s lab, engineered a solution to this problem. The robots he developed in Anikeeva’s lab are not uniformly magnetized. Instead, they are strategically magnetized in different zones and directions so a single magnetic field can enable a movement-driving profile of magnetic forces.

Before they are magnetized, however, the flexible, lightweight bodies of the robots must be fabricated. Lee starts this process with two kinds of rubber, each with a different stiffness. These are sandwiched together, then heated and stretched into a long, thin fiber. Because of the two materials’ different properties, one of the rubbers retains its elasticity through this stretching process, but the other deforms and cannot return to its original size. So when the strain is released, one layer of the fiber contracts, tugging on the other side and pulling the whole thing into a tight coil. Anikeeva says the helical fiber is modeled after the twisty tendrils of a cucumber plant, which spiral when one layer of cells loses water and contracts faster than a second layer.

A third material — one whose particles have the potential to become magnetic — is incorporated in a channel that runs through the rubbery fiber. So once the spiral has been made, a magnetization pattern that enables a particular type of movement can be introduced.

“Youngbin thought very carefully about how to magnetize our robots to make them able to move just as he programmed them to move,” Anikeeva says. “He made calculations to determine how to establish such a profile of forces on it when we apply a magnetic field that it will actually start walking or crawling.”

To form a caterpillar-like crawling robot, for example, the helical fiber is shaped into gentle undulations, and then the body, head, and tail are magnetized so that a magnetic field applied perpendicular to the robot’s plane of motion will cause the body to compress. When the field is reduced to zero, the compression is released, and the crawling robot stretches. Together, these movements propel the robot forward. Another robot in which two foot-like helical fibers are connected with a joint is magnetized in a pattern that enables a movement more like walking.

Biomedical potential

This precise magnetization process generates a program for each robot and ensures that that once the robots are made, they are simple to control. A weak magnetic field activates each robot’s program and drives its particular type of movement. A single magnetic field can even send multiple robots moving in opposite directions, if they have been programmed to do so. The team found that one minor manipulation of the magnetic field has a useful effect: With the flip of a switch to reverse the field, a cargo-carrying robot can be made to gently shake and release its payload.

Anikeeva says she can imagine these soft-bodied robots — whose straightforward production will be easy to scale up — delivering materials through narrow pipes, or even inside the human body. For example, they might carry a drug through narrow blood vessels, releasing it exactly where it is needed. She says the magnetically-actuated devices have biomedical potential beyond robots as well, and might one day be incorporated into artificial muscles or materials that support tissue regeneration.

• PAPER – Magnetically Actuated Fiber-Based Soft Robots. Youngbin Lee, Florian Koehler, Tom Dillon, Gabriel Loke, Yoonho Kim, Juliette Marion, Marc-Joseph Antonini, Indie Garwood, Atharva Sahasrabudhe, Keisuke Nagao, Xuanhe Zhao, Yoel Fink, Ellen T. Roche, and Polina Anikeeva. Advanced Materials, 2301916.

Figure 1: stepwise behavior in self-supervised learning. When training common SSL algorithms, we find that the loss descends in a stepwise fashion (top left) and the learned embeddings iteratively increase in dimensionality (bottom left). Direct visualization of embeddings (right; top three PCA directions shown) confirms that embeddings are initially collapsed to a point, which then expands to a 1D manifold, a 2D manifold, and beyond concurrently with steps in the loss.

It is widely believed that deep learning’s stunning success is due in part to its ability to discover and extract useful representations of complex data. Self-supervised learning (SSL) has emerged as a leading framework for learning these representations for images directly from unlabeled data, similar to how LLMs learn representations for language directly from web-scraped text. Yet despite SSL’s key role in state-of-the-art models such as CLIP and MidJourney, fundamental questions like “what are self-supervised image systems really learning?” and “how does that learning actually occur?” lack basic answers.

Our recent paper (to appear at ICML 2023) presents what we suggest is the first compelling mathematical picture of the training process of large-scale SSL methods. Our simplified theoretical model, which we solve exactly, learns aspects of the data in a series of discrete, well-separated steps. We then demonstrate that this behavior can be observed in the wild across many current state-of-the-art systems. This discovery opens new avenues for improving SSL methods, and enables a whole range of new scientific questions that, when answered, will provide a powerful lens for understanding some of today’s most important deep learning systems.

Read More

A United Nations technology agency assembled a group of robots that physically resembled humans at a news conference Friday, inviting reporters to ask them questions in an event meant to spark discussion about the future of artificial intelligence.

Mars rovers have teams of human experts on Earth telling them what to do. But robots on lander missions to moons orbiting Saturn or Jupiter are too far away to receive timely commands from Earth.

Today’s robots are often static and isolated from humans in structured environments — you can think of robot arms employed by Amazon for picking and packaging products within warehouses. But the true potential of robotics lies in mobile robots operating alongside humans in messy environments like our homes and hospitals — this requires navigation skills.

Imagine dropping a robot in a completely unseen home and asking it to find an object, let’s say a toilet. Humans can do this effortlessly: when looking for a glass of water at a friend’s house we’re visiting for the first time, we can easily find the kitchen without going to bedrooms or storage closets. But teaching this kind of spatial common sense to robots is challenging.

Many learning-based visual navigation policies have been proposed to tackle this problem. But learned visual navigation policies have predominantly been evaluated in simulation. How well do different classes of methods work on a robot?

We present a large-scale empirical study of semantic visual navigation methods comparing representative methods from classical, modular, and end-to-end learning approaches across six homes with no prior experience, maps, or instrumentation. We find that modular learning works well in the real world, attaining a 90% success rate. In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.

Object goal navigation

We instantiate semantic navigation with the Object Goal navigation task, where a robot starts in a completely unseen environment and is asked to find an instance of an object category, let’s say a toilet. The robot has access to only a first-person RGB and depth camera and a pose sensor.

This task is challenging. It requires not only spatial scene understanding of distinguishing free space and obstacles and semantic scene understanding of detecting objects, but also requires learning semantic exploration priors. For example, if a human wants to find a toilet in this scene, most of us would choose the hallway because it is most likely to lead to a toilet. Teaching this kind of common sense or semantic priors to an autonomous agent is challenging. While exploring the scene for the desired object, the robot also needs to remember explored and unexplored areas.

Methods

So how do we train autonomous agents capable of efficient navigation while tackling all these challenges? A classical approach to this problem builds a geometric map using depth sensors, explores the environment with a heuristic, like frontier exploration, which explores the closest unexplored region, and uses an analytical planner to reach exploration goals and the goal object as soon as it is in sight. An end-to-end learning approach predicts actions directly from raw observations with a deep neural network consisting of visual encoders for image frames followed by a recurrent layer for memory. A modular learning approach builds a semantic map by projecting predicted semantic segmentation using depth, predicts an exploration goal with a goal-oriented semantic policy as a function of the semantic map and the goal object, and reaches it with a planner.

Large-scale real-world empirical evaluation

While many approaches to navigate to objects have been proposed over the past few years, learned navigation policies have predominantly been evaluated in simulation, which opens the field to the risk of sim-only research that does not generalize to the real world. We address this issue through a large-scale empirical evaluation of representative classical, end-to-end learning, and modular learning approaches across 6 unseen homes and 6 goal object categories.

Results

We compare approaches in terms of success rate within a limited budget of 200 robot actions and Success weighted by Path Length (SPL), a measure of path efficiency. In simulation, all approaches perform comparably, at around 80% success rate. But in the real world, modular learning and classical approaches transfer really well, up from 81% to 90% and 78% to 80% success rates, respectively. While end-to-end learning fails to transfer, down from 77% to 23% success rate.

We illustrate these results qualitatively with one representative trajectory. All approaches start in a bedroom and are tasked with finding a couch. On the left, modular learning first successfully reaches the couch goal. In the middle, end-to-end learning fails after colliding too many times. On the right, the classical policy finally reaches the couch goal after a detour through the kitchen.

Result 1: modular learning is reliable

We find that modular learning is very reliable on a robot, with a 90% success rate. Here, we can see it finds a plant in a first home efficiently, a chair in a second home, and a toilet in a third.

Result 2: modular learning explores more efficiently than classical

Modular learning improves by 10% real-world success rate over the classical approach. On the left, the goal-oriented semantic exploration policy directly heads towards the bedroom and finds the bed in 98 steps with an SPL of 0.90. On the right, because frontier exploration is agnostic to the bed goal, the policy makes detours through the kitchen and the entrance hallway before finally reaching the bed in 152 steps with an SPL of 0.52. With a limited time budget, inefficient exploration can lead to failure.

Result 3: end-to-end learning fails to transfer

While classical and modular learning approaches work well on a robot, end-to-end learning does not, at only 23% success rate. The policy collides often, revisits the same places, and even fails to stop in front of goal objects when they are in sight.

Analysis

Insight 1: why does modular transfer while end-to-end does not?

Why does modular learning transfer so well while end-to-end learning does not? To answer this question, we reconstructed one real-world home in simulation and conducted experiments with identical episodes in sim and reality.

The semantic exploration policy of the modular learning approach takes a semantic map as input, while the end-to-end policy directly operates on the RGB-D frames. The semantic map space is invariant between sim and reality, while the image space exhibits a large domain gap. In this example, this gap leads to a segmentation model trained on real-world images to predict a bed false positive in the kitchen.

The semantic map domain invariance allows the modular learning approach to transfer well from sim to reality. In contrast, the image domain gap causes a large drop in performance when transferring a segmentation model trained in the real world to simulation and vice versa. If semantic segmentation transfers poorly from sim to reality, it is reasonable to expect an end-to-end semantic navigation policy trained on sim images to transfer poorly to real-world images.

Insight 2: sim vs real gap in error modes for modular learning

Surprisingly, modular learning works even better in reality than simulation. Detailed analysis reveals that a lot of the failures of the modular learning policy that occur in sim are due to reconstruction errors, which do not happen in reality. Visual reconstruction errors represent 10% out of the total 19% episode failures, and physical reconstruction errors another 5%. In contrast, failures in the real world are predominantly due to depth sensor errors, while most semantic navigation benchmarks in simulation assume perfect depth sensing. Besides explaining the performance gap between sim and reality for modular learning, this gap in error modes is concerning because it limits the usefulness of simulation to diagnose bottlenecks and further improve policies. We show representative examples of each error mode and propose concrete steps forward to close this gap in the paper.

Takeaways

For practitioners:

• Modular learning can reliably navigate to objects with 90% success.

For researchers:

• Models relying on RGB images are hard to transfer from sim to real => leverage modularity and abstraction in policies.
• Disconnect between sim and real error modes => evaluate semantic navigation on real robots.

For more content about robotics and machine learning, check out my blog.

• PAPER – Navigating to Objects in the Real World. Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, and Devendra Singh Chaplot. Science Robotics, 8(79), eadf6991.

A panel of AI-enabled humanoid robots took the microphone Friday at a United Nations conference with the message: they could eventually run the world better than humans.

The fossil fuel-free Aigen Element helps farmers reduce chemicals in food, and improve human and planetary health.

Researchers have combined research with real and robotic insects to better understand how they sense forces in their limbs while walking, providing new insights into the biomechanics and neural dynamics of insects and informing new applications for large legged robots. They presented their findings at the SEB Centenary Conference 2023.

Imagine a warehouse where every item you need is just a few clicks away. That is the beauty of Goods-to-Person Picking Systems (G2P or GTP). These innovative automated systems are revolutionizing the world of order fulfillment

Meet Axl and Slash. The pair of mechanical arms named after Guns N' Roses members each weigh more than a ton but possess the deftness to nail wood without splitting. "Metallic beasts," is how BotBuilt cofounder Brent Wadas describes his yellow robots.

Competitive intralogistics requires extensive automation, something that is inconceivable without micromotors. Drive systems from FAULHABER prove their ability to perform here during sorting, transporting and with robotics solutions.

Welcome to the inaugural edition of Robo-Insight, a biweekly robotics news update! In this post, we are thrilled to present a range of remarkable advancements in the field, highlighting robotics progress in terrain traversability, shape morphing, object avoidance, mechanical memory, physics-based AI techniques, and new home robotics kits. These developments exemplify the continuous evolution and potential of robotics technology.

Four-legged robot traverses tricky terrains thanks to improved 3D vision

Recently, researchers from the University of California San Diego have given four-legged robots forward-facing depth cameras to enable them to clearly analyze the environment around and below them. The researchers utilized a model that obtains 3D information from short 2D frame videos. This data can also be compared with past images to estimate possible 3D transformation. Furthermore, their system is also self-checking, as it fuses information to give it a sort of short-term memory. Although the model does not guide the robot to a specific location, it enables the robot to traverse challenging terrain. The full paper, more videos, and the code (coming soon) can be found here.

Neural Volumetric Memory for Visual Locomotion Control

Mori3: A polygon shape-shifting robot for space travel

Along the lines of performing in difficult settings, Mori3, a robot that can change shape and interact with objects and people, was created by researchers at the Engineering School of EPFL. The modular Mori3 robot can change from 2D triangles into numerous 3D shapes by fusing digital polygon meshing with swarm behavior. The study helps highlight how modular robotics can be used for tasks like space exploration. The robot shows a great deal of versatility thanks to its adaptability and ability to assemble and disassemble. The Mori3 robots will be used by the crew to communicate with spacecraft and perform exterior repairs.

Mori3, the shape-shifter and modular origami robot

A step toward safe and reliable autopilots for flying

And speaking off-ground, a machine-learning method has recently been devised by MIT researchers to address challenging stabilize-avoid issues in autonomous aircraft. The method offers a tenfold increase in stability and outperforms previous techniques in terms of safety. The researchers were able to attain stable trajectories while avoiding obstacles by redefining the issue as a restricted optimization and employing a deep reinforcement learning technique. The method avoided crashing a simulated jet aircraft when it was flown in a tight space. The method may be used to create dynamic robot controllers and maintain stability and safety in mission-critical systems. Improvements to uncertainty accounting and hardware testing will be made in the future.

Metamaterials with built-in frustration have mechanical memory

A breakthrough in the development of materials with mechanical memory has been reached by researchers from the University of Amsterdam and ENS de Lyon. They created materials that can remember how they were previously bent or stretched and that have a special part or line that won’t change shape when pushed or pulled. This development in metamaterials may be used in mechanical and quantum computers, as well as in robotics and photonics. To create this mechanical memory effect, the researchers used the idea of non-orientable order, which is present in items like Möbius strips.

Metamaterials with built-in frustration have mechanical memory

Hybrid AI-powered computer vision combines physics and Big Data

On the topic of enhancing computer vision technology, researchers from UCLA and the United States Army Research Laboratory have developed a hybrid strategy that integrates physics-based awareness into data-driven algorithms. The article presents multiple approaches to integrate physics and data in AI like physics-based AI datasets, network designs, and network loss functions. The hybrid technique has demonstrated promising outcomes in image enhancement, motion prediction, and object tracking. Deep learning-based AI systems may eventually be able to autonomously master the rules of physics, according to the researchers.

myCobot 320 AI Kit 2023

On the industry side, the myCobot 320 AI Kit 2023, a ground-breaking robotic arm built for user-programmable development, was just released by Elephant Robotics. It offers flexibility for business, research, and creative endeavors because of its increased working radius, higher payload capacity, and intelligent grasping abilities. The kit features considerable advancements over earlier designs, supports five sophisticated vision recognition algorithms, includes grippers, and comes with user-friendly visualization software.

Bowl Bot

Finally, the Bowl Bot is an autonomous, self-cleaning robot recently created by Nala Robotics that can prepare a wide range of individualized food bowls. It offers a wide variety of 28 ingredients for bases, proteins, garnishes, and sauces in a small footprint. The Bowl Bot, which is outfitted with cutting-edge AI and vision technologies, runs at rapid speeds while upholding cleanliness and eliminating cross-contamination with its self-cleaning system.

These remarkable breakthroughs are merely a glimpse into the vibrant and dynamic world of robotics. The field continues to inspire and push boundaries, propelling us toward a future where robotics technology plays an increasingly pivotal role. Stay tuned for more exciting updates in our next edition!

Sources:

1. “Four-Legged Robot Traverses Tricky Terrains Thanks to Improved 3D Vision.” Accessed 1 July 2023.
2. Christoph H. Belke, Kevin Holdcroft, Alexander Sigrist, Jamie Paik. Morphological flexibility in robotic systems through physical polygon meshing. Nature Machine Intelligence, 2023; DOI: 10.1038/s42256-023-00676-8
3. “A Step toward Safe and Reliable Autopilots for Flying.” MIT News | Massachusetts Institute of Technology, Accessed 12 June 2023
4. Amsterdam, Universiteit van. “Metamaterials with Built-in Frustration Have Mechanical Memory.” University of Amsterdam. Accessed 1 July 2023.
5. Hybrid AI-Powered Computer Vision Combines Physics and Big Data. Accessed 1 July 2023.
6. Empowering Research and Development: Introducing the MyCobot 320 AI Kit 2023 by Elephant Robotics. Accessed 1 July 2023.
7. “The Bowls, a Fully Automated Robotic Salad Bowl Maker – Nala Robotics.”. Accessed 1 July 2023.

This year, RoboCup will be held in Bordeaux, from 4-10 July. The event will see around 2500 participants, from 45 different countries take part in competitions, training sessions, and a symposium. You can see the schedule for the week here.

The leagues and their competitions

The league competitions will take place on 6-9 July. You can find out more about the different leagues at these links:

• RoboCupSoccer
  • Humanoid
  • Standard platform
  • Middle size
  • Small size
  • Simulation
• RoboCupRescue
  • Robot
  • Simulation
• RoboCup@Home
  • This league comprises: Open platform, Domestic standard platform, and Social standard platform
• RoboCupIndustrial
  • RoboCup@Work
  • Logistics
• RoboCupJunior
  • Soccer
  • Onstage
  • Rescue

Symposium

The RoboCup symposium will take place on 10 July. The programme can be found here.

There will be three keynote talks:

• Cynthia Breazeal, Social Robots: Reflections and Predictions of Our Future Relationship with Personal Robots
• Ben Moran and Guy Lever, Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
• Laurence Devillers, Socio-affective robots: ethical issues

Find out more at the event website.