Archive 27.07.2022

The 2021 World Robot Report shows that the global number of robots per 10,000 employees has risen from 66 to 126 between 2015–2020. So which sectors are reaping the rewards?

By Kim Martineau | MIT Schwarzman College of Computing

More than a decade ago, Ted Adelson set out to create tactile sensors for robots that would give them a sense of touch. The result? A handheld imaging system powerful enough to visualize the raised print on a dollar bill. The technology was spun into GelSight, to answer an industry need for low-cost, high-resolution imaging.

An expert in both human and machine vision, Adelson was pleased to have created something useful. But he never lost sight of his original dream: to endow robots with a sense of touch. In a new Science Hub project with Amazon, he’s back on the case. He plans to build out the GelSight system with added capabilities to sense temperature and vibrations. A professor in MIT’s Department of Brain and Cognitive Sciences, Adelson recently sat down to talk about his work.

Q: What makes the human hand so hard to recreate in a robot?

A: A human finger has soft, sensitive skin, which deforms as it touches things. The question is how to get precise sensing when the sensing surface itself is constantly moving and changing during manipulation.

Q: You’re an expert on human and computer vision. How did touch grab your interest?

A: When my daughters were babies, I was amazed by how skillfully they used their fingers and hands to explore the world. I wanted to understand the way they were gathering information through their sense of touch. Being a vision researcher, I naturally looked for a way to do it with cameras.

Q: How does the GelSight robot finger work? What are its limitations?

A: A camera captures an image of the skin from inside, and a computer vision system calculates the skin’s 3D deformation. GelSight fingers offer excellent tactile acuity, far exceeding that of human fingers. However, the need for an inner optical system limits the sizes and shapes we can achieve today.

Q: How did you come up with the idea of giving a robot finger a sense of touch by, in effect, giving it sight?

A: A camera can tell you about the geometry of the surface it is viewing. By putting a tiny camera inside the finger, we can measure how the skin geometry is changing from point to point. This tells us about tactile properties like force, shape, and texture.

Q: How did your prior work on cameras figure in?

A: My prior research on the appearance of reflective materials helped me engineer the optical properties of the skin. We create a very thin matte membrane and light it with grazing illumination so all the details can be seen.

Q: Did you know there was a market for measuring 3D surfaces?

A: No. My postdoc Kimo Johnson posted a YouTube video showing GelSight’s capabilities about a decade ago. The video went viral, and we got a flood of email with interesting suggested applications. People have since used the technology for measuring the microtexture of shark skin, packed snow, and sanded surfaces. The FBI uses it in forensics to compare spent cartridge casings.

Q: What’s GelSight’s main application?  

A: Industrial inspection. For example, an inspector can press a GelSight sensor against a scratch or bump on an airplane fuselage to measure its exact size and shape in 3D. This application may seem quite different from the original inspiration of baby fingers, but it shows that tactile sensing can have many uses. As for robotics, tactile sensing is mainly a research topic right now, but we expect it to increasingly be useful in industrial robots.

Q: You’re now building in a way to measure temperature and vibrations. How do you do that with a camera? How else will you try to emulate human touch?

A: You can convert temperature to a visual signal that a camera can read by using liquid crystals, the molecules that make mood rings and forehead thermometers change color. For vibrations we will use microphones. We also want to extend the range of shapes a finger can have. Finally, we need to understand how to use the information coming from the finger to improve robotics.

Q: Why are we sensitive to temperature and vibrations, and why is that useful for robotics?

A: Identifying material properties is an important aspect of touch. Sensing temperature helps you tell whether something is metal or wood, and whether it is wet or dry. Vibrations can help you distinguish a slightly textured surface, like unvarnished wood, from a perfectly smooth surface, like wood with a glossy finish.

Q: What’s next?

A: Making a tactile sensor is the first step. Integrating it into a useful finger and hand comes next. Then you have to get the robot to use the hand to perform real-world tasks.

Q: Evolution gave us five fingers and two hands. Will robots have the same?

A: Different robots will have different kinds of hands, optimized for different situations. Big hands, small hands, hands with three fingers or six fingers, and hands we can’t even imagine today. Our goal is to provide the sensing capability, so that the robot can skillfully interact with the world.

What factors influence the embodiment felt towards parts of our bodies controlled by others? Using a new "joint avatar" whose left and right limbs are controlled by two people simultaneously, researchers have revealed that the visual information necessary to predict the partner's intentions behind limb movements can significantly enhance the sense of embodiment towards partner-controlled limbs during virtual co-embodiment. This finding may contribute to enhancing the sense of embodiment towards autonomous prosthetic limbs.

Researchers at Ulm University in Germany have recently developed a new framework that could help to make self-driving cars safer in urban and highly dynamic environments. This framework, presented in a paper pre-published on arXiv, is designed to identify potential threats around the vehicle in real-time.

Today’s integrated power modules are addressing the demanding size, weight, power budgets and cost efficiencies required to move robots from factory, residential and commercial applications into a vast new landscape bounded only by the imagination.

A chess-playing robot grabbed the finger of its 7-year-old opponent and broke it during last week's Moscow Chess Open tournament, Russian media reported Monday.

AGV’s and AMR’s address a number of key manufacturing process and facility-related concerns as well as key cost-related metrics in the migration towards continuous improvement philosophies such as lean manufacturing and six sigma.

Teams of mobile robots could be highly effective in helping humans to complete straining manual tasks, such as manufacturing processes or the transportation of heavy objects. In recent years, some of these robots have already been tested and introduced in real-world settings, attaining very promising results.

Dr. Akira Kakugo and his team from Hokkaido University in Japan sent us a video presentation of his recent paper ‘Cooperative cargo transportation by a swarm of molecular machines’, published in Science Robotics.

‘Despite the advancements in macro-scale robots, the development of a large number of small-size robots is still challenging, which is crucial to enhance the scalability of swarm robots,’ says Dr. Kakugo. In the paper, researchers showed it is possible to collectively transport molecular cargo by a swarm of artificial molecular robots (engineered systems with biological/molecular sensors, processors and actuators) responding to light.

• PAPER – Cooperative cargo transportation by a swarm of molecular machines. M. Akter, J. J. Keya, K. Kayano, A. M. R. Kabir, D. Inoue, H. Hess, K. Sada, A. Kuzuya, H. Asanuma, and A. Kakugo. Science Robotics, 7(65), p.eabm0677.

How do people communicate when they are underwater? With body language, of course.

Marine environments present a unique set of challenges that render several technologies that were developed for land applications completely useless. Communicating using sound, or at least as people use sound to communicate, is one of them.

Michael Fulton tackles this challenge with his presentation at ICRA 2022 by using body language to communicate with an AUV underwater. Tune in for more.

His poster can be viewed here.

Michael Fulton

Michael Fulton is a Ph.D. Candidate at the University of Minnesota Twin Cities. His research focuses primarily on underwater robotics with a focus on applications where robots work with humans. Specifically, human-robot interaction and robot perception using computer vision and deep learning, with the intent of creating systems that can work collaboratively with humans in challenging environments.

Simulation models show that this new mode of transport could have huge cost advantages, with savings between 20 and 50 per cent compared to traditional multi-tiered land transport systems.

When we announced AlphaFold 2 last December, it was hailed as a solution to the 50-year old protein folding problem. Last week, we published the scientific paper and source code explaining how we created this highly innovative system, and today we’re sharing high-quality predictions for the shape of every single protein in the human body, as well as for the proteins of 20 additional organisms that scientists rely on for their research.

When we announced AlphaFold 2 last December, it was hailed as a solution to the 50-year old protein folding problem. Last week, we published the scientific paper and source code explaining how we created this highly innovative system, and today we’re sharing high-quality predictions for the shape of every single protein in the human body, as well as for the proteins of 20 additional organisms that scientists rely on for their research.

As the underwater robot OceanOneK carefully navigated toward the upper deck railing of the sunken Italian steamship Le Francesco Crispi about 500 m below the Mediterranean's surface this month (roughly a third of a mile), Stanford University roboticist Oussama Khatib felt as though he himself was there.

The traditional portfolio mix of heavy hardware-orientation with associated software is transitioning to application and non-control software. The move to solutions combining hardware, software, and services is real.