The new IIT’s “Robot Teleoperativo” (teleoperation robot)

The Robot Teleoperativo is a new robotics result achieved at IIT-Istituto Italiano di Tecnologia and combining the rugged locomotion of HyQReal robot with dexterous and powerful manipulation assured by a new robotic arm, controlled by immersive VR visualization and haptic teleoperation. It represents a significant contribution to the state-of-the-art robotics dedicated to intervention on unstructured, difficult terrain environments, requiring powerful capabilities. The human operator can always be in control, not being replaced by the robot but substituted and assisted by it, in the case of situations where humans may be exposed to risks for their health, such as disaster response or emergency in nuclear, marine, chemical, oil-&-gas, environments.

In a new video, researchers show the achieved results: the Robot Teleoperativo is able to open doors, access fire equipment, gather precious items, navigate in the dark, and walk over difficult terrain. The video was showed during the international conference ICRA 2021, on May 31st.

With the effective execution of such tasks, albeit in simulated lab setup scenarios, the three years of developmental research in the Robot Teleoperativo project has shown that although the challenge of substituting human workers is a difficult one, it is certainly not an insurmountable one.

The project is a result of the close collaboration between IIT-Istituto Italiano di Tecnologia and INAIL (Italian National Worker’s Compensation Authority). The 3-year project started in 2017 with an aim to enhance occupational safety and health (OSH) in demanding and dangerous occupations. The goal was to keep the worker who is engaged in high-risk interventions in demanding environments, at the core of the development; researchers worked with firefighters to define specific needs and they will further explore the use of Robot Teleoperativo in realistic scenario.

Robot Teleoperativo consists of two modules, the field robot and the pilot station. The field robot integrates the versatile locomotion of the HyQReal hydraulic quadruped robot with the dexterity and power of the INAIL-IIT multi-degrees-of-freedom (DOFs) robotic arm. At the pilot station, the 6-DOF haptic teleoperation device REMOTArm, including the 3-finger HEXOTrAc-Plus hand exoskeleton, combines with the 3D virtual reality (VR) based intuitive, information-rich interface for perception, interaction, and teleoperation. The field-pilot combination offers an immersive telepresence experience to the operator (worker), allowing them to project their actions to and perceive the information from the remote environment through the field robot.

The HyQReal quadruped, with its 90 cm height, 133 cm length, and 130 kilograms of weight, demonstrated in 2019 being able to develop sufficient force to tow a passenger plane of over three tons. With its advanced locomotion, the HyQReal is capable of navigating difficult terrain, characterized by slopes, stairs, or debris, with a degree of mobility far greater than in the case of wheeled or tracked vehicles. The HyQReal is equipped with a variety of cameras and multimodal sensors for real-time environment perception.

INAIL-IIT arm is a robotic manipulator with strength and dexterity. The 5-DOF serial robotic arm provides the ability to interact with, and manipulate the environment with robustness, precision, and power. It is about 1.0m in length, and has a 4 kg payload capacity. Its 5 joints integrate compact and lightweight, torque controlled actuators, which allow the manipulator arm to operate in position and / or torque-control depending on the task. The HERI II robotic hand, with its multi-finger configuration, combines strength and dexterity for grasping heavy as well as delicate objects.

REMOTArm is the haptic teleoperation pilot interface for precise tele-manipulation. The REMOTArm is a desk-mounted, 6-DOF, haptic teleoperation device. It tracks the position and orientation of the operator’s wrist with high accuracy, and offers a much larger workspace than similar commercial devices. It provides 3-DOF force feedback at the wrist (> 20N). The 3-finger HEXOTrAc-Plus hand exoskeleton is an ergonomic haptic glove that accurately tracks the operator’s gestures when tele-manipulating the field manipulator. Its actuators can display the remotely sensed interaction forces (< 10 N) to the user at the robotic hand as they happen. The interface is lightweight, easy-to-wear, and includes gravity compensation for user comfort.

The operator uses a VR system for an immersive 3D perception of the remote environment. The user interface, using commercial head-mount display devices and VR software, makes for an intuitive and effective real-time projection of the field of action for the remote operator. Using the multimodal vision, depth, and ambient sensing system on-board the field robot, the interface “reconstructs” the remote field context for the operator, providing an information-rich interface for perception, interaction, and control.

This successful outcome has encouraged researchers to push the boundaries of technological innovation further. IIT and INAIL continue their close collaboration in this field aiming at expanding the possibilities in remote telerobotics research and its transition from the lab to the real hazardous work environments.

Further information

The research study was coordinated by Nikhil Deshpande, Researcher at IIT’s Advanced Robotics Line, and involved the research groups led by Claudio Semini (for HyQReal), Nikolaos Tsagarakis (for manipulation) and Ioannis Sarakoglou (for teleoperation) at IIT in Genova (Italy).


Two teams from Istituto Italiano di Tecnologia will compete at the Cybathlon Global Edition 2020, the Olympics dedicated to new prosthetic technologies

SoftHand Pro
SoftHand Pro. Credits: Istituto Italiano di Tecnologia

IIT’s teams will compete in the “Powered Arm Prosthesis” category showing two different robotic arm prostheses made in Italy: SoftHandPro and Hannes. The race course is about 30 metres long and will see the pilots compete in three races on 6 stations reproducing daily tasks. 60 teams from 23 countries will be involved in the event “remotely”, streamed on Cybathlon website starting from November 13th.

Maria Fossati interview. Credits: Istituto Italiano di Tecnologia

The IIT-Istituto Italiano di Tecnologia (Italian Institute of Technology) will participate in the Cybathlon Global Edition 2020, an international event organised by the Federal Institute of Technology (ETH) in Zurich, Switzerland, and dedicated to new prosthetic devices. People with physical disabilities from all over the world will compete, as pilots, in different disciplines that reproduce daily useful tasks, using the latest discoveries in technology such as robotic prostheses, exoskeletons and new generation wheelchairs. IIT will participate by presenting two robotic arm prostheses: the SoftHand Pro, stemming from a research project funded by the European Research Council (ERC), and the Hannes robotic hand developed together with the Italian Centro Protesi INAIL (the prosthetic unit of the National Institute for Insurance against Accidents at Work).

Cybathlon 2020 should have taken place in May at the Letzigrund Stadium in Zurich. However, this year for the first time, due to the current global health crisis, it will take place remotely in 23 countries around the world, from Genoa to Hong Kong, Cleveland to Paris and Melbourne to Tokyo. These countries will transmit their own races to the control center in Zurich where a single programming video will be created and streamed online on 13 November. The competition will be available on Cybathlon website.

The event aims to raise awareness among the general public on the needs of people living with a disability, and to showcase the state-of-the-art of technologies developed in this area also promoting collaboration between research centers.

Preparing for Cybathlon 2020 Credits: Istituto Italiano di Tecnologia

IIT will vie with two made-in-Italy robotics solutions in the “Powered Arm Prosthesis” category, lining up two different teams: Rehab Tech and SoftHand Pro. The two teams belongs to different IIT research groups, both engaged in research on the movements of the human hand and robotics. They will face competition along the same race track prepared at the IIT’s Center for Human Technologies (CHT-IIT) based in Genova (Italy).

The two teams will use two independent devices that have a common history: they were created in IIT starting from the Pisa/IIT SoftHand robotic hand technology, initially developed for humanoid robots and supported by European Research Council (ERC)’s grant.

Rehab Tech team
This team will use the biomimetic prosthetic hand Hannes, developed in the joint laboratory IIT-INAIL Rehab Technologies Lab, a Class 1 CE product which this year was awarded the Compasso d’Oro Award for design, with the goal of reaching the market in the short-term. The pilot is Colian Rossi, a technical designer who had a work accident in 2015, causing him an arm amputation. He is now an INAIL patient.

Colian Rossi from the Rehab Tech team during training

SoftHand Pro team
They will use the SoftHand Pro realized in the IIT’s Soft Robotics for Human Cooperation and Rehabilitation laboratory in collaboration with the University of Pisa. SoftHand Pro ensues from the ERC SoftHand project and it is now at the center of an ERC’s Synergy Grant (Natural Bionics) which, in future, aims to directly integrate the prosthesis with the subjects’ spinal circuits via neurosurgery. The pilot is Maria Fossati, industrial designer and IIT researcher, who constantly wears the prosthesis.

Maria Fossati from the SoftHand Pro team during training

Each team consists of a pilot and her/his entourage made up of researchers having the role of coaches, athletic trainers and real race mechanics, just as in motor racing competitions.

The race course for the “Powered Arm Prosthesis” category is about 30 metres long and will see the pilots compete in three races on six stations: preparing the table for breakfast, laundry, re-setting a table, using domestic tools (hammer, scissors, etc.), recognising shapes blindly and arranging glasses in the shape of a pyramid. The race will have a maximum duration of 8 minutes. The score is assigned on the basis of the number of completed tasks and the completion time.

Competing teams in this category are 14, from Italy, Sweden, India, South Africa, Russia, Croatia, Great Britain, China, Germany and Switzerland. Teams achieving the 4 best times in their discipline will receive an award on Saturday, 14 November.

Cybathlon 2020 will have a total of 60 teams divided into 6 race disciplines, identified by the pilot’s disability and technology in use. Besides the Power Arm Prosthesis category there is: The Powered Leg Prosthesis, the Powered Exoskeleton, the Powered Wheel Chair, the Brain Computer Interface and the Functional Electrical Stimulation Bike. The event will kick off at 3pm UTC. You can read more information about it in this post and on their website.