Archive 30.09.2017

Page 1 of 3
1 2 3

Robohub Podcast #244: Robot Pediatric Coach, with Ayanna Howard

In this episode, Audrow Nash interviews Ayanna Howard, Professor at the Georgia Institute of Technology, about her work to help children with the movement disorder cerebral palsy. Howard discusses how robots and tablet can be used to “gamify” pediatric therapy. The idea is that if therapy is fun and engaging children are more likely to do it, and thus, they are more likely to see the long-term benefits of the therapy. Howard discusses how therapy is “gamified,” how a small humanoid robot is used to coach children, and how they work with pediatricians.


Ayanna Howard

Ayanna Howard, Ph.D. is Professor and Linda J. and Mark C. Smith Endowed Chair in Bioengineering in the School of Electrical and Computer Engineering at the Georgia Institute of Technology. She also holds the position of Associate Chair for Faculty Development in ECE. She received her B.S. in Engineering from Brown University, her M.S.E.E. from the University of Southern California, and her Ph.D. in Electrical Engineering from the University of Southern California.

Her area of research is centered around the concept of humanized intelligence, the process of embedding human cognitive capability into the control path of autonomous systems. This work, which addresses issues of autonomous control as well as aspects of interaction with humans and the surrounding environment, has resulted in over 200 peer-reviewed publications in a number of projects – from scientific rover navigation in glacier environments to assistive robots for the home. To date, her unique accomplishments have been highlighted through a number of awards and articles, including highlights in USA Today, Upscale, and TIME Magazine, as well as being named a MIT Technology Review top young innovator and recognized as one of the 23 most powerful women engineers in the world by Business Insider.

In 2013, she also founded Zyrobotics, which is currently licensing technology derived from her research and has released their first suite of therapy and educational products for children with differing needs. From 1993-2005, Dr. Howard was at NASA’s Jet Propulsion Laboratory, California Institute of Technology. She has also served a term as the Associate Director of Research for the Georgia Tech Institute for Robotics and Intelligent Machines and a term as Chair of the multidisciplinary Robotics Ph.D. program at Georgia Tech.



Energy, enthusiasm and spirit of cooperation: Award winners of ERL Emergency Robots 2017 announced

The European Robotics League (ERL) announced the winners of ERL Emergency Robots 2017 major tournament, during the awards ceremony held on Saturday, 23rd September at Giardini Pro Patria, in Piombino, Italy.

The ERL Emergency Robots 2017 competition consisted of four scenarios, inspired by the nuclear accident of Fukushima (Japan, 2011) and designed specifically for multi-domain human-robot teams. The first scenario is The Grand Challenge made up of three domains – sea, air, land, and the other three scenarios are made of only two domains.

The Awards, given for each scenario to the best performing teams, were introduced by Alan Winfield from Bristol Robotics Laboratory and ERL Emergency Coordinator. “The energy, enthusiasm and spirit of cooperation among the teams competing in ERL Emergency was amazing. We witnessed not only great performances from the teams and their robots, but also the drama and excitement of last minute field repairs and workarounds to the robots”, said Alan Winfield.

The Grand Challenge (Scenario 1: land, sea, and air)

After a nuclear power plant has been struck by a potent earthquake and a tsunami, it’s time for the emergency response team to act. Due to high radiation levels, the cooperation of land, sea and air robots is essential. The robots have to find as soon as possible three missing workers and deploy an emergency kit next to them. Secondly, the robots check for any structural damage of the building and to the pipes connecting the reactor to the sea for cooling purposes. In case of damaged or leaking pipes, the corresponding valves are to be closed both in the machine room and underwater to avoid radioactive contamination. Closing the wrong valves may cause a reduction in the amount of seawater available for cooling down the reactor.

1st Prize: Telerob, Germany (land) + Universitat de Girona, Spain (sea) + INESTEC/ISEP Aerial Robotics, Portugal (air)

“Our underwater robot Sparus II AUV was used to create maps of the underwater environment and to autonomously detect some targets. The algorithms developed by master and PhD students and the robustness of the platform allowed us to obtain good results even in the challenging conditions of the competition. The multi-domain competition required the coordination with the other robots (land and air), which offered us a unique opportunity for testing our communication capabilities.”, said Marc Carreras from the University of Girona.

“Autonomy was showing its advantages as well as a good situational awareness. The advanced mission documentation, which was requested in the competition, enables the first responders to get a fast and reliable situational understanding to finally reach the necessary situational ownership. The ERL Emergency competition should further focus on improving the robot-human-teaming”, said Andreas Ciossek from Telerob.

“The INESC TEC participation in ERL Emergency 2017 allowed us to validate our robotics technology in a real-world scenario with a relevant social and economical impact. Furthermore, it helped raising public awareness of the role that advanced robots can play in disaster scenarios aiding human teams in critical operations, and it confirmed once again the leading role of European robotics research”, said Eduardo Silva from ISEP/INESC TEC.

Read more about their experience here.

2nd Prize: IIS Piombino CVP, Italy (air) + Robdos, Spain, + IMM, Poland (land and sea)

3rd Prize: Raptors, Poland (air & land) + Oubot, Hungary (sea)

Survey the building and search for missing workers (Scenario 2: land and sea)

The ground and aerial robots perform a reconnaissance mission of the area and create a map of the surrounding area in order to increase the awareness of the emergency response team. Additionally, the robots find two missing workers outdoors and deploy first-aid kits near them.

1st Prize: IMM, Poland + IIS Piombino CVP, Italy

2nd Prize: Raptors Team, Poland

3rd Prize: Telerob, Germany + INESTEC/ISEP Aerial Robotics, Portugal

Pipe inspection and search for missing workers (Scenario 3: sea and air)

After the earthquake and tsunami, the pipes connecting the reactor to the sea might be leaking radioactive substances, therefore the emergency team has to find the damaged ones on land or underwater. Robots must find two missing workers: one outside the building, to whom an emergency kit should be deployed, and another one dragged by the tsunami to the sea, expected to be a casualty.

1st Prize: Universitat de Girona, Spain + INESTEC/ISEP Aerial Robotics, Portugal

2nd Prize: Tuscany Robotics Team, Italy

3rd Prize: AUV Tomkyle, Germany + HSR Search and Rescue Team, Switzerland

“Aerial robots have shown great improvements with respect to euRathlon 2015, most of them being fully operational from the first day of the competition. They have been able to quickly provide information about inaccessible areas, structural damages or other possible threats. As it happened with robots from the other two domains, aerial teams also struggled with communication issues to properly command and control their platforms. This further confirms the need for autonomous capabilities onboard the aerial robots, to become even more powerful tools for emergency response teams”, said Francisco Javier Pérez Grau from the Advanced Center for Aerospace Technologies.

Stem the leak (Scenario 4: land and sea)

The land robots have to inspect the pipes in the building’s machine room and marine robots the underwater pipes in order to close the correct valves and prevent leakage. Land and marine robots must cooperate to identify the valves and synchronize the process of closing them, by communicating directly or via their operators.

1st Prize: Telerob, Germany + Universitat de Girona, Spain

2nd Prize: Raptors, Poland + Oubot, Hungary

3rd Prize: bebot, Switzerland + AUV Tomkyle Team, Germany

Winning teams were given a diploma, prize money and in-kind, sponsored by the Platinum Sponsor IEEE Oceanic Engineering Society, silver sponsors SBG systems and Texas Instruments.

In addition, Marta Palau Franco, Bristol Robotics Laboratory, ERL Emergency project manager introduced the referees’ special awards. Find out who the winners are here!

“This great event has been possible thanks to the work and effort of an amazing local organising team. Special thanks to Fausto Ferreira, the ERL Emergency 2017 Deputy Director, for his continuous support. I want to thank all the sponsors, especially our platinum sponsor IEEE OES, the project partners, referees, local associations and schools for their support. Huge thanks go to the participating teams, which were the heart of this great event. Their competitiveness pushed the robots to accomplish great results, nevertheless the competition has always been accompanied by fair play. I believe this is the perfect formula for team members to improve their professional and human skills”, said Gabriele Ferri, ERL Emergency 2017 Director.


 Watch the ERL Emergency 2017 Awards Ceremony video

More info

The European Robotics League is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement n° 688441.

The ERL Emergency Robots 2017 competition was organised locally by the NATO STO-Centre for Maritime Research and Experimentation (CMRE) of La Spezia, Italy.

The European Robotics League is part of the SPARC public-private partnership set up by the European Commission and euRobotics to extend Europe’s leadership in civilian robotics. SPARC’s €700 million of funding from the Commission in 2014̶20 is being combined with €1.4 billion of funding from European industry.

euRobotics is a European Commission-funded non-profit organisation which promotes robotics research and innovation for the benefit of Europe’s economy and society. It is based in Brussels and has more than 250-member organisations.

The referees’ special awards ERL Emergency Robots 2017

The European Robotics League (ERL) announced the winners of ERL Emergency Robots 2017 major tournament, during the awards ceremony held on Saturday, 23rd September at Giardini Pro Patria, in Piombino, Italy.

In addition to the Competition Awards, Marta Palau Franco from Bristol Robotics Laboratory and ERL Emergency project manager introduced the referees’ special awards.

“Behind a multi-domain competition there is always a large technical committee, I feel privileged to have worked with such an amazing team of volunteer referees, technical assistants and safety pilots and divers. We were delighted to give these awards to recognise teams’ effort, fair play and hard work. The experience of participating in this robotics competition will prove beneficial for team members to develop further their professional career”, said Marta Palau Franco.

Mapping Award, handed by Vladimir Djapic from AFAK, for good quality georeferenced undersea mapping.
Winner: AUV Team Tomkyle, Germany (sea)

Navigation Award, handed by Pino Casalino from the University of Genova, for the effort to change and adapt algorithms to navigate without a Doppler Velocity Log (DVL) sensor, important for the AUV (autonomous Undersea Vehicle) navigation.
Winner: Oubot Team, Hungary (sea)

Fair Play Award, handed by Marta Palau Franco from the University of the West of England, Bristol, for lending to the Tuscany Robotics Team a wheel platform for their new robot and for lending the batteries of their aerial robot to ISEP/INESC TEC aerial team.
Winners: ENSTA Team and ENSTA Bretagne, France (land, air, sea)

Creativity Award, handed by Bernd Bruggermann from Fraunhofer FKIE, for building a land robot from scratch in less than two days when their ground platform broke.
Winner: Tuscany Robotics, Italy (land, air, sea)

Multi-domain Cooperation Award, handed by Fausto Ferreira from CMRE, for cooperation between domains. The teams used a graphical interface in which each robot from sea and air domain reported its findings in the competition arena in real-time.
Winner: Universitat de Girona, Spain (sea) + ISEP/INESC TEC, Portugal (air)

Perseverance Award, handed by Francisco Javier Perez Grau from FADA-CATEC, for hard work on the development and integration of the aerial robot. The team competed in the Grand Challenge the day after their aerial platform suffered a severe crash, working overnight to fix it.
Winner: HSR Search and Rescue Team, Switzerland (air)

Piloting Award, handed by Stjepan Bogdan from University of Zagreb-FER , for outstanding UAV piloting skills. The team was able to recover the aerial robot after an unintentional landing without incurring manual intervention.
Winner: Raptors, Poland (land & air)

Autonomy Award, handed by Frank Schneider from Fraunhofer FKIE, for the best autonomy of land robots. Outstanding autonomous navigation and automatic object detection.
Winner: IMM, Poland (land)

SAUC-E Student Award – handed by Bill Kirkwood (IEEE OES), Kelly Cooper (ONR) and Hitesh Patel (AUVSI) to the best student marine team.
Winner: AUV Team Tomkyle, Germany (sea)

Teams were given a diploma and set of eZ430-Chronos development tool sponsored by Texas Instruments.


Watch the ERL Emergency 2017 Awards Ceremony video

More info

The European Robotics League is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement n° 688441.

The ERL Emergency Robots 2017 competition was organised locally by the NATO STO-Centre for Maritime Research and Experimentation (CMRE) of La Spezia, Italy.

A language to design control algorithms for robot swarms

Designing and representing control algorithms is challenging in swarm robotics, where the collective swarm performance depends on interactions between robots and with their environment. The currently available modeling languages, such as UML, cannot fully express these interactions. The Behaviour-Data Relations Modeling Language (BDRML) explicitly represents robot behaviours and data that robots utilise, as well as relationships between them. This allows BDRML to express control algorithms where robots cooperate and share information with each other while interacting with the environment. Here’s the work I presented this week at #IROS2017.

BDRML primitives

Primitives are the basic blocks of BDRML. They include:

  • Behaviour: A set of processes that deal with a particular situation a robot finds itself in, for example “Scout”
  • Internal data structure: Information that is stored in a robot’s memory
  • External data structure: Information that is stored in a non-robot entity, i.e., in the robot’s environment

BDRML primitives

BDRML relations

The following relations between entities can exist:

  • Transition: The robot transitions from one behavioural mode to another
  • Read \ Write: Internal data is used \ stored by the robot engaged in particular behaviour
  • Receive \ Send: External data is used \ stored by the robot. In the case of the Send relation, a robot may also send the data to another robot that stores it in its own internal data structure
  • Copy: Information is copied from one data structure to another
  • Update: The value of a data structure is updated from that in the previous time step by a subroutine not visualised in the BDRML diagram (for example, a pheromone level may spontaneously decrease over time).

The write and send relations can optionally define the new data structure value or a function that updates the value, indicated by a dashed line extending from the end of the relation arrow in a visual description, and written before a colon proceeding the data structure name in a textual description. The \textit{update} relation always must specify the new value or the value update function.

BDRML relations

BRML relation conditions

Each relation or operation occurs under a specific set of conditions. A condition is visually represented as an annotated triangle at the beginning of a relation or operation arrow. In a textual representation, a condition set follows a relation signature and is separated from it by a colon. Unless otherwise specified, the “or” logical operator is used when multiple conditions affect a single relation.

BDRML relation conditions


A full BDRML representation consists of both visual and textual specification. A set of behaviours, B, internal data structures, Di and external data structures, De, are first defined, followed by a list of relations between them. Each box, circle and arrow in the visual representation must have a corresponding element or line in the textual representation and vice versa.

A BDRML representation exampleAn example is shown in the picture. The described algorithm allows robots to search for worksites and recruit each other to perform work and it can be applied for decentralised task allocation. A robot performs the “Scout” behaviour by searching the environment for worksites that can be found with a probability p(F). A successful Scout, that finds a worksite, performs the “Work” behaviour, during which it reads from and writes into its internal data structure, “Worksite location”, to keep track of where the worksite is located. Additionally, a working robot sends Worksite location to any Scout that it encounters in order to recruit it.

Note how the condition that allows a robot to transition from the Scout to the Work behaviour can be triggered by both p(F) or by recruitment, i.e., by existence of the internal data structure in the Scout’s memory. Also note that the condition of recruitment, “scout encountered” signifies that the two robots have to be at a similar place at a similar time for recruitment to occur. The BDRML diagram fully and unambiguously describes when recruitment is performed, what information is exchanged between robots and how it affects robot behaviour.

Pitonakova, L., Crowder, R. & Bullock, S. (in press). Behaviour-Data Relations Modelling Language For Multi-Robot Control Algorithms. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017), IEEE.

“Superhero” robot wears different outfits for different tasks

Dubbed “Primer,” a new cube-shaped robot can be controlled via magnets to make it walk, roll, sail, and glide. It carries out these actions by wearing different exoskeletons, which start out as sheets of plastic that fold into specific shapes when heated. After Primer finishes its task, it can shed its “skin” by immersing itself in water, which dissolves the exoskeleton. Credit: the researchers.

From butterflies that sprout wings to hermit crabs that switch their shells, many animals must adapt their exterior features in order to survive. While humans don’t undergo that kind of metamorphosis, we often try to create functional objects that are similarly adaptive — including our robots.

Despite what you might have seen in “Transformers” movies, though, today’s robots are still pretty inflexible. Each of their parts usually has a fixed structure and a single defined purpose, making it difficult for them to perform a wide variety of actions.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are aiming to change that with a new shape-shifting robot that’s something of a superhero: It can transform itself with different “outfits” that allow it to perform different tasks.

Dubbed “Primer,” the cube-shaped robot can be controlled via magnets to make it walk, roll, sail, and glide. It carries out these actions by wearing different exoskeletons, which start out as sheets of plastic that fold into specific shapes when heated. After Primer finishes its task, it can shed its “skin” by immersing itself in water, which dissolves the exoskeleton.

“If we want robots to help us do things, it’s not very efficient to have a different one for each task,” says Daniela Rus, CSAIL director and principal investigator on the project. “With this metamorphosis-inspired approach, we can extend the capabilities of a single robot by giving it different ‘accessories’ to use in different situations.”

Primer’s various forms have a range of advantages. For example, “Wheel-bot” has wheels that allow it to move twice as fast as “Walk-bot.” “Boat-bot” can float on water and carry nearly twice its weight. “Glider-bot” can soar across longer distances, which could be useful for deploying robots or switching environments.

Primer can even wear multiple outfits at once, like a Russian nesting doll. It can add one exoskeleton to become “Walk-bot,” and then interface with another, larger exoskeleton that allows it to carry objects and move two body lengths per second. To deploy the second exoskeleton, “Walk-bot” steps onto the sheet, which then blankets the bot with its four self-folding arms.

“Imagine future applications for space exploration, where you could send a single robot with a stack of exoskeletons to Mars,” says postdoc Shuguang Li, one of the co-authors of the study. “The robot could then perform different tasks by wearing different ‘outfits.’”

The project was led by Rus and Shuhei Miyashita, a former CSAIL postdoc who is now director of the Microrobotics Group at the University of York. Their co-authors include Li and graduate student Steven Guitron. An article about the work appears in the journal Science Robotics on Sept. 27.

Robot metamorphosis

Primer builds on several previous projects from Rus’ team, including magnetic blocks that can assemble themselves into different shapes and centimeter-long microrobots that can be precisely customized from sheets of plastic.

While robots that can change their form or function have been developed at larger sizes, it’s generally been difficult to build such structures at much smaller scales.

“This work represents an advance over the authors’ previous work in that they have now demonstrated a scheme that allows for the creation of five different functionalities,” says Eric Diller, a microrobotics expert and assistant professor of mechanical engineering at the University of Toronto, who was not involved in the paper. “Previous work at most shifted between only two functionalities, such as ‘open’ or ‘closed’ shapes.”

The team outlines many potential applications for robots that can perform multiple actions with just a quick costume change. For example, say some equipment needs to be moved across a stream. A single robot with multiple exoskeletons could potentially sail across the stream and then carry objects on the other side.

“Our approach shows that origami-inspired manufacturing allows us to have robotic components that are versatile, accessible, and reusable,” says Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT.

Designed in a matter of hours, the exoskeletons fold into shape after being heated for just a few seconds, suggesting a new approach to rapid fabrication of robots.

“I could envision devices like these being used in ‘microfactories’ where prefabricated parts and tools would enable a single microrobot to do many complex tasks on demand,” Diller says.

As a next step, the team plans to explore giving the robots an even wider range of capabilities, from driving through water and burrowing in sand to camouflaging their color. Guitron pictures a future robotics community that shares open-source designs for parts much the way 3-D-printing enthusiasts trade ideas on sites such as Thingiverse.

“I can imagine one day being able to customize robots with different arms and appendages,” says Rus. “Why update a whole robot when you can just update one part of it?”

This project was supported, in part, by the National Science Foundation.

#ERLEmergency2017 in tweets

The ERL Emergency Robots 2017 (#ERLemergency2017) major tournament in Piombino, Italy, gathered 130 participants from 16 universities and companies from 8 European countries. Participating teams designed robots able to bring the first relief to survivors in disaster-response scenarios. The #ERLemergency2017 scenarios were inspired by the Fukushima 2011 nuclear accident. The robotics competition took place from 15-23 September 2017 at Enel’s Torre del Sale, and saw sea, land and air robots collaborating.

Teams worked very hard during the practice and competition days:

Robots could be found also in the exhibition area:

Or enjoying the sea:

Robotics experts held presentations and demos for the general public during the Opening Ceremony in Piazza Bovio.

The public got to know the teams,

And to see some emergency robots in action with the demo of TRADR project:

The competition site benefitted from the visit of some personalities:

Since they are the new generation of roboticists, children were not forgotten either: they enjoyed the free classes given by Scuola di Robotica.

At the Piombino Castle, the public attended more #robotics presentations.

After days of hard work, passion and enjoyment, the winners of the Grand challenge were announced:

Drones can almost see in the dark

We want to share with you our recent breakthrough teaching drones to fly using an eye-inspired camera, which opens the door to them performing fast, agile maneuvers and flying in low-light environments, where all commercial drones fail. Possible applications include supporting rescue teams with search missions at dusk or dawn. We submitted this work to the IEEE Robotics and Automation Letters.

How it works
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high speed motions or in scenes characterized by high dynamic range. However, event cameras output only little information when the amount of motion is limited, such as in the case of almost still motion. Conversely, standard cameras provide instant and rich information about the environment most of the time (in low-speed and good lighting scenarios), but they fail severely in case of fast motions or difficult lighting such as high dynamic range or low light scenes. In this work, we present the first state estimation pipeline that leverages the complementary advantages of these two sensors by fusing in a tightly-coupled manner events, standard frames, and inertial measurements. We show that our hybrid pipeline leads to an accuracy improvement of 130% over event-only pipelines, and 85% over standard-frames only visual-inertial systems, while still being computationally tractable. Furthermore, we use our pipeline to demonstrate—to the best of our knowledge—the first autonomous quadrotor flight using an event camera for state estimation, unlocking flight scenarios that were not reachable with traditional visual inertial odometry, such as low-light environments and high dynamic range scenes: we demonstrate how we can even fly in low light (such as after switching completely off the light in a room) or scenes characterized by a very high dynamic range (one side of the room highly illuminated and another side of the room dark).

T. Rosinol Vidal, H.Rebecq, T. Horstschaefer, D. Scaramuzza
Hybrid, Frame and Event based Visual Inertial Odometry for Robust, Autonomous Navigation of Quadrotors, submitted to IEEE Robotics and Automation Letters PDF

#IROS2017 Live Coverage

The 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017) is being held in Vancouver Canada this week. The theme of IROS 2017 is “Friendly People, Friendly Robots”. Robots and humans are becoming increasingly integrated in various application domains. We work together in factories, hospitals and households, and share the road. This collaborative partnership of humans and robots gives rise to new technological challenges and significant research opportunities in developing friendly robots that can work effectively with, for, and around people.

And it’s also IROS’s 30th birthday this year! The occasion for much celebration.

Many Robohubbers will be at IROS, watch out for Sabine, Audrow, Andra, Hallie, and AJung. We’re also looking for new members to join our community. If you’re interested, email, and we’ll make sure to meet during the conference!

Udacity Robotics video series: Interview with Chris Anderson from 3D Robotics

Mike Salem from Udacity’s Robotics Nanodegree is hosting a series of interviews with professional roboticists as part of their free online material.

This week we’re featuring Mike’s interview with Chris Anderson, Co-Founder and CEO of 3D Robotics. Chris is a former Wired magazine editor turned robotics company co-founder and CEO. Learn about Chris’s amazing journey into the world of unmanned aerial vehicles.

You can find all the interviews here. We’ll be posting them regularly on Robohub.

Talking Machines: The long view and learning in person, with John Quinn

In episode nine of season three we chat about the difference between models and algorithms, take a listener question about summer schools and learning in person as opposed to learning digitally, and we chat with John Quinn of the United Nations Global Pulse lab in Kampala, Uganda and Makerere University’s Artificial Intelligence Research group.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Robotics, the traditional path and new approaches

The hype cycle representation of the robotics field based on the general interest since its inception obtained from a joint review of publications, conferences, events and solutions. From “Dissecting Robotics — historical overview and future perspectives”.

Robotics, like many other technologies, suffered from an inflated set of expectations resulting in a decrease of the developments and results during the 90s. Over the last years, several groups thought that flying robots, commonly known as drones, would address these limitations however it seems unlikely that the popularity of these flying machines will drive and push the robot growth as expected. This article aims to summarize traditional techniques used to build and program robots together with new trends that aim to simplify and enhance the progress in the field.

Building robots

It’s a rather popular thought that building a robot and programing its behavior remain two highly complicated tasks. Recent advances in adopting ROS as a standardized software framework for developing robot applications helped with the latter, however building a robot remains a challenge. The lack of compatible systems in terms of hardware, the non existing marketplace of reusable modules, or the expertise required to develop the most basic behaviors are some of the few listed hurdles.

The integration-oriented approach
Robots are typically built by following the step-by-step process described below:

1. Buy parts: We typically decide on what components our robot will need. A mobile base, a range finder, a processing device, etc. Once decided we fetch those that match our requirements and proceed towards integration.

2. Integration: Making different components speak to each other and cooperate towards achieving the end goal of the robot. Surprisingly, that’s where most of our time is spent.

3. “build the robot”: Assembling all of the components into joints and mechanically linking them. This step might also get executed together with step

4. Programming the robot: Making the robot do what it’s meant to do.

5. Test & adapt: Robots are typically programmed in predictable scenarios. Testing the pre-programmed behavior in real scenarios is always critical. Generally, these tests deliver results that indicate where adaptations are needed, which in many cases pushes de process of building a robot down to step 2 again, integration.

6. Deploy: Ship it!

The “integration-oriented” approach for building a robot.

It’s well understood that building a robot is a technically challenging task. Engineers often face situations where the integration effort of the robot, generally composed by diverse sub-components, supersedes many other tasks. Furthermore, every hardware modification/adaptation while programming or building the robot demands further integration.

This method for building robots produces results that become obsolete within a short period.

Moreover, modules within the robots aren’t reusable in most of the cases since the integration effort makes reusability an incredibly expensive (manpower-wise) and time-consuming task.

The modular approach

The existing trend in robotics is producing a significant number of hardware devices. Although there’s an existing trend towards using the Robot Operating System (ROS), when compared to each other, these components typically consist of incompatible electronic components with different software interfaces.

Now, imagine building robots by connecting interoperable modules together. Actuator, sensors, communication modules, UI devices, provided everything interoperates together, the whole integration effort could be eliminated. The overall process of building robots could be simplified and the development effort and time will be reduced significantly.

Comparison between the “integration-oriented” and the “modular” approaches for building robots.

Modular components could be reused among robots and that’s exactly what we’re working on with H-ROS, the Hardware Robot Operating System.
H-ROS is a vendor-agnostic infrastructure for the creation of robot modules that interoperate and can be exchanged between robots. H-ROS builds on top of ROS, the Robot Operating System, which is used to define a set of standardized logical interfaces that each physical robot component must meet if compliant with H-ROS.

Programming robots

The robotics control pipeline

Traditionally, the process of programming a robot for a given task T is described as follows:
Observation: Robot’s sensors produce measurements. All these measurements receive the name of “observations” and are the inputs that the robot receives to execute task T.

State estimation: Given the observations of step 1, we describe the robot’s motion over time by inferring a set of characteristics of the robot such as its position, its orientation or its velocity. Obviously, mistakes in the observations will lead to errors in the state estimation.

Modeling & Prediction: Determine the dynamics of the robot (rules for how to move it around) using a) the robot model (typically the URDF of the robot in the ROS world) and b) the state estimation. Similarly to what happened with the previous step, errors in “state estimation” will impact the results obtained in this step.

Planning: this step determines the actions required to execute task T and uses both the state estimation and the dynamical model from previous steps in the pipeline.

Low level control: the final step in the pipeline consists of transforming the “plan” into low level control commands that steer the robot actuators.

The traditional “robotics control pipeline”

Bio-inspired techniques

Artificial Intelligence methods and, particularly, bio-inspired techniques such as artificial neural networks (ANNs) are becoming more and more relevant in robotics. Starting from 2009, ANNs gained popularity and started delivering good results in the fields of computer vision (2012) or machine translation (2014). Nowadays, these fields are completely filled by techniques that simulate the neural/synaptic activity of the brain of a living organism.

During the last years we have seen how these techniques have been translated to robotics for tasks such as robotic grasping (2016). Our team has been putting resources into exploring these techniques
that enable to train a robotic device in a manner conceptually similar to the mode in which one goes about training a domesticated animal such as a dog or cat.

Training robots end-to-end for a given task. This integrated and bio-inspired approach conflicts with the traditional robotics pipeline, however it’s already showing promising results of behaviors that generalize.
We are excited to share that it’s within our expectations to see more active use of these bio-inspired techniques. We are confident that its use will drive innovations with high impact for robotics and we hope to contribute by opening part of our work and results.

Programming robots versus training robots. This image pictures the traditional robotics approach named as the “robotics control pipeline” and the new “bio-inspired” approach that makes use of AI techniques that simulate the neural/synaptic activity of the brain.

The roboticist matrix

All these new approaches for both building and programming robots bring a dilemma to roboticists. What should they focus on? Which approach should they follow for each particular use case? Let’s analyze the different combinations:

The roboticist matrix presents a comparison between traditional and new approaches for building and programming robots.

Integration-oriented + robotics control pipeline:
This combination represents the “traditional approach” in all senses. It’s the process that most robot solutions use nowadays in industry. Integrated robots that typically belong to a single manufacturer. Such robots are programmed in a structured way to execute a well defined task. Typically achieving high levels of accuracy and repeatability. However, any uncertainty in the environment will typically drive the robot to fail on its task. Expenses related to develop such systems are typically in the range of 10.000–100.000 € for the simplest behaviors and an order of magnitud above for the more complex tasks.

Integration-oriented + bio-inspired:
Behaviors that evolve, but with strong hardware constraints and limitations. Traditional robots enhanced with bio-inspired approaches. Robots using this combination will be able to learn by themselves and adapt to changes in the environment however any modification, repurpose or extension within the robot hardware will require big integration efforts. The expenses for developing these robots are similar to the ones presented for the “traditional approach”.

Modular + robotics control pipeline:
Flexible hardware with structured behaviors. These robots will be built in a modular way. Building, repairing and/or repurposing these robots will be extremely affordable when compared to traditional robots (built with the integration-oriented approach), we estimate an order of magnitude less (1.000–10.000 €). Furthermore, modularity will introduce new opportunities for these robots.

Modular + bio-inspired:
This combination represents the most innovative one and has the potential to disrupt the whole robotics market changing both the way we build and program/train robots. Yet it’s also the most immature one.
Similar to the previous approach group, our team foresees that the expenses for putting together these robots can also be reduced when compared to the more traditional approaches. We estimate that building and training these robots should range in terms of expenses from 1.000 to 10.000 € for simple scenarios and up to 50.000 € for the more elaborated ones.

Our path towards the future: modular robots and H-ROS

A modular robot built using H-ROS compatible components.

The team behind Erle Robotics is proud to announce that together with Acutronic Robotics (Switzerland), Sony (Japan) is now also pushing the development of H-ROS, the Hardware Robot Operating System. A technology that aims to change the landscape of robotics by creating an ecosystem where hardware components can be reused among different robots, regardless of the original manufacturer. Our team strongly believe that the future of robotics will be about modular robots that can be easily repaired and reconfigured. H-ROS aims to shape this future. Sony’s leadership and vision in robotics is widely recognized in the community. We are confident that, with the addition of Sony as a supporter, our present innovations will spread even more rapidly.

Our team is focused in exploring these new opportunities and will introduce some results this week in Vancouver during ROSCon. Show your interest and join us in Canada!

The importance of research reproducibility in robotics

As highlighted in a previous post, despite the fact that robotics is increasingly regarded as a ‘Science’, as shown by the launch of new journals such as Science Robotics, reproducibility of experiments is still difficult or entirely lacking.

This is quite unfortunate as the possibility of reproducing experimental results is a cornerstone of the scientific method. This situation pushes serious discussions (What’s ‘soft robotics’? Is it needed? What has to be ’soft’?) and paradigm clashes (Good Old Fashioned Artificial Intelligence vs. Deep Learning vs. Embodied Cognition) into the realm of literary controversy or even worse religious territory fights, with quite little experimental evidence supporting the claims of the different parties. Not even wrong, as they say (following Peter Woit’s arguments on String Theory)?

The robotics community has been aware of these issues for a long time and more and more researchers in recent years have published datasets, code and other valuable information to allow others to reproduce their results. We are heading in the right direction, but we probably need to do more.

I think we should therefore welcome the fact that for the first time ever, IEEE R&A Mag. will start accepting R-articles (i.e., papers that report experiments aiming to be fully reproducible) beginning this September. Actually, they will also accept short articles reporting on the replication of R-Article results, and author replies are solicited and will be published after peer-review. The result will be a two-stage high-quality review process. The first stage will be the ordinary rigorous review process of a top-tier publishing venue. The second stage will be the replication of the experiments by the community (which is the core of the scientific method).

This seems like a historical improvement, doesn’t it?

There is more information on this in the column I wrote in the September issue of IEEE Robotics and Automation.

Automatic code reuse

“CodeCarbonCopy enables one of the holy grails of software engineering: automatic code reuse,” says Stelios Sidiroglou-Douskos, a research scientist at CSAIL. Credit: MIT News

by Larry Hardesty

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a new system that allows programmers to transplant code from one program into another. The programmer can select the code from one program and an insertion point in a second program, and the system will automatically make modifications necessary — such as changing variable names — to integrate the code into its new context.

Crucially, the system is able to translate between “data representations” used by the donor and recipient programs. An image-processing program, for instance, needs to be able to handle files in a range of formats, such as jpeg, tiff, or png. But internally, it will represent all such images using a single standardized scheme. Different programs, however, may use different internal schemes. The CSAIL researchers’ system automatically maps the donor program’s scheme onto that of the recipient, to import code seamlessly.

The researchers presented the new system, dubbed CodeCarbonCopy, at the Association for Computing Machinery’s Symposium on the Foundations of Software Engineering.

“CodeCarbonCopy enables one of the holy grails of software engineering: automatic code reuse,” says Stelios Sidiroglou-Douskos, a research scientist at CSAIL and first author on the paper. “It’s another step toward automating the human away from the development cycle. Our view is that perhaps we have written most of the software that we’ll ever need — we now just need to reuse it.”

The researchers conducted eight experiments in which they used CodeCarbonCopy to transplant code between six popular open-source image-processing programs. Seven of the eight transplants were successful, with the recipient program properly executing the new functionality.

Joining Sidiroglou-Douskos on the paper are Martin Rinard, a professor of electrical engineering and computer science; Fan Long, an MIT graduate student in electrical engineering and computer science; and Eric Lahtinen and Anthony Eden, who were contract programmers at MIT when the work was done.

Mutatis mutandis

With CodeCarbonCopy, the first step in transplanting code from one program to another is to feed both of them the same input file. The system then compares how the two programs process the file.

If, for instance, the donor program performs a series of operations on a particular piece of data and loads the result into a variable named “mem_clip->width,” and the recipient performs the same operations on the same piece of data and loads the result into a variable named “picture.width,” the system will infer that the variables are playing the same roles in their respective programs.

Once it has identified correspondences between variables, CodeCarbonCopy presents them to the user. It also presents all the variables in the donor for which it could not find matches in the recipient, together with those variables’ initial definitions. Frequently, those variables are playing some role in the donor that’s irrelevant to the recipient. The user can flag those variables as unnecessary, and CodeCarbonCopy will automatically excise any operations that make use of them from the transplanted code.

New order

To map the data representations from one program onto those of the other, CodeCarbonCopy looks at the precise values that both programs store in memory. Every pixel in a digital image, for instance, is governed by three color values: red, green, and blue. Some programs, however, store those triplets of values in the order red, green, blue, and others store them in the order blue, green, red.

If CodeCarbonCopy finds a systematic relationship between the values stored by one program and those stored by the other, it generates a set of operations for translating between representations.

CodeCarbonCopy works well with file formats, such as images, whose data is rigidly organized, and with programs, such as image processors, that store data representations in arrays, which are essentially rows of identically sized memory units. In ongoing work, the researchers are looking to generalize their approach to file formats that permit more flexible data organization and programs that use data structures other than arrays, such as trees or linked lists.

“In general, code quoting is where a lot of problems in software come from,” says Vitaly Shmatikov, a professor of computer science at Cornell Tech, a joint academic venture between Cornell University and Israel’s Technion. “Both bugs and security vulnerabilities — a lot of them occur when there is functionality in one place, and someone tries to either cut and paste or reimplement this functionality in another place. They make a small mistake, and that’s how things break. So having an automated way of moving code from one place to another would be a huge, huge deal, and this is a very solid step toward having it.”

“Recognizing irrelevant code that’s not important for the functionality that they’re quoting, that’s another technical innovation that’s important,” Shmatikov adds. “That’s the kind of thing that was an obstacle for a lot of previous approaches — that you know the right code is there, but it’s mixed up with a lot of code that is not relevant to what you’re trying to do. So being able to separate that out is a fairly significant technical contribution.”

Descartes revisited: Do robots think?

This past week, a robotic first happened: ABB’s Yumi robot conducted the Lucca Philharmonic Orchestra in Pisa, Italy. The dual-armed robot overshadowed even his vocal collaborator, Italian tenor Andrea Bocelli. While many will try to hype the performance as ushering in a new new era of mechanical musicians, Yumi’s artistic career was short-lived as it was part of the opening ceremonies of Italy’s First International Festival of Robotics.

Italian conductor Andrea Colombini said of his student, “The gestural nuances of a conductor have been fully reproduced at a level that was previously unthinkable to me. This is an incredible step forward, given the rigidity of gestures by previous robots. I imagine the robot could serve as an aid, perhaps to execute, in the absence of a conductor, the first rehearsal, before the director steps in to make the adjustments that result in the material and artistic interpretation of a work of music.”

Harold Cohen with his robot AARON

Yumi is not the first computer artist. In 1973, professor and artist, Harold Cohen created a software program called AARON – a mechanical painter. AARON’s works have been exhibited worldwide, including at the prestigious Venetian Biennale. Following Cohen’s lead, Dr Simon Colton of London’s Imperial College created “The Painting Fool,” with works on display in Paris’ prestigious Galerie Oberkampf in 2013. Colton wanted to test if he could cross the emotional threshold with an artistic Turning Test. Colton explained, “I realized that the Painting Fool was a very good mechanism for testing out all sorts of theories, such as what it means for software to be creative. The aim of the project is for the software itself to be taken seriously as a creative artist in its own right, one day.”

In June 2015, Google’s Brain AI research team took artistic theory to the next level by infusing its software with the ability to create a remarkably human-like quality of imagination. To do this, Google’s programmers took a cue from one of the most famous masters of all time, Leonardo da Vinci. Da Vinci suggested that aspiring artists should start by looking at stains or marks on walls to create visual fantasies. Google’s neural net did just that, translating the layers of the image into spots and blotches with new stylized painterly features (see examples below).

1) Google uploaded a photograph of a standard Southwestern scene:

2) The computer then translated the layers as below:

In describing his creation, Google Brain senior scientist Douglas Eck said this past March, “I don’t think that machines themselves just making art for art’s sake is as interesting as you might think. The question to ask is, can machines help us make a new kind of art?” The goal of Eck’s platform called Magenta is to enable laypeople (without talent) to design new kinds of music and art, similar to synthetic keyboards, drums and camera filters. Dr. Eck himself is an admittedly frustrated failed musician who hopes that Magenta will revolutionize the arts in the same way as the electric guitar. “The fun is in finding new ways to break it and extend it,” Eck said excitedly.

The artistic development and growth of these computer programs is remarkable. Cohen, who passed away last year, said in a 2010 lecture regarding AARON “with no further input from me, it can generate unlimited numbers of images, it’s a much better colorist than I ever was myself, and it typically does it all while I’m tucked up in bed.” Feeling proud, he later corrected himself, “Well, of course, I wrote the program. It isn’t quite right to say that the program simply follows the rules I gave it. The program is the rules.”

In reflecting on the societal implications of creative bots, one can not help to be reminded of the famous statement by philosopher René Decartes: “I think, therefore I am.” Challenging this idea for the robotic age, Professor Arai Noriko tested the thinking capabilities of robots. Noriko led a research team in 2011 at Japan’s National Institute of Informatics to build an artificial intelligence program smart enough to pass the  rigorous entrance exam of the University of Tokyo.

“Passing the exam is not really an important research issue, but setting a concrete goal is useful. We can compare the current state-of-the-art AI technology with 18-year-old students,” explained Dr. Noriko. The original goal set out by Noriko’steam was for the Todai robot (named for the University) to be admitted to college by 2021. At a Ted conference earlier this year, Noriko shocked the audience by revealing the news that Todai beat 80% of the students taking the exam, which consisted of seven sections, including math, English, science, and even a 600-word essay. Rather than celebrating, Noriko shared with the crowd her fear,”I was alarmed.”

Todai is able to search and process an immense amount of data, but unlike humans it does not read, even with 15 billion sentences already in its neural network. Noriko reminds us that “humans excel at pattern recognition, creative projects, and problem solving. We can read and understand.” However, she is deeply concerned that modern educational systems are more focused on facts and figures than creative reasoning, especially because humans could never compete with the fact-checking of an AI. Noriko pointed to the entrance exam as an example, the Todai robot failed to grasp a multiple choice question that would have been obvious even to young children. She tested her thesis at a local middle school, and was dumfounded when one-third of students couldn’t even “answer a simple reading comprehension question.” She concluded that in order for humans to compete with robots, “We have to think about a new type of education. ”

Cohen also wrestled with the question of a thinking robot and whether his computer program could ever have the emotional impact of a human artist like Monet or Picasso. In his words, to reach that kind of level a machine would have to “develop a sense of self.” Cohen professed that “if it doesn’t, it means that machines will never be creative in the same sense that humans are creative.” Cohen later qualified his remarks about robotic creativity, adding, “it doesn’t mean that machines have no part to play with respect to creativity.”

Noriko is much more to the point, “How we humans will coexist with AI is something we have to think about carefully, based on solid evidence. At the same time, we have to think in a hurry because time is running out.” John Cryan, CEO of Deutsche Bank, echoed Noriko’s sentiment at a banking conference last week. Cryan said “In our banks we have people behaving like robots doing mechanical things, tomorrow we’re going to have robots behaving like people. We have to find new ways of employing people and maybe people need to find new ways of spending their time.”

iRobot on the defensive

SharkNinja, a well-known marketer of home consumer products, has entered the American robotic vacuum market with a product that is priced to compete against iRobot’s Roomba line of floor cleaners. Their new ION Robot navigates floors and carpets and docks and recharges automatically. It sells at a very favorable price point to iRobot’s.

SharkNinja has partnered with ECOVACS, a Chinese manufacturer of many robotic products including robotic vacuums and floor cleaners, to custom manufacture the new Shark ION Robot – thus SharkNinja isn’t starting from scratch. [ECOVACS is a big seller in China. On Singles Day (11/11/2016), online via the e-commerce giant Alibaba, ECOVACS sold $60.2 million of robotic products, up from $47.6 million in 2015. The star performer was a DEEBOT robotic vacuum which sold 135,000 units. The ECOVACS window-cleaning robot was another standout product, with more than 10,000 units sold.]

iRobot’s stock took an 18% negative hit – perhaps on the news of the product launch by SharkNinja, or perhaps because some prominent analysts downgraded their ratings of the company saying that iRobot is susceptible to a lower-cost similarly capable well-regarded branded product. The SharkNinja robotic vacuums fits those criteria.

SharkNinja is a fast-growing vendor of blenders, vacuums and other household products. They displaced Dyson in vacuums by engineering a superior product at a value price point (the Dyson robot vacuum sold for $1,000). SharkNinja, by using disruptive pricing and infomercial marketing, has garnered around 20% of the U.S. market for vacuums in just 10 years. SharkNinja’s non-robotic vacuums and blenders command significant shelf space and are very popular with customers and sellers alike. Thus they are a formidable competitor.

Also this month, SharkNinja raised an undisclosed sum from CDH Investments, a private equity fund with $20 billion of assets under management. CDH said they purchased “a significant equity interest” in SharkNinja.

iRobot’s Defensive Moves

iRobot has been making defensive moves recently. It acquired its two main distributors: Robopolis in Europe and Sales on Demand in Japan. It has used up much of its cash reserve to buy back shares of the company. And it sued what it considered to be patent violations by Hoover, Black & Decker, Bobsweep, Bissell Homecare, and Micro-Star International (MSI) (which manufacturers the Hoover and Black & Decker vacuums).

According to Zacks Equity Research, iRobot just favorably settled with MSI in an agreement where MSI will exit the global robotic cleaning industry and also provide a undisclosed compensation fee to iRobot.

“This settlement represents the first successful milestone on the enforcement effort iRobot initiated earlier this year,” said Glen Weinstein, executive vice president and chief legal officer at iRobot. “The agreement by MSI to exit the robotic cleaning industry signifies the value of iRobot’s intellectual property and the company’s efforts to protect it.”

Nevertheless, iRobot may be vulnerable to an international consumer products company with a full range of consumer products who competes with similar products at lower prices.

Page 1 of 3
1 2 3