By Annie Xie
In many animals, tool-use skills emerge from a combination of observational learning and experimentation. For example, by watching one another, chimpanzees can learn how to use twigs to “fish” for insects. Similarly, capuchin monkeys demonstrate the ability to wield sticks as sweeping tools to pull food closer to themselves. While one might wonder whether these are just illustrations of “monkey see, monkey do,” we believe these tool-use abilities indicate a greater level of intelligence.
Left: A chimpanzee fishing for termites. Right: A gorilla using a stick to gather herbs. (source)
The question our new work explores is: can we enable robots to use tools in the same way — through observation and experimentation?
A requisite for performing complex multi-object manipulation tasks, such as those involved in tool use, is an understanding of physical cause-and-effect relationships. Therefore, the ability to predict how one object might interact with another is crucial. Our prior work has investigated how visual predictive models of cause-and-effect can be learned from unsupervised robot interaction with the world. After learning such a model, the robot can plan to accomplish a diverse set of simple tasks, including cloth folding and object arrangement. However, if we consider the more complex interactions that occur in tool-use tasks, such as how a broom can sweep dirt into a dustpan, undirected experimentation isn’t enough.
Hence, taking inspiration from how animals learn, we designed an algorithm that allows robots to learn tool-use skills through a similar paradigm of imitation and interaction. In particular, we show that, with a mix of demonstration data and unsupervised experience, a robot can use novel objects as tools and even improvise tools in the absence of traditional ones. Further, depending on the demands of the task, our method demonstrates the ability to decide whether to use the provided tools. In this post, we will describe how this works.
Dylan Glas is a Senior Robotics Software Architect at Futurewei Technologies, a research division of Huawei in Silicon Valley. He was previously a senior researcher in social robotics at Hiroshi Ishiguro Laboratories at ATR and a Guest Associate Professor at the Intelligent Robotics Laboratory at Osaka University. He was the chief architect for the ERICA android in the ERATO Ishiguro Symbiotic Human-Robot Interaction Project. His research interests include social human-machine interaction, ubiquitous sensing, network robot systems, teleoperation for social robots, and machine learning.