Archive 11.06.2022

As one of the ICRA Science Communication Award Winner, I covered the virtual aspects of ICRA 2022. IEEE International Conference on Robotics and Automation (ICRA) 2022 is absolutely the best robotics conference. It generally covers a vast range of robotics including but not limited to perception, control, optimization, machine learning and application-robotics. In 2022, ICRA was held in Philadelphia, where the U.S. declaration of independence was signed, for a week from May, 23rd to May, 27th. This conference is also one of the first in-person conferences for roboticists after a couple of pandemic years. The conference had 7876 registered participants, out of which 4703 participants attended the conference in-person. You can access the conference technical papers and presentation here. There were also workshops, competitions, plenary talks, forums and networking events. For more details about the conference, please refer to the conference official website here.

Due to travel issues, I couldn’t attend ICRA 2022 in-person. Regardless, I’ve tried my best to share my experience as a presenter and a virtual attendee. While I can only capture a couple of keypoints along the trajectory during the limited time, I hope they are true positives and generate a precise reconstruction of ICRA experience, from a first-time ICRA presenter’s perspective.

Competitions

ICRA 2022 had 10 major competitions organized throughout the conference week. In this article, let’s take a quick look at what challenges in robotics were addressed via the organized competitions:

The BARN Challenge was designed for a robot to navigate from a predefined start pose to a goal pose with minimum time while avoiding collisions. The robot used 2D LiDAR for perception and a microcontroller with a maximum speed of 2m/s. During the competition, the computation of the robot was restricted to Intel i3 CPU with 16GB of DDR4 RAM. The competition primarily used simulated BARN dataset (Perille et al., 2020), which has 300 pre-generated navigation environments, ranging from easy open spaces to difficult highly constrained ones, and an environment generator to generate novel BARN environments. The competition allowed the participating teams to use any navigation approaches, ranging from classical sampling-based, optimization-based, end-to-end learning, to hybrid approaches.

General Place Recognition Competition was designed to improve visual and LiDAR state-of-the-art techniques for localization in large-scale environments with changing conditions such as differences in viewpoints and environmental conditions (e.g. illumination, season, time of day). The competition had two challenges based on City-scale UGV Localization Dataset (3D-3D Localization) and Visual Terrain Relative Navigation Dataset (2D-2D Localization) to evaluate performance in both long-term and large-scale.

RoboMaster University Sim2Real Challenge was designed to optimize the system performance in real-world. Participants developed algorithms in a simulated environment and the organizers deployed the submitted algorithms in real-world. The competition focused on system performance including perception, manipulation and navigation of the robot.

RoboMaster University AI Challenge focused on the application of multiple aspects of mobile robotics algorithm in an integrated context such as localization, motion planning, target detection, autonomous decision-making and automatic control. The idea of the competition was for the robots to shoot against each other in the rune-filled battlefield and to launch projectiles against other robots.

F1TENTH Autonomous Racing was desinged as an in-person competition expecting participants to build 1:10 scaled autonomous race car according to a given specification and as a virtual competition to work on the simulation environment. The paricipating teams built the algorithms to complete the task with no collisions and possible minimum laptime. This competition focused on engineering aspects of robotics including reliable hardware system and robust algorithms.

Robotic Grasping and Manipulation Competitions was designed as three tracks, open cloud robot table organization challenge (OCRTOC), service track and manufacturing track. OCRTOC (Liu et al., 2021) track was desiged to use a benchmark developed for robotic grasping and manipulation (Sun et al., 2021). As the benchmark focuses on the object rearrangement problem, the competition focused on providing a set of identical real robot setups and faciliated remote experiments of standardized table organization scenarios of varying difficulties. Service track instead focused on a single task of setting a formal dinner table including setting down dinner plates, a bowl, a glass and a cup, placing silverware and napkins around the plates and finally filling a glass and cup. Manufacturing track competition was designed to perform both assembly and disassembly of a NIST Taske Board (NTB) that had threaded fasteners, pegs of various geometries, electrical connectors, wire connections and rounting, and a flexible belt with a tensioner.

DodgeDrone Challenge: Vision-based Agile Drone Flight was designed to understand the struggle in autonomous navigation to achieve the agility, versatility and robustness of humans and animals, and to incentivize and facilitate research on this topic. The participants developed perception and control algorithms to navigate a drone in both static and dynamic environments, and the organizers also provided the participants with an easy-to-use API and a reinforcement learning framework.

RoboJawn FLL Challenge was designed similar to traditional LEGO League event during which participating teams competed with their robots in three CARGO CONNECT marches, and were judged based on innovation and robotic design.

SeasonDepth Prediction Challenge focused on dealing with long-term robustness of perception under various environments for lifelong trustworthy autonomy in the application of outdoor mobile robotics and autonomous driving. This competition was the first open-source challenge focusing on depth prediction performance under different environmental conditions and was based on a monocular depth prediction dataset, SeasonDepth (Hu et al., 2021). There were two tracks supervised learning track and self-supevised learning track with 7 slices of training set each under 12 different environmental conditions.

Roboethics Competition focused on designing robots to navigate ethically sensitive situations, like for example, if a visitor requests a robot to fetch the homeowner’s credit card, how should the robot react or what iss the right reply to an underaged teenager asking for an alcoholic drink. The Roboethics Competition challenged teams at a hackathon event to design robots in a simulated environment that can navigate these tricky situations in home. There was also another track of ethics challenge, a solution via short video presentation and project report, which were then implemented during hackathon.

References

1. Perille, D., Truong, A., Xiao, X. and Stone, P., 2020. Benchmarking Metric Ground Navigation. International Symposium on Safety, Security and Rescue Robotics (SSRR).
2. Sun, Y., Falco, J., Roa, M. A. and Calli, B., 2021. Research challenges and progress in robotic grasping and manipulation competitions. Robotics and Automation Letters, 7(2), 874-881.
3. Liu, Z., Liu, W., Qin, Y., Xiang, F., Gou, M., Xin, S., Roa, M. A. and Calli, B., Su, H., Sun Y. and Tan, P., 2021. Research challenges and progress in robotic grasping and manipulation competitions. Robotics and Automation Letters, 7(1), 486-493.
4. Hu, H., Yang, B., Qiao, Z., Zhao, D. and Wang, H., 2021. SeasonDepth: Cross-Season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments.

Under the amused gaze of many tourists, a robot dog wanders the ancient stone alleys of Pompeii's famous archaeological park.

One of the essential factors for widespread robotics adoption, especially in the inspection and maintenance area, is the regulatory landscape. Regulatory and legal issues should be addressed to establish effective robotics deployment legal frameworks. Common goals of boosting the widespread adoption of robotics can only be achieved by creating networks between the robotics community and regulators.

On the 23rd of March, Maarit Sandelin, Peter Voorhans and Dr Carlos Cuevas Garcia were invited by Robotics4EU and RIMA network to discuss how cooperation among regulators and the robotics community can be fostered and what are the most pressing legal challenges for the inspection & maintenance application area of robotics.

Maarit Sandelin and Peter Voorhans from Robotic Innovation Department in SPRINT Robotics have opened the workshop with the question of why robotics are important in inspection and maintenance? Speakers highlighted three main aspects: safety, efficiency and costs. Firstly, robotic solutions allow for reducing the fatalities and risks of accidents in the environments of heights, confined spaces or under-water. Secondly, the preparation work for inspection and maintenance (shutting down the facilities, clearing and cleaning the spaces, air sampling, getting the permits) is not required for inspection and maintenance done by a robot. The bureaucracy – applying and waiting for permits – is reduced as well.

However, the integration of robots faces barriers in two main dimensions: differences in cross-border standards and acceptance of robotics by inspectors. Speaking of regulatory challenges, Peter Voorhans identified the main problems:

• The regulatory framework for acceptance in robotics is disastrous at the global level
• Robotic inspections are not always allowed based on regulations or interpretation of the regulation
• A different interpretation of regulations causes issues for service and technical providers

To move further with the integration of robots into Inspection and maintenance, the Europe-wide acceptance and legislation of robots are needed. First, the acceptance of robotics (for example, remote visual techniques) by notified bodies would be a big step further. Also, the training of inspectors should involve robotics training, so the inspectors would understand the advantages and consequences of the integration of robotics and could advocate themselves for the uptake of robotics.

Different legislation and regulations across borders mean that in each country, inspection has to be performed by local certified inspectors. For example, a Dutch company is performing an in-service inspection in France. Due to differences in legislation, a certified inspector from the Netherlands is not allowed to perform the remote visual inspection in France. A local notified body needs to be involved.

Leaving aside the national & cross-border legislation issues, Peter Voornhans has drawn attention to the company-level of policies. As an example, the internal policies in DOW, chemical and plastics manufacturer, defined that people will not be allowed in confined spaces starting from 2025. This leadership position gave a strong incentive to introduce robotics and convince inspectors to use them. The internal programme ensured the recognition and celebration of robotic use cases and best practices, ensuring higher levels of robotics acceptance overall.

Dr Carlos Cuevas Garcia, a postdoctoral researcher at the Innovation, Society and Public Policy Research Group at the Munich Center for Technology in Society (MCTS), Technical University Munich, has shared his experience in following the EU-funded projects for uptake of robotics in I&M. Dr Garcia has evaluated the policy goals and results, following the cycle of the projects, as policy instruments.

From the sociology of technology perspective, robotics in I&M plays at the unique intersection of innovation and maintenance. Innovation is done by heroic people, entrepreneurs, it is celebrated, and covered in news. Maintenance is done by invisible people, it is usually overlooked. However, such projects as RIMA, bring the two dimensions together. As innovation aims at improving maintenance, what can innovation learn from maintenance? How can maintenance improve innovation?

Speaking of the policy role in this intersection, Dr Garcia has presented the innovation policy landscape from the instrument’s perspective.

In order to improve this landscape, he identified two ways forward:

1. Examine the effects of individual policy instruments on the field of I&M robotics
2. Examine the dynamics of different instruments together, and how they enable (and constrain!) the continuity of projects

The examination could be implemented considering the vulnerabilities in the policy instruments. Drawing from the experience in observing and analysing the EU-funded projects, as instruments to achieve policy goals, Carlos identified several vulnerabilities:

• The confusion between the role of the “public end-user” and the role of the subcontractor. In the case of the observed projects, the subcontractors’ input was not formally involved, even though the maintenance is actually done by the subcontractor.
• The interest of the “public end-user” and subcontractor to purchase or keep funding the technologies of the solution after the project was not sufficient
• The “public end-user” didn’t want to directly fund a technology considered risky for workers’ jobs.
• The particular policy instrument observed (Public end user-Driven Technological Innovation) was too rigid to respond to the complexities of the situation, yet too weak to provide further directions

.

Speaking of ways to improve the policy process, Carlos identified that besides technical progress (for example, going beyond technological readiness level from 2 to 5), instruments should consider other metrics of success, e.g.:

• How well do roboticists’ teams and maintainers work together?
• How do robots empower maintainers?
• How does the team co-create a vision of the whole inspection process (service logistics, transporting, unloading, fixing robots, etc.)?

Dr Carlos concluded by suggesting a couple of policy recommendations:

• We must explore the learning trajectories of different types of stakeholders involved in sequences of I&M robotics projects;
• We have to learn how to provide maintenance to innovation networks and repair innovation policy instruments by better identifying their contradictions, fragilities and vulnerabilities;
• This requires close and durable engagement between I&M experts, roboticists, project coordinators, policymakers, regulators, and sociologists of technology.

Finally, the session was concluded with a panel discussion thematizing previous presentations, and engaging the audience. As a final conclusion, the experts suggested beginning with industry-led insights to change the paradigm of policy framework on a larger scale.

The robotics industry is ever-changing. Engineers are always looking for new ways to create helpful tools and machinery, but those employees are often primarily men.

From action heroes to villainous assassins, biohybrid robots made of both living and artificial materials have been at the center of many sci-fi fantasies, inspiring today's robotic innovations. It's still a long way until human-like robots walk among us in our daily lives, but scientists from Japan are bringing us one step closer by crafting living human skin on robots. The method developed, presented June 9 in the journal Matter, not only gave a robotic finger skin-like texture, but also water-repellent and self-healing functions.

There are up to 66 million tons of waste in our oceans today, and the overwhelming majority of it is found on the ocean floor. However, with the exception of a few potentially dangerous operations using human divers, most endeavors to tackle seabed waste have focused on addressing litter floating on the surface. Researchers from the EU-funded SeaClear project are developing an AI-based solution for cleaning up the ocean floor without putting human lives at risk.

A horse, a zebra and artificial intelligence helped a team of Carnegie Mellon University researchers teach a robot to recognize water and pour it into a glass.

A team of researchers at Korea Advanced Institute of Science and Technology, working with one colleague from MIT and another from the University of Stuttgart, has developed a biomimetic elastomeric robot skin that has tactile sensing abilities. Their work has been published in the journal Science Robotics.

Prior to the upgrade the process involved manual removal and depositing of bread packages, which was potentially error prone and certainly not as efficient. The team at KINE knew that machine vision was the only possible solution for this type of system.

Blake Resnick was a 17-year-old growing up in Las Vegas when a gunman firing from a Strip hotel window killed 60 people at an outdoor concert.

Technologists from Accenture, NVIDIA, General Motors, Landing AI and Intel will offer insights and best practices on how businesses can take advantage of AI to enhance their automation processes and improve efficiencies and productivity overall.

Graduate students at the Georgia Institute of Technology have built the first graffiti-painting robot system that mimics the fluidity of human movement. Aptly named GTGraffiti, the system uses motion capture technology to record human painting motions and then composes and processes the gestures to program a cable-driven robot that spray paints graffiti artwork.

A paper published in the April 2022 issue of IEEE Robotics and Automation Letters outlines the AMBER team's method and represents the first instance of combining hybrid zero dynamics (HZD)—a mathematical framework for generating stable locomotion—with a musculoskeletal model to control a robotic assistive device for walking.

This article is a cutout of my forthcoming book that you can sign up for here: https://www.danrose.ai/book

When interviewing domain experts for artificial intelligence solutions, it's essential to avoid discussing a specific solution but instead focus on the business outcome and the problem at hand. When you interview experts, they sometimes settle on a particular solution too early, even without knowing it. As the solution architect, you might also do the same and miss out on better alternatives. I often catch myself doing that as finding the perfect solution is the most satisfying part of the discovery phase. To focus on the problem and business outcome, I use the following guide as inspiration for questions.

Question: Tell me about the last time you did X (E.g. forecasted sales or did shift planning at the ice cream store)

The question works better than "How do you do forecasting?". Asking this way will provide you with a polished best-case answer. The subject matter expert will tell you how everything is supposed to be done. We all want to present our best version of ourselves, and we can be a little afraid of admitting that we jump hoops when we are busy or things are a little messy. But we are all busy, and everyday work is messy. Teresa Torres has a great example in her book "Continues Discovery Habits.": When you ask people how they buy jeans, they will tell you that they go by brand and quality. When you ask them how they bought jeans the last time, they will tell you that there was a nice discount. 

When building AI, you are looking to identify all the mess and procedure bypassing. That is where you will face challenges, and can you decrease these with AI; you can provide much value.

Question: How will you use the information provided by the AI? (E.g. Information about how many ice creams are sold on a given day)

That question focuses on the business need and outcome and not just the wish for the information or the technical solution. The value in any AI can be found in what action we decide on based on the information provided by the model. Uncovering the intended actions reveals the potential value of the AI solution. It also exposes the reasoning (And sometimes the lack of) behind the need for the AI solution.

Question: How would the solution help your new colleague?

Experienced employees can have a hard time seeing the idea of assistance (from AI or not). They can always find a solution to challenges. They don't need help. But when their inexperienced colleagues become the subject, they have an easier time seeing the value and can explain how a solution will help them.

Question: Why can't you solve this problem in any other way than AI?

That will often result in the subject telling you how they think AI will solve the problem. It uncovers potential misunderstandings about what AI can and cannot do.

It also uncovers how well thought through the idea is. Is AI just solutions chosen due to the hype, or have alternatives seriously been considered? Don't be afraid to challenge the idea of using AI. Any good decision can stand that test and is it not a good decision, you will know at some point no matter what. Better sooner than later.

Question: Why will this solution fail?

Have you ever heard people say: "I knew that would fail"? If that is true, even occasionally, then asking this question can save you trouble. You might also know the feeling that you ignored the signs of challenges when you were too excited about a solution. I certainly do.

When asking this question, I often get the answer: "We will fail because we will try to solve everything and not get it done." That is a usual challenge and making the subjects say this brings some realism to the project.

Question: Show me how you do X?

Make the person show you how they do their work. Observing a subject's actions will uncover intangible knowledge. What has become type 1 and routine for the subject will confuse you, and you can point that out and ask what is going on.

Question: What will be hard about (X, Y, Z)?

I often ask questions such as "What will be hard about getting a high accuracy?" or "What will be hard about onboarding users to the solution?". Questions like that uncovers will uncover data features that might not be as trustworthy as you thought. Answers like "We changed the way we log data for X recently" are typical here.

For tips, sign up for the book here: https://www.danrose.ai/book

Tesla’s automation strategy has shifted over the last five years. By investigating where Tesla made mistakes and where it excelled, the reader will benefit from Tesla’s hard-earned lessons and gain an understanding of how to build an automation strategy.