Page 411 of 427
1 409 410 411 412 413 427

RoboBusiness 2017: What’s cooking in robotics?

Mike Toscano, the former president of the Association for Unmanned Vehicle Systems International, emphatically declared at the September RobotLab forum that “anyone who claims to know the future of the [robotics] industry is lying, I mean no one could’ve predicted the computing mobile revolution.” These words acted as a guiding principle when walking around RoboBusiness in Silicon Valley last week.

The many keynotes, pitches and exhibits in the Santa Clara Convention Center had the buzz of an industry racing towards mass adoption, similar to the early days of personal computing. The inflection point in the invention that changed the world, the PC, was 1995. During that year, Sun Microsystems released Java to developers with promise of “write once, publish anywhere,” followed weeks later by Microsoft’s consumer software package, Windows ’95. Greater accessibility led to full ubiquity and applications unthinkable by the original engineers. In many ways, the robot market is standing a few years before its own watershed moment.

In my last post, I highlighted mechanical musicians and painters, this week it is time to see what is cooking, literally, in robotics. Next year, startup Moley plans to introduce the “first fully-automated and integrated intelligent cooking robot,” priced under $100,000. It already has a slick video that is reminiscent of Lily’s Robotics’ rise to the headlines; needless to say Moley has created quite a stir in the culinary community.

Austin Gresham, executive chef at The Kitchen by Wolfgang Puck is very skeptical, “Professional chefs have to improvise constantly as they prepare dishes. If a recipe says to bake a potato for 25 minutes and the potatoes are more or less dense than the previous batch, then cooking times will vary. I would challenge any machine to make as good a mashed potato (from scratch).” Gresham’s challenge is really the crux of the matter, creativity is driven by human’s desire for food, without taste could a robot chef have the intuition to improvise?

Acting as a judge of the RoboBusiness Pitch Fire Competition, I met entrepreneurs undiscouraged by the market challenges ahead. In addition, throughout my Valley visit, I encountered five startups building commercial and consumer culinary applications. Any time this happens within such a short timespan, I stop and take notice. Automated restaurants seem to be a growing trend across the nation with a handful of upstarts on both coasts. Eatsa is a chain of quinoa-salad restaurants sans cashiers and servers. Customers order via mobile devices or on-site kiosks, picking up their ready dishes through an automated floor-to-ceiling lockbox fixture. However, behind the wall Eatsa has hourly workers manually preparing the salad bowls. Cafe X in San Francisco offers a completely automated experience with a robot-arm barista preparing, brewing and serving espressos, cappuccinos, and Americanos. After raising $5 million from venture investors, Cafe X plans to expand with robot kiosks throughout the city. Probably the most end-to-end automated restaurant concept I visited can be found tucked away on Berkeley University’s Global Campus called BBox by Nourish. BBox is currently running a trial on campus and planning to open its first store next year to conquer the multi-billion dollar breakfast market with egg sandwiches and gourmet coffee (see video below).

According to Nourish’s CEO Greg Becker, BBox will “reengineer the food ecosystem, from farm to mouth.” Henry Hu, Cafe X’s founder, also aims to revolutionize “the supply chain, recipes, maintenance, and customer support.” To date, the most successful robotic concept is Zume Pizza. Founder Julia Collins made headlines last year with her groundbreaking spin on the traditional pizzeria. Today she is taking on Dominos dollar for dollar in the San Francisco area, delivering pies in under 22 minutes. Collins, a former Chief Financial Officer of a Mexican restaurant chain, challenges the food industry, “Why don’t we just re-write the rules— forget about everything we learned about running a restaurant?” Already, Zume is serving hundreds of satisfied customers daily, proving at least with pizza it is possible to innovate.

“We realized we could automate more of the unsafe repetitive tasks of operating a kitchen using flexible, dynamic robots,” explains Collins, who currently employees over 50 human workers that do everything from software engineering to supervising the robots to delivering the pizza. “The humans that work at Zume are making dough from scratch, working with farmers to source products, recipe development—more collaborative, creative human tasks. [We have] lower rent costs because we don’t have a storefront; delivery only and lower labor costs. We reinvest those savings into locally sourced, responsibly farmed food.” Collins also boasts that her human workforce has access to free vision, dental, and health insurance due to the cost savings.

Even Shake Shack could have competition very soon as Google Ventures-backed Momentum Machines is launching an epicurean robot bistro in San Francisco’s chic SoMa district later next year. The machine that has been clocked at 400 burgers an hour, guarantees “to slice toppings, grill a patty, assemble, and bag the burger without any help from humans,” at prices that “everyone can afford.” Momentum’s proposition prompted former McDonald’s CEO Ed Rensi to controversially state that “it’s cheaper to buy a $35,000 robotic arm than it is to hire an employee who’s inefficient making $15 an hour bagging french fries.” Comments like Rensi’s do not further the industry, in fact it probably led to the controversy last month with the launch of Bodega, an automated convenience store that even enraged Lin-Manuel Miranda below.

The bad press was multiplied further by Elizabeth Segran’s article in Fast Company, which read, “the major downside to this concept — should it take off — is that it would put a lot of mom-and-pop stores out of business.” Founder Paul McDonald responded on Medium, “Rather than take away jobs, we hope Bodega will help create them. We see a future where anyone can own and operate a Bodega — delivering relevant items and a great retail experience to places no corner store would ever open.” While Bodega is not exactly a robotic concept, it is similar to the automated marketplace of AmazonGo with 10 computer vision sensors tracking the consumer and inventory management via a mobile checkout app. “We’re shrinking the store and putting it in a box,” said McDonald. The founder has publicly declared war on 7-Eleven’s 5,000 stores, in addition to the 4 million vending machines across the US. Realizing the pressures to innovate, last year 7-Eleven made history with the first drone Slurpee delivery. “Drone delivery is the ultimate convenience for our customers and these efforts create enormous opportunities to redefine convenience,” said Jesus H. Delgado-Jenkins, 7-Eleven EVP and Chief Merchandising Officer. “This delivery marks the first time a retailer has worked with a drone delivery company to transport immediate consumables from store to home. In the future, we plan to make the entire assortment in our stores available for delivery to customers in minutes. Our customers have demanding schedules, are on-the-go 24/7 and turn to us to help navigate the challenges of their daily lives. We look forward to working with Flirtey to deliver to our customers exactly what they need, whenever and wherever they need it.”

As mom & pop stores compete for market share, one wonders with more Kitchen OS concepts if home cooked meals will join the list of outdated cultural trends. Serenti Kitchen in San Francisco plans to bring the Keurig pod revolution to food with its proprietary machine that includes prepared culinary recipe pods that are dropped into a bowl and whipped to perfection by a robotic arm (see above). Serenti Founder Tim Chen was featured last year at the Smart Kitchen Summit, which reconvenes later this month in Seattle. Chen said, “We’re building something that’s quite hard, mechanically, so it’s more from a vision where we wanted to initially develop a machine that could cook, and make cooking easier and automate cooking for the home.” Initially Chen plans to target business catering, “In the near term, we need to focus on placing these machines where there’s the highest amount of density, which is right in the offices,” but long-term Serenti plans to join the appliance counter. Chen explained his inspiration, “Our Mom is a great cook, so they’ve watched her execute the meals. Then realized a lot of it is repetitive, and what recipes are, is essentially just a machine language.” Chen’s observations are shared by many in the IoT and culinary space, as this year’s finalists in the Smart Kitchen Summit include more robotic of inventions, such as Crepe Robot that automatically dispense, cook and flavors France’s favorite snack and GammaChef, a robotic appliance that promises like Serenti to whip up anything in a bowl. Clearly, these inventions will eventually lead to a redesign of the physical home kitchen space that is already crowded with appliances. Some innovators are even using robotic arms tucked away in cabinets and specialized drawers, ovens and refrigeration units that communicate seamlessly to serve up dinner.

The automated kitchen illuminated by Moley and others might be coming sooner than anyone expects; then again it could be a rotten egg. In almost every Sci-Fi movie and television show the kitchen is reduced to a replicator that synthesizes food to the wishes of the user. Three years ago, it was rumored that food-powerhouse Nestle was working on a machine that could produce nutritional supplements on demand, code name Iron Man. While Iron Man has yet to be released to the public, it does illustrate the convergence of 3D printing, robotics and kitchen appliances. While the Consumer Electronics Show is still months away, my appetite has just been whetted for more automated culinary treats, stay tuned!

Global robot growth causing shortages in critical components

Two reputable research resources are reporting that the robotics industry is growing more rapidly than expected. BCG (Boston Consulting Group) is conservatively projecting that the market will reach $87 billion by 2025; Tractica, incorporating the robotic and AI elements of the emerging self-driving industry, is forecasting the market will reach $237 billion by 2022.

Both research firms acknowledge that yesterday’s robots — which were blind, big, dangerous and difficult to program and maintain — are being replaced and supplemented with newer, more capable ones. Today’s new – and future robots will – have voice and language recognition, access to super-fast communications, data and libraries of algorithms, learning capability, mobility, portability and dexterity. These new precision robots can sort and fill prescriptions, pick and pack warehouse orders, sort, inspect, process and handle fruits and vegetables, plus a myriad of other industrial and non-industrial tasks, most faster than humans, yet all the while working safely along side them.

Boston Consulting Group (BCG)

Gaining Robotic Advantage, June 2017, 13 pages, free

BCG suggests that business executives be aware of ways robots are changing the global business landscape and think and act now. They see robotics-fueled changes coming in retail, logistics, transportation healthcare, food processing, mining and agriculture.

BCG cites the following drivers:

  • Private investment in the robotic space has continued to amaze with exponential year-over-year funding curves and sensational billion dollar acquisitions.
  • Prices continue to fall on robots, sensors, CPUs and communications while capabilities continue to increase.
  • Robot programming is being transformed by easier interfaces, GUIs and ROS.
  • The prospect of a self-driving vehicles industry disrupting transportation is propelling a talent grab and strategic acquisitions by competing international players with deep pockets.
  • 40% of robotic startups have been in the consumer sector and will soon augment humans in high-touch fields such as health and elder care.

 BCG also cites the following example as an example of paying close attention to gain advantage:

“Amazon gained a first-mover advantage in 2012 when it bought Kiva Systems, which makes robots for warehouses. Once a Kiva customer, Amazon acquired the robot maker to improve the productivity and margins of its network of warehouses and fulfillment centers. The move helped Amazon maintain its low costs and expand its rapid delivery capabilities. It took five years for a Kiva alternative to hit the market. By then, Amazon had a jump on its rivals and had developed an experienced robotics team, giving the company a sustainable edge.”

Tractica

Robotics Market Forecast – June 2017, 26 pages, $4,200
Drones for Commercial Applications – June 2017, 196 pages, $4,200
AI for Automotive Applications – May 2017, 63 pages, $4,200
Consumer Robotics – May 2017, 130 pages, $4,200

The key story is that industrial robotics, the traditional pillar of the robotics market, dominated by Japanese and European robotics manufacturers, has given way to non-industrial robot categories like personal assistant robots, UAVs, and autonomous vehicles, with the epicenter shifting toward Silicon Valley, which is now becoming a hotbed for artificial intelligence (AI), a set of technologies that are, in turn, driving a lot of the most significant advancements in robotics. Consequently, Tractica forecasts that the global robotics market will grow rapidly between 2016 and 2022, with revenue from unit sales of industrial and non-industrial robots rising from $31 billion in 2016 to $237.3 billion by 2022.  The market intelligence firm anticipates that most of this growth will be driven by non-industrial robots.

Tractica is headquartered in Boulder and analyzes global market trends and applications for robotics and related automation technologies within consumer, enterprise, and industrial marketplaces and related industries.

General Research Reports

  • Global autonomous mobile robots marketJune 2017, 95 pages, TechNavio, $2,500
    TechNavio forecasts that the global autonomous mobile robots market will grow at a CAGR of more than 14% through 2021.
  • Global underwater exploration robotsJune 2017, 70 pages, TechNavio, $3,500
    TechNavio forecasts that the global underwater exploration robots market will grow at a CAGR of 13.92 % during the period 2017-2021.
  • Household vacuum cleaners market, March 2017, 134 pages, Global Market Insights, $4,500
    Global Market Insights forecasts that household vacuum cleaners market size will surpass $17.5 billion by 2024 and global shipments are estimated to exceed 130 million units by 2024, albeit at a low 3.0% CAGR. Robotic vacuums show a slightly higher growth CAGR.
  • Global unmanned surface vehicle market, June 2017, Value Market Research, $3,950
    Value Market Research analyzed drivers (security and mapping) versus restraints such as AUVs and ROVs and made their forecasts for the period 2017-2023.
  • Innovations in Robotics, Sensor Platforms, Block Chain, and Artificial Intelligence for Homeland Security, May 2017, Frost & Sullivan, $6,950
    This Frost & Sullivan report covers recent developments such as co-bots for surveillance applications, airborne sensor platforms for border security, blockchain tech, AI as first responder, and tech for detecting nuclear threats.
  • Top technologies in advanced manufacturing and automation, April 2017, Frost & Sullivan, $4,950
    This Frost & Sullivan report focuses on exoskeletons, metal and nano 3D printing, co-bots and agile robots – all of which are in the top 10 technologies covered.
  • Mobile robotics market, December 2016, 110 pages, Zion Market Research, $4,199
    Global mobile robotics market will reach $18.8 billion by end of 2021, growing at a CAGR of slightly above 13.0% between 2017 and 2021.
  • Unmanned surface vehicle (USV) market, May 2017, MarketsandMarkets, $5,650
    MarketsandMarkets forecasts the unmanned surface vehicle (USV) market to grow from $470.1 Million in 2017 to $938.5 Million by 2022, at a CAGR of 14.83%.
  • Military/Civil UAS markets, May 2017, 608 pages, Teal Group
    The Teal Group’s 2016 world military market study estimates that UAV production will soar from current worldwide UAV production of $2.8 billion annually in 2016 to $9.4 billion in 2025, a 15.4% CAGR and that civil UAS production will soar from $2.6 billion worldwide in 2016 to $10.9 billion in 2025, a 15.4% CAGR.

Agricultural Research Reports

  • Global agricultural robots market, May 2017, 70 pages, TechNavio, $2,500
    Forecasts the global agricultural robots market will grow steadily at a CAGR of close to 18% through 2021.
  • Agriculture robots market, June 2017, TMR Research, $3,716
    Robots are poised to replace agricultural hands. They can pluck fruits, sow and reap crops, and milk cows. They carry out the tasks much faster and with a great degree of accuracy. This coupled with mandates on higher minimum pay being levied in most countries, have spelt good news for the global market for agriculture robots.
  • Agricultural Robots, December 2016, 225 pages, Tractica, $4,200
    Forecasts that shipments of agricultural robots will increase from 32,000 units in 2016 to 594,000 units annually in 2024 and that the market is expected to reach $74.1 billion in annual revenue by 2024. Report, done in conjunction with The Robot Report, profiles over 165 companies involved in developing robotics for the industry.

Bottom Line

The disparity between the projections of these research reports is wide but the CAGRs are mostly all double digit. It is easy to conclude as BCG did – that the robotics industry is growing faster than expected

Robohub Podcast #244: Robot Pediatric Coach, with Ayanna Howard



In this episode, Audrow Nash interviews Ayanna Howard, Professor at the Georgia Institute of Technology, about her work to help children with the movement disorder cerebral palsy. Howard discusses how robots and tablet can be used to “gamify” pediatric therapy. The idea is that if therapy is fun and engaging children are more likely to do it, and thus, they are more likely to see the long-term benefits of the therapy. Howard discusses how therapy is “gamified,” how a small humanoid robot is used to coach children, and how they work with pediatricians.

 

Ayanna Howard

Ayanna Howard, Ph.D. is Professor and Linda J. and Mark C. Smith Endowed Chair in Bioengineering in the School of Electrical and Computer Engineering at the Georgia Institute of Technology. She also holds the position of Associate Chair for Faculty Development in ECE. She received her B.S. in Engineering from Brown University, her M.S.E.E. from the University of Southern California, and her Ph.D. in Electrical Engineering from the University of Southern California.

Her area of research is centered around the concept of humanized intelligence, the process of embedding human cognitive capability into the control path of autonomous systems. This work, which addresses issues of autonomous control as well as aspects of interaction with humans and the surrounding environment, has resulted in over 200 peer-reviewed publications in a number of projects – from scientific rover navigation in glacier environments to assistive robots for the home. To date, her unique accomplishments have been highlighted through a number of awards and articles, including highlights in USA Today, Upscale, and TIME Magazine, as well as being named a MIT Technology Review top young innovator and recognized as one of the 23 most powerful women engineers in the world by Business Insider.

In 2013, she also founded Zyrobotics, which is currently licensing technology derived from her research and has released their first suite of therapy and educational products for children with differing needs. From 1993-2005, Dr. Howard was at NASA’s Jet Propulsion Laboratory, California Institute of Technology. She has also served a term as the Associate Director of Research for the Georgia Tech Institute for Robotics and Intelligent Machines and a term as Chair of the multidisciplinary Robotics Ph.D. program at Georgia Tech.

 

Links

Energy, enthusiasm and spirit of cooperation: Award winners of ERL Emergency Robots 2017 announced


The European Robotics League (ERL) announced the winners of ERL Emergency Robots 2017 major tournament, during the awards ceremony held on Saturday, 23rd September at Giardini Pro Patria, in Piombino, Italy.

The ERL Emergency Robots 2017 competition consisted of four scenarios, inspired by the nuclear accident of Fukushima (Japan, 2011) and designed specifically for multi-domain human-robot teams. The first scenario is The Grand Challenge made up of three domains – sea, air, land, and the other three scenarios are made of only two domains.

The Awards, given for each scenario to the best performing teams, were introduced by Alan Winfield from Bristol Robotics Laboratory and ERL Emergency Coordinator. “The energy, enthusiasm and spirit of cooperation among the teams competing in ERL Emergency was amazing. We witnessed not only great performances from the teams and their robots, but also the drama and excitement of last minute field repairs and workarounds to the robots”, said Alan Winfield.

The Grand Challenge (Scenario 1: land, sea, and air)

After a nuclear power plant has been struck by a potent earthquake and a tsunami, it’s time for the emergency response team to act. Due to high radiation levels, the cooperation of land, sea and air robots is essential. The robots have to find as soon as possible three missing workers and deploy an emergency kit next to them. Secondly, the robots check for any structural damage of the building and to the pipes connecting the reactor to the sea for cooling purposes. In case of damaged or leaking pipes, the corresponding valves are to be closed both in the machine room and underwater to avoid radioactive contamination. Closing the wrong valves may cause a reduction in the amount of seawater available for cooling down the reactor.

1st Prize: Telerob, Germany (land) + Universitat de Girona, Spain (sea) + INESTEC/ISEP Aerial Robotics, Portugal (air)

“Our underwater robot Sparus II AUV was used to create maps of the underwater environment and to autonomously detect some targets. The algorithms developed by master and PhD students and the robustness of the platform allowed us to obtain good results even in the challenging conditions of the competition. The multi-domain competition required the coordination with the other robots (land and air), which offered us a unique opportunity for testing our communication capabilities.”, said Marc Carreras from the University of Girona.

“Autonomy was showing its advantages as well as a good situational awareness. The advanced mission documentation, which was requested in the competition, enables the first responders to get a fast and reliable situational understanding to finally reach the necessary situational ownership. The ERL Emergency competition should further focus on improving the robot-human-teaming”, said Andreas Ciossek from Telerob.

“The INESC TEC participation in ERL Emergency 2017 allowed us to validate our robotics technology in a real-world scenario with a relevant social and economical impact. Furthermore, it helped raising public awareness of the role that advanced robots can play in disaster scenarios aiding human teams in critical operations, and it confirmed once again the leading role of European robotics research”, said Eduardo Silva from ISEP/INESC TEC.

Read more about their experience here.

2nd Prize: IIS Piombino CVP, Italy (air) + Robdos, Spain, + IMM, Poland (land and sea)

3rd Prize: Raptors, Poland (air & land) + Oubot, Hungary (sea)

Survey the building and search for missing workers (Scenario 2: land and sea)

The ground and aerial robots perform a reconnaissance mission of the area and create a map of the surrounding area in order to increase the awareness of the emergency response team. Additionally, the robots find two missing workers outdoors and deploy first-aid kits near them.

1st Prize: IMM, Poland + IIS Piombino CVP, Italy

2nd Prize: Raptors Team, Poland

3rd Prize: Telerob, Germany + INESTEC/ISEP Aerial Robotics, Portugal

Pipe inspection and search for missing workers (Scenario 3: sea and air)

After the earthquake and tsunami, the pipes connecting the reactor to the sea might be leaking radioactive substances, therefore the emergency team has to find the damaged ones on land or underwater. Robots must find two missing workers: one outside the building, to whom an emergency kit should be deployed, and another one dragged by the tsunami to the sea, expected to be a casualty.

1st Prize: Universitat de Girona, Spain + INESTEC/ISEP Aerial Robotics, Portugal

2nd Prize: Tuscany Robotics Team, Italy

3rd Prize: AUV Tomkyle, Germany + HSR Search and Rescue Team, Switzerland

“Aerial robots have shown great improvements with respect to euRathlon 2015, most of them being fully operational from the first day of the competition. They have been able to quickly provide information about inaccessible areas, structural damages or other possible threats. As it happened with robots from the other two domains, aerial teams also struggled with communication issues to properly command and control their platforms. This further confirms the need for autonomous capabilities onboard the aerial robots, to become even more powerful tools for emergency response teams”, said Francisco Javier Pérez Grau from the Advanced Center for Aerospace Technologies.

Stem the leak (Scenario 4: land and sea)

The land robots have to inspect the pipes in the building’s machine room and marine robots the underwater pipes in order to close the correct valves and prevent leakage. Land and marine robots must cooperate to identify the valves and synchronize the process of closing them, by communicating directly or via their operators.

1st Prize: Telerob, Germany + Universitat de Girona, Spain

2nd Prize: Raptors, Poland + Oubot, Hungary

3rd Prize: bebot, Switzerland + AUV Tomkyle Team, Germany

Winning teams were given a diploma, prize money and in-kind, sponsored by the Platinum Sponsor IEEE Oceanic Engineering Society, silver sponsors SBG systems and Texas Instruments.

In addition, Marta Palau Franco, Bristol Robotics Laboratory, ERL Emergency project manager introduced the referees’ special awards. Find out who the winners are here!

“This great event has been possible thanks to the work and effort of an amazing local organising team. Special thanks to Fausto Ferreira, the ERL Emergency 2017 Deputy Director, for his continuous support. I want to thank all the sponsors, especially our platinum sponsor IEEE OES, the project partners, referees, local associations and schools for their support. Huge thanks go to the participating teams, which were the heart of this great event. Their competitiveness pushed the robots to accomplish great results, nevertheless the competition has always been accompanied by fair play. I believe this is the perfect formula for team members to improve their professional and human skills”, said Gabriele Ferri, ERL Emergency 2017 Director.

 

 Watch the ERL Emergency 2017 Awards Ceremony video

More info

The European Robotics League is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement n° 688441.

The ERL Emergency Robots 2017 competition was organised locally by the NATO STO-Centre for Maritime Research and Experimentation (CMRE) of La Spezia, Italy.

The European Robotics League is part of the SPARC public-private partnership set up by the European Commission and euRobotics to extend Europe’s leadership in civilian robotics. SPARC’s €700 million of funding from the Commission in 2014̶20 is being combined with €1.4 billion of funding from European industry. www.eu-robotics.net/sparc

euRobotics is a European Commission-funded non-profit organisation which promotes robotics research and innovation for the benefit of Europe’s economy and society. It is based in Brussels and has more than 250-member organisations.

www.eu-robotics.net

The referees’ special awards ERL Emergency Robots 2017

The European Robotics League (ERL) announced the winners of ERL Emergency Robots 2017 major tournament, during the awards ceremony held on Saturday, 23rd September at Giardini Pro Patria, in Piombino, Italy.

In addition to the Competition Awards, Marta Palau Franco from Bristol Robotics Laboratory and ERL Emergency project manager introduced the referees’ special awards.

“Behind a multi-domain competition there is always a large technical committee, I feel privileged to have worked with such an amazing team of volunteer referees, technical assistants and safety pilots and divers. We were delighted to give these awards to recognise teams’ effort, fair play and hard work. The experience of participating in this robotics competition will prove beneficial for team members to develop further their professional career”, said Marta Palau Franco.

Mapping Award, handed by Vladimir Djapic from AFAK, for good quality georeferenced undersea mapping.
Winner: AUV Team Tomkyle, Germany (sea)

Navigation Award, handed by Pino Casalino from the University of Genova, for the effort to change and adapt algorithms to navigate without a Doppler Velocity Log (DVL) sensor, important for the AUV (autonomous Undersea Vehicle) navigation.
Winner: Oubot Team, Hungary (sea)

Fair Play Award, handed by Marta Palau Franco from the University of the West of England, Bristol, for lending to the Tuscany Robotics Team a wheel platform for their new robot and for lending the batteries of their aerial robot to ISEP/INESC TEC aerial team.
Winners: ENSTA Team and ENSTA Bretagne, France (land, air, sea)

Creativity Award, handed by Bernd Bruggermann from Fraunhofer FKIE, for building a land robot from scratch in less than two days when their ground platform broke.
Winner: Tuscany Robotics, Italy (land, air, sea)

Multi-domain Cooperation Award, handed by Fausto Ferreira from CMRE, for cooperation between domains. The teams used a graphical interface in which each robot from sea and air domain reported its findings in the competition arena in real-time.
Winner: Universitat de Girona, Spain (sea) + ISEP/INESC TEC, Portugal (air)

Perseverance Award, handed by Francisco Javier Perez Grau from FADA-CATEC, for hard work on the development and integration of the aerial robot. The team competed in the Grand Challenge the day after their aerial platform suffered a severe crash, working overnight to fix it.
Winner: HSR Search and Rescue Team, Switzerland (air)

Piloting Award, handed by Stjepan Bogdan from University of Zagreb-FER , for outstanding UAV piloting skills. The team was able to recover the aerial robot after an unintentional landing without incurring manual intervention.
Winner: Raptors, Poland (land & air)

Autonomy Award, handed by Frank Schneider from Fraunhofer FKIE, for the best autonomy of land robots. Outstanding autonomous navigation and automatic object detection.
Winner: IMM, Poland (land)

SAUC-E Student Award – handed by Bill Kirkwood (IEEE OES), Kelly Cooper (ONR) and Hitesh Patel (AUVSI) to the best student marine team.
Winner: AUV Team Tomkyle, Germany (sea)

Teams were given a diploma and set of eZ430-Chronos development tool sponsored by Texas Instruments.

 

Watch the ERL Emergency 2017 Awards Ceremony video

More info

The European Robotics League is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement n° 688441.

The ERL Emergency Robots 2017 competition was organised locally by the NATO STO-Centre for Maritime Research and Experimentation (CMRE) of La Spezia, Italy.

A language to design control algorithms for robot swarms


Designing and representing control algorithms is challenging in swarm robotics, where the collective swarm performance depends on interactions between robots and with their environment. The currently available modeling languages, such as UML, cannot fully express these interactions. The Behaviour-Data Relations Modeling Language (BDRML) explicitly represents robot behaviours and data that robots utilise, as well as relationships between them. This allows BDRML to express control algorithms where robots cooperate and share information with each other while interacting with the environment. Here’s the work I presented this week at #IROS2017.

BDRML primitives

Primitives are the basic blocks of BDRML. They include:

  • Behaviour: A set of processes that deal with a particular situation a robot finds itself in, for example “Scout”
  • Internal data structure: Information that is stored in a robot’s memory
  • External data structure: Information that is stored in a non-robot entity, i.e., in the robot’s environment

BDRML primitives

BDRML relations

The following relations between entities can exist:

  • Transition: The robot transitions from one behavioural mode to another
  • Read \ Write: Internal data is used \ stored by the robot engaged in particular behaviour
  • Receive \ Send: External data is used \ stored by the robot. In the case of the Send relation, a robot may also send the data to another robot that stores it in its own internal data structure
  • Copy: Information is copied from one data structure to another
  • Update: The value of a data structure is updated from that in the previous time step by a subroutine not visualised in the BDRML diagram (for example, a pheromone level may spontaneously decrease over time).

The write and send relations can optionally define the new data structure value or a function that updates the value, indicated by a dashed line extending from the end of the relation arrow in a visual description, and written before a colon proceeding the data structure name in a textual description. The \textit{update} relation always must specify the new value or the value update function.

BDRML relations

BRML relation conditions

Each relation or operation occurs under a specific set of conditions. A condition is visually represented as an annotated triangle at the beginning of a relation or operation arrow. In a textual representation, a condition set follows a relation signature and is separated from it by a colon. Unless otherwise specified, the “or” logical operator is used when multiple conditions affect a single relation.

BDRML relation conditions

Example

A full BDRML representation consists of both visual and textual specification. A set of behaviours, B, internal data structures, Di and external data structures, De, are first defined, followed by a list of relations between them. Each box, circle and arrow in the visual representation must have a corresponding element or line in the textual representation and vice versa.

A BDRML representation exampleAn example is shown in the picture. The described algorithm allows robots to search for worksites and recruit each other to perform work and it can be applied for decentralised task allocation. A robot performs the “Scout” behaviour by searching the environment for worksites that can be found with a probability p(F). A successful Scout, that finds a worksite, performs the “Work” behaviour, during which it reads from and writes into its internal data structure, “Worksite location”, to keep track of where the worksite is located. Additionally, a working robot sends Worksite location to any Scout that it encounters in order to recruit it.

Note how the condition that allows a robot to transition from the Scout to the Work behaviour can be triggered by both p(F) or by recruitment, i.e., by existence of the internal data structure in the Scout’s memory. Also note that the condition of recruitment, “scout encountered” signifies that the two robots have to be at a similar place at a similar time for recruitment to occur. The BDRML diagram fully and unambiguously describes when recruitment is performed, what information is exchanged between robots and how it affects robot behaviour.

Publication:
Pitonakova, L., Crowder, R. & Bullock, S. (in press). Behaviour-Data Relations Modelling Language For Multi-Robot Control Algorithms. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017), IEEE.

“Superhero” robot wears different outfits for different tasks

Dubbed “Primer,” a new cube-shaped robot can be controlled via magnets to make it walk, roll, sail, and glide. It carries out these actions by wearing different exoskeletons, which start out as sheets of plastic that fold into specific shapes when heated. After Primer finishes its task, it can shed its “skin” by immersing itself in water, which dissolves the exoskeleton. Credit: the researchers.

From butterflies that sprout wings to hermit crabs that switch their shells, many animals must adapt their exterior features in order to survive. While humans don’t undergo that kind of metamorphosis, we often try to create functional objects that are similarly adaptive — including our robots.

Despite what you might have seen in “Transformers” movies, though, today’s robots are still pretty inflexible. Each of their parts usually has a fixed structure and a single defined purpose, making it difficult for them to perform a wide variety of actions.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are aiming to change that with a new shape-shifting robot that’s something of a superhero: It can transform itself with different “outfits” that allow it to perform different tasks.

Dubbed “Primer,” the cube-shaped robot can be controlled via magnets to make it walk, roll, sail, and glide. It carries out these actions by wearing different exoskeletons, which start out as sheets of plastic that fold into specific shapes when heated. After Primer finishes its task, it can shed its “skin” by immersing itself in water, which dissolves the exoskeleton.

“If we want robots to help us do things, it’s not very efficient to have a different one for each task,” says Daniela Rus, CSAIL director and principal investigator on the project. “With this metamorphosis-inspired approach, we can extend the capabilities of a single robot by giving it different ‘accessories’ to use in different situations.”

Primer’s various forms have a range of advantages. For example, “Wheel-bot” has wheels that allow it to move twice as fast as “Walk-bot.” “Boat-bot” can float on water and carry nearly twice its weight. “Glider-bot” can soar across longer distances, which could be useful for deploying robots or switching environments.

Primer can even wear multiple outfits at once, like a Russian nesting doll. It can add one exoskeleton to become “Walk-bot,” and then interface with another, larger exoskeleton that allows it to carry objects and move two body lengths per second. To deploy the second exoskeleton, “Walk-bot” steps onto the sheet, which then blankets the bot with its four self-folding arms.

“Imagine future applications for space exploration, where you could send a single robot with a stack of exoskeletons to Mars,” says postdoc Shuguang Li, one of the co-authors of the study. “The robot could then perform different tasks by wearing different ‘outfits.’”

The project was led by Rus and Shuhei Miyashita, a former CSAIL postdoc who is now director of the Microrobotics Group at the University of York. Their co-authors include Li and graduate student Steven Guitron. An article about the work appears in the journal Science Robotics on Sept. 27.

Robot metamorphosis

Primer builds on several previous projects from Rus’ team, including magnetic blocks that can assemble themselves into different shapes and centimeter-long microrobots that can be precisely customized from sheets of plastic.

While robots that can change their form or function have been developed at larger sizes, it’s generally been difficult to build such structures at much smaller scales.

“This work represents an advance over the authors’ previous work in that they have now demonstrated a scheme that allows for the creation of five different functionalities,” says Eric Diller, a microrobotics expert and assistant professor of mechanical engineering at the University of Toronto, who was not involved in the paper. “Previous work at most shifted between only two functionalities, such as ‘open’ or ‘closed’ shapes.”

The team outlines many potential applications for robots that can perform multiple actions with just a quick costume change. For example, say some equipment needs to be moved across a stream. A single robot with multiple exoskeletons could potentially sail across the stream and then carry objects on the other side.

“Our approach shows that origami-inspired manufacturing allows us to have robotic components that are versatile, accessible, and reusable,” says Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT.

Designed in a matter of hours, the exoskeletons fold into shape after being heated for just a few seconds, suggesting a new approach to rapid fabrication of robots.

“I could envision devices like these being used in ‘microfactories’ where prefabricated parts and tools would enable a single microrobot to do many complex tasks on demand,” Diller says.

As a next step, the team plans to explore giving the robots an even wider range of capabilities, from driving through water and burrowing in sand to camouflaging their color. Guitron pictures a future robotics community that shares open-source designs for parts much the way 3-D-printing enthusiasts trade ideas on sites such as Thingiverse.

“I can imagine one day being able to customize robots with different arms and appendages,” says Rus. “Why update a whole robot when you can just update one part of it?”

This project was supported, in part, by the National Science Foundation.

#ERLEmergency2017 in tweets


The ERL Emergency Robots 2017 (#ERLemergency2017) major tournament in Piombino, Italy, gathered 130 participants from 16 universities and companies from 8 European countries. Participating teams designed robots able to bring the first relief to survivors in disaster-response scenarios. The #ERLemergency2017 scenarios were inspired by the Fukushima 2011 nuclear accident. The robotics competition took place from 15-23 September 2017 at Enel’s Torre del Sale, and saw sea, land and air robots collaborating.

Teams worked very hard during the practice and competition days:

Robots could be found also in the exhibition area:

Or enjoying the sea:

Robotics experts held presentations and demos for the general public during the Opening Ceremony in Piazza Bovio.

The public got to know the teams,

And to see some emergency robots in action with the demo of TRADR project:

The competition site benefitted from the visit of some personalities:

Since they are the new generation of roboticists, children were not forgotten either: they enjoyed the free classes given by Scuola di Robotica.

At the Piombino Castle, the public attended more #robotics presentations.

After days of hard work, passion and enjoyment, the winners of the Grand challenge were announced:

Drones can almost see in the dark

We want to share with you our recent breakthrough teaching drones to fly using an eye-inspired camera, which opens the door to them performing fast, agile maneuvers and flying in low-light environments, where all commercial drones fail. Possible applications include supporting rescue teams with search missions at dusk or dawn. We submitted this work to the IEEE Robotics and Automation Letters.

How it works
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high speed motions or in scenes characterized by high dynamic range. However, event cameras output only little information when the amount of motion is limited, such as in the case of almost still motion. Conversely, standard cameras provide instant and rich information about the environment most of the time (in low-speed and good lighting scenarios), but they fail severely in case of fast motions or difficult lighting such as high dynamic range or low light scenes. In this work, we present the first state estimation pipeline that leverages the complementary advantages of these two sensors by fusing in a tightly-coupled manner events, standard frames, and inertial measurements. We show that our hybrid pipeline leads to an accuracy improvement of 130% over event-only pipelines, and 85% over standard-frames only visual-inertial systems, while still being computationally tractable. Furthermore, we use our pipeline to demonstrate—to the best of our knowledge—the first autonomous quadrotor flight using an event camera for state estimation, unlocking flight scenarios that were not reachable with traditional visual inertial odometry, such as low-light environments and high dynamic range scenes: we demonstrate how we can even fly in low light (such as after switching completely off the light in a room) or scenes characterized by a very high dynamic range (one side of the room highly illuminated and another side of the room dark).

Paper:
T. Rosinol Vidal, H.Rebecq, T. Horstschaefer, D. Scaramuzza
Hybrid, Frame and Event based Visual Inertial Odometry for Robust, Autonomous Navigation of Quadrotors, submitted to IEEE Robotics and Automation Letters PDF

#IROS2017 Live Coverage

The 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017) is being held in Vancouver Canada this week. The theme of IROS 2017 is “Friendly People, Friendly Robots”. Robots and humans are becoming increasingly integrated in various application domains. We work together in factories, hospitals and households, and share the road. This collaborative partnership of humans and robots gives rise to new technological challenges and significant research opportunities in developing friendly robots that can work effectively with, for, and around people.

And it’s also IROS’s 30th birthday this year! The occasion for much celebration.

Many Robohubbers will be at IROS, watch out for Sabine, Audrow, Andra, Hallie, and AJung. We’re also looking for new members to join our community. If you’re interested, email sabine.hauert@robohub.org, and we’ll make sure to meet during the conference!


Udacity Robotics video series: Interview with Chris Anderson from 3D Robotics


Mike Salem from Udacity’s Robotics Nanodegree is hosting a series of interviews with professional roboticists as part of their free online material.

This week we’re featuring Mike’s interview with Chris Anderson, Co-Founder and CEO of 3D Robotics. Chris is a former Wired magazine editor turned robotics company co-founder and CEO. Learn about Chris’s amazing journey into the world of unmanned aerial vehicles.

You can find all the interviews here. We’ll be posting them regularly on Robohub.

Talking Machines: The long view and learning in person, with John Quinn


In episode nine of season three we chat about the difference between models and algorithms, take a listener question about summer schools and learning in person as opposed to learning digitally, and we chat with John Quinn of the United Nations Global Pulse lab in Kampala, Uganda and Makerere University’s Artificial Intelligence Research group.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Robotics, the traditional path and new approaches

The hype cycle representation of the robotics field based on the general interest since its inception obtained from a joint review of publications, conferences, events and solutions. From “Dissecting Robotics — historical overview and future perspectives”.

Robotics, like many other technologies, suffered from an inflated set of expectations resulting in a decrease of the developments and results during the 90s. Over the last years, several groups thought that flying robots, commonly known as drones, would address these limitations however it seems unlikely that the popularity of these flying machines will drive and push the robot growth as expected. This article aims to summarize traditional techniques used to build and program robots together with new trends that aim to simplify and enhance the progress in the field.

Building robots

It’s a rather popular thought that building a robot and programing its behavior remain two highly complicated tasks. Recent advances in adopting ROS as a standardized software framework for developing robot applications helped with the latter, however building a robot remains a challenge. The lack of compatible systems in terms of hardware, the non existing marketplace of reusable modules, or the expertise required to develop the most basic behaviors are some of the few listed hurdles.

The integration-oriented approach
Robots are typically built by following the step-by-step process described below:

1. Buy parts: We typically decide on what components our robot will need. A mobile base, a range finder, a processing device, etc. Once decided we fetch those that match our requirements and proceed towards integration.

2. Integration: Making different components speak to each other and cooperate towards achieving the end goal of the robot. Surprisingly, that’s where most of our time is spent.

3. “build the robot”: Assembling all of the components into joints and mechanically linking them. This step might also get executed together with step

4. Programming the robot: Making the robot do what it’s meant to do.

5. Test & adapt: Robots are typically programmed in predictable scenarios. Testing the pre-programmed behavior in real scenarios is always critical. Generally, these tests deliver results that indicate where adaptations are needed, which in many cases pushes de process of building a robot down to step 2 again, integration.

6. Deploy: Ship it!

The “integration-oriented” approach for building a robot.

It’s well understood that building a robot is a technically challenging task. Engineers often face situations where the integration effort of the robot, generally composed by diverse sub-components, supersedes many other tasks. Furthermore, every hardware modification/adaptation while programming or building the robot demands further integration.

This method for building robots produces results that become obsolete within a short period.

Moreover, modules within the robots aren’t reusable in most of the cases since the integration effort makes reusability an incredibly expensive (manpower-wise) and time-consuming task.

The modular approach

The existing trend in robotics is producing a significant number of hardware devices. Although there’s an existing trend towards using the Robot Operating System (ROS), when compared to each other, these components typically consist of incompatible electronic components with different software interfaces.

Now, imagine building robots by connecting interoperable modules together. Actuator, sensors, communication modules, UI devices, provided everything interoperates together, the whole integration effort could be eliminated. The overall process of building robots could be simplified and the development effort and time will be reduced significantly.

Comparison between the “integration-oriented” and the “modular” approaches for building robots.

Modular components could be reused among robots and that’s exactly what we’re working on with H-ROS, the Hardware Robot Operating System.
H-ROS is a vendor-agnostic infrastructure for the creation of robot modules that interoperate and can be exchanged between robots. H-ROS builds on top of ROS, the Robot Operating System, which is used to define a set of standardized logical interfaces that each physical robot component must meet if compliant with H-ROS.

Programming robots

The robotics control pipeline

Traditionally, the process of programming a robot for a given task T is described as follows:
Observation: Robot’s sensors produce measurements. All these measurements receive the name of “observations” and are the inputs that the robot receives to execute task T.

State estimation: Given the observations of step 1, we describe the robot’s motion over time by inferring a set of characteristics of the robot such as its position, its orientation or its velocity. Obviously, mistakes in the observations will lead to errors in the state estimation.

Modeling & Prediction: Determine the dynamics of the robot (rules for how to move it around) using a) the robot model (typically the URDF of the robot in the ROS world) and b) the state estimation. Similarly to what happened with the previous step, errors in “state estimation” will impact the results obtained in this step.

Planning: this step determines the actions required to execute task T and uses both the state estimation and the dynamical model from previous steps in the pipeline.

Low level control: the final step in the pipeline consists of transforming the “plan” into low level control commands that steer the robot actuators.

The traditional “robotics control pipeline”

Bio-inspired techniques

Artificial Intelligence methods and, particularly, bio-inspired techniques such as artificial neural networks (ANNs) are becoming more and more relevant in robotics. Starting from 2009, ANNs gained popularity and started delivering good results in the fields of computer vision (2012) or machine translation (2014). Nowadays, these fields are completely filled by techniques that simulate the neural/synaptic activity of the brain of a living organism.

During the last years we have seen how these techniques have been translated to robotics for tasks such as robotic grasping (2016). Our team has been putting resources into exploring these techniques
that enable to train a robotic device in a manner conceptually similar to the mode in which one goes about training a domesticated animal such as a dog or cat.

Training robots end-to-end for a given task. This integrated and bio-inspired approach conflicts with the traditional robotics pipeline, however it’s already showing promising results of behaviors that generalize.
We are excited to share that it’s within our expectations to see more active use of these bio-inspired techniques. We are confident that its use will drive innovations with high impact for robotics and we hope to contribute by opening part of our work and results.

Programming robots versus training robots. This image pictures the traditional robotics approach named as the “robotics control pipeline” and the new “bio-inspired” approach that makes use of AI techniques that simulate the neural/synaptic activity of the brain.

The roboticist matrix

All these new approaches for both building and programming robots bring a dilemma to roboticists. What should they focus on? Which approach should they follow for each particular use case? Let’s analyze the different combinations:

The roboticist matrix presents a comparison between traditional and new approaches for building and programming robots.

Integration-oriented + robotics control pipeline:
This combination represents the “traditional approach” in all senses. It’s the process that most robot solutions use nowadays in industry. Integrated robots that typically belong to a single manufacturer. Such robots are programmed in a structured way to execute a well defined task. Typically achieving high levels of accuracy and repeatability. However, any uncertainty in the environment will typically drive the robot to fail on its task. Expenses related to develop such systems are typically in the range of 10.000–100.000 € for the simplest behaviors and an order of magnitud above for the more complex tasks.

Integration-oriented + bio-inspired:
Behaviors that evolve, but with strong hardware constraints and limitations. Traditional robots enhanced with bio-inspired approaches. Robots using this combination will be able to learn by themselves and adapt to changes in the environment however any modification, repurpose or extension within the robot hardware will require big integration efforts. The expenses for developing these robots are similar to the ones presented for the “traditional approach”.

Modular + robotics control pipeline:
Flexible hardware with structured behaviors. These robots will be built in a modular way. Building, repairing and/or repurposing these robots will be extremely affordable when compared to traditional robots (built with the integration-oriented approach), we estimate an order of magnitude less (1.000–10.000 €). Furthermore, modularity will introduce new opportunities for these robots.

Modular + bio-inspired:
This combination represents the most innovative one and has the potential to disrupt the whole robotics market changing both the way we build and program/train robots. Yet it’s also the most immature one.
Similar to the previous approach group, our team foresees that the expenses for putting together these robots can also be reduced when compared to the more traditional approaches. We estimate that building and training these robots should range in terms of expenses from 1.000 to 10.000 € for simple scenarios and up to 50.000 € for the more elaborated ones.

Our path towards the future: modular robots and H-ROS

A modular robot built using H-ROS compatible components.

The team behind Erle Robotics is proud to announce that together with Acutronic Robotics (Switzerland), Sony (Japan) is now also pushing the development of H-ROS, the Hardware Robot Operating System. A technology that aims to change the landscape of robotics by creating an ecosystem where hardware components can be reused among different robots, regardless of the original manufacturer. Our team strongly believe that the future of robotics will be about modular robots that can be easily repaired and reconfigured. H-ROS aims to shape this future. Sony’s leadership and vision in robotics is widely recognized in the community. We are confident that, with the addition of Sony as a supporter, our present innovations will spread even more rapidly.

Our team is focused in exploring these new opportunities and will introduce some results this week in Vancouver during ROSCon. Show your interest and join us in Canada!

The importance of research reproducibility in robotics

As highlighted in a previous post, despite the fact that robotics is increasingly regarded as a ‘Science’, as shown by the launch of new journals such as Science Robotics, reproducibility of experiments is still difficult or entirely lacking.

This is quite unfortunate as the possibility of reproducing experimental results is a cornerstone of the scientific method. This situation pushes serious discussions (What’s ‘soft robotics’? Is it needed? What has to be ’soft’?) and paradigm clashes (Good Old Fashioned Artificial Intelligence vs. Deep Learning vs. Embodied Cognition) into the realm of literary controversy or even worse religious territory fights, with quite little experimental evidence supporting the claims of the different parties. Not even wrong, as they say (following Peter Woit’s arguments on String Theory)?

The robotics community has been aware of these issues for a long time and more and more researchers in recent years have published datasets, code and other valuable information to allow others to reproduce their results. We are heading in the right direction, but we probably need to do more.

I think we should therefore welcome the fact that for the first time ever, IEEE R&A Mag. will start accepting R-articles (i.e., papers that report experiments aiming to be fully reproducible) beginning this September. Actually, they will also accept short articles reporting on the replication of R-Article results, and author replies are solicited and will be published after peer-review. The result will be a two-stage high-quality review process. The first stage will be the ordinary rigorous review process of a top-tier publishing venue. The second stage will be the replication of the experiments by the community (which is the core of the scientific method).

This seems like a historical improvement, doesn’t it?

There is more information on this in the column I wrote in the September issue of IEEE Robotics and Automation.

Automatic code reuse

“CodeCarbonCopy enables one of the holy grails of software engineering: automatic code reuse,” says Stelios Sidiroglou-Douskos, a research scientist at CSAIL. Credit: MIT News

by Larry Hardesty

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a new system that allows programmers to transplant code from one program into another. The programmer can select the code from one program and an insertion point in a second program, and the system will automatically make modifications necessary — such as changing variable names — to integrate the code into its new context.

Crucially, the system is able to translate between “data representations” used by the donor and recipient programs. An image-processing program, for instance, needs to be able to handle files in a range of formats, such as jpeg, tiff, or png. But internally, it will represent all such images using a single standardized scheme. Different programs, however, may use different internal schemes. The CSAIL researchers’ system automatically maps the donor program’s scheme onto that of the recipient, to import code seamlessly.

The researchers presented the new system, dubbed CodeCarbonCopy, at the Association for Computing Machinery’s Symposium on the Foundations of Software Engineering.

“CodeCarbonCopy enables one of the holy grails of software engineering: automatic code reuse,” says Stelios Sidiroglou-Douskos, a research scientist at CSAIL and first author on the paper. “It’s another step toward automating the human away from the development cycle. Our view is that perhaps we have written most of the software that we’ll ever need — we now just need to reuse it.”

The researchers conducted eight experiments in which they used CodeCarbonCopy to transplant code between six popular open-source image-processing programs. Seven of the eight transplants were successful, with the recipient program properly executing the new functionality.

Joining Sidiroglou-Douskos on the paper are Martin Rinard, a professor of electrical engineering and computer science; Fan Long, an MIT graduate student in electrical engineering and computer science; and Eric Lahtinen and Anthony Eden, who were contract programmers at MIT when the work was done.

Mutatis mutandis

With CodeCarbonCopy, the first step in transplanting code from one program to another is to feed both of them the same input file. The system then compares how the two programs process the file.

If, for instance, the donor program performs a series of operations on a particular piece of data and loads the result into a variable named “mem_clip->width,” and the recipient performs the same operations on the same piece of data and loads the result into a variable named “picture.width,” the system will infer that the variables are playing the same roles in their respective programs.

Once it has identified correspondences between variables, CodeCarbonCopy presents them to the user. It also presents all the variables in the donor for which it could not find matches in the recipient, together with those variables’ initial definitions. Frequently, those variables are playing some role in the donor that’s irrelevant to the recipient. The user can flag those variables as unnecessary, and CodeCarbonCopy will automatically excise any operations that make use of them from the transplanted code.

New order

To map the data representations from one program onto those of the other, CodeCarbonCopy looks at the precise values that both programs store in memory. Every pixel in a digital image, for instance, is governed by three color values: red, green, and blue. Some programs, however, store those triplets of values in the order red, green, blue, and others store them in the order blue, green, red.

If CodeCarbonCopy finds a systematic relationship between the values stored by one program and those stored by the other, it generates a set of operations for translating between representations.

CodeCarbonCopy works well with file formats, such as images, whose data is rigidly organized, and with programs, such as image processors, that store data representations in arrays, which are essentially rows of identically sized memory units. In ongoing work, the researchers are looking to generalize their approach to file formats that permit more flexible data organization and programs that use data structures other than arrays, such as trees or linked lists.

“In general, code quoting is where a lot of problems in software come from,” says Vitaly Shmatikov, a professor of computer science at Cornell Tech, a joint academic venture between Cornell University and Israel’s Technion. “Both bugs and security vulnerabilities — a lot of them occur when there is functionality in one place, and someone tries to either cut and paste or reimplement this functionality in another place. They make a small mistake, and that’s how things break. So having an automated way of moving code from one place to another would be a huge, huge deal, and this is a very solid step toward having it.”

“Recognizing irrelevant code that’s not important for the functionality that they’re quoting, that’s another technical innovation that’s important,” Shmatikov adds. “That’s the kind of thing that was an obstacle for a lot of previous approaches — that you know the right code is there, but it’s mixed up with a lot of code that is not relevant to what you’re trying to do. So being able to separate that out is a fairly significant technical contribution.”

Page 411 of 427
1 409 410 411 412 413 427