Sailors assigned to Explosive Ordnance Disposal Mobile Unit 5 (EODMU5) Platoon 142 recover an unmanned underwater vehicle onto a Coastal Riverine Group 1 Detachment Guam MK VI patrol boat in the Pacific Ocean May 10, 2017. Credit: Mass Communication Specialist 1st Class Torrey W. Lee/ U.S. Navy
May 8, 2017 – May 14, 2017
If you would like to receive the Weekly Roundup in your inbox, please subscribe at the bottom of the page.
News
The International Civil Aviation Organization announced that it plans to develop global standards for small unmanned aircraft traffic management. In a statement at the Association of Unmanned Vehicle Systems International’s Xponential trade conference, the United Nations agency said that as part of the initiative it has issued a Request for Information on air traffic management systems for drones. (GPS World)
Virginia Governor Terry McAuliffe has created a new office dedicated to drones and autonomous systems. According to Gov. McAuliffe, the Autonomous Systems Center for Excellence will serve as a “clearinghouse and coordination point” for research and development programs related to autonomous technologies. (StateScoop)
Commentary, Analysis, and Art
At the Telegraph, Alan Tovey writes that the U.K.’s exit from the European Union is unlikely to affect cross-channel cooperation on developing fighter drones.
Nautilus, a California startup, is developing a cargo drone that could carry thousands of pounds of goods over long distances. (Air & Space Magazine)
Drone maker Pulse Aerospace unveiled two new rotorcraft drones for military and commercial applications, the Radius 65 and the Vapor 15. (Press Release)
Piaseki Aerospace will likely submit its ARES demonstrator drone for the U.S. Marine Corps’ Unmanned Expeditionary Capabilities program. (FlightGlobal)
Defense firm Kratos confirmed that it has conducted several demonstration flights of a high performance jet drone for an undisclosed customer. (FlightGlobal)
Technology firm Southwest Research Institute has been granted a patent for a system by which military drones can collaborate with unmanned ground vehicles. (Unmanned Aerial Online)
The U.S. Army is interested in developing a mid-size unmanned cargo vehicle that could carry up to 800 pounds of payload. (FlightGlobal)
A drone flying over a bike race in in Rancho Cordova, California crashed into a cyclist. (Market Watch)
Meanwhile, a consumer drone crashed into a car crossing the Sydney Harbor Bridge in Australia. It is the second time a drone has crashed at the site of the bridge in the past nine months. (Sydney Morning Herald)
Insurance company Travelers has trained over 150 drone operators to use drones for insurance appraisals over properties. (Insurance Journal)
A Latvian technology firm used a large multirotor drone to carry a skydiver to altitude before he parachuted back down to earth. (Phys.org)
Clear Flight Solutions and AERIUM Analytics are set to begin integrating the Robird drone system, a falcon-like drone that scares birds away from air traffic, at Edmonton International Airport. (Unmanned Systems Technology)
Industry Intel
The U.S. Army awarded General Atomics Aeronautical Systems a $221.6 million contract modification for 20 extended range Gray Eagle drones and associated equipment. (DoD)
The U.S. Air Force awarded General Electric a $14 million contract for work that includes the Thermal Management System for unmanned aircraft. (DoD)
Turkish Aerospace Industries will begin cooperating with ANTONOV Company on the development of unmanned systems. (Press Release)
Aker, a company that develops drones for agriculture, won $950,000 in funding from the Clean Energy Trust Challenge. (Chicago Tribune)
For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.
JD Claridge’s story epitomizes the current state of the drone industry. Claridge, founder of xCraft, is best known for being the first contestant on Shark Tank to receive money from all the Sharks – even Kevin O’Leary! Walking the floor of Xponential 2017, the annual convention of the Association for Unmanned Vehicle Systems Integration (AUVSI), Claridge remarked to me how the drone industry has grown up since his TV appearance.
Claridge has gone from pitching cellphone cases that turn into drones (aka phonedrone) to solving mission critical problems. The age of fully autonomous flight is near and the drone industry is finally recovering from the hangover of overhyped Kickstarter videos (see Lily drone’s $34 million fraud). xCraft’s pivot to lightweight, power efficient, enterprise drones is an example of this evolved marketplace. During the three days of Xponential 2017, several far-reaching announcements were made between stalwarts of the tech industry and aviation startups. Claridge introduced me to his new partner, Rajant, which is a leader in industrial wireless networks. xCraft’s latest models utilize Rajant’s mesh networks to launch swarms of drones with one controller. More drones flying simultaneously enables users to maximize the flight time limitations of lithium batteries by covering greater areas within a single mission.
Bob Schena, Rajant’s CEO, said, “Rajant’s network technology now makes it possible for one pilot to operate many aircrafts concurrently, with flight times of 45 minutes. We’re pleased to partner with xCraft and bring more intelligence, mobility and autonomy to UAV communication infrastructures covering greater aerial distances while supporting various drone payloads.”
The battery has been the Achilles heel of the small drone industry since inception. While large winged craft relies heavily on fossil fuels, multirotor battery-operated drones have been plagued with shorter missions of under 45 minutes. Innovators like Claridge are leading the way for a new wave of creative solutions:
Solar Powered Wings
Solar Powered Wings
Airbus showcased its Zephyr drone products or HAPS (High Altitude Pseudo-Satellite) UAVs using solar-winged craft for power. Zephyr UAVs can fly for months at a time, saving thousands of tons of fuel. The HAPS also offers a number of lightweight payload options from voice communications to persistent internet to real-time surveillance. Airbus was not the only solar solution on display; there were a handful of Chinese upstarts and solar cell purveyors for retrofitting existing aircrafts.
Hybrid Fuel Solutions
In the Startup Pavilion, William Fredericks of the Advanced Aircraft Company (AAC) demoed a novel technology using a hybrid of diesel fuel and lithium batteries with flexible fixed wings and multirotors, resulting in over 3 hours of flying time. AAC’s prototype, the Hercules (above) is remarkably lightweight and fast. Fredricks is an aircraft designer by trade with 12 designs flying in the air, including NASA’s Greased Lightning that looks remarkably similar to Boeing’s Osprey. The Hercules is available for sale on the company’s website for multiple use cases, including: agricultural, first responders, and package delivery. It is interesting to note that a few rows from Frederick’s booth was his former employer, NASA, promoting their new Autonomy Incubator for “intelligent flight systems” and its “autonomy innovation lab,” (definitely an incubator to watch).
Vertical Take Off & Landing
In addition to hybrid fuel strategies, entrepreneurs are also rethinking the launch procedures. AAC’s Hercules and XCraft’s commercial line of drones vertically takeoff to reduce wind resistance and maximize energy consumption. Australian Startup Iridium Dynamics takes this approach to a new level with astonishing results. Its winged craft, Halo, uses a patent-pending “hover thrust” of its entire craft so its wings actually create the vertical lift to hover with minimal power. The drone also has two rotors to fly horizontally. According to Dion Gonano, Control Systems Engineer, it can fly for over 2 hours. The Halo also lands vertically into a stationary mechanical arm. While the website lists a number of commercial applications for this technology, it was unclear in my discussions with Gonano if they have deployed this technology in real tests.
New Charging Efficiencies
Prior to Xponential, Seattle-based WiBotic announced the closing of its $2.5 seed round to fund its next generation of battery charging technologies. The company has created a novel approach to wireless inductive charging for robotics. Its wireless inductive charging platform includes a patent-pending auto detect feature that can begin recharging once the robot enters the proximity of the base station, even during flight. According to Dr. Ben Waters, (CEO), its charge is faster than traditional solutions presently on the market. Dr. Waters demonstrated for me its suite of software tools that monitor battery performance, providing clients with a complete power management analytics platform. WiBotic is already piloting its technology with leading commercial customers in the energy and security sectors. WiBotic is the first inductive charging platform; other companies have created innovating battery-swapping techniques. Airobotics unique drone storage box that is deployed currently at power plants in Israel, includes a robotic arm, housed inside, that services the robot post flight by switching out the payload and battery:
Reducing Payload Weight
In addition to aircraft design, payload weight is a big factor of battery drain. A growing trend within the industry is miniaturizing the size and cost of the components. Ultimately, the mission of a drone is directly related to the type of payload from cameras for collecting images to precise measurements using Light Detection and Ranging sensors (or Lidar). Lidar is typically deployed in autonomous vehicles to provide the most precise position for the robot in a crowded area, like a self-driving car on the road. However, Lidar is currently extremely expensive and large for many multirotor surveys. Chris Brown of Z-Senz, a former scientist with the The National Institute of Standards and Technology (NIST), hopes to change the landscape of drones with his miniaturized Lidar sensor. Brown’s reduced sensor, SKY1, offers major advantages for size, weight, and power consumption without losing accuracy of high distance sensing. A recent study estimates the Lidar market is expected to exceed $5 billion by 2022, with Velodyne and Quanergy already gaining significant investment. Z-Senz is aiming to be commercially available by 2018.
Lidar is not the only measuring methodology, Global Positioning Solutions (GPS) have been deployed widely. Two of the finalists of the Xponetial Startup Showdown were startups focused on reducing GPS chip sizes and increasing functionality. Inertial Sense has produced a chip the size of a dime that is capable of housing an Inertial Measurement Unit (IMU), Attitude Heading Reference System (AHRS), and GPS-aided Inertial Navigation System (INS). Their website claims that their “advanced algorithms fuse output from MEMs inertial sensors, magnetometers, barometric pressure, and a high-sensitivity GPS (GNSS) receiver to deliver fast, accurate, and reliable attitude, velocity, and position even in the most dynamic environments.” The chips and micro navigation accessories are available on the company’s e-store.
The winner of the Showdown, uAvionix, is a leading developer of avionics for both manned and unmanned flight. Their new transceivers and transponders claim to be “the smallest, and lightest and most affordable on the market” (already GPS is a commodity). uAvionix presented its “Ping Network System that reduces weight on average by 40% as compared to the two-piece installations.” The Ping products also claim barometric altitude precision with accuracy beyond 80,000 ft.
Paul Beard, CEO of uAvionix, said, “our customers have asked for even smaller and lighter solutions; integrating the transceivers, GPS receivers, GPS antennas, and barometric pressure sensors into a single form factor facilitates easier installation and lowers weight and power draw requirements resulting in a longer usable flight time.”
As I rushed to the airport to catch my manned flight, I felt reenergized about the drone industry, although follies will persist. I mean who wouldn’t want a pool deckchair drone this summer?
This and all other autonomous subjects will be explored at RobotLabNYC’s next event with Dr. Howard Morgan (FirstRound Capital) and Tom Ryden (MassRobotics) – RSVP.
Over 800 leading scientists, companies, and policymakers working in robotics will convene at the European Robotics Forum (#ERF2017) in Edinburgh, 22-24 March. This year’s theme is “Living and Working With Robots” with a focus on applications in manufacturing, disaster relief, agriculture, healthcare, assistive living, education, and mining.
The 3-day programme features keynotes, panel discussions, workshops, and plenty of robots roaming the exhibit floor.
We’ll be updating this post regularly with live tweets and videos. You can also follow all the Robohub coverage here. #erf2017 Tweets
Engineers have designed a microfluidic device they call a “tree-on-a-chip,” which mimics the pumping mechanism of trees and other plants.
Trees and other plants, from towering redwoods to diminutive daisies, are nature’s hydraulic pumps. They are constantly pulling water up from their roots to the topmost leaves, and pumping sugars produced by their leaves back down to the roots. This constant stream of nutrients is shuttled through a system of tissues called xylem and phloem, which are packed together in woody, parallel conduits.
Now engineers at MIT and their collaborators have designed a microfluidic device they call a “tree-on-a-chip,” which mimics the pumping mechanism of trees and plants. Like its natural counterparts, the chip operates passively, requiring no moving parts or external pumps. It is able to pump water and sugars through the chip at a steady flow rate for several days. The results are published this week in Nature Plants.
Anette “Peko” Hosoi, professor and associate department head for operations in MIT’s Department of Mechanical Engineering, says the chip’s passive pumping may be leveraged as a simple hydraulic actuator for small robots. Engineers have found it difficult and expensive to make tiny, movable parts and pumps to power complex movements in small robots. The team’s new pumping mechanism may enable robots whose motions are propelled by inexpensive, sugar-powered pumps.
“The goal of this work is cheap complexity, like one sees in nature,” Hosoi says. “It’s easy to add another leaf or xylem channel in a tree. In small robotics, everything is hard, from manufacturing, to integration, to actuation. If we could make the building blocks that enable cheap complexity, that would be super exciting. I think these [microfluidic pumps] are a step in that direction.”
Hosoi’s co-authors on the paper are lead author Jean Comtet, a former graduate student in MIT’s Department of Mechanical Engineering; Kaare Jensen of the Technical University of Denmark; and Robert Turgeon and Abraham Stroock, both of Cornell University.
A hydraulic lift
The group’s tree-inspired work grew out of a project on hydraulic robots powered by pumping fluids. Hosoi was interested in designing hydraulic robots at the small scale, that could perform actions similar to much bigger robots like Boston Dynamic’s Big Dog, a four-legged, Saint Bernard-sized robot that runs and jumps over rough terrain, powered by hydraulic actuators.
“For small systems, it’s often expensive to manufacture tiny moving pieces,” Hosoi says. “So we thought, ‘What if we could make a small-scale hydraulic system that could generate large pressures, with no moving parts?’ And then we asked, ‘Does anything do this in nature?’ It turns out that trees do.”
The general understanding among biologists has been that water, propelled by surface tension, travels up a tree’s channels of xylem, then diffuses through a semipermeable membrane and down into channels of phloem that contain sugar and other nutrients.
The more sugar there is in the phloem, the more water flows from xylem to phloem to balance out the sugar-to-water gradient, in a passive process known as osmosis. The resulting water flow flushes nutrients down to the roots. Trees and plants are thought to maintain this pumping process as more water is drawn up from their roots.
“This simple model of xylem and phloem has been well-known for decades,” Hosoi says. “From a qualitative point of view, this makes sense. But when you actually run the numbers, you realize this simple model does not allow for steady flow.”
In fact, engineers have previously attempted to design tree-inspired microfluidic pumps, fabricating parts that mimic xylem and phloem. But they found that these designs quickly stopped pumping within minutes.
It was Hosoi’s student Comtet who identified a third essential part to a tree’s pumping system: its leaves, which produce sugars through photosynthesis. Comtet’s model includes this additional source of sugars that diffuse from the leaves into a plant’s phloem, increasing the sugar-to-water gradient, which in turn maintains a constant osmotic pressure, circulating water and nutrients continuously throughout a tree.
Running on sugar
With Comtet’s hypothesis in mind, Hosoi and her team designed their tree-on-a-chip, a microfluidic pump that mimics a tree’s xylem, phloem, and most importantly, its sugar-producing leaves.
To make the chip, the researchers sandwiched together two plastic slides, through which they drilled small channels to represent xylem and phloem. They filled the xylem channel with water, and the phloem channel with water and sugar, then separated the two slides with a semipermeable material to mimic the membrane between xylem and phloem. They placed another membrane over the slide containing the phloem channel, and set a sugar cube on top to represent the additional source of sugar diffusing from a tree’s leaves into the phloem. They hooked the chip up to a tube, which fed water from a tank into the chip.
With this simple setup, the chip was able to passively pump water from the tank through the chip and out into a beaker, at a constant flow rate for several days, as opposed to previous designs that only pumped for several minutes.
“As soon as we put this sugar source in, we had it running for days at a steady state,” Hosoi says. “That’s exactly what we need. We want a device we can actually put in a robot.”
Hosoi envisions that the tree-on-a-chip pump may be built into a small robot to produce hydraulically powered motions, without requiring active pumps or parts.
“If you design your robot in a smart way, you could absolutely stick a sugar cube on it and let it go,” Hosoi says.
This research was supported, in part, by the Defense Advance Research Projects Agency.
The Nereis virens worm inspired new research out of the MIT Laboratory for Atomistic and Molecular Mechanics. Its jaw is made of soft organic material, but is as strong as harder materials such as human dentin. Photo: Alexander Semenov/Wikimedia Commons
A new material that naturally adapts to changing environments was inspired by the strength, stability, and mechanical performance of the jaw of a marine worm. The protein material, which was designed and modeled by researchers from the Laboratory for Atomistic and Molecular Mechanics (LAMM) in the Department of Civil and Environmental Engineering (CEE), and synthesized in collaboration with the Air Force Research Lab (AFRL) at Wright-Patterson Air Force Base, Ohio, expands and contracts based on changing pH levels and ion concentrations. It was developed by studying how the jaw of Nereis virens, a sand worm, forms and adapts in different environments.
The resulting pH- and ion-sensitive material is able to respond and react to its environment. Understanding this naturally-occurring process can be particularly helpful for active control of the motion or deformation of actuators for soft robotics and sensors without using external power supply or complex electronic controlling devices. It could also be used to build autonomous structures.
“The ability of dramatically altering the material properties, by changing its hierarchical structure starting at the chemical level, offers exciting new opportunities to tune the material, and to build upon the natural material design towards new engineering applications,” wrote Markus J. Buehler, the McAfee Professor of Engineering, head of CEE, and senior author of the paper.
The research, recently published in ACS Nano, shows that depending on the ions and pH levels in the environment, the protein material expands and contracts into different geometric patterns. When the conditions change again, the material reverts back to its original shape. This makes it particularly useful for smart composite materials with tunable mechanics and self-powered roboticists that use pH value and ion condition to change the material stiffness or generate functional deformations.
Finding inspiration in the strong, stable jaw of a marine worm
In order to create bio-inspired materials that can be used for soft robotics, sensors, and other uses — such as that inspired by the Nereis — engineers and scientists at LAMM and AFRL needed to first understand how these materials form in the Nereis worm, and how they ultimately behave in various environments. This understanding involved the development of a model that encompasses all different length scales from the atomic level, and is able to predict the material behavior. This model helps to fully understand the Nereis worm and its exceptional strength.
“Working with AFRL gave us the opportunity to pair our atomistic simulations with experiments,” said CEE research scientist Francisco Martin-Martinez. AFRL experimentally synthesized a hydrogel, a gel-like material made mostly of water, which is composed of recombinant Nvjp-1 protein responsible for the structural stability and impressive mechanical performance of the Nereis jaw. The hydrogel was used to test how the protein shrinks and changes behavior based on pH and ions in the environment.
The Nereis jaw is mostly made of organic matter, meaning it is a soft protein material with a consistency similar to gelatin. In spite of this, its strength, which has been reported to have a hardness ranging between 0.4 and 0.8 gigapascals (GPa), is similar to that of harder materials like human dentin. “It’s quite remarkable that this soft protein material, with a consistency akin to Jell-O, can be as strong as calcified minerals that are found in human dentin and harder materials such as bones,” Buehler said.
At MIT, the researchers looked at the makeup of the Nereis jaw on a molecular scale to see what makes the jaw so strong and adaptive. At this scale, the metal-coordinated crosslinks, the presence of metal in its molecular structure, provide a molecular network that makes the material stronger and at the same time make the molecular bond more dynamic, and ultimately able to respond to changing conditions. At the macroscopic scale, these dynamic metal-protein bonds result in an expansion/contraction behavior.
Combining the protein structural studies from AFRL with the molecular understanding from LAMM, Buehler, Martin-Martinez, CEE Research Scientist Zhao Qin, and former PhD student Chia-Ching Chou ’15, created a multiscale model that is able to predict the mechanical behavior of materials that contain this protein in various environments. “These atomistic simulations help us to visualize the atomic arrangements and molecular conformations that underlay the mechanical performance of these materials,” Martin-Martinez said.
Specifically, using this model the research team was able to design, test, and visualize how different molecular networks change and adapt to various pH levels, taking into account the biological and mechanical properties.
By looking at the molecular and biological makeup of a the Nereis virens and using the predictive model of the mechanical behavior of the resulting protein material, the LAMM researchers were able to more fully understand the protein material at different scales and provide a comprehensive understanding of how such protein materials form and behave in differing pH settings. This understanding guides new material designs for soft robots and sensors.
Identifying the link between environmental properties and movement in the material
The predictive model explained how the pH sensitive materials change shape and behavior, which the researchers used for designing new PH-changing geometric structures. Depending on the original geometric shape tested in the protein material and the properties surrounding it, the LAMM researchers found that the material either spirals or takes a Cypraea shell-like shape when the pH levels are changed. These are only some examples of the potential that this new material could have for developing soft robots, sensors, and autonomous structures.
Using the predictive model, the research team found that the material not only changes form, but it also reverts back to its original shape when the pH levels change. At the molecular level, histidine amino acids present in the protein bind strongly to the ions in the environment. This very local chemical reaction between amino acids and metal ions has an effect in the overall conformation of the protein at a larger scale. When environmental conditions change, the histidine-metal interactions change accordingly, which affect the protein conformation and in turn the material response.
“Changing the pH or changing the ions is like flipping a switch. You switch it on or off, depending on what environment you select, and the hydrogel expands or contracts” said Martin-Martinez.
LAMM found that at the molecular level, the structure of the protein material is strengthened when the environment contains zinc ions and certain pH levels. This creates more stable metal-coordinated crosslinks in the material’s molecular structure, which makes the molecules more dynamic and flexible.
This insight into the material’s design and its flexibility is extremely useful for environments with changing pH levels. Its response of changing its figure to changing acidity levels could be used for soft robotics. “Most soft robotics require power supply to drive the motion and to be controlled by complex electronic devices. Our work toward designing of multifunctional material may provide another pathway to directly control the material property and deformation without electronic devices,” said Qin.
By studying and modeling the molecular makeup and the behavior of the primary protein responsible for the mechanical properties ideal for Nereis jaw performance, the LAMM researchers are able to link environmental properties to movement in the material and have a more comprehensive understanding of the strength of the Nereis jaw.
The research was funded by the Air Force Office of Scientific Research and the National Science Foundation’s Extreme Science and Engineering Discovery Environment (XSEDE) for the simulations.
Over 800 leading scientists, companies, and policymakers working in robotics will convene at the European Robotics Forum (#ERF2017) in Edinburgh, 22-24 March. This year’s theme is “Living and Working With Robots” with a focus on applications in manufacturing, disaster relief, agriculture, healthcare, assistive living, education, and mining.
The 3-day programme features keynotes, panel discussions, workshops, and plenty of robots roaming the exhibit floor. Visitors may encounter a humanoid from Pal Robotics, a bartender robot from KUKA, Shadow’s human-like hands, or the latest state-of-the-art robots from European research. Success stories from Horizon 2020, the European Union’s framework programme for research and innovation, and FP7 European projects will be on display.
Dr Cécile Huet Deputy Head of European Commission Robotics & Artificial Intelligence Unit, said, “A set of EU projects will demonstrate the broad impact of the EU funding programme in robotics: from progress in foundational research in robot learning, to in touch sensing for a new dimension in intuitive Human-Robot cooperation, to inspection in the oil-and-gas industry, security, care, manufacturing for SMEs, or the vast applications enabled by the progress in drones autonomous navigation.”
Reinhard Lafrenz, Secretary General of euRobotics said, “A rise in sales in robotics is driving the industry forward, and it’s not just benefiting companies who sell robots, but also SMEs and larger industries that use robots to increase their productivity and adopt new ways of thinking about their business. Around 80 robotics start-ups were created last year in Europe, which is truly remarkable. At euRobotics, we nurture the robotics industry ecosystem in Europe; keep an eye out for the Tech Transfer award and the Entrepreneurship award we’ll be giving out at ERF.”
Projects presented will include:
FUTURA – Focused Ultrasound Therapy Using Robotic Approaches
PETROBOT – Use cases for inspection robots opening up the oil-, gas- and petrochemical markets
SMErobotics – The European Robotics Initiative for Strengthening the Competitiveness of SMEs in Manufacturing by Integrating aspects of Cognitive Systems
STRANDS – Spatio-Temporal Representations and Activities For Cognitive Control in Long-Term Scenarios
Xperience – Robots Bootstrapped through Learning from Experience
The increased use of Artificial Intelligence and Machine Learning in robotics will be highlighted in two keynote presentations. Raia Hadsell, Senior Research Scientist at DeepMind will focus on deep learning, and strategies to make robots that can continuously learn and improve over time. Stan Boland, CEO of FiveAI, will talk about his company’s aim to accelerate the arrival of fully autonomous vehicles.
Professor David Lane, ERF2017 General Chair and Director of the Edinburgh Centre for Robotics, said, “We’re delighted this year to have two invited keynotes of outstanding quality and relevance from the UK, representing both research and disruptive industrial application of robotics and artificial intelligence. EURobotics and its members are committed to the innovation that translates technology from research to new products and services. New industries are being created, with robotics providing the essential arms, legs and sensors that bring big data and artificial intelligence out of the laboratory and into the real world.”
Throughout ERF2017, emphasis will be given to the impact of robots on society and the economy. Keith Brown MSP, Cabinet Secretary for Economy, Jobs and Fair Work, will open the event, said, “The European Robotics Forum provides an opportunity for Scotland to showcase our world-leading research and expertise in robotics, artificial intelligence and human-robot interaction. This event will shine a light on some of the outstanding developments being pioneered and demonstrates Scotland’s vital role in this globally significant area.”
In discussing robots and society, Dr Patricia A. Vargas, ERF2017 General Chair and Director of the Robotics Laboratory at Heriot-Watt University, said, “As robots gradually move to our homes and workplace, we must make sure they are fully ethical. A potential morality code for robots should include human responsibilities, and take into account how humans can interact with robots in a safe way. The European Robotics Forum is the ideal place to drive these discussions.”
Ultimately, the forum aims to understand how robots can benefit small and medium-sized businesses, and how links between industry and academia can be improved to better exploit the strength of European robotics and AI research. As robots start leaving the lab to enter our home and work environments, it becomes increasingly important to understand how they will best work alongside human co-workers and users. Issues of policy, the law, and ethics will be debated during dedicated workshops.
Dr Katrin Lohan, General Chair and Deputy Director of the Robotics Laboratory at Heriot-Watt University said, “It is important how to integrate robotics into the workflow so that it support and not disrupt the human workers. The potential of natural interaction interfaces and non-verbal communication cues needs to be further explored. The synergies of robots and human workers could make all the difference for small and medium-sized businesses to discuss this the European Robotics Forum is the ideal place as it joins industry and academia community. ”
______________________
Confirmed keynote speakers include:
Keith Brown, Cabinet Secretary for the Economy, Jobs and Fair Work, Member of the Scottish Parliament
Raia Hadsell, Senior Research Scientist at DeepMind
Stan Boland, CEO of FiveAI
Press Passes:
Journalists may request free press badges, or support with interviews, by emailing publicity.chairs@erf2017.eu. Please see the website for additional information.
Organisers The European Robotics Forum is organised by euRobotics under SPARC, the Public-Private partnership for Robotics in Europe. This year’s conference is hosted by the Edinburgh Centre for Robotics.
About euRobotics and SPARC
euRobotics is a non-profit organisation based in Brussels with the objective to make robotics beneficial for Europe’s economy and society. With more than 250 member organisations, euRobotics also provides the European Robotics Community with a legal entity to engage in a public/private partnership with the European Commission, named SPARC.
SPARC, the public-private partnership (PPP) between the European Commission and euRobotics, is a European initiative to maintain and extend Europe’s leadership in civilian robotics. Its aim is to strategically position European robotics in the world thereby securing major benefits for the European economy and the society at large.
SPARC is the largest research and innovation programme in civilian robotics in the world, with 700 million euro in funding from the European Commission between 2014 to 2020, which is tripled by European industry to yield a total investment of 2.1 billion euro. SPARC will stimulate an ever more vibrant and effective robotics community that collaborates in the successful development of technical transfer and commercial exploitation.
The U.S. Army deployed a company of MQ-1C Gray Eagle drones to Kunsan Air Base in South Korea. Credit: Staff Sgt. Christopher Calvert/U.S. Army
March 13, 2017 – March 19, 2017
At the Center for the Study of the Drone
We spoke to Rolling Stone about the implications of recent advances in swarming drone technology for the future of warfare.
News
A U.S. airstrike in Syria involving U.S. MQ-9 Reaper drones may have resulted in the deaths of noncombatants. According to the U.K.-based Syrian Observatory for Human Rights, the strike, which reportedly hit a mosque in Jinah, killed at least 46 people. In a statement to reporters, a Pentagon spokesperson said that U.S. aircraft had not targeted the mosque, but rather al-Qaeda fighters at a community center nearby. (Washington Post)
The Wall Street Journal has reported that the Trump administration has given the CIA greater latitude to order drone strikes. If confirmed to be true, the policy shift would appear to reverse restrictions placed by the Obama administration on the intelligence agency’s role in strikes, and may reopen a disagreement with the Department of Defense over the CIA’s authority to carry out strike operations.
The U.S. Army deployed an MQ-1C Gray Eagle surveillance and strike drone unit to Kunsan Air Base in South Korea. The Gray Eagle company will be assigned to the 2nd Combat Aviation Brigade, 2nd Aviation Regiment. (AIN Online)
Canada announced new rules for recreational drone users, including a flight ceiling of 295 feet and a prohibition against flying near airports. Infractions could result in fines of over $2,000. In a statement, Transport Minister Marc Garneau said that the measures were aimed at preventing an accident involving a drone and a manned aircraft. (ABC News)
Commentary, Analysis, and Art
The U.S. Senate Committee on Commerce, Science, and Transportation held a hearing on integrating drones into the national airspace. (UAS Magazine)
At the New York Times, Rachel Nuwer takes a closer look at the benefits and challenges of using drones to fight poachers.
The New York Times Editorial Board argues that the Trump administration should not loosen the rules of engagement for strikes and counterterrorism operations in Yemen and Somalia.
At Lawfare, Robert Chesney considers the possible consequences of the Trump administration’s reported decision to allow the CIA to order drone strikes.
The Australian Transport Safety Bureau released a report in which it found that there was a 75 percent rise in the number of reported close encounters between drones and manned aircraft between 2012 and 2016. (PerthNow)
Drone manufacturer DJI released a paper in which it argues that drones have saved 59 lives over the past several years. (Drone360)
At Breaking Defense, Sydney J. Freedberg Jr. looks at how automation and robotics figure into the U.S. Army’s plans for its next generation battle tank.
U.K. firm Windhorse Aerospace revealed new details about its edible humanitarian drones, which will likely be made of compressed vegetable honeycomb and salami. (The Verge)
Online retail giant Amazon has been granted two patents for its proposed delivery drone system: an adjustable landing gear system and a propeller system with adjustable wingtips. (CNBC)
Meanwhile, Amazon displayed two of its Prime Air delivery drones at the South by Southwest festival in Texas, the first time the systems had been displayed publicly. (Fortune)
Drone maker QuadH2O unveiled the HexH2O Pro, a waterproof commercial drone. (Unmanned Systems Technology)
Defense firm BAE is once again displaying its Armed Robotic Combat Vehicle, a weaponized unmanned ground vehicle that it developed for the U.S. Army’s cancelled Future Combat Systems program. (Defense News)
China Daily reported that China Aerospace Science and Industry Corporation, a state-owned company, is developing drones capable of evading radar detection. (IHS Jane’s 360)
A software upgrade to the U.S. Navy’s Boeing P-8 maritime surveillance aircraft will enable it to work with unmanned systems. (Defense Systems)
Drones at Work
New Zealand firm Drone Technologies conducted the country’s first beyond-line-of-sight flight of a drone to inspect transmission lines and towers in the Rimutaka Ranges. (Stuff)
The Cecil County Sheriff’s Office in Maryland used a drone to discover a trove of stolen heavy machinery. (ABC2 News)
A Skylark 1 drone operated by the Israel Defense Forces crashed during a flight in Gaza. (Jerusalem Post)
The Defense Advanced Research Projects Agency awarded Dynetics and General Atomics Aeronautical Systems phase two contracts for the Gremlins low-cost, reusable drone program. (Shephard Media)
The U.S. Air Force will reportedly award General Atomics Aeronautical Systems contracts for upgrading the MQ-9 Reaper Block 5 systems to an extended range configuration. (IHS Jane’s 360)
The National Oceanic and Atmospheric Administration awarded Aerial Imaging Solutions a $61,850 contract for three hexacopter drone systems. (FBO)
The U.S. Geological Survey awarded Rock House Products International a $13,011 contract for a thermal imaging system for an unmanned aircraft. (FBO)
The U.S. Navy awarded Northrop Grumman Systems a $3.6 million contract for the installation and flight testing of the Selex ES Osprey 30 RADAR for the MQ-8C Fire Scout drone. (FBO)
The U.S. Navy announced that it will award Boeing Insitu a $112,842 foreign military sales contract for spare parts for the ScanEagle drone for Kenya. (FBO)
For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.
Japan is holding a huge robot celebration in 2018 in Tokyo and 2020 in Aichi, Fukushima, hosted by the Ministry of Economy, Trade and industry (METI) and the New Energy Industrial Technology Development Organization (NEDO). This is a commercial robotics Expo and a series of robotics Challenges with the goal of bringing together experts from around the world to advance human focused robotics.
The World Robot Summit website was just launched on March 2, 2017. The results of tenders for standard robot platforms for the competitions are being announced soon and the first trials for competition teams should happen in summer 2017.
There are a total of 8 challenges that fall into 4 categories: Industrial Robotics, Service Robotics, Disaster Robotics and Junior.
Industrial: Assembly Challenge – quick and accurate assembly of model products containing technical components require in assembling industrial products and other goods.
Service: Partner Robot Challenge – setting tasks equivalent to housework and making robots that complete such tasks – utilizing a standard robot platform.
Service: Automation of Retail Work Challenge – making robots to complete tasks eg. shelf stocking and replenishment multiple types of products such as foods, interaction between customers and staffs and cleaning restrooms.
Disaster: Plant Disaster Prevention Challenge – inspecting or maintaining infrastructures based on set standards eg. opening/closing valves and exchanging consumable supplies and searching for disaster victims.
Disaster: Tunnel Disaster Response and Recovery Challenge – collecting information and providing emergency response in case of a tunnel disaster eg. saving lives and removing vehicles from tunnels.
Disaster: Standard Disaster Robotics Challenge – assessing standard performance levels eg. mobility, sensing, information collection, wireless communication, remote control on-site deployment and durability, etc. require in disaster prevention and response.
Junior (aged 19 or younger): School Robot Challenge – making robots to complete tasks that might be useful in a school environment – utilizing a standard robot platform.
Junior (aged 19 or younger): Home Robot Challenge – setting tasks equivalent to housework and making robots that complete such tasks.
The World Robot Summit, Challenge, Expo and Symposiums are looking for potential teams and major sponsors.
Researchers including MIT professor Daniela Rus (left) and research scientist Stephanie Gil (right) have developed a technique for preventing malicious hackers from commandeering robot teams’ communication networks. To verify the theoretical predictions, the researchers implemented their system using a battery of distributed Wi-Fi transmitters and an autonomous helicopter. Image: M. Scott Brauer.
Distributed planning, communication, and control algorithms for autonomous robots make up a majorarea of research in computer science. But in the literature on multirobot systems, security has gotten relatively short shrift.
In the latest issue of the journal Autonomous Robots, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory and their colleagues present a new technique for preventing malicious hackers from commandeering robot teams’ communication networks. The technique could provide an added layer of security in systems that encrypt communications, or an alternative in circumstances in which encryption is impractical.
“The robotics community has focused on making multirobot systems autonomous and increasingly more capable by developing the science of autonomy. In some sense we have not done enough about systems-level issues like cybersecurity and privacy,” says Daniela Rus, an Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT and senior author on the new paper.
“But when we deploy multirobot systems in real applications, we expose them to all the issues that current computer systems are exposed to,” she adds. “If you take over a computer system, you can make it release private data — and you can do a lot of other bad things. A cybersecurity attack on a robot has all the perils of attacks on computer systems, plus the robot could be controlled to take potentially damaging action in the physical world. So in some sense there is even more urgency that we think about this problem.”
Identity theft
Most planning algorithms in multirobot systems rely on some kind of voting procedure to determine a course of action. Each robot makes a recommendation based on its own limited, local observations, and the recommendations are aggregated to yield a final decision.
A natural way for a hacker to infiltrate a multirobot system would be to impersonate a large number of robots on the network and cast enough spurious votes to tip the collective decision, a technique called “spoofing.” The researchers’ new system analyzes the distinctive ways in which robots’ wireless transmissions interact with the environment, to assign each of them its own radio “fingerprint.” If the system identifies multiple votes as coming from the same transmitter, it can discount them as probably fraudulent.
“There are two ways to think of it,” says Stephanie Gil, a research scientist in Rus’ Distributed Robotics Lab and a co-author on the new paper. “In some cases cryptography is too difficult to implement in a decentralized form. Perhaps you just don’t have that central key authority that you can secure, and you have agents continually entering or exiting the network, so that a key-passing scheme becomes much more challenging to implement. In that case, we can still provide protection.
“And in case you can implement a cryptographic scheme, then if one of the agents with the key gets compromised, we can still provide protection by mitigating and even quantifying the maximum amount of damage that can be done by the adversary.”
Hold your ground
In their paper, the researchers consider a problem known as “coverage,” in which robots position themselves to distribute some service across a geographic area — communication links, monitoring, or the like. In this case, each robot’s “vote” is simply its report of its position, which the other robots use to determine their own.
The paper includes a theoretical analysis that compares the results of a common coverage algorithm under normal circumstances and the results produced when the new system is actively thwarting a spoofing attack. Even when 75 percent of the robots in the system have been infiltrated by such an attack, the robots’ positions are within 3 centimeters of what they should be. To verify the theoretical predictions, the researchers also implemented their system using a battery of distributed Wi-Fi transmitters and an autonomous helicopter.
“This generalizes naturally to other types of algorithms beyond coverage,” Rus says.
The new system grew out of an earlier project involving Rus, Gil, Dina Katabi — who is the other Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT — and Swarun Kumar, who earned master’s and doctoral degrees at MIT before moving to Carnegie Mellon University. That project sought to use Wi-Fi signals to determine transmitters’ locations and to repairad hoc communication networks. On the new paper, the same quartet of researchers is joined by MIT Lincoln Laboratory’s Mark Mazumder.
Typically, radio-based location determination requires an array of receiving antennas. A radio signal traveling through the air reaches each of the antennas at a slightly different time, a difference that shows up in the phase of the received signals, or the alignment of the crests and troughs of their electromagnetic waves. From this phase information, it’s possible to determine the direction from which the signal arrived.
Space vs. time
A bank of antennas, however, is too bulky for an autonomous helicopter to ferry around. The MIT researchers found a way to make accurate location measurements using only two antennas, spaced about 8 inches apart. Those antennas must move through space in order to simulate measurements from multiple antennas. That’s a requirement that autonomous robots meet easily. In the experiments reported in the new paper, for instance, the autonomous helicopter hovered in place and rotated around its axis in order to make its measurements.
When a Wi-Fi transmitter broadcasts a signal, some of it travels in a direct path toward the receiver, but much of it bounces off of obstacles in the environment, arriving at the receiver from different directions. For location determination, that’s a problem, but for radio fingerprinting, it’s an advantage: The different energies of signals arriving from different directions give each transmitter a distinctive profile.
There’s still some room for error in the receiver’s measurements, however, so the researchers’ new system doesn’t completely ignore probably fraudulent transmissions. Instead, it discounts them in proportion to its certainty that they have the same source. The new paper’s theoretical analysis shows that, for a range of reasonable assumptions about measurement ambiguities, the system will thwart spoofing attacks without unduly punishing valid transmissions that happen to have similar fingerprints.
“The work has important implications, as many systems of this type are on the horizon — networked autonomous driving cars, Amazon delivery drones, et cetera,” says David Hsu, a professor of computer science at the National University of Singapore. “Security would be a major issue for such systems, even more so than today’s networked computers. This solution is creative and departs completely from traditional defense mechanisms.”
If you enjoyed this article from CSAIL, you might also be interested in:
Norway has established a test site at Trondheim Fjord for unmanned and autonomous vessels like these concept container ships of the future. Credit: Kongsberg Seatex
March 6, 2017 – March 12, 2017
News
Germany reportedly intends to acquire the Northrop Grumman MQ-4C Triton high-altitude surveillance drone, according to a story in Sueddeutsche Zeitung. In 2013, Germany cancelled a similar program to acquire Northrop Grumman’s RQ-4 Global Hawk, a surveillance drone on which the newer Triton is based, due to cost overruns. The Triton is a large, long-endurance system that was originally developed for maritime surveillance by the U.S. Navy. (Reuters)
The U.S. Army released a report outlining its strategy for obtaining and using unmanned ground vehicles. The Robotics and Autonomous Systems strategy outlines short, medium, and long-term goals for the service’s ground robot programs. The Army expects a range of advanced unmanned combat vehicles to be fielded in the 2020 to 2030 timeframe. (IHS Jane’s 360)
The U.S. Air Force announced that there are officially more jobs available for MQ-1 Predator and MQ-9 Reaper pilots than for any manned aircraft pilot position. Following a number of surges in drone operations, the service had previously struggled to recruit and retain drone pilots. The Air Force is on track to have more than 1,000 Predator and Reaper pilots operating its fleet. (Military.com)
At FlightGlobal, Dominic Perry writes that France’s Dassault is not concerned that the U.K. decision to leave the E.U. will affect a plan to develop a combat drone with BAE Systems.
At the Los Angeles Times, Bryce Alderton looks at how cities in California are addressing the influx of drones with new regulations.
At CBS News, Larry Light looks at how Bill Gates has reignited a debate over taxes on companies that use robots.
In an interview with the Wall Street Journal, Andrew Ng and Neil Jacobstein argue that artificial intelligence will bring about significant changes to commerce and society in the next 10 to 15 years.
In testimony before the House Armed Services Committee’s subcommittee on seapower, panelists urged the U.S. Navy to develop and field unmanned boats and railguns. (USNI News)
At DefenseTech.org, Richard Sisk looks at how a U.S.-made vehicle-mounted signals “jammer” is helping Iraqi forces prevent ISIS drone attacks in Mosul.
In a Drone Radio Show podcast, Steven Flynn discusses why prioritizing drone operators who comply with federal regulations is important for the drone industry.
At ABC News, Andrew Greene examines how a push by the Australian military to acquire armed drones has reignited a debate over targeted killings.
At Smithsonian Air & Space, Tim Wright profiles the NASA High Altitude Shuttle System, a glider drone that is being used to test communications equipment for future space vehicles.
Researchers at Virginia Tech are flying drones into crash-test dummies to evaluate the potential harm that a drone could cause if it hits a human. (Bloomberg)
Meanwhile, researchers at École Polytechnique Fédérale de Lausanne are developing flexible multi-rotor drones that absorb the impact of a collision without breaking. (Gizmodo)
Recent satellite images of Russia’s Gromov Flight Research Institute appear to show the country’s new Orion, a medium-altitude long-endurance military drone. (iHLS)
The Fire Department of New York used its tethered multi-rotor drone for the first time during an apartment fire in the Bronx. (Crain’s New York)
The Michigan State Police Bomb Squad used an unmanned ground vehicle to inspect the interior of two homes that were damaged by a large sinkhole. (WXYZ)
A video posted to YouTube appears to show a woman in Washington State firing a gun at a drone that was flying over her property. (Huffington Post)
Meanwhile, a bill being debated in the Oklahoma State Legislature would remove civil liability for anybody who shoots a drone down over their private property. (Ars Technica)
An Arizona man who leads an anti-immigration vigilante group is using a drone to patrol the U.S border with Mexico in search of undocumented crossings. (Voice of America)
A man who attempted to use a drone to smuggle drugs into a Scottish prison has been sentenced to five years in prison. (BBC)
Industry Intel
The Turkish military has taken a delivery of six Bayraktar TB-2 military drones, two of which are armed, for air campaigns against ISIL and Turkish forces. (Defense News)
General Atomics Aeronautical Systems awarded Hughes Network Systems a contract for satellite communications for the U.K.’s Predator B drones. (Space News)
Schiebel awarded CarteNav Solutions a contact for its AIMS-ISR software for the S-100 Camcopter unmanned helicopters destined for the Royal Australian Navy. (Press Release)
Defence Research and Development Canada awarded Ontario Drive & Gear a $1 million contract for trials of the Atlas J8 unmanned ground vehicle. (Canadian Manufacturing)
Deveron UAS will provide Thompsons, a subsidiary of Lansing Trade Group and The Andersons, with drone data for agricultural production through 2018. (Press Release)
Precision Vectors Aerial selected the Silent Falcon UAS for its beyond visual line-of-sight operations in Canada. (Shephard Media)
Rolls-Royce won a grant from Tekes, a Finnish government research funding agency, to continue developing remote and autonomous shipping technologies. (Shephard Media)
Israeli drone manufacturer BlueBird is submitting an updated MicroB UAV system for the Indian army small UAV competition. (FlightGlobal)
A Romanian court has suspended a planned acquisition of Aeronautics Defense Systems Orbiter 4 drones for the Romanian army. (FlightGlobal)
Deere & Co.—a.k.a. John Deere—announced that it will partner with Kespry, a drone startup, to market drones for the construction and forestry industries. (TechCrunch)
For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.
The National Science Foundation (NSF) announced a $6.1 million, five-year award to accelerate fundamental research on wireless communication and networking technologies through the foundation’s Platforms for Advanced Wireless Research (PAWR) program.
Through the PAWR Project Office (PPO), award recipients US Ignite, Inc. and Northeastern University will collaborate with NSF and industry partners to establish and oversee multiple city-scale testing platforms across the United States. The PPO will manage nearly $100 million in public and private investments over the next seven years.
“NSF is pleased to have the combined expertise from US Ignite, Inc. and Northeastern University leading the project office for our PAWR program,” said Jim Kurose, NSF assistant director for Computer and Information Science and Engineering. “The planned research platforms will provide an unprecedented opportunity to enable research in faster, smarter, more responsive, and more robust wireless communication, and move experimental research beyond the lab — with profound implications for science and society.”
Over the last decade, the use of wireless, internet-connected devices in the United States has nearly doubled. As the momentum of this exponential growth continues, the need for increased capacity to accommodate the corresponding internet traffic also grows. This surge in devices, including smartphones, connected tablets and wearable technology, places an unprecedented burden on conventional 4G LTE and public Wi-Fi networks, which may not be able to keep pace with the growing demand.
NSF established the PAWR program to foster use-inspired, fundamental research and development that will move beyond current 4G LTE and Wi-Fi capabilities and enable future advanced wireless networks. Through experimental research platforms that are at the scale of small cities and communities and designed by the U.S. academic and industry wireless research community, PAWR will explore robust new wireless devices, communication techniques, networks, systems and services that will revolutionize the nation’s wireless systems. These platforms aim to support fundamental research that will enhance broadband connectivity and sustain U.S. leadership and economic competitiveness in the telecommunications sector for many years to come.
“Leading the PAWR Project Office is a key component of US Ignite’s mission to help build the networking foundation for smart communities,” said William Wallace, executive director of US Ignite, Inc., a public-private partnership that aims to support ultra-high-speed, next-generation applications for public benefit. “This effort will help develop the advanced wireless networks needed to enable smart and connected communities to transform city services.”
Establishing the PPO with this initial award is the first step in launching a long-term, public-private partnership to support PAWR. Over the next seven years, PAWR will take shape through two multi-stage phases:
Design and Development. The PPO will assume responsibility for soliciting and vetting proposals to identify the platforms for advanced wireless research and work closely with sub-awardee organizations to plan the design, development, deployment and initial operations of each platform.
Deployment and Initial Operations. The PPO will establish and manage each platform and document best practices as it progresses through the lifecycle.
“We are delighted that our team of wireless networking researchers has been selected to take the lead of the PAWR Project Office in partnership with US Ignite, Inc.,” said Dr. Nadine Aubry, dean of the college of engineering and university distinguished professor at Northeastern University. “I believe that PAWR, by bringing together academia, industry, government and communities, has the potential to make a transformative impact through advances spanning fundamental research and field platforms in actual cities.”
The PPO will work closely with NSF, industry partners and the wireless research community in all aspects of PAWR planning, implementation and management. Over the next seven years, NSF anticipates investing $50 million in PAWR, combined with approximately $50 million in cash and in-kind contributions from over 25 companies and industry associations. The PPO will disperse these investments to support the selected platforms.
Additional information can be found on the PPO webpage.
This announcement will also be highlighted this week during the panel discussion, “Wireless Network Innovation: Smart City Foundation,” at the South by Southwest conference in Austin, Texas.
Yesterday, the UK government announced their budget plans to invest in robotics, artificial intelligence, driverless cars, and faster broadband. The spending commitments include:
£16m to create a 5G hub to trial the forthcoming mobile data technology. In particular, the government wants there to better mobile network coverage over the country’s roads and railway lines
£200m to support local “full-fibre” broadband network projects that are designed to bring in further private sector investment
£270m towards disruptive technologies to put the UK “at the forefront” including cutting-edge artificial intelligence and robotics systems that will operate in extreme and hazardous environments, including off-shore energy, nuclear energy, space and deep mining; batteries for the next generation of electric vehicles; and biotech.
Investing £300 million to further develop the UK’s research talent, including through creating an additional 1,000 PhD places.
Several experts in the robotics community agree that progress is shifting in the right direction, however, more needs to happen if the UK is to remain competitive in the robotics sector:
“The UK understand the very real positive impact that RAS [robotics & autonomous systems] will have on our society from now, of all time. It continues to see the big picture and today’s announcement by the Chancellor is a clear indication of that. We can have better roads, cleaner cities, healthier oceans and bodies, safer skies, deeper mines, better jobs and more opportunity. That’s what machines are for.”
“We are at a real inflection point in the development of autonomous technology. The UK has a number of nascent world class companies in the area of self-driving vehicles, which have a huge potential to change the world, whilst creating jobs and producing exportable UK goods and services. We have a head start and now we need to take advantage of it.” [from FT]
“Some of the great robotics companies of the future are being launched by British entrepreneurs and the support announced in today’s budget will to strengthen their impact and global competitiveness. We’re currently seeing strong appetite from private investors to back locally-grown robotics businesses and this money will help bring even more interest in this space”
“This is welcome news for the many research organisations developing robotics applications. As a leading UK robotics research group specialising in extreme and challenging environments, we welcome the allocation of significant funding in this field as part of the Government’s evolving Industrial Strategy. RACE and the rest of the robotics R&D sector are looking forward to working with industry to fully utilise this funding.”
“Robotics and AI is set to be a driving force in increasing productivity, but also in solving societal and environmental challenges. It’s opening new frontiers in off-shore and nuclear energy, space and deep mining. Investment from government will be key in helping the UK stay at the forefront of this field.” [from BBC]
“We lost our best machine learning group to Amazon just recently. The money means there will be more resources for universities, which may help them retain their staff. But it’s not nearly enough for all of the disruptive technologies being developed in the UK. The government says it want this to be the leading robotics country in the world, but Google and others are spending far more, so it’s ultimately chicken feed by comparison.” [from BBC]
“I’m pleased by the additional funding, and, in fact, my group is a partner in a new £4.6M EPSRC grant to develop robots for nuclear decommissioning announced last week.
But having just returned from Tokyo (from AI in Asia: AI for Social Good), I’m well aware that other countries are investing much more heavily than the UK. China was for instance described as an emerging powerhouse of AI. A number of colleagues at that meeting also made the same point as Noel, that universities are haemorrhaging star AI/robotics academics to multi-national companies with very deep pockets.”
“I, like many others, was pleased to hear more money going into robotics and AI research, but I was disappointed – though completely unsurprised – to see nothing about how to restructure the economy to deal with the consequences of increasing research into and use of robots and AI. Hammond’s blunder on the relationship of productivity to wages – and it can’t be seen as anything other than a blunder – means that he doesn’t even seem to appreciate that there is a problem.
The truth is that increased automation means fewer jobs and lower wages and this needs to be addressed with some concrete measures. There will be benefits to society with increased automation, but we need to start thinking now (and taking action now) to ensure that those benefits aren’t solely economic gain for the already-wealthy. The ‘robot dividend’ needs to be shared across society, as it can have far-reaching consequences beyond economics: improving our quality of life, our standard of living, education, health and accessibility.”
“America has the American Manufacturing Initiative which, in 2015, was expanded to establish Fraunhofer-like research facilities around the US (on university campuses) that focus on particular aspects of the science of manufacturing.
Robotics were given $50 million of the $500 million for the initiative and one of the research facilities was to focus on robotics. Under the initiative, efforts from the SBIR, NSF, NASA and DoD/DARPA were to be coordinated in their disbursement of fundings for science in robotics. None of these fundings comes anywhere close to the coordinated funding programs and P-P-Ps found in the EU, Korea and Japan, nor the top-down incentivized directives of China’s 5-year plans. Essentially American robotic funding is (and has been) predominantly entrepreneurial with token support from the government.
In the new Trump Administration, there is no indication of any direction nor continuation (funding) of what little existing programs we have. At a NY Times editorial board sit-down with Trump after his election, he was quoted as saying that “Robotics is becoming very big and we’re going to do that. We’re going to have more factories. We can’t lose 70,000 factories. Just can’t do it. We’re going to start making things.” Thus far there is no followup to those statements nor has Trump hired replacements for the top executives at the Office of Science and Technology Policy, all of which are presently vacant.”
And finally, a few comments from the business sector on Twitter:
International Women’s Day is raising discussion about the lack of diversity and role models in STEM and the potential negative outcomes of bias and stereotyping in robotics and AI. Let’s balance the words with positive actions. Here’s what we can all do to support women in robotics and AI, and thus improve diversity, innovation and reduce skills shortages for robotics and AI.
Join WomeninRobotics.org – a network of women working in robotics (or who aspire to work in robotics). We are a global discussion group supporting local events that bring women together for peer networking. We recognize that lack of support and mentorship in the workplace holds women back, particularly if there is only one woman in an organization/company.
Although the main group is only for women, we are going to start something for male ‘Allies’ or ‘Champions’. So men, you can join women in robotics too! Women need champions and while it would be ideal to have an equal number of women in leadership roles, until then, companies can improve their hiring and retention by having visible and vocal male allies. We all need mentors as our careers progress.
Women also need visibility and high profile projects for their careers to progress on par. One way of improving that is to showcase the achievements of women in robotics. Read and share all four year’s worth of our annual “25 Women in Robotics you need to know about” – that’s more than 100 women already because we have some groups in there. (There has always been a lot of women on the core team at Robohub.org, so we love showing our support.) Our next edition will come out on October 10 2017 to celebrate Ada Lovelace Day.
Change starts at the top of an organization. It’s very hard to hire women if you don’t have any women, or if they can’t see pathways for advancement in your organization. However, there are many things you can do to improve your hiring practices. Some are surprisingly simple, yet effective. I’ve collected a list and posted it at Silicon Valley Robotics – How to hire women.
And you can invest in women entrepreneurs. All the studies show that you get a higher rate of return, and higher likelihood of success from investments in female founders. And yet, proportionately investment is much less. You don’t need to be a VC to invest in women either. Kiva.org is matching loans today and $25 can empower an entrepreneur all over the world. #InvestInHer
And our next Silicon Valley/ San Francisco Women in Robotics event will be on March 22 at SoftBank Robotics – we’d love to see you there – or in support!
Guest post by José Hernández-Orallo, Professor at Technical University of Valencia
Two decades ago I started working on metrics of machine intelligence. By that time, during the glacial days of the second AI winter, few were really interested in measuring something that AI lacked completely. And very few, such as David L. Dowe and I, were interested in metrics of intelligence linked to algorithmic information theory, where the models of interaction between an agent and the world were sequences of bits, and intelligence was formulated using Solomonoff’s and Wallace’s theories of inductive inference.
In the meantime, seemingly dozens of variants of the Turing test were proposed every year, the CAPTCHAs were introduced and David showed how easy it is to solve some IQ tests using a very simple program based on a big-switch approach. And, today, a new AI spring has arrived, triggered by a blossoming machine learning field, bringing a more experimental approach to AI with an increasing number of AI benchmarks and competitions (see a previous entry in this blog for a survey).
Last year also witnessed the introduction of a different kind of AI evaluation platforms, such as Microsoft’s Malmö, GoodAI’s School, OpenAI’s Gym and Universe, DeepMind’s Lab, Facebook’s TorchCraft and CommAI-env. Based on a reinforcement learning (RL) setting, these platforms make it possible to create many different tasks and connect RL agents through a standard interface. Many of these platforms are well suited for the new paradigms in AI, such as deep reinforcement learning and some open-source machine learning libraries. After thousands of episodes or millions of steps against a new task, these systems are able to excel, with usually better than human performance.
Despite the myriads of applications and breakthroughs that have been derived from this paradigm, there seems to be a consensus in the field that the main open problem lies in how an AI agent can reuse the representations and skills from one task to new ones, making it possible to learn a new task much faster, with a few examples, as humans do. This can be seen as a mapping problem (usually under the term transfer learning) or can be seen as a sequential problem (usually under the terms gradual, cumulative, incremental, continual or curriculum learning).
One of the key notions that is associated with this capability of a system of building new concepts and skills over previous ones is usually referred to as “compositionality”, which is well documented in humans from early childhood. Systems are able to combine the representations, concepts or skills that have been learned previously in order to solve a new problem. For instance, an agent can combine the ability of climbing up a ladder with its use as a possible way out of a room, or an agent can learn multiplication after learning addition.
In my opinion, two of the previous platforms are better suited for compositionality: Malmö and CommAI-env. Malmö has all the ingredients of a 3D game, and AI researchers can experiment and evaluate agents with vision and 3D navigation, which is what many research papers using Malmö have done so far, as this is a hot topic in AI at the moment. However, to me, the most interesting feature of Malmö is building and crafting, where agents must necessarily combine previous concepts and skills in order to create more complex things.
CommAI-env is clearly an outlier in this set of platforms. It is not a video game in 2D or 3D. Video or audio don’t have any role there. Interaction is just produced through a stream of input/output bits and rewards, which are just +1, 0 or -1. Basically, actions and observations are binary. The rationale behind CommAI-env is to give prominence to communication skills, but it still allows for rich interaction, patterns and tasks, while “keeping all further complexities to a minimum”.
Examples of interaction within the CommAI-mini environment.
When I was aware that the General AI Challenge was using CommAI-env for their warm-up round I was ecstatic. Participants could focus on RL agents without the complexities of vision and navigation. Of course, vision and navigation are very important for AI applications, but they create many extra complications if we want to understand (and evaluate) gradual learning. For instance, two equal tasks for which the texture of the walls changes can be seen as requiring higher transfer effort than two slightly different tasks with the same texture. In other words, this would be extra confounding factors that would make the analysis of task transfer and task dependencies much harder. It is then a wise choice to exclude this from the warm-up round. There will be occasions during other rounds of the challenge for including vision, navigation and other sorts of complex embodiment. Starting with a minimal interface to evaluate whether the agents are able to learn incrementally is not only a challenging but an important open problem for general AI.
Also, the warm-up round has modified CommAI-env in such a way that bits are packed into 8-bit (1 byte) characters. This makes the definition of tasks more intuitive and makes the ASCII coding transparent to the agents. Basically, the set of actions and observations is extended to 256. But interestingly, the set of observations and actions is the same, which allows many possibilities that are unusual in reinforcement learning, where these subsets are different. For instance, an agent with primitives such as “copy input to output” and other sequence transformation operators can compose them in order to solve the task. Variables, and other kinds of abstractions, play a key role.
This might give the impression that we are back to Turing machines and symbolic AI. In a way, this is the case, and much in alignment to Turing’s vision in his 1950 paper: “it is possible to teach a machine by punishments and rewards to obey orders given in some language, e.g., a symbolic language”. But in 2017 we have a range of techniques that weren’t available just a few years ago. For instance, Neural Turing Machines and other neural networks with symbolic memory can be very well suited for this problem.
By no means does this indicate that the legion of deep reinforcement learning enthusiasts cannot bring their apparatus to this warm-up round. Indeed they won’t be disappointed by this challenge if they really work hard to adapt deep learning to this problem. They won’t probably need a convolutional network tuned for visual pattern recognition, but there are many possibilities and challenges in how to make deep learning work in a setting like this, especially because the fewer examples, the better, and deep learning usually requires many examples.
As a plus, the simple, symbolic sequential interface opens the challenge to many other areas in AI, not only recurrent neural networks but techniques from natural language processing, evolutionary computation, compression-inspired algorithms or even areas such as inductive programming, with powerful string-handling primitives and its appropriateness for problems with very few examples.
I think that all of the above makes this warm-up round a unique competition. Of course, since we haven’t had anything similar in the past, we might have some surprises. It might happen that an unexpected (or even naïve) technique could behave much better than others (and humans) or perhaps we find that no technique is able to do something meaningful at this time.
I’m eager to see how this round develops and what the participants are able to integrate and invent in order to solve the sequence of micro and mini-tasks. I’m sure that we will learn a lot from this. I hope that machines will, too. And all of us will move forward to the next round!
Guest post by Simon Andersson, Senior Research Scientist @GoodAI
Executive summary
Tracking major unsolved problems in AI can keep us honest about what remains to be achieved and facilitate the creation of roadmaps towards general artificial intelligence.
This document currently identifies 29 open problems.
For each major problem, example tests are suggested for evaluating research progress.
Introduction
This document identifies open problems in AI. It seeks to provide a concise overview of the greatest challenges in the field and of the current state of the art, in line with the “open research questions” theme of focus of the AI Roadmap Institute.
The challenges are grouped into AI-complete problems, closed-domain problems, and fundamental problems in commonsense reasoning, learning, and sensorimotor ability.
I realize that this first attempt at surveying the open problems will necessarily be incomplete and welcome reader feedback.
To help accelerate the search for general artificial intelligence, GoodAI is organizing the General AI Challenge (GoodAI, 2017), that aims to solve some of the problems outlined below, through a series of milestone challenges starting in early 2017.
Sources, method, and related work
The collection of problems presented here is the result of a review of the literature in the areas of
Machine learning
Machine perception and robotics
Open AI problems
Evaluation of AI systems
Tests for the achievement of human-level intelligence
Benchmarks and competitions
To be considered for inclusion, a problem must be
Highly relevant for achieving general artificial intelligence
Closed in scope, not subject to open-ended extension
Testable
Problems vary in scope and often overlap. Some may be contained entirely in others. The second criterion (closed scope) excludes some interesting problems such as learning all human professions; a few problems of this type are mentioned separately from the main list. To ensure that problems are testable, each is presented together with example tests.
Several websites, some listed below, provide challenge problems for AI.
OpenAI Requests for research (OpenAI, 2016) presents machine learning problems of varying difficulty with an emphasis on deep and reinforcement learning.
In the context of evaluating AI systems, Hernández-Orallo (2016a) reviews a number of open AI problems. Lake et al. (2016) offers a critique of the current state of the art in AI and discusses problems like intuitive physics, intuitive psychology, and learning from few examples.
The rest of the document lists AI challenges as outlined below.
AI-complete problems
Closed-domain problems
Commonsense reasoning
Learning
Sensorimotor problems
AI-complete problems
AI-complete problems are ones likely to contain all or most of human-level general artificial intelligence. A few problems in this category are listed below.
Open-domain dialog
Text understanding
Machine translation
Human intelligence and aptitude tests
Coreference resolution (Winograd schemas)
Compound word understanding
Open-domain dialog
Open-domain dialog is the problem of conducting competently a dialog with a human when the subject of the discussion is not known in advance. The challenge includes language understanding, dialog pragmatics, and understanding the world. Versions of the tasks include spoken and written dialog. The task can be extended to include multimodal interaction (e.g., gestural input, multimedia output). Possible success criteria are usefulness and the ability to conduct dialog indistinguishable from human dialog (“Turing test”).
Tests
Dialog systems are typically evaluated by human judges. Events where this has been done include
Text understanding is an unsolved problem. There has been remarkable progress in the area of question answering, but current systems still fail when common-sense world knowledge, beyond that provided in the text, is required.
Tests
McCarthy (1976) provided an early text understanding challenge problem.
Brachman (2006) suggested the problem of reading a textbook and solving its exercises.
Machine translation
Machine translation is AI-complete since it includes problems requiring an understanding of the world (e.g., coreference resolution, discussed below).
Tests
While translation quality can be evaluated automatically using parallel corpora, the ultimate test is human judgement of quality. Corpora such as the Corpus of Contemporary American English (Davies, 2008) contain samples of text from different genres. Translation quality can be evaluated using samples of
Newspaper text
Fiction
Spoken language transcriptions
Intelligence tests
Human intelligence and aptitude tests (Hernández-Orallo, 2017) are interesting in that they are designed to be at the limit of human ability and to be hard or impossible to solve using memorized knowledge. Human-level performance has been reported for Raven’s progressive matrices (Lovett and Forbus, 2017) but artificial systems still lack the general reasoning abilities to deal with a variety of problems at the same time (Hernández-Orallo, 2016b).
Tests
Brachman (2006) suggested using the SAT as an AI challenge problem.
In many languages, there are compound words with set meanings. Novel compound words can be produced, and we are good at guessing their meaning. We understand that a water bird is a bird that lives near water, not a bird that contains or is constituted by water, and that schadenfreude is felt when others, not we, are hurt.
Closed-domain problems are ones that combine important elements of intelligence but reduce the difficulty by limiting themselves to a circumscribed knowledge domain. Game playing agents are examples of this and artificial agents have achieved superhuman performance at Go (Silver et al., 2016) and more recently poker (Aupperlee, 2017; Brown and Sandholm, 2017). Among the open problems are:
Learning to play board, card, and tile games from descriptions
Producing programs from descriptions
Source code understanding
Board, card, and tile games from descriptions
Unlike specialized game players, systems that have to learn new games from descriptions of the rules cannot rely on predesigned algorithms for specific games.
Tests
The problem of learning new games from formal-language descriptions has appeared as a challenge at the AAAI conference (Genesereth et al., 2005; AAAI, 2013).
Even more challenging is the problem of learning games from natural language descriptions; such descriptions for card and tile games are available from a number of websites (e.g., McLeod, 2017).
Programs from descriptions
Producing programs in a programming language such as C from natural language input is a problem of obvious practical interest.
Tests
The “Description2Code” challenge proposed at (OpenAI, 2016) has 5000 descriptions for programs collected by Ethan Caballero.
Source code understanding
Related to source code production is source code understanding, where the system can interpret the semantics of code and detect situations where the code differs in non-trivial ways from the likely intention of its author. Allamanis et al. (2016) reports progress on the prediction of procedure names.
Tests
The International Obfuscated C Code Contest (OCCC, 2016) publishes code that is intentionally hard to understand. Source code understanding could be tested as the ability to improve the readability of the code as scored by human judges.
Commonsense reasoning
Commonsense reasoning is likely to be a central element of general artificial intelligence. Some of the main problems in this area are listed below.
Causal reasoning
Counterfactual reasoning
Intuitive physics
Intuitive psychology
Causal reasoning
Causal reasoning requires recognizing and applying cause-effect relations.
Counterfactual reasoning is required for answering hypothetical questions. It uses causal reasoning together with the system’s other modeling and reasoning capabilities to consider situations possibly different from anything that ever happened in the world.
Despite remarkable advances in machine learning, important learning-related problems remain mostly unsolved. They include:
Gradual learning
Unsupervised learning
Strong generalization
Category learning from few examples
Learning to learn
Compositional learning
Learning without forgetting
Transfer learning
Knowing when you don’t know
Learning through action
Gradual learning
Humans are capable of lifelong learning of increasingly complex tasks. Artificial agents should be, too. Versions of this idea have been discussed under the rubrics of life-long (Thrun and Mitchell, 1995), continual, and incremental learning. At GoodAI, we have adopted the term gradual learning (Rosa et al., 2016) for the long-term accumulation of knowledge and skills. It requires the combination of several abilities discussed below:
Compositional learning
Learning to learn
Learning without forgetting
Transfer learning
Tests
A possible test applies to a household robot that learns household and house maintenance tasks, including obtaining tools and materials for the work. The test evaluates the agent on two criteria: Continuous operation (Nilsson in Brooks, et al., 1996) where the agent needs to function autonomously without reprogramming during its lifetime, and improving capability, where the agent must exhibit, at different points in its evolution, capabilities not present at an earlier time.
Unsupervised learning
Unsupervised learning has been described as the next big challenge in machine learning (LeCun 2016). It appears to be fundamental to human lifelong learning (supervised and reinforcement signals do not provide nearly enough data) and is closely related to prediction and common-sense reasoning (“filling in the missing parts”). A hard problem (Yoshua Bengio, in the “Brains and bits” panel at NIPS 2016) is unsupervised learning in hierarchical systems, with components learning jointly.
Tests
In addition to the possible tests in the vision domain, speech recognition also presents opportunities for unsupervised learning. While current state-of-the-art speech recognizers rely largely on supervised learning on large corpora, unsupervised recognition requires discovering, without supervision, phonemes, word segmentation, and vocabulary. Progress has been reported in this direction, so far limited to small-vocabulary recognition (Riccardi and Hakkani-Tur, 2003, Park and Glass, 2008, Kamper et al., 2016).
A full-scale test of unsupervised speech recognition could be to train on the audio part of a transcribed speech corpus (e.g., TIMIT (Garofolo, 1993)), then learn to predict the transcriptions with only very sparse supervision.
Strong generalization
Humans can transfer knowledge and skills across situations that share high-level structure but are otherwise radically different, adapting to the particulars of a new setting while preserving the essence of the skill, a capacity that (Tarlow, 2016; Gaunt et al., 2016) refer to as strong generalization. If we learn to clean up a room, we know how to clean up most other rooms.
Tests
A general assembly robot could learn to build a toy castle in one material (e.g., lego blocks) and be tested on building it from other materials (sand, stones, sticks).
A household robot could be trained on cleaning and cooking tasks in one environment and be tested in highly dissimilar environments.
Category learning from few examples
Lake et al. (2015) achieved human-level recognition and generation of characters using few examples. However, learning more complex categories from few examples remains an open problem.
Tests
The ImageNet database (Deng et al., 2009) contains images organized by the semantic hierarchy of WordNet (Miller, 1995). Correctly determining ImageNet categories from images with very little training data could be a challenging test of learning from few examples.
Learning to learn
Learning to learn or meta-learning (e.g., Harlow, 1949; Schmidhuber, 1987; Thrun and Pratt, 1998; Andrychowicz et al., 2016; Chen et al., 2016; de Freitas, 2016; Duan et al., 2016; Lake et al., 2016; Wang et al., 2016) is the acquisition of skills and inductive biases that facilitate future learning. The scenarios considered in particular are ones where a more general and slower learning process produces a faster, more specialized one. An example is biological evolution producing efficient learners such as human beings.
Tests
Learning to play Atari video games is an area that has seen some remarkable recent successes, including in transfer learning (Parisotto et al., 2016). However, there is so far no system that first learns to play video games, then is capable of learning a new game, as humans can, from a few minutes of play (Lake et al., 2016).
Compositional learning
Compositional learning (de Freitas, 2016; Lake et al., 2016) is the ability to recombine primitive representations to accelerate the acquisition of new knowledge. It is closely related to learning to learn.
Tests
Tests for compositional learning need to verify both that the learner is effective and that it uses compositional representations.
Some ImageNet categories correspond to object classes defined largely by their arrangements of component parts, e.g., chairs and stools, or unicycles, bicycles, and tricycles. A test could evaluate the agent’s ability to learn categories with few examples and to report the parts of the object in an image.
Compositional learning should be extremely helpful in learning video games (Lake et al., 2016). A learner could be tested on a game already mastered, but where component elements have changed appearance (e.g., different-looking fish in the Frostbite game). It should be able to play the variant game with little or no additional learning.
Learning without forgetting
In order to learn continually over its lifetime, an agent must be able to generalize over new observations while retaining previously acquired knowledge. Recent progress towards this goal is reported in (Kirkpatrick et al., 2016) and (Li and Hoiem, 2016). Work on memory augmented neural networks (e.g., Graves et al., 2016) is also relevant.
Tests
A test for learning without forgetting needs to present learning tasks sequentially (earlier tasks are not repeated) and test for retention of early knowledge. It may also test for declining learning time for new tasks, to verify that the agent exploits the knowledge acquired so far.
A challenging test for learning without forgetting would be to learn to recognize all the categories in ImageNet, presented sequentially.
Transfer learning
Transfer learning (Pan and Yang, 2010) is the ability of an agent trained in one domain to master another. Results in the area of text comprehension are currently poor unless the agent is given some training on the new domain (Kadlec, et al., 2016).
Tests
Sentiment classification (Blitzer et al., 2007) provides a possible testing ground for transfer learning. Learners can be trained on one corpus, tested on another, and compared to a baseline learner trained directly on the target domain.
Reviews of movies and of businesses are two domains dissimilar enough to make knowledge transfer challenging. Corpora for the domains are Rotten Tomatoes movie reviews (Pang and Lee, 2005) and the Yelp Challenge dataset (Yelp, 2017).
Knowing when you don’t know
While uncertainty is modeled differently by different learning algorithms, it seems to be true in general that current artificial systems are not nearly as good as humans at “knowing when they don’t know.” An example are deep neural networks that achieve state-of-the-art accuracy on image recognition but assign 99.99% confidence to the presence of objects in images completely unrecognizable to humans (Nguyen et al., 2015).
Human performance on confidence estimation would include
In induction tasks, like program induction or sequence completion, knowing when the provided examples are insufficient for induction (multiple reasonable hypotheses could account for them)
In speech recognition, knowing when an utterance has not been interpreted reliably
In visual tasks such as pedestrian detection, knowing when a part of the image has not been analyzed reliably
Tests
A speech recognizer can be compared against a human baseline, measuring the ratio of the average confidence to the confidence on examples where recognition fails.
The confidence of image recognition systems can be tested on generated adversarial examples.
Learning through action
Human infants are known to learn about the world through experiments, observing the effects of their own actions (Smith and Gasser, 2005; Malik, 2015). This seems to apply both to higher-level cognition and perception. Animal experiments have confirmed that the ability to initiate movement is crucial to perceptual development (Held and Hein, 1963) and some recent progress has been made on using motion in learning visual perception (Agrawal et al., 2015). In (Agrawal et al., 2016), a robot learns to predict the effects of a poking action.
“Learning through action” thus encompasses several areas, including
Active learning, where the agent selects the training examples most likely to be instructive
Undertaking epistemological actions, i.e., activities aimed primarily at gathering information
Learning to perceive through action
Learning about causal relationships through action
Perhaps most importantly, for artificial systems, learning the causal structure of the world through experimentation is still an open problem.
Tests
For learning through action, it is natural to consider problems of motor manipulation where in addition to the immediate effects of the agent’s actions, secondary effects must be considered as well.
Learning to play billiards: An agent with little prior knowledge and no fixed training data is allowed to explore a real or virtual billiard table and should learn to play billiards well.
Sensorimotor problems
Outstanding problems in robotics and machine perception include:
Autonomous navigation in dynamic environments
Scene analysis
Robust general object recognition and detection
Robust, life-time simultaneous location and mapping (SLAM)
Multimodal integration
Adaptive dexterous manipulation
Autonomous navigation
Despite recent progress in self-driving cars by companies like Tesla, Waymo (formerly the Google self-driving car project) and many others, autonomous navigation in highly dynamic environments remains a largely unsolved problem, requiring knowledge of object semantics to reliably predict future scene states (Ess et al., 2010).
Tests
Fully automatic driving in crowded city streets and residential areas is still a challenging test for autonomous navigation.
Scene analysis
The challenge of scene analysis extends far beyond object recognition and includes the understanding of surfaces formed by multiple objects, scene 3D structure, causal relations (Lake et al., 2016), and affordances. It is not limited to vision but can depend on audition, touch, and other modalities, e.g., electroreception and echolocation (Lewicki et al., 2014; Kondo et al., 2017). While progress has been made, e.g., in recognizing anomalous and improbable scenes (Choi et al., 2012), predicting object dynamics (Fouhey and Zitnick, 2014), and discovering object functionality (Yao et al., 2013), we are still far from human-level performance in this area.
Tests
Some possible challenges for understanding the causal structure in visual scenes are:
Recognizing dangerous situations: A corpus of synthetic images could be created where the same objects are recombined to form “dangerous” and “safe” scenes as classified by humans.
Recognizing physically improbable scenes: A synthetic corpus could be created to show physically plausible and implausible scenes containing the same objects.
Recognizing useless objects: Images of useless objects have been created by (Kamprani, 2017).
Object recognition
While object recognition has seen great progress in recent years (e.g., Han et al., 2016), matches or surpasses human performance for many problems (Karpathy, 2014), and can approach perfection in closed environments (Song et al., 2015), state-of-the-art systems still struggle with the harder cases such as open objects (interleaved with background), broken objects, truncation and occlusion in dynamic environments (e.g., Rajaram et al., 2015).
Tests
Environments that are cluttered and contain objects drawn from a large, open-ended, and changing set of types are likely to be challenging for an object recognition system. An example would be
Seeing photos of the insides of pantries and refrigerators and listing the ingredients available to the owners
Simultaneous location and mapping
While the problem of simultaneous location and mapping (SLAM) is considered solved for some applications, the challenge of SLAM for long-lived autonomous robots, in large-scale, time-varying environments, remains open (Cadena et al., 2016).
Tests
Lifetime location and mapping, without detailed maps provided in advance and robust to changes in the environment, for an autonomous car based in a large city
Multimodal integration
The integration of multiple senses (Lahat, 2015) is important, e.g., in human communication (Morency, 2015) and scene understanding (Lewicki et al., 2014; Kondo et al., 2017). Having multiple overlapping sensory systems seems to be essential for enabling human children to educate themselves by perceiving and acting in the world (Smith and Gasser, 2005).
Tests
Spoken communication in noisy environments, where lip reading and gestural cues are indispensable, can provide challenges for multimodal fusion. An example would be
A robot bartender: The agent needs to interpret customer requests in a noisy bar.
Adaptive dexterous manipulation
Current robot manipulators do not come close to the versatility of the human hand (Ciocarlie, 2015). Hard problems include manipulating deformable objects and operating from a mobile platform.
Tests
Taking out clothes from a washing machine and hanging them on clothes lines and coat hangers in varied places while staying out of the way of humans
Open-ended problems
Some noteworthy problems were omitted from the list for having a too open-ended scope: they encompass sets of tasks that evolve over time or can be endlessly extended. This makes it hard to decide whether a problem has been solved. Problems of this type include
Enrolling in a human university and take classes like humans (Goertzel, 2012)
Automating all types of human work (Nilsson, 2005)
Puzzlehunt challenges, e.g., the annual TMOU game in the Czech republic (TMOU, 2016)
Conclusion
I have reviewed a number of open problems in an attempt to delineate the current front lines of AI research. The problem list in this first version, as well as the problem descriptions, example tests, and mentions of ongoing work in the research areas, are necessarily incomplete. I plan to extend and improve the document incrementally and warmly welcome suggestions either in the comment section below or at the institute’s discourse forum.
Acknowledgements
I thank Jan Feyereisl, Martin Poliak, Petr Dluhoš, and the rest of the GoodAI team for valuable discussion and suggestions.
Agrawal, Pulkit, Joao Carreira, and Jitendra Malik. “Learning to see by moving.” Proceedings of the IEEE International Conference on Computer Vision. 2015.
Agrawal, Pulkit, et al. “Learning to poke by poking: Experiential learning of intuitive physics.” arXiv preprint arXiv:1606.07419 (2016).
AI•ON. “The AI•ON collection of open research problems.” Online under http://ai-on.org/projects(2016)
Allamanis, Miltiadis, Hao Peng, and Charles Sutton. “A convolutional attention network for extreme summarization of source code.” arXiv preprint arXiv:1602.03001 (2016).
Andrychowicz, Marcin, et al. “Learning to learn by gradient descent by gradient descent.” Advances in Neural Information Processing Systems. 2016.
Blitzer, John, Mark Dredze, and Fernando Pereira. “Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification.” ACL. Vol. 7. 2007.
Brachman, Ronald J. “AI more than the sum of its parts.” AI Magazine 27.4 (2006): 19.
Brooks, R., et al. “Challenge problems for artificial intelligence.” Thirteenth National Conference on Artificial Intelligence-AAAI. 1996.
Cadena, Cesar, et al. “Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age.” IEEE Transactions on Robotics 32.6 (2016): 1309–1332.
Chang, Michael B., et al. “A compositional object-based approach to learning physical dynamics.” arXiv preprint arXiv:1612.00341 (2016).
Chen, Yutian, et al. “Learning to Learn for Global Optimization of Black Box Functions.” arXiv preprint arXiv:1611.03824 (2016).
Choi, Myung Jin, Antonio Torralba, and Alan S. Willsky. “Context models and out-of-context objects.” Pattern Recognition Letters 33.7 (2012): 853–862.
de Freitas, Nando. “Learning to Learn and Compositionality with Deep Recurrent Neural Networks: Learning to Learn and Compositionality.” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016.
Degrave, Jonas, Michiel Hermans, and Joni Dambre. “A Differentiable Physics Engine for Deep Learning in Robotics.” arXiv preprint arXiv:1611.01652 (2016).
Deng, Jia, et al. “Imagenet: A large-scale hierarchical image database.” Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.
Denil, Misha, et al. “Learning to Perform Physics Experiments via Deep Reinforcement Learning.” arXiv preprint arXiv:1611.01843 (2016).
Duan, Yan, et al. “RL²: Fast Reinforcement Learning via Slow Reinforcement Learning.” arXiv preprint arXiv:1611.02779 (2016).
Ess, Andreas, et al. “Object detection and tracking for autonomous navigation in dynamic environments.” The International Journal of Robotics Research 29.14 (2010): 1707–1725.
Finn, Chelsea, and Sergey Levine. “Deep Visual Foresight for Planning Robot Motion.” arXiv preprint arXiv:1610.00696 (2016).
Fouhey, David F., and C. Lawrence Zitnick. “Predicting object dynamics in scenes.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.
Fragkiadaki, Katerina, et al. “Learning visual predictive models of physics for playing billiards.” arXiv preprint arXiv:1511.07404 (2015).
Garofolo, John, et al. “TIMIT Acoustic-Phonetic Continuous Speech Corpus LDC93S1.” Web Download. Philadelphia: Linguistic Data Consortium, 1993.
Gaunt, Alexander L., et al. “Terpret: A probabilistic programming language for program induction.” arXiv preprint arXiv:1608.04428 (2016).
Genesereth, Michael, Nathaniel Love, and Barney Pell. “General game playing: Overview of the AAAI competition.” AI magazine 26.2 (2005): 62.
Graves, Alex, et al. “Hybrid computing using a neural network with dynamic external memory.” Nature 538.7626 (2016): 471–476.
Hamrick, Jessica B., et al. “Imagination-Based Decision Making with Physical Models in Deep Neural Networks.” Online under http://phys.csail.mit.edu/papers/5.pdf(2016)
Han, Dongyoon, Jiwhan Kim, and Junmo Kim. “Deep Pyramidal Residual Networks.” arXiv preprint arXiv:1610.02915 (2016).
Harlow, Harry F. “The formation of learning sets.” Psychological review 56.1 (1949): 51.
Held, Richard, and Alan Hein. “Movement-produced stimulation in the development of visually guided behavior.” Journal of comparative and physiological psychology 56.5 (1963): 872.
Hernández-Orallo, José. “Evaluation in artificial intelligence: from task-oriented to ability-oriented measurement.” Artificial Intelligence Review(2016a): 1–51.
Hernández-Orallo, José, et al. “Computer models solving intelligence test problems: progress and implications.” Artificial Intelligence 230 (2016b): 74–107.
Hernández-Orallo, José. “The measure of all minds.” Cambridge University Press, 2017.
IOCCC. “The International Obfuscated C Code Contest.” Online under http://www.ioccc.org(2016)
Kadlec, Rudolf, et al. “Finding a jack-of-all-trades: an examination of semi-supervised learning in reading comprehension.” Under review at ICLR 2017, online under https://openreview.net/pdf?id=rJM69B5xx
Kamper, Herman, Aren Jansen, and Sharon Goldwater. “Unsupervised word segmentation and lexicon discovery using acoustic word embeddings.” IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP) 24.4 (2016): 669–679.
Kirkpatrick, James, et al. “Overcoming catastrophic forgetting in neural networks.” arXiv preprint arXiv:1612.00796 (2016).
Kondo, H. M., et al. “Auditory and visual scene analysis: an overview.” Philosophical transactions of the Royal Society of London. Series B, Biological sciences 372.1714 (2017).
Lahat, Dana, Tülay Adali, and Christian Jutten. “Multimodal data fusion: an overview of methods, challenges, and prospects.” Proceedings of the IEEE 103.9 (2015): 1449–1477.
Lake, Brenden M., Ruslan Salakhutdinov, and Joshua B. Tenenbaum. “Human-level concept learning through probabilistic program induction.” Science 350.6266 (2015): 1332–1338.
Lake, Brenden M., et al. “Building machines that learn and think like people.” arXiv preprint arXiv:1604.00289 (2016).
Lewicki, Michael S., et al. “Scene analysis in the natural environment.” Frontiers in psychology 5 (2014): 199.
Li, Wenbin, Aleš Leonardis, and Mario Fritz. “Visual stability prediction and its application to manipulation.” arXiv preprint arXiv:1609.04861 (2016).
Li, Zhizhong, and Derek Hoiem. “Learning without forgetting.” European Conference on Computer Vision. Springer International Publishing, 2016.
McLeod, John. “Card game rules — card games and tile games from around the world.” Online under https://www.pagat.com (2017)
Miller, George A. “WordNet: a lexical database for English.” Communications of the ACM 38.11 (1995): 39–41.
Mottaghi, Roozbeh, et al. ““What happens if…” Learning to Predict the Effect of Forces in Images.” European Conference on Computer Vision. Springer International Publishing, 2016.
Nair, Ashvin, et al. “Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation.” Online under http://phys.csail.mit.edu/papers/15.pdf (2016)
Nguyen, Anh, Jason Yosinski, and Jeff Clune. “Deep neural networks are easily fooled: High confidence predictions for unrecognizable images.” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015.
Nilsson, Nils J. “Human-level artificial intelligence? Be serious!.” AI magazine 26.4 (2005): 68.
Pan, Sinno Jialin, and Qiang Yang. “A survey on transfer learning.” IEEE Transactions on knowledge and data engineering 22.10 (2010): 1345–1359.
Pang, Bo, and Lillian Lee. “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales.” Proceedings of the 43rd annual meeting on association for computational linguistics. Association for Computational Linguistics, 2005.
Parisotto, Emilio, Jimmy Lei Ba, and Ruslan Salakhutdinov. “Actor-mimic: Deep multitask and transfer reinforcement learning.” arXiv preprint arXiv:1511.06342 (2015).
Park, Alex S., and James R. Glass. “Unsupervised pattern discovery in speech.” IEEE Transactions on Audio, Speech, and Language Processing 16.1 (2008): 186–197.
Rajaram, Rakesh Nattoji, Eshed Ohn-Bar, and Mohan M. Trivedi. “An exploration of why and when pedestrian detection fails.” 2015 IEEE 18th International Conference on Intelligent Transportation Systems. IEEE, 2015.
Riccardi, Giuseppe, and Dilek Z. Hakkani-Tür. “Active and unsupervised learning for automatic speech recognition.” Interspeech. 2003.
Rosa, Marek, Jan Feyereisl, and The GoodAI Collective. “A Framework for Searching for General Artificial Intelligence.” arXiv preprint arXiv:1611.00685 (2016).
Schmidhuber, Jurgen. “Evolutionary principles in self-referential learning.” On learning how to learn: The meta-meta-… hook.) Diploma thesis, Institut f. Informatik, Tech. Univ. Munich (1987).
Silver, David, et al. “Mastering the game of Go with deep neural networks and tree search.” Nature 529.7587 (2016): 484–489.
Smith, Linda, and Michael Gasser. “The development of embodied cognition: Six lessons from babies.” Artificial life 11.1–2 (2005): 13–29.
Song, Shuran, Linguang Zhang, and Jianxiong Xiao. “Robot in a room: Toward perfect object recognition in closed environments.” CoRR (2015).
Stewart, Russell, and Stefano Ermon. “Label-free supervision of neural networks with physics and domain knowledge.” arXiv preprint arXiv:1609.05566 (2016).
Verschae, Rodrigo, and Javier Ruiz-del-Solar. “Object detection: current and future directions.” Frontiers in Robotics and AI 2 (2015): 29.
Wang, Jane X., et al. “Learning to reinforcement learn.” arXiv preprint arXiv:1611.05763 (2016).
Yao, Bangpeng, Jiayuan Ma, and Li Fei-Fei. “Discovering object functionality.” Proceedings of the IEEE International Conference on Computer Vision. 2013.
International Women’s Day is raising discussion about the lack of diversity and role models in STEM and the potential negative outcomes of bias and stereotyping in robotics and AI. Let’s balance the words with positive actions. Here’s what we can all do to support women in robotics and AI, and thus improve diversity, innovation and reduce skills shortages for robotics and AI.
