With a superb organization and a beautiful location, the event included conferences of leading researchers and companies from all around the world, as well as workshops and an exhibitors area. This latter is where I spent most of my time, as I love direct interaction with the companies and research centres. Also, in this kind of academic events, compared to trade fairs, you usually have the chance to directly find technical people who are able to explain in deep detail all the ins and outs of their products.
The robotics community is not so big yet, so we still know each other. I had the pleasure to meet good friends from companies like Infinium Robotics, PAL Robotics and Fetch Robotics between others. Infinium Robotics is the company in Singapore where I work as CTO. I already wrote about this great company in one of my previous posts: “Infinium Robotics. Flying Robots“.
PAL Robotics is a company in Spain well known for having developed some of the best humanoid robots in the world. I have a very good relationship with this companies team since more than ten years ago. Great people, well motivated, well managed, that has been able to look outside of the box and enter with bravery in the world of the robotics warehouse solutions with robots like Tiago or StockBot.
Fetch Robotics is also one of the big players in the robotics solutions for warehousing industry. But I met other interesting people and had amazing chats with people from companies so key in this field as: Amazon Robotics, DJI or Clearpath Robotics. At the end of this post, there is a full list of exhibiting companies.
I saw interesting technology, like the rehabilitation exoskeletons from Fourier Intelligence (Shanghai), the Spidar-G 6DOF Haptic interface from Tokyo Institute of Technology, the Haptic systems of Force Dimension and Moog, the dexterous manipulators of Kinova, Kuka, ABB, ITRI, the modular robot components of HEBI Robotics, Keyi Technology, the D motion capture technologies of Phoenix Technologies and Optitrack, the educational solutions of Ubitech, GT Robot or Robotis and many, many others, most of them ROS enabled.
As I usually do at these events, I recorded a video of the exhibition area to provide an idea of the technologies shown there.
Last but not least, I want to thank my friends from SIAA (Singapore International Automation Association) for their kind friendship and support. They also organized also the Singapore Pavilion in this event.
Ms LIM Sue Yin, Civic Seh (both SIAA) and Alejandro Alonso (IR/Hisparob VP)
Humankind lost another important battle with artificial intelligence (AI) last month when AlphaGo beat the world’s leading Go player Ke Jie by three games to zero.
AlphaGo is an AI program developed by DeepMind, part of Google’s parent company Alphabet. Last year it beat another leading player, Lee Se-dol, by four games to one, but since then AlphaGo has substantially improved.
AlphaGo will now retire from playing Go, leaving behind a legacy of games played against itself. They’ve been described by one Go expert as like “games from far in the future”, which humans will study for years to improve their own play.
Ready, set, Go
Go is an ancient game that essentially pits two players – one playing black pieces the other white – for dominance on board usually marked with 19 horizontal and 19 vertical lines.
A typical game of Go: simple to learn but a lifetime to master. Flickr/Alper Cugun, CC BY
Go is a far more difficult game for computers to play than chess, because the number of possible moves in each position is much larger. This makes searching many moves ahead – feasible for computers in chess – very difficult in Go.
DeepMind’s breakthrough was the development of general-purpose learning algorithms that can, in principle, be trained in more societal-relevant domains than Go.
DeepMind says the research team behind AplhaGo is looking to pursue other complex problems, such as finding new cures for diseases, dramatically reducing energy consumption or inventing revolutionary new materials. It adds:
If AI systems prove they are able to unearth significant new knowledge and strategies in these domains too, the breakthroughs could be truly remarkable. We can’t wait to see what comes next.
This does open up many opportunities for the future, but challenges still remain.
Neuroscience meets AI
AlphaGo combines the two most powerful ideas about learning to emerge from the past few decades: deep learning and reinforcement learning. Remarkably, both were originally inspired by how biological brains learn from experience.
In the human brain, sensory information is processed in a series of layers. For instance, visual information is first transformed in the retina, then in the midbrain, and then through many different areas of the cerebral cortex.
This creates a hierarchy of representations where simple, local features are extracted first, and then more complex, global features are built from these.
The AI equivalent is called deep learning; deep because it involves many layers of processing in simple neuron-like computing units.
But to survive in the world, animals need to not only recognise sensory information, but also act on it. Generations of scientists and psychologists have studied how animals learn to take a series of actions that maximise their reward.
This has led to mathematical theories of reinforcement learning that can now be implemented in AI systems. The most powerful of these is temporal difference learning, which improves actions by maximising its expectation of future reward.
The best moves
By combining deep learning and reinforcement learning in a series of artificial neural networks, AlphaGo first learned human expert-level play in Go from 30 million moves from human games.
But then it started playing against itself, using the outcome of each game to relentlessly refine its decisions about the best move in each board position. A value network learned to predict the likely outcome given any position, while a policy network learned the best action to take in each situation.
Although it couldn’t sample every possible board position, AlphaGo’s neural networks extracted key ideas about strategies that work well in any position. It is these countless hours of self-play that led to AlphaGo’s improvement over the past year.
Unfortunately, as yet there is no known way to interrogate the network to directly read out what these key ideas are. Instead we can only study its games and hope to learn from these.
This is one of the problems with using such neural network algorithms to help make decisions in, for instance, the legal system: they can’t explain their reasoning.
We still understand relatively little about how biological brains actually learn, and neuroscience will continue to provide new inspiration for improvements in AI.
Humans can learn to become expert Go players based on far less experience than AlphaGo needed to reach that level, so there is clearly room for further developing the algorithms.
Also much of AlphaGo’s power is based on a technique called back-propagation learning that helps it correct errors. But the relationship between this and learning in real brains is still unclear.
What’s next?
The game of Go provided a nicely constrained development platform for optimising these learning algorithms. But many real world problems are messier than this and have less opportunity for the equivalent of self-play (for instance self-driving cars).
So are there problems to which the current algorithms can be fairly immediately applied?
One example may be optimisation in controlled industrial settings. Here the goal is often to complete a complex series of tasks while satisfying multiple constraints and minimising cost.
As long as the possibilities can be accurately simulated, these algorithms can explore and learn from a vastly larger space of outcomes than will ever be possible for humans. Thus DeepMind’s bold claims seem likely to be realised, and as the company says, we can’t wait to see what comes next.
The world’s brightest minds in Artificial Intelligence (AI) and humanitarian action will meet with industry leaders and academia at the AI for Good Global Summit, 7-9 June 2017, to discuss how AI will assist global efforts to address poverty, hunger, education, healthcare and the protection of our environment. The event will in parallel explore means to ensure the safe, ethical development of AI, protecting against unintended consequences of advances in AI.
The event is co-organized by ITU and the XPRIZE Foundation, in partnership with 20 other United Nations (UN) agencies, and with the participation of more than 70 leading companies and academic and research institutes.
“Artificial Intelligence has the potential to accelerate progress towards a dignified life, in peace and prosperity, for all people,” said UN Secretary-General António Guterres. “The time has arrived for all of us – governments, industry and civil society – to consider how AI will affect our future. The AI for Good Global Summit represents the beginnings of our efforts to ensure that AI charts a course that will benefit all of humanity.”
The AI for Good Global Summit will emphasize AI’s potential to contribute to the pursuit of the UN Sustainable Development Goals.
Opening sessions will share expert insight into the state of play in AI, with leading minds in AI giving voice to their greatest ambitions in driving AI towards social good. ‘Breakthrough’ sessions will propose strategies for the development of AI applications and systems able to promote sustainable living, reduce poverty and deliver citizen-centric public services.
“Today, we’ve gathered here to discuss how far AI can go, how much it will improve our lives, and how we can all work together to make it a force for good,” said ITU Secretary-General Houlin Zhao. “This event will assist us in determining how the UN, ITU and other UN Agencies can work together with industry and the academic community to promote AI innovation and create a good environment for the development of artificial intelligence.”
“The AI for Good Global Summit has assembled an impressive, diverse ecosystem of thought leaders who recognize the opportunity to use AI to solve some of the world’s grandest challenges,” said Marcus Shingles, CEO of the XPRIZE Foundation. “We look forward to this Summit providing a unique opportunity for international dialogue and collaboration that will ideally start to pave the path forward for a new future of problem solvers working with XPRIZE and beyond.”
The AI for Good Global Summit will be broadcast globally as well as captioned to ensure accessibility.
RoboCup is an international scientific initiative with the goal to advance the state of the art of intelligent robots. Established in 1997, the original mission was to field a team of robots capable of winning against the human soccer World Cup champions by 2050.
To celebrate 20 years of RoboCup, the Federation is launching a video series featuring each of the leagues with one short video for those who just want a taster, and one long video for the full story. Robohub will be featuring one league every week leading up to RoboCup 2017 in Nagoya, Japan.
This week, we consider being part of the RoboCup@Home league. Robots helping at home can certainly ‘feel’ like the future. One day, these robots might help with various tasks around the house. You’ll hear about the history and ambitions of RoboCup from the trustees, and inspiring teams from around the world.
Short version:
Long version:
Want to watch the rest? You can view all the videos on the RoboCup playlist below:
In 2015, after much research, I wrote about China having 194 robot companies and used screen shots of The Robot Report’s Global Map to show where they were and a chart to show their makeup. We’ve just concluded another research project and have added hundreds of new Chinese companies to the database and global map.
Why is China so focused on robots?
China installed 90,000 robots in 2016, 1/3 of the world’s total and a 30% increase over 2015. Why?
Simply said, China has three drivers helping them move toward country-wide adoption of robotics: scale, growth momentum, and money. Startup companies can achieve scale quickly because the domestic market is so large. Further, companies are under pressure to automate thereby causing double-digit demand for industrial robots (according to the International Federation of Robotics). Third, the government is strongly behind the move.
Made in China 2025 and 5-Year Plans
Chinese President Xi Jinping has called for “a robot revolution” and initiated the “Made in China 2025” program. More than 1,000 firms and a new robotics association, CRIA (Chinese Robotics Industry Alliance) have emerged (or begun to transition) into robotics to take advantage of the program, according to a 2016 report by the Ministry of Industry and Information Technology. By contrast, according to the same report, the sector was virtually non-existent a decade ago.
Under “Made in China 2025,” and the five-year robot plan launched last April, Beijing is focusing on automating key sectors of the economy including car manufacturing, electronics, home appliances, logistics, and food production. At the same time, the government wants to increase the share of in-country-produced robots to more than 50% by 2020; up from 31% last year.
Robot makers, and companies that automate, are both eligible for subsidies, low-interest loans, tax waivers, rent-free land and other incentives. One such program lured back Chinese engineers working overseas; another oversaw billions of dollars poured into technology parks dedicated to robotics production and related businesses; another encouraged local governments to help regional companies deploy robots in their production processes; and despite its ongoing crackdown on capital outflows, green lights have been given to Chinese companies acquiring Western robotics technology companies.
Many of those acquisitions were reported by The Robot Report during 2016 and are reflected (with little red flags) in the chart reporting the top 15 acquisitions of robotic-related companies:
Midea, a Chinese consumer products manufacturer, acquired KUKA, one of the Big 4 global robot manufacturers
The Kion Group, a predominately Chinese-funded warehousing systems and equipment conglomerate, acquired Dematic, a large European AGV and material handling systems company
KraussMaffei, a big German industrial robots integrator, was acquired by ChemChina
Paslin, a US-based industrial robot integrator, was acquired by Zhejiang Wanfeng Technology, a Chinese industrial robot integrator
China has set goals to be able to make 150,000 industrial robots in 2020; 260,000 in 2025; and 400,000 by 2030. If achieved, the plan should help generate $88 billion over the next decade. China’s stated goal in both their 5-year plan and Made in China 2025 program is to overtake Germany, Japan, and the United States in terms of manufacturing sophistication by 2049, the 100th anniversary of the founding of the People’s Republic of China. To make that happen, the government needs Chinese manufacturers to adopt robots by the millions. It also wants Chinese companies to start producing more of those robots.
Analysts and Critics
Various research reports are predicting that more than 250,000 industrial pick and place, painting and welding robots will be purchased and deployed in China by 2019. That figure represents more than the total global sales of all types of industrial robots in 2014!
Research firms predicting dramatic growth for the domestic Chinese robotics industry are also predicting very low-cost devices. Their reports are contradicted by academics, roboticists and others who point out that there are so many new robot manufacturing companies in China that none will be able to manufacture many thousand robots per year and thus benefit from scale. Further, many of the components that comprise a robot are intricate and costly, e.g., speed reducers, servo motors and control panels. Consequently these are purchased from Nabtesco, Harmonic Drive, Sumitomo and other Japanese, German and US companies. Although a few of the startups are attempting to make reduction gears and other similar devices, the lack of these component manufacturers in China may put a cap on how low costs can go and on how much can be done in-country for the time being.
“We aim to increase the market share of homegrown servomotors, speed reducers and control panels in China to over 30 percent by 2018 or 2019,” said Qu Xianming, an expert with the National Manufacturing Strategy Advisory Committee, which advises the government on plans to upgrade the manufacturing sector. “By then, these indigenous components could be of high enough quality to be exported to foreign countries,” Qu said in an interview with China Daily. “Once the target is met, it will lay down a strong foundation for Chinese parts makers to expand their presence.”
Regardless, China, with governmental directives and incentives, has become both the world’s biggest buyer of robots and also is growing a very large in-country industry to make and sell robots of all types.
The Robot Report now has over 500 Chinese companies in its online directories and on its Global Map
The Robot Report and its research team have been able to identify over 500 companies that make or are directly involved in making robots in China. The CRIA (China Robot Industry Alliance), and other sources, proffer the number to be closer to 800. The Robot Report is limited by our own research capabilities, language translation limitations, and scarcity of information about robotics companies and their websites and contact people in China.
These companies are combined with other global companies – now totaling over 5,300 – in our online directories and plotted on our global map so that you can research by area. You can explore online and filter in a variety of ways.
Use Google’s directional and +/- markers to navigate, enlarge, and hone in on a geographical area of interest (or double click near where you want to enlarge). Click on one of the colored markers to get a pop-up window with the name, type, focus, location and a link to the company’s website.
[NOTE: the map shows a single entry for the company headquarters regardless how many branches, subsidiaries and factory locations that company might have, consequently international companies with factories and service centers in China won’t appear. Further note that The Robot Report’s database doesn’t contain companies that just use robots; it focuses on those involved in making robots.]
The Filter pull-down menu lets you choose any one of the seven major categories:
Industrial robot makers
Service robots used by corporations and governments
Service robots for personal and private use
Integrators
Robotics-related start-up companies
Universities and research labs with special emphasis on robotics
Ancillary businesses providing engineering, software, components, sensors and other products and services to the industry.
In the chart below, 500 Chinese companies are tabulated by their business type and area of focus. Please note that your help would be greatly appreciated by adding to the map and making it as accurate and up-to-date as possible. Please send robotics-related companies that we have missed (or are new) to info@therobotreport.com.
Autonomous exploration and reliable mapping of unknown environments corresponds to a major challenge for mobile robotic systems. For many important application domains, such as industrial inspection or search and rescue, this task is further challenged from the fact that such operations often have to take place in GPS-denied environments and possibly visually-degraded conditions.
Source: Dr Kostas Alexis, UNR
In this work, we move away from deterministic approaches on autonomous exploration and we propose a localization uncertainty-aware autonomous receding horizon exploration and mapping planner verified using aerial robots. This planner follows a two-step optimization paradigm. At first, in an online computed random tree the algorithm finds a finite-horizon branch that optimizes the amount of space expected to be explored. The first viewpoint configuration of this branch is selected, but the path towards it is decided through a second planning step. Within that, a new tree is sampled, admissible branches arriving at the reference viewpoint are found and the robot belief about its state and the tracked landmarks of the environment is propagated. The branch that minimizes the expected localization uncertainty is selected, the corresponding path is executed by the robot and the whole process is iteratively repeated.
The algorithm has been experimentally verified with aerial robotic platforms equipped with a stereo visual-inertial system operating in both well-lit and dark conditions, as shown in our videos:
To enable further developments, research collaboration and consistent comparison, we have released an open source version of our localization uncertainty-aware exploration and mapping planner, experimental datasets and interfaces. To get the code, please visit: https://github.com/unr-arl/rhem_planner
Christos Papachristos, Shehryar Khattak, Kostas Alexis, “Uncertainty-aware Receding Horizon Exploration and Mapping using Aerial Robots,” IEEE International Conference on Robotics and Automation (ICRA), May 29-June 3, 2017, Singapore
If you liked this article, you may also want to read:
Figure 1: A distributed robotic system managing the logistics of a warehouse.
It isn’t a secret that distributed robotic systems are starting to revolutionize many applications from targeted material delivery (e.g., Amazon Robotics) to precision farming. Assisted by technological advancements such as cloud computing, novel hardware design, and manufacturing techniques, nowadays distributed robot systems are starting to become an important part of industrial activities including warehouse logistics or autonomous transportation.
However, as many engineers and scientists in this field know, several of the heterogeneous characteristics of these systems that make them ideal for certain future applications — robot autonomy, decentralized control, collective emergent behavior, collective learning, knowledge sharing, etc. — hinder the evolution of the technology from academic institutions to the public sphere. For instance, controlling the motion and behavior of large teams of robots still presents unique challenges for human operators, who cannot yet effectively convey their high-level intentions in application. Moreover, robots collaborating through the cloud might find difficulties applying shared knowledge due to physical hardware differences. Solutions to these issues might be necessary steps towards mainstream adoption.
Figure 2: Different types of robots share a blockchain communication channel using their public keys as main identifiers.
In order to understand the current state of the art of the distributed robotic systems field and foresee its breakthroughs, the International Journal of Advanced Robotic Systems decided to launch a special issue titled “Distributed Robotic Systems and Society”. This special issue seeks to move beyond the classical view of distributed robotic systems to advance our understanding about the future role of these systems in the marketplace and public society. Insights to unasked questions in the field are especially suited to this issue. For instance, what security methods are available and are efficient for these systems? What kind of distributed robotic algorithms are suitable to introduce human-oriented interactions? Are there new interfaces to connect with these systems or reduce their complexity? Are distributed networks such as Bitcoin a feasible way to integrate distributed robotic systems in our society? Are there new business models for distributed robot ventures? How can distributed robotic systems make use of unlimited access information in the cloud?
We also welcome submissions on other topics addressing multi-robot systems in the society. We seek papers with conceptual and theoretical contributions as well as papers documenting valuable results of experiments conducted with real-robots. Finally, the editorial team of this special issue (Dr. Penaloza, Dr. Hauert, and myself) would like to encourage researchers and scientists to submit their manuscripts. We are confident that the ideas, methods, and results included in this special issue will assist the scientific community as well as the industry to reach new horizons in the field of distributed robotics systems.
A German Heron 1 UAV. Credit: Airbus Defence and Space
May 29, 2017 – June 4, 2017
News
A German court has thrown out a protest filed by U.S. drone maker General Atomics Aeronautical Systems over the German military’s decision to acquire the Israel Aerospace Industries Heron TP drone over the U.S. firm’s Reaper. The decision by the Oberlandesgericht, the country’s high court, allows the Bundeswehr to proceed with the planned acquisition of five Heron TPs, a medium-altitude long-endurance surveillance and reconnaissance drone. (DefenseNews)
At the Washington Post, Michael Laris considers what is likely to happen in the wake of the federal appeals court’s decision to strike down the FAA’s drone registration rule for hobbyists.
At the Jamestown Foundation, Tobias J. Burgers and Scott N. Romaniuk look at how al-Qaeda learned to adapt to U.S. drone strikes.
In the Air & Space Power Journal, Lt. Col. Thomas S. Palmer and Dr. John P. Geis II argue that effective counter-drone weapons will be essential in future conflicts.
Belorussian firms presented a range of new military unmanned aircraft at the MILEX 2017 exhibition in Minsk, including the Belar YS-EX, a medium-altitude long-endurance system. (IHS Jane’s 360)
In a test, the U.S. Army used two Raytheon Stinger anti-aircraft missiles to intercept two drones. (Press Release)
The U.S. Special Operations Command has completed testing for its Joint Threat Warning System sensor for the Puma hand-launched tactical drone. (Shephard Media)
Snap, the social media company that owns Snapchat, has acquired Ctrl Me Robotics, a California-based drone startup. (Buzzfeed)
The Australian Ministry of Defense announced that it will invest $75 million in small unmanned aircraft systems, including the AeroVironment Wasp AE, for the Australian Army. (Press Release)
The Netherlands Ministry of Defence selected the Insitu Integrator to replace the Insitu ScanEagle. (Unmanned Systems Technology)
The U.S. Army awarded Six3 Advanced Systems a $10.5 million contract to design and develop a prototype for a squad of human and unmanned assets. (DoD)
Three German firms, ESG Elektroniksystem und Logistik, Diehl Defence and Rohde & Schwarz, have partnered to market the Guardion counter-drone solution. (Shephard Media)
For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.
A quick, hassle-free way to stay on top of robotics news, our robotics digest is released on the first Monday of every month. Sign up to get it in your inbox.
20 years of RoboCup
20 years in the books! RoboCup, which first started in 1997, was originally established to bring forth a team of robots that could beat the human soccer World Cup champions. Twenty years on, RoboCup is so more than just a soccer competition. In fact, the competition has grown into an international movement with a variety of leagues. Teams compete against each other in four different leagues and many sub-competitions, including home, work, and rescue missions. The complexity of missions in RoboCup requires intelligent, dynamic, sensing robots that can react to chaotic and changing environments. And in its 20-year history, the competition has brought forth numerous winners who have gone on to achieve great things.
Without robot competitions like RoboCup, the field of robotics wouldn’t be where it is today. So to celebrate 20 years of RoboCup, the Federation launched a video series featuring each of the leagues with one short video for those who just want a taster, and one long video for the full story. Robohub will be featuring one league every week leading up to RoboCup 2017 in Nagoya, Japan. You can watch all videos from the playlist here.
ICRA 2017 in Singapore
While we have RoboCup 2017 to look forward to, theIEEE 2017 International Conference on Robotics and Automation (ICRA) took place in Singapore. Under the conference theme “Innovation, Entrepreneurship, and Real-world Solutions”, the event brought together engineers, researchers, entrepreneurs and industry to address some of the major challenges of our times.
This year’s keynote speakers included Louis Phee, who described development and implementation of a surgical robot called EndoMaster, andPeter Luh, giving an overview of Industry 4.0. Alongside the speeches, the event included 25 workshops and tutorials, various exhibitions, as well as four Robot Challenges.
Self-driving cars: Competition heats up
May wasn’t just events and competitions. In the world of autonomous vehicles, French Groupe PSA, the second largest car manufacturer in Europe which owns brands such as Peugeot and Citroen, have teamed up with autonomous car maker nuTonomy. The collaboration will seek to build a self-driving Peugeot 3008, which they hope will hit the roads of Singapore in the not-too-distant future.
Groupe PSA and other well-known car manufacturers, including Ford Motors, who are just starting their bid to enter the autonomous car market, are lagging way behind Waymo (the self-driving car developed by Google’s parent Alphabet) when it comes to putting their cars on actual roads. And with Waymo about to team up with ride-hailing start-up Lyft, Google is getting ever closer to making their autonomous car part of mainstream traffic.
Another player in the self-driving car game we haven’t mentioned yet is cab service Uber. The company is locked in a dispute with Google over allegedly stolen design secrets and will likely be going to public trial later in the year. Linked to the lawsuit, Uber fired the engineer at the heart of the dispute, Anthony Levandowski, who is suspected of having stolen company secrets when he left Google and founded his own company, Otto, which was acquired by Uber last year. The information war continues.
Self-driving cars: Innovation
Meanwhile, a group of veterans previously linked with Alphabet founded their own company – DeepMap Inc. – which aims to develop systems that allow cars to navigate complex cityscapes.
It’s not just cityscapes that are complex to navigate. Considering other vehicles, cyclists, and pedestrians on busy roads form part of the difficult environment a self-driving car has to cope with due to its unpredictable nature. It will, therefore, become necessary for autonomous cars to understand and predict behaviour. Here, machine learning will be key, as Dr Nathan Griffiths explains in The Royal Society’s blog on machine learning in research.
And with so much interest and research into autonomous, intelligent cars, it’s not surprising that some, like Chris Urmson in a recent lecture at CMU, predict we will see a shift from the traditional transport model where people own their cars to a more dynamic, responsive model of “Transportation as a Service”, in which, companies own fleets of cars that can be used by anyone when and where they’re needed.
Robots in the fields
Much of the innovations that have enabled autonomous cars are transferable to industrial, commercial and agricultural vehicles. In the case of the latter, self-driving tractors and precision agribots have already increased productivity and made 24-hour autonomous, high-yield farming a possibility.
It is believed the uptake of farming technologies is not progressing as quickly as it could be, due to farmers being slow to accept precision agriculture products, software, equipment and practices. But the farming (r)evolution is well underway, as agricultural robotics are already a 3 billion dollar industry set to grow to an impressive 12 billion by 2026.
Robots in the skies
While robots are still waiting to be fully accepted in agriculture, it’s no secret that the US military uses drones extensively. What came as a surprise was the strange video feed that surfaced in May from what appeared to be a drone flying over Florida’s panhandle, apparently sponsored by the National Reconnaissance Office – a body that doesn’t usually publicise its drone-related activities. The footage, now believed to have been shot in February, was likely uploaded as a demo video by a contractor.
While the Florida video caused quite a bit of confusion this month, another drone was making headlines for very different reasons. Previously used in military operations, the ArcticShark (a modified version of the military TigerShark) drone is now helping to fight climate change. And it’s now on a scientific mission to help scientists understand cloud formation and other atmospheric processes.
In other drone-related news, the Alaska Department of Transportation and Public Facilities has allowed some of its employees to receive licenses to operate drones to support projects involving roads, bridged and other structures. And a report has shown that drone funding fell by 64% in 2016 compared to 2015, with DJI dominating the market and issues such as battery life, connectivity problems and drone regulations stifling development efforts.
Robots in the water
From the skies to the sea: this May, NATO Nations agreed to use JANUS for their digital underwater communications. JANUS has the potential to make military and civilian, NATO and non-NATO, devices fully interoperable, doing away with the communication problems between systems and models by different manufacturers that have made underwater communication difficult up to this point.
Robots in the lab
Researchers at the Institute for Human and Machine Cognition in Pensacola, Florida, have developed a two-legged robot that can run without using sensors and a computer. The robot, called Planar Elliptical Runner, is stable through its physical design, which makes it different to other two or four-legged robots.
Meanwhile, a team at the University of California, Berkeley, has come up with a nimble-fingered robot that is able to pick up a wide range of objects using a 3D sensor and a deep learning neural network. It may not be perfect, but it’s the most nimble-fingered robot yet and the technology may find applications in picking and manufacturing in future.
Human-Machine Interaction
Most of the robots we interact with are practical helpers, offering support in the home, on the road or at work. Innovations some of us may have interacted with are autonomous lawn mowers, cars with self-driving features, or Amazon’s Alexa. With there plenty more robots in the pipeline.
A four-wheeled, waterproof, battery-powered robot inspector developed by a team in Nevada, may soon be supporting civil engineers and safety inspectors with bridge checks to reduce the chance of human error and omissions that could lead to a collapse such as the I-35W bridge disaster in Minnesota on 2007.
And finally, engineers in Germany have built a robot priest called BlessU-2 that can beam light from its hands and deliver blessings in five languages. The robot is meant to spark discussions about faith, the church and the potential of AI.
Learning resources: Robot Academy
And to finish off our digest for May, we wanted to highlight a new open online resource for robotics education: the Robot Academy. Developed by Professor Peter Corke and the Queensland University of Technology (QUT), the Academy offers more than 200 lessons from robot joint control architecture to limits of electric motor performance. So if you find yourself with a bit of time on your hands, why not try a Robot Academy Masterclass?
Or if you’re after something a little bit different, there’s a new toolkit on “Computational Abstractions for Interactive Design of Robotic Devices” that allows you to drag and drop parts of a virtual robot on screen without the need to know exactly what needs to connect to what as a complex physics engine ensures your robot won’t fail or fall over.
Missed any previous Digests? Click here to check them out.
A GelSight sensor attached to a robot’s gripper enables the robot to determine precisely where it has grasped a small screwdriver, removing it from and inserting it back into a slot, even when the gripper screens the screwdriver from the robot’s camera. Photo: Robot Locomotion Group at MIT
Eight years ago, Ted Adelson’s research group at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) unveiled a new sensor technology, called GelSight, that uses physical contact with an object to provide a remarkably detailed 3-D map of its surface.
Now, by mounting GelSight sensors on the grippers of robotic arms, two MIT teams have given robots greater sensitivity and dexterity. The researchers presented their work in two papers at the International Conference on Robotics and Automation last week.
In one paper, Adelson’s group uses the data from the GelSight sensor to enable a robot to judge the hardness of surfaces it touches — a crucial ability if household robots are to handle everyday objects.
In the other, Russ Tedrake’s Robot Locomotion Group at CSAIL uses GelSight sensors to enable a robot to manipulate smaller objects than was previously possible.
The GelSight sensor is, in some ways, a low-tech solution to a difficult problem. It consists of a block of transparent rubber — the “gel” of its name — one face of which is coated with metallic paint. When the paint-coated face is pressed against an object, it conforms to the object’s shape.
The metallic paint makes the object’s surface reflective, so its geometry becomes much easier for computer vision algorithms to infer. Mounted on the sensor opposite the paint-coated face of the rubber block are three colored lights and a single camera.
“[The system] has colored lights at different angles, and then it has this reflective material, and by looking at the colors, the computer … can figure out the 3-D shape of what that thing is,” explains Adelson, the John and Dorothy Wilson Professor of Vision Science in the Department of Brain and Cognitive Sciences.
In both sets of experiments, a GelSight sensor was mounted on one side of a robotic gripper, a device somewhat like the head of a pincer, but with flat gripping surfaces rather than pointed tips.
Contact points
For an autonomous robot, gauging objects’ softness or hardness is essential to deciding not only where and how hard to grasp them but how they will behave when moved, stacked, or laid on different surfaces. Tactile sensing could also aid robots in distinguishing objects that look similar.
In previous work, robots have attempted to assess objects’ hardness by laying them on a flat surface and gently poking them to see how much they give. But this is not the chief way in which humans gauge hardness. Rather, our judgments seem to be based on the degree to which the contact area between the object and our fingers changes as we press on it. Softer objects tend to flatten more, increasing the contact area.
The MIT researchers adopted the same approach. Wenzhen Yuan, a graduate student in mechanical engineering and first author on the paper from Adelson’s group, used confectionary molds to create 400 groups of silicone objects, with 16 objects per group. In each group, the objects had the same shapes but different degrees of hardness, which Yuan measured using a standard industrial scale.
Then she pressed a GelSight sensor against each object manually and recorded how the contact pattern changed over time, essentially producing a short movie for each object. To both standardize the data format and keep the size of the data manageable, she extracted five frames from each movie, evenly spaced in time, which described the deformation of the object that was pressed.
Yuan is joined on the paper by her two thesis advisors, Adelson and Mandayam Srinivasan, a senior research scientist in the Department of Mechanical Engineering; Chenzhuo Zhu, an undergraduate from Tsinghua University who visited Adelson’s group last summer; and Andrew Owens, who did his PhD in electrical engineering and computer science at MIT and is now a postdoc at the University of California at Berkeley.
Obstructed views
The paper from the Robot Locomotion Group was born of the group’s experience with the Defense Advanced Research Projects Agency’s Robotics Challenge (DRC), in which academic and industry teams competed to develop control systems that would guide a humanoid robot through a series of tasks related to a hypothetical emergency.
Typically, an autonomous robot will use some kind of computer vision system to guide its manipulation of objects in its environment. Such systems can provide very reliable information about an object’s location — until the robot picks the object up. Especially if the object is small, much of it will be occluded by the robot’s gripper, making location estimation much harder. Thus, at exactly the point at which the robot needs to know the object’s location precisely, its estimate becomes unreliable. This was the problem the MIT team faced during the DRC, when their robot had to pick up and turn on a power drill.
“You can see in our video for the DRC that we spend two or three minutes turning on the drill,” says Greg Izatt, a graduate student in electrical engineering and computer science and first author on the new paper. “It would be so much nicer if we had a live-updating, accurate estimate of where that drill was and where our hands were relative to it.”
That’s why the Robot Locomotion Group turned to GelSight. Izatt and his co-authors — Tedrake, the Toyota Professor of Electrical Engineering and Computer Science, Aeronautics and Astronautics, and Mechanical Engineering; Adelson; and Geronimo Mirano, another graduate student in Tedrake’s group — designed control algorithms that use a computer vision system to guide the robot’s gripper toward a tool and then turn location estimation over to a GelSight sensor once the robot has the tool in hand.
In general, the challenge with such an approach is reconciling the data produced by a vision system with data produced by a tactile sensor. But GelSight is itself camera-based, so its data output is much easier to integrate with visual data than the data from other tactile sensors.
In Izatt’s experiments, a robot with a GelSight-equipped gripper had to grasp a small screwdriver, remove it from a holster, and return it. Of course, the data from the GelSight sensor don’t describe the whole screwdriver, just a small patch of it. But Izatt found that, as long as the vision system’s estimate of the screwdriver’s initial position was accurate to within a few centimeters, his algorithms could deduce which part of the screwdriver the GelSight sensor was touching and thus determine the screwdriver’s position in the robot’s hand.
“I think that the GelSight technology, as well as other high-bandwidth tactile sensors, will make a big impact in robotics,” says Sergey Levine, an assistant professor of electrical engineering and computer science at the University of California at Berkeley. “For humans, our sense of touch is one of the key enabling factors for our amazing manual dexterity. Current robots lack this type of dexterity and are limited in their ability to react to surface features when manipulating objects. If you imagine fumbling for a light switch in the dark, extracting an object from your pocket, or any of the other numerous things that you can do without even thinking — these all rely on touch sensing.”
“Software is finally catching up with the capabilities of our sensors,” Levine adds. “Machine learning algorithms inspired by innovations in deep learning and computer vision can process the rich sensory data from sensors such as the GelSight to deduce object properties. In the future, we will see these kinds of learning methods incorporated into end-to-end trained manipulation skills, which will make our robots more dexterous and capable, and maybe help us understand something about our own sense of touch and motor control.”
VENTURER is the first Connected and Autonomous Vehicle project to start in the UK. The results of VENTURER preliminary trials show that the handover process is a safety critical issue in the development of Autonomous Vehicles (AVs).
The first VENTURER trials set out to investigate ‘takeover’ (time taken to reengage with vehicle controls) and ‘handover’ (time taken to regain a baseline/normal level of driving behaviour and performance) when switching frequently between automated and manual driving modes within urban and extra-urban settings. This trial is believed to be the first to directly compare handover to human driver-control from autonomous mode in both simulator and autonomous road vehicle platforms.
The handover process is important from a legal and insurance perspective – the length of time it takes people to regain full control of the vehicle represents a meaningful risk to insurers and understanding when control is transferred between the vehicle and the driver has liability implications.
David Williams from AXA outlined that, “The results of this trial have been very useful as we consider the issues that the handover process raises for insurers. Although some motor manufacturers have said they will skip SAE Level 3, some are progressing with vehicles that will require the driver to take back control of the vehicle. The insurance industry will need to assess the relative safety of the handover systems as they come to market but VENTURER’s trial 1 results show that with robust testing we can properly assess how humans and autonomous vehicles interact during this crucial phase of the technologies’ evolution.”
VENTURER designed, tested and analysed both simulator and road vehicle-based handover trials.
50 participants were tested in a simulator and/or in the autonomous vehicle on roads on UWE Bristol campus. The tests were at speeds of 20, 30, 40 and 50 mph in the simulator and 20 mph in the autonomous vehicle; speeds common in urban and extra-urban settings. Baseline driving behaviour of participants was also tested, and then the length of time it took them to return to this baseline following handover.
During the trial, the driver was aware that they might be alerted to take control of the vehicle at any moment, either due to the decisions made by the driver, or the capabilities of the vehicle in particular situations. VENTURER has classified this as planned handover.
The 20- and 30- mph scenarios involved town/city urban driving and the 40- and 50- mph scenarios involved outer-town/city extra-urban driving. Driving speed, lateral lane position, and braking behaviour (amongst other measures) were taken.
A key finding is that it took 2-3 seconds for participants to ‘takeover’ manual controls and resume active driving after short periods of autonomous driving in urban environments.
They also found that participants drove more slowly than the recommended speed limit for up to 55 seconds following a handover request, which suggests more cautious, but not necessarily safer, driving. This could be important for traffic management – if drivers on the road replicated this behaviour it might impact the flow of traffic and mitigate some of the predicated benefits of AVs.
Image: Local World
In addition, participants returned to their baseline manual driving behaviour after handover within 10-20 seconds, with most measures including speed, stabilising after 20-30 seconds. This was not the case within the highest speed simulator condition where stabilisation did not seem to occur on most measures within the 55 second measured handover period.
The team says these results have implications for the designers of autonomous vehicles with handback functionality, for example, in terms of phased handover systems. The results also inform the emerging market for insurance for autonomous vehicles.
Chair of the project, Lee Woodcock (Atkins) said, “The outcome of this research for trial one is significant and must provide food for thought as the market develops for driverless cars and how we progress through the different levels of automation. Further research must also explore interaction not just between vehicles but also with network operations and city management systems.”
Dr Phillip Morgan (UWE Bristol) said, “Designers need to proceed with caution and consider human performance under multiple driving conditions and scenarios in order to plot accurate takeover and handover time safety curves. In the time it takes for drivers to reach their baseline behaviour, the vehicle may have travelled some distance, depending on the speed. These initial trials show that there are some risk elements in the handover process and bigger studies with more participants may be needed to ensure there is sufficient data to build safe handover support systems.”
Professor Graham Parkhurst (UWE Bristol) said, “The results of these tests suggest that autonomous vehicles on highways should slow to a safe speed before handover is attempted. Further research is required to clarify what that safe speed is, but it would be substantially slower than the 70 mph motorway limit, and somewhat lower than the highest speed (50 mph) considered in our simulator trials.”
The trial clearly demonstrated that there were no major differences between control of the simulator and the Wildcat platforms used within the trial, validating the future use of simulators for the development of autonomous vehicles and associated technologies.
Levandowski (right) at MCE 2016. Source: Wikipedia Commons
Andy Rubin, who developed the Android operating system at Google then went on to lead Google through multiple acquisitions into robotics, has launched a new consumer products company. Anthony Levandowski, who, after many years with Google and their autonomous driving project, launched Otto which Uber acquired, was sued by Google, and just got fired by Uber.
People involved in robotics – from the multi-disciplined scientists turned entrepreneurs to all the specialists and other engineers involved in any aspect of the industry of making things robotic – are a relatively small community. Most people know (or know of) most of the others, and get-togethers like ICRA, the IEEE International Conference on Robotics and Automation, being held this week in Singapore, are an opportunity to meet new up-and-coming talent as they present their papers and product ideas and mingle with older, more established players. Two of those players made headline news this week: Rubin, launching Essential, and Levandowski, getting fired.
Andy Rubin
Rubin came to Google in 2005 when they acquired Android and left in 2014 to found an incubator for hardware startups, Playground Global. While at Google Rubin became an SVP of Mobile and Digital content including the open-source smartphone operating system Android and then started Google’s robotics group through a series of acquisitions. Android can be found in more than 2 billion phones, TVs, cars and watches.
2008 Google Developer Day in Japan – Press Conference: Andy Rubin
In 2007, Rubin was developing his own version of a smartphone at Google, also named Android, when Apple launched their iPhone, a much more capable and stylish device. Google’s product was scrapped but their software was marketed to HTC and their phone became Google’s first Android-based phone. The software was similar enough to Apple’s that Steve Jobs was furious and, as reported in Fred Vogelstein’s ‘Dogfight: How Apple and Google Went to War and Started a Revolution,’ called Rubin a “big, arrogant f–k” and “everything [he’s doing] is a f–king rip-off of what we’re doing.”
Jobs had trusted Google’s cofounders, Larry Page and Sergey Brin and Google’s CEO Eric Schmidt who was on Apple’s board. All three had been telling Jobs about Android, but they kept telling him it would be different from the iPhone. He believed them until he actually saw the phone and its software and how similar it was to the iPhone’s, whereupon he insisted Google make a lot of changes and removed Schmidt from Apple’s board. Rubin was miffed and had a sign on his office white board that said “STEVE JOBS STOLE MY LUNCH MONEY.”
Quietly, stealthily, Rubin went about creating “a new kind of company using 21st-century methods to build products for the way people want to live in the 21st century.” That company is Essential and Essential just launched and is taking orders for its new $699 phone and a still-stealthy home assistant to compete with Amazon’s Echo and Google’s Home devices.
Wired calls the new Essential Phone “the anti-iPhone.” The first Phones will ship in June.
Anthony Levandowski
In 2004, Levandowski and a team from UC Berkeley built and entered an autonomous motorcycle into the DARPA Grand Challenge. In 2007 he joined Google to work with Sebastian Thrun on Google Street View. Outside of Google he started a mobile mapping company that experimented with LiDAR technology and another to build a self-driving LiDAR-using a Prius. Google acquired both companies including their IP.
In 2016 Levandowski left Google to found Otto, a company making self-driving kits to retrofit semi-trailer trucks. Just as the kit was launched, Uber acquired Otto and Levandowski became the head of Uber’s driverless car operation in addition to continuing his work at Otto.
Quoting Wikipedia,
According to a February 2017 lawsuit filed by Waymo, the autonomous vehicle research subsidiary of Alphabet Inc, Levandowski allegedly “downloaded 9.7 GB of Waymo’s highly confidential files and trade secrets, including blueprints, design files and testing documentation” before resigning to found Otto.
In March 2017, United States District Judge William Haskell Alsup, referred the case to federal prosecutors after Levandowski exercised his Fifth Amendment right against self-incrimination. In May 2017, Judge Alsup ordered Levandowski to refrain from working on Otto’s LiDAR and required Uber to disclose its discussions on the technology. Levandowski was later fired by Uber for failing to cooperate in an internal investigation.
The device named “Spark” flew high above the man on stage with his hands waving in the direction of the flying object. In a demonstration of DJI’s newest drone, the audience marveled at the Coke can-sized device’s most compelling feature: gesture controls. Instead of a traditional remote control, this flying selfie machine follows hand movements across the sky. Gestures are the most innate language of mammals, and including robots in our primal movements means we have reached a new milestone of co-existence.
Madeline Gannon of Carnegie Mellon University is the designer of Mimus, a new gesture controlled robot featured in an art installation at The Design Museum in London, England. Gannon explained: “In developing Mimus, we found a way to use the robot’s body language as a medium for cultivating empathy between museum-goers and a piece of industrial machinery. Body language is a primitive yet fluid means of communication that can broadcast an innate understanding of the behaviors, kinematics and limitations of an unfamiliar machine.” Gannon wrote about her experiences recently in the design magazine Dezeen: “In a town like Pittsburgh, where crossing paths with a driverless car is now an everyday occurrence, there is still no way for a pedestrian to read the intentions of the vehicle…it is critical that we design more effective ways of interacting and communicating with them.”
So far, the biggest commercially deployed advances in human-robot interactions have been conversational agents by Amazon, Google and Apple. While natural language processing has broken new ground in artificial intelligence, the social science of its acceptability in our lives might be its biggest accomplishment. Japanese roboticist Masahiro Mori described the danger of making computer generated voices too indistinguishable from humans as the “uncanny valley.” Mori cautioned inventors from building robots that are too human sounding (and possibly looking) as the result elicits negative emotions best described as “creepy” and “disturbing.”
Recently, many toys have embraced conversational agents as a way of building greater bonds and increasing the longevity of play with kids. Barbie’s digital speech scientist, Brian Langner of ToyTalk, detailed his experiences with crossing into the “Uncanny Valley” as: “Jarring is the way I would put it. When the machine gets some of those things correct, people tend to expect that it will get everything correct.”
Kate Darling of MIT’s Media Lab, whose research centers on human-robot interactions, suggested that “if you get the balance right, people will like interacting with the robot, and will stop using it as a device and start using it as a social being.”
This logic inspired Israeli startup Intuition Robotics to create ElliQ—a bobbing head (eyeless) robot. The purpose of the animatronics is to break down barriers between its customer base of elderly patients and their phobias of technology. According to Intuition Robotics’ CEO, Dor Skuler, the range of motion coupled with a female voice helps create a bond between the device and its user. Don Norman, usability designer of ElliQ, said: “It looks like it has a face even though it doesn’t. That makes it feel approachable.”
Mayfield Robotics decided to add cute R2D2-like sounds to its newest robot, Kuri. Mayfield hired former Pixar designers Josh Morenstein and Nick Cronan of Branch Creative with the sole purpose of making Kuri more adorable. To accomplish this mission, Morenstein and Cronan gave Kuri eyes, but not a mouth as that would be, in their words “creepy.” Conan shares the challenges with designing the eyes: “Just by moving things a few millimeters, it went from looking like a dumb robot to a curious robot to a mean robot. It became a discussion of, how do we make something that’s always looking optimistic and open to listen to you?” Kuri has a remarkable similarity to Morenstein and Cronan’s former theatrical robot, EVA.
In the far extreme of making robots act and behave human, RealDoll has been promoting six thousand dollar sex robots. To many, RealDoll has crossed the “Uncanny Valley” of creepiness with sex dolls that look and talk like humans. In fact, there is a growing grassroots campaign to ban RealDoll’s products globally, as it endangers the very essence of human relationships. Florence Gildea writes on the organization’s blog: “The personalities and voices that doll owners project onto their dolls is pertinent for how sex robots may develop, given that sex doll companies like RealDoll are working on installing increasing AI capacities in their dolls and the expectation that owners will be able to customize their robots’ personalities.” The example given is how the doll expresses her “feelings” for her owner on Twitter:
Obviously a robot companion has no feelings, however it is a projection of the doll owners’. “To anthropomorphize their dolls to sustain the fantasy that they have feelings for the owner. The Twitter accounts seemingly manifest the dolls’ independent existence so that their dependence on their owners can seem to signify their emotional attachment, rather than it following inevitable from their status as objects. Immobility, then, can be misread as fidelity and devotion.” The implications of this behavior is that their female companion, albeit mechanical, enjoys “being dominated.” The fear that the Campaign Against Sex Robots expresses is the objectification of women (even robotic ones) reinforces problematic human sexual stereotypes.
Today, with technology at our fingertips, there is growing phenomena of preferring one-directional device relationships over complicated human encounters. MIT Social Sciences Professor Sherry Turkle writes in her essay,Close Engagements With Artificial Companionship, that “over-stressed, overworked, people claim exhaustion and overload. These days people will admit they’d rather leave a voice mail or send an email than talk face-to-face. And from there, they say: ‘I’d rather talk to the robot. Friends can be exhausting. The robot will always be there for me. And whenever I’m done, I can walk away.’”
In the coming years humans will communicate more with robots in their lives from experiences in the home to the office to their leisure time. The big question will be not the technical barriers, but the societal norms that will evolve to accept Earth’s newest species.
“What do we think a robot is?” asked robot designer Norm. “Some people think it should look like an animal or a person, and it should move around. Or it just has to be smart, sense the environment, and have motors and controllers.”
Norm’s answer, like beauty, could be in the eye of the beholder.
New algorithms power a prototype system for helping visually impaired users avoid obstacles and identify objects. Courtesy of the researchers.
Computer scientists have been working for decades on automatic navigation systems to aid the visually impaired, but it’s been difficult to come up with anything as reliable and easy to use as the white cane, the type of metal-tipped cane that visually impaired people frequently use to identify clear walking paths.
White canes have a few drawbacks, however. One is that the obstacles they come in contact with are sometimes other people. Another is that they can’t identify certain types of objects, such as tables or chairs, or determine whether a chair is already occupied.
Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a new system that uses a 3-D camera, a belt with separately controllable vibrational motors distributed around it, and an electronically reconfigurable Braille interface to give visually impaired users more information about their environments.
“We did a couple of different tests with blind users,” says Robert Katzschmann, a graduate student in mechanical engineering at MIT and one of the paper’s two first authors. “Having something that didn’t infringe on their other senses was important. So we didn’t want to have audio; we didn’t want to have something around the head, vibrations on the neck — all of those things, we tried them out, but none of them were accepted. We found that the one area of the body that is the least used for other senses is around your abdomen.”
Katzschmann is joined on the paper by his advisor Daniela Rus, an Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science; his fellow first author Hsueh-Cheng Wang, who was a postdoc at MIT when the work was done and is now an assistant professor of electrical and computer engineering at National Chiao Tung University in Taiwan; Santani Teng, a postdoc in CSAIL; Brandon Araki, a graduate student in mechanical engineering; and Laura Giarré, a professor of electrical engineering at the University of Modena and Reggio Emilia in Italy.
Parsing the world
The researchers’ system consists of a 3-D camera worn in a pouch hung around the neck; a processing unit that runs the team’s proprietary algorithms; the sensor belt, which has five vibrating motors evenly spaced around its forward half; and the reconfigurable Braille interface, which is worn at the user’s side.
The key to the system is an algorithm for quickly identifying surfaces and their orientations from the 3-D-camera data. The researchers experimented with three different types of 3-D cameras, which used three different techniques to gauge depth but all produced relatively low-resolution images — 640 pixels by 480 pixels — with both color and depth measurements for each pixel.
The algorithm first groups the pixels into clusters of three. Because the pixels have associated location data, each cluster determines a plane. If the orientations of the planes defined by five nearby clusters are within 10 degrees of each other, the system concludes that it has found a surface. It doesn’t need to determine the extent of the surface or what type of object it’s the surface of; it simply registers an obstacle at that location and begins to buzz the associated motor if the wearer gets within 2 meters of it.
Chair identification is similar but a little more stringent. The system needs to complete three distinct surface identifications, in the same general area, rather than just one; this ensures that the chair is unoccupied. The surfaces need to be roughly parallel to the ground, and they have to fall within a prescribed range of heights.
Tactile data
The belt motors can vary the frequency, intensity, and duration of their vibrations, as well as the intervals between them, to send different types of tactile signals to the user. For instance, an increase in frequency and intensity generally indicates that the wearer is approaching an obstacle in the direction indicated by that particular motor. But when the system is in chair-finding mode, for example, a double pulse indicates the direction in which a chair with a vacant seat can be found.
The Braille interface consists of two rows of five reconfigurable Braille pads. Symbols displayed on the pads describe the objects in the user’s environment — for instance, a “t” for table or a “c” for chair. The symbol’s position in the row indicates the direction in which it can be found; the column it appears in indicates its distance. A user adept at Braille should find that the signals from the Braille interface and the belt-mounted motors coincide.
In tests, the chair-finding system reduced subjects’ contacts with objects other than the chairs they sought by 80 percent, and the navigation system reduced the number of cane collisions with people loitering around a hallway by 86 percent.
May 2017 had two robotics-related companies get $9.5 billion in funding and 22 others raised $249 million. Acquisitions also continued to be substantial with Toyota Motor’s $260 million acquisition of Bastian Solutions plus three others (where the amounts weren’t disclosed).
Fundings
Didi Chuxing, the Uber of China, raised $5.5 billion in a round led by SoftBank with new investors Silver Lake Kraftwerk joining previous investors SoftBank, China Merchants Bank and Bank of Communications. According to TechCrunch, this latest round brings the total raised by DiDi to about $13 billion. Uber, by comparison, has raised $8.81 billion.
Nvidia Corp, a Santa Clara, CA-based speciality GPU maker, raised $4 billion (representing a 4.9% stake in the company) according to Bloomberg. Nvidia’s newest chips are focused on providing power for deep learning for self-driving vehicles.
ClearMotion, a Woburn, MA automotive technology startup that’s building shock absorbers with robotic, software-driven adaptive actuators for car stability, has raised $100 million in a Series C round led by a group of JP Morgan clients and NEA, Qualcomm Ventures and more.
Echodyne, a Bellevue, WA developer of radar vision technology used in drones and self-driving cars, has raised $29 million in a Series B round led by New Enterprise Associates and joined by Bill Gates, Madrona Venture Group, and others.
DeepMap, a Silicon Valley mapping startup, raised $25 million in a round led by Accel and included GSR Ventures and Andreessen Horowitz.
“Autonomous vehicles are tempting us with a radically new future. However, this level of autonomy requires a highly sophisticated mapping and localization infrastructure that can handle massive amounts of data. I’m very excited to work with the DeepMap team, who have the requisite expertise in mapping, vision, and large scale operations, as they build the core technology that will fuel the next generation of transportation,” said Martin Casado, general partner at Andreessen Horowitz.
Hesai Photonics Technology, a transplanted Silicon Valley-to-Shanghai sensor startup, raised $16 million in a Series A round led by Pagoda Investment with participation from Grains Valley Venture Capital, Jiangmen Venture Capital and LightHouse Capital Management. Hesai is developing a hybrid LiDAR device for self-driving cars. Hesai has already partnered with a number of autonomous driving technology and car companies including Baidu’s Chinese electric vehicle start-up NIO and self-driving tech firm UiSee.
Abundant Robotics, a Menlo Park, CA-based automated fruit-picking tech developer, raised $10 million in venture funding. GV (Google Ventures) led the round, and was joined by BayWa AG and Tellus Partners. Existing partners Yamaha Motor Company, KPCB Edge, and Comet Labs also participated.
TriLumina Corp., an Albuquerque, NM-based developer of solid-state automotive LiDAR illumination for ADAS and autonomous driving, closed a $9 million equity and debt financing. Backers included new investors Kickstart Seed Fund and existing stakeholders Stage 1 Ventures, Cottonwood Technology Fund, DENSO Ventures and Sun Mountain Capital.
Bowery Farming, a NYC indoor vertical farm startup, raised $7.5 million (in February) in a seed round led by First Round Capital and including Box Group, Homebrew, Flybridge, Red Swan, RRE, Lerer Hippeau Ventures, and Tom Colicchio – a restauranteur and judge on reality cooking show, Top Chef.
Taranis, an Israel-based precision agriculture intelligence platform raised $7.5 million in Series A funding. Finistere Ventures led the round, and was joined by Vertex Ventures. Existing investors Eshbol Investments, Mindset Ventures, OurCrowd, and Eyal Gura participated.
Ceres Imaging, the Oakland, CA aerial imagery and analytics company, raised a $5 million Series A round of funding led by Romulus Capital.
Stanley Robotics, a Paris-based automated valet parking service developer, raised $4 million in funding. Investors included Elaia Partners, Idinvest Partners and Ville de Demain. Stanley’s new parking robot is a mobile car-carrying lift that moves and tightly parks cars in outdoor locations.
AIRY3D Inc, a Canadian start-up in 3D computer vision, raised $3.5 million in a seed round co-led by CRCM Ventures and R7 Partners. Other investors include WI Harper Group, Robert Bosch Venture Capital, Nautilus Venture Partners and several angel investors that are affiliates of TandemLaunch, the Montreal-based incubator that spun out AIRY3D.
SkyX Systems, a Canada-based unmanned aircraft system developer, raised around $3 million in funding from Kuang-Chi Group.
Catalia Health, a San Francisco-based patient care management company applying robotics to improve personal health, raised $2.5 million in funding. Khosla Ventures led the round.
vHive, an Israeli startup developing software to operate autonomous drone fleets, raised $2 million (in April) in an A round led by StageOne VC and several additional private investors.
Vivacity Labs, a London AI tech and sensor startup, raised $2 million from Tracsis, Downing Ventures and the London Co-Investment Fund and was also granted an additional $1.3 million from Innovate UK to create sensors with built-in machine learning to identify individual road users and manage traffic accordingly.
Bluewrist, a Canadian integrator of vision systems, raised around $1.5 million (in February) from Istuary Toronto Capital.
American Robotics, a Boston-based commercial farming drone system and analytics developer, raised $1.1 million in seed funding. Investors included Brain Robotics Capital.
Kubo, a Danish educational robot startup, raised around $1 million from the Danish Growth Fund. Kubo is an educational robot that helps kids learn coding, math, language and music in a screenless, tangible environment.
Zeals, a Japanese startup which produces interaction software for robots such as Palmi and Sota, has closed a $720k investment from Japanese adtech firm FreakOut Holdings.
Kitty Hawk, a San Francisco drone platform startup, raised $600k in seed money in March from The Flying Object VC.
Kraken Sonar, a Newfoundland marine tech startup, raised around $500k from RDC, a provincial Crown corporation responsible for improving Newfoundland and Labrador’s research and development. The funding will be used to develop the ThunderFish program which will combine smart sonar, laser and optical sensors, advanced pressure tolerant battery and thruster technologies and cutting edge artificial intelligence algorithms integrated onboard a cost effective AUV capable of 20,000 foot depths.
Motörleaf, a Canadian ag sensor, communications and software startup, raised an undisclosed amount in a seed round (in March).
Acquisitions
Toyota Motor Corp paid $260 million to acquire Bastian Solutions, a U.S.-based materials handling systems integrator. Toyota is the world’s No. 1 forklift truck manufacturer in terms of global market share. With this acqisition Toyota is making a “full-scale entry” into the North American logistics technology sector and will also use Bastian’s systems to make its own global supply chain more efficient.
Ctrl.Me Robotics, a Hollywood, CA drone startup, was acquired by Snap, Inc. for “an amount less than $1 million.” Ctrl Me developed a system for capturing movie-quality aerial video but was recently winding down its operations. Snap acquired its assets and technology as well as talent. Snap was already using one of Ctrl.me’s products: Spectacles, which captures video for Snap’s mobile app.
Applied Research Associates, Inc. (ARA), an employee-owned scientific research and engineering company, acquired Neya Systems LLC on April 28, 2017. Neya Systems LLC is known for their development of unmanned systems for defense, homeland security, and commercial users. Terms of the deal were not disclosed.
Trimble has acquired Müller-Elektronik and all its subsidiary companies, for an undisclosed amount. Müller is a German manufacturer and integrator of farm implement controls, steering kits and precision farming solutions. The transaction is expected to close in the Q3 2017. Financial terms were not disclosed. Müller was key in the development of the ISOBUS communication protocol found in most tractors and towed implements, which allows one terminal to control several implements and machines, regardless of manufacturer.
IPOs
Gamma 2 Robotics, a security robot maker, launched a $6 million private offering to accredited investors.
Aquabotix, a Fall River, MA-headquartered company, raised $5.5 million from their IPO of UUV (ASX:UUV) on the Australian Securities Exchange (ASX). Aquabotix manufactures commercial and industrial underwater drone/camera systems and has shipped over 350 units worldwide.
As long as the possibilities can be accurately simulated, these algorithms can explore and learn from a vastly larger space of outcomes than will ever be possible for humans. Thus DeepMind’s bold claims seem likely to be realised, and as the company says, we can’t wait to see what comes next.
