The Second Edition of the award-winning Springer Handbook of Robotics edited by Bruno Siciliano and Oussama Khatib has recently been published. The contents of the first edition have been restructured to achieve four main objectives: the enlargement of foundational topics for robotics, the enlightenment of design of various types of robotic systems, the extension of the treatment on robots moving in the environment, and the enrichment of advanced robotics applications. Most previous chapters have been revised, fifteen new chapters have been introduced on emerging topics, and a new generation of authors have joined the handbook’s team. Like for the first edition, a truly interdisciplinary approach has been pursued in line with the expansion of robotics across the boundaries with related disciplines. Again, the authors have been asked to step outside of their comfort zone, as the Editorial Board have teamed up authors who never worked together before.
No doubt one of the most innovative elements is the inclusion of multimedia content to leverage the valuable written content inside the book. Under the editorial leadership of Torsten Kröger, a web portal has been created to host the Multimedia Extension of the book, which serves as a quick one-stop shop for more than 700 videos associated with the specific chapters. In addition, video tutorials have been created for each of the seven parts of the book, which benefit everyone from PhD students to seasoned robotics experts who have been in the field for years. A special video related to the contents of the first chapter shows the journey of robotics with the latest and coolest developments in the last 15 years. As publishing explores new interactive technologies, an App has been made available in the Google/IOS stores to introduce an additional multimedia layer to the reader’s experience. With the app, readers can use the camera on their smartphone or tablet, hold it to a page containing one or more special icons, and produce an augmented reality on the screen, watching videos as they read along the book.
The Multimedia Portal offers free access to more than 700 accompanying videos. In addition, a Multimedia App is now downloadable: AppStore and GooglePlay for smartphones and tablets, allowing you to easily access multimedia content while reading the book.
In 2016, the European Union co-funded 17 new robotics projects from the Horizon 2020 Framework Programme for research and innovation. 16 of these resulted from the robotics work programme, and 1 project resulted from the Societal Challenges part of Horizon 2020. The robotics work programme implements the robotics strategy developed by SPARC, the Public-Private Partnership for Robotics in Europe (see the Strategic Research Agenda).
Every week, euRobotics will publish a video interview with a project, so that you can find out more about their activities. This week features CYBERLEGs++: The CYBERnetic LowEr-Limb CoGnitive Ortho-prosthesis Plus Plus.
Objectives
The goal of CYBERLEGs++ is to validate the technical and economic viability of the powered robotic ortho-prosthesis developed within the FP7-ICT-CYBERLEGs project. The aim is to enhance/restore the mobility of transfemoral amputees and to enable them to perform locomotion tasks such as ground-level walking, walking up and down slopes, climbing/descending stairs, standing up, sitting down and turning in scenarios of real life. Restored mobility will allow amputees to perform physical activity thus counteracting physical decline and improving the overall health status and quality of life.
Expected Impact
By demonstrating in an operational environment (TRL=7) – from both the technical and economic viability view point – a modular robotics technology for healthcare, with the ultimate goal of fostering its market exploitation CYBERLEGs Plus Pus will have an impact on:
Society: CLs++ technology will contribute to increase the mobility of dysvascular amputees, and, more generally, of disabled persons with mild lower-limb impairments;
Science and technology: CLs++ will further advance the hardware and software modules of the ortho-prosthesis developed within the FP7 CYBERLEGs project and validate its efficacy through a multi-centre clinical study;
Market: CLs++ will foster the market exploitation of high-tech robotic systems and thus will promote the growth of both a robotics SME and a large healthcare company.
Partners
SCUOLA SUPERIORE SANT’ANNA (SSSA)
UNIVERSITÉ CATHOLIQUE DE LOUVAIN (UCL)
VRIJE UNIVERSITEIT BRUSSEL (VUB)
UNIVERZA V LJUBLJANI (UL)
FONDAZIONE DON CARLO GNOCCHI (FDG)
ÖSSUR (OSS)
IUVO S.R.L. (IUVO)
Coordinator
Prof. Nicola Vitiello, The BioRobotics Institute
Scuola Superiore Sant’Anna, Pisa, Italy
nicola.vitiello@santannapisa.it
The need for fast, accurate 3D mapping solutions has quickly become a reality for many industries wanting to adopt new technologies in AI and automation. New applications requiring these 3D mapping platforms include surveillance, mining, automated measurement & inspection, construction management & decommissioning, and photo-realistic rendering. Here at Clearpath Robotics, we decided to team up with Mandala Robotics to show how easily you can implement 3D mapping on a Clearpath robot.
3D Mapping Overview
3D mapping on a mobile robot requires Simultaneous Localization and Mapping (SLAM), for which there are many different solutions available. Localization can be achieved by fusing many different types of pose estimates. Pose estimation can be done using combinations of GPS measurements, wheel encoders, inertial measurement units, 2D or 3D scan registration, optical flow, visual feature tracking and others techniques. Mapping can be done simultaneously using the lidars and cameras that are used for scan registration and for visual position tracking, respectively. This allows a mobile robot to track its position while creating a map of the environment. Choosing which SLAM solution to use is highly dependent on the application and the environment to be mapped. Although many 3D SLAM software packages exist and cannot all be discussed here, there are few 3D mapping hardware platforms that offer full end-to-end 3D reconstruction on a mobile platform.
Existing 3D Mapping Platforms
We will briefly highlight some more popular alternatives of commercialized 3D mapping platforms that have one or many lidars, and in some cases optical cameras, for point cloud data collection. It is important to note that there are two ways to collect a 3D point cloud using lidars:
1. Use a 3D lidar which consists of one device with multiple stacked horizontally laser beams
2. Tilt or rotate a 2D lidar to get 3D coverage
Tilting of a 2D lidar typically refers to back-and-forth rotating of the lidar about its horizontal plane, while rotating usually refers to continuous 360 degree rotation of a vertically or horizontally mounted lidar.
Example 3D Mapping Platforms: 1. MultiSense SL (Left) by Carnegie Robotics, 2. 3DLS-K (Middle) by Fraunhofer IAIS Institute, 3. Cartographer Backpack (Right) by Google.
1. MultiSense SL
The MultiSense SL was developed by Carnegie Robotics and provides a compact and lightweight 3D data collection unit for researchers. The unit has a tilting Hokuyo 2D lidar, a stereo camera, LED lights, and is pre-calibrated for the user. This allows for the generation of coloured point clouds. This platform comes with a full software development kit (SDK), open source ROS software, and is the sensor of choice for the DARPA Robotics Challenge for humanoid robots.
2. 3DLS-K
The 3DLS-K is a dual-tilting unit made by Fraunhofer IAIS Institute with the option of using SICK LMS-200 or LMS-291 lidars. Fraunhofer IAIS also offers other configurations with continuously rotating 2D SICK or Hokuyo lidars. These systems allow for the collection of non-coloured point clouds. With the purchase of these units, a full application program interface (API) is available for configuring the system and collecting data.
3. Cartographer Backpack
The Cartographer Backpack is a mapping unit with two static Hokuyo lidars (one horizontal and one vertical) and an on-board computer. Google released cartographer software as an open source library for performing 3D mapping with multiple possible sensor configurations. The Cartographer Backpack is an example of a possible configuration to map with this software. Cartographer allows for integration of multiple 2D lidars, 3D lidars, IMU and cameras, and is also fully supported in ROS. Datasets are also publicly available for those who want to see mapping results in ROS.
Mandala Mapping – System Overview
Thanks to the team at Mandala Robotics, we got our hands on one of their 3D mapping units to try some mapping on our own. This unit consists of a mount for a rotating vertical lidar, a fixed horizontal lidar, as well as an onboard computer with an Nvidia GeForce GTX 1050 Ti GPU. The horizontal lidar allows for the implementation of 2D scan registration as well as 2D mapping and obstacle avoidance. The vertical rotating lidar is used for acquiring the 3D point cloud data. In our implementation, real-time SLAM was performed solely using 3D scan registration (more on this later) specifically programmed for full utilization of the onboard GPU. The software used to implement this mapping can be found on the mandala-mapping github repository.
Scan registration is the process of combining (or stitching) together two subsequent point clouds (either in 2D or 3D) to estimate the change in pose between the scans. This results in motion estimates to be used in SLAM and also allows a new point cloud to be added to an existing in order to build a map. This process is achieved by running iterative closest point (ICP) between the two subsequent scans. ICP performs a closest neighbour search to match all points from the reference scan to a point on the new scan. Subsequently, optimization is performed to find rotation and translation matrices that minimise the distance between the closest neighbours. By iterating this process, the result converges to the true rotation and translation that the robot underwent between the two scans. This is the process that was used for 3D mapping in the following demo.
Mandala Robotics has also released additional examples of GPU computing tasks useful for robotics and SLAM. These examples can be found here.
Mandala Mapping Results
The following video shows some of our results from mapping areas within the Clearpath office, lab and parking lot. The datasets collected for this video can be downloaded here.
The Mandala Mapping software was very easy to get up and running for someone with basic knowledge in ROS. There is one launch file which runs the Husky base software as well as the 3D mapping. Initiating each scan can be done by sending a simple scan request message to the mapping node, or by pressing one button on the joystick used to drive the Husky. Furthermore, with a little more ROS knowledge, it is easy to incorporate autonomy into the 3D mapping. Our forked repository shows how a short C++ script can be written to enable constant scan intervals while navigating in a straight line. Alternatively, one could easily incorporate 2D SLAM such as gmapping together with the move_base package in order to give specific scanning goals within a map.
Why use Mandala Mapping on your robot?
If you are looking for a quick and easy way to collect 3D point clouds, with the versatility to use multiple lidar types, then this system is a great choice. The hardware work involved with setting up the unit is minimal and well documented, and it is preconfigured to work with your Clearpath Husky. Therefore, you can be up and running with ROS in a few days! The mapping is done in real time, with only a little lag time as your point cloud size grows, and it allows you to visualize your map as you drive.
The downside to this system, compared to the MultiSense SL for example, is that you cannot yet get a coloured point cloud since no cameras have been integrated into this system. However, Mandala Robotics is currently in the beta testing stage for a similar system with an additional 360 degree camera. This system uses the Ladybug5 and will allow RGB colour to be mapped to each of the point cloud elements. Keep an eye out for a future Clearpath blogs in case we get our hands on one of these systems! All things considered, the Mandala Mapping kit offers a great alternative to the other units aforementioned and fills many of the gaps in functionality of these systems.
In the race to develop self-driving technology, Chinese Internet giant Baidu unveiled its 50+ partners in an open source development program, revised its timeline for introducing autonomous driving capabilities on open city roads, described the Project Apollo consortium and its goals, and declared Apollo to be the ‘Android of the autonomous driving industry’.
At a developer's conference last week in Beijing, Baidu described its plans and timetable for its self-driving car technology. It will start test-driving in restricted environments immediately, before gradually introducing fully autonomous driving capabilities on highways and open city roads by 2020. Baidu's goal is to get those vehicles on the roads in China, the world's biggest auto market, with the hope that the same technology, embedded in exported Chinese vehicles, can then conquer the United States. To do so, Baidu has compiled a list of cooperative partners, a consortium of 50+ public and private entities, and named it Apollo, after NASA's massive Apollo moon-landing program.
Project Apollo
The program is making its autonomous car software open source in the same way that Google released its Android operating system for smartphones. By encouraging companies to build upon the system and share their results, it hopes to overtake rivals such as Google/Waymo, Tencent, Alibaba and others researching self-driving technology.
The Apollo platform consists of a core software stack, a number of cloud services, and self-driving vehicle hardware such as GPS, cameras, lidar, and radar.
The software currently available to outside developers is relatively simple: it can record the behavior of a car being driven by a person and then play that back in autonomous mode. This November, the company plans to release perception capabilities that will allow Apollo cars to identify objects in their vicinity. This will be followed by planning and localization capabilities, and a driver interface.
The cloud services being developed by Baidu include mapping services, a simulation platform, a security framework, and Baidu’s DuerOS voice-interface technology.
Members of the project include Chinese automakers Chery, Dongfeng Motor, Foton, Nio, Yiqi and FAW Group. Tier 1 members include Continental, Bosch, Intel, Nvidia, Microsoft and Velodyne. Other partners include Chinese universities, governmental agencies, Autonomous Stuff, TomTom, Grab and Ford. The full list of members can be seen here.
Quoting from Bloomberg News regarding the business aspect of Project Apollo:
China has set a goal for 10 to 20 percent of vehicles to be highly autonomous by 2025, and for 10 percent of cars to be fully self-driving in 2030. Didi Chuxing, the ride-sharing app that beat Uber in China, is working on its own product, as are several local automakers. It’s too early to tell which will ultimately succeed though Baidu’s partnership approach is sound, said Marie Sun, an analyst with Morningstar Investment Service.
“This type of technology needs cooperation between software and hardware from auto-manufacturers so it’s not just Baidu that can lead this,” she said. If Baidu spins off the car unit, “in the longer term, Baidu should maintain a major shareholder position so they can lead the growth of the business.”
Baidu and Apollo have a significant advantage over Google's Waymo: Baidu has a presence in the United States, whereas Alphabet has none in China because Google closed down its search site in 2010 rather than give in to China's internet censorship.
Strategic Issue
According to the Financial Times, “autonomous vehicles pose an existential threat [to global car manufacturers]. Instead of owning cars, consumers in the driverless age will simply summon a robotic transportation service to their door. One venture capitalist says auto executives have come to him saying they know they are “screwed”, but just want to know when it will happen.”
This desperation has prompted a string of big acquisitions and joint ventures amongst competing providers including those in China. Citing just a few:
Last year GM paid $1bn for Cruise, a self-driving car start-up.
Uber paid $680m for Otto, an autonomous trucking company that was less than a year old.
In March, Intel spent $15bn to buy Israel’s Mobileye, which makes self-driving sensors and software.
Baidu acquired Raven Tech, an Amazon Echo competitor; 8i, an augmented reality hologram startup; Kitt, a conversational language engine; and XPerception, a vision systems developer.
Tencent invested in mapping provider Here and acquired 5% of Tesla.
Alibaba announced that it is partnering with Chinese Big 4 carmaker SAIC in their self-driving effort.
China Network
Baidu’s research team in Silicon Valley is pivotal to their goals. Baidu was one of the first of the Chinese companies to set up in Silicon Valley, initially to tap into SV's talent pool. Today it is the center of a “China network” of almost three dozen firms, through investments, acquisitions and partnerships.
Baidu is rapidly moving forward from the SV center:
It formed a self-driving car sub-unit in April which now employs more than 100 researchers and engineers.
It partnered with chipmaker Nvidia.
It acquired vision systems startup XPerception.
It has begun testing its autonomous vehicles in China and California.
Regarding XPerception, Gartner research analyst Michael Ramsey said in a CNBC interview:
“XPerception has expertise in processing and identifying images, an important part of the sensing for autonomous vehicles. The purchase may help push Baidu closer to the leaders, but it is just one piece.”
XPerception is just one of many Baidu puzzle pieces intended to bring talent and intellectual property to the Apollo project. It acquired Raven Tech and Kitt AI to gain conversational transaction processing. It acquired 8i, an augmented reality hologram startup, to add AR — which many expect to be crucial in future cars — to the project. And it suggested that the acquisition spree will continue as needed.
Bottom Line
China has set a goal for 10 to 20 percent of vehicles to be highly autonomous by 2025, and for 10 percent of cars to be fully self-driving in 2030 and Baidu wants to provide the technology to get those vehicles on the roads in China with the hope that the same technology, embedded in exported Chinese vehicles, can then conquer the United States. It seems well poised to do so.
In this episode, MeiXing Dong conducts interviews at the 2017 Midwest Speech and Language Days workshop in Chicago. She talks with Michael White of Ohio State University about question interpretation in a dialogue system; Dmitriy Dligach of Loyola University Chicago about extracting patient timelines from doctor’s notes; and Denis Newman-Griffiths of Ohio State University about connecting words and phrases to relevant medical topics.
Mike Salem from Udacity’s Robotics Nanodegree is hosting a series of interviews with professional roboticists as part of their free online material.
This week we’re featuring Mike’s interview with Nick Kohut, Co-Founder and CEO of Dash Robotics.
Nick is a former robotics postdoc at Stanford and received his PhD in Control Systems from UC Berkeley. At Dash Robotics, Nick handles team-building and project management.
You can find all the interviews here. We’ll be posting one per week on Robohub.
China’s second-biggest e-commerce company, JD.com (Alibaba is first), is testing mobile robots to make deliveries to its customers, and imagining a future with fully unmanned logistics systems.
On the last day of a two-week-long shopping bonanza that recorded sales of around $13 billion, some deliveries were made using mobile robots designed by JD. It’s the first time that the company has used delivery robots in the field. The bots delivered packages to multiple Beijing university campuses such as Tsinghua University and Renmin University.
JD has been testing delivery robots since November last year. At that time, the cost of a single robot was almost $88,000.
They have been working on lowering the cost and increasing the capabilities since then. The white, four-wheeled UGVs can carry five packages at once and travel 13 miles on a charge. They can climb up a 25° incline and find the shortest route from warehouse to destination.
Once it reaches its destination, the robot sends a text message to notify the recipient of the delivery. Users can accept the delivery through face-recognition technology or by using a code.
The UGVs now cost $7,300 per robotic unit which JD figures can reduce delivery costs from less than $1 for a human delivery to about 20 cents for a robot delivery.
JD is also testing the world’s largest drone-delivery network, including flying drones carrying products weighing as much as 2,000 pounds.
“Our logistics systems can be unmanned and 100% automated in 5 to 8 years,” said Liu Qiangdong, JD’s chairman.
We are excited to show off a simulation of a Prius in Mcity using ROS Kinetic and Gazebo 8. ROS enabled the simulation to be developed faster by using existing software and libraries. The vehicle’s throttle, brake, steering, and transmission are controlled by publishing to a ROS topic. All sensor data is published using ROS, and can be visualized with RViz.
We leveraged Gazebo’s capabilities to incorporate existing models and sensors.
The world contains a new model of Mcity and a freeway interchange. There are also models from the gazebo models repository including dumpsters, traffic cones, and a gas station. On the vehicle itself there is a 16 beam lidar on the roof, 8 ultrasonic sensors, 4 cameras, and 2 planar lidar.
The simulation is open source and available at on GitHub at osrf/car_demo. Try it out by installing nvidia-docker and pulling “osrf/car_demo” from Docker Hub. More information about building and running is available in the README in the source repository.
In episode four of season three Neil introduces us to the ideas behind the bias variance dilemma (and how how we can think about it in our daily lives). Plus, we answer a listener question about how to make sure your neural networks don’t get fooled. Our guest for this episode is Jeff Dean, Google Senior Fellow in the Research Group, where he leads the Google Brain project. We talk about a closet full of robot arms (the arm farm!), image recognition for diabetic retinopathy, and equality in data and the community.
June, 2017 saw two robotics-related companies get $50 million each and 17 others raised $248 million for a monthly total of $348 million. Acquisitions also continued to be substantial with SoftBank's acquisition of Google's robotic properties Boston Dynamics and Schaft plus two others acquisitions.
Fundings
Drive.ai raised $50 million in a Series B funding round, led by New Enterprise Associates, Inc. (NEA) with participation from GGV Capital and existing investors including Northern Light Venture Capital. Andrew Ng who led AI projects at Baidu and Google (and is husband to Drive.ai’s co-founder and president Carol Reiley) joined the board of directors and said:
“The cutting-edge of autonomous driving has shifted squarely to deep learning. Even traditional autonomous driving teams have 'sprinkled on' some deep learning, but Drive.ai is at the forefront of leveraging deep learning to build a truly modern autonomous driving software stack.
Aera Technology, renamed from FusionOps, a Silicon Valley software and AI provider, raised $50 million from New Enterprise Associates. Aera seems to be the first RPA to actuate in the physical world. Merck uses Aera to predict demand, determine product routing and interact with warehouse management systems to enact what’s needed.
“The leap from transactional automation to cognitive automation is imminent and it will forever transform the way we work,” says Frederic Laluyaux, President and CEO of Aera. “At Aera, we deliver the technology that enables the Self-Driving Enterprise: a cognitive operating system that connects you with your business and autonomously orchestrates your operations.”
Swift Navigation, the San Francisco tech firm building centimeter-accurate GPS technology to power a world of autonomous vehicles, raised $34 million in a Series B financing round led by New Enterprise Associates (NEA), with participation from existing investors Eclipse and First Round Capital. Swift provides solutions to over 2,000 customers-including autonomous vehicles, precision agriculture, unmanned aerial vehicles (UAVs), robotics, maritime, transportation/logistics and outdoor industrial applications. By moving GPS positioning from custom hardware to a flexible software-based receiver, Swift Navigation delivers Real Time Kinematics (RTK) GPS (100 times more accurate than traditional GPS) at a fraction of the cost ($2k) of alternative RTK systems.
AeroFarmsraised over $34 million of a $40 million Series D. The New Jersey-based indoor vertical farming startup has raised over $130 million since 2014 and now has 9 operating indoor farms. AeroFarms grows leafy greens using aeroponics –- growing them in a misting environment without soil using LED lights, and growth algorithms. The round brings AeroFarms’ total fundraising to over $130 million since 2014 including a $40 million note from Goldman Sachs and Prudential.
Seven Dreamers Labs, a Tokyo startup commercializing the Laundroid robot, raised $22.8 million KKR’s co-founders Henry Kravis and George Roberts, Chinese conglomerate Fosun International, and others. Laundroid is being developed with Panasonic and Daiwa House.
Bowery Farming, which raised $7.5 million earlier this year, raised an additional $20 million from General Catalyst, GGV Capital and GV (formerly Google Ventures). Bowery’s first indoor farm in Kearny, NJ, uses proprietary computer software, LED lighting and robotics to grow leafy greens without pesticides and with 95% less water than traditional agriculture.
Drone Racing League raised $20 million in a Series B investment round led by Sky, Liberty Media and Lux Capital, and new investors Allianz and World Wrestling Entertainment, plus existing investors Hearst Ventures, RSE Ventures, Lerer Hippeau Ventures, and Courtside Ventures.
Momentum Machines, the SF-based startup developing a hamburger-making robot, raised $18.4 million in an equity offering of $21.8 million, from existing investors Lemnos Labs, GV, K5 Ventures and Khosla Ventures. The company has been working on its first retail location since at least June of last year. There is still no scheduled opening date for the flagship, though it's expected to be located in San Francisco's South of Market neighborhood.
AEye, a startup developing a solid state LiDAR and other vision systems for self-driving cars, raised $16 million in a Series A round led by Kleiner Perkins Caufield & Byers, Airbus Ventures, Intel Capital, Tyche Partners and others.
Said Luis Dussan, CEO of AEye. “The biggest bottleneck to the rollout of robotic vision solutions has been the industry’s inability to deliver a world-class perception layer. Quick, accurate, intelligent interpretation of the environment that leverages and extends the human experience is the Holy Grail, and that’s exactly what AEye intends to deliver.”
Snips, an NYC voice recognition AI startup, raised $13 million in a Series A round led by MAIF Avenir with PSIM Fund managed by Bpifrance, as well as previous investor Eniac Ventures and K-Fund 1 and Korelya Capital, joining the round. Snip makes an on-device system that parses and understands better than Amazon's Alexa.
Misty Robotics, a spin-out from Orbotix/Sphero, raised $11.5 million in Series A funding from Venrock, Foundry Group and others. Ian Bernstein, former Sphero co-founder and CTO, will be taking the role of Head of Product and is joined by five other autonomous robotics division team members. Misty Robotics will use its new capital to build out the team and accelerate product development. Sphero and Misty Robotics will have a close partnership and have signed co-marketing and co-development agreements.
Superflex, a spin-off from SRI, has raised $10.2 million in equity financing from 10 unnamed investors. Superflex is developing a powered suit designed for individuals experiencing mobility difficulties and working in challenging environments to support the wearer’s torso, hips and legs.
Nongtian Guanjia (FarmFriend), a Chinese drone/ag industry software startup, raised $7.36 million led by Gobi Partners and existing investors GGV Capital, Shunwei Capital, the Zhen Fund and Yunqi Partners.
Carmera, a NYC-based auto tech startup, unstealthed this week with $6.4M in funding led by Matrix Partners. The two-year-old company has been quietly collecting data for its 3D mapping solution, partnering with delivery fleets to install its sensor and data collection platform.
Cognata, an Israeli deep learning simulation startup, raised $5 million from Emerge, Maniv Mobility, and Airbus Ventures. Cognata recently launched a self-driving vehicle road-testing simulation package
“Every autonomous vehicle developer faces the same challenge—it is really hard to generate the numerous edge cases and the wide variety of real-world environments. Our simulation platform rapidly pumps out large volumes of rich training data to fuel these algorithms,” said Cognata’s Danny Atsmon
SoftWear Automation, the GA Tech and DARPA sponsored startup developing sewing robots for apparel manufacturing, raised $4.5 million in a Series A round from CTW Venture Partners.
Knightscope, a startup developing robotic security technologies, raised $3 million from Konica Minolta. The capital is to be invested in Knightscope’s current Reg A+ “mini-IPO” offering of Series M Preferred Stock.
Multi Tower Co, a Danish medical device startup, raised around $1.12 million through a network of private and public investors most notable of which were Syddansk Innovation, Rikkesege Invest, M. Blæsbjerg Holding and Dahl Gruppen Holding. The Multi Tower Robot used to lift and move hospital patients, is developed through Blue Ocean Robotics’ partnership program, RoBi-X, in a public-private partnership (PPP) between University Hospital Køge, Multi Tower Company and Blue Ocean Robotics.
Optimus Ride, a MIT spinoff developing self-driving tech, raised $1.1 million in financing from an undisclosed investor.
Acquisitions
SoftBank acquired Boston Dynamics and Schaft from Google Alphabet for an undisclosed amount.
Boston Dynamics, a DARPA and DoD-funded 25 year old company, designs two and four-legged robots for the military. Videos of BD’s robots WildCat, Big Dog, Cheetah and most recently Handle, continue to be YouTube hits. Handle is a two-wheeled, four-legged hybrid robot that can stand, walk, run and roll at up to 9 MPH.
Schaft, a Japanese participant in the DARPA Robotics Challenge, recently unveiled an updated version of a two-legged robot that climbed stairs, can carry 125 pounds of payload, move in tight spaces and keep its balance throughout.
IPG Photonics, a laser component manufacturer/integrator of welding and laser-cutting systems, including robotic ones, acquired Innovative Laser Technologies, a Minnesota laser systems maker, for $40 million.
Motivo Engineering, an engineering product developer, has acquired Robodondo, an ag tech integrator focused on food processing, for an undisclosed amount.
Neural networks learn to perform computational tasks by analyzing large sets of training data. But once they’ve been trained, even their designers rarely have any idea what data elements they’re processing. Image: Christine Daniloff/MIT
By Larry Hardesty
Neural networks, which learn to perform computational tasks by analyzing large sets of training data, are responsible for today’s best-performing artificial intelligence systems, from speech recognition systems, to automatic translators, to self-driving cars.
But neural nets are black boxes. Once they’ve been trained, even their designers rarely have any idea what they’re doing — what data elements they’re processing and how.
Two years ago, a team of computer-vision researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) described a method for peering into the black box of a neural net trained to identify visual scenes. The method provided some interesting insights, but it required data to be sent to human reviewers recruited through Amazon’s Mechanical Turk crowdsourcing service.
At this year’s Computer Vision and Pattern Recognition conference, CSAIL researchers will present a fully automated version of the same system. Where the previous paper reported the analysis of one type of neural network trained to perform one task, the new paper reports the analysis of four types of neural networks trained to perform more than 20 tasks, including recognizing scenes and objects, colorizing grey images, and solving puzzles. Some of the new networks are so large that analyzing any one of them would have been cost-prohibitive under the old method.
The researchers also conducted several sets of experiments on their networks that not only shed light on the nature of several computer-vision and computational-photography algorithms, but could also provide some evidence about the organization of the human brain.
Neural networks are so called because they loosely resemble the human nervous system, with large numbers of fairly simple but densely connected information-processing “nodes.” Like neurons, a neural net’s nodes receive information signals from their neighbors and then either “fire” — emitting their own signals — or don’t. And as with neurons, the strength of a node’s firing response can vary.
In both the new paper and the earlier one, the MIT researchers doctored neural networks trained to perform computer vision tasks so that they disclosed the strength with which individual nodes fired in response to different input images. Then they selected the 10 input images that provoked the strongest response from each node.
In the earlier paper, the researchers sent the images to workers recruited through Mechanical Turk, who were asked to identify what the images had in common. In the new paper, they use a computer system instead.
“We catalogued 1,100 visual concepts — things like the color green, or a swirly texture, or wood material, or a human face, or a bicycle wheel, or a snowy mountaintop,” says David Bau, an MIT graduate student in electrical engineering and computer science and one of the paper’s two first authors. “We drew on several data sets that other people had developed, and merged them into a broadly and densely labeled data set of visual concepts. It’s got many, many labels, and for each label we know which pixels in which image correspond to that label.”
The paper’s other authors are Bolei Zhou, co-first author and fellow graduate student; Antonio Torralba, MIT professor of electrical engineering and computer science; Aude Oliva, CSAIL principal research scientist; and Aditya Khosla, who earned his PhD as a member of Torralba’s group and is now the chief technology officer of the medical-computing company PathAI.
The researchers also knew which pixels of which images corresponded to a given network node’s strongest responses. Today’s neural nets are organized into layers. Data are fed into the lowest layer, which processes them and passes them to the next layer, and so on. With visual data, the input images are broken into small chunks, and each chunk is fed to a separate input node.
For every strong response from a high-level node in one of their networks, the researchers could trace back the firing patterns that led to it, and thus identify the specific image pixels it was responding to. Because their system could frequently identify labels that corresponded to the precise pixel clusters that provoked a strong response from a given node, it could characterize the node’s behavior with great specificity.
The researchers organized the visual concepts in their database into a hierarchy. Each level of the hierarchy incorporates concepts from the level below, beginning with colors and working upward through textures, materials, parts, objects, and scenes. Typically, lower layers of a neural network would fire in response to simpler visual properties — such as colors and textures — and higher layers would fire in response to more complex properties.
But the hierarchy also allowed the researchers to quantify the emphasis that networks trained to perform different tasks placed on different visual properties. For instance, a network trained to colorize black-and-white images devoted a large majority of its nodes to recognizing textures. Another network, when trained to track objects across several frames of video, devoted a higher percentage of its nodes to scene recognition than it did when trained to recognize scenes; in that case, many of its nodes were in fact dedicated to object detection.
One of the researchers’ experiments could conceivably shed light on a vexed question in neuroscience. Research involving human subjects with electrodes implanted in their brains to control severe neurological disorders has seemed to suggest that individual neurons in the brain fire in response to specific visual stimuli. This hypothesis, originally called the grandmother-neuron hypothesis, is more familiar to a recent generation of neuroscientists as the Jennifer-Aniston-neuron hypothesis, after the discovery that several neurological patients had neurons that appeared to respond only to depictions of particular Hollywood celebrities.
Many neuroscientists dispute this interpretation. They argue that shifting constellations of neurons, rather than individual neurons, anchor sensory discriminations in the brain. Thus, the so-called Jennifer Aniston neuron is merely one of many neurons that collectively fire in response to images of Jennifer Aniston. And it’s probably part of many other constellations that fire in response to stimuli that haven’t been tested yet.
Because their new analytic technique is fully automated, the MIT researchers were able to test whether something similar takes place in a neural network trained to recognize visual scenes. In addition to identifying individual network nodes that were tuned to particular visual concepts, they also considered randomly selected combinations of nodes. Combinations of nodes, however, picked out far fewer visual concepts than individual nodes did — roughly 80 percent fewer.
“To my eye, this is suggesting that neural networks are actually trying to approximate getting a grandmother neuron,” Bau says. “They’re not trying to just smear the idea of grandmother all over the place. They’re trying to assign it to a neuron. It’s this interesting hint of this structure that most people don’t believe is that simple.”
The OR-3 autonomous security robot will begin patrolling parts of Dubai. Credit: Otsaw Digital
At the Center for the Study of the Drone
In a podcast at The Drone Radio Show, Arthur Holland Michel discusses the Center for the Study of the Drone’s recent research on local drone regulations, public safety drones, and legal incidents involving unmanned aircraft.
In a series of podcasts at the Center for a New American Security, Dan Gettinger discusses trends in drone proliferation and the U.S. policy on drone exports.
News
The U.S. Court of Appeals for the D.C. Circuit dismissed a lawsuit over the death of several civilians from a U.S. drone strike in Yemen, concurring with the decision of a lower court. In the decision, Judge Janice Rogers Brown argued that Congress had nevertheless failed in its oversight of the U.S. military. (The Hill)
Commentary, Analysis, and Art
At the Bulletin of Atomic Scientist, Michael Horowitz argues that the Missile Technology Control Regime is poorly suited to manage international drone proliferation.
At the MIT Technology Review, Bruce Y. Lee argues that though medicine-carrying drones may be expensive, they will save lives.
In a speech at the SMi Future Armoured Vehicles Weapon Systems conference in London, U.S. Marine Corps Colonel Jim Jenkins discussed the service’s desire to use small, cheap autonomous drones on the battlefield. (IHS Jane’s 360)
Meanwhile, China Aerospace Science and Technology Corporation conducted a 15-hour test flight of its Cai Hong solar-powered drone at an altitude of over 65,000 feet. (IHS Jane’s 360)
The Defense Advanced Research Projects Agency successfully tested autonomous quadcopters that were able to navigate a complex obstacle course without GPS. (Press Release)
The government of Malawi and humanitarian agency Unicef have launched an air corridor to test drones for emergency response and medical deliveries. (BBC)
French police have begun using drones to search for migrants crossing the border with Italy. (The Telegraph)
Portugal’s Air Traffic Accident Prevention and Investigation Office is leading an investigation into a number of close encounters between drones and manned aircraft in the country’s airspace. (AIN Online)
The U.S. Federal Aviation Administration and app company AirMap are developing a system that will automate low-altitude drone operation authorizations. (Drone360)
Dubai’s police will deploy the Otsaw Digital O-R3, an autonomous security robot equipped with facial recognition software and a built-in drone, to patrol difficult-to-reach areas. (Washington Post)
The University of Southampton writes that Boaty McBoatface, an unmanned undersea vehicle, captured “unprecedented data” during its voyage to the Orkney Passage.
The Department of the Interior awarded NAYINTY3 a $7,742 contract for the Agisoft Photo Scan, computer software designed to process images from drones. (FBO)
The Federal Aviation Administration awarded Computer Sciences Corporation a $200,000 contract for work on drone registration. (USASpending)
The U.S. Navy awarded Hensel Phelps a $36 million contract to build a hangar for the MQ-4C Triton surveillance drone at Naval Station Mayport in Florida. (First Coast News)
The U.S. Navy awarded Kratos Defense & Security Solutions a $35 million contract for the BQM-177A target drones. (Military.com)
Clobotics, a Shanghai-based startup that develops artificial intelligence-equipped drones for infrastructure inspection, announced that it has raised $5 million in seed funding. (GeekWire)
For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.
At the Empa and Eawag NEST building in Dübendorf, eight ETH Zurich professors as part of the Swiss National Centre of Competence in Research (NCCR) Digital Fabrication are collaborating with business partners to build the three-storey DFAB HOUSE. It is the first building in the world to be designed, planned and built using predominantly digital processes.
Robots that build walls and 3D printers that print entire formworks for ceiling slabs – digital fabrication in architecture has developed rapidly in recent years. As part of the National Centre of Competence in Research (NCCR) Digital Fabrication, architects, robotics specialists, material scientists, structural engineers and sustainability experts from ETH Zurich have teamed up with business partners to put several new digital building technologies from the laboratory into practice. Construction is taking place at NEST, the modular research and innovation building that Empa and Eawag built on their campus in Dübendorf to test new building and energy technologies under real conditions. NEST offers a central support structure with three open platforms, where individual construction projects – known as innovation units – can be installed. Construction recently began on the DFAB HOUSE.
Digitally Designed, Planned and Built
The DFAB HOUSE is distinctive in that it was not only digitally designed and planned, but is also built using predominantly digital processes. With this pilot project, the ETH professors want to examine how digital technology can make construction more sustainable and efficient, and increase the design potential. The individual components were digitally coordinated based on the design and are manufactured directly from this data. The conventional planning phase is no longer needed. As of summer 2018, the three-storey building, with a floor space of 200 m2, will serve as a residential and working space for Empa and Eawag guest researchers and partners of NEST.
Four New Building Methods Put to the Test
At the DFAB HOUSE, four construction methods are for the first time being transferred from research to architectural applications. Construction work began with the Mesh Mould technology, which received the Swiss Technology Award at the end of 2016. The result will be a double-curved load-bearing concrete wall that will shape the architecture of the open-plan living and working area on the ground floor. A “Smart Slab” will then be installed – a statistically optimised and functionally integrated ceiling slab, for which the researchers used a large-format 3D sand printer to manufacture the formwork.
Smart Dynamic Casting technology is being used for the façade on the ground floor: the automated robotic slip-forming process can produce tailor-made concrete façade posts. The two upper floors, with individual rooms, are being prefabricated at ETH Zurich’s Robotic Fabrication Laboratory using spatial timber assemblies; cooperating robots will assemble the timber construction elements.
More Information in ETH Zurich Press Release and on Project Website
Detailed information about the building process, quotes as well as image and video material can be found in the extended press release by ETH Zurich. In addition, a project website for the DFAB HOUSE is currently in development and will soon be available at the following link: www.dfabhouse.ch. Until then, Empa’s website offers information about the project: https://www.empa.ch/web/nest/digital-fabrication
I’ve written a few times that perhaps the biggest unsolved problem in robocars is how to know we have made them safe enough. While most people think of that in terms of government certification, the truth is that the teams building the cars are very focused on this, and know more about it than any regulator, but they still don’t know enough. The challenge is going to be convincing your board of directors that the car is safe enough to release, for if it is not, it could ruin the company that releases it, at least if it’s a big company with a reputation.
We don’t even have a good definition of what “safe enough” is though most people are roughly taking that as “a safety record superior to the average human.” Some think it should be much more, few think it should be less. Tesla, now with the backing of the NTSB, has noted that their autopilot system — combined with a mix of mostly attentive but some inattentive humans, may have a record superior to the average human, for example, even though with the inattentive humans it is worse.
Last week I attended a conference in Stuttgart devoted to robocar safety testing, part of a larger auto show including an auto testing show. It was interesting to see the main auto testing show — scores of expensive and specialized machines and tools that subject cars to wear and tear, slamming doors thousands of times, baking the surfaces, rattling and vibrating everything. And testing the electronics, too.
In Europe, the focus of testing is very strongly on making sure you are compliant with standards and regulations. That’s true in the USA but not quite as much. It was in Europe some time ago that I learned the word “homologation” which names this process.
There is a lot to be learned from the previous regimes of testing. They have built a lot of tools and learned techniques. But robocars are different beasts, and will fail in different ways. They will definitely not fail the way human drivers do, where usually small things are always going wrong, and an accident happens when 2 or 3 things go wrong at once. The conference included a lot of people working on simulation, which I have been promoting for many years. The one good thing in the NHTSA regulations — the open public database of all incidents — may vanish in the new rules, and it would have made for a great simulator. The companies making the simulators (and the academic world) would have put every incident into a shared simulator so every new car could test itself in every known problem situation.
Still, we will see lots of simulators full of scenarios, and also ways to parameterize them. That means that instead of just testing how a car behaves if somebody cuts it off, you test what it does if it gets cut off with a gap of 1cm, or 10cm, or 1m, or 2m, and by different types of vehicles, and by two at once etc. etc. etc. The nice thing about computers is you can test just about every variation you can think of, and test it in every road situation and every type of weather, at least if your simulator is good enough,
Yoav Hollander, who I met when he came as a student to the program at Singularity U, wrote a report on the approaches to testing he saw at the conference that contains useful insights, particularly on this question of new and old thinking, and what regulations drive vs. liability and fear of the public. He puts it well — traditional and certification oriented testing has a focus on assuring you don’t have “expected bugs” but is poor at finding unexpected ones. Other testing is about finding unexpected bugs. Expected bugs are of the “we’ve seen this sort of thing before, we want to be sure you don’t suffer from it” kind. Unexpected bugs are “something goes wrong that we didn’t know to look for.”
Avoiding old thinking
I believe that we are far from done on the robocar safety question. I think there are startups who have not yet been founded who, in the future, will come up with new techniques both for promoting safety and testing it that nobody has yet thought of. As such, I strongly advise against thinking that we know very much about how to do it yet.
A classic example of things going wrong is the movement towards “explainable AI.” Here, people are concerned that we don’t really know how “black box” neural network tools make the decisions they do. Car regulations in Europe are moving towards banning software that can’t be explained in cars. In the USA, the draft NHTSA regulations also suggest the same thing, though not as strongly.
We may find ourselves in a situation where we take to systems for robocars, one explainable and the other not. We put them through the best testing we can, both in simulator and most importantly in the real world. We find the explainable system has a “safety incident” every 100,000 miles, and the unexplainable system has an incident every 150,000 miles. To me it seems obvious that it would be insane to make a law that demands the former system which, when deployed, will hurt more people. We’ll know why it hurt them. We might be better at fixing the problems, but we also might not — with the unexplainable system we’ll be able to make sure that particular error does not happen again, but we won’t be sure that others very close it it are eliminated.
Testing in sim is a challenge here. In theory, every car should get no errors in sim, because any error found in sim will be fixed or judged as not really an error or so rare as to be unworthy of fixing. Even trained machine learning systems will be retrained until they get no errors in sim. The only way to do this sort of testing in sim will be to have teams generate brand new scenarios in sim that the cars have never seen, and see how they do. We will do this, but it’s hard. Particularly because as the sims get better, there will be fewer and fewer real world situations they don’t contain. At best, the test suite will offer some new highly unusual situations, which may not be the best way to really judge the quality of the cars. In addition, teams will be willing to pay simulator companies well for new and dangerous scenarios in sim for their testing — more than the government agencies will pay for such scenarios. And of course, once a new scenario displays a problem, every customer will fix it and it will become much less valuable. Eventually, as government regulations become more prevalent, homologation companies will charge to test your compliance rate on their test suites, but again, they will need to generate a new suite every time since everybody will want the data to fix any failure. This is not like emissions testing, where they tell you that you went over the emissions limit, and it’s worth testing the same thing again.
The testing was interesting, but my other main focus was on the connected car and security sessions. More on that to come.
The Robot Academy is a new learning resource from Professor Peter Corke and the Queensland University of Technology (QUT), the team behind the award-winning Introduction to Robotics and Robotic Vision courses. There are over 200 lessons available, all for free.
The lessons were created in 2015 for the Introduction to Robotics and Robotic Vision courses. We describe our approach to creating the original courses in the article, An Innovative Educational Change: Massive Open Online Courses in Robotics and Robotic Vision. The courses were designed for university undergraduate students but many lessons are suitable for anybody, as you can easily see the difficulty rating for each lesson. Below are lessons from inverse kinematics and robot motion.
In this video lecture, we will learn about inverse kinematics, that is, how to compute the robot’s joint angles given the desired pose of their end-effector and knowledge about the dimensions of its links. We will also learn about how to generate paths that lead to a smooth coordinated motion of the end-effector.
Inverse kinematics for a 2-joint robot arm using geometry
In this lesson, we revisit the simple 2-link planar robot and determine the inverse kinematic function using simple geometry and trigonometry.
Inverse kinematics for a 2-joint robot arm using algebra
by Nicholas Charron
The Apollo platform consists of a core software stack, a number of cloud services, and self-driving vehicle hardware such as GPS, cameras, lidar, and radar.
China Network
