Tag news

The Robotics Hub, in collaboration with Silicon Valley Robotics, is currently investing in robotics, AI and sensor startups, with checks between $250,000 and $500,000. Current portfolio companies include Agility Robotics, RoBotany, Travelwits and Ariel Precision Technologies.

A team of judges has shortlisted 25 robotics startups who all deserve mention. Eight startups will be in our public voting which will start on Dec 1st and continue till December 10 on Robohub.org. Also eight startups are currently giving longer pitches to a panel of judges, so that the final winner(s) can be announced at the Silicon Valley Robotics investor showcase on December 14.

The Top 25 in alphabetical order are:

Achille, Inc.
Apellix
Augmented Robots (spin-off from GESTALT Robotics)
Betterment Labs (formerly known as MOTI)
BotsAndUs
C2RO Cloud Robotics
DroidX
Fotokite
Fruitbot, Inc.
Holotron
INF Robotics Inc.
Kinema Systems Inc.
Kiwi Campus
KOMPAÏ robotics
krtkl inc.
Mothership Aeronautics
Northstar Robotics Inc
Rabbit Tractors, Inc
Semio
TatuRobotics PTY LTD
Tennibot
UniExo
Woobo Inc.

The winners of last year’s Robot Launch 2016 startup competition, Vidi Systems, were acquired by Cognex earlier this year for an undisclosed amount. Some of the other finalists have gone on to expo at TechCrunch, and other competitions. Franklin Robotics raised $312,810 in a Kickstarter campaign, more than doubling their target. Business Insider called Franklin’s Tertill weed whacker ‘a Roomba for your garden’.

Modular Science were accepted into YCombinators Summer 2017 intake, and Dash Robotics, the spin off from Berkeley Biomimetics Lab, make the Kamigami foldable toy robots that are now being sold at all major retailers.

This year, the top 8 startups will receive space in the Silicon Valley Robotics Cowork Space @CircuitLaunch in Oakland. The space has lots of room for testing, full electronics lab and various prototyping equipment such as laser cutters, cnc machines, 3d printers. It’s located near Oakland International Airport and is convenient to San Francisco and the rest of Silicon Valley. There are also plenty of meeting and conference rooms. We also hold networking/mentor/investor events so you can connect with the robotics community.

Finalists also receive invaluable exposure on Robohub.org to an audience of robotics professionals and those interested in the latest robotics technologies, as well as the experience of pitching their startup to an audience of top VCs, investors and experts.

Robot Launch is supported by Silicon Valley Robotics to help more robotics startups present their technology and business models to prominent investors. Silicon Valley Robotics is the not-for-profit industry group supporting innovation and commercialization in robotics technologies. The Robotics Hub is the first investor in advanced robotics and AI startups, helping to get from ‘zero to one’ with their network of robotics and market experts.

Learn more about previous Robot Launch competitions here.

By Michael Laskey, Jonathan Lee, and Ken Goldberg

In Imitation Learning (IL), also known as Learning from Demonstration (LfD), a robot learns a control policy from analyzing demonstrations of the policy performed by an algorithmic or human supervisor. For example, to teach a robot make a bed, a human would tele-operate a robot to perform the task to provide examples. The robot then learns a control policy, mapping from images/states to actions which we hope will generalize to states that were not encountered during training.

There are two variants of IL: Off-Policy, or Behavior Cloning, where the demonstrations are given independent of the robot’s policy. However, when the robot encounters novel risky states it may not have learned corrective actions. This occurs because of “covariate shift” a known challenge, where the states encountered during training differ from the states encountered during testing, reducing robustness. Common approaches to reduce covariate shift are On-Policy methods, such as DAgger, where the evolving robot’s policy is executed and the supervisor provides corrective feedback. However, On-Policy methods can be difficult for human supervisors, potentially dangerous, and computationally expensive.

This post presents a robust Off-Policy algorithm called DART and summarizes how injecting noise into the supervisor’s actions can improve robustness. The injected noise allows the supervisor to provide corrective examples for the type of errors the trained robot is likely to make. However, because the optimized noise is small, it alleviates the difficulties of On-Policy methods. Details on DART are in a paper that will be presented at the 1st Conference on Robot Learning in November.

We evaluate DART in simulation with an algorithmic supervisor on MuJoCo tasks (Walker, Humanoid, Hopper, Half-Cheetah) and physical experiments with human supervisors training a Toyota HSR robot to perform grasping in clutter, where a robot must search through clutter for a goal object. Finally, we show how DART can be applied in a complex system that leverages both classical robotics and learning techniques to teach the first robot to make a bed. For researchers who want to study and use robust Off-Policy approaches, we additionally announce the release of our codebase on GitHub.

Read More

Wanda Tuerlinckx and Erwin R. Boer have fused their scientific and photographic interests in robots and traveled the world since 2016 to visit roboticists to discuss and photograph their creations. The resulting set of photographs documents the technical robot revolution that is unfolding before us. The portfolio of photographs below presents the androids from Wanda’s collection of robot photographs.

But first, here’s a note from Erwin R. Boer, a scientist who connects humans and machines using symbiosis facilitating techniques mirrored after the way humans interact with each other in the here and now.

Man has created machines in the form of mechanical humans since antiquity. The sculpted faces of the early automatons gave us a glimpse of the future we currently live in. Today’s machines look like humans, move like humans, talk like humans, and at a rapidly increasing pace even think like humans. We marvel at the technological capabilities of these robots and how they are being integrated into our daily lives. The integration of robots into society requires vast technological advances. Successful interactions and communications with humans takes more than nimble technology and raw artificial intelligence – it requires the robot to have emotional intelligence, exhibiting empathy, compassion, forgiveness, and playfulness. At the same time, we fearfully watch how robots reach human potential. Human like robots come in many incarnations ranging from humanoids that have human forms but their bodies and faces are clearly robotic to androids that look in all aspects like humans and are hard to tell apart from humans. Today most androids act on the edge of the uncanny valley, a valley that reflects the fact that the complex behavior of androids, at times, is highly disturbing to humans; these disturbances are caused by unrealistic humanistic expectations of complete human ability projected onto these highly advanced androids that through interaction often gets broken by sometimes creepy realizations that they are not human. This valley is an extremely delicate space, where human and robot apparently overlap in appearance, movement and speech,. Researchers are working feverishly to remove the uncanny valley and create a flat playing field where robots are capable of producing emotions and become an integral part of society through tranquil harmonious cooperation, servitude and symbiotic interactions with humans.

Imagine seeing yourself in the mirror and then that mirror image takes on a reality that reflects your own and walks away to represent you around the world. This is what Professor Hiroshi Ishiguro envisioned when he created his HI-2 and later in life HI-4 geminoids; these geminoids are life size robotic replicates of himself. He created these geminoids to travel for him to far away conferences so that he could from the comfort of his home or office talk and act through these geminoids to give lectures and make appearances. A geminoid with its human twin offers a perfect test bed to explore the question that has inspired scientists and philosophers through the ages namely: what does it mean to be human? To be human also means to have emotional intelligence and thus to be able to understand emotions.

Humans understand emotions because when we see an emotion it triggers in us the feelings that we have when we produce that emotion and therefore we naturally project our feelings onto robots that are capable of producing emotions. Dr. David Hanson has produced a facial rubber called frubber that is perfectly suited to be pulled on the inside by little actuators as if a muscle underneath the skin contracts. His robots are capable of producing a series of emotions that elicit mirror emotions in us. The child like android Diego-san has been capable of instilling the joys of youth in many humans he interacted with. The emotional riches of Hanson’s androids help to create emotional robots that find tremendous value especially in the medical field where human compassion is critical for healing and where autistics children are benefitting from the unfailing compassion that these androids offer.

Recently, a recipient of the Nobel price of literature, Japanese author Natsume Sōseki (1867- 1916) was reincarnated in the form of his android who will give lectures at the university where professor Sōseki taught back in the 1880s. The fact that Wanda photographed android Sōseki with a camera that was used in Sōseki’s own time to take portraits of notable people creates a loop that not only transcends time but also connects two key industrial revolutions; the industrial revolution around 1900 and the robot revolution around 2000. The connection across a similar time scale is also beautifully embodied in Dr. Hanson’s android Einstein whose clones are currently being used as science teachers in many classrooms and homes around the world. Photography continues to enlighten use through imagery while robots enlighten us through physical embodied actions enriched by intelligent emotional sensitive speech.

Wanda Tuerlinckx is a photographer who connects humans and robots using a 180 year old photographic technique that mirrors how humans connect with each other across the boundaries of time through the soft understanding eye from our great grandfathers who have lived through earlier technological revolutions and presents these new technological marvels in a manner that exudes a comfortable familiarity that instills acceptance. The human element in science imposes its presence nowhere stronger than in the incarnation of a human robot that in many respects is indistinguishable from a human human. More information about Wanda and her work can be found here. You can also see her previous set of robot portraits here.

Jibo is a personal robot with a difference. It is unlike the stationary Amazon Alexa or Google Home. It attempts to offer the same repertoire of features while adding its physical presence and mobility to the mix.

Quoting Time Magazine, “Jibo looks like something straight out of a Pixar movie, with a big, round head and a face that uses animated icons to convey emotion. It’s not just that his body swivels and swerves while he speaks, as if he’s talking with his nonexistent hands. It’s not just that he can giggle and dance and turn to face you, wherever you are, as soon as you say, “Hey, Jibo.” It’s that, because of all this, Jibo seems downright human in a way that his predecessors do not. Jibo could fundamentally reshape how we interact with machines.”

Jibo can recognize up to six faces and voices yet it still has a lot to learn. Although he can help users in basic ways, like by summarizing news stories and taking photos, he can’t yet play music or work with third-party apps like Domino’s and Uber.

As an original IndieGoGo backer back in 2014, it’s been a long wait. Three years! Yes, this version of Jibo still has a lot to learn. But those skills are coming in 2018 as Jibo’s SDK becomes available to developers.

Huachangda Intelligent Equipment, a Chinese industrial robot integrator primarily servicing China’s auto industry, has acquired Swedish Robot System Products (RSP), a 2003 spin-off from ABB with 70 employees in Sweden, Germany and China, for an undisclosed amount. RSP manufactures grippers, welding equipment, tool changers and other peripheral products for robots.

Last month HTI Cybernetics, a Michigan industrial robotics integrator and contract manufacturer, was acquired by Chongqing Nanshang Investment Group for around $50 million. HTI provides robotic welding systems to the auto industry and also has a contract welding services facility in Mexico.

China is in the midst of a national program to develop or acquire its own technology to rival similar technologies in the West, particularly in futuristic industries such as robotics, electric cars, self-driving vehicles and artificial intelligence. China’s Made in China 2025 program will “support state capital in becoming stronger, doing better, and growing bigger, turning Chinese enterprises into world-class, globally competitive firms,” said President Xi at the recent party congress meeting in Beijing.

Made in China 2025 has specific targets and quotas. It envisions China domestically supplying 3/4 of its own industrial robots and more than 1/3 of its demand for smartphone chips by 2025, for example. These goals are backed with money: $45 billion in low-cost loans, $3 billion for advanced manufacturing efforts and billions more in other types of financial incentives and support.

Over the last two years there have been many targeted acquisitions by Chinese companies, of robotic companies in the EU and US. Following are the major ones:

• Midea Group, for around $4.5 billion, last year acquired the world’s 4th largest robot manufacturer, Germany-based Kuka AG.
• The Kion Group, a predominately Chinese-funded warehousing systems and equipment conglomerate, acquired Dematic, a large European AGV and material handling systems company. Kion also acquired Egemin Automation, Retrotech and Linde Material Handling.
• KraussMaffei, a big German industrial robots integrator, was acquired by ChemChina.
• Paslin, a US-based industrial robot integrator, was acquired by Zhejiang Wanfeng Technology, a Chinese industrial robot integrator.
• Baidu acquired Silicon Valley vision systems startup Xperception  for an undisclosed amount.
• Servotronix, an Israeli motion control and automation systems company, was acquired by Midea Group for $170 million. Servotronix, founded in 1987, develops and manufactures comprehensive and high performance motion control solutions.
• Shenyang Blue Silver Group (SBS Group), China, acquired 85% of the Ecoclean Group within the Dürr Group for around $134 million.
• RoboRobo, a Korean educational robotics kit and course provider startup, was acquired by Shengtong Printing for $62 million in cash and stock.
• Hocoma, a Swiss provider of robotic and sensor-based rehabilitation solutions, merged with Chinese DIH International to provide comprehensive rehab solutions.

Bottom line:

The consequences of China’s relentless quest for technology acquisitions may upset global trade. Their efforts have many American and European officials and business leaders pushing for tougher rules on technology purchases. Jeremie Waterman, President of the China Center at the U.S. Chamber of Commerce said the following to the NY Times.

“If Made in China 2025 achieves its goals, the U.S. and other countries would likely become just commodity exporters to China — selling oil, gas, beef and soybeans.”

Bossa Nova Robotics, a Silicon Valley developer of autonomous service robots for the retail industry, announced the close of a $17.5 million Series B funding round led by Paxion Capital Partners and participation by Intel Capital, WRV Capital, Lucas Venture Group (LVG), and Cota Capital. This round brings Bossa Nova’s total funding to date to $41.7 million.

Bossa Nova helps large scale stores automate the collection and analysis of on-shelf inventory data by driving their sensor-laden mobile robots autonomously through aisles, navigating safely among customers and store associates. The robots capture images of store shelves and use AI to analyze the data and calculate the status of each product including location, price, and out-of-stocks which is then aggregated and delivered to management in the form of a restock action plan.

They recently began testing their robots and analytic services in 50 Walmart stores across the US. They first deployed their autonomous robots in retail stores in 2013 and have since registered more than 710 miles and 2,350 hours of autonomous inventory scanning, capturing more than 80 million product images.

“We have worked closely with Bossa Nova to help ensure this technology, which is designed to capture and share in-store data with our associates in near real time, works in our unique store environment,” said John Crecelius, vice president of central operations at Walmart. “This is meant to be a tool that helps our associates quickly identify where they can make the biggest difference for our customers.”

CMU grads launched Bossa Nova Robotics in Pittsburgh as a designer of robotic toys. In 2009 they launched two new products: Penbo, a fuzzy penguin-like robot that sang, danced, cuddled and communicated with her baby in their own Penbo language; and Prime-8, a gorilla-like loud fast-moving robot for boys. In 2011 and 2012 they changed direction: they sold off the toy business and focused on developing a mobile robot based on CMU’s ballbot technology. Later they converted to normal casters and mobility methods and spent their energies on developing camera, vision and AI analytics software to produce their latest round of shelf-scanning mobile robots.

Again, it is better not to write but just to show the video, when it is a robot from Boston Dynamics, as it speaks for itself

The post Boston Dynamics Atlas Robot can now do backflips appeared first on Roboticmagazine.

The acquisition and processing of a video stream can be very computationally expensive. Typical image processing applications split the work across multiple threads, one acquiring the images, and another one running the actual algorithms. In MATLAB we can get multi-threading by interfacing with other languages, but there is a significant cost associated with exchanging data across the resulting language barrier. In this blog post, we compare different approaches for getting data through MATLAB’s Java interface, and we show how to acquire high-resolution video streams in real-time and with low overhead.

Motivation

For our booth at ICRA 2014, we put together a demo system in MATLAB that used stereo vision for tracking colored bean bags, and a robot arm to pick them up. We used two IP cameras that streamed H.264 video over RTSP. While developing the image processing and robot control parts worked as expected, it proved to be a challenge to acquire images from both video streams fast enough to be useful.

• IP Camera Support only supports MJPEG over HTTP and didn’t exist at the time

• USB Webcam Support only supports USB cameras

• imread and webread are limited to HTTP and too slow for real-time

Since we did not want to switch to another language, we decided to develop a small library for acquiring video streams. The project was later open sourced as HebiCam.

Technical Background

In order to save bandwidth most IP cameras compress video before sending it over the network. Since the resulting decoding step can be computationally expensive, it is common practice to move the acquisition to a separate thread in order to reduce the load on the main processing thread.

Unfortunately, doing this in MATLAB requires some workarounds due to the language’s single threaded nature, i.e., background threads need to run in another language. Out of the box, there are two supported interfaces: MEX for calling C/C++ code, and the Java Interface for calling Java code.

While both interfaces have strengths and weaknesses, practically all use cases can be solved using either one. For this project, we chose the Java interface in order to simplify cross-platform development and the deployment of binaries. The diagram below shows an overview of the resulting system.

Figure 1. System overview for a stereo vision setup

Starting background threads and getting the video stream into Java was relatively straightforward. We used the JavaCV library, which is a Java wrapper around OpenCV and FFMpeg that includes pre-compiled native binaries for all major platforms. However, passing the acquired image data from Java into MATLAB turned out to be more challenging.

The Java interface automatically converts between Java and MATLAB types by following a set of rules. This makes it much simpler to develop for than the MEX interface, but it does cause additional overhead when calling Java functions. Most of the time this overhead is negligible. However, for certain types of data, such as large and multi-dimensional matrices, the default rules are very inefficient and can become prohibitively expensive. For example, a 1080x1920x3 MATLAB image matrix gets translated to a byte[1080][1920][3] in Java, which means that there is a separate array object for every single pixel in the image.

As an additional complication, MATLAB stores image data in a different memory layout than most other libraries (e.g. OpenCV’s Mat or Java’s BufferedImage). While pixels are commonly stored in row-major order ([width][height][channels]), MATLAB stores images transposed and in column-major order ([channels][width][height]). For example, if the Red-Green-Blue pixels of a BufferedImage would be laid out as [RGB][RGB][RGB]…​, the same image would be laid out as [RRR…​][GGG…​][BBB…​] in MATLAB. Depending on the resolution this conversion can become fairly expensive.

In order to process images at a frame rate of 30 fps in real-time, the total time budget of the main MATLAB thread is 33ms per cycle. Thus, the acquisition overhead imposed on the main thread needs to be sufficiently low, i.e., a low number of milliseconds, to leave enough time for the actual processing.

Data Translation

We benchmarked five different ways to get image data from Java into MATLAB and compared their respective overhead on the main MATLAB thread. We omitted overhead incurred by background threads because it had no effect on the time budget available for image processing.

The full benchmark code is available here.

1. Default 3D Array

By default MATLAB image matrices convert to byte[height][width][channels] Java arrays. However, when converting back to MATLAB there are some additional problems:

• byte gets converted to int8 instead of uint8, resulting in an invalid image matrix

• changing the type back to uint8 is somewhat messy because the uint8(matrix) cast sets all negative values to zero, and the alternative typecast(matrix, 'uint8') only works on vectors

Thus, converting the data to a valid image matrix still requires several operations.

% (1) Get matrix from byte[height][width][channels]
data = getRawFormat3d(this.javaConverter);
[height,width,channels] = size(data);

% (2) Reshape matrix to vector
vector = reshape(data, width * height * channels, 1);

% (3) Cast int8 data to uint8
vector = typecast(vector, 'uint8');

% (4) Reshape vector back to original shape
image = reshape(vector, height, width, channels);

2. Compressed 1D Array

A common approach to move image data across distributed components (e.g. ROS) is to encode the individual images using MJPEG compression. Doing this within a single process is obviously wasteful, but we included it because it is common practice in many distributed systems. Since MATLAB did not offer a way to decompress jpeg images in memory, we needed to save the compressed data to a file located on a RAM disk.

% (1) Get compressed data from byte[]
data = getJpegData(this.javaConverter);

% (2) Save as jpeg file
fileID = fopen('tmp.jpg','w+');
fwrite(fileID, data, 'int8');
fclose(fileID);

% (3) Read jpeg file
image = imread('tmp.jpg');

3. Java Layout as 1D Pixel Array

Another approach is to copy the pixel array of Java’s BufferedImage and to reshape the memory using MATLAB. This is also the accepted answer for How can I convert a Java Image object to a MATLAB image matrix?.

% (1) Get data from byte[] and cast to correct type
data = getJavaPixelFormat1d(this.javaConverter);
data = typecast(data, 'uint8');
[h,w,c] = size(this.matlabImage); % get dim info

 % (2) Reshape matrix for indexing 
pixelsData = reshape (data, 3 , w, h);

 % (3) Transpose and convert from row major to col major format (RGB case) 
image = cat (3 , ...
    transpose(reshape (pixelsData(3 , :, :), w, h)), ...
    transpose(reshape (pixelsData(2 , :, :), w, h)), ...
    transpose(reshape (pixelsData(1 , :, :), w, h)));

4. MATLAB Layout as 1D Pixel Array

The fourth approach also copies a single pixel array, but this time the pixels are already stored in the MATLAB convention.

 % (1) Get data from byte[] and cast to correct type 
data = getMatlabPixelFormat1d(this.javaConverter);
[h,w,c] = size (this.matlabImage);   % get dim info 
vector = typecast(data, 'uint8' );

 % (2) Interpret pre-laid out memory as matrix 
image = reshape (vector,h,w,c);

Note that the most efficient way we found for converting the memory layout on the Java side was to use OpenCV’s split and transpose functions. The code can be found in MatlabImageConverterBGR and MatlabImageConverterGrayscale.

5. MATLAB Layout as Shared Memory

The fifth approach is the same as the fourth with the difference that the Java translation layer is bypassed entirely by using shared memory via memmapfile. Shared memory is typically used for inter-process communication, but it can also be used within a single process. Running within the same process also simplifies synchronization since MATLAB can access Java locks.

 % (1) Lock memory 
lock(this.javaObj);

 % (2) Force a copy of the data 
image = this.memFile.Data.pixels * 1 ;

 % (3) Unlock memory 
unlock(this.javaObj);

Note that the code could be interrupted (ctrl+c) at any line, so the locking mechanism would need to be able to recover from bad states, or the unlocking would need to be guaranteed by using a destructor or onCleanup.

The multiplication by one forces a copy of the data. This is necessary because under-the-hood memmapfile only returns a reference to the underlying memory.

Results

All benchmarks were run in MATLAB 2017b on an Intel NUC6I7KYK. The performance was measured using MATLAB’s timeit function. The background color of each cell in the result tables represents a rough classification of the overhead on the main MATLAB thread.

The two tables below show the results for converting color (RGB) images as well as grayscale images. All measurements are in milliseconds.

Figure 2. Conversion overhead on the MATLAB thread in [ms]

The results show that the default conversion, as well as jpeg compression, are essentially non-starters for color images. For grayscale images, the default conversion works significantly better due to the fact that the data is stored in a much more efficient 2D array (byte[height][width]), and that there is no need to re-order pixels by color. Unfortunately, we currently don’t have a good explanation for the ~10x cost increase (rather than ~4x) between 1080p and 4K grayscale. The behavior was the same across computers and various different memory settings.

When copying the backing array of a BufferedImage we can see another significant performance increase due to the data being stored in a single contiguous array. At this point much of the overhead comes from re-ordering pixels, so by doing the conversion beforehand, we can get another 2-3x improvement.

Lastly, although accessing shared memory in combination with the locking overhead results in a slightly higher fixed cost, the copying itself is significantly cheaper, resulting in another 2-3x speedup for high-resolution images. Overall, going through shared memory scales very well and would even allow streaming of 4K color images from two cameras simultaneously.

Final Notes

Our main takeaway was that although MATLAB’s Java interface can be inefficient for certain cases, there are simple workarounds that can remove most bottlenecks. The most important rule is to avoid converting to and from large multi-dimensional matrices whenever possible.

Another insight was that shared-memory provides a very efficient way to transfer large amounts of data to and from MATLAB. We also found it useful for inter-process communication between multiple MATLAB instances. For example, one instance can track a target while another instance can use its output for real-time control. This is useful for avoiding coupling a fast control loop to the (usually lower) frame rate of a camera or sensor.

As for our initial motivation, after creating HebiCam we were able to develop and reliably run the entire demo in MATLAB. The video below shows the setup using old-generation S-Series actuators.

Governor Andrew Cuomo of the State of New York declared last month that New York City will join 13 other states in testing self-driving cars: “Autonomous vehicles have the potential to save time and save lives, and we are proud to be working with GM and Cruise on the future of this exciting new technology.” For General Motors, this represents a major milestone in the development of its Cruise software, since the the knowledge gained on Manhattan’s busy streets will be invaluable in accelerating its deep learning technology. In the spirit of one-upmanship, Waymo went one step further by declaring this week that it will be the first car company in the world to ferry passengers completely autonomously (without human engineers safeguarding the wheel).

As unmanned systems are speeding ahead toward consumer adoption, one challenge that Cruise, Waymo and others may counter within the busy canyons of urban centers is the loss of Global Positioning System (GPS) satellite data. Robots require a complex suite of coordinating data systems that bounce between orbiting satellites to provide positioning and communication links to accurately navigate our world. The only thing that is certain, as competing technologies and standards wrestle in this nascent marketplace for adoption, is the critical connection between Earth and space. Based upon the estimated growth of autonomous systems on the road, in the workplace and home in the next ten years, most unmanned systems rely heavily on the ability of commercial space providers to fulfill their boastful mission plans to launch thousands of new satellites into an already crowded lower earth orbit.

As shown by the chart below, the entry of autonomous systems will drive an explosion of data communications between terrestrial machines and space, leading to tens of thousands of new rocket launches over the next two decades. In a study done by Northern Sky Research (NSR) it projected that by 2023 there will be an estimated 5.8 million satellite Machine-to-Machine (M2M) and Internet Of Things (IOT) connections to approximately 50 billion global Internet-connected devices. In order to meet this demand, satellite providers are racing to the launch pads and raising billions in capital, even before firing up the rockets. As an example, OneWeb, which has raised more than $1.5 billion from Softbank, Qualcomm and Airbus, plans to launch its first 10 satellite constellations in 2018 which will eventually grow to 650 in the next decade. OneWeb competes with Space X, Boeing, Immarsat, Iridium, and others in deploying new satellites offering high-speed communication spectrums, such as Ku Band (12 GHz Wireless), K Band (18 GHz – 27 GHz), Ka Band (27 GHz – 40 GHz) and V Band (40 GHz – 75 GHz). The opening of new higher frequency spectrums is critical to support the explosion of increased data demands. Today there are more than 250 million cars on the road in the United States and in the future these cars will connect to the Internet, transmitting 200 million lines of code or 50 billion pings of data to safely and reliably transport passengers to their destinations everyday.

Satellites already provide millions of GPS coordinates for connected systems. However, the accuracy of GPS has been off  by as many as 5 meters, which in a fully autonomous world could mean the difference between life and death. Chip manufacturer Broadcom aims to reduce the error margin to 30 centimeters. According to a press release this summer, Broadcom’s technology works better in concrete canyons like New York which have plagued Uber drivers for years with wrong fare destinations. Using new L5 satellite signals, the chips are able to calculate receptions between points at a fast rate with lower power consumption (see diagram). Manuel del ­Castillo of Broadcom explained, “Up to now there haven’t been enough L5 satellites in orbit.” Currently there are approximately 30 L5 satellites in orbit. However, del ­Castillo suggests that could be enough to begin shipping the new chip next year, “[Even in a city’s] narrow window of sky you can see six or seven, which is pretty good. So now is the right moment to launch.”

Leading roboticist and business leader in this space, David Bruemmer explained to me this week that GPS is inherently deficient, even with L5 satellite data. In addition, current autonomous systems rely too heavily on vision systems like LIDAR and cameras, which can only see what is in front of them but not around the corner. In Bruemmer’s opinion the only solution to provide the greatest amount of coverage is one that combines vision, GPS with point-to-point communications such as Ultra Wide Band and RF beacons. Bruemmer’s company Adaptive Motion Group (AMG) is a leading innovator in this space. Ultimately, in order for AMG to efficiently work with unmanned systems it requires a communication pipeline that is wide enough to transmit space signals within a network of terrestrial high-speed frequencies.

AMG is not the only company focused on utilizing a wide breadth of data points to accurately steer robotic systems. Sandy Lobenstein, Vice President of Toyota Connected Services, explains that the Japanese car maker has been working with the antenna satellite company Kymeta to expand the data connectivity bandwidth in preparation for Toyota’s autonomous future. “We just announced a consortium with companies such as Intel and a few others to find ways to use edge computing and create standards around managing data flow in and out of vehicles with the cellphone industries or the hardware industries. Working with a company like Kymeta helps us find ways to use their technology to handle larger amounts of data and make use of large amounts of bandwidth that is available through satellite,” said Lobenstein.

In a world of fully autonomous vehicles the road of the next decade truly will become an information superhighway – with data streams flowing down from thousands of satellites to receiving towers littered across the horizon, bouncing between radio masts, antennas and cars (Vehicle to Vehicle [V2V] and Vehicle to Infrastructure [V2X] communications). Last week, Broadcom ratcheted up its autonomous vehicle business by announcing the largest tech-deal ever to acquire Qualcomm for $103 billion. The acquisition would enable Broadcom to dominate both aspects of autonomous communications that rely heavily on satellite uplinks, GPS and vehicle communications. Broadcom CEO Hock Tan said, “This complementary transaction will position the combined company as a global communications leader with an impressive portfolio of technologies and products.” Days earlier, Tan attend a White House press conference with President Trump boasting of plans to move Broadcom’s corporate office back to the United States, a very timely move as federal regulators will have to approve the Broadcom/Qualcomm merger.

The merger news comes months after Intel acquired Israeli computer vision company, Mobileye for $15 billion. In addition to Intel, Broadcom also competes with Nvidia which is leading the charge to enable artificial intelligence on the road. Last month, Nvidia CEO Jensen Huang predicted that “It will take no more than 4 years to have fully autonomous cars on the road. How long it takes for the vast majority of cars on the road to become that, it really just depends.” Nvidia, which traditionally has been a computer graphics chip company, has invested heavily in developing AI chips for automated systems. Huang shares his vision, “There are many tasks in companies that can be automated… the productivity of society will go up.”

Industry consolidation represents the current state of the autonomous car race as chip makers volley to own the next generation of wireless communications. Tomorrow’s 5G mobile networks promise a tenfold increase in data streams for phones, cars, drones, industrial robots and smart city infrastructure. Researchers estimate that the number of Internet-connected chips could grow from 12 million to 90 million by the end of this year; making connectivity as ubiquitous as gasoline for connected cars. Karl Ackerman, analyst at Cowen & Co., said it best, “[Broadcom] would basically own the majority of the high-end components in the smart phone market and they would have a very significant influence on 5G standards, which are paramount as you think about autonomous vehicles and connected factories.”

The topic of autonomous transportation and smart cities will be featured at the next RobotLabNYC event series on November 29th @ 6pm with New York Times best selling author Dan Burstein/Millennium Technology Value Partners and Rhonda Binda of Venture Smarter, formerly with the Obama Administration – RSVP today.