Category robots in business

The key takeaway from Tuesday’s RobotLabNYC forum, on “Exploring The Autonomous Future,” was humans are the key to robot adoption. Dr. Howard Morgan of First Round Capital expressed to the audience of more than 100 innovators working within the automation ecosystem, the necessity of embracing “entrepreneurial marketing” to reach customers. Tom Ryden echoed Morgan’s sentiment in his presentation about Mass Robotics, conveying his startups’ frustrations with the pace of adoption. Dr. Eric Daimler, formerly of the Obama Administration, concluded the evening succinctly by exclaiming, “we only adopt what we trust.” Trust is key for crossing the chasm.

Intuitive Surgical this past year celebrated its 17 year of operations with close to a million robotic surgeries completed last year. According to a recent Gallup Poll, medical professionals are the most trusted individuals in our society, even more than one’s clergy. The fact that robot-assisted surgery has become so routine and accepted by doctors and their patients is proof positive that in some industries we have already crossed the trust threshold.

Understanding how Intuitive’s Da Vinci robot built trust within the medical community could offer parallels to other areas of the automation industry. Robotic-assisted surgery or “telerobotics,” is the evolution of two modern technologies: 1) telepresence or telemanipulation; and 2) laparoscopic surgery. In 1987, French physician Dr. Philippe Mouret performed the first minimally invasive gallbladder surgery using an endoscope-like device to remotely guide his instruments via video to remove the damaged organ. By the 1990’s, laparoscopic surgery became commonplace, driving the demand for more precision through mechanics and computer-aided techniques. A decade later, Intuitive received FDA approval for its Da Vinci robot for general surgery, which has since been expanded for prostate, neurological, and thoracic procedures. Telerobotics evolved not just from the availability of advanced technologies, but from the demand for less invasive procedures by the most trusted people in America.

Last month, the FDA approved the Da Vinci Xi Systems, enabling Intuitive Surgical to market less expensive systems and gain marketshare with smaller medical institutions globally. “This new system enables access to Intuitive’s leading and proven robotic-assisted surgical technology at a lower price point. Customers around the globe have different needs from a clinical, cost and technology perspective; Intuitive’s goal is to meet those needs by providing a range of products and solutions: the da Vinci X System helps us continue to do so,” said CEO Dr. Gary Guthart.

According to the press release, Da Vinci X System is “a focused-quadrant surgery and features flexible port placement and 3D digital optics, while also including advanced instruments and accessories from its Xi system.” Another determining factor of Intuitive Surgical’s success is the interoperability of the instruments. Rather than just an endoscope that provides video feeds, Da Vinci is equipped with multiple end effectors that mimics traditional instruments guided by experienced surgeons telerobotically.

Patients trust the robot because it is simply augmenting their doctor’s skills with greater precision. This is reinforced by shorter recovery periods and better outcomes. Recently, Oxford published a research study which took place over a nine-year period that concluded patients who opted for robotic lobectomies had better lung cancer outcomes. As a new generation of surgeons embraces the robotic future, the market for abdominal surgical robots is expected to grow from $2.9 billion in 2017 to $12.9 billion by 2022.

Trusting robots with our bodies might seem like a difficult premise to uphold, but robots have been saving lives on the front lines since 1972. Bomb-defusing machines have been utilized in the most dangerous situations worldwide from Afghanistan to Jerusalem to New York City. Today, almost every police department and military have an arsenal of remote-controlled explosive removal devices.

Dr. Sethu Vijayakumar, director of the Edinburgh Centre for Robotics in the United Kingdom, explains, “One of the target areas, in terms of [the] use of robots, is for going into dangerous situations. Robots can go in, be operated from a safe distance, and, in a worst-case scenario, be sacrificed.”

Similar to robotic medicine, trust-based systems for the military are built by teleporting human expertise into dangerous situations. Pittsburgh-based RE2 Robotics took this concept to a new level with its Robotic Manipulation System announced last week. The RE2 System now enables users to actually use their limbs and hands to control the robot’s movements and grippers to quickly defuse explosives.

RE2 CEO explained the rationale for his new product, “Often times, you still need the human intellect to perform those tasks. But they’re dangerous, so the question is, how can we project that human capability remotely, so they’re still able to do their job and leverage the human intellect to solve a really big problem? That’s what we’re trying to do — keep the human safe, but allowing them to still do their job.”

While rover looks remarkably similar to Endeavor’s (formerly iRobot) Packbot that has been widely deployed by the US military in Iraq, Afghanistan and elsewhere, the control system is novel and more reliable in the high-pressure situations. Pedersen says, “If you’re going to project that human capability, the most human way to control it is to have it be as much like you as possible. That’s where we’ve come over the past decade, having true human-like capability. It’s no coincidence that these robots look like human torsos. These systems are a projection of you, remotely. It’s almost like an avatar, where you’re dealing with a threat out of harm’s way.”

While today the operator of RE2’s robot stands at a safe distance watching the video feed on a laptop, the company is developing a virtual reality headset accessory for the control system to enable the professional to completely immerse himself into the situation. Pedersen also plans to expand the use cases for his technology to civilian markets such as search & rescue, disaster recovery (like Fukushima) and medicine.

“Yes, people could use this technology for other means. But our charter is saving lives and extending it into new markets like health care, where we can do patient assist. [We can] help a person from a wheelchair to a bed or a wheelchair to a toilet, as the brawn for a caregiver,” touts Pedersen.

While we are years away from fully trusting autonomous systems with our lives, it appears from these two examples that the first step is enabling machines to augment our most trusted citizens. As yesterday was Father’s Day it is only appropriate I share with my readers my gift – GrillBot. The disclaimer on the box, however, does make me question when I plan to use it; fear of death is kind of a big deal!

By Peter Reuell, Harvard Staff Writer

At the beginning of the decade, George Whitesides helped rewrite the rules of what a machine could be with the development of biologically inspired “soft robots.” Now he’s poised to rewrite them again, with help from some plastic drinking straws.

Inspired by arthropod insects and spiders, Whitesides and Alex Nemiroski, a former postdoctoral fellow in Whitesides’ Harvard lab, have created a type of semi-soft robot capable of standing and walking. The team also created a robotic water strider capable of pushing itself along the liquid surface. The robots are described in a recently published paper in the journal Soft Robotics.

Unlike earlier generations of soft robots, which could stand and awkwardly walk by inflating air chambers in their bodies, the new robots are designed to be far nimbler. Though real-world applications are still far off, the researchers hope the robots eventually could be used in search operations following natural disasters or in conflict zones.

“If you look around the world, there are a lot of things, like spiders and insects, that are very agile,” said Whitesides, the Woodford L. and Ann A. Flowers University Professor at Harvard. “They can move rapidly, climb on various items, and are able to do things that large, hard robots can’t do because of their weight and form factor. They are among the most versatile organisms on the planet. The question was, how can we build something like that?”

The answer, Nemiroski said, came in the form of your average drinking straw.

“This all started with an observation that George made, that polypropylene tubes have an excellent strength-to-weight ratio. That opened the door to creating something that has more structural support than purely soft robots have,” he said. “That was the building block, and then we took inspiration from arthropods to figure out how to make a joint and how to use the tubes as an exoskeleton. From there it was a question of how far can your imagination go? Once you have a Lego brick, what kind of castle can you build with it?”

What they built, he said, is a surprisingly simple joint.

Whitesides and Nemiroski began by cutting a notch in the straws, allowing them to bend. The scientists then inserted short lengths of tubing which, when inflated, would force the joints to extend. A rubber tendon attached on either side would then cause the joint to retract when the tubing deflated.

Armed with that simple concept, the team built a one-legged robot capable of crawling, and moved up in complexity as they added a second and then a third leg, allowing the robot to stand on its own.

“With every new level of systems complexity, we would have to go back to the original joint and make modifications to make it capable of exerting more force or to be able to support the weight of larger robots,” Nemiroski said. “Eventually, when we graduated to six- or eight-legged arthrobots, making them walk became a challenge from a programming perspective. For example, we looked at the way ants and spiders sequence the motion of their limbs and then tried to figure out whether aspects of these motions were applicable to what we were doing or whether we’d need to develop our own type of walking tailored to these specific types of joints.”

While Nemiroski and colleagues were able to control simple robots by hand, using syringes, they turned to computers to control the sequencing of their limbs as the designs increased in complexity.

“We put together a microcontroller run by Arduino that uses valves and a central compressor,” he said. “That allowed us the freedom to evolve their gait rapidly.”

Though Nemiroski and colleagues were able to replicate ants’ distinctive “triangle” gait using their six-legged robot, duplicating a spider-like gait proved far trickier.

“A spider has the ability to modulate the speed at which it extends and contracts its joints to carefully time which limbs are moving forward or backward at any moment,” Nemiroski said. “But in our case, the joints’ motion is binary due to the simplicity of our valving system. Either you switch the valve to the pressure source to inflate the balloon in the joint, and thus extend the limb, or you switch the valve to atmosphere to deflate the joint and thus retract the limb. So in the case of the eight-legged robot, we had to develop our own gait compatible with the binary motion of our joints. I’m sure it’s not a brand-new gait, but we could not duplicate precisely how a spider moves for this robot.”

Developing a system that can fine-tune the speed of actuation of the legs, Nemiroski said, would be a useful goal for future research, and would require programmable control over the flow rate supplied to each joint.

“We hit that limitation in the system, which I’m actually pretty proud of, because it means we pushed it to its absolute limit,” he said. “We took the basic concept and asked how far can we go before we would have to make radical alterations to how these limbs work, and we found that limit at the eight-legged robot. We were able to make it walk, but if you wanted to make it walk faster, or to add more limbs — for example, to support a load — you would have to start rethinking the system from the ground up.”

Though it may be years before the robots find their way into real-world applications, Whitesides believes the techniques used in their development — particularly the use of everyday, off-the-shelf materials — can point the way toward future innovations.

“I don’t see any reason to reinvent wheels,” he said. “If you look at drinking straws, they can make them at, effectively, zero cost and with great strength, so why not use them? These are academic prototypes, so they’re very light weight, but it would be fairly easy to imagine building these with a lightweight structural polymer that could hold a substantial weight.”

“What’s really attractive here is the simplicity,” added Nemiroski. “This is something George has been championing for some time, and something I grew to appreciate deeply while I was in his lab. For all the complexity of movement and structural integrity we get out of these robots, they’re remarkably simple in terms of construction and control. Using a single, easy-to-find material and a single concept for an actuator, we could achieve complex, multidimensional motion.”

This post was originally published on The Harvard Gazette. Click here to view the original.

This research was supported with funding from the U.S. Department of Energy, DARPA, the Natural Sciences and Engineering Research Council of Canada, the National Science Foundation, the Swedish Research Council, and the Wyss Institute for Biologically Inspired Engineering at Harvard University.

In episode three, season three of Talking Machines, we dive into overfitting, take a listener question about unbalanced data and speak with Professor (Emeritus) Tom Dietterich from Oregon State University.

If you enjoyed this episode, you may also want to listen to:

• Talking Machines: Restricted Boltzmann Machines, with Eric Lander
• Talking Machines: Automatic Translation and t-SNE, with Hal Daume
• Talking Machines: Generative art and Hamiltonian Monte Carlo, with Doug Eck
• Talking Machines: Machine learning and the Flint water crisis, with Jake Abernethy
• Talking Machines: Gaussian processes and OpenAI, with IIya Sutskever

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Prof. Pierre Dillenbourg and the team from the Computer-Human Interaction in Learning and Instruction (CHILI) Lab, explain how they are building robots to use in the classrooms of tomorrow. It is CHILI’s goal to deeply integrate Human-Computer Interaction (HCI) and learning sciences, especially in addressing practical problems in learning, teaching, and instruction.

If you enjoyed this ‘meet the lab’ video, you can also watch another in the NCCR Robotics series below:

• Meet the labs of NCCR Robotics: Paik Lab
• Meet the labs of NCCR Robotics: Auke Ijspeert and BioRob

It was a hectic but interesting week at the 2017 IEEE International Conference on Robotics and Automation in Singapore. Here’s a brief recap of some of my observations and experiences.

Popular Fields

Computer/robotic vision and navigation-related (SLAM, localization) are still some of the most popular, or at least well-attended fields at these conferences. My colleagues said that some of the manipulation / grasping sessions were well attended too.

The influence and growth in machine learning / deep learning seems to have slowed somewhat. This is possibly related to the fact we’re increasingly seeing some of the best robotics learning work going to machine learning or computer vision conferences, or potentially to new conferences like the Conference on Robot Learning.

Workshops continue to grow in importance

There seems to be an increasing number of attendees coming primarily to attend workshops, some of which very well attended, even at the tail end of the conference on Friday afternoon (a good sign!). This trend mirrors what we’re seeing at some of the other robotics and computer vision conferences, where there are very well attended workshops.

The event-vision based workshop, in particular, was impressive in how different it was to the 2015 incarnation, which was sparsely attended. This one had heaps of top researchers, industry, and lots of demos around the room.

Some of the mystery stealth startups in self-driving cars like Zoox showed off a little of what they are doing:

The shark tank environment has died down a little

Although I did manage to witness a couple of specific individuals spending the entire week being whisked away for recruiting lunches / dinners / drinks, the crazy recruiting environment of recent conferences seems to have died down, at least a little.

This is perhaps a testament to the fact that the fields of AI and self-driving cars are starting to mature a little – most of the major players now have a more stable, ongoing strategy so there’s not so much chaotic recruiting activity. It’s also clear that there’s still a significant talent shortage, especially at a middle seniority level – highly qualified postdocs with several years of domain experience (machine learning, perception, autonomous vehicles, SLAM) and leadership experience are like hens’ teeth!

Major local startups like NuTonomy showcased their work at a special event. I managed to get along separately and see that they are improving leaps and bounds in their capability and now have some pretty impressive capability showcases.

The bigger picture

Plenaries and keynotes like that of Hiroaki Kitano (Sony Labs, “Nobel Turing Challenge: Grand Challenge of AI, Robotics, and Systems Biology”) reminded us of the big picture regarding robotics and AI. We always seem to use inspiration from the space race:

And we got to see Peter Corke introduce ICRA2018 in Brisbane, Australia!

I enjoyed talking at the Visual Localization session (another packed out session with standing room only) on our paper, “Deep Learning Features at Scale for Visual Place Recognition“.

The social functions were good and the conference hotel was incredible, especially the infinity pool at the top of the hotel, 57 floors up, looking over the Marina. The night safari was fun too.

Overall, a great week and always great to reconnect with hundreds of colleagues and collaborators from around the world. See you next year in Brisbane!

Mosquitos kill more humans every year than any other animal on the planet and conventional methods to reduce mosquito-borne illnesses haven’t worked as well as many hoped. So we’ve been hard at work since receiving this USAID grant six months ago to reduce Zika incidence and related threats to public health.

Our partners at the joint FAO/IAEA Insect Pest Control Lab in Vienna, Austria have been working to perfect the Sterile Insect Technique (SIT) in order to sterilize and release male mosquitos in Zika hotspots. Releasing millions of said male mosquitos increases competition for female mosquitos, making it more difficult for non-sterilized males to find a mate.

We learned last year at a USAID Co-Ideation Workshop that this technique can reduce the overall mosquito population in a given area by 90%. The way this works is by releasing millions of sterilized mosquitos using cars, helicopters and/or planes, or even backpacks.

Our approach seeks to complement and extend (not replace) these existing delivery methods. The challenge with manned aircraft is that they are expensive to operate and maintain. They may also not be able to target areas with great accuracy given the altitudes they have to fly at.
Cars are less expensive, but they rely on ground infrastructure. This can be a challenge in some corners of the world when roads become unusable due to rainy seasons or natural disasters. What’s more, not everyone lives on or even close to a road.

Our IAEA colleagues thus envision establishing small mosquito breeding labs in strategic regions in order to release sterilized male mosquitos and reduce the overall mosquito population in select hotspots. The idea would be to use both ground and aerial release methods with cars and flying robots.

The real technical challenge here, besides breeding millions of sterilized mosquitos, is actually not the flying robot (drone/UAV) but rather the engineering that needs to go into developing a release mechanism that attaches to the flying robot. In fact, we’re more interested in developing a release mechanism that will work with any number of flying robots, rather than having a mechanism work with one and only one drone/UAV. Aerial robotics is evolving quickly and it is inevitable that drones/UAVs available in 6-12 months will have greater range and payload capacity than today. So we don’t want to lock our release mechanism into a platform that may be obsolete by the end of the year. So for now we’re just using a DJI Matrice M600 Pro so we can focus on engineering the release mechanism.

Developing this release mechanism is anything but trivial. Ironically, mosquitos are particularly fragile. So if they get damaged while being released, game over. What’s more, in order to pack one million mosquitos (about 2.5kg in weight) into a particularly confined space, they need to be chilled or else they’ll get into a brawl and damage each other, i.e., game over. (Recall the last time you were stuck in the middle seat in Economy class on a transcontinental flight). This means that the release mechanism has to include a reliable cooling system. But wait, there’s more. We also need to control the rate of release, i.e., to control how many thousands mosquitos are released per unit of space and time in order to drop said mosquitos in a targeted and homogenous manner. Adding to the challenge is the fact that mosquitos need time to unfreeze during free fall so they can fly away and do their thing, i.e., before they hit the ground or else, game over.

We’ve already started testing our early prototype using “mosquito substitutes” like cumin and anise as the latter came recommended by mosquito experts. Next month, we’ll be at the FAO/IAEA Pest Control Lab in Vienna to test the release mechanism indoors using dead and live mosquitos. We’ll then have 3 months to develop a second version of the prototype before heading to Latin America to field test the release mechanism with our Peru Flying Labs. One of these tests will involve the the integration of the flying robot and the release mechanism in terms of both hardware and software. In other words, we’ll be testing the integrated system over different types of terrain and weather conditions in Peru specifically.

We are already developing the mission control app pictured above to program our autonomous flights. The app will let the operator decide how many sterilized mosquitos to release at any given time and location. Our field tests in Peru will also seek to identify the optimal flight parameters for the targeted and homogenous delivery of sterilized mosquitos. For example, what is the optimal speed and altitude of the flying robot to ensure that the mosquitos are released over the intended areas?

Our Peru Flying Labs has already developed expertise and capacity in cargo drone delivery, most recently in projects in the Amazon Rainforest with the Ministry of Health (more here). This new Zika reduction project –and in particularly the upcoming field tests — will enable us to further build our Peruvian team’s capacity in cargo space. The plan is for Peru Flying Labs to operate the flying robots and release mechanisms as need once we have a more robust version of the release mechanism. The vision here is to have a fleet of flying robots at our Flying Labs equipped with release mechanisms in order to collectively release millions of sterilized mosquitos over relatively large areas. And because our Peruvian colleagues are local, they can rapidly deploy as needed.

For now, though, our WeRobotics Engineering Team (below) is busy developing the prototype out of our Zurich office. So if you happen to be passing through, definitely let us know, we’d love to show you the latest and give you a demo. We’ll also be reaching out the Technical University of Peru who are members of our Peru Flying Labs to engage with their engineers as we get closer to the field tests in country.

As an aside, our USAID colleagues recently encouraged us to consider an entirely separate, follow up project totally independently of IAEA whereby we’d be giving rides to Wolbachia treated mosquitos. Wolbachia is the name of bacteria that is used to infect male mosquitos so they can’t reproduce. IAEA does not focus on Wolbachia at all, but other USAID grantees do. Point being, the release mechanism could have multiple applications. For example, instead of releasing mosquitos, the mechanism could scatter seeds. Sound far-fetched? Think again.

RoboCup is an international scientific initiative with the goal to advance the state of the art of intelligent robots. Established in 1997, the original mission was to field a team of robots capable of winning against the human soccer World Cup champions by 2050.

To celebrate 20 years of RoboCup, the Federation is launching a video series featuring each of the leagues with one short video for those who just want a taster, and one long video for the full story. Robohub will be featuring one league every week leading up to RoboCup 2017 in Nagoya, Japan.

This week, we consider being part of the RoboCupIndustrial league. RoboCupIndustrial is a competition between industrial mobile robots focusing on logistics and warehousing systems. In anticipation of Industry 4.0, participants compete in automation through robots, autonomous systems, and mobile robot technology. You’ll hear about the history and ambitions of RoboCup from the trustees, and inspiring teams from around the world.

Short version:

Long version:

Want to watch the rest? You can view all the videos on the RoboCup playlist below:

https://www.youtube.com/playlist?list=PLEfaZULTeP_-bqFvCLBWnOvFAgkHTWbWC

Please spread the word! If you would like to join a team, click here for more information.

• RoboCup video series: 20 years of history
• RoboCup video series: Soccer league
• RoboCup video series: Rescue league
• RoboCup video series: @Home league
• What has twenty years of RoboCup taught us?
• How we evolved a winning strategy for the RoboCup competition by imitating nature

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

For the first time on Broadway human and drone performances fuse to create a new form of artistic expression. The magic happened in Cirque du Soleil’s first musical on Broadway: ‘Paramour’ at the Lyric Theatre. The show is themed on the Golden Age of Hollywood and follows the life of a poet who is forced to choose between love and art. The contributions of the technology firm Verity Studios include the choreography of the drone show segment, the frame and lighting design of the drone costumes, and all underlying drone technologies. The system was operated by the show’s automation team, with Verity Studios providing maintenance services twice per year. The dancing drones completed almost 400 shows, including more than 7,000 autonomous takeoffs, flights, and landings.

Over the past year, 398 audiences of up to 2,000 people witnessed an octet of colorful lampshades perform an airborne choreography during Cirque du Soleil’s Broadway show Paramour, which ran until April 20^th. The work behind the design and choreography of the flying lampshades, which turn out to be self-piloted drones, bears the signature of the Swiss high-tech company Verity Studios.

But how novel is it really that robots have appeared in theater? Since Karel Capek’s science fiction play R.U.R. (short for Rossum’s Universal Robots) introduced the word “robot” to the English language and to science fiction almost 100 years ago, the technical challenges of incorporating robots into live performance and theater have been overwhelming. Before these Broadway drones, nearly all theater robots were remote-controlled puppets, relying on humans hidden off-scene to steer their movements and provide their intelligence. What holds for Broadway, also holds for the even bigger picture.

It is intriguing that the aerial revolution gained momentum on the ground first: Even though the complexity of most advanced industrial automation systems found in state-of-the art factories and warehouses hardly compares to the complexity of drone show systems that allow for the autonomous navigation of dozens, hundreds, or thousands of drones in a safety-critical environment, one single example stands out. The only comparable systems that can serve as a template for operating large numbers of autonomous mobile robots with high demands for reliability and safety are those developed by the technology company Kiva Systems (bought by Amazon for USD 775 million in 2012), where Verity Studios’ founder Professor Raffaello D’Andrea and his former colleagues developed a solution for automated storage and retrieval in warehouses. His team unleashed thousands of mobile robots in warehouses, which started the robotics gold rush in the Silicon Valley. Now, Professor D’Andrea’s team is pursuing an analogous, even more ambitious goal in the air.

Placing intelligent, autonomous flying machines in live theater presents a multi-faceted challenge: creating a compelling performance with safety, reliability, and ease of operation. While the compelling performance translates into shaping a convincing creative concept around the drones’ choreography, the latter chiefly points to designing computerized systems in lieu of making use of human pilots: Verity Studios’ drones are flying mobile robots and navigate autonomously, piloting themselves, only supervised by a human operator. To navigate autonomously, these drones require a reliable method for determining their position in space. Since GPS is usually not available indoors, Verity Studios has built on more than a decade of research and development at the Flying Machine Arena of Switzerland’s Federal Institute of Technology (ETH Zurich) to provide a novel localization method.

The first groups of autonomous robots are now meeting the live event industries’ high requirements: Initially, Verity Studios set the stage for widely recognized flying machines in entertainment in 2014, by collaborating with Cirque de Soleil on Sparked, a short film in which 10 siblings of the similarly dressed 8 quadcopters from Paramour dance, hover, and flicker around a stunned human actor. Sparked was named a Winner of the 2016 New York City Drone Film Festival. However, it is one thing to feature drones in a 4-minute video, where the makers have the luxury to shoot it 100 times and then choose the best shot. Live performances on Broadway running 8 times per week in front of up to 2,000 people every time, by contrast, require the drone shows to work every single time. Verity Studios has met these requirements in the best sense: During their one-year run, the drones on Broadway safely completed more than 7,000 autonomous take-offs, flights, and landings. How was that possible?

In contrast to the long list of safety incidents involving remote-controlled drones at events, performance and reliability are key to Paramour’s dancing drones which are custom-made and hand-crafted in Switzerland. Each of the eight flying machines featured in Paramour uses 80 sensors to fly and performs roughly 1.5 billion calculations per second. Each critical calculation is double-checked every time it is performed in case any of the concerned processors makes a mistake. In other words, accidents are prevented by self-test and monitoring. And even if something does happen, a back-up is in place. For example, on the show of 2017-02-08, the battery from one of the eight vehicles could not provide sufficient power during the takeoff phase. The quadrocopter detected the problem and returned to the ground on only two propellers.

In most cases, the drones withstand any point of failure, but where a single point of failure is unavoidable, a fail-safe or weakest link design is used. The failure of any single system component cannot result in an unmanaged loss of control, but must be handled appropriately. For instance, a drone should be able to land in a controlled fashion despite a failure of any one of its motors, any part of its electronics, any one of its cables or connectors, and any one of its batteries. Similarly, loss of communication between a drone and its ground station must not create a dangerous situation. This can be achieved by designing a fully redundant system, i.e., a system that can continue operation, or trigger a safe emergency behavior, if any one of its components fails. While such designs can be more expensive to create than designs without duplication, they provide much higher safety (and help reduce insurance costs). Their decades-long history in the manned aviation industry has shown this approach’s effectiveness and provides a treasure trove of experience and evidence for the emerging drone industry.

One way to design such a redundant system is to first design a simple flight-capable system, to then duplicate all its components (e.g. sensors, processors, actuators) to achieve the necessary redundancy, and to carefully design switch-over from a failed system to its backup. Importantly, the design of such a system cannot stop with the design of fully redundant drones, but needs to extend to all other critical components of the drone show system, including the positioning system, communications architecture, e-stop systems, and control stations. In the end, this uncompromising approach allows for a degree of safety and robustness that makes those intelligent and autonomous robots suitable for performance next to theater audiences. The result is a one-of-a-kind performance that extends the traditional palette of light, sound, stage effects, and interaction with human performers by translating an intimate character beat into an unexpected visual motif.

The enticing reality is that there are many more areas to explore, ranging from live concerts to active scenography, that offer dramatic possibilities for performing robots on a much broader scale. Once again, not only such stage flyers, but also Lucie micro-drones will make history, not only on Broadway or at Amazon, but all across the domain of performing arts revolutionizing the scene with a new realm of creative expression.

Update: The response to Tertill’s crowdfunding campaign has amazed and delighted us! Pledges totalling over $250,000 have come from 1000+ backers. We’re shipping to all countries, with over a fifth of Tertill’s supporters coming from outside the United States. But the end is near; Tuesday (11 July) is the last full day of the campaign. After that Tertill’s discounted campaign price will no longer be available and delivery in time for next year’s (northern hemisphere) growing season cannot be assured.

Franklin Robotics has launched a Kickstarter campaign for Tertill, their solar-powered, garden-weeding robot.

Tertill lives in your garden, collecting sunlight to power its weed patrol, and cutting down short plants with a string trimmer/weed whacker with almost no intervention required. Available for about $300USD, the fully autonomous Tertill is the first weeding robot available to home gardeners.

Tertill is round, short, has four wheel drive and extreme camber wheels. It uses proprietary algorithms to ensure that it finds as many weeds as it can, using its sensors to distinguish between weeds and crops based on height. As Tertill sees it: small plants are weeds, big plants are crops. But what if you have seedlings? Simply place a plant collar around seedlings to inform Tertill that this plant is wanted.

Tertill doesn’t use a camera for weed discrimination because Franklin wants to put Tertill to work in your garden now. In the not-too-distant future, inexpensive vision systems may reliably distinguish between plants and crops even when both are seedlings. But, according to Franklin, such system are not yet ready.

Tertill is designed such that one robot can easily keep up with the weeds in a typical sized garden (that’s about 100 square feet in the USA). Tertill will usually handle gardens considerably larger than this, but how much larger depends on weather and local weed types. And Terrill’s can work in teams, coexisting harmoniously in the same space. Multiple Tertill robots will simply avoid each other.

Check out the video below:

But Tertill needs a small barrier to keep it from wandering off beyond the borders of your garden. A garden fence, the wooden border of a raised bed, or any sort of edging that is two or more inches tall will work.

Otherwise, Tertill is designed to live in your garden and require very little attention. Summer showers won’t bother it. And, under normal conditions, you won’t have to worry about charging Tertill—the sun will see to that. But, if Tertill has been in the dark for so long (months) that its battery is completely exhausted, a USB connection can be used to charge the battery.

Franklin believes that small, simple robots can help solve big, complex problems. Franklin’s robots aim to help gardeners by reducing the tedium and physical challenge of weeding, making gardening more fun. The data their robots collect helps gardeners garden better, improving yield and quality. Their robots aim to benefit the environment by eliminating the need for herbicides and returning organic matter to the soil.

Click here to visit Franklin’s website and learn more.

The Robot Academy is a new learning resource from Professor Peter Corke and the Queensland University of Technology (QUT), the team behind the award-winning Introduction to Robotics and Robotic Vision courses. There are over 200 lessons available, all for free.

The lessons were created in 2015 for the Introduction to Robotics and Robotic Vision courses. We describe our approach to creating the original courses in the article, An Innovative Educational Change: Massive Open Online Courses in Robotics and Robotic Vision. The courses were designed for university undergraduate students but many lessons are suitable for anybody, as you can easily see the difficulty rating for each lesson. Below are several examples of image formation and 3D vision.

The geometry of image formation

The real world has three dimensions but an image has only two. We can use linear algebra and homogeneous coordinates to understand what’s going on. This more general approach allows us to model the positions of pixels in the sensor array and to derive relationships between points on the image and points on an arbitrary plane in the scene.

Watch the rest of the Masterclass here.

How images are formed

How is an image formed? The real world has three dimensions but an image has only two: how does this happen and what are the consequences? We can use simple geometry to understand what’s going on.

Watch the rest of the Masterclass here.

3D vision

An image is a two-dimensional projection of a three-dimensional world. The big problem with this projection is that big distant objects appear the same size as small close objects. For people, and robots, it’s important to distinguish these different situations. Let’s look at how humans and robots can determine the scale of objects and estimate the 3D structure of the world based on 2D images.

Watch the rest of the Masterclass here.

If you liked this article, you may also enjoy:

• Smart Grasping System available on ROS Development Studio
• Programming your NAO robot for human interaction
• The Robot Academy: The open online robotics education resource 
• Machine Learning with OpenAI Gym on ROS Development Studio
• SICK LMS full LIDAR teardown
• Envisioning the future of robotics
• How to mount an external removable hard drive on a robot

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

No matter how great a surgeon is, robotic assistance can bring a higher level of precision to the operating table. The ability to remotely operate a robot that can hold precision instruments greatly increases the accuracy of surgical procedures like thoracoscopic surgery, which is used to treat lung cancer.

Of the two most common types of lung cancer, non-small cell lung cancer (NSCLC) is a good candidate for surgery because the tumors spread slowly and are more localized. Since more than 80 percent of people with lung cancer have NSCLC, surgery is a common treatment.

Lung cancer usually starts when epithelial cells that line the inside of the lungs rapidly reproduce into cancerous cells, creating tumors inside the lungs. These tumors have been traditionally removed directly by the skilled hands of a surgeon. Today, we’re starting to see more tumors being removed by robots that are controlled by skilled surgeons. This process is known as robotic assisted lung surgery.

The key benefits of robotic assisted lung surgery are:

It’s minimally invasive and small incisions are used. Traditional lung surgery procedures utilize a large incision across the chest wall. When using robotic assisted surgery, the incision is about half as long. The robotic arms can be maneuvered more intricately, so the incision doesn’t need to be as large. Often a second incision is made for the removal of tissue.

Patients often experience a faster recovery time. When a patient has a robotic lobectomy, they’re often back to their normal routine in a week’s time. Older patients in their 70s and 80s also experience a good recovery with this type of procedure, which is good news considering they’re usually not great candidates for open surgeries. The small incisions are responsible for a faster recovery time compared to the long recovery time after traditional lung surgery.

There are several robotic assisted procedures used to treat lung cancer:

Video-Assisted Thoracoscopic Surgery (VATS)

Fewer than 5 percent of thoracic surgeries in the US are performed robotically. Rush University is one of a few US medical institutions that offer a full range of robotic-assisted lung cancer procedures, including VATS. In fact, Rush is known as a leader in VATS.

VATS is the most common type of robotic assisted lung surgery. With this procedure, one incision is made to insert the surgical instruments for the removal of tissue, while another incision is used to place the camera. The surgeon maneuvers the surgical instruments while an assistant operates the camera so the surgeon can see what’s happening.

The da Vinci Si robotic system

Rush University is now using the da Vinci Si robotic system for lung surgery, where it was once only used for other, more general surgeries.

How the da Vinci Si robotic system works

A surgeon sits at a console to control a four-armed robot that’s been positioned above the patient lying on the operating table. The surgeon observes the scene from a screen that displays images coming from a camera. As the surgeon moves his or her controls, the robot responds accordingly in real-time, and the surgical instruments attached to the robotic arms perform the surgery.

One of Rush University Medical Center’s thoracic surgeons, Gary Chmielewski, says, “any motion I can do with my hands, the robot can simulate inside the patient with more precision and less tissue trauma. It all works together for a better operation that’s easier on the patient.”

Nine years of positive results

A study published in 2014, analyzed the results of different types of robotic lobectomies for treating lung cancer over a period of 9 years. The study was designed to evaluate the evolution of technique as well as the robotic technology. The study found a positive trend in patient outcomes when they opted for the upgraded robotic systems compared to the standard systems.

Twenty years ago, who would have thought having robotic assisted lung surgery would become the most popular, most effective option?

If robots can be used to help surgeons remove tumors from lung cancer patients, what else is possible with robotics? The possibilities are endless.

June 5, 2017 – June 11, 2017

At the Center for the Study of the Drone

At Washington D.C.’s WTOP radio, Center for the Study of the Drone co-director Dan Gettinger discusses the growing use of drones by public safety agencies in the U.S.

News

A U.S. drone strike reportedly killed eight individuals at an al-Shabab command post in southern Somalia. It is believed to be the first U.S. drone strike in Somalia since President Trump relaxed rules for targeting al-Shabab militants. (New York Times)

Canada announced that it is moving ahead with its acquisition of strike-capable drones. The planned acquisition is part of a $62 billion investment in new military systems and technologies. In a press conference, Canadian Prime Minister Justin Trudeau said that the government will carefully review how to best use the drones prior to deployment. (CBC)

Alphabet will sell Boston Dynamics, which makes quadruped and bipedal robots, as well as robotics firm Schaft, to SoftBank Group, a Japanese telecommunications corporation. Boston Dynamics has become well-known for developing advanced, dexterous robots. Alphabet acquired Boston Dynamics in 2013 and Schaft in 2014. (Recode)

A U.S. fighter jet shot down an Iranian drone near the Al Tanf border crossing between Syria and Iraq. In a statement, the U.S.-led coalition said that the drone had earlier attacked U.S. allies on the ground and that it was approximately the size of a U.S. MQ-1 Predator, indicating that it was likely a Shahed-129 drone. A variety of Iranian drones have supported pro-regime forces in Syria since at least 2012. (The Daily Beast) For more on the drones operating in Syria and Iraq, click here.

Commentary, Analysis, and Art

In an in-depth interview at Drone Wars UK, a former British drone pilot discusses his experience flying the MQ-9 Reaper.

Meanwhile, at Over the Horizon, U.S. Air Force Colonel Joseph Campo discusses U.S. drone operations and the psychological toll of those operations on drone pilots.

At the MIT Technology Review, Jonathan W. Rosen looks at how Zipline’s drones are reducing the time it takes to deliver blood to remote medical facilities in Rwanda.

The WeRobotics Global conference featured several panels that considered the use of unmanned systems for humanitarian and conservation purposes. (iRevolutions)

At Recode, April Glaser writes that the Trump administration’s plan to privatize air traffic control could accelerate the push to develop a national drone tracking system.

Meanwhile, at Business Insider, Jonathan Camhi writes that it may be difficult for privately-organized drone air traffic control systems to make it to market.

At IEEE Spectrum, David Schneider looks at some failed drone bills and asks why they never became laws.

DRONEII.com considers the different energy sources available for drones and alternatives to the popular Li-Po batteries.

In a study published in the online Journal of Unmanned Vehicle Systems, Paul Nesbit examines the growing number of close encounters between drones and manned aircraft in Canada. (Phys.org)

In a paper published in the Arabian Journal of Geosciences, David R. Green and Cristina Gomez argue that drones are especially useful for pipeline monitoring.  

In the Georgetown Security Studies Review, Shannon Dick considers the U.S. targeted killing program in the context of Just War Theory.

The U.S. Department of Defense’s 2017 report on Chinese military capabilities highlights China’s progress in developing and exporting drones. (PDF)  For more on China’s drones, click here.

At TechCrunch, Brian Heater looks at how the Robust Adaptive Systems Lab at Carnegie Mellon is developing drones and robots that can work alongside humans.

At CBC, Kyle Bakx visits a village in Canada that is seeking to transform itself into a hub for companies that develop and test drones.

At Radio New Zealand, Roy Olsson, the head of the ICARUS project at the Defense Advanced Research Projects Agency, discusses the future of disposable drones.

A comic strip at xkcd considers the challenges of countering drones. (xkcd)

Know Your Drone

NASA successfully tested five drones in a number of scenarios as part of its unmanned aircraft traffic management system program. (Military Embedded Systems)

Meanwhile, Alphabet conducted a live test of its developmental air traffic control system for drones. (MIT Technology Review)

California startup Airspace Systems has unveiled a drone designed to hunt and catch rogue drones. (Forbes)

Turkish defense firm Meteksan Savunma has developed the country’s first indigenous automatic takeoff and landing system for drones. (Defense News)

Amazon has been awarded a patent for a safety system for its delivery drones that shuts off the aircraft’s rotors if it detects an imminent collision. (Drone Life)

U.S. drone maker General Atomics Aeronautical Systems is planning to demonstrate an ultra high-speed laser satellite communication link for drones as well as a drone-mounted high-power kill laser. (Aviation Week)  

U.K. technology firm Wirth Research has unveiled a hydrogen-powered vertical take-off and landing mapping drone. (Unmanned Systems Technology)

Researchers at Georgia Tech are developing the Miniature Autonomous Blimp, an indoor unmanned blimp that has a camera that can detect faces and autonomously follow individuals. (IEEE Spectrum)

Researchers at MIT have developed a small low-cost gas powered drone that can remain aloft for up to five days at a time. (New Atlas)

The China Electronics Technology Group Corporation conducted a flight test of a swarm of 119 small drones. (The Economic Times)   

Several Japanese shipbuilders and shipping firms are partnering to develop remote controlled cargo ships. (The Guardian)

French drone maker Delair-Tech conducted a beyond-line-of-sight flight of a commercial drone at a distance of 30 miles. (Drone Life)

Undersea drone maker Robosea has unveiled the BIKI, a small remote controlled unmanned undersea vehicle. (Unmanned Systems Technology)

Drones at Work

The city of Garfield in New Jersey is revising a recently adopted drone ordinance after it was found to contravene FAA authority. (NorthJersey.com) For more on local drone laws, click here.

Navajo County sheriff’s deputies in Arizona used a drone to assist in the search for a man who went missing in a local forest. (White Mountain Independent)

A group of women in Majorca, Spain claim that a drone spied on them while they were sunbathing on a boat. (The Telegraph)

A man in Mountain View, California crashed a drone into a high voltage power line, causing a power outage that affected 1,600 people. (San Francisco Chronicle)

New Zealand’s first drone store opened to large crowds in Auckland. (News Hub)

A Hezbollah TV channel has broadcast footage of what it claims shows U.S. drone that was filmed by an Iranian drone over southeast Syria. (Reuters)

Officials in Fiji are planning to use drones to assist with the country’s 2017 census. (Fiji Sun)

A helicopter operated by the Charlotte-Mecklenburg Police Department in North Carolina was involved in a close encounter incident with a drone. (WSOCTV)

Meanwhile, aerial firefighting operations over the Lizard Fire in Arizona were interrupted for four hours after a drone was spotted in the airspace over the fire. (Sierra Vista Herald)

The Philippines has been revealed to be an operator of the Boeing Insitu ScanEagle drone. (IHS Jane’s 360)

South Korea has found what it claims to be a North Korean surveillance drone near the border between the two countries. (Reuters)

Police in Lawrenceville, Georgia are planning to purchase a drone for a range of operations. (AJC.com)

In Nepal, the Kanjirowa-Bullis STEM project tested a drone near Mt Everest at a high altitude. (The Himalayan Times)

An Australian filmmaker has been charged with jeopardizing Cambodia’s national security after he flew a drone over an election rally. (The Phnom Penh Post)

Australian computer vision firm Sentient Vision has announced that its Kestrel automated surveillance system will be deployed aboard the Australian Defence Force’s AeroVironment Wasp tactical drones. (Unmanned Systems Technology)

Researchers used an autonomous boat to obtain data from an acidic crater lake at the Poás Volcano in Costa Rica. (The Atlantic)

Industry intel

The European Commission officially launched the 5.5 billion Euro European Defence Fund, which will include funding for unmanned systems development and acquisition. (Press Release)

Japanese investor Kotaro Chiba launched the $9 million DroneFund to invest in drone startups. (The Bridge)

The U.S. Navy awarded Academi Training Center, Insitu, PAE ISR, and AAI contracts for sea and land-based drones for intelligence, surveillance, and reconnaissance. The total value of the contracts is $1.73 billion. (DoD)

The U.S. Special Operations Command awarded AAI Corp. a multiple award with a $475 million maximum order ceiling for Mid-Endurance Unmanned Aircraft Systems. (DoD)

The U.S. Special Operations Command awarded Insitu a multiple award with a $475 million maximum order ceiling for Mid-Endurance Unmanned Aircraft Systems. (DoD)

The U.S. Air Force awarded SpaceX a contract to launch the X-37B unmanned spacecraft in August. (Reuters)

The U.S. Air Force awarded Radio Hill Technologies a $2.5 million contract for 100 Block 3 Dronebuster counter-UAS detection and jamming systems. (Shephard Media)

The Defense Advanced Research Projects Agency awarded BAE Systems two contracts worth $5.4 million to develop a payload architecture that will enable smaller drones to multitask, a program known as CONCERTO. (IHS Jane’s 360)

The Defense Advanced Research Projects Agency awarded Oregon State University a $6.5 million grant to improve the trustworthiness of artificial intelligence in robots and drones. (Press Release)

Boeing is partnering with Huntington Ingalls, the largest military shipbuilder in the U.S., to help to develop the company’s Echo Voyager extra large unmanned undersea vehicle (XLUUV). (Defense One)

ECA Group was awarded a contract by an undisclosed NATO navy for its A9-M autonomous underwater vehicles for mine countermeasure operations. (Naval Technology)

Dronefence, a German counter-UAS startup, announced seed funding from Larnabel Ventures, VP Capital, Boundary Holding, and Technology and Business Consulting Group. (Press Release)

Maritime Robotics, a Norwegian company specializing in unmanned maritime vehicles, will provide Seabed Geosolutions with its Mariner unmanned surface vehicle for seismic exploration. (Maritime Journal)

The Duke Energy Foundation awarded Butler Technology and Career Development School in Hamilton, Ohio a $45,000 grant to teach students how to use drones. (WLWT5)

Virginia Governor Terry McAuliffe awarded three $50,000 grants to the Counter-Drone Research Corporation, TruWeather Solutions, and dbS Productions to conduct research into drones and autonomous vehicles. (StateScoop)  

Nongtian Guanjia, a Chinese agricultural drone startup, raised $7 million in a funding round led by Gobi Partners. (Deal Street Asia)  

Autonomous vehicle and artificial intelligence company Cognata raised $5 million from Emerge, Maniv Mobility, and Airbus Ventures. (TechCrunch)

For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.

Uber, the global ride-sharing transportation company, has named two replacements to recover from the recent firing of Anthony Levandowski who headed their Advanced Technologies Group, their OTTO trucking unit, and their self-driving team. Levandowski was fired May 30th.

Eric Meyhofer

Meyhofer, who before coming to Uber was the co-founder of Carnegie Robotics and a CMU robotics professor, was also part of the group that came to Uber from CMU (see below). He has now been named to head Uber’s Advanced Technologies Group (ATG) self-driving group and will report directly to Uber CEO Travis Kalanick.

The ATG group is charged with developing the self-driving technologies of mapping, perception, safety, data collection and learning, and self-driving for cars and trucks.

Sensors that determine distances are integral to the process. Elon Musk said recently that LiDAR isn’t needed because cameras, sensors, software and high-speed GPUs can do the same tricks at a fraction of the cost. Levandowski favored LiDARs, particularly newly developed solid state LiDAR technologies.

Anthony Levandowski

Levandowski, the previous head of the ATG, joined Google to work with Sebastian Thrun on Google Street View, started a mobile mapping company that experimented with LiDAR technology and another to build a self-driving LiDAR-using car (a Prius). Google acquired both companies including their IP. In 2016 Levandowski left Google to found Otto, a company making self-driving kits to retrofit semi-trailer trucks. Just as the kit was launched, Uber acquired Otto and Levandowski became the head of Uber’s driverless car operation in addition to continuing his work at Otto.

The Levandowski case, which caused Uber to fire him, revolves around the intellectual property and particularly the LiDAR-related technologies that Google’s and Uber’s self-driving plans revolve around. Getting the cost of perception down to a reasonable level has been part of the bigger challenge of self-driving technology and LiDAR technology is integral to that plan.

Google’s Waymo self-driving unit is implying in their suit that in return for bringing Google’s IP to Uber, Uber gave Levandowski $250 million in stock grants. Uber has called Waymo’s claims baseless and an attempt to slow down a competitor.

Waymo also claims that Uber has a history of “stealing” technology and includes the time in 2015 when Uber hired away 50+ of the Carnegie Mellon University robotics team – a move that cost Uber a reported $5 billion and created havoc at CMU and the National Robotics Engineering Centre (NREC) which lost 1/3 of their staff to Uber. The move was preceded by establishing a strategic partnership with CMU to work together on self-driving technologies. Four months later, Uber hired the 50.

The Daily Mail headlines said:

Carnegie Mellon left decimated after Uber poaches 40 top-rated robotic researchers to help them develop self-driving cars

• Carnegie Mellon ‘in crisis’ after mass defection of scientists to Uber
• Uber hope their fleet of taxis will not require drivers in the future
• Used $5 billion from investors to poach at least 40 from the National Robotics Engineering Center
• Uber took six principal investigators and 34 engineers

Brian Zajac

Zajac has been on Uber’s self-driving team since 2015 after stints with Shell and the U.S. Army. Now he becomes the new chief of hardware development and reports to Meyhofer.

David Morris, writing for Fortune, wrote:

“Zajac will now bear a great deal of responsibility for cracking the driverless car problem, which Uber CEO Travis Kalanick has described as “existential” to the company. Uber loses huge amounts of money, and many observers think eliminating the cost of drivers is its only realistic path to profitability.”

Bottom line:

Uber has research teams in Silicon Valley, Toronto and Pittsburgh all working to perfect Level 5 autonomous driving capabilities before any of their competitors are able to duplicate the process. Google, Baidu, Yandex, Didi Chuxing, a few of the Tier 1 component makers, and many others including all the major car companies are racing forward with the same intentions. Levandowski’s firing caused a big gap in Uber’s self-driving project management and fear amongst their investors. Uber hopes that these two changes, Meyhofer as overall head and Zajac as hardware chief, will quell the fears that Uber is losing their momentum.

It is unclear if Masayoshi Son, Chairman of Softbank, was one of the 17 million YouTube viewers of Boston Dynamic’s Big Dog before acquiring the company for an undisclosed amount this past Thursday. What is clear is the acquisition of Boston Dynamics by Softbank is a big deal. Softbank’s humanoid robot Pepper is trading up her dainty wheels for a pair of sturdy legs.

In expressing his excitement for the acquisition, Masayoshi Son said, “Today, there are many issues we still cannot solve by ourselves with human capabilities. Smart robotics are going to be a key driver of the next stage of the Information Revolution, and Marc and his team at Boston Dynamics are the clear technology leaders in advanced dynamic robots. I am thrilled to welcome them to the SoftBank family and look forward to supporting them as they continue to advance the field of robotics and explore applications that can help make life easier, safer and more fulfilling.”

Marc Raibert, CEO of Boston Dynamics, previously sold his company to Google in 2013. Following the departure of Andy Rubin from Google, the internet company expressed buyers remorse. Raibert’s company failed to advance from being a military contractor to a commercial enterprise. It became very challenging incorporating Boston Dynamic’s zoo of robots (mechanical dogs, cheetahs, bulls, mules and complex humanoids) into Google’s autonomous strategy. Since Rubin’s exit in 2014, rumors of buyers acquiring Boston Dynamics from Google have ranged from Toyota Research to Amazon Robotics. Softbank represents a new chapter for Raibert, and possible the entire industry.

Raibert’s statement to the press gave astute readers a peek of what to expect: “We at Boston Dynamics are excited to be part of SoftBank’s bold vision and its position creating the next technology revolution, and we share SoftBank’s belief that advances in technology should be for the benefit of humanity. We look forward to working with SoftBank in our mission to push the boundaries of what advanced robots can do and to create useful applications in a smarter and more connected world.” A quick study of the assets of both companies reveals how Boston Dynamics could help Softbank in its mission to build robots to benefit humanity.

Softbank’s premier robot is Pepper, a four-foot tall social robot that has been mostly deployed in Asia as a customer service agent. Recently, Pepper, as part of Softbank’s commitment to the Trump administration to invest $50 billion in the United States has been spotted in stores in California. As an example, Pepper proved itself as a valuable asset last year to Palo Alto’s premier tech retailer B8ta, accounting for a 6 times increase in sales. To date, there are close to 10,000 Pepper robots deployed worldwide, mostly in Asian retail stores. However, Softbank is also the owner of Sprint with 4,500 cell phone stores across the USA, and a major investor in WeWork with 140 locations globally servicing 100,000 members – could Pepper be the customer service agent or receptionist of the future?

According to Softbank’s website, Pepper is designed to be a “day-to-day companion,” with its most compelling feature being the ability to perceive emotions. Softbank boasts that their humanoid is the first robot ever to recognize moods of homo sapiens and adapt its behavior accordingly. While this is extremely relevant for selling techniques, Softbank is most proud of Pepper being the first robot to be adopted into homes in Japan. It is believed that Pepper is more than a point-of-purchase display gimmick, but an example of the next generation of caregivers for the rising elderly populations in Japan and the United States. According to the Daily Good, “Pepper could do wonders for the mental engagement and continual monitoring of those in need.” Its under $2,000 price point also provides an attractive incentive to introduce the robot into new environments, however wheel-based systems are a limitation in the home with clutter floors, stairs and other unforeseen obstacles.

Boston Dynamics is almost the complete opposite of Softbank; it is a research group spun out of MIT. Its expertise is not in social robots but in military “proofs of concepts” like futuristic combat mules. The company has touted some of the most frightening mechanical beasts to ever walk the planet from metal Cheetahs that sprint at over 25 miles per hour to mechanized dogs that scale mountains with ease to one of the largest humanoids every built that has an uncanny resemblance to Cyberdyne’s T-800. In a step towards commercialization, Boston Dynamics released earlier this year its newest monster – a wheel-biped leg robot named Handle that can easily lift over a hundred pounds and jump higher than Lebron James. Many analysts pontificated that this appeared to be Boston Dynamics attempt to prove its relevance to Google with a possible last mile delivery bot.

In an IEEE interview when Handle debuted last February, Raibert exclaimed, “Wheels are a great invention. But wheels work best on flat surfaces and legs can go anywhere. By combining wheels and legs, Handle can have the best of both worlds.” IEEE writer Evan Ackerman questioned, after seeing Handle, if the next generation of Boston Dynamic’s humanoids could feature legs with roller-skate like shoes. One is certain that Boston Dynamics is the undisputed leader of dynamic control and balance systems for complex mechanical designs.

Leading roboticist Dan Kara of ABI Research confirmed that “these [Boston Dynamics] are the world’s greatest experts on legged mobility.”

If walking is the expertise of Raibert’s team and Softbank is the leader of cognitive robotics with a seemingly endless supply of capital, the combination could be the first real striding humanoid capable of human-like emotions. By 2030 there will be 70 million people over the age of 65 years in America, with a considerable smaller amount of caregivers. To answer this call researchers are already converting current versions of Pepper into sophisticated robotic assistants. Last year, Rice University unveiled a “Multi-Purpose Eldercare Robot Assistant (MERA)” which is essentially a customized version of the Softbank’s robot. MERA is specifically designed to be a home companion for seniors that “records and analyzes videos of a person’s face and calculates vital signs such as heart and breathing rates.” Rice University partnered with IBM’s Aging-in-Place Research Lab to create MERA’s speech technology. IBM’s Lab founder, Susann Keohane, explained that Pepper “has everything bundled into one adorable self.” Now with Boston Dynamic’s legs Pepper could be a friend, physical therapist, and life coach walking side by side with its human companion.

Daniel Theobald, founder of Vecna Technologies – a healthcare robotic company, summed it best last week, “I think Softbank has made a major commitment to the future of robotics. They understand that the world economy is going to be driven by robotics more and more.”

Next Tuesday we will dive further into the implications of Softbank’s purchase of Boston Dynamics with Dr. Howard Morgan/First Round Capital, Tom Ryden/MassRobotics and Dr. Eric Daimler/Obama White House at RobotLabNYC’s event on 6/13 @ 6pm WeWork Grand Central (RSVP).

3! 2! 1! Go! Suddenly, robots jerk into motion and zoom across the field to score points, crossing over several types of terrain and shooting balls into high and low goals. Another buzzer sounds, drivers pick up their controls and all six robots—three per alliance—are now under human control. As these huge 120-pound robots score points, cheers ring through a packed stadium, fueled by high school students who worked hard to build their robot in just six weeks. As the match ends, nervous and excited students wait to see who is the winner of the 2016 world championship.

This was my last match as a member of the Girls of Steel FIRST Robotics Competition Team #3504. FIRST (For Recognition and Inspiration of Science and Technology) is a robotics program for students from K-12, and I was in the last division, FRC. The program is about more than introducing students to STEM and giving them hands-on experience, it’s about helping students to grow and have positive impacts by recognizing community service efforts, celebrating good values, developing soft skills, and guiding students to pursue higher education.

The next fall, I was off to college at the Illinois Institute of Technology in Chicago, studying to become a mechanical engineer. For the first time in my life, I was on my own. My time was so swept away by schoolwork, clubs, exploring the city, and making new friends that FIRST became a distant memory. Now, I fear that if I hadn’t bumped back into it, I would have lost touch with the program that played such a critical role in my life. While at a robotics club meeting, sign ups for the FRC Midwest Regional Planning Committee were passed around. Wanting to somehow get involved, I signed up.

I had no idea what to expect as I walked into a big conference room with a friend, and fellow FRC alum, for our first committee meeting. As the meeting progressed—densely filled with information and detailed plans for upcoming seasons and younger leagues—I sat there stunned. Regardless of if they were an alum or not, there were so many people who dedicated their time and effort to make this program work for students. It was a wake-up call. During my four years of being a member, I took so much for granted and didn’t realize the magnitude of hard work that volunteers and mentors put in for the students. They were the ones who supported and helped make me the person that I am today. So, to all of the volunteers and mentors who may be reading this, thank you for everything, I couldn’t have done it without you.

Just like that, I was hooked on FIRST once again. I kept going to meetings, trying to help as much as I could while making connections with other volunteers. Right before the season started, there were a couple teams who had a shortage of mentors, which is how I found Hawks on the Horizon FRC team #5125. The first meeting I went to with the students opened my eyes; I was so used to being a member, I had no idea how to mentor FRC. Eventually, I learned how to be a different figure in a familiar situation and how to adjust to the differences between a large all-girls team, with many resources, and a small family-like team. Yet, without a doubt, I knew this was my new team. From the very first day, I was welcomed with open arms by the mentors and students, who made sure I came back.

“Who is your role model?” was a common question for me when I was very young, and I’d respond with a superhero. Now it’s never asked, but I have a better answer since I’ve gotten the chance to meet some of them. One of my role models is a student mentor from my old team. Although I haven’t seen her in a long time, I found myself remembering conversations we’d had years ago about some of the challenges of being a student mentor. Knowing I wasn’t alone, and how she’d dealt with some sticky situations, helped me guide students to find their own answers and become comfortable being hands-off myself. I’ve learned so much from past mentors like her, fellow mentors, and the students themselves. I’m happy to have found my FIRST family again.

The Midwest Regional was a whole new experience for me. During high school, my competition days were hectic, filled with fixing the robot, talking to other teams, and cheering for our alliance. At this past regional, it was an entirely different world. I was the student volunteer coordinator, helping run the student ambassador program and talking to people about how to get involved. I only visited my team when I got the chance. I was less aware of specific robots and paid more attention to what happens behind the scenes to ensure regional runs like a well-oiled machine. I want to be more involved in the process, since these competitions ignited a passion for engineering within me, and I cannot express with words how grateful I am.

Regardless of the changes within the past year, by the end of the competition, I still rocked the uniform. I’m incredibly proud of Hawks on the Horizon, thankful for Girls of Steel, and amazed by everyone in the Midwest Planning Committee. This transition from being a member to mentor has been an amazing journey and it doesn’t stop here. I’ll be working to give back for the rest of my life by helping make programs like FIRST happen for future students. Thank you to everyone who is a part of this organization and to find a team or event, go to firstinspires.org.

If you enjoyed this article, you may also want to read:

• FIRST 2013 photo essay: Girls of Steel in the trenches at the FRC championship finals
• FIRST 2014 photo essay: Girls of Steel back in St. Louis
• ‘It’s a reminder we are making a difference in the world’: Girls of Steel follow-up
• Getting into the driver’s seat: Girls of Steel tell us how they got hooked on robotics
• The story of Atlas, FRC Team #3504 Girls of Steel
• A vision for Nigeria’s youth: FIRST LEGO league competition held in Nigeria