If you are involved in the UK Robotics and Autonomous Systems (RAS) sector, we’d love to hear from you. Please fill in this survey.
In January this year, the UK Government published a Green Paper on “Building our Industrial Strategy” (https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/611705/building-our-industrial-strategy-green-paper.pdf). In it is set an ‘open door’ challenge to industry to come to the Government with proposals to transform and upgrade their sector through ‘Sector Deals’. Businesses rather than the Government are being encouraged to identify what companies need in order to enhance their competitiveness as a sector.
This is not about the Government providing additional funding; rather, it is an open call to business to organise behind strong leadership, like the automotive and aerospace sectors, to address shared challenges and opportunities.
Government is looking for businesses to collaborate with other stakeholders, such as universities and local leaders to produce a clear proposal for boosting the productivity of their sector, setting out detailed plans to address challenges such as:
delivering upgrades in productivity, including in supply chains;
promoting competition and innovation;
facilitating long term investment and coordination between suppliers and primes;
accelerating growth across the value chain, including by identifying where the greatest value can be gained from technology development and investment;
developing and growing the strengths of particular clusters;
increasing exports; and looking at how we can use trade and investment deals to help the sector;
commercialising research across sectors; and
boosting skills and the number of high value, high productivity jobs.
To help provide evidence for the proposed Robotics Sector Deal, we would like to understand what activities are taking place in the UK that are in alignment with the existing RAS Strategy, and what new ones could be enabled by Government action. To this end we are reaching out to the UK RAS Community to collect this information. All you need to do is fill in this short survey.
When answering the questions, please endeavour to be specific and thorough. Your answers will not be publicly published, and will only be used to inform the proposed Sector Deal (and will therefore remain confidential between the RAS Special Interested Group Advisory Board and the Government).
Please feel free to give us more than one set of answers to this questionnaire. We will collate the answers and provide a high-level synthesis of them, rather than providing the details, so please don’t worry about overwhelming Government with detail!
If you are not familiar with the way we use the terms Asset, Skills, Coordination, Clusters and Challenges, then please have a quick look at the RAS UK Strategy here.
Thanks very much for your help. Your input is greatly valued and will contribute to something that will be of huge benefit to our sector, as well as the wider community.
PARAMOUR on Broadway – A Cirque du Soleil Musical. Credit: Richard Termine
Last year, Intel partnered with Lady Gaga on the Super Bowl Halftime Show to showcase its latest aerial technology called “Shooting Star.” Intel did a reprise performance of its Shooting Star technology for Singapore’s 52nd birthday this past week. Instead of fireworks, the tech-savvy country celebrated its National Day Parade with a swarm of 300 LED drones animating the night sky with shapes, logos, and even a map of the country.
Intel’s global drone chief, Anil Nanduri, explained, “There’s considerably more operational complexity in handling a 300 drone fleet, compared with 100 drones in a show. It’s like juggling balls in your hand. You may be able to juggle three, but if you juggle nine, you may have to throw them higher and faster to get more time.” Earlier this year, Intel first showcased its 300 drone show at Coachella music festival on the heels of claiming the Guinness World Record of a 500 drone performance.
Choreographed drones are winning the hearts of Cirque du Soleil theatergoers with a fleet of flying acrobatic vehicles dancing around its human performers. These drones are the brain child of Professor Raffaello D’Andrea of the ETH Zurich, Switzerland and his new startup Verity Studios. D’Andrea is probably best known as one of the three founders of Kiva Systems and now he is taking the same machine intelligence that sorts and delivers goods within Amazon’s warehouses to safely wow audiences worldwide. The flying lampshades (shown in the video below) are actually autonomous drones that magically synchronize with the dancers, without safety nets or human operators.
Verity’s customer, Cirque du Soleil’s Chief Creative Officer Jean-Francois Bouchard, said D’Andrea’s “flying machines are unquestionably one of the most important statements of the PARAMOUR show.” The key to the flying machines’ success over 7,000 autonomous flights on Broadway is its proprietary technologies that enable multiple self-piloted drones to be synchronized within flight. Verity’s drone is part of a larger performance system called “Stage Flyers.”
The Stage Flyer platform has proven itself in the field, flying next to thousands of people each evening by having built-in redundancy to any single failure. According to Verity’s website, the system is “capable of continuing operation in spite of a failed battery, a failed motor, a failed connector, a failed propeller, a failed sensor, or a failure of any other component. This is achieved through the duplication of critical components and the use of proprietary algorithms, which enable safe emergency responses to component failures.” This means that the drones can operate safely around audiences and performers alike, carrying payloads of cameras, mirrors, and special lighting effects. As shown in the above diagram, the drone system includes a fleet of self-piloted drones that utilize one positioning system and control unit. The company boasts that its system only takes a few hours to install, calibrate and learn how to operate.
Drone swarms are not just for entertainment, as today there are a number of upstarts and established players utilizing these mechanics for e-commerce fulfillment centers. Last June, Amazon was issued a patent for a “Multi-Level Fulfillment Center for Unmanned Aerial Vehicles.” The beehive-looking distribution center (above) is designed to facilitate traffic between inbound and outbound delivery drones. The patent illustration details “multiple levels with multiple landing and take-off locations, depending on local zoning regulations.”
This is all part of Amazon’s larger plan to grow its robotic workforce over the next two to three years. Instead of human truck drivers, the patent displays delivery bays that open and close automatically based on the direction of the drones and interior platforms that cycle around the hive. CB Insights reports that the patent describes “impact dampeners,” such as nets, for receiving inbound drones and “launch assist mechanisms,” such as fans, for launching outbound drones. It appears that Amazon will be looking again to technology like D’Andrea’s research to revolutionize its global network of warehouses with synchronized swarms of drones that safely soar above human workers.
Amazon’s competitor Walmart announced last summer its plans to utilize swarms of indoor drones for inventory management, replacing the need for people climbing dangerous ladders to manually scan labels. The New York Times first reported last July that the retailer applied for a FAA exemption to begin testing drones inside its massive distribution centers. Shekar Natarajan, the vice president of last mile and emerging science for Walmart, demonstrated for the newspaper how swarms of drones could easily move up and down aisles, from floor to ceiling, to scan 30 barcode images a second (an efficiency that would be impossible for even the most agile humans). Walmart has publicly boasted that it will spend close to $3 billion on new technology and other cost-saving investments to bolster its e-commerce business which is growing, but at a slower pace than its nemesis Amazon.
The race to dominate the warehouse has led to increased investments in the logistics sector and even an accelerator dedicated solely to technology around distribution centers. Chattanooga, Tennessee-based Dynamo Accelerator showcased last May its second cohort of startups. One of the most successful showings was Chicago-based Corvus Robotics, a software company that uses indoor aerial drones to scan inventory (similar to the Walmart example above). According to Dynamo’s managing directors, Corvus is building enabling tools that allow operators to fly drones autonomously, scan & sync barcodes, and enter the SKU data into the existing warehouse management system.
Santosh Sankar, the director at Dynamo, explained his accelerator’s mission succinctly in a recent blog post: “We believe our focus and hands-on approach is one of our value-adds. As such, we’re leaning into being seed investors and upholding our commitment to transforming our industry by focusing on our founders and our corporate partners. We’ve opted to not hold a quota for our programs and hone in on companies we can truly help because that ultimately makes for good seed investments.” Sankar added that several of the program participants “are already well on their way to generate ($1 million or more) in annual revenue and/or have raised their initial round of capital.”
Corvus may be the latest indoor drone startup to enter an already crowded warehouse market, which includes established players like the Hardis Group, Smartx, and DJI. Drones continue to amuse, amaze and evolve as the growing need for more unmanned systems in our lives appears to be almost insatiable. Next month, we plan to dig deeper into the drone market with our RobotLabNYC event series on September 19th at 6pm in WeWork Grand Central. Joining us in the discussions include thought-leaders from NASA, AUVSI, and Genius NY, RSVP today, as space is limited.
Australia retired the IAI Heron drone after eight years of service. The Royal Australian Air Force is currently seeking a replacement. Image via Aviation Analysis Wing
August 7, 2017 – August 13, 2017
News
Two U.S. airstrikes targeted members of al-Shabab in Somalia. In a statement, the U.S. Africa Command said the strikes were “conducted within the parameters of the proposal approved by the President in March 2017.” A spokesperson said the strikes were carried out by drones. (ABC News)
In a classified guidance issued last month, the Department of Defense authorized the U.S. military to seize or destroy drones that appear to endanger the airspace or pose a threat to military installations. Pentagon spokesperson Captain Jeff Davis told reporters that the move was a response to the “increase of commercial and private drones” in the U.S. (Reuters)
An Iranian drone interfered with a U.S. F/A-18E Super Hornet jet in the Persian Gulf. In a statement, the U.S. military said that the drone flew within 100 feet of the manned fighter as it was preparing to land on the USS Nimitz. (Washington Post)
Turkish authorities have detained a Russian citizen with ties to ISIS for allegedly planning to use a drone to attack a U.S. military base. Police in Adana claim that Renad Bakiev admitted to reconnoitering Incirlik Air Base in southern Turkey. (Associated Press)
Commentary, Analysis, and Art
At the Financial Times, Louise Lucas writes that China-based drone manufacturer DJI is considering making a move toward the commercial sector.
The Los Angeles Times editorial board argues that the Los Angeles County Sheriff should adopt a more transparent process for integrating drones into police work.
At the Wall Street Journal, Nicole Friedman writes that insurance companies are increasingly relying on drones to inspect physical damage to properties.
At the San Francisco Chronicle, Carolyn Said looks at how robots and drones are taking on more roles in food delivery.
At WFTV, Lauren Seabrook writes that a new state law in Florida that offers more opportunities for drone businesses conflicts with local drone ordinances.
Kratos Defense and Security Solutions announced that a classified military drone that it is developing for an unnamed program will enter into production this year. (FlightGlobal)
Northrop Grumman is using an X-47B drone as a testbed for the Navy’s MQ-25A Stingray aerial refueling drone. (Aviation Week) For more on the X-47B, click here.
The San Francisco Public Utilities Commission approved a policy to use drones for construction management, environmental monitoring, and inspection. (San Francisco Examiner)
The Worldview International Foundation is planning to use drones to plant tree seeds as part of a reforestation effort in Myanmar. (Fast Company)
The Kansas Department of Transportation and app maker AirMap are partnering to develop a drone air traffic management system. (The Wichita Eagle)
The Israeli Defence Ministry is investigating reports that defense firm Aeronautics was asked by Azerbaijan to carry out a live demonstration of a loitering munition drone against Armenian forces. (Jerusalem Post)
The Muriwai Surf Life Saving Club in New Zealand is planning to use drones for patrolling a popular beach. (News Hub)
Researchers who used a drone last year to collect pollutant samples from the open burning of waste at an Army ammunition facility in Virginia found arsenic and other pollutants. (Phys.org)
For updates, news, and commentary, follow us on Twitter. The Weekly Drone Roundup is a newsletter from the Center for the Study of the Drone. It covers news, commentary, analysis and technology from the drone world. You can subscribe to the Roundup here.
In episode six of season three we chat about the difference between frequentists and Bayesians, take a listener question about techniques for panel data, and have an interview with Katherine Heller of Duke.
If you enjoyed this episode, you may also want to listen to:
The U.S. Army has ordered its members to stop using drones made by Chinese manufacturer SZ DJI Technology because of “cyber vulnerabilities.” The directive applies to all DJI drones and systems that use DJI components or software. It requires service members to “cease all use, uninstall all DJI applications, remove all batteries and storage media and secure equipment for follow-on direction.”
DJI has about 70% of the global commercial and consumer drone market according to Goldman Sachs analysts. The market, including military, is expected to be worth more than $100 billion over the next five years.
The Army's move appears to follow studies conducted by the Army Research Laboratory and the Navy which said there were risks and vulnerabilities in DJI products. The directive cites a classified Army Research Laboratory report and a Navy memo, as references for the order to cease use of DJI drones and related equipment.
DJI responded with the following statement on their website:
Some recent news stories have claimed DJI routinely shares customer information and drone video with authorities in China, where DJI is headquartered. This is false. A junior DJI staffer misspoke during an impromptu interview with reporters who were touring the DJI headquarters; we have attempted to correct the facts since then, but inaccurate stories are still posted online.
We want to emphasize that DJI does not routinely share customer information or drone video with Chinese authorities — or any authorities Any claims to the contrary are false.
In other DJI-related news, 3D Robotics (3DR), a previous camera drone competitor, announced a product partnership with DJI. Its 'Site Scan' aerial data analytics software platform now works with DJI drones, and is aimed at large construction and engineering companies using drones.
DJI director of strategic partnerships Michael Perry stated: “This integration is a significant milestone for the AEC industry. We’re excited that 3DR Site Scan users can now use DJI drones to convert images into actionable data that helps project stakeholders save time and manage costs.”
As AI surpasses human abilities in Go and poker – two decades after Deep Blue trounced chess grandmaster Garry Kasparov – it is seeping into our lives in ever more profound ways. It affects the way we search the web, receive medical advice and whether we receive finance from our banks.
The most innovative AI breakthroughs, and the companies that promote them – such as DeepMind, Magic Pony, Aysadi, Wolfram Alpha and Improbable – have their origins in universities. Now AI will transform universities.
We believe AI is a new scientific infrastructure for research and learning that universities will need to embrace and lead, otherwise they will become increasingly irrelevant and eventually redundant.
Through their own brilliant discoveries, universities have sown the seeds of their own disruption. How they respond to this AI revolution will profoundly reshape science, innovation, education – and society itself.
Deep Mind was created by three scientists, two of whom met while working at University College London. Demis Hassabis, one of Deep Mind’s founders, who has a PhD in cognitive neuroscience from UCL and has undertaken postdoctoral studies at MIT and Harvard, is one of many scientists convinced that AI and machine learning will improve the process of scientific discovery.
It is already eight years since scientists at the University of Aberystwyth created a robotic system that carried out an entire scientific process on its own: formulating hypotheses, designing and running experiments, analysing data, and deciding which experiments to run next.
Complex data sets
Applied in science, AI can autonomously create hypotheses, find unanticipated connections, and reduce the cost of gaining insights and the ability to be predictive.
AI is being used by publishers such as Reed Elsevier for automating systematic academic literature reviews, and can be used for checking plagiarism and misuse of statistics. Machine learning can potentially flag unethical behaviour in research projects prior to their publication.
AI can combine ideas across scientific boundaries. There are strong academic pressures to deepen intelligence within particular fields of knowledge, and machine learning helps facilitate the collision of different ideas, joining the dots of problems that need collaboration between disciplines.
As AI gets more powerful, it will not only combine knowledge and data as instructed, but will search for combinations autonomously. It can also assist collaboration between universities and external parties, such as between medical research and clinical practice in the health sector.
The implications of AI for university research extend beyond science and technology.
Philosophical questions
In a world where so many activities and decisions that were once undertaken by people will be replaced or augmented by machines, profound philosophical questions arise about what it means to be human. Computing pioneer Douglas Engelbert – whose inventions include the mouse, windows and cross-file editing – saw this in 1962 when he wrote of “augmenting human intellect”.
Expertise in fields such as psychology and ethics will need to be applied to thinking about how people can more rewardingly work alongside intelligent machines and systems.
Research is needed into the consequences of AI on the levels and quality of employment and the implications, for example, for public policy and management.
When it comes to AI in teaching and learning, many of the more routine academic tasks (and least rewarding for lecturers), such as grading assignments, can be automated. Chatbots, intelligent agents using natural language, are being developed by universities such as the Technical University of Berlin; these will answer questions from students to help plan their course of studies.
Virtual assistants can tutor and guide more personalized learning. As part of its Open Learning Initiative (OLI), Carnegie Mellon University has been working on AI-based cognitive tutors for a number of years. It found that its OLI statistics course, run with minimal instructor contact, resulted in comparable learning outcomes for students with fewer hours of study. In one course at the Georgia Institute of Technology, students could not tell the difference between feedback from a human being and a bot.
Global classroom
Mixed reality and computer vision can provide a high-fidelity, immersive environment to stimulate interest and understanding. Simulations and games technology encourage student engagement and enhance learning in ways that are more intuitive and adaptive. They can also engage students in co-developing knowledge, involving them more in university research activities. The technologies also allow people outside of the university and from across the globe to participate in scientific discovery through global classrooms and participative projects such as Galaxy Zoo.
As well as improving the quality of education, AI can make courses available to many more people. Previously access to education was limited by the size of the classroom. With developments such as Massive Open Online Courses (MOOCs) over the last five years, tens of thousands of people can learn about a wide range of university subjects.
It still remains the case, however, that much advanced learning, and its assessment, requires personal and subjective attention that cannot be automated. Technology has ‘flipped the classroom’, forcing universities to think about where we can add real value – such as personalised tuition, and more time with hands-on research, rather than traditional lectures.
Monitoring performance
University administrative processes will benefit from utilising AI on the vast amounts of data they produce during their research and teaching activities. This can be used to monitor performance against their missions, be it in research, education or promotion of diversity, and can be produced frequently to assist more responsive management. It can enhance the quality of performance league tables, which are often based on data with substantial time lags. It can allow faster and more efficient applicant selection.
AI allows the tracking of individual student performance, and universities such as Georgia State and Arizona State are using it to predict marks and indicate when interventions are needed to allow students to reach their full potential and prevent them from dropping out.
Such data analytics of students and staff raises weighty questions about how to respect privacy and confidentiality, that require judicious codes of practice.
The blockchain is being used to record grades and qualifications of students and staff in an immediately available and incorruptible format, helping prevent unethical behaviour, and could be combined with AI to provide new insights into student and career progression.
Universities will need to be attuned to the new opportunities AI produces for supporting multidisciplinarity. In research this will require creating new academic departments and jobs, with particular demands for data scientists. Curricula will need to be responsive, educating the scientists and technologists who are creating and using AI, and preparing students in fields as diverse as medicine, accounting, law and architecture, whose future work and careers will depend on how successfully they ally their skills with the capabilities of machines.
New curricula should allow for the unpredictable path of AI’s development, and should be based on deep understanding, not on the immediate demands of companies.
Addressing the consequences
Universities are the drivers of disruptive technological change, like AI and automation. It is the duty of universities to reflect on their broader social role, and create opportunities that will make society resilient to this disruption.
We must address the consequences of technological unemployment, and universities can help provide skills and opportunities for people whose jobs have been adversely affected.
There is stiff competition for people skilled in the development and use of AI, and universities see many of their talented staff attracted to work in the private sector. One of the most pressing AI challenges for universities is the need for them to develop better employment conditions and career opportunities to retain and incentivize their own AI workers. They need to create workplaces that are flexible, agile and responsive to interactions with external sources of ideas, and are open to the mixing of careers as people move between universities and business.
The fourth industrial revolution is profoundly affecting all elements of contemporary societies and economies. Unlike the previous revolutions, where the structure and organization of universities were relatively unaffected, the combinations of technologies in AI is likely to shake them to their core. The very concept of ‘deep learning’, central to progress in AI, clearly impinges on the purpose of universities, and may create new competition for them.
If done right, AI can augment and empower what universities already do; but continuing their missions of research, teaching and external engagement will require fundamental reassessment and transformation. Are universities up to the task?
This article was originally posted on the World Economic Forum. Click here to view the original.
Top 3 Robotic Applications in Primary Food Processing
Primary processing involves handling raw food products, which are cleaned, sorted, chopped, packaged, etc. Some foods, like raw vegetables, will only undergo primary processing before they are packaged for the consumer. Other foods will undergo secondary processing before packaging.
Up until quite recently, robotic processing at this stage has been limited or non-existent. Raw foods are variable in size, weight and shape. This makes it difficult for robots to handle them. However, recent developments in sensing and soft gripping has made it possible for robots to handle many raw foods.
1. Robotic Butchery
Butchery is a very difficult task to automate. Every animal carcass is different. A skilled butcher will adapt each cut to the shape and position of bones and meat. Some butchery tasks are simpler to automate than others. For example, high-volume chicken leg deboning is an established part of the meat processing industry.
Beef butchery has traditionally been very difficult to automate. Recently,beef manufacturer JBS has started looking for ways to introduce robots into their factories. Parts of the process are very dangerous for human workers. Rib cutting, for example, involves operating a high-speed circular saw for several hours. JBS has managed to automate this action using robot manipulators and various vision sensors. The application has improved safety and product consistency.
2. Fruit and Vegetable Pick and Place
Fruits and vegetables are challenging to handle with a robot due to their variable sizes and shapes. They also require delicate handling to avoid damage. For these reasons they have traditionally been handled by human workers. However, recent developments in gripping technologies look to change all that. Soft Robotics Inc has introduced a flexible gripper which can handle very delicate foods, even individual lettuce leaves!
Another example is Lacquey’s gripper, which uses paddles to lift soft fruits and vegetables.
3. Robotic Cutting and Slicing
Some cutting and slicing tasks are easy to automate. For example, even kitchen food processors can slice vegetables into uniform shapes. Robots are not needed for this type of simple automation.
For more advanced cutting and slicing, however, the food industry has relied on human workers but robotics is starting to make its way into the industry. Fish cutting, for example, involves detecting and removing defects from the fish as well as cutting fillets to uniform shapes and sizes.
Top 3 Robotic Applications in Secondary Food Processing
Secondary processing involves handling products which have already undergone primary processing. Robots have been used for several applications for a long time, particularly pick and place. However, recent developments have opened the door to even more advanced applications.
1. Product Pick and Place
You may be familiar with the high speed delta robots which are used to move food products around a production line. If not, here is a video:
This is an example of secondary processing pick and place. It is distinct from the vegetable pick and place mentioned above because the products are more uniform in shape and size. Uniform foods are much easier to handle robotically, so this application has been available in the food industry for many years.
2. Cake Decorating
One impressive application is robotic cake decoration. This involves using a robotic arm much like a 3D printer to pipe icing onto a cake. The Deco-Bot from Unifiller can pipe hand-drawn decorations onto cakes on a moving conveyor.
Cake cutting can also be done robotically, like the Katana waterjet cutting robot which can cut out intricate shapes in cakes using high pressure water.
3. Pizza Making
Artisan food producers sometimes worry that adding robots their process will make their products less “hand-made.” However, Silicon Valley pizza producer Zume is showing how robots can be produced to look like they have the human touch. Their pizzeria uses two robots: a delta robot to spread the tomato sauce and a ABB manipulator to tend the pre-baking ovens. While their system is far from fully automated, their goal is to make the pizza delivery industry a labor-free business.
Finally… Washing Up!
Contaminated food causes 48 million people in the USA to become sick annually. Robotic food processing has the potential to reduce this, by removing human workers from parts of the process, but this is only possible if the robots themselves do not cause contamination.
One of the more challenging issues for food automation is the fact that every piece of machinery must be thoroughly cleaned to avoid contamination. Robot manufacturers have been working to make their robot casings smoother, with better ingress ratings and no loose wires. This allows them to be thoroughly washed down at the end of each cycle.
In this system from JMP Automation, the two robots wash down the workcell with high powered water, and even wash down each other:
Researchers have devised a new way to monitor sleep stages without sensors attached to the body. Their device uses an advanced artificial intelligence algorithm to analyze the radio signals around the person and translate those measurements into sleep stages: light, deep, or rapid eye movement (REM). Image: Christine Daniloff/MIT
More than 50 million Americans suffer from sleep disorders, and diseases including Parkinson’s and Alzheimer’s can also disrupt sleep. Diagnosing and monitoring these conditions usually requires attaching electrodes and a variety of other sensors to patients, which can further disrupt their sleep.
To make it easier to diagnose and study sleep problems, researchers at MIT and Massachusetts General Hospital have devised a new way to monitor sleep stages without sensors attached to the body. Their device uses an advanced artificial intelligence algorithm to analyze the radio signals around the person and translate those measurements into sleep stages: light, deep, or rapid eye movement (REM).
“Imagine if your Wi-Fi router knows when you are dreaming, and can monitor whether you are having enough deep sleep, which is necessary for memory consolidation,” says Dina Katabi, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science, who led the study. “Our vision is developing health sensors that will disappear into the background and capture physiological signals and important health metrics, without asking the user to change her behavior in any way.”
Katabi worked on the study with Matt Bianchi, chief of the division of sleep medicine at MGH, and Tommi Jaakkola, the Thomas Siebel Professor of Electrical Engineering and Computer Science and a member of the Institute for Data, Systems, and Society at MIT. Mingmin Zhao, an MIT graduate student, is the paper’s first author, and Shichao Yue, another MIT graduate student, is also a co-author.
The researchers will present their new sensor at the International Conference on Machine Learning on Aug. 9.
Remote sensing
Katabi and members of her group in MIT’s Computer Science and Artificial Intelligence Laboratory have previously developed radio-based sensors that enable them to remotely measure vital signs and behaviors that can be indicators of health. These sensors consist of a wireless device, about the size of a laptop computer, that emits low-power radio frequency (RF) signals. As the radio waves reflect off of the body, any slight movement of the body alters the frequency of the reflected waves. Analyzing those waves can reveal vital signs such as pulse and breathing rate.
“It’s a smart Wi-Fi-like box that sits in the home and analyzes these reflections and discovers all of these changes in the body, through a signature that the body leaves on the RF signal,” Katabi says.
Katabi and her students have also used this approach to create a sensor called WiGait that can measure walking speed using wireless signals, which could help doctors predict cognitive decline, falls, certain cardiac or pulmonary diseases, or other health problems.
After developing those sensors, Katabi thought that a similar approach could also be useful for monitoring sleep, which is currently done while patients spend the night in a sleep lab hooked up to monitors such as electroencephalography (EEG) machines.
“The opportunity is very big because we don’t understand sleep well, and a high fraction of the population has sleep problems,” says Zhao. “We have this technology that, if we can make it work, can move us from a world where we do sleep studies once every few months in the sleep lab to continuous sleep studies in the home.”
To achieve that, the researchers had to come up with a way to translate their measurements of pulse, breathing rate, and movement into sleep stages. Recent advances in artificial intelligence have made it possible to train computer algorithms known as deep neural networks to extract and analyze information from complex datasets, such as the radio signals obtained from the researchers’ sensor. However, these signals have a great deal of information that is irrelevant to sleep and can be confusing to existing algorithms. The MIT researchers had to come up with a new AI algorithm based on deep neural networks, which eliminates the irrelevant information.
“The surrounding conditions introduce a lot of unwanted variation in what you measure. The novelty lies in preserving the sleep signal while removing the rest,” says Jaakkola. Their algorithm can be used in different locations and with different people, without any calibration.
Using this approach in tests of 25 healthy volunteers, the researchers found that their technique was about 80 percent accurate, which is comparable to the accuracy of ratings determined by sleep specialists based on EEG measurements.
“Our device allows you not only to remove all of these sensors that you put on the person, and make it a much better experience that can be done at home, it also makes the job of the doctor and the sleep technologist much easier,” Katabi says. “They don’t have to go through the data and manually label it.”
Sleep deficiencies
Other researchers have tried to use radio signals to monitor sleep, but these systems are accurate only 65 percent of the time and mainly determine whether a person is awake or asleep, not what sleep stage they are in. Katabi and her colleagues were able to improve on that by training their algorithm to ignore wireless signals that bounce off of other objects in the room and include only data reflected from the sleeping person.
The researchers now plan to use this technology to study how Parkinson’s disease affects sleep.
“When you think about Parkinson’s, you think about it as a movement disorder, but the disease is also associated with very complex sleep deficiencies, which are not very well understood,” Katabi says.
The sensor could also be used to learn more about sleep changes produced by Alzheimer’s disease, as well as sleep disorders such as insomnia and sleep apnea. It may also be useful for studying epileptic seizures that happen during sleep, which are usually difficult to detect.
Instead of worrying so much about robots taking away jobs, maybe we should worry more about wages being too low for robots to even get a chance. Seasonal labor for harvesting agricultural products, particularly fruits and vegetables, is dependent on human labor from a diminishing universe of willing workers.
Robots that can supplement or replace human workers in the harvesting process are being developed and tested in startups and academia, but almost all are not yet ready for prime time.
In a NY Times article written by Neil Irwin entitled Rethinking Low Productivity, productivity growth has been on a downward path since the Financial Crisis. Irwin, who writes about economic trends, asks whether the downward trend is the cause of low growth, or the result, a troubling question in the dynamics of the agriculture industry.
“Inventors and business innovators are always developing better ways to do things, but it takes a labor shortage and high wages to coax firms to deploy the investment it takes to actually put those innovations into widespread use.”
Those two phenomena are happening today in the U.S. agricultural industry. Labor rates are rising while willing laborers are diminishing — yet farmers are not yet investing in robotics. In fact, they are accommodating to the labor crisis by planting less and changing the crops they grow to be less labor intensive.
J.W. Mason, the author of a paper by the Roosevelt Institute cited as the basis for the thesis of Irwin’s article, says:
“If the labor market tightens and wages rise, that will be the impetus to get companies to consider more big-ticket innovations that generate productivity growth so that we don’t perpetuate the present conundrum in which both arguments cannot be true:
“On Mondays and Wednesdays, economists argue that wages are low because robots are taking people’s jobs.
On Tuesdays and Thursdays, it’s that we can’t have wages rise because productivity growth is low.”
This situation cannot continue without radical changes else we will become buyers of foreign fruits and vegetables paying prices set by foreign providers and dependent on them for availability and quality.
A number of factors are causing a reset in the ag industry in addition to the declining availability of farm workers: the challenges and complexities of employing and retaining farm labor; the rising cost of farm workers; changing farmlands; climate change; the growth of indoor farming; and the broader automation of the agriculture industry. All are propelling farmers to make changes in how they farm (or change downward the product mix they farm (which is often a path toward failure)). Market challenges for the sector include unclear value propositions, limited awareness of robotic systems among growers, insufficient robotic solutions, the difficulty of matching human-like dexterity with machines, fragmented technology development, and weak support.
Tractica, in their report on agricultural robotics, forecast an optimistic resolution to this temporary conundrum in the very near term and that that shipments of agricultural robots will increase significantly in the years ahead, rising from 32,000 units in 2016 to 594,000 units annually by 2024, by which time the market is expected to reach $74 billion in annual revenue. Certainly the time is ripe and, like Mason said, “Both arguments can’t be true.”
If you would like to receive the Weekly Roundup in your inbox, please subscribe at the bottom of the page.
At the Center for the Study of the Drone
In an interview with OpenGov, Center for the Study of the Drone co-director Arthur Holland Michel discusses the main trendlines in the ongoing evolution of drone technology.
News
A U.S. drone strike in Somalia reportedly killed a member of al-Shabab. In a statement, the U.S. Africa Command said that the strike took place in the Lower Shabelle region, an al-Shabab stronghold. (Associated Press)
The Trump administration is reviewing a drone exports policy established by the Obama administration. According to an official who spoke to DefenseNews, the review is part of a broader effort to find “smarter new approaches to U.S. defense trade policy.” The Obama administration placed controls on drone technology exports to U.S. allies in 2015.
A small tethered drone will help the U.S. Secret Service provide perimeter security during President Trump’s visit to the Trump National Golf Club in Bedminster, New Jersey this month. In an announcement, the agency said that the test is part of an initiative to explore new technologies for security operations. (Reuters)
Citing cyber vulnerabilities, the U.S. Army has instructed its units to discontinue the use of all drones made by DJI, the popular Chinese manufacturer. The decision appears to have been based on a classified study and a Navy memo on security issues in DJI products. (Reuters)
At an event hosted by the Center for Strategic and International Studies and the U.S. Naval Institute, Adm. Paul Zukunft discussed the Coast Guard’s history with drone acquisition and operations. (USNI News)
The China Aerospace Science and Technology Corporation has made a number of upgrades to its CH-4 Rainbow surveillance and strike drone. (IHS Jane’s International Defence Review)
A team of security researchers has demonstrated that sonic blasts can be used to hack a number of electronic devices, including drones. (Fox News)
Researchers at the Swiss Federal Institute of Technology and Zurich University of the Arts have developed a hexacopter with independently rotating propellers that is capable of flying in far more acrobatic ways than traditional multirotor drones. (Yanko Design)
The University of Michigan has announced that it is developing an outdoor flight testing facility for drones. (Unmanned Systems Technology)
A team at the China Aerospace Science and Technology Corporation is developing an app-based management system for large military drones. (IHS Jane’s International Defence Review)
Following a 10-month definition process, the Organisation for Joint Armament Cooperation has decided that its European Medium-Altitude Long-Endurance drone will have a twin turboprop design. (IHS Jane’s International Defence Review)
Ukraine’s SpetsTechnoExport has conducted a weapons test of its Fantom-2 unmanned ground vehicle. (IHS Jane’s International Defence Review)
The Federal Aviation Administration is investigating an incident in which a drone was spotted near a runway at Newark International Airport. (USA Today)
China’s People’s Liberation Army confirmed that it is now operating the CH-901, a loitering munition drone. (Popular Science) For more on loitering munitions, click here.
The Duluth Fire Department in Minnesota is testing EMILY, an unmanned surface vehicle designed for saving stranded swimmers, with an eye to possibly acquiring the system to use on Lake Superior. (Grand Forks Herald)
Sen. Sheldon Whitehouse (D-RI) and Rep. Jim Langevin (D-RI) introduced a bill that makes it illegal to fly drones near airports without permission. (The Hill)
The Fort Wayne Police Department in Indiana has acquired two drones for a range of operations, including emergency response and environmental surveys. (News Sentinel)
Kentucky governor Matt Bevin has accused a local news channel of invading his privacy after it flew a drone over his private property. (CNET)
The U.S. Army and the Army of the Republic of Macedonia are constructing a 300m-long runway for drones in the Krivolak Training Area. (IHS Jane’s Defense Weekly)
The North Dakota Air National Guard took delivery of its first new MQ-9 Reaper drone. (Associated Press)
Snap is reportedly negotiating to buy Zero Zero Robotics, the China-based manufacturer of the Hover Camera selfie drone, for between $150 and $200 million. (TechCrunch)
The U.S. Navy awarded Northrop Grumman Systems a $19.9 million contract to identify solutions to “near-term emergent obsolescence issues” for the MQ-4C Triton. (DoD)
The U.S. Navy awarded Northrop Grumman Systems a $2.94 million contract for MQ-8 Fire Scout logistics and training sustainment. (FBO)
A quick, hassle-free way to stay on top of robotics news, our robotics digest is released on the first Monday of every month. Sign up to get it in your inbox.
Robots in action
From wacky talking Einsteins to clumsy security ‘bots, from speedy drones to the underwater operations at Fukushima, it’s been another busy month. So let’s kick off our July review with a look at robots in action!
Up in the air
You’d be forgiven for missing this first one. Zipping by at a record speed of 179.3 mp/h (288.6 km/h), The Drone Racing League’s new RacerX drone lays claim to the “fastest ground speed by a battery-powered remote-controlled quadcopter.” The team behind the drone buzzed into the Guinness Book of World Records while performing a number of drag races along a 100-meter course in upstate New York. The aircraft, which weighs in at 0.8 kg, recorded an average top speed of 163.5 mp/h (263.1 km/h).
Flying a lot higher and—technically—a lot faster, this next innovation takes the form of a cute, spheroid camera drone that predictably drew comparisons to everyone’s favorite interstellar droid-ball, BB-8. Released aboard the International Space Station (ISS), the Japanese made Int-Ball will not only save crewmembers time by snapping pictures of experiments, but could also improve robotic-human cooperation in future space expeditions according to a statement from the Japanese Aerospace Exploration Agency (JAXA).
But if you really don’t want flying robots in your airspace (and you don’t have a trained sparrowhawk on-hand), then check out this testing of anti-drone weapons designed to blow prying eyes out of the sky. Could we be on the cusp of an anti-drone arms race?
In the lab
July saw more researchers challenging the traditional anthropomorphic vision of robot motion. Take a look at these new innovations and you’ll see that the concept of a rickety, two legged android helper is beginning to look rather dated.
For example: a new type of vine-inspired robot created by mechanical engineers at Stanford University could soon be squirming through the rubble of collapsed buildings. Like natural organisms that cover distance by growing—such as fungi and nerve cells—the researchers have made a proof of concept of their soft, growing robot and have run it through some challenging tests. From one end of the cylinder, a tendril can extend into a mass of stones or dirt, like a fast-climbing vine. And a camera at the tip of the tendril can potentially offer rescuers a view of otherwise unreachable places.
Meanwhile, a team of researchers at the Wyss Institute for Biologically Inspired Engineering and the John A. Paulson School of Engineering and Applied Sciences (SEAS) at Harvard University has created battery-free folding robots that are capable of complex, repeatable movements powered and controlled through a wireless magnetic field. There are many applications for this kind of minimalist robotic technology. For example: rather than having an uncomfortable endoscope put down their throat to assist a doctor with surgery, a patient could just swallow a micro-robot that could move around and perform simple tasks like holding tissue or filming—all powered by a coil outside the patient’s body. Furthermore, using large source coils could enable wireless, battery-free communication between multiple “smart” objects in an entire home.
And check this out: a pair of new computational methods developed by a team of researchers from Massachusetts Institute of Technology (MIT), University of Toronto and Adobe Research has taken steps towards automating the design of the dynamic mechanisms behind jumping movements in robots. Their methods generate simulations that match the real-world behaviors of flexible devices at rates 70-times faster than previously possible and provide critical improvements in the accuracy of simulated collisions and rebounds. These methods are both fast and accurate enough to be used to automate the design process used to create dynamic mechanisms for controlled jumping.
Health and rehabilitation
One of the most inspiring and positive applications of robotics and AI is within the field of medicine and rehabilitation. Just take the following story for example. With the assistance of its human handlers, Toyota’s Human Support Robot wheeled into a paralyzed military veteran’s home on a mission: to support the quadriplegic patient and, in the process, pave the way for truly useful care robots. Check out the video below.
Leveraging similar technology to that applied in their self-driving golf buggies and autonomous electric cars, MIT and Singaporean researchers have developed and deployed a self-driving wheelchair at a hospital. Spearheaded by Daniela Rus and Erna Viterbi (professor of Electrical Engineering and Computer Science and director of MIT’s Computer Science and Artificial Intelligence Laboratory), this autonomous wheelchair is an extension of the self-driving scooter that launched at MIT last year.
Meanwhile, July also brought some fantastic new developments in robotic driven harness devices designed to improve mobility in disabled people. A team led by Sunil Agrawal, professor of mechanical engineering and of rehabilitation and regenerative medicine at Columbia Engineering has published a pilot study in Science Robotics that demonstrates a robotic training method that improves posture and walking in children with crouch gait—caused by cerebral palsy—by enhancing their muscle strength and coordination.
Similarly, a recent paper published in Science Translational Medicine by a team lead by Courtine-Lab addresses the issue of regaining movement in spinal chord injury (SPI) and stroke patients. The research focuses on multi-directional gravity assist harnesses to aid in rehabilitation. Check out the video below.
Mapping and exploration
An underwater drone, nicknamed “Little Sunfish”, has captured previously unseen shots of underwater damage at the crippled Fukushima nuclear plant. The RC marine vehicle, which is about the size of a loaf of bread, has been sent into the primary containment vessel of the Unit 3 reactor in an attempt to locate melted fuel. Tepco spokesman Takahiro Kimoto told the Japan Times that video taken by the robot over three days shows clumps of what is likely to be melted fuel. “This means something of high temperature melted some structural objects and came out. So it is natural to think that melted fuel rods are mixed with them,” he said.
Exemplified by the operation at Fukushima, the need for fast, accurate 3D mapping solutions has quickly become a reality for many industries. As such, Clearpath Robotics decided this month to team up with Mandala Robotics to demonstrate how easily you might implement 3D mapping on a Clearpath robot.
Fun and games
Comedy is subjective. One person’s epic fail is another person’s fit of hilarity. This month, a lot of people were having a good laugh at a suicidal security ‘bot at an office and retail complex in Washington D.C. who drove itself into a fountain. The egg-shaped robot, known as the K5 Autonomous Data Machine, drew both sympathy and jeers after it stumbled down a set of steps and into the water. The photos were widely shared. Fish that one out.
The K5 security robot fell into a fountain in Washington, D.C.
If expensive punchlines are your thing, then you might also be interested in the $300 USD Professor Einstein robot now available on eBay. Hanson Robotics’ expressive, wacky robotic character can chat about science, tell jokes, check on the weather and, naturally, quote Einstein himself. It connects to a companion app with games, videos, and interactive lessons And yeah, it’s constantly sticking its tongue out (of course).
Finally, last month we saw a group of students in England develop a robot to pull the perfect pint. This month, it’s cocktails! The Cocktail Bot 4.0 consists of five robots with one high-level goal: mix more than 20 possible drink combination for you!
Business & politics
Last month saw two robotics-related companies get $50 million each and 17 others raised $248 million for a monthly total of $348 million. Acquisitions also continued to be substantial with SoftBank’s acquisition of Google’s robotic properties Boston Dynamics and Schaft plus two others acquisitions.
Indeed, two reputable research resources in July reported that the robotics industry is growing more rapidly than expected. BCG (Boston Consulting Group) is conservatively projecting that the market will reach $87 billion by 2025; Tractica, incorporating the robotic and AI elements of the emerging self-driving industry, is forecasting the market will reach $237 billion by 2022.
Meanwhile, Singapore Technologies Engineering Ltd (ST Engineering) has acquired Pittsburgh, PA-based robotics firm Aethon Inc through Vision Technologies Land Systems, Inc. (VTLS), and its wholly-owned subsidiary, VT Robotics, Inc, for $36 million. The acquisition will be carried out by way of a merger with VT Robotics, a special newly incorporated entity established for the transaction. The merger will see Aethon as the surviving entity that will operate as a subsidiary of VTLS, and will be part of the the ST Group’s Land Systems sector. Aethon’s leadership team and employees will remain in place and the company will continue to operate out of its Pittsburgh, PA location.
And if you’re looking for funding them check this out: The Robotics Hub, in collaboration with Silicon Valley Robotics, is looking to invest up to $500,000 in robotics, AI and sensor startups! Finalists also receive exposure on Robohub and space in the new Silicon Valley Robotics Cowork Space. Plus you get to pitch your startup to an audience of top VCs, investors and experts. Entries close August 31.
Remember Tertill, the weed whacking robot? The response to Tertill’s crowdfunding campaign has amazed and delighted! Pledges totalling over $250,000 have come from 1000+ backers, and Tertill is shipping to all countries, with over a fifth of Tertill’s supporters coming from outside the United States. 11th of July was the last full day of the campaign. The discounted campaign price is no longer available and delivery in time for next year’s—northern hemisphere—growing season cannot be assured.
Self-driving news
In the race to develop self-driving technology, Chinese Internet giant Baidu unveiled its 50+ partners in an open source development program, revised its timeline for introducing autonomous driving capabilities on open city roads, described the Project Apollo consortium and its goals, and declared Apollo to be the ‘Android of the autonomous driving industry’.
And there could be more trouble ahead in the Asian market as India—another powerhouse in the region—could say no to self driving cars in general by banning their use entirely. ‘In a country where you have unemployment, you can’t have a technology that ends up taking people’s jobs,’ roads minister Nitin Gadkari says.
Conversely, lawmakers in the USA say self-driving carsare the future and federal law needs updating to ensure they’re developed and deployed in the United States. A panel approved a bill to boost testing of self-driving vehicles. The bill prohibits any state from imposing its own laws related to the design and construction of self-driving cars. Federal officials say 94% of auto accidents are caused by human error, so self-driving technology has the potential to save thousands of lives and improve the mobility of many elderly and disabled Americans. The Safely Ensuring Lives Future Deployment and Research In Vehicle Evolution Act, or SELF DRIVE, bill has hit the House of Representatives.
And you can read Brad Templeton’s take on all the news and commentary from July’s AUVSI/TRB Automated Vehicle Symposium 2017. It’s an odd mix of business and research, but also the oldest self-driving car conference.
Drone News
Drone manufacturers and designers have managed to shrink drone technology in recent years, even creating flying prototypes at a near insect scale. But the toughest task has been shrinking down the brains of the operation. Now, engineers at MIT have taken a first step in designing a computer chip that uses a fraction of the power of larger drone computers and is tailored for a drone as small as a bottlecap. They presented a new methodology and design, which they call “Navion,” at the Robotics: Science and Systems conference, held at MIT.
Perhaps shrinking UAVs might also go some way to addressing the problem of sonic irritation? A preliminary NASA study has discovered that people find the noise of drones more annoying than that of ground vehicles, even when the sounds are the same volume. “We didn’t go into this test thinking there would be this significant difference,” says Andrew Christian of NASA’s Langley Research Center, Virginia. “It is almost unfortunate the research has turned up this difference in annoyance levels,” he adds, “as its purpose was merely to prove that Langley’s acoustics research facilities could contribute to NASA’s wider efforts to study drones.”
Meanwhile, an unrelated study by the UK government on the danger of drones colliding with aircraft has drawn criticism from manufacturers. The Department for Transport (DfT) report recommended registration and competency testing, saying helicopters were especially vulnerable to drones. The Drone Manufacturers Alliance Europe (DMAE) has questioned the evidence gathered in the report and says some of the testing is flawed.
In other safety news, DJI has responded to reports of its drones randomly switching off mid-flight and dropping out of the sky. According to at least 14 users, DJI Spark drones have switched off and crashed into various areas ranging from open fields to lakes or forests. But luckily no crowded areas yet. DJI told Fortune in a statement that it is working to address the crash incidents going forward: “DJI is aware of a small number of reports involving Spark drones that have lost power mid-flight. Flight safety and product reliability are top priorities. Our engineers are thoroughly reviewing each customer case and working to address this matter urgently,” the statement read. “We are looking to implement additional safeguards with a firmware update which will be issued soon. When prompted on the DJI GO 4 App, we recommend all customers to connect to the internet and update their aircraft’s firmware to ensure a safe flight when flying their Spark,” the company added.
Learn
An interdisciplinary workshop on self-organization and swarm intelligence in cyber physical systems was held at Lakeside Labs in July. Experts presented their work and discussed open issues in this exciting field. Click here to watch some videos from the workshop.
July also welcomed The Second Edition of the award-winning Springer Handbook of Robotics, edited by Bruno Siciliano and Oussama Khatib. The contents of the first edition have been restructured to achieve four main objectives: the enlargement of foundational topics for robotics, the enlightenment of design of various types of robotic systems, the extension of the treatment on robots moving in the environment, and the enrichment of advanced robotics applications. Most previous chapters have been revised, fifteen new chapters have been introduced on emerging topics, and a new generation of authors have joined the handbook’s team.
Elsewhere, Mike Salem from Udacity’s Robotics Nanodegree is hosting a series of interviews with professional roboticists as part of their free online material. You can watch the interview with Nick Kohut, Co-Founder and CEO of Dash Robotics, below. Stay tuned to Robohub throughout August to see more featured interviews from Udacity.
And finally, check out Peter Feuilherade’s article on rescue robots, and Christoph Salge’s article on how Azimov’s laws might not be sufficient to rescue humanity from robots. I’ll leave it to you to muddle through the implicit dichotomy. Or I’ll just see you next month for all the latest Robot related news and views!
Enjoy.
Upcoming events for August – September 2017
Farm Progress Show: August 29–31, 2017, Decatur, IL
World of Drones Congress (WoDC): August 31–September 2, 2017, Brisbane, Australia
Interdrone: September 6–8, 2017, Las Vegas, NV
FSR 2017: September 12–15, 2017, Zurich, Switzerland
RobotWorld: September 13–16, 2017, Seoul, South Korea
IEEE Africon: September 18–20, 2017, Cape Town, South Africa
ROS Con: September 21–22, 2017, Vancouver, BC
IROS 2017: September 24–28, 2017, Vancouver, BC
RoboBusiness: September 27–28, 2017, Santa Clara, CA
Auris Surgical Robotics, the Silicon Valley startup headed by Frederic Moll who previously co-founded Hansen Medical and Intuitive Surgical, raised $280 million in a Series D round led by Coatue Management and including earlier investors Mithril Capital Management, Lux Capital, Highland Capital and 24 others.
An Auris spokesman said that the company has raised a total of $530 million and is developing targeted, minimally invasive therapies that treat only the diseased cells in order to prevent the progression of a patient’s illness. Lung cancer is the first disease they are targeting.
There are 1 billion smokers worldwide creating an epidemic of 6 million deaths per year. More patients die every year from lung cancer than from prostate, breast and colon cancer combined.
The reason lung cancer is so deadly is that the diagnostic and treatment processes are ineffective. The majority of lung cancer patients are diagnosed in late stage, when the cancer has already spread beyond its primary location. With our technology, physicians will be able to access early stage lung cancer without incisions, allowing accurate diagnosis and targeted treatment.
In this episode, MeiXing Dong interviews Matthias Vanoni, co-founder and CEO of Biowatch. Vanoni speaks about Biowatch, a wrist-veins biometric reader that functions as a security solution for mobile payments and smart devices. They discuss the technical challenges of building a miniaturized wrist-vein reader and how this device changes the usual user authentication process.
Matthias Vanoni
Matthias Vanoni is the co-founder and CEO of Biowatch. Previously, he studied vein biometrics as a PhD student at EPFL.
SEM images of the hybrid soft pop-up actuators. The image has been colored in post processing to differentiate between the soft (in yellow) and the rigid structure (in blue). Credit: Wyss Institute at Harvard University
By Leah Burrows, SEAS Communications
Flexible endoscopes can snake through narrow passages to treat difficult to reach areas of the body. However, once they arrive at their target, these devices rely on rigid surgical tools to manipulate or remove tissue. These tools offer surgeons reduced dexterity and sensing, limiting the current therapeutic capabilities of the endoscope.
Now, researchers from the Wyss Institute for Biologically Inspired Engineering at Harvard University and the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed a hybrid rigid-soft robotic arm for endoscopes with integrated sensing, flexibility, and multiple degrees of freedom. This arm — built using a manufacturing paradigm based on pop-up fabrication and soft lithography — lies flat on an endoscope until it arrives at the desired spot, then pops up to assist in surgical procedures.
Soft robots are so promising for surgical applications because they can match the stiffness of the body, meaning they won’t accidentally puncture or tear tissue. However, at small scales, soft materials cannot generate enough force to perform surgical tasks.
“At the millimeter scale, a soft device becomes so soft that it can’t damage tissue but it also can’t manipulate the tissue in any meaningful way,” said Tommaso Ranzani, Ph.D., a Postdoctoral Fellow at the Wyss Institute and SEAS and coauthor of the paper. “That limits the application of soft microsystems for performing therapy. The question is, how can we develop soft robots that are still able to generate the necessary forces without compromising safety.”
Inspired by biology, the team developed a hybrid model that used a rigid skeleton surrounded by soft materials. The manufacturing method drew on previous work in origami-inspired, pop-up fabrication developed by Robert Wood, Ph.D., who coauthored the paper and is a Core Faculty Member of the Wyss Institute and the Charles River Professor of Engineering and Applied Sciences at SEAS.
Soft pop-up arm performing tissue counter-traction during an ex-vivo test on a porcine stomach. Credit: Harvard University
Previous pop-up manufacturing techniques — such as those used with the RoboBees — rely on actuation methods that require high voltages or temperatures to operate, something that wouldn’t be safe in a surgical tool directly manipulating biological tissues and organs.
So, the team integrated soft actuators into the pop-up system.
“We found that by integrating soft fluidic microactuators into the rigid pop-up structures, we could create soft pop-up mechanisms that increased the performance of the actuators in terms of the force output and the predictability and controllability of the motion,” said Sheila Russo, Ph.D., Postdoctoral Fellow at the Wyss Institute and SEAS and lead author of the paper. “The idea behind this technology is basically to obtain the best of both worlds by combining soft robotic technologies with origami-inspired rigid structures. Using this fabrication method, we were able to design a device that can lie flat when the endoscope is navigating to the surgical area, and when the surgeon reaches the area they want to operate on, they can deploy a soft system that can safely and effectively interact with tissue.”
The soft actuators are powered by water. They are connected to the rigid components with an irreversible chemical bond, without the need of any adhesive. The team demonstrated the integration of simple capacitive sensing that can be used to measure forces applied to the tissue and to give the surgeon a sense of where the arm is and how it’s moving. The fabrication method allows for bulk manufacturing, which is important for medical devices, and allows for increased levels of complexity for more sensing or actuation. Furthermore, all materials used are biocompatible.
The arm is also equipped with a suction cup — inspired by octopus tentacles — to safely interact with tissue. The team tested the device ex vivo, simulating a complicated endoscopic procedure on pig tissue. The arm successfully manipulated the tissue safely.
“The ability to seamlessly integrate gentle yet effective actuation into millimeter-scale deployable mechanisms fits naturally with a host of surgical procedures,” said Wood. “We are focused on some of the more challenging endoscopic techniques where tool dexterity and sensor feedback are at a premium and can potentially make the difference between success and failure.”
Multi-articulated soft pop-up robotic arm. Concept of the system (left): An endoscope navigating in the GI tract and detail of the arm mounted at the tip of the endoscope. Soft pop-up arm (right) performing tissue counter-traction during an ex-vivo test on a porcine stomach. Credit: Harvard University
The researchers demonstrated that the device could be scaled down to 1 millimeter, which would allow it to be used in even tighter endoscopic procedures, such as in lungs or the brain.
Next, the researchers hope to test the device in vivo.
“Our technology paves the way to design and develop smaller, smarter, softer robots for biomedical applications,” said Russo.
The paper was coauthored by Conor Walsh, Ph.D., a Core Faculty Member of the Wyss Institute and the John L. Loeb Associate Professor of Engineering and Applied Sciences at SEAS.
The research was supported by the DARPA “Atoms to Product” program and the Wyss Institute for Biologically Inspired Engineering.
If you are involved in the UK Robotics and Autonomous Systems (RAS) sector, we’d love to hear from you. Please fill in this survey. 
Perhaps shrinking UAVs might also go some way to addressing the problem of sonic irritation? A preliminary NASA study has discovered that people find the 
