Category robots

By Lindsay Brownell

DNA has often been compared to an instruction book that contains the information needed for a living organism to function, its genes made up of distinct sequences of the nucleotides A, G, C, and T echoing the way that words are composed of different arrangements of the letters of the alphabet. DNA, however, has several advantages over books as an information-carrying medium, one of which is especially profound: based on its nucleotide sequence alone, single-stranded DNA can self-assemble, or bind to complementary nucleotides to form a complete double-stranded helix, without human intervention. That would be like printing the instructions for making a book onto loose pieces of paper, putting them into a box with glue and cardboard, and watching them spontaneously come together to create a book with all the pages in the right order.

But just as paper can also be used to make origami animals, cups, and even the walls of houses, DNA is not limited to its traditional purpose as a passive repository of genetic blueprints from which proteins are made – it can be formed into different shapes that serve different functions, simply by controlling the order of As, Gs, Cs, and Ts along its length. A group of scientists at the Wyss Institute for Biologically Inspired Engineering at Harvard University is investigating this exciting property of DNA molecules, asking, “What types of systems and structures can we build with them?”

They’ve decided to build robots.

At first glance, there might not seem to be much similarity between a strand of DNA and, say, a Roomba or Rosie the Robot from The Jetsons. “Looking at DNA versus a modern-day robot is like comparing a piece of string to a tractor trailer,” says Wyss Faculty member Wesley Wong, Ph.D., Assistant Professor of Biological Chemistry and Molecular Pharmacology (BCMP) at Harvard Medical School (HMS) and Investigator at Boston Children’s Hospital. Despite the vast difference in their physical form, however, robots and DNA share the ability to be programmed to complete a specific function – robots with binary computer code, DNA molecules with their nucleotide sequences.

Recognizing that commonality, the Wyss Institute created the cross-disciplinary Molecular Robotics Initiative in 2016, which brings together researchers with experience in the disparate disciplines of robotics, molecular biology, and nanotechnology to collaborate and help inform each other’s work to solve the fields’ similar challenges. Wong is a founding member of the Initiative, along with Wyss Faculty members William Shih, Ph.D., Professor of BCMP at HMS and Dana-Farber Cancer Institute; Peng Yin, Ph.D., Professor of Systems Biology at HMS; and Radhika Nagpal, Ph.D., Fred Kavli Professor of Computer Science at Harvard’s John A. Paulson School of Engineering and Applied Sciences (SEAS); as well as other Wyss scientists and support staff.

“We’re not used to thinking about molecules inside cells doing the same things that computers do. But they’re taking input from their environment and performing actions in response – a gene is either turned on or off, a protein channel is either open or closed, etc. – in ways that can resemble what computer-controlled systems do,” says Shih. “Molecules can do a lot of things on their own that robots usually have trouble with (move autonomously, self-assemble, react to the environment, etc.), and they do it all without needing motors or an external power supply,” adds Wyss Founding Director Don Ingber, M.D., Ph.D., who is also the Judah Folkman Professor of Vascular Biology at HMS and the Vascular Biology Program at Boston Children’s Hospital, as well as a Professor of Bioengineering at SEAS. “Programmable biological molecules like DNA have almost limitless potential for creating transformative nanoscale devices and systems.”

Molecular Robotics capitalizes on the recent explosion of technologies that read, edit, and write DNA (like next-generation sequencing and CRISPR) to investigate the physical properties of DNA and its single-stranded cousin RNA. “We essentially treat DNA not only as a genetic material, but as an incredible building block for creating molecular sensors, structures, computers, and actuators that can interact with biology or operate completely separately,” says Tom Schaus, M.D., Ph.D., a Staff Scientist at the Wyss Institute and Molecular Robotics team member.

Many of the early projects taking advantage of DNA-based self-assembly were static structures.  These include DNA “clamshell” containers that can be programmed to snap open and release their contents in response to specific triggers, and DNA “bricks” whose nucleotide sequences allow their spontaneous assembly into three-dimensional shapes, like tiny Lego bricks that put themselves together to create sculptures automatically. Many of these structures are three-dimensional, and some incorporate as many as 10,000 unique DNA strands in a single complete structure.

The reliable specificity of DNA and RNA (where A always binds with T or U, C always with G) allows for not only the construction of static structures, but also the programming of dynamic systems that sense and respond to environmental cues, as seen in traditional robotics. For example, Molecular Robotics scientists have created a novel, highly controllable mechanism that automatically builds new DNA sequences from a mixture of short fragments in vitro. It utilizes a set of hairpin-shaped, covalently-modified DNA strands with a single-stranded “overhang” sequence dangling off one end of the hairpin. The overhang sequence can bind to a complementary free-floating fragment of DNA (a “primer”) and act as a template for its extension into a double-stranded DNA sequence. The hairpin ejects the new double strand and can then be re-used in subsequent reactions to produce multiple copies of the new strand.

Such extension reactions can be programmed to occur only in the presence of signal molecules, such as specific RNA sequences, and can be linked together to create longer DNA product strands through “Primer Exchange Reactions” (PER). PER can in turn be programmed to enzymatically cut and destroy particular RNA sequences, record the order in which certain biochemical events happen, or generate components for DNA structure assembly.

PER reactions can also be combined into a mechanism called “Autocycling Proximity Recording” (APR), which records the geometry of nano-scale structures in the language of DNA. In this instance, unique DNA hairpins are attached to different target molecules in close proximity and, if any two targets are close enough together, produce new pieces of DNA containing the molecular identities (“names”) of those two targets, allowing the shape of the underlying structure to be determined by sequencing that novel DNA.

Another tool, called “toehold switches,” can be used to exert complex and precise control over the machinery inside living cells. Here, a different, RNA-based hairpin is designed to “open” when it binds to a specific RNA molecule, exposing a gene sequence in its interior that can be translated into a protein that then performs some function within the cell. These synthetic circuits can even be built with logic-based sequences that mimic the “AND,” “OR,” and “NOT” system upon which computer languages are based, which prevents the hairpin from opening and its gene from being translated except under very specific conditions.

Such an approach could induce cells that are deficient in a given protein to produce more of it, or serve as a synthetic immune system that, when it detects a given problem in the cell, produces a toxin that kills it to prevent it from spreading an infection or becoming cancerous. [toeholds] “Because we have a thorough understanding of DNA and RNA’s properties and how their bases pair together, we can use that simple machinery to design complex circuits that allow us to precisely interact with the molecular world,” says Yin. “It’s an ability that has been dreamed about for a long time, and now, we’re actually making it a reality.”

The potential applications of that ability are seemingly endless. In addition to the previously mentioned tools, Molecular Robotics researchers have created loops of DNA attached to microscopic beads to create “calipers” that can both measure the size, structure, and stiffness of other molecules, and form the basis of inexpensive protein recognition tests. Another advance is folding single-stranded DNA into molecular origami to create molecular structures, rather than traditional double-stranded DNA. Some academic projects are already moving into the commercial sector. These include a low-cost alternative to super-resolution microscopy that can image up to 100 different molecular targets in a single sample (DNApaint), as well as a multiplexed imaging technique that integrates fluorescent probes into self-folding DNA structures and enables simultaneous visualization of ultra-rare DNA and/or RNA molecules.

We’re trying to push the limits of these really dumb little molecules to get them to behave in sophisticated, collective ways – it’s a new frontier for DNA nanotechnology.

One of the major benefits of engineering molecular machines is that they’re tiny, so it’s relatively easy to create a large amount of them to complete any one task (for example, circulating through the body to detect any rogue cancer DNA). Getting simple, individual molecules to interact with each other to achieve a more complex, collective task (like relaying the information that cancer has been found), however, is a significant challenge, and one that the roboticists in Molecular Robotics are tackling at the macroscopic scale with inch-long “Kilobots.”

Taking cues from colonies of insects like ants and bees, Wyss researchers are developing swarms of robots that are themselves limited in function but can form complex shapes and complete tasks by communicating with each other via reflected infrared light. The insights gained from studies with the Kilobots are likely to be similar to those needed to solve similar problems when trying to coordinate molecular robots made of DNA.

“In swarm robotics, you have multiple robots that explore their environment on their own, talk to each other about what they find, and then come to a collective conclusion. We’re trying to replicate that with DNA but it’s challenging because, as simple as Kilobots are, they’re brilliant compared to DNA in terms of computational power,” says Justin Werfel, Ph.D., a Senior Research Scientist at the Wyss Institute and director of the Designing Emergence Laboratory at Harvard. “We’re trying to push the limits of these really dumb little molecules to get them to behave in sophisticated, collective ways – it’s a new frontier for DNA nanotechnology.”

Given the magnitude of the challenge and the short time the Molecular Robotics Initiative has existed, it is already making significant progress, with more than two dozen papers published and two companies (Ultivue and NuProbe) founded around its insights and discoveries. It may take years of creative thinking, risk taking, and swapping ideas across the members’ different expertise areas before a molecule of DNA is able to achieve the same task on the nanoscale that a robot can do on the human scale, but the team is determined to see it happen.

“Our vision with Molecular Robotics is to solve hard problems humanity currently faces using smaller, simpler tools, like a single loop of DNA or a single Kilobot that can act cooperatively en masse, instead of bigger, more complex ones that are harder to develop and become useless should any one part fail,” says Wong. “It’s an idea that definitely goes against the current status quo, and we’re lucky enough to be pursuing it here at the Wyss Institute, which brings together people with common goals and interests to create new things that wouldn’t exist otherwise.”

Click on the links below to explore research from the Molecular Robotics Initiative.

1. Researchers at Harvard’s Wyss Institute Develop DNA Nanorobot to Trigger Targeted Therapeutic Responses
2. A 100-fold leap to GigaDalton DNA nanotech
3. Autonomously growing synthetic DNA strands
4. High-fidelity recording of molecular geometry with DNA “nanoscopy”
5. Programming cells with computer-like logic
6. Democratizing high-throughput single molecule force analysis
7. Single-stranded DNA and RNA origami go live
8. Capturing ultrasharp images of multiple cell components at once
9. A self-organizing thousand-robot swarm
10. Discrete Molecular Imaging

In 2016, the European Union co-funded 17 new robotics projects from the Horizon 2020 Framework Programme for research and innovation. 16 of these resulted from the robotics work programme, and 1 project resulted from the Societal Challenges part of Horizon 2020. The robotics work programme implements the robotics strategy developed by SPARC, the Public-Private Partnership for Robotics in Europe (see the Strategic Research Agenda). 

EuRobotics regularly publishes video interviews with projects, so that you can find out more about their activities. This week features REELER: Responsible Ethical Learning with Robotics.

Objectives

The project aims at aligning roboticists’ visions of a future with robots with empirically-based knowledge of human needs and societal concerns through a new proximity-based human-machine ethics that take into account how individuals and community connect with robot technologies.
The main outcome of REELER is a research-based roadmap presenting:

• ethical guidelines for Human Proximity Levels,
• prescriptions for how to include the voice of new types of users and affected stakeholders through Mini-Publics,
• assumptions in robotics through socio-drama
• agent-based simulations of the REELER research for policymaking.

At the core of these guidelines is the concept of collaborative learning, which permeates all aspects of REELER and will guide future SSH-ICT research.

Expected Impact

Integrating the recommendations of the REELER Roadmap for responsible and ethical learning in robotics in future robot design processes will enable the European robotics community to addresses human needs and societal concerns. Moreover, the project will give powerful instruments able to foster networking and exploit potentialities of future robotics projects.

Partners

AARHUS UNIVERSITY, DPU
AB.ACUS SRL
DE MONTFORT UNIVERSITY, CCSR
HOHENHEIM UNIVERSITY

Coordinator:

Stine Trentemøller

Project website:

http://www.reeler.eu/

Watch all EU-projects videos

If you enjoyed reading this article, you may also want to read:

• New Horizon 2020 robotics projects, 2016: ILIAD
• New Horizon 2020 robotics projects, 2016: HEPHAESTUS
• New Horizon 2020 robotics projects, 2016: Co4Robots
• New Horizon 2020 robotics projects, 2016: An.Dy
• New Horizon 2020 robotics projects, 2016: BADGER
• Two Horizon 2020 projects researching EU Digital Industrial Platform for Robotics
• EU’s Horizon 2020 has funded $179 million in robotics PPPs

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

• Vacuum generation that’s 100% electrical; • Integrated intelligence for energy and process control; • Extensive communication options through IO-Link interface; Schmalz already offers a large range of solutions that can optimize handling process from single components such as vacuum generators to complete gripping systems. Particularly when used in autonomous warehouse, conventional vacuum generation with compressed air reaches its limits. Compressed air often is unavailable in warehouses. Schmalz therefore is introducing a new technology development: a gripper with vacuum generation that does not use compressed air. The vacuum is generated 100% electrically. This makes the gripper both energy efficient and mobile. At the same time, warehouses need systems with integrated intelligence to deliver information and learn. This enables the use of mobile and self-sufficient robots, which pick production order at various locations in the warehouse. Furthermore, Schmalz provides various modular connection options from its wide range of end effectors in order to handle different products reliably and safely.

Well, this year’s videos are getting creative!
Have a holiday robot video of your own that you’d like to share? Send your submissions to editors@robohub.org.

“Cozmo stars in Christmas Wrap” by Life with Cozmo

“Don’t be late for Christmas!” by FZI Living Lab

“LTU Robotics Team Christmas Video 2017” by the Control Engineering Group of Luleå University of Technology, Sweden.

Warning: This video is insane . . .

“Misletoe: A robot love story” by the Robot Drive-In Movies.

For more holiday videos, check last week’s post. Email us your holiday robot videos at editors@robohub.org!

When you think of a treadmill, what comes to your mind?

Perhaps the images of a person burning calories, or maybe the treadmill fail videos online. But almost certainly not a miniature treadmill for insects, and particularly not as a tool for understanding fundamental biology and its applications to technology.

Researchers have been studying insects walking on a treadmill.

But why!?

Traditional methods for investigating an insect’s biology include observing them in their natural habitat or in the lab, and manipulating the animal or its surroundings. While this is sufficient for some research questions, it has its limitations. It is challenging to study certain behaviours like flight and navigation as it is difficult to manipulate insects in motion.  Scientists have been using the simple concept of a treadmill to address this. (1, 2). When insects fly or navigate, they typically use visual cues from their surroundings. So a screen with images/videos projected on it can be used to study how the insects behave with such cues. Alternatively, a virtuality reality setup added to the treadmill can help in manipulating the cues in real-time.

How do you make a treadmill for insects?

A miniature insect treadmill is a light-weight hollow Styrofoam ball suspended on an airflow. An ant, bee or a fly is tethered using a dental floss or a metal wire and placed on the top of the ball. Motion of the ball as the insect walks on it is recorded by two optical sensors similar to the one you find in a desktop mouse. This setup can be used as is outdoors, or with stationary images projected on a screen, or with a virtual reality screen instead. For virtual reality, as the ant walks on the ball, the sensors record the movement of the ball to extract the fictive movement of the insect in two dimensional space. This information is then transmitted to a computer which creates corresponding movement in the images/video on the virtual reality screen. For ants, this is almost as if they are walking and experiencing the change in the surroundings in real-time.

What can you learn from this about the insects?

Scientists have been able to learn about how visual cues influence flight and navigation in bees and ants by projecting them on a screen while tethered insects walk on a treadmill. Neural responses in different parts of their brain can also be recorded while the tethered insects are performing different behaviour. Such experiments can inform us about how they learn and remember different visual cues.

Do they show naturalistic behaviour on the treadmill?

At least in some ants like Cataglyphis fortis, the behaviours on the treadmill are similar to natural behaviour. However, the treadmill setup is still not free of shortcomings.

For example, restricting the movement of a flying insect like bees or flies tethered over the treadmill can affect their sensorymotor experience. Insect brains are evolved such that certain sensory feedback is required to elicit motor actions (behaviour). Flying on the treadmill might not feel the same for the insects. But recent technology has made it possible to use the virtual reality in real time for freely moving insects (and also mice and fish). High speed cameras can now record the 3D position of a freely flying insect, and transmit that to a computer which updates the visuals on the screen accordingly. The whole setup looks as if the insects are in a computer game.

The experimenters control the fly’s position (red circles) and its flight direction by providing strong visual motion stimuli. Left: live camera footage, Right: plot of flight positions. Credit: https://strawlab.org/freemovr

On the other hand, this setup cannot be used to study depth perception or 3D vision (stereopsis) in insects like praying mantises as the projections on the screen are two dimensional. Luckily, researchers at Newcastle University (link) have found another ingenious way — 3D movie glasses! They cut out mantis-eye-sized glasses out of an ordinary human 3D glasses and attach them to the mantis eyes using beeswax. The visuals on the screen can now be similar to any 3D movie. This technique can potentially help to build simpler 3D vision systems for robots.

Another challenge with the treadmill setup include not being able to re-create different kinds of sensory information that they experience in nature. This may also be achieved in future.

What are the applications of this fundamental research?

The treadmill with virtual reality setup is an example of how technology can advance science, and how fundamental biological research in turn can inspire technology.  Since insects have simpler nervous and sensory systems than humans, they are easier to mimic. While the latest technology has helped uncover biological secrets of insects, that in turn can be an inspiration for bio-robots.

Take for example, the Moth robots. Moths use chemicals (pheromones) to communicate. So moths on a treadmill can navigate towards the smell. The motion of the treadmill as the tethered moth walks towards the smell can drive a small robot. Using the insect pilot in the cockpit of a robot, one could locate necessary odour signals in areas humans cannot reach.

Ants navigating on the treadmill can also inspire visually navigating robots and driverless cars (link). This can have applications ranging from disaster management to extra-terrestrial navigation. Perhaps in the future, ants-sized robots could visually navigate and search for the victims stuck under rubble after a devastating earthquake.

So the simple concept of a treadmill and the latest virtual reality can help biological research and inspire technology in different ways. What might be next, an insect gym?

By Paul Salmon, University of the Sunshine Coast

Millions of fans all over the world eagerly anticipated this week’s release of Star Wars: The Last Jedi, the eighth in the series. At last we will get some answers to questions that have been vexing us since 2015’s The Force Awakens.

Throughout the franchise, the core characters have been accompanied by a number of much-loved robots, including C-3PO, R2-D2 and more recently, BB-8 and K2-SO. While often fulfilling the role of wise-cracking sidekicks, these and other robots also play an integral role in events.

Interestingly, they can also tell us useful things about automation, such as whether it poses dangers to us and whether robots will ever replace human workers entirely. In these films, we see the good, bad and ugly of robots – and can thus glean clues about what our technological future might look like.

The fear of replacement

One major fear is that robots and automation will replace us, despite work design principles that tell us technology should be used as a tool to assist, rather than replace, humans. In the world of Star Wars, robots (or droids as they are known) mostly assist organic lifeforms, rather than completely replace them.

So for instance, C-3PO is a protocol droid who was designed to assist in translation, customs and etiquette. R2-D2 and the franchise’s new darling, BB-8, are both “astromech droids” designed to assist in starship maintenance.

In the most recent movie, Rogue One, an offshoot of the main franchise, we were introduced to K2-SO, a wisecracking advanced autonomous military robot who was caught and reprogrammed to switch allegiance to the rebels. K2-SO mainly acts as a co-pilot, for example when flying a U-Wing with the pilot Cassian Andor to the planet of Eadu.

In most cases then, the Star Wars droids provide assistance – co-piloting ships, helping to fix things, and even serving drinks. In the world of these films, organic lifeforms are still relied upon for most skilled work.

When organic lifeforms are completely replaced, it is generally when the work is highly dangerous. For instance, during the duel between Annakin and Obi Wan on the planet Mustafar in Revenge of the Sith, DLC-13 mining droids can be seen going about their work in the planet’s hostile lava rivers.

Further, droid armies act as the frontline in various battles throughout the films. Perhaps, in the future, we will be OK with losing our jobs if the work in question poses a significant risk to our health.

However, there are some exceptions to this trend in the Star Wars universe. In the realm of healthcare, for instance, droids have fully replaced organic lifeforms. In The Empire Strikes Back a medical droid treats Luke Skywalker after his encounter with a Wampa, a yeti-like snow beast on the planet Hoth. The droid also replaces his hand following his battle with Darth Vadar on the planet Bespin.

Likewise, in Revenge of the Sith, a midwife droid is seen delivering the siblings Luke and Leia on Polis Massa.

Perhaps this is one area in which Star Wars has it wrong: here on earth, full automation is a long way off in healthcare. Assistance from robots in healthcare is the more realistic prospect and is in fact, already here. Indeed, robots have been assisting surgeons in operating theatres for some time now.

Automated vehicles

Driverless vehicles are currently flavour of the month – but will we actually use them? In Star Wars, despite the capacity for spacecraft and star ships to be fully automated, organic lifeforms still take the controls. The spaceship Millenium Falcon, for example, is mostly flown by the smuggler Han Solo and his companion Chewbacca.

Most of the Star Wars starship fleet (A-Wings, X-Wings, Y-Wings, Tie Fighters, Star Destroyers, Starfighters and more) ostensibly possess the capacity for fully automated flight, however, they are mostly flown by organic lifeforms. In The Phantom Menace the locals on Tatooine have even taken to building and manually racing their own “pod racers”.

It seems likely that here on earth, humans too will continue to prefer to drive, fly, sail, and ride. Despite the ability to fully automate, most people will still want to be able to take full control.

Flawless, error proof robots?

Utopian visions often depict a future where sophisticated robots will perform highly skilled tasks, all but eradicating the costly errors that humans make. This is unlikely to be true.

A final message from the Star Wars universe is that the droids and advanced technologies are often far from perfect. In our own future, costly human errors may simply be replaced by robot designer errors.

The B1 Battle Droids seen in the first and second Star Wars films lack intelligence and frequently malfunction. C-3PO is notoriously error prone and his probability-based estimates are often wide of the mark.

In the fourth film, A New Hope, R5-D4 (another astromech droid) malfunctions and explodes just as the farmer Owen Lars is about to buy it. Other droids are slow and clunky, such as the GNK Power droid and HURID-327, the groundskeeper at the castle of Maz Kanata in The Force Awakens.

The much feared scenario, whereby robots become so intelligent that they eventually take over, is hard to imagine with this lot.

Perhaps the message from the Star Wars films is that we need to lower our expectations of robot capabilities, in the short term at least. Cars will still crash, mistakes will still be made, regardless of whether humans or robots are doing the work.

Paul Salmon, Professor of Human Factors, University of the Sunshine Coast

This article was originally published on The Conversation. Read the original article.

In this episode of Robots in Depth, Per Sjöborg speaks with Ian Bernstein, the founder of several robotics companies including Sphero. He shares his experience from completing 5 successful rounds of financing, raising 17 million dollars in the 5th one.

Ian also talks about building a world-wide distribution network and the complexity of combining software and hardware development. We then discuss what is happening in robotics and where future successes may come from, including the importance of Kickstarter and Indiegogo.

If you view this episode, you will also learn which day of the week people don’t play with their Sphero :-).

By Changliu Liu, Masayoshi Tomizuka

Democratization of Robots in Factories

In modern factories, human workers and robots are two major workforces. For safety concerns, the two are normally separated with robots confined in metal cages, which limits the productivity as well as the flexibility of production lines. In recent years, attention has been directed to remove the cages so that human workers and robots may collaborate to create a human-robot co-existing factory.

Manufacturers are interested in combining human’s flexibility and robot’s productivity in flexible production lines. The potential benefits of industrial co-robots are huge and extensive, e.g. they may be placed in human-robot teams in flexible production lines, where robot arms and human workers cooperate in handling workpieces, and automated guided vehicles (AGV) co-inhabit with human workers to facilitate factory logistics. In the factories of the future, more and more human-robot interactions are anticipated to take place. Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. The fundamental problem is how to ensure that co-robots operate efficiently and safely in dynamic uncertain environments. In this post, we introduce the robot safe interaction system developed in the Mechanical System Control (MSC) lab.

Fig. 1. The factory of the future with human-robot collaborations.

Existing Solutions

Robot manufacturers including Kuka, Fanuc, Nachi, Yaskawa, Adept and ABB are providing or working on their solutions to the problem. Several safe cooperative robots or co-robots have been released, such as Collaborative Robots CR family from FANUC (Japan), UR5 from Universal Robots (Denmark), Baxter from Rethink Robotics (US), NextAge from Kawada (Japan) and WorkerBot from Pi4_Robotics GmbH (Germany). However, many of these products focus on intrinsic safety, i.e. safety in mechanical design, actuation and low level motion control. Safety during social interactions with humans, which are key to intelligence (including perception, cognition and high level motion planning and control), still needs to be explored.

Technical Challenges

Technically, it is challenging to design the behavior of industrial co-robots. In order to make the industrial co-robots human-friendly, they should be equipped with the abilities to: collect environmental data and interpret such data, adapt to different tasks and different environments, and tailor itself to the human workers’ needs. For example, during human-robot collaborative assembly shown in the figure below, the robot should be able to predict that once the human puts the two workpieces together, he will need the tool to fasten the assemble. Then the robot should be able to get the tool and hand it over to the human, while avoid colliding with the human.

Fig. 2. Human-robot collaborative assembly.

To achieve such behavior, the challenges lie in (1) the complication of human behaviors, and (2) the difficulty in assurance of real time safety without sacrificing efficiency. The stochastic nature of human motions brings huge uncertainty to the system, making it hard to ensure safety and efficiency.

The Robot Safe Interaction System and Real-time Non-convex Optimization

The robot safe interaction system (RSIS) has been developed in the Mechanical System Control lab, which establishes a methodology to design the robot behavior to achieve safety and efficiency in peer-to-peer human-robot interactions.

As robots need to interact with humans, who have long acquired interactive behaviors, it is natural to let robot mimic human behavior. Human’s interactive behavior can result from either deliberate thoughts or conditioned reflex. For example, if there is a rear-end collision in the front, the driver of a following car may instinctively hit the brake. However, after a second thought, that driver may speed up to cut into the other lane to avoid chain rear-end. The first is a short-term reactive behavior for safety, while the second needs calculation on current conditions, e.g. whether there is enough space to achieve a full stop, whether there is enough gap for a lane change, and whether it is safer to change lane or do a full stop.

A parallel planning and control architecture has been introduced mimicking these kind of behavior, which included both long term and short term motion planners. The long term planner (efficiency controller) emphasizes efficiency and solves a long-term optimal control problem in receding horizons with low sampling rate. The short term planner (safety controller) addresses real time safety by solving a short-term optimal control problem with high sampling rate based on the trajectories planned by the efficiency controller. This parallel architecture also addresses the uncertainties, where the long term planner plans according to the most-likely behavior of others, and the short term planner considers almost all possible movements of others in the short term to ensure safety.

Fig. 3. The parallel planning and control architecture in the robot safe interaction system.

However, the robot motion planning problems in clustered environment are highly nonlinear and non-convex, hence hard to solve in real time. To ensure timely responses to the change of the environment, fast algorithms are developed for real-time computation, e.g. the convex feasible set algorithm (CFS) for the long term optimization, and the safe set algorithm (SSA) for the short term optimization. These algorithms achieve faster computation by convexification of the original non-convex problem, which is assumed to have convex objective functions, but non-convex constraints. The convex feasible set algorithm (CFS) iteratively solves a sequence of sub-problems constrained in convex subsets of the feasible domain. The sequence of solutions will converge to a local optima. It converges in fewer iterations and run faster than generic non-convex optimization solvers such as sequential quadratic programming (SQP) and interior point method (ITP). On the other hand, the safe set algorithm (SSA) transforms the non convex state space constraints to convex control space constraints using the idea of invariant set.

Fig. 4. Illustration of convexification in the CFS algorithm.

With the parallel planner and the optimization algorithms, the robot can interact with the environment safely and finish the tasks efficiently.

Fig. 5. Real time motion planning and control.

Towards General Intelligence: the Safe and Efficient Robot Collaboration System (SERoCS)

We now work on an advanced version of RSIS in the Mechanical System Control lab, the safe and efficient robot collaboration system (SERoCS), which is supported by National Science Foundation (NSF) Award #1734109. In addition to safe motion planning and control algorithms for safe human-robot interactions (HRI), SERoCS also consists of robust cognition algorithms for environment monitoring, optimal task planning algorithms for safe human-robot collaboration. The SERoCS will significantly expand the skill sets of the co-robots and prevent or minimize occurrences of human-robot collision and robot-robot collision during operation, hence enables harmonic human-robot collaboration in the future.

Fig. 6. SERoCS Architecture.

This article was initially published on the BAIR blog, and appears here with the authors’ permission.

References

C. Liu, and M. Tomizuka, “Algorithmic safety measures for intelligent industrial co-robots,” in IEEE International Conference on Robotics and Automation (ICRA), 2016.

C. Liu, and M. Tomizuka, “Designing the robot behavior for safe human robot interactions”, in Trends in Control and Decision-Making for Human-Robot Collaboration Systems (Y. Wang and F. Zhang (Eds.)). Springer, 2017.

C. Liu, and M. Tomizuka, “Real time trajectory optimization for nonlinear robotic systems: Relaxation and convexification”, in Systems & Control Letters, vol. 108, pp. 56-63, Oct. 2017.

C. Liu, C. Lin, and M. Tomizuka, “The convex feasible set algorithm for real time optimization in motion planning”, arXiv:1709.00627.

Shelby Rogers for Interesting Engineering: Each robot has its own functions, and all of them have names reminiscent of Star Wars droids.

The Robot Launch global startup competition is over for 2017. We’ve seen startups from all over the world and all sorts of application areas – and we’d like to congratulate the overall winner Semio, and runners up Apellix and Mothership Aeronautics. All three startups met the judges criteria; to be an early stage platform technology in robotics or AI with great impact, large market potential and near term customer pipeline.

Semio from Southern California is a software platform for developing and deploying social robot skills. Ross Mead, founder and CEO of Semio said that “he was greatly looking forward to spending more time with The Robotics Hub, and is excited about the potential for Semio moving forward.”

Apellix from Florida provides software controlled aerial robotic systems that utilize tethered and untethered drones to move workers from harm’s way; such as window washers on skyscrapers (window washing drone, windmill blade cleaning and coating drone), painters on scaffolding (spray painting drone, graffiti removal drone), and workers spraying toxic chemicals (corrosion control).

Robert Dahlstrom, founder and CEO of Apellix said, “As an entrepreneur I strongly believe in startup’s potential to improve lives, create jobs, and make the world a more exciting place. I also know first hand how difficult and challenging a startup can be (an emotional roller coaster ride) and how valuable the work Robot Launch is.”

Mothership Aeronautics from Silicon Valley have a solar powered drone capable of ‘infinity cruise’ where more power is generated than consumed. The drone can perform aerial surveillance and inspection for large scale infrastructures, like pipelines, railways and powerlines. Mothership may also fulfill the ‘warehouse in the sky’ vision that both Amazon and Walmart have tried to patent.

The other awardees are.

• Kinema Systems, impressive approach to logistical challenges from the original Silicon Valley team that developed ROS.
• BotsandUs, highly awarded UK startup with a beautifully designed social robot for retail.
• Fotokite, smart team from ETHZurich with a unique approach to using drones in large scale venues.
• C2RO, from Canada are creating an expansive cloud based AI platform for service robots.
• krtkl, from Silicon Valley are high end embedded board designed for both prototyping and deployment.
• Tennibot, from Alabama have a well designed tennis ball collecting robot. And it’s portable and it’s cute.

And as mentioned in our previous article, the three startups who won the Robohub Choice award were UniExo, BotsAndUs and Northstar Robotics. All the award winners will be featured on Robohub and get access to the Silicon Valley Robotics accelerator program and cowork space, where the award ceremony took place as part of a larger investor/startup showcase.

 

The Silicon Valley Robotics cowork space is at the newly opened Circuit Launch, and provides more than 30,000 sq ft of hot desks and office spaces with professional prototyping facilities. Access to the space is for interesting robotics, AI, AR/VR and sensor technologies, and can include access to the Silicon Valley Robotics startup accelerator program.

The other startups that pitched on the day were; Vecna, Twisted Field, RoboLoco, Dash Shipping, Tekuma, Sake Robotics and Kinema Systems.

Not all of the startups were from the Bay Area – Dash flew up from LA, and Vecna/Twisted Field from Boston, while Tekuma came from Australia as part of an Australian government startup program.

This latest video from the D’Andrea lab shows a quadrotor performing a looping trajectory while balancing an inverted pendulum at the same time.

The video is pretty self-explanatory and includes lots of the technical details – enjoy!

The work, which will be detailed in an upcoming paper, was done by Julien Kohler, Michael Mühlebach, Dario Brescianini, and Raffaello D’Andrea at ETH Zürich. You can learn more about the Flying Machine Arena here.

Our first few submissions have now arrived! Have a holiday robot video of your own that you’d like to share? Send your submissions to editors@robohub.org.

“I made 2000 ugly holiday cards with a $100k robot arm” by Simone Giertz

“Making Ideas Come True” by Danish Technological Institute

“Hey, Jibo. Welcome home for the holidays.” by Jibo

“Bake Together” and “Decorate Together” by iRobot

Keep them coming! Email us your holiday robot videos at editors@robohub.org!

Josh Tenenbaum, a professor of brain and cognitive sciences at MIT, directs research on the development of intelligence at the Center for Brains, Minds, and Machines, a multiuniversity, multidisciplinary project based at MIT that seeks to explain and replicate human intelligence.

Presenting their work at this year’s Conference on Neural Information Processing Systems, Tenenbaum and one of his students, Jiajun Wu, are co-authors on four papers that examine the fundamental cognitive abilities that an intelligent agent requires to navigate the world: discerning distinct objects and inferring how they respond to physical forces.

By building computer systems that begin to approximate these capacities, the researchers believe they can help answer questions about what information-processing resources human beings use at what stages of development. Along the way, the researchers might also generate some insights useful for robotic vision systems.

“The common theme here is really learning to perceive physics,” Tenenbaum says. “That starts with seeing the full 3-D shapes of objects, and multiple objects in a scene, along with their physical properties, like mass and friction, then reasoning about how these objects will move over time. Jiajun’s four papers address this whole space. Taken together, we’re starting to be able to build machines that capture more and more of people’s basic understanding of the physical world.”

Three of the papers deal with inferring information about the physical structure of objects, from both visual and aural data. The fourth deals with predicting how objects will behave on the basis of that data.

Two-way street

Something else that unites all four papers is their unusual approach to machine learning, a technique in which computers learn to perform computational tasks by analyzing huge sets of training data. In a typical machine-learning system, the training data are labeled: Human analysts will have, say, identified the objects in a visual scene or transcribed the words of a spoken sentence. The system attempts to learn what features of the data correlate with what labels, and it’s judged on how well it labels previously unseen data.

In Wu and Tenenbaum’s new papers, the system is trained to infer a physical model of the world — the 3-D shapes of objects that are mostly hidden from view, for instance. But then it works backward, using the model to resynthesize the input data, and its performance is judged on how well the reconstructed data matches the original data.

For instance, using visual images to build a 3-D model of an object in a scene requires stripping away any occluding objects; filtering out confounding visual textures, reflections, and shadows; and inferring the shape of unseen surfaces. Once Wu and Tenenbaum’s system has built such a model, however, it rotates it in space and adds visual textures back in until it can approximate the input data.

Indeed, two of the researchers’ four papers address the complex problem of inferring 3-D models from visual data. On those papers, they’re joined by four other MIT researchers, including William Freeman, the Perkins Professor of Electrical Engineering and Computer Science, and by colleagues at DeepMind, ShanghaiTech University, and Shanghai Jiao Tong University.

Divide and conquer

The researchers’ system is based on the influential theories of the MIT neuroscientist David Marr, who died in 1980 at the tragically young age of 35. Marr hypothesized that in interpreting a visual scene, the brain first creates what he called a 2.5-D sketch of the objects it contained — a representation of just those surfaces of the objects facing the viewer. Then, on the basis of the 2.5-D sketch — not the raw visual information about the scene — the brain infers the full, three-dimensional shapes of the objects.

“Both problems are very hard, but there’s a nice way to disentangle them,” Wu says. “You can do them one at a time, so you don’t have to deal with both of them at the same time, which is even harder.”

Wu and his colleagues’ system needs to be trained on data that include both visual images and 3-D models of the objects the images depict. Constructing accurate 3-D models of the objects depicted in real photographs would be prohibitively time consuming, so initially, the researchers train their system using synthetic data, in which the visual image is generated from the 3-D model, rather than vice versa. The process of creating the data is like that of creating a computer-animated film.

Once the system has been trained on synthetic data, however, it can be fine-tuned using real data. That’s because its ultimate performance criterion is the accuracy with which it reconstructs the input data. It’s still building 3-D models, but they don’t need to be compared to human-constructed models for performance assessment.

In evaluating their system, the researchers used a measure called intersection over union, which is common in the field. On that measure, their system outperforms its predecessors. But a given intersection-over-union score leaves a lot of room for local variation in the smoothness and shape of a 3-D model. So Wu and his colleagues also conducted a qualitative study of the models’ fidelity to the source images. Of the study’s participants, 74 percent preferred the new system’s reconstructions to those of its predecessors.

All that fall

In another of Wu and Tenenbaum’s papers, on which they’re joined again by Freeman and by researchers at MIT, Cambridge University, and ShanghaiTech University, they train a system to analyze audio recordings of an object being dropped, to infer properties such as the object’s shape, its composition, and the height from which it fell. Again, the system is trained to produce an abstract representation of the object, which, in turn, it uses to synthesize the sound the object would make when dropped from a particular height. The system’s performance is judged on the similarity between the synthesized sound and the source sound.

Finally, in their fourth paper, Wu, Tenenbaum, Freeman, and colleagues at DeepMind and Oxford University describe a system that begins to model humans’ intuitive understanding of the physical forces acting on objects in the world. This paper picks up where the previous papers leave off: It assumes that the system has already deduced objects’ 3-D shapes.

Those shapes are simple: balls and cubes. The researchers trained their system to perform two tasks. The first is to estimate the velocities of balls traveling on a billiard table and, on that basis, to predict how they will behave after a collision. The second is to analyze a static image of stacked cubes and determine whether they will fall and, if so, where the cubes will land.

Wu developed a representational language he calls scene XML that can quantitatively characterize the relative positions of objects in a visual scene. The system first learns to describe input data in that language. It then feeds that description to something called a physics engine, which models the physical forces acting on the represented objects. Physics engines are a staple of both computer animation, where they generate the movement of clothing, falling objects, and the like, and of scientific computing, where they’re used for large-scale physical simulations.

After the physics engine has predicted the motions of the balls and boxes, that information is fed to a graphics engine, whose output is, again, compared with the source images. As with the work on visual discrimination, the researchers train their system on synthetic data before refining it with real data.

In tests, the researchers’ system again outperformed its predecessors. In fact, in some of the tests involving billiard balls, it frequently outperformed human observers as well.

Matt Simon for Wired: “Companies like Amazon and others are now delivering products at an unprecedented rate, something like 500 packages per second. And that is only going to grow.”

Robotiq Updates FT 300 Sensitivity For High Precision Tasks With Universal Robots
Force Torque Sensor feeds data to Universal Robots force mode

Quebec City, Canada, November 13, 2017 – Robotiq launches a 10 times more sensitive version of its FT 300 Force Torque Sensor. With Plug + Play integration on all Universal Robots, the FT 300 performs highly repeatable precision force control tasks such as finishing, product testing, assembly and precise part insertion.

This force torque sensor comes with an updated free URCap software able to feed data to the Universal Robots Force Mode. “This new feature allows the user to perform precise force insertion assembly and many finishing applications where force control with high sensitivity is required” explains Robotiq CTO Jean-Philippe Jobin*.

The URCap also includes a new calibration routine. “We’ve integrated a step-by-step procedure that guides the user through the process, which takes less than 2 minutes” adds Jobin. “A new dashboard also provides real-time force and moment readings on all 6 axes. Moreover, pre-built programming functions are now embedded in the URCap for intuitive programming.”

See some of the FT 300’s new capabilities in the following demo videos:

#1 How to calibrate with the FT 300 URCap Dashboard

#2 Linear search  demo

#3 Path recording demo

Visit the FT 300 webpage  or get a quote here

Get the FT 300 specs here

Get more info in the FAQ

Get free Skills to accelerate robot programming of force control tasks.

Get free robot cell deployment resources on leanrobotics.org

* Available with Universal Robots CB3.1 controller only

About Robotiq

Robotiq’s Lean Robotics methodology and products enable manufacturers to deploy productive robot cells across their factory. They leverage the Lean Robotics methodology for faster time to production and increased productivity from their robots. Production engineers standardize on Robotiq’s Plug + Play components for their ease of programming, built-in integration, and adaptability to many processes. They rely on the Flow software suite to accelerate robot projects and optimize robot performance once in production.

Robotiq is the humans behind the robots: an employee-owned business with a passionate team and an international partner network.

Media contact

David Maltais, Communications and Public Relations Coordinator

d.maltais@robotiq.com

1-418-929-2513

////

Press Release Provided by: Robotiq.Com

The post FT 300 force torque sensor appeared first on Roboticmagazine.