Since 2007, two professors at the TU Delft have been researching ways to harvest energy from the wind using a kite. The robotic kite looks set to make its debut in the energy sector, but often inventions are used in unexpected ways. In this series of articles, we take robot innovations from their test-lab and bring them to a randomly selected workplace in the outside world. From kindergarten teacher Fransien, we learn that big kites could also be child’s play, quite literally.
A robot wheels in the kite and then slowly releases it, painting 8-shaped loops on the sky. As this happens, the motion of the unwinding rope generates electricity. This innovative solution harmonises power generation with the flows of the atmosphere. Kitepower uses wind to provide us with energy, but rather than simply sticking a big fan in the air and waiting for air flows to come and go, it latches on to the wind and stays with it.
Professor Schmehl explains his research: “We explored the idea of wind energy through the use of robotically engineered kites”. The result is a big sail floating high up in the air with a small unit that automatically steers, controls the pulling force and connects to the ground station. This ground station senses when to reel in the kite to convert a maximum of motion – or kinetic energy – into electricity. Using only the vital parts of a wind-turbine, KitePower is cheaper to make, less noisy and more durable than traditional windmills.
KitePower is the company founded to bring this product to market. In England, the idea is to build kite stations in the sea, but in the Netherlands, KitePower is looking to launch them on solid land. The technical challenge now is to get fly the kite above a certain altitude and make its flight autonomous so the system can work without human interaction. While this technology is being optimised, we were wondering: could there be a different, totally unexpected way of using this technology?
To explore this, we took Kitepower to Fransien Godeke, a kindergarten teacher in the Bilt. She has been educating four to six year-old children for 36 years. Her understanding of how kids grow and learn is incredible, and we were curious how Fransien would use this robot innovation in the context of her work.
Copied by a six-year old
“We once tried to build our own mini-parachutes in class, maybe we could make this as well?”, is her first thought. Fransien tells me that children learn most through playing and making things themselves. A re-engineering process would allow them to experience what is needed to keep a kite like this floating in the wind, without any help. She says: “We could use straws to make the shape, pull through little cords and find very thin paper to lift the entire toy in the air.”
Make a big dragon
Children also love pretending to be something they’re not. “Kitepower looks like it has a futuristic dragon skin”, Fransien fantasises. With a kite this large, the kids could transform into a ten meter long dragon. They could jump and run around and show of to other kids. Or build a big hut to shelter in and maybe also become a train. In short, she imagines that kids would make anything they want with this versatile piece of kite fabric created by Professor Schmehl and his colleagues.
Physics of flight
If the class would get access to a real Kitepower kite, we could try to get it up in the air together says Fransien: “Children somehow have the impulse to start running when they want to get something airborne. Perhaps this robot could show them how speed translates to lift and power”.
A theatrical flying dragon show for entertaining people in nature that also generates green dragon energy. How cool is that as a spectacle?
Finally, we come to the question whether childer would be clever enough to understand the concept of Kitepower. Fransien says: “If you show them the physics, they will understand it”. To illustrate how fast Fransien’s kids learn, she tells me her toddlers already code their own programs: “They get a little matrix, I tell them where the little robot – called Bbot – should go and they program it within minutes”.
KitePower is being developed to solve the world’s energy crisis, but in the kindergarten it becomes a wonderful tool for irreverent experimentation by the world’s youngest engineers. Just imagining what children would do with Kitepower, already opens up exciting new avenues that we didn’t see before. A theatrical flying dragon show for entertaining people in nature that also generates green dragon energy: how cool is that as a spectacle?
The challenge for many companies is how to consistently maintain a safe workplace without creating barriers to productivity or innovation. Enter autonomous mobile robots (AMRs).
Carnegie Mellon University's acclaimed snake-like robot can now slither its way underwater, allowing the modular robotics platform to inspect ships, submarines and infrastructure for damage.
A team of researchers at the University of California, Berkeley, has built a two-legged robot with t he ability to teach itself to walk using reinforcement learning. The team has written a paper describing their work and has uploaded it to the arXiv preprint server.
In this webinar we will look at how integrating robotics into the working world means that both cobots and autonomous systems will be important differentiators for production, processes, and innovation.
Waypoint Robotics/Productive Robotics omnidirectional 7 DoF mobile manipulator combines the easiest to use, most capable omnidirectional autonomous mobile robot with the simplest and most flexible 7 axis collaborative robot arm. The Vector AMR’s omnidirectional mobility enables fast, precise docking in any direction or orientation so the OB7 can perform accurate and precise grasping or picking tasks, taking full advantage of the cobots’ 7 degrees of freedom (7 DoF) and superior dexterity. It can be powered using Waypoint’s EnZone wireless charger that provides on-demand energy and opportunity charging for longer run times. Workers can use this mobile platform to perform repetitive tasks such as machine tending, quality assurance sampling, material replenishment, packaging, and many others, so they can focus on the high skilled jobs for which they are uniquely qualified. It has never been easier or more cost effective to deploy a mobile manipulator for manufacturing and logistics applications.
Physician driving CorPath GRX from the interventional cockpit. Image Credit: Corindus
In 2018, five patients at the Apex Heart Institute in Ahmedabad, India, received treatment for coronary artery disease (CAD) the same way 3 million others do each year: a small balloon was inserted into an artery in the heart and inflated, making way for the placement of a stent to keep the vital pathway open.
The procedure, known as percutaneous coronary intervention (PCI), is the standard treatment for atherosclerosis, a common CAD marked by the buildup of plaque inside the arteries and a subsequent restriction of blood flow. Like many patients before them, their operation was assisted by a robot—the CorPath GRX robotic platform from Corindus, A Siemens Healthineers Company.
Yet unlike anyone else before them, these five patients were part of an amazing first: their principal physician was not in the room with them during the procedure. In fact, he was 20 miles away, guiding the robot—performing the operation to perfection—from a remote workstation.
Welcome to the new frontier of telemedicine.
Robotic Surgery Tackles Leading Cause of Death
According to the World Health Organization (WHO), cardiovascular disease (CVD) is the number one cause of death globally, claiming 17.9 million lives per year. The impact is greatest in locations lacking ready access to critical care, such as developing countries and rural areas.
The WHO notes that over three-quarters of deaths from CVD occur in low- and middle-income countries. In developed countries, the issue arises in rural communities where smaller hospitals often do not have specialists, such as interventional cardiologists, on staff. It is in these areas where remote solutions like the one Corindus has pioneered offer hope and tantalizing possibilities.
Dr. Tejas Patel performed these remote interventions and wrote about them with his colleagues Sanjay Shah and Samir Pancholy in EClinicalMedicine. According to the article, “The vast majority of patients with CAD or acute coronary syndrome in developing countries have little or no access to immediate interventional therapy. Current robotic technology combined with improvements in network connectivity and operator expertise in R-PCI [robot-assisted PCI] procedures can be employed as a front-line service in regions where such expertise is not available.” The article also noted that the system could be used as a supplemental service to provide expertise to a larger number of patients.
“Go Remote or Go Home”
Reaching patients around the world wherever and whenever they need treatment has been on the mind of the Corindus team since the introduction of their CorPath robotic platform. So has physician safety.
While CorPath’s robotic advances have provided physicians with exceptional precision and control during PCI procedures, the system still needed to do more to safeguard physician health—exposure to radiation from imaging techniques used in the procedures was common, as were orthopedic injuries stemming from the necessary heavy protective equipment doctors were forced to wear.
“Sensing an opportunity to increase the safety of surgeons while at the same time addressing the growing global need for innovative telemedicine solutions, Corindus CEO Mark Toland famously told our team that it was time for us to ‘go remote or go home,’” recalls Nicholas Kottenstette, Ph.D., R&D Fellow at Corindus. “This battle cry resonated across the company, forcing us to push the envelope, further evolve the capabilities and possibilities of our CorPath system, and achieve another industry milestone: enabling the first remote robotic-assisted PCI.”
Technology for Aircraft Control Leads to Robotic Surgery
Kottenstette has worked on the robot product line since joining Corindus. But he started his career in a different field. He received his Ph.D. in electrical engineering at Notre Dame, where he designed a framework to control systems over networks, ensuring stability while accounting for time delay and data loss. He then began his career at Vanderbilt University, developing robotic control systems for aircraft using Model-Based Design. It was this work that led to a different type of control system that operated over the network: the surgical robot.
All of that work, which involved Model-Based Design using MATLAB and Simulink, has proven integral to meeting the challenges of developing our precision robotic systems.
Nicholas Kottenstette, Ph.D., R&D Fellow at Corindus
To understand the complex workings of the robotic system, it helps to take a basic inventory of its key components. The system comprises two main stations: a bedside unit, the part of the system which allows for device manipulation within the patient, and an interventional cockpit, from which the physician guides the devices during an intervention.
The bedside unit has an extended reach arm and bedside touch screen. This is operated by the scrub staff. Image Credit: CorindusThe interventional cockpit comprises a radiation shield, monitor, and control console that enable the interventional cardiologist to work from a seated position without wearing lead. Image Credit: Corindus
The bedside unit consists of the extended reach arm, the robotic drive, and the single-use sterile cassette. The arm positions the robotic drive and cassette. The robotic drive, which receives inputs from the control console in the cockpit, guides the cassette, which in turn manipulates the guidewire, balloon/stent catheter, and guide catheter in the patient’s body.
The interventional cardiologist works from the interventional cockpit, which contains both the robot control subsystem and a telepresence communication system, and links the physician to the robot in the bedside unit.
“The robot control substation contains a control computing system, monitors, networking equipment (i.e., connectivity) and a robotic console with three joysticks,” says Dr. Patel. “The monitors display real-time hemodynamic variables and fluoroscopic video, providing the operator enhanced visualization of the PCI procedure. One joystick is for balloon/stent manipulation, one for guidewire manipulation, and the third for guide catheter manipulation.”
A closer look at these two main systems—the bedside unit and interventional cockpit—reveals a collaboration and interdependence essential to the successful performance of the robotic system. Combined, these systems have delivered quantifiable benefits, including increased accuracy in measuring anatomy to determine lesion size and stent length, and increased precision in stent positioning.
Long-Distance Surgery
Performing an intricate procedure remotely posed significant design challenges, chief among them real-time, end-to-end video capture and processing: physicians may operate less effectively when there is significant lag or latency in the images they see and commands they send over a network. In addition, it is important that the physician is aware of the quality of the network connection, which includes network latencies and received frames of images per second (throughput). The system should limit the physician from operating under poor networking conditions in order to limit risks of patient harm.
To overcome the challenges of real-time systems operations, Kottenstette drew on his track record of innovation using Model-Based Design.
MATLAB, Simulink, and Simulink Real-Time have long been staples of my application-development work on the CorPath system, from embedded motor controls for guiding the robotic arm’s motions, to the way the fluoroscopic images are communicated with the workstations. My team modeled the remote system using Model-Based Design.
Nicholas Kottenstette, Ph.D., R&D Fellow at Corindus.
That approach has paid dividends. For example, as Corindus began to develop its next generation platform, CorPath GRX, it used universal cameras well before there was real-time support for USB 3.0 devices.
“As we pushed to develop an advanced real-time video capability that didn’t interrupt the physicians’ normal workflow, MathWorks worked hand in hand with us to develop the needed support,” says Kottenstette. “Once we had that, we could compress and decompress images as needed to facilitate their real-time transmission and consumption by the remote operator.”
As for ensuring a dedicated real-time network capability, Corindus used the Speedgoat family of target machines—high-performance computers optimized for specific applications—to execute his system’s mission-critical applications. With a Speedgoat target at the operating site and one at the remote location, where it supported the interventional cockpit, CorPath GRX performed admirably.
Dr. Patel notes in his report, “The remote R-PCI procedure was successful in all aspects. The remote interventionalist operator rated the functionality of the robotic platform and the network connection system as equivalent to an in-lab manual PCI procedure with no significant procedural delays or technical difficulties. This was corroborated by the 53 millisecond mean delay recorded by the network, which is most likely imperceptible to the operator.”
Kottenstette designed Simulink Real-Time models that control the bedside unit.
Encouraged by its breakthrough success with remote R-PCI procedures, Corindus is continuing to think big—big enough to lead a telemedicine revolution in brain treatment as well.
“Every second matters for stroke victims, just as it does for those who suffer heart attacks,” says Kottenstette. “Our ability to treat patients wherever they are with our remote robotic protocol is the wave of the future and is driving our next set of CorPath innovations as we take on strokes, which is the number one cause of disability and fifth leading cause of death in the U.S.”
Watch this video (3:11) to see how Corindus runs real-time applications on Speedgoat targets, using Precision Time Protocol to synchronize the local and remote site clocks.
Teams from Bosnia and Herzegovina and Croatia have concocted using robots to screen honey bees while they work. An automated vehicle flies around, catching pictures of the bug, which PCs subsequently examine to uncover potential mines covered up in the ground.
Do you fancy making yourself an industrial robot to enjoy at home? Jeremy Fielding, a passionate fan of mechanical engineering, did. So he built one. Good news is: he’s preparing a series of videos to teach you the whole process from scratch. How much power do you need to run 7 motors at one time? If you lose power, how do you prevent the arm from collapsing on you or dropping the load? How do you keep the cost down? He’s recorded over 100 hours of video, and he’s planning to teach you how he used the servo motors, how you can use them for your projects and how he designed his 7 axis, articulated robot.
Jeremy’s aim (website, YouTube, Twitter, Instagram) is simple: draw people to engineering with amazing projects, inspire them with ideas, then teach them how to do it. And for this video series, he’s also looking for your collaboration. So if you’ve got experience and knowledge on building this type of robots and you’d like to share it, maybe you end up being part of the series!
We’d like to thank Black in Robotics for making it possible for us to discover Jeremy. Here’s the video Jeremy has released to introduce his project:
Robots are writing more of what we read on the internet. And artificial intelligence (AI) writing tools are becoming freely available for anyone, including students, to use.
Can loads be positioned dynamically, with high precision and no oscillation, despite mechanical backlash and elastic components? Yes — using an intelligent system.
Biologists have long experienced the challenges of documenting ocean life, with many species of fish proving quite sensitive to the underwater movements of humans.
Talking Robotics is a series of virtual seminars about Robotics and its interaction with other relevant fields, such as Artificial Intelligence, Machine Learning, Design Research, Human-Robot Interaction, among others. They aim to promote reflections, dialogues, and a place to network. In this seminars compilation, we bring you 7 talks (and a half?) from current roboticists for your enjoyment.
Filipa Correia “Group Intelligence on Social Robots”
Filipa Correia received a M.Sc. in Computer Science from University of Lisbon, Portugal, 2015. She is currently a junior researcher at GAIPSLab and she is pursuing a Ph.D. on Human-Robot Interaction at University of Lisbon, Portugal.
Her PhD thesis is focused on the challenges of creating social robots that are capable of sustaining cohesive alliances in team settings with humans. Moreover, it contributes with computational mechanisms for the robotic teammate to autonomously express group-based emotions or to gaze human teammates in multi-party settings.
For more information about the speaker and related papers, please see this website.
Ross Mead “Bringing Robots To Life”
Dr. Ross Mead is the Founder and CEO of Semio. Ross received his PhD and MS in Computer Science from the University of Southern California in 2015, and his BS in Computer Science from Southern Illinois University Edwardsville in 2007.
In this talk, Dr. Ross Mead discussed the technological advances paving the way for the personal robotics revolution, and the robotics hardware companies leading the charge along that path. He also introduced the software innovations in development at Semio for bringing robots to life.
For more information about the speaker and related papers, please see this website.
Kim Baraka “Humans and Robots Teaching and Learning through Social Interaction”
Kim Baraka is currently a postdoctoral fellow at the Socially Intelligent Machines Lab at the University of Texas at Austin. He holds a dual Ph.D. in Robotics from Carnegie Mellon University and Instituto Superior Técnico (Portugal), an M.S. in Robotics from Carnegie Mellon, and a Bachelor in Electrical and Computer Engineering from the American University of Beirut.
In the first part of the talk, focusing on robots teaching humans, Kim discussed algorithmic solutions that enable socially assistive robots to teach both children and therapists in a personalized way. In the second part of the talk, focusing on humans teaching robots, Kim discussed some preliminary efforts towards developing ways in which robots can learn tasks from human teachers in richer and more natural ways.
For more information about the speaker and related papers, please see this website.
Glenda Hannibal “Trust in HRI: Probing Vulnerability as an Active Precondition”
Glenda has previously worked in the Department of Sociology at the University of Vienna and as an expert for the HUMAINT project at the European Commission. Glenda holds a BA and MA in Philosophy from Aarhus University and is currently a PhD student in the Trust Robots Doctoral College and Human-Computer Interaction group at TU Wien.
In this talk, Glenda presented her research on vulnerability as a precondition of trust in HRI. In the first part, she argued that while the most commonly cited definitions of trust used in HRI recognize vulnerability as an essential element of trust, it is also often considered somewhat problematic too. In the second part of her talk, she presented the results of two empirical studies she has undertaken to explore trust in HRI in relation to vulnerability. Finally, she reflected on few ethical aspects related to this theme to end this talk.
For more information about the speaker and related papers, please see this website.
Carl Mueller “Robot Learning from Demonstration Driven Constrained Skill Learning & Motion Planning”
Carl Mueller is a Ph.D. student of computer science at the University of Colorado – Boulder, advised by Professor Bradley Hayes within the Collaborative Artificial Intelligence and Robotics Laboratory. He graduated from the University of California – Santa Barbara with a degree in Biopsychology and after a circuitous route through the pharmaceutical industry, he ended up in the tech, founding his own company building intelligent chat agents for business analytics.
The major theme of his research is the enablement of human users to communicate additional information to the robot learning system through ‘concept constraints’. Concept Constraints are abstract behavioral restrictions grounded as geometric and kinodynamical planning predicates that prohibit or limit the behavior of the robot resulting in more robust, generalizable, and safe skill execution. In this talk, Carl discussed how conceptual constraints are integrated into existing LfD methods, how unique interfaces can further enhance the communication of such constraints, and how the grounding of these constraints requires constrained motion planning techniques.
For more information about the speaker and related papers, please see this website.
Daniel Rakita “Methods and Applications for Generating Accurate and Feasible Robot-arm Motions in Real-time”
Daniel Rakita is a Ph.D. student of computer science at the University of Wisconsin-Madison advised by Michael Gleicher and Bilge Mutlu. He received a Bachelors of Music Performance from the Indiana University Jacobs School of Music in 2012.
In this talk, he overviewed technical methods they have developed that attempt to achieve feasible, accurate, and time-sensitive robot-arm motions. In particular, he detailed their inverse kinematics solver called RelaxedIK that utilizes both non-linear optimization and machine learning to achieve a smooth, feasible, and accurate end-effector to joint-space mapping on-the-fly. He highlighted numerous ways they have applied their technical methods to real-world-inspired problems, such as mapping human-arm-motion to robot-arm-motion in real-time to afford effective shared-control interfaces and automatically moving a camera-in-hand robot in a remote setting to optimize a viewpoint for a teleoperator.
For more information about the speaker and related papers, please see this website.
Barbara Bruno “Culture-Aware Robotics”
Barbara Bruno is a post-doc researcher at the École Polytechnique Fédérale de Lausanne (EPFL), in Lausanne, Switzerland, in the CHILI lab. Barbara received the M.Sc. and the Ph.D. in Robotics from the University of Genoa in 2011 and 2015, respectively. She is part of the NCCR Robotics organisation and currently involved in the EU ITN ANIMATAS.
In this talk, she explored how existing quantitative and qualitative methods for the assessment of culture, and cultural differences, can be combined with knowledge representation and reasoning tools such as ontologies and fuzzy controllers to endow robots with the capability of taking cultural factors into account in a range of tasks going from low-level motion planning to high-level dialogue management and user adaptation.
For more information about the speaker and related papers, please see this website.
Extra: When you forget to hit the record button but you value the speaker so much that you compose a musical summary of his talk
Because as they say, we are human after all!
Nils Hagberg “A call for a Human Approach to Technology”
Nils Hagberg is the Product Owner at Furhat Robotics. He is a computer linguist with almost ten years of industry experience in human-machine interaction and conversational technology.
In this talk, he gave a few examples of human-centric business case designs that he’s come across earlier in his career and at Furhat Robotics – the social robotics company that have set out to make technology more human. His hope for this talk is that it will put your own work in a larger context and nudge you towards a path that will ensure humans will be allowed to remain human.
For more information about the speaker and related papers, please see this website.
If a robot worker makes a mistake on the job, or annoys customers, businesses may not give it a pink slip and a cardboard box for its office belongings, but companies may be forced to shut down these expensive machines, according to a team of researchers.
