In 2018, five patients at the Apex Heart Institute in Ahmedabad, India, received treatment for coronary artery disease (CAD) the same way 3 million others do each year: a small balloon was inserted into an artery in the heart and inflated, making way for the placement of a stent to keep the vital pathway open.
The procedure, known as percutaneous coronary intervention (PCI), is the standard treatment for atherosclerosis, a common CAD marked by the buildup of plaque inside the arteries and a subsequent restriction of blood flow. Like many patients before them, their operation was assisted by a robot—the CorPath GRX robotic platform from Corindus, A Siemens Healthineers Company.
Yet unlike anyone else before them, these five patients were part of an amazing first: their principal physician was not in the room with them during the procedure. In fact, he was 20 miles away, guiding the robot—performing the operation to perfection—from a remote workstation.
Welcome to the new frontier of telemedicine.
According to the World Health Organization (WHO), cardiovascular disease (CVD) is the number one cause of death globally, claiming 17.9 million lives per year. The impact is greatest in locations lacking ready access to critical care, such as developing countries and rural areas.
The WHO notes that over three-quarters of deaths from CVD occur in low- and middle-income countries. In developed countries, the issue arises in rural communities where smaller hospitals often do not have specialists, such as interventional cardiologists, on staff. It is in these areas where remote solutions like the one Corindus has pioneered offer hope and tantalizing possibilities.
Dr. Tejas Patel performed these remote interventions and wrote about them with his colleagues Sanjay Shah and Samir Pancholy in EClinicalMedicine. According to the article, “The vast majority of patients with CAD or acute coronary syndrome in developing countries have little or no access to immediate interventional therapy. Current robotic technology combined with improvements in network connectivity and operator expertise in R-PCI [robot-assisted PCI] procedures can be employed as a front-line service in regions where such expertise is not available.” The article also noted that the system could be used as a supplemental service to provide expertise to a larger number of patients.
Reaching patients around the world wherever and whenever they need treatment has been on the mind of the Corindus team since the introduction of their CorPath robotic platform. So has physician safety.
While CorPath’s robotic advances have provided physicians with exceptional precision and control during PCI procedures, the system still needed to do more to safeguard physician health—exposure to radiation from imaging techniques used in the procedures was common, as were orthopedic injuries stemming from the necessary heavy protective equipment doctors were forced to wear.
“Sensing an opportunity to increase the safety of surgeons while at the same time addressing the growing global need for innovative telemedicine solutions, Corindus CEO Mark Toland famously told our team that it was time for us to ‘go remote or go home,’” recalls Nicholas Kottenstette, Ph.D., R&D Fellow at Corindus. “This battle cry resonated across the company, forcing us to push the envelope, further evolve the capabilities and possibilities of our CorPath system, and achieve another industry milestone: enabling the first remote robotic-assisted PCI.”
Kottenstette has worked on the robot product line since joining Corindus. But he started his career in a different field. He received his Ph.D. in electrical engineering at Notre Dame, where he designed a framework to control systems over networks, ensuring stability while accounting for time delay and data loss. He then began his career at Vanderbilt University, developing robotic control systems for aircraft using Model-Based Design. It was this work that led to a different type of control system that operated over the network: the surgical robot.
All of that work, which involved Model-Based Design using MATLAB and Simulink, has proven integral to meeting the challenges of developing our precision robotic systems.
Nicholas Kottenstette, Ph.D., R&D Fellow at Corindus
To understand the complex workings of the robotic system, it helps to take a basic inventory of its key components. The system comprises two main stations: a bedside unit, the part of the system which allows for device manipulation within the patient, and an interventional cockpit, from which the physician guides the devices during an intervention.
The bedside unit consists of the extended reach arm, the robotic drive, and the single-use sterile cassette. The arm positions the robotic drive and cassette. The robotic drive, which receives inputs from the control console in the cockpit, guides the cassette, which in turn manipulates the guidewire, balloon/stent catheter, and guide catheter in the patient’s body.
The interventional cardiologist works from the interventional cockpit, which contains both the robot control subsystem and a telepresence communication system, and links the physician to the robot in the bedside unit.
“The robot control substation contains a control computing system, monitors, networking equipment (i.e., connectivity) and a robotic console with three joysticks,” says Dr. Patel. “The monitors display real-time hemodynamic variables and fluoroscopic video, providing the operator enhanced visualization of the PCI procedure. One joystick is for balloon/stent manipulation, one for guidewire manipulation, and the third for guide catheter manipulation.”
A closer look at these two main systems—the bedside unit and interventional cockpit—reveals a collaboration and interdependence essential to the successful performance of the robotic system. Combined, these systems have delivered quantifiable benefits, including increased accuracy in measuring anatomy to determine lesion size and stent length, and increased precision in stent positioning.
Performing an intricate procedure remotely posed significant design challenges, chief among them real-time, end-to-end video capture and processing: physicians may operate less effectively when there is significant lag or latency in the images they see and commands they send over a network. In addition, it is important that the physician is aware of the quality of the network connection, which includes network latencies and received frames of images per second (throughput). The system should limit the physician from operating under poor networking conditions in order to limit risks of patient harm.
To overcome the challenges of real-time systems operations, Kottenstette drew on his track record of innovation using Model-Based Design.
MATLAB, Simulink, and Simulink Real-Time have long been staples of my application-development work on the CorPath system, from embedded motor controls for guiding the robotic arm’s motions, to the way the fluoroscopic images are communicated with the workstations. My team modeled the remote system using Model-Based Design.
Nicholas Kottenstette, Ph.D., R&D Fellow at Corindus.
That approach has paid dividends. For example, as Corindus began to develop its next generation platform, CorPath GRX, it used universal cameras well before there was real-time support for USB 3.0 devices.
“As we pushed to develop an advanced real-time video capability that didn’t interrupt the physicians’ normal workflow, MathWorks worked hand in hand with us to develop the needed support,” says Kottenstette. “Once we had that, we could compress and decompress images as needed to facilitate their real-time transmission and consumption by the remote operator.”
As for ensuring a dedicated real-time network capability, Corindus used the Speedgoat family of target machines—high-performance computers optimized for specific applications—to execute his system’s mission-critical applications. With a Speedgoat target at the operating site and one at the remote location, where it supported the interventional cockpit, CorPath GRX performed admirably.
Dr. Patel notes in his report, “The remote R-PCI procedure was successful in all aspects. The remote interventionalist operator rated the functionality of the robotic platform and the network connection system as equivalent to an in-lab manual PCI procedure with no significant procedural delays or technical difficulties. This was corroborated by the 53 millisecond mean delay recorded by the network, which is most likely imperceptible to the operator.”
Encouraged by its breakthrough success with remote R-PCI procedures, Corindus is continuing to think big—big enough to lead a telemedicine revolution in brain treatment as well.
“Every second matters for stroke victims, just as it does for those who suffer heart attacks,” says Kottenstette. “Our ability to treat patients wherever they are with our remote robotic protocol is the wave of the future and is driving our next set of CorPath innovations as we take on strokes, which is the number one cause of disability and fifth leading cause of death in the U.S.”
Watch this video (3:11) to see how Corindus runs real-time applications on Speedgoat targets, using Precision Time Protocol to synchronize the local and remote site clocks.
Do you fancy making yourself an industrial robot to enjoy at home? Jeremy Fielding, a passionate fan of mechanical engineering, did. So he built one. Good news is: he’s preparing a series of videos to teach you the whole process from scratch. How much power do you need to run 7 motors at one time? If you lose power, how do you prevent the arm from collapsing on you or dropping the load? How do you keep the cost down? He’s recorded over 100 hours of video, and he’s planning to teach you how he used the servo motors, how you can use them for your projects and how he designed his 7 axis, articulated robot.
Jeremy’s aim (website, YouTube, Twitter, Instagram) is simple: draw people to engineering with amazing projects, inspire them with ideas, then teach them how to do it. And for this video series, he’s also looking for your collaboration. So if you’ve got experience and knowledge on building this type of robots and you’d like to share it, maybe you end up being part of the series!
We’d like to thank Black in Robotics for making it possible for us to discover Jeremy. Here’s the video Jeremy has released to introduce his project:
We had to go find this guy on YouTube because he looked very cool! https://t.co/xomix1TZXo @jeremy_fielding #BlackInRobotics #NationalRoboticsWeek @RoboWeek
— Black in Robotics (@BlackInRobotics) April 9, 2021
Check out Jeremy’s YouTube channel to discover many more instructional videos. You can also support his work on Patreon.
Talking Robotics is a series of virtual seminars about Robotics and its interaction with other relevant fields, such as Artificial Intelligence, Machine Learning, Design Research, Human-Robot Interaction, among others. They aim to promote reflections, dialogues, and a place to network. In this seminars compilation, we bring you 7 talks (and a half?) from current roboticists for your enjoyment.
Filipa Correia “Group Intelligence on Social Robots”
Filipa Correia received a M.Sc. in Computer Science from University of Lisbon, Portugal, 2015. She is currently a junior researcher at GAIPSLab and she is pursuing a Ph.D. on Human-Robot Interaction at University of Lisbon, Portugal.
Her PhD thesis is focused on the challenges of creating social robots that are capable of sustaining cohesive alliances in team settings with humans. Moreover, it contributes with computational mechanisms for the robotic teammate to autonomously express group-based emotions or to gaze human teammates in multi-party settings.
For more information about the speaker and related papers, please see this website.
Ross Mead “Bringing Robots To Life”
Dr. Ross Mead is the Founder and CEO of Semio. Ross received his PhD and MS in Computer Science from the University of Southern California in 2015, and his BS in Computer Science from Southern Illinois University Edwardsville in 2007.
In this talk, Dr. Ross Mead discussed the technological advances paving the way for the personal robotics revolution, and the robotics hardware companies leading the charge along that path. He also introduced the software innovations in development at Semio for bringing robots to life.
For more information about the speaker and related papers, please see this website.
Kim Baraka “Humans and Robots Teaching and Learning through Social Interaction”
Kim Baraka is currently a postdoctoral fellow at the Socially Intelligent Machines Lab at the University of Texas at Austin. He holds a dual Ph.D. in Robotics from Carnegie Mellon University and Instituto Superior Técnico (Portugal), an M.S. in Robotics from Carnegie Mellon, and a Bachelor in Electrical and Computer Engineering from the American University of Beirut.
In the first part of the talk, focusing on robots teaching humans, Kim discussed algorithmic solutions that enable socially assistive robots to teach both children and therapists in a personalized way. In the second part of the talk, focusing on humans teaching robots, Kim discussed some preliminary efforts towards developing ways in which robots can learn tasks from human teachers in richer and more natural ways.
For more information about the speaker and related papers, please see this website.
Glenda Hannibal “Trust in HRI: Probing Vulnerability as an Active Precondition”
Glenda has previously worked in the Department of Sociology at the University of Vienna and as an expert for the HUMAINT project at the European Commission. Glenda holds a BA and MA in Philosophy from Aarhus University and is currently a PhD student in the Trust Robots Doctoral College and Human-Computer Interaction group at TU Wien.
In this talk, Glenda presented her research on vulnerability as a precondition of trust in HRI. In the first part, she argued that while the most commonly cited definitions of trust used in HRI recognize vulnerability as an essential element of trust, it is also often considered somewhat problematic too. In the second part of her talk, she presented the results of two empirical studies she has undertaken to explore trust in HRI in relation to vulnerability. Finally, she reflected on few ethical aspects related to this theme to end this talk.
For more information about the speaker and related papers, please see this website.
Carl Mueller “Robot Learning from Demonstration Driven Constrained Skill Learning & Motion Planning”
Carl Mueller is a Ph.D. student of computer science at the University of Colorado – Boulder, advised by Professor Bradley Hayes within the Collaborative Artificial Intelligence and Robotics Laboratory. He graduated from the University of California – Santa Barbara with a degree in Biopsychology and after a circuitous route through the pharmaceutical industry, he ended up in the tech, founding his own company building intelligent chat agents for business analytics.
The major theme of his research is the enablement of human users to communicate additional information to the robot learning system through ‘concept constraints’. Concept Constraints are abstract behavioral restrictions grounded as geometric and kinodynamical planning predicates that prohibit or limit the behavior of the robot resulting in more robust, generalizable, and safe skill execution. In this talk, Carl discussed how conceptual constraints are integrated into existing LfD methods, how unique interfaces can further enhance the communication of such constraints, and how the grounding of these constraints requires constrained motion planning techniques.
For more information about the speaker and related papers, please see this website.
Daniel Rakita “Methods and Applications for Generating Accurate and Feasible Robot-arm Motions in Real-time”
Daniel Rakita is a Ph.D. student of computer science at the University of Wisconsin-Madison advised by Michael Gleicher and Bilge Mutlu. He received a Bachelors of Music Performance from the Indiana University Jacobs School of Music in 2012.
In this talk, he overviewed technical methods they have developed that attempt to achieve feasible, accurate, and time-sensitive robot-arm motions. In particular, he detailed their inverse kinematics solver called RelaxedIK that utilizes both non-linear optimization and machine learning to achieve a smooth, feasible, and accurate end-effector to joint-space mapping on-the-fly. He highlighted numerous ways they have applied their technical methods to real-world-inspired problems, such as mapping human-arm-motion to robot-arm-motion in real-time to afford effective shared-control interfaces and automatically moving a camera-in-hand robot in a remote setting to optimize a viewpoint for a teleoperator.
For more information about the speaker and related papers, please see this website.
Barbara Bruno “Culture-Aware Robotics”
Barbara Bruno is a post-doc researcher at the École Polytechnique Fédérale de Lausanne (EPFL), in Lausanne, Switzerland, in the CHILI lab. Barbara received the M.Sc. and the Ph.D. in Robotics from the University of Genoa in 2011 and 2015, respectively. She is part of the NCCR Robotics organisation and currently involved in the EU ITN ANIMATAS.
In this talk, she explored how existing quantitative and qualitative methods for the assessment of culture, and cultural differences, can be combined with knowledge representation and reasoning tools such as ontologies and fuzzy controllers to endow robots with the capability of taking cultural factors into account in a range of tasks going from low-level motion planning to high-level dialogue management and user adaptation.
For more information about the speaker and related papers, please see this website.
Because as they say, we are human after all!
Nils Hagberg “A call for a Human Approach to Technology”
Nils Hagberg is the Product Owner at Furhat Robotics. He is a computer linguist with almost ten years of industry experience in human-machine interaction and conversational technology.
In this talk, he gave a few examples of human-centric business case designs that he’s come across earlier in his career and at Furhat Robotics – the social robotics company that have set out to make technology more human. His hope for this talk is that it will put your own work in a larger context and nudge you towards a path that will ensure humans will be allowed to remain human.
For more information about the speaker and related papers, please see this website.
iRobot Corp. unveiled new coding resources through iRobot Education that promote more inclusive, equitable access to STEM education and support social-emotional development. iRobot also updated its iRobot Coding App with the introduction of Python coding support and a new 3D Root coding robot simulator environment that is ideal for hybrid and remote learning landscapes.
The updates coincide with the annual National Robotics Week, a time when kids, parents and teachers across the nation tap into the excitement of robotics for STEM learning.
Supporting Social and Emotional Learning
The events of the past year changed the traditional learning environment with students, families and educators adapting to hybrid and remote classrooms. Conversations on the critical importance of diversity, equity and inclusion have also taken on increased importance in the classroom. To address this, iRobot Education has introduced social and emotional learning (SEL) lessons to its Learning Library that tie SEL competencies, like peer interaction and responsible decision-making, into coding and STEM curriculum. These SEL learning lessons, such as The Kind Playground, Seeing the Whole Picture and Navigating Conversations, provide educators with new resources that help students build emotional intelligence and become responsible global citizens, through a STEM lens.
Language translations for iRobot Coding App
More students can now enjoy the free iRobot Coding App with the introduction of Spanish, French, German, Czech and Japanese language support. iRobot’s mobile and web coding app offers three progressively challenging levels of coding language that advances users from graphical coding to hybrid coding, followed by full-text coding. Globally, users can now translate graphical and hybrid block coding levels into their preferred language, helping beginners and experts alike hone their language and computational thinking skills.
Introducing Python coding language support
One of the most popular coding languages, Python is now available to iRobot Coding App users in level 3, full-text coding. This new functionality provides an avenue to gain more complex coding experience in a coding language that is currently used in both academic and professional capacities worldwide, preparing the next generation of students for STEM curriculums and careers.
New Root Coding Robot 3D Simulator
Ready to code in 3D? The iRobot Coding App is uniquely designed to help kids learn coding at home and in school, with a virtual Root coding robot available for free within the app. iRobot updated the virtual Root SimBot with a fun and interactive 3D experience, allowing students to control their programmable Root® coding robot right on the screen.
“The COVID-19 pandemic has had, and continues to have, a tangible impact on students who’ve been learning in remote environments, which is why we identified solutions to nurture and grow SEL skills in students,” said Colin Angle, chairman and CEO of iRobot. “The expansion of these new iRobot Education resources, which are free to anyone, will hopefully facilitate greater inclusivity and accessibility for those who want to grow their coding experience and pursue STEM careers.”
In celebration of National Robotics Week, iRobot Education will also release weekly coding challenges focused on learning how to use code to communicate. Each challenge can be completed online in the iRobot Coding App with the Root SimBot or in-person with a Root coding robot. The weekly challenges build upon each other and include guided questions to facilitate discussions about the coding process, invite reflections, and celebrate new learning.
For more information on iRobot Education, Root coding robots and the iRobot Coding App, visit: https://edu.irobot.com/.
Information about National Robotics Week can be found at www.nationalroboticsweek.org.