All posts by Ella Scallan

From the 1st-5th June, the robots descended on Vienna. The 2026 IEEE International Conference on Robotics & Automation (ICRA) brought together the top minds in robotics for one short week to showcase the latest technologies, form new collaborations, and exchange ideas. Held at the Messe Wien, a stone’s throw from the bank of the Danube, ICRA proved to be equal parts technological marvel and thought-provoking discussion. 

The host venue for ICRA 2026: Messe Wien, also known as VIECON.

Workshop on robot ethics

My week at ICRA began with the 2nd ICRA 2026 Workshop on Robot Ethics: Ethical, Legal and User Perspectives in Robotics & Automation (WOROBET). WOROBET provided a space for researchers to share ideas, thoughts, and concerns on the future of robot-human interaction, and how to create ethical frameworks to navigate this rapidly changing technology. 

Yasuhisa Hirata, Professor at Tohoku University, began by presenting his vision of a world with physically assistive robots, such as detachable exoskeletons or cycling wheelchairs. These tools can affect people’s sense of self-efficacy and motivation, and there are a host of ethical implications that come with this – how do you help people just enough to build their confidence, without slipping into deception? 

We then heard from Prof. Minoru Asada from Osaka University, who discussed his aim to implement pain signals into robots, so they can experience the world as we do. This was a highly interesting and niche proposition that brought up more ethical questions than we have answers for at the moment. Perhaps most pertinent to a technical conference: is a sense of embodied morality necessary for true intelligence? 

Alan Winfield, Professor of Robot Ethics at UWE Bristol, presented a vision of robotics that acted as a counterweight to Prof. Asada’s: robots as tools, not potential beings. This also somewhat reflects differing attitudes in Eastern vs Western cultures. Thinking about the practical issues we are likely to face in the near term, he outlined a framework for social robot accident investigation. In his view, robot ethics is not just an engineering problem, but requires appropriate social and governance frameworks, just as we do for aviation. His talk also emphasised the risk that programming ethics into robots runs the risk of removing moral responsibility from the roboticist, and can always give rise to unethical robots via malicious hacking. The theme that we must focus on human morality, as opposed to machine morality, was repeated throughout the day. 

After a morning of differing ideas and visions of what robot-human interaction could be, we were invited to ground this into a real robot social care scenario by Praminda Caleb-Solly, Professor of Embodied Intelligence at the University of Nottingham. In this red-teaming exercise, we examined the safety risks and possible mitigations of an assistive robot for a schoolteacher recovering from a stroke at home. Our group discussion circled around human agency: how ethical is it to make design choices for people that take away some of their autonomy, in the name of their best interests? As robots in social care will become a more urgent need in the years to come, these questions may become more salient. 

I left WOROBET with plenty to think about, and a renewed sense of appreciation for the ethicists who are already grappling with the problems that are to come. I hope that progress in robot ethics keeps pace with progress in robotics, so that we are well prepared as robots become more of a part of our daily lives.

Welcome to the jungle – the robot exhibition floor

Every time you entered the exhibition hall, you were greeted by one of the child-sized Booster robots, either playing football, dancing, or demonstrating some kung fu. They were always an endearing welcome to the sea of robots. 

The robots ranged from the endearing to the uncanny, but the common thread was their technical capabilities were astounding. Veteran attendees consistently remarked on how much the robots had improved year on year.

The cute.

The slightly uncanny.

The below clips show the robots that most caught my eye. After admiring a phosphorescent, Stranger Things-esque robotic flower display, I was blown away by the D1-modular robot from Direct Drive. Unlike most robot dogs, it can split into two halves, with the ability to jump, twist, and traverse difficult terrain. 

Sharpa’s North was always a friendly face, waving and making love-hearts at visitors to its booth. Around the back of the booth, you could even challenge it to a round at blackjack. We saw humanoids zipping up rucksacks and trying to fold laundry. Enchanted Tools’ social care robot, Mirokaï, was an unusual sight among the mass of black and steel, with a bright orange body, feline ears and an orange, furry face. Tesollo’s humanoid spent much of its time at ICRA using its long, wavering arms to pick up fruit and drop it into baskets, with impressive dexterity. Vietnamese company Vinrobotics’ humanoid offering was reminiscent of the Cybermen, with its gently wheezing joints, but the team assured me it was much friendlier. The pint-sized Boosters were almost always playing football, not far from their similarly sized Agibot cousins. 

However, humanoids didn’t steal the show this year, as they have done previously. The big trend this year was robotic hands, and the levels of dexterity were truly impressive. Closing the gap between human and robot abilities here would unlock whole new swathes of tasks to automate – and who wouldn’t want a robot folding their laundry? 

Industrial challenges: solving dexterity 

Tackling the dexterity challenge really defined the industrial talks for me this year. A talk by ARIA’s program director, Prof. Jenny Read, demonstrated how the UK government is already laying the groundwork here.

They outlined their funding proposals as part of their Smarter Robot Bodies program, which is split into two branches: robot locomotion and robot dexterity. The robot locomotion branch aims to enable robots to traverse messy, unpredictable physical environments, while the robot dexterity branch will try to break the bottleneck of adept physical manipulation by robotic hands. With the programme set to launch in early 2027, it’ll be exciting to see what kind of innovations this attracts. 

One standout innovation in the realm of dexterity was TARS, a record-setting newcomer in the Chinese robotics market. Co-founded by Dr Ding Wenchao just 18 months ago, TARS has already raised the most funding in angel and pre-seed rounds of any company in the Chinese embodied intelligence sector, and achieved a Guinness World Record for robotic flexible robotic flexible wiring-harness insertion completed in one hour. 

TARS’ DexHand. Image credits: TARS.

DexHand is a 1:1 model of the human hand, even replicating the 21 degrees of freedom we have in the wrist joint and hand. According to TARS, “it can interpret tactile data to distinguish slipperiness, roughness, and hardness in real time and perform 26 English alphabet hand gestures with high-precision finger control.” I was given the chance to control the hand using my own, and was impressed by how well it emulated my own movements. I was also impressed by their humanoid zipping up a backpack – despite the technical abilities of all the robots on the floor, few were able to perform tasks which required such fine motor skills. 

TARS robot zipping up a rucksack.

On Thursday, Dr Wenchao Ding delivered an industry keynote where he presented TARS’ roadmap, charting the path from academia to industrial deployment. With an impressive academic team behind them, TARS may be one to watch. 

Plenary talks

Aside from the wealth of invention and innovation taking place in the exhibition hall, the breadth and depth of academic research at ICRA was fantastic. The plenaries especially gave an insight into the research trends that are currently defining the field. 

Ken Goldberg delivered an electrifying plenary, titled “A Tale of Two Cultures:  Can Agentic Coding Close the Gap?” In this talk, he called for a step change to close the data gap faced by robot manipulation. With the rise of diffusion models and LLMs, it is clear that big data has solved computer vision and language. He challenged the audience – when will the ChatGPT moment for robotics come? With state spaces larger than 50 dimensions in robotics, there is not enough training data to close this gap. Currently, the data required to train vision-language models is equivalent to 100,000 years of real physical experience. 

According to Goldberg, the 2 dominant cultures in engineering – model free “good old fashioned” engineering (GOFE), and, the currently more popular model based engineering.  GOFE encapsulates rigorous engineering methods that pre-date AI, but may have been slightly forgotten about in the AI wave of recent years. He also highlighted how, in his career, he has always been working to bridge the gap between 2 cultures: from science and art; to robotics and automation. 

He outlined 4 possible solutions to the data gap:

1. Simulations – these work incredibly well for locomotion and body control, but less so for manipulation due to the number of forces and instabilities involved.
2. World models – they do not currently properly capture the physics, and hallucinations can be problematic. 
3. Human teleoperation – this is currently big business and a good way to obtain high quality data. However, the largest dataset is currently only equivalent to a year’s worth of data. 
4. Real data from functioning robots – this is less commonly used, but can be powerful. 

Prof Goldberg described how he used the fourth approach in his robotic delivery packing company, Ambi Robotics, for 22 years. Picking up bags is an example of variational automation –  one task is done repeatedly, but with different initial conditions each time. From this rich dataset, they created a generative model to train robots in the best way to pick up bags. Here, they close the gap between model-free and model-based methods – Ambi exploits both to achieve industry-leading results. This plenary served as a call for other researchers to use their own production data to do the same. 

During Thursday’s keynote on Robot Learning, Planning & Foundation Models, Stefanie Tellex from Brown University gave a compelling talk titled “Towards Complex Language in Partially Observed Environments”. While current research is bounded in known, predictable scenarios, using action-based language, this does not reflect what the real world is like, nor how people would naturally communicate with robots. Prof. Tellex described her work creating robots that can understand complex, goal-based commands in only  partially observed, dynamic environments, and outlined the grounded Turing test – a reimagining of the Turing test for embodied AI. 

An example of a robot performing a goal-based task in a dynamic environment. Credits: Tellex et al, 2026.

Both plenaries spoke to current pinch points in robotics: data, reasoning, and operating in complex real-world environments. It’ll be interesting to see what solutions are developed in the coming years. 

Science communications crash course

One of my favourite parts of ICRA was delivering the Science Communications Crash Course. Along with Robohub Executive Trustee Sabine Hauert, IEEE Spectrum Senior Editor Evan Ackerman, and IEEE Spectrum Community Manager Kohava Mendelsohn, we gave our guidance on effective science communication to an audience of 100 academics. It was encouraging to see so many people interested in communicating their research effectively – it is a crucial skill, especially in the era of AI and robotics when mainstream narratives can be hijacked by doom-mongering, hype, and corporate interests. More academics communicating their work clearly and neutrally will go a long way to grounding our societal discussion in technical reality, not sci-fi futures. 

Sabine kicking off the science communications crash course. Image credits: Taraja Arnold

Delivering my part of the course. Image credits: Taraja Arnold

Art and robotics 

The arts and robotics section was rich and interesting. There was a lot to visually take in, with constant background music from a robotic saxophone. 

Masatoshi Hamanaka’s robotic saxophone. Image credits: ©Denes Erdos – Your Event Photographer

PET – marked by a large “PET ME” sign – was a white and orange mass of connecting, rotating pyramids, that gently pulsed and hummed in response to touch, responding by curling towards or away from you depending on how you touched it, The effect was strangely lifelike. 

Rhombus Research presented “Reptile: A Bio-Mimetic Choreography Engine for V2X and A2X Swarms”. This artistic simulation visualises the contracts negotiated within autonomous vehicle and swarm fleets in dreamy blue, browser-based visualisation. Performance artist and former Cirque de Soleil acrobat Silke Grabinger explored human-robot interaction via her piece, AREYOUARE. 

Silke Grabinger performing AREYOUARE. Image credits: ©Denes Erdos – Your Event Photographer

I’d also like to acknowledge some of the video creators I met there. YouTubers Back to Engineering and the. Amazing, PhD are making some fantastic videos about physical AI. 

Nothing lasts forever – ending with the robot parade 

ICRA 2026 ended with the robot parade, which attracted quite a crowd. See if you can spot the panda, dragon, and headless humanoid!

Seeing all the robots gathered together was a real spectacle. The technology on display was state-of-the-art, and it only improves year on year. What struck me most was the sense that robotics is moving from proving what is possible to tackling the remaining barriers to real-world deployment. Across the exhibition floor, industry keynotes, and plenary talks, the focus was often on the same challenges: dexterity, data, and operating reliably in complex environments. Solving these bottlenecks will open up new avenues to real-world applications of robots. 

While workshops such as WOROBET highlighted important questions around ethics, agency, and governance, the overwhelming emphasis at ICRA 2026 was on capability. Researchers and companies alike are working to close the gap between what robots can do in carefully controlled demonstrations and what they can do in the messy reality of the world outside the lab. Judging by the pace of progress on display in Vienna, that gap may be narrowing faster than many of us expected. I hope that the kinds of conversations around human-robot interaction and ethics that were commonplace at WOROBET and in the Arts exhibition space will become more mainstream – we may need to face the questions that they raise sooner than we think. 

Note: Where image and video credits are not stated, they belong to Ella Scallan.

AIhub is excited to launch a new series, speaking with leading researchers to explore the breakthroughs driving AI and the reality of the future promises – to give you an inside perspective on the headlines. The first interviewee is Ross King, who created the first robot scientist back in 2009. He spoke to us about the nature of scientific discovery, the role AI has to play, and his recent work in DNA computing.

Automated science is a really exciting area, and it feels like everyone’s talking about it at the moment – e.g. AlphaFold sharing the 2024 Nobel Prize. But you’ve been working in this field for many years now. In 2009 you developed Adam, the first robot scientist to generate novel scientific knowledge. Could you tell me some more about that?

So the history goes back to before Adam. Back in the late 1990s, I moved from a postdoc at what was then the Imperial Cancer Research Fund – now Cancer Research UK – and got my first academic job at the University of Wales, Aberystwyth. That’s where I had the original idea of trying to automate scientific research.

Our first publication on this was in 2004. It was a paper about robot scientists, published in Nature. That was the start. We showed that the different steps in the scientific method – forming hypotheses, determining experiments to test them, analysis of the results – could all be individually automated. But the whole cycle wasn’t fully automated, and the AI system didn’t do any novel science at that point.

In 2009, we built the Adam system. Adam was a (physically) large laboratory automation system, combined with AI that could perform full cycles of scientific research, and had knowledge about yeast functional genomics. Adam hypothesised and experimentally confirmed novel scientific knowledge about yeast metabolism, which we manually verified in the lab. 

How has the field evolved since then?

For many years, not much happened. Funding was difficult due to the financial crisis, which made the British Research Councils much more conservative. Before that period, panels would choose the most exciting science. Afterwards, they focused more on what would help Britain financially in the near term.

We couldn’t get funding for many years, and few others were interested. There was some work in symbolic regression – finding interpretable mathematical models to fit phenomena – but not much automation of science. What changed was the general rise of AI. As AI became more prominent, interest picked up, especially after 2017.

What are the potential upsides and downsides of AI scientists? 

I’ll start with the big picture: I think that science is positive for humanity. I think our lives in the 21st century are better than those of kings and queens in the 17th century, when modern science started. We have better food from around the world, beautiful fruits for breakfast, and much better healthcare – a 17th-century dentist was not pleasant. My mobile phone can communicate with billions of people at the touch of a button, and I can fly around the world. These are unbelievably good standards of living for billions of people, not just elites. The application of science to technology has provided this.  Of course there are downsides – pollution, environmental damage – but generally, for humans, I think life is better than in the 17th century. 

However, we still have huge problems. We can’t stop global warming or many diseases, and a billion people still live with food insecurity. I think we have sufficient technology to solve these problems if the nations of the world collaborated and shared resources. But I see no prospect of that happening in the current world situation, and I see no examples from history where these things have happened. So my only hope is that science becomes more efficient. If AI can help achieve that, then perhaps we can overcome these challenges. If we have better technology and we treat people badly after that, then it’s not down to constraints in the world, it’s down to human beings. 

As for having AI scientists as colleagues: AI systems don’t understand the big picture. They can’t do really clever things, like Einstein seeing space and time as a four-dimensional continuum as opposed to quite separate things. If you read the 1905 paper by Einstein, it starts off with this philosophical problem about electricity and magnets – AI systems are nowhere near as clever as being able to do anything like that. They can’t see deep analogies or connections, but they are brilliant at other parts of science. They can literally read everything – they have read every paper in the world 1000 times. If you have a small amount of data, machine learning systems can analyze it better than humans would. In this sense, they have superhuman powers. 

One interesting thing now is that if you’re a working scientist and you’re not using AI, in almost all fields you’re not going to be competitive anymore. AI on its own is not better than humans – yet. But a human plus AI is better than a human alone. Human scientists need to embrace AI and use it to do better science.

Do you think we’ll reach a point where autonomous AI will be able to generate the research questions and direct the movement of research?

Yes, I think so, although we’re not close to that at the moment. They can generate new ideas in constrained spaces, often better than humans, but they don’t really have the big picture yet. 

I think that will come sooner or later. I’m involved in a project called the Nobel Turing Challenge. The goal of that is to build an AI robotic system able to do autonomous science at the level of a Nobel Prize winner, by the year 2050. And if you can do that, we can build two machines, a hundred machines, a million machines – and we’d transform society.

Do you think that’s feasible by 2050? 

Just before the pandemic and during the pandemic, I thought the probability of hitting that target was dropping. But then there was the breakthrough of large language models, which are amazing in many ways – often remarkably stupid too, but generally very clever. I think that they alone will not be enough to beat the Nobel Turing Challenge, but I think they’ve made the probability of hitting that target much more likely.

What is interesting – and I don’t know the answer to this – is whether you need to solve AI in general to solve science, or whether it’s more like chess, where you can build a special machine which is genius at chess but not anything else. Imagine some machine which is a genius at physics but doesn’t know anything about poetry or history. Would that be enough? 

My instinct would be to say that it’s not, because everything’s so interlinked – poetry has rhythm, music contains mathematical structures. I think an AI scientist would need a broader understanding of reality than just its specific domain. 

People used to think that we needed those things to solve chess, so our human intuition is not very good at these things. For example, I didn’t expect LLMs to work so well, just by building a bigger network and putting in more data. I assumed they’d need some deep internal model of the world, or even that they would need a body to really understand how things move around in the world.

LLMs raise some interesting questions – are they just mimicking intelligence, as they lack internal models? 

I think AI must have, in some sense, some internal model inside. It’s just we don’t really understand why they work. It’s purely empirical, which is very unusual. I don’t remember a case where we have such an important technology, but we have so little understanding of it.

It is quite mysterious. Especially because science is always asking “what’s the mechanism?”  With AI, it’s the opposite. The question is “does it work?” We don’t know what the mechanism is. 

It’s not even clear what the theory to explain it is. Coming from machine learning, I assumed it would be some sort of Bayesian inference or something. But the mathematicians say no, it’s all to do with function mapping in some high dimensional space. These don’t seem to be the same, so it’s not even clear what framework we should use to explain it. 

And, mapping in a high dimensional space is something that’s fundamentally not intuitively understandable to humans. 

Yes, so it’s a mystery. So why do they do so well, and why do they not overfit over so many parameters. How do they manage to come to a reasonable answer? Generally, it’s easy to understand why they make mistakes, but it’s not so easy to understand why they actually work so well. 

Can you speak about your work in DNA computing, and how it relates to automated science?

With automated science, we’re using computer science to understand, for instance, biology or chemistry. With DNA computing we’re using technology from biology and chemistry to improve computer science. With DNA, you have the potential to have many, many orders of magnitude greater computing density than with electronics. This is because the bases in DNA are roughly the same size as the smallest transistors, but you can pack DNA in three dimensions, whereas transistors can only be in two dimensions. In our design for DNA, every DNA strand is a tiny computer. 

And the beautiful thing with DNA is that it can replicate itself – nature has made ways of copying DNA which are very effective. That’s how we as humans and all animals and plants and bacteria replicate, whereas electronic computers don’t replicate themselves – they’re built in factories costing billions. We can piggyback on top of this wonderful technology which nature has given us.

How does a DNA computer work? 

One of the greatest discoveries ever made was by Alan Turing, who discovered, or invented, the concept of the universal Turing machine. So this is an abstract mathematical object which can essentially compute anything which any other computer can compute. You can’t make a more powerful computer, in the sense that it can compute a function which that universal Turing machine can’t compute.

And there’s many different ways of physically implementing a universal Turing machine. The most common one is to build an electronic computer. But you could, in principle, build a Turing machine out of tin cans, for instance – the only difference is how fast they go and how much memory they have. The reason that your computer can do multiple tasks is because it can be programmed to do.

The beautiful thing which you can do with DNA is you can make a non deterministic universal Turing machine. These compute the same functions as normal universal Turing machines, but they do so exponentially faster – every time there is a decision point in the program, rather than having to explore only one path, it can go both ways simultaneously. So you can make a computer which, like an organism (think rabbits), can replicate and replicate and replicate until we solve the problem, or you run out of space. So space becomes the limiting factor rather than time. 

You can imagine that if you wanted to search through a tree to find something, you could put down all the branches in parallel, whereas a normal computer would go down one branch at a time. If you do the sums for DNA computing, you could have more memory and more compute on a desktop than all the electronic computers on the planet, which seems incredible. That’s just because of the density of compute. 

That would be an incredible scale-up – like how a modern smartphone is so  much more powerful than NASA’s supercomputers in the 60s. But computing isn’t improving at the same rate as it used to. 

Yes. Computers are not improving like they used to for many decades (Moore’s law). That’s why these big tech companies are building big compute farms the size of Manhattan or soon maybe Texas. So the world does need more efficient ways of doing compute.

If we had a lot of compute, what kinds of scientific problems or areas do you think AI-enabled science could best be applied to? Are there any low-hanging fruits?

What’s very important is to integrate AI systems with actual experiments and laboratories. You can’t just think about science and get the right answer. We need to actually go into the labs and test things, but a lot of AI people and AI companies don’t really appreciate that. They’ve been so successful in science with AI plus simulation that they don’t realize simulation is only so good as something that’s testable.

Areas with low-hanging fruit include materials science, as we need better battery materials, better solar panels, and lots more. There’s something of a gold rush happening there right now, with many startup companies getting huge valuations.

The other area of automation, which is in some sense easier, is drug design, because it’s much easier to move liquids around than solid phase materials. Closed-loop automation has sort of transformed early-stage drug design, and there are lots of companies in that space now.

The big picture is that the economic cost of science is dropping. A lot of the actual thinking involved in science can now be done by AI systems, and the experimental work can be done very well by lab automation. You don’t need to employ people to move things around, and people aren’t as accurate and don’t record things as well as automation does. So that’s the big picture: what can we do if we can make science much cheaper?

Where do you think AI science is headed next?

I think there’s an analogy with computer games like chess and Go. In my lifetime, computers went from playing chess pretty poorly to being able to beat the world champion. I think it’s the same in science. There’s a continuum of ability from what current technology can do, from the average human, to grandmasters of science like Newton, Einstein, Darwin and others. If you agree there is no sharp cutoff on that path, then I think that with faster computers, better algorithms, and better data, there’s nothing stopping them getting better and better at science. Whereas there’s evidence that humans are getting worse at science – the average economic benefit per scientist is decreasing. I think they’ll get better and better and sooner or later overtake humans in science. We shall see, but I’m optimistic. If we get through this period, better science can improve the standard of living and happiness of humanity,  and save the planet at the same time.

And now we have so much data, we need that raw power and intelligence to look at it all.

Yes, we need factories doing a lot of automation to scale things up. There’s no point in AI having brilliant ideas if we can’t test them in the lab. In my mind, science is still at the pre-industrial level. A PI with some post-docs and a few students is like a cottage industry, as opposed to a factory of science. I think humans will still be doing science, but we won’t be actually pipetting things in the future. It’s one reason we chose the name Adam (Adam Smith), we want to change the economics of science. 

And Eve?

Eve was a system we developed some years ago to look at early-stage drug design. Eve optimises a process, rather than doing pure science. Most systems don’t actually do hypothesis-driven science, they optimise something, e.g. find a better material for batteries, which is useful, but not necessarily science. 

Our new system is called Genesis. There we’re trying to scale up the experiments we can do and build up a lot of data. We’re using a continuous flow bioreactor, which enables you to control the growth rate of microorganisms. This is important if you want to understand their internal workings.

And you’re beginning with microorganisms because they’re a fundamental unit of life? 

Yes, we want to understand the eukaryotic cells. There are three branches of life, and the other two are bacteria. Eukaryotes evolved more than 1 billion years ago. We are eukaryotes. Biology is conservative, so the design of yeast and human cells is pretty much the same, but yeast cells are much simpler than human ones. To understand how we work, first we need to understand yeast, then human cells. Once we understand how human cells work, we can understand how organs work, then how humans work, and then we can solve medicine. It’s a reductionist approach to science – we understand something simple first, and then build from there. 

I like the progression, that approach makes sense. 

Unfortunately, it doesn’t make sense to our funders. They generally want to fund practical work on human cells now. They don’t easily fund research on fundamental questions. 

That’s the problem with the funding system. Most great discoveries in science over the last few centuries would not have been funded – they happened because people were doing the most impractical things for the most impractical reasons. And maybe a century later they were found to have a practical purpose. 

Exactly. Some years ago in the UK you had to write a 2-pages for every Research Council grant on how your research was going to make Britain richer or healthier. What would Alan Turing have written on his grant application for the Entscheidungsproblem? 

Thank you. This has been a very interesting conversation.

Thank you, happy to discuss this. It’s a very interesting topic. 

About Ross King

Ross King is a Professor with joint positions at the University of Cambridge, and Chalmers Institute of Technology, Sweden. He originated the idea of a ‘Robot Scientist’: integrating AI and laboratory robotics to physically implement scientific discovery. His research has been published in top scientific journals – Science, Nature, etc. – and received wide publicity. His other core research interest is DNA computing. He developed the first nondeterministic universal Turing machine, and is now working on a DNA computer that can solve larger NP complete problems than conventional or quantum computers.

Illustration of the radiotherapy room and the occlusion problem faced by ceiling-mounted cameras in this application.

What was the topic of your PhD research and why was it an interesting area?

My topic of research was developing an optical tactile sensor to track head motion during radiotherapy. I worked on both the hardware and software development of this sensor, though my focus was mostly on the software side. Its importance comes from the fact that during radiotherapy, patients undergoing head and neck cancer treatment are typically immobilised. This is usually done using a thermoplastic mask, which can feel very claustrophobic, or a stereotactic frame. Frames are more common for brain cancers, but they have to be surgically inserted into the patient’s skull using pins. Either of these immobilisation tools may be used depending on the situation. When patients are uncomfortable, they are more likely to move, which affects the accuracy of treatment, especially with thermoplastic masks.

Another major issue is that current systems use ceiling-mounted cameras to record patient motion. These cameras cannot be placed too close to the patient because of the electromagnetic environment around the equipment. Their view is also frequently occluded because the patient moves into a tunnel to receive the ionising beams, which makes it difficult to capture rotational motion.

One alternative is an infrared camera with a nose marker, but this only captures translational motion. Currently, when a nose tracker detects movement beyond a certain threshold, treatment is paused, the patient is repositioned, and treatment resumes. It is difficult to adapt this system to reliably measure the rotational motion of the patient’s head in the radiotherapy environment.

This is where the Motion Capture Pillow (MCP) comes in, which contains the optical tactile sensor I developed. The goal with this system is similar to the nose tracker, but with more accurate rotational feedback for the radiographer. It can be placed beneath the patient’s head and attached to the treatment bed. It estimates how much the patient’s neck is rotating and improves patient comfort. Radiographers can receive real-time feedback on both translational and rotational movement. The advantages of this system are that there are no occlusions, because the pillow is in direct contact with the patient’s head, and it is more compatible with radiotherapy environments because the sensor is non-ferromagnetic. Its premise is to maintain patient comfort, stay compact and easy to integrate into the pre-existing systems for radiotherapy, whilst improving the accuracy of the treatment through real-time head tracking.

Labelled diagram of the Motion Capture Pillow – Optical tactile sensor for head tracking during radiotherapy. The pneumatic pillow is a deformable rubber-like sheet with embedded white markers, held in its convex shape using air pressure. The fibrescope represents a non-ferromagnetic fibre optic bundle used as a lens extension to an area scan camera. The camera is ferromagnetic and will require safe positioning and fixation.

What were the main contributions of your work?

There were four main contributions to my work. The first contribution focused on making the system more non-ferromagnetic and improving the imaging and tracking approach. Previous work used a webcam and binary image processing within the optical tactile sensor to track marker displacement. I ultimately decided to use a fibrescope, optical flow tracking algorithm, and grayscale imaging instead, which improved the sensor’s tracking ability.

The second contribution focused on optimising marker density. The optical tactile sensor consists of an array of markers on a deformable rubber-like sheet, resembling a pillow. The deformation of these markers is captured by the camera. I investigated how dense the marker array needed to be by adjusting the spacing between markers to determine what worked best for this application.

The third contribution involved sensor fusion to improve reliability and robustness. To do this, I integrated a gyroscope and used Kalman filtering to fuse data from the gyroscope and the MCP. This was important for Gamma Knife systems, which are radiosurgery platforms used for brain cancers. They tend to have higher accuracy requirements than linear accelerators, which are commonly used for head and neck cancers, and lower constraints on the use of ferromagnetic components.

The final contribution was a participatory design study conducted in collaboration with clinicians and the social sciences department. We explored how the MCP could be integrated into hospital workflows and assessed its feasibility.

How feasible is it to integrate this sensor into hospital workflows?

Clinicians did seem to be very on board with it, but the study was more qualitative than quantitative. While they felt the idea had merit, there were reservations about adopting new technology and the associated learning curve.

They were also concerned about accuracy. Improving accuracy and reliability is essential for clinicians to feel confident using the system. At present, further development is needed before it can be widely implemented.

What future work is planned in this area?

One area to investigate is the differences between the mannequin and participant data. The pillow shape is controlled by a pneumatic system with a pressure sensor and air pump. When the patient or mannequin moves, pressure changes occur. The system compensates to maintain a set pressure, but this introduces errors in the motion readings. The mannequin produced more errors than the participant data. It may not accurately simulate human motion on the pillow, and the testing setup may introduce discrepancies that do not reflect real-world behaviour.
So, future work includes stabilising and refining the pressure control system to improve reliability. If necessary, reconsidering the use of gel on the sensors could be an option. Gel had been used previously but was abandoned due to clinician concerns about attenuation of ionising beams. However, if avoiding gel significantly compromises sensor performance, revisiting this approach may be worthwhile.

In addition, more participant data collection is needed. Not all previously collected data could be used due to ground-truth measurements being partially occluded in the experimental setup. Additional participant studies would provide a clearer understanding of performance across different individuals. Another priority is improving the fibrescope’s resolution and angle to better visualise high-density marker arrays. Hardware upgrades would help ensure a clearer field of view and improve overall system performance.

About Bhoomika

Bhoomika Gandhi is a recent PhD graduate from the University of Sheffield Medical Robotics group. Her undergraduate degree was in Bioengineering – Medical Devices and Instruments, with control engineering and robotics being the key themes.