Everyone claims AI governance now. 15 questions to test the claim.
“Governance” has become a fixture in enterprise AI announcements. Identity, security, orchestration, infrastructure, and data platform vendors offer products for agent safety, governance, and control, bringing welcome attention to a problem enterprises need to solve. As the claims grow broader, CIOs and risk leaders need to get more precise about what they are buying.
Look closely at those offerings and the capabilities differ. Some restrict an agent’s access, test how an agent behaves before launch, or collect records for an audit. Your exposure depends on how those capabilities connect and where their coverage ends. A vendor evaluation needs to show whether the controls cover your agents, apply your business rules, and keep working as both change.
One floor, no loop
Controls within that stack test behavior, restrict access, stop activity, and document outcomes. Each serves a necessary purpose, and each has boundaries that a governance claim should make clear.
The pattern to watch for is one floor and no loop: a useful capability presented as complete governance, with little clarity about what happens after deployment. A stack without a lifecycle is incomplete. Buyers need evidence that the controls will remain effective as agents, rules, and environments change.
Two gaps deserve particular attention.
- The first is a missing foundation: an incomplete view of which agents exist and what they can access. A platform might tightly control its registered agents while leaving agents that were never registered ungoverned. Those unknown agents can expose customer data without ever encountering the controls you evaluated.
- The second is a missing lifecycle. After an agent passes its launch review, someone may connect another tool, replace its model, or expand its permissions. Each change can affect behavior and protection. An unchanged approval says little about the agent’s current risk.
The lifecycle gap cuts both ways. A restriction introduced without testing can disrupt legitimate work, prompting a team to disable it to restore service. Governance must account for that pressure through testing, authorized changes, and checks that the protection still works.
A foundation and a continuous lifecycle
The foundation begins with a registry: a live record of every agent and what it is made of, including its model, tools, accounts it acts through, and data it can reach. When a sales agent gains access to another customer database, its owner should see the change and review the permissions involved.
Each agent also needs a distinct identity so investigators can trace its actions separately from its builder’s activity. With that foundation in place, governance becomes a loop that runs for as long as the agent does.
The lifecycle connects five stages throughout the agent’s use:
- Define sets the rules for the agent’s work, drawing on existing business obligations and risk decisions. For the sales agent, those rules specify which customer records it can use, which services it can contact, and which actions require approval.
- Test checks the agent against those rules before release and after relevant changes. A test should confirm that the sales agent can complete an approved customer request while resisting instructions to send protected records elsewhere.
- Enforce applies the rules while the agent works. A monitoring tool records a prohibited transfer and alerts security after the fact. An enforcement control prevents the transfer through the controlled route. Buyers need to know which actions the product can prevent and which it can only report.
- Prove preserves evidence of what happened and why. For the attempted transfer, that means identifying the agent, the rule that blocked it, and any approval or override that affected the decision.
- Adapt updates the controls as agents, requirements, and environments change. If the sales agent gains a new outside tool, its permissions need review and its protections need testing again. That change brings the process back through the loop.

15 questions to ask any AI governance vendor
These questions cover the full governance lifecycle, from the foundation that makes agents visible to the process that keeps them governed. No single vendor answers all 15 today, and any vendor that claims to is defining governance narrowly or selling you a roadmap. Your job isn’t to find a perfect score. It’s to evaluate coverage, direction, and architectural fit against your operating requirements.
Use these questions with every vendor. Ground the answers in your workflows and operating environments. Record what works today, what requires additional tools, and what your team will own after deployment. The answers labeled “flag” identify claims that deserve a closer look.
Foundation: agent registry and identity
1. Can you show us every agent we run, what each is made of, and when its ingredients change?
An agent can gain access to sensitive customer data after its original approval, leaving the business exposed to a change nobody reviewed. Ask for a registry that tracks models, tools, data access, and accounts, with changes reported to the responsible owner. Buyers may hear the ingredient list called an AI bill of materials (AI-BOM). Ask how agents built elsewhere enter the registry and how unregistered agents are found.
Flag: The vendor claims visibility into all your agents but is only describing agents built on their own platform.
2. Does each agent have its own identity, separate from the person who built it?
Agents that act through shared employee accounts leave investigators unable to distinguish an agent’s actions from a person’s decisions. Have the vendor show you how activity remains traceable when the builder changes roles or leaves.
Flag: Traceability depends on the builder’s account. If that person leaves or changes roles, the audit trail breaks.
Define: business rules and compliance requirements
3. Can we apply our existing rules and work with the risk and compliance tools we already use?
Separate rulebooks can leave risk leaders approving one restriction while agents operate under another. Ask how existing obligations reach agent controls and how evidence returns to the team’s current systems.
Flag: Integration means manual copying between systems, with no process to keep rules and evidence in sync.
4. Which regulatory frameworks come ready to use, and how does the evidence map to their requirements?
An audit can stall when activity records don’t answer a question about a specific requirement. Ask which obligations under the EU AI Act or recommendations in the NIST AI Risk Management Framework are supported. Request an example linking a requirement, its control, and supporting evidence.
Flag: The vendor claims logs are compliance-ready but can’t show which requirement a specific log entry satisfies.
Test: agent behavior and control effectiveness
5. How can we try to make an agent misbehave before launch?
Biased answers, invented information, or an agent manipulated through prompt injection can put customers at risk. Ask to see results from deliberate attempts to make the agent break its rules and whether a failed test blocks release.
Flag: The vendor shows passing results from standard tests but no evidence from deliberate attempts to break the agent.
6. What’s the plan when a rule change breaks something?
A restriction meant to protect customer data can cause an outage if it also blocks approved work. Ask how quickly the previous state can be restored, who is notified, and what the vendor does to isolate the rule change from running agents. Then ask what steps the vendor takes to reduce that risk before enforcement: staging, scoping the change to a subset of agents, or testing against recorded traffic.
Flag: The vendor’s only answer is to apply the change and watch for problems.
Enforce: prevention, boundaries, and exceptions
7. If our sales agent tries to send customer records to an outside tool, what stops it before the data leaves?
An alert after the transfer leaves you managing a data incident. Request a demonstration of prevention and a clear account of transfers the product can only report. Ask which controls or additional tools are needed for the workflow you run.
Flag: The vendor claims full visibility but can’t show what prevents the transfer, only what reports it.
8. Where do your controls sit, and what falls outside them?
A blocked transfer through one connection can still become a data incident if the agent reaches the same destination through another route. Ask the vendor to document where each control’s authority ends and what protection the buyer is responsible for beyond that boundary.
Flag: The vendor claims all traffic is governed but can’t identify which connections pass through the controls and which don’t.
9. What happens when a control fails or someone overrides it?
A failed control can expose data, while an unexplained shutdown can interrupt service. Ask whether the agent stops, continues, or seeks approval; who is notified; and how any override is authorized and recorded. Have the vendor explain what happens to work already in progress.
Flag: The vendor says an administrator can turn it off but can’t explain what happens to ongoing work or how that decision is recorded.
Prove: evidence and accountability
10. Can you explain an action or blocked attempt, including the agent, rules, and approvals behind it?
Boards and investigators need to understand why an action was permitted and who authorized exceptions. The record should identify the rule version and give reviewers enough context to judge whether the block was justified or interrupted legitimate work.
Flag: The log shows whether the action succeeded but not which rule applied or who approved the exception.
11. How would we know if a prevention control failed silently?
A team can believe it’s protected because alerts still fire, even when the blocking control behind them is down. Ask what evidence shows that prevention — not just detection — is still working, how a gap between the two reaches the responsible owner, and what the most recent verification looks like.
Flag: The vendor points to a green dashboard but can’t show evidence that prevention, not just monitoring, is operational.
Adapt: changes and authorization
12. When an urgent restriction is needed, what’s the fastest path to enforcement?
A critical restriction can be delayed when every change requires downtime or redeployment. Ask which updates take effect during operation and which require a maintenance window. Have the vendor walk through a time-sensitive scenario, blocking a newly discovered data exposure, for example, and explain how long enforcement takes from the moment you decide.
Flag: The vendor can’t walk through the steps or timeline for applying an urgent restriction.
13. Who can approve, change, or revoke an agent’s permissions, and is every change recorded?
Unclear authority can leave sensitive access in place after the business need ends. Ask for the approval trail, how an authorized owner can withdraw access, and when that withdrawal takes effect.
Flag: The vendor says the owner can remove access but can’t show who holds that authority or how revocation is recorded.
Across environments: consistency and portability
14. Do the same rules, controls, and evidence hold in our cloud, data center, and disconnected sites?
Uneven protection can leave a data center or remote site exposed under a policy that works in the cloud. Follow the same workflow through each environment. Ask what enforces the rule when outside services are unavailable, where records are stored, and how they reach the risk team later. Include agents built outside the vendor’s platform. Have the vendor identify what works today and what needs additional controls or rebuilding.
Flag: The vendor claims full cloud support but puts other environments on the roadmap.
15. How much depends on your infrastructure, and what can we take with us if we move?
A platform change can interrupt enforcement or make historical evidence difficult to use during an audit. Ask which rules, approvals, and records remain usable elsewhere, and which controls need replacement. Request an example of what can be exported and what your team would need to do to use it. If you replace the model or the framework used to build the agent, which rules and evidence remain valid, and what needs retesting?
Flag: The vendor says you can export your data but can’t show whether policies and audit records remain usable outside their platform.
Ask every vendor to document what its product doesn’t cover. An explicit limitation is something you can plan for. An unanswered question is unresolved risk.
Where single-environment governance stops
Governance built into the agent platform reduces integration work and keeps controls close to the agent. The question is how far that coverage reaches. Governance limited to one environment leaves your team responsible for protection beyond it.
Consider an insurer with a customer service agent in the cloud and claims data in its own data center. If it moves the agent into the data center or adds an agent built elsewhere, the test is whether the same rules, enforcement, and evidence follow. At disconnected sites, controls must work without outside services, with records kept locally until they can be transferred.
The insurer should set policy, own its evidence, and retain both when it changes infrastructure. Ask what remains usable and what needs rebuilding. Integrated governance should support your rules and your evidence across the environments where your agents run.
Establishing a standard for governed agents
A governed agent should have an identifiable owner, a current registry entry, and a traceable set of rules. Your organization should be able to show how those rules are tested and enforced, explain recorded decisions, and keep the process working as the agent changes. That standard applies wherever the agent operates.
The checklist sets a minimum evaluation standard for the complete governance process. An unanswered question identifies work that needs an owner, a control that needs to be supplied, or a deployment decision that deserves review. Vendors should make their contribution and limits clear enough for your team to act on them.
Begin with one agent that handles consequential data or actions, and follow it through the full lifecycle in the environments where it runs. Reviewing the evidence at each stage will show where your governance works and where exposure remains. The gap between claiming governance and running it is where enterprise risk lives.
See how DataRobot defines policy once and enforces it across the agent lifecycle.
The post Everyone claims AI governance now. 15 questions to test the claim. appeared first on DataRobot.
The Robotics Engineer Skill Set Is Changing. Here’s What Senior Engineers Need Now
Self-adjusting drone controls could make for smoother formation flying
Gemini 4 Carbon: Our next frontier model
A new flexible foot pad enables this one-legged, hopping robot to go where few robots have gone before
How Should Engineers and Buyers Evaluate Robot AI
Insect-inspired flying robot bounces back after collisions
Competing in RoboCup@Home and winning the 2026 best paper award: an interview with Raphael Memmesheimer
Team NimbRo’s robot in action during the RoboCup@Home competition at RoboCup 2026. Raphael leads the team.
Raphael Memmesheimer has been successfully competing in the RoboCup@Home league winning the RoboCup@Home competition five times. He and his colleagues won the best paper award at RoboCup 2026 for their work “Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation”. We caught up with Raphael to find out more about this research, his history with RoboCup, and the challenges of the @Home competition.
Could you start by telling us how you got involved with RoboCup@Home?
I’ve actually been involved in the @Home league since I was a student, and my first time going to a World Cup for @Home was in 2013. That competition motivated me to do a PhD and infected me in a way that meant I basically aligned my academic career to it. I always return to robot competitions because this is one of the rare places where you can benchmark robots in a real environment and have a real competition. Commonly in research you tune your approaches on data sets and show that they perform in a one-shot video. In robot competitions, you basically have to roll out your robot, it enters the arena, and then it operates autonomously on its own. It is always interesting to see how new developments make the robots perform better.
You and your co-authors won the best paper award at the RoboCup symposium this year. Congratulations! Could you give us a quick introduction to the paper?
The motivation for this work is that we only have two preparation days at the RoboCup competitions. So we arrive there, we have to assemble our robots, and the robots need to get used to the new environment [a model apartment] that we haven’t seen before. Also, the robots don’t have a clue about anything in this apartment. And within the two preparation days, the robots need to learn the objects they are interacting with. For example, they have to recognise what a spoon is, or a specific drink.
Our paper basically supports us in very efficiently generating this data such that the robots can more quickly learn models for the objects that are involved in this competition. And the interesting thing is it’s not just a few objects that we can distinguish between; there can be up to 50 different objects that we haven’t seen before. Our approach basically optimizes the process so that we can quickly get a good vision model for the robot. We are aware of open-vocabulary approaches but found them to have issues distinguishing between objects of a category like drinks.
Is it possible to do some pre-training to anticipate objects that might be used in the competition?
Partially that’s possible, but most objects cannot really be anticipated. We know that there are drinks, there are snacks, that there are cutlery items in the apartment. But what cannot be anticipated is the details because objects come from different areas in the world. The organisers will usually go to a shopping mall and just get local groceries there. Those objects are then distributed in the apartment. Part of the object set comes from the publicly available YCB Dataset, for those we can gather data beforehand, and of course pre-training with large general object sets.
Could you talk about your methodology?
We use a multi-stage approach. Firstly, we segment, such that we learn what an object actually is on a segmentation level. We look at all the objects that robots saw in the past editions of these competitions, and we train one model that just says, “oh, there is something which might be an object”. And then we get reference images of the actual objects and we semi-supervised or semi-automatically label those objects. The idea is that we combine identifying something as an object with a reference of what could actually be there. That allows us to simplify the labeling approach. Around 60% of the data is automatically labeled based on our approach. That would take a lot of hours by human annotation.
Labeled example images from different competition venues. Figure from “Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation”.
Have you been able to use the method already in RoboCup competitions?
Yes, we used it in Brazil for RoboCup 2025 and at RoboCup GermanOpen for the local RoboCup competitions. The method is continuously evolving. Now that we have this very good vision model, it means that we save a lot of time during the two preparation days and we can concentrate more on testing and achieving the actual tasks instead of just focusing on the vision element. One side note: nowadays there are promptable object detection or segmentation methods, but we found that they don’t scale yet to the number of classes that are involved in this competition. They work fine with 5 to 10 different classes, but we had issues distinguishing all 50 classes just by prompting them.
In this year’s World Cup (in Korea) we used an evolved approach where we used videos to generate even more data.
Team NimbRo’s robot picking laundry from the washing machine.
How did your team get on in the competition this time?
It went well. We had issues, as always during competitions, but we ended up in 3rd place. There was a new “Doing Laundry” task introduced this year. The robot had to carry a laundry basket from a washing machine to a folding table and open the washing machine. It had to detect the washing machine door, then find the specific point to grasp the handle to open the machine door. Then it had to pull out t-shirts and fold them.
Our robot performed the best in this task. It was able to open the washing machine and start folding the laundry. And this was possible because we used these semi-automatic labeling approaches.
What were the other tasks that teams needed to carry out?
The first big hurdle at the competition is to make the robot pass the initial inspection. Robots must satisfy certain safety and interactive capabilities and if they fail they are not allowed to take part in the competition.
In the competition itself there are five main tasks.
One is human-robot interaction. The robots have to open the door for party guests and greet them. Here, the robot has to go to the door, find the handle, align the end-effector to the handle, open the door, let the human enter, speak with the guest while holding eye-contact, guide them to the living room, find them a free seat, and introduce this guest to any other guests. This process continues on repeat as long as there are new guests arriving.
Team NimbRo’s robot opening the door.
Another task is a pick and place task, and this time we had to clear a table after breakfast. So the robot had to bring the cutlery to the dishwasher, put a dishwasher tablet into the dishwasher, close the dishwasher, and then prepare the table ready for the next meal. That’s a very manipulation intensive task.
There is also a general-purpose service robot task. This task is undefined because the robot has to deal with a random natural language command that can incorporate all elements of the apartment. For example it might be asked to go and fetch a particular drink or snack and hand it to someone. In the past there were people from the audience called into the arena and asked to give a command. It’s interesting to see how good these robots are at interacting with a new person and not someone they are used to receiving commands from.
Another interesting task is the restaurant task. Here, the robots are not in the known environment that we trained them in for two days, instead they are taken to an unknown restaurant location. The robot is placed there and needs to serve naturally. Only the menu is known to the teams in advance. The robots need to scan the restaurant to autonomously operate and find waving customers. The order is then given by voice, and usually these restaurant environments are very noisy. This is a very interesting task for the audience because it’s in the wild and people gather around it to watch it.
Then, of course, there’s the new laundry task that I described above. To add to what I said before, this is a very hard task because the robot cannot see into the washing machine. So it has to feel if it has a T-shirt in its hand and take the t-shirt out of the machine. Then the robot has to go to a table and start folding other t-shirts.
Team NimbRo
Interesting. I was at RoboCup in Eindhoven and I found the @Home competition fascinating.
We actually won the competition that year!
The league was more divided then, with three different leagues: two standard platforms and an open platform league. It was the open platform league that we won.
Now, all of those leagues have been merged. This year there were 24 teams from all over the world participating in just one league, which is amazing. It also drives competition a lot.
How much of an improvement have you seen over the years that you’ve been doing RoboCup@Home?
There are always phases of improvement, as I would describe them. I started pre-deep learning, and there were no vision robust models for seeing objects in unknown environments. What was working well was mapping and navigation. When I started in the league, the robots were already talking, but often, instead of talking to a human like they were supposed to, they might talk to a potted plant or a coat hanger because they had a shape similar to a human. This is no longer the case – nowadays we perceive humans quite well. We can also recognize speech quite well. Task planning is still a challenge, but it has improved. Then there are things that appear and push the boundaries, such as large language models, open vocabulary approaches and, most recently, agentic systems. But every now and then there’s stagnation because teams are just overwhelmed by the wide amount of different challenges that this competition throws at you. Basically your robot has to do everything. It needs to navigate in an apartment, it needs to be able to interact with humans, it has to manipulate items, and it has to do task planning. And all of those are very intensive research areas that are challenging in themselves. In addition the implementations must be very robust, as there are only a few trials.
How are you and the team planning to improve the robot for the next competition?
We are working on improving manipulation capabilities so that the robot can more quickly pick various objects. We are also continuously testing our robots’ performance to make our robots more human-like, and using two arms for manipulation is something we want to improve.
About Raphael
|
Raphael Memmesheimer’s long-term research goal is to develop autonomous service robots that assist people in everyday environments: robots that perceive, understand, and learn from human behavior using multi-modal sensory information and that prove themselves outside the lab. He is currently a postdoctoral researcher with the Autonomous Intelligent Systems (AIS) group at the University of Bonn, where he re-established and leads the domestic service robot team NimbRo@Home. He is also a founding member of the Center for Robotics at the University of Bonn. Raphael Memmesheimer has extensive experience in benchmark robotics competitions, having led teams at two universities to multiple international titles. Notably, he has won the RoboCup@Home World Championship five times (including three consecutive titles), along with multiple victories at the RoboCup German Open and top placements in the European Robotics League and the World Robot Summit. |
Genie 4: A new generation of world simulation
The Humanoid Robotics Funding Race: What the Money Tells Us About Commercial Readiness
Artificial skin helps robots grasp pencil lead and pull away from needles
Balancing Torque and Battery Life in Next-Generation Autonomous Mobile Robots
Robots in Society, Business and Culture: September 2026
A walking robotic hand, from Fingers as Legs: Learning Self-Supported Locomotion and Manipulation with an Anthropomorphic Hand, Amirhossein Kazemipour, Hehui Zheng, Robert Katzschmann.
By Emmet Cole
In our monthly series, we showcase a selection of robotics stories from society, business, research and culture, as we track the field’s ongoing journey from specialized industrial machines to an increasingly visible social phenomenon.
Five million and counting
More than five million industrial robots are now operating in factories worldwide, according to the International Federation of Robotics’ newly released World Robotics 2026 report.
Factories installed more than 600,000 industrial robots during 2025, an increase of 11% on the previous year. The global operational stock grew by 9%.
China continued to pull away from the rest of the world. Its factories installed 354,000 robots, accounting for 59% of the global total. Chinese manufacturers supplied 55% of their domestic market.
The United States overtook Japan to become the world’s second-largest market, with almost 38,500 installations. India installed nearly 10,500 robots, an increase of 15%. Europe moved in the opposite direction: installations fell by 8% in Germany, 11% in Italy, 8% in France and 15% in Spain.
IFR forecasts that worldwide installations will reach 655,000 units in 2026 and 806,000 in 2029.
Separately, Toyota told investors that modernizing factories operated by the company, its group businesses and major suppliers could require around 400,000 robots. The estimate includes replacement machines, new industrial robots, automated logistics systems and possible future humanoid deployments. Toyota said the programme could cost ¥1 trillion, or approximately $6.4 billion, annually from 2028, although it has not made a firm spending commitment.
Elsewhere, Universal Robots used the IMTS trade show in Chicago to launch Gen 7, a redesigned cobot platform built around greater computing power, tool-flange connectivity, sensor integration and support for AI-assisted applications.
The hands behind the robots
In Rajasthan, India, women are being paid to record household activities from cameras worn on their heads, according to an investigation by Decode. Their footage captures everyday tasks such as cooking, cleaning and folding clothes from the worker’s point of view.
The recordings can be divided into individual actions and used to teach robots how people manipulate objects in real homes. This kind of data is especially valuable because household environments contain clutter, variation and improvised behaviour that are difficult to reproduce in a robotics laboratory.
Decode estimates that between 20,000 and 30,000 people may be doing this work in India, although that figure is an extrapolation rather than an official count. The article also stops short of establishing which robot companies ultimately receive the featured workers’ recordings.
Lethal autonomous robots
International efforts to regulate autonomous weapons reached a significant point in September. Three years of discussions by the United Nations Group of Governmental Experts ended with growing support for formal negotiations on new international rules.
Human Rights Watch says 76 states now support negotiations on legally binding measures governing autonomous weapons systems. However, the group’s final report was weakened during negotiations. References to ethical concerns and to the design and development of the weapons were among the passages removed or reduced following objections from states including Russia and the United States.
The issue will now move to the Convention on Certain Conventional Weapons Review Conference in November, where states may decide whether to establish a negotiating mandate.
The Stop Killer Robots campaign argues that further discussion is no longer enough and that states should begin negotiating prohibitions and regulations.
Computer scientist Stuart Russell also entered the debate in a short UC Berkeley video. Russell called for a ban on small autonomous anti-personnel weapons, warning that machines capable of selecting and attacking people could become cheap, scalable weapons of mass destruction.
His intervention is more specific than a call to prohibit every military application of AI. It focuses on systems that can independently identify human targets and apply lethal force at scale.
Does the robot play fair?
A study published in the International Journal of Human–Computer Interaction examined whether perceptions of fairness can reduce conflict between people and security robots.
Xin Ye and Lionel Robert considered two forms of fairness. Distributive fairness concerns whether the robot’s decision or its consequences appear fair. Informational fairness concerns whether the robot provides an adequate explanation for that decision.
The researchers found that both can help reduce conflict and improve acceptance. The broader implication is that security robots may need to justify their actions as well as perform their assigned tasks effectively.
The results come from controlled human–robot interaction research, so they should not be treated as evidence from robots exercising authority in operational policing environments. Nevertheless, the study raises a question that will become more pressing as autonomous systems move into security roles: what does it mean for a machine’s exercise of authority to be regarded as legitimate?
Give us a hand
Finally, researchers at ETH Zurich have taught an anthropomorphic robotic hand to walk on its fingertips.
The self-contained hand uses reinforcement-learning policies to crawl, steer and recover after falling. It can also support its own weight while pressing successive keyboard commands, and it can push an object towards targets using overhead visual feedback.
The work is currently an eight-page arXiv preprint, and separate policies are used for different tasks. Even so, it offers an inventive demonstration of a robot repurposing the same hardware for movement, balance and manipulation.