Harold Soh in conversation with Dr. Bangdao Chen at the opening fireside chat of AIMX Singapore 2026, beneath the session title 'Embodied Intelligence: Bridging AI and Physical Action'.
AIMX Singapore 2026 · Opening fireside chat

← Back to Horizons

IndustryOpinion pieceEmbodied AI

Embodied AI: Between Demo and Deployment.

Notes from the opening fireside chat at AIMX Singapore 2026 — on why commercial embodied AI is still much closer to demo, and the three blockers in the way.

Director, Horizons OfficeAssociate Professor, School of Computing, NUSDate27 August 2026Read~8 minutes

Yesterday morning I participated in the opening fireside chat at AIMX Singapore 2026 on "Embodied Intelligence: Bridging AI and Physical Action." The session was moderated by Dr. Bangdao Chen of The Blockhouse Technology. Bangdao asked probing questions and I have been thinking about our conversation since. This post is a longer version of what we discussed, with links to articles and numbers. To be clear, when discussing Embodied AI (EAI) below, I'm referring to general-purpose learning-based robot policies.

The short version: commercial EAI is between demo and deployment, and still much closer to demo. There have been a lot of beautiful proofs-of-concept in the last two years (my social media streams are full of amazing robot video demonstrations). But there have been very few clear wins where a robot was used somewhere real and stayed deployed.

There are blockers that prevent sustained deployment, and I will mention just three of them: Reliability, Throughput, and Safety (RTS for short, since we Singaporeans love our acronyms). Addressing these blockers requires significant work on both the research and development fronts.

01

R is for Reliability

The problem is not that robots fail. Everything fails at some point. The problem is that it remains hard to predict when and how badly current EAI models fail.

A group at Huawei Noah's Ark did something interesting this year: they took the winning entries from the BEHAVIOR-1K 2025 Challenge and re-ran them. Across twenty moderate-difficulty household tasks, the best model averaged a score of 0.256. Putting aside that this average score is not close to what we need for deployment, it is instructive to look at the performance ranges: the per-task score spread was from 0.000 to 0.800.

Two further findings are, to me, also consequential:

First, roughly half of all scenarios were bimodal; the policy either completed the task or failed completely. This is not a distribution with a mean and a comfortable variance. It's like a coin flip whose bias we cannot reliably estimate in advance.

Second, they re-ran the official checkpoints with the official seeds and saw per-task gaps exceeding 0.27 on some tasks. The attributed cause was simulator non-determinism, for example, minor differences in lighting or object appearance between runs. Whatever the cause, the policies were brittle.

On RoboDojo's real-robot leaderboard (July 2026), π0.5 averages ≈12.8% success across three embodiments on 18 tasks — with per-embodiment numbers ranging 3.3% → 21.7%. The average hides substantial performance heterogeneity.

This matters commercially. You can plan or engineer around a known failure rate. If a system fails 20% of the time and you're aware of that, you can work around it with buffers or fallback procedures. But you cannot plan around a system that is excellent one day and gets zero work done the next, without any advance indication whether it's a good or bad day. Many unpredicted failures require a person to fix the problem, which is why robot deployments can lead to more work or potentially increase friction in existing workflows.

02

T is for Throughput

If you watch robot videos, you will notice many of them are sped up (a small 5x or 10x at one of the corners usually indicates the speed-up). It's not a big secret that EAI models + robots are typically slow, and it is worth understanding why. Often, it is not a problem of weak actuators. Rather, we can isolate three issues:

First, the training data is slow. Many EAI models today are trained with behavioral cloning from teleoperated demonstrations, and teleoperation remains awkward. The operator typically has to control the robot using a SpaceMouse, VR controller, or puppeteering device. I've tried one myself with a data collection setup we built at our lab and at first I had trouble even picking up a simple cube. The problem is that the common setups provide limited depth cues and little or no tactile/force feedback, so control is challenging. The operators then move slowly and deliberately, and learned policies inherit that pace. There is research work in speeding up learned robot policies, through modulation or through metadata conditioning. But these come with costs: executing faster than the demonstrations creates a distribution shift, which can break policies.

Second, we need to pay attention to the interface between the policy and the robot. Robot policies typically output position targets (where the joints or end-effector should go). These targets are fed into a low-level controller that converts them into motor commands. The controller then acts as an interface the policy learns and acts through, and can have a significant effect on robot performance; this effect was nicely described in recent work from Pulkit Agrawal's lab at MIT. The interface parameters are often not reported in publicly-available robot datasets, and this is problematic since it affects what is learned. Achieving higher performance and throughput requires us to better consider the robot system as a whole.

Third, model inference takes time. Many models employ architectures that require "pausing" for small amounts of time during execution. These pauses are typically larger for models that leverage test-time compute (e.g., reasoning or world models). For example, on ALOHA, NVIDIA's Cosmos Policy can pause for almost a second to generate a two-second action chunk using a H100. There is research in reducing policy latency, such as real-time chunking, streaming flow policies, and streaming stochastic interpolant policies. But more work needs to happen to scale these methods up and get latency down.

For many commercial applications, robots don't need to be faster than us. They need to clear a threshold to be economically viable. Speed multiplied by uptime is what determines this viability. Right now both terms are too small.

03

S is for Safety

Safety is a nuanced concept when it comes to "generalist robots" that share human spaces and perform multiple tasks. I recently gave a talk at the RSS workshop on Rethinking What It Means to be "Safe" for Generalist Robots. It was a room full of people who work on robot safety, and we found it difficult to even define what safety is in this context. Classical robot safety was largely built for "specialists" in factories, for example, a welding arm in a cage with joint limits and no humans around. But in the generalist context, safety cannot be reduced to a fixed set of context-free constraints — it requires reasoning about the context, environment, intent, and human expectations. For example, breaking an egg is a failure if the robot is putting groceries away, but it is a necessary step if the robot is making an omelette. A robot can set a knife down cleanly (no contact, no collision, nobody hurt) and still have done something unsafe, because it left the knife within reach of a toddler. This matters because a robot can avoid immediate faults and still create an unsafe situation.

There are also technical issues that need fixing. A 2026 simulation benchmark showed that EAI models can have rather weak safety properties. None of the tested models were violation-free, and even among the best model's successful episodes, 12.5% involved a safety violation. On the theoretical front, in work with my student Eugene Lim (who recently defended and is now Dr. Lim), we looked at whether certain model failures (specifically, action hallucinations) are simply bugs we can train away with more data. Our conclusion is no; some errors are a consequence of specific model designs rather than limited training data. It is important to consider such issues so that we can correctly design the models now, rather than discovering these faults at deployment. Beyond just the model design, we should also consider model scaffolding (or "harnesses", as they are often called nowadays): verifiers, checkers, safety layers. This requires a systems approach to safety that goes beyond just a data-centric view.

—

Closing

I am more optimistic about Embodied AI than the above may suggest. The reason is that we are recognizing these issues now, and we are beginning to measure how bad things are. Fixing a problem begins with a strong understanding of our current state; the demo is just a sample from a distribution. The deployment is the whole distribution, every day, in a building with people in it. Proper evaluation is a key part of moving our robots along the path from demo to deployment.

I'll discuss potential ways of moving forward in a future post. Briefly, I believe the next phase of embodied AI will involve fewer cool videos and more "boring" reliable robots; robots that predict and recover from failures, sustain throughputs that clear economic thresholds, and are engineered for safety. That is how we move from demo to deployment.

Thanks to Dr. Bangdao Chen for a good conversation, to the AIMX Singapore team for the invitation, and to my students at CLeAR whose work shaped my thinking about this.

AI usage: LLMs (Claude and ChatGPT) were used to find relevant resources and in the editing of this article.

Industry← Back to the Horizons Office
DiscussWrite to us →