Researchers have published ThorArena, a simulation-based benchmark for evaluating humanoid robots under realistic physical forces, addressing a significant gap in how current robot systems are tested. The paper was published on the arXiv preprint server. Most existing humanoid benchmarks assess robot movement in empty spaces without applied weight or resistance – conditions that systematically overstate performance on the tasks that matter commercially.
The problem is structural: a robot that looks stable and coordinated in a benchmark environment without external forces may lose balance the moment it carries a heavy box, pushes furniture, or exerts meaningful effort against resistance. ThorArena is designed to expose that gap before it becomes a deployment problem.
How ThorArena Works
The benchmark was built by capturing real-world human interaction data. Human operators wore VR headsets and motion trackers while performing daily physical tasks including lifting and lowering a water container, pushing and pulling a chair, and carrying an object with a partner. Specialized 3D-printed hand tools equipped with sensors measured the directional forces exerted during each task.
These recorded human movements and hand forces were then mapped onto a virtual humanoid robot inside a physics simulator. The system replays the exact force patterns against the virtual robot as it attempts to follow the movements – subjecting the robot’s control system to the same physical loads that a human performing the task would experience. Robots are scored using a metric the team developed called FATS – Force-Aware Tracking Score – which evaluates both tracking accuracy and balance under load.
What the Results Showed
The researchers ran four humanoid control systems through the benchmark and found that force-aware evaluation revealed performance differences that conventional tests obscured. When physical forces were turned off in the simulation, all four systems performed well and maintained balance across the test tasks. When the recorded forces were replayed, scores declined and meaningful performance gaps between systems emerged.
“Experiments demonstrate that force-aware evaluation reveals substantial performance differences that remain largely hidden under conventional no-force evaluation,” the team wrote in the paper.
The team’s own control system, Thor2, performed best across the six tasks, achieving the highest average FATS score and maintaining near-perfect balance throughout. The specific performance gaps between the other three systems were not publicly detailed but were described as substantial relative to what conventional benchmarks would suggest.
Why This Matters for Deployment
The commercial deployment of humanoid robots in manufacturing, logistics, and services is accelerating – BMW is deploying Figure 03 at Spartanburg, Agibot has G2 units running on electronics production lines, and Hyundai is planning 25,000 Atlas units across its plants. All of these use cases involve robots exerting meaningful forces against objects and environments rather than moving through empty space.
The credibility of capability claims made by humanoid manufacturers depends on the quality of the benchmarks against which those claims are validated. A benchmark that does not include realistic physical forces does not test the capability that matters for industrial deployment. ThorArena provides a more representative evaluation surface – one that is likely to become a reference point as the industry matures and enterprise customers seek more rigorous performance guarantees before committing to large-scale purchases.
The research team plans to expand ThorArena’s dataset to include more diverse tasks and force conditions, and to evaluate real-world physical robots rather than simulated ones in future work.
