What Is Physical AI? The Data Behind Embodied Intelligence
Physical AI is AI that perceives and acts in the physical world - robots and embodied agents. Its bottleneck is data. Here is what that data is and why it is scarce.
TL;DR. Physical AI is artificial intelligence that perceives and acts in the physical world - humanoid robots, manipulators and embodied agents - rather than only producing text or images. Its main bottleneck is not compute or model size but data: first-person demonstrations of real physical tasks, which have to be recorded rather than scraped.
Physical AI vs digital AI
Digital AI (chatbots, image models) learns from abundant web data. Physical AI has to interact with objects, contact forces and consequences, and there is no comparable archive of physical action. Vendors and labs - including NVIDIA with its Isaac GR00T work on humanoid foundation models - frame "physical AI" as the next frontier precisely because the data layer is immature.
Why data is the bottleneck
- There is no internet of actions; manipulation has to be demonstrated.
- Generalisation depends on diversity. The ICLR 2025 paper Data Scaling Laws in Imitation Learning found policy generalisation tracks the number of distinct environments and objects, not just demonstration count.
- Skilled and industrial tasks are under-represented in open datasets and hard for gig crowds to reach.
What training data physical AI needs
- Egocentric demonstrations of real tasks - see what is egocentric data.
- Rich signals: depth, hand pose, 6-DoF trajectory, action labels.
- Diversity of workers, tools and environments.
- Robotics formats: LeRobot, RLDS, HDF5.
- Provenance and consent, increasingly required under the EU AI Act.
How VLA models use it
Modern robot policies are often vision-language-action (VLA) models that map camera input and a language instruction to actions. They are hungry for exactly this demonstration data - covered in VLA models: the data they need.
FAQ
What is physical AI? AI that senses and acts in the physical world - robots and embodied agents - as opposed to purely digital models that output text or images.
Why is data the bottleneck for physical AI? Because physical action has no web-scale corpus. Robots must be shown tasks through first-person demonstrations, and diverse, skilled demonstrations are scarce.
What data does physical AI need? Egocentric demonstrations with depth, pose and action labels, across many environments, delivered in robotics formats with consent and provenance.
nxted supplies that data layer: nxted Capture · or compare providers in our buyer guide.
Physical-AI data specialists at OFORO LTD (UK). We write about egocentric data, robotics dataset formats, RLHF and data governance. See what we build.