Jisoo · May 17, 2026
Frontier Systems for the Physical World
As of 2026, AI is centered on language and code. The scaling laws of LLMs are already well understood, and the virtuous cycle in which data, compute, and algorithms push one another upward is working well. Even a small performance gain turns directly into money. It is only natural that capital and attention are flowing into this paradigm.
Alongside it, several fields are quietly gathering strength: general-purpose robotics models such as VLA and WAM, physical reasoning for AI scientists, and new interfaces such as BCI and neurotechnology. These fields are not advancing through technology alone; talent, capital, and founders are beginning to flow into them as well.
The technical pieces needed to extend AI into the physical world are maturing at the same time, and judging by the pace of the past 18 months, these fields may soon enter a serious scaling phase too.
The largest opportunities inside a technology paradigm usually sit in places like this: one step away from the mainstream, while still governed by the same scaling principles. Close enough to inherit the momentum of existing infrastructure and research, but far enough away that it cannot be copied easily. That distance creates two things: a moat against fast followers, and a broader, less explored problem space because the easy paths have not yet been claimed. That also makes unexpected capabilities more likely to emerge.
Three fields fit those conditions today: robot learning; autonomous science, especially materials science and life science; and new human-machine interfaces. These include new sensory modes such as brain-computer interfaces, silent speech, neural wearables, and digital olfaction. The three are not separate, unrelated fields. They belong to the same current: frontier systems for the physical world.
They share technical foundations such as learned representations of physical dynamics, architectures for embodied action, simulation and synthetic data infrastructure, an expanding sensory manifold, and closed-loop agentic orchestration. They reinforce one another and create compounding effects across domains. They are also the areas where qualitatively new AI capabilities are most likely to appear when model scale, physical grounding, and new data formats meet.
This essay examines the foundational technologies behind these systems, considers why these three fields are frontier opportunities, and argues that their mutual reinforcement becomes a structural flywheel for extending AI into the physical world.
Foundational Technologies
Before looking at specific applications, we need to understand the common technical base that makes these frontier systems possible. Five key foundational technologies support frontier AI's move into the physical world. They are not limited to particular applications. They are building blocks for systems that extend AI into the physical world. The fact that they are maturing at the same time is what makes this moment special.
Learned Representations of Physical Dynamics
The most fundamental foundational technology is the ability to learn a compressed, general-purpose representation of how the physical world works: how objects move, deform, collide, and respond to force. Without this, every physical-world AI system would have to learn the physics of its own domain from scratch, and the cost would be unbearable.
Several architectures are converging on the same capability from different directions. Vision-Language-Action models, or VLAs, approach from the top down. They take pretrained vision-language models that already understand objects, spatial relationships, and language, then attach an action decoder that emits motor commands. The core idea is that the enormous cost of learning how to see and understand the world can be amortized through internet-scale image-text pretraining. Models such as Physical Intelligence's π₀, Google DeepMind's Gemini Robotics, and NVIDIA's GR00T N1 have demonstrated this structure at increasing scale.
World Action Models, or WAMs, approach from the bottom up. Built on video diffusion transformers pretrained on internet-scale video, they inherit rich priors about how objects fall, disappear behind occlusions, and interact under force, then combine those priors with action generation. NVIDIA's DreamZero demonstrated zero-shot generalization to entirely new tasks and environments, and even transfer from human video demonstrations to different robot bodies with only a small amount of adaptation data.
There is also a third path, one that may be the most revealing for understanding where the field is going. It uses neither a pretrained VLM nor a video diffusion backbone. Generalist's GEN-1 is a native embodied foundation model trained from scratch on more than 500,000 hours of real-world physical interaction data. That data was collected mainly by attaching low-cost wearables to people performing everyday manipulation tasks. It is not a VLA in the standard sense, because it does not fine-tune a vision-language backbone. It is not a WAM either. It is a foundation model for physical interaction, designed from the start to learn representations of dynamics from the statistics of humans touching objects, rather than from internet images, text, or video.
The spatial intelligence being built by companies such as World Labs also matters to this foundation, because it fills a representational gap shared by VLAs, WAMs, and native embodied models. None of these models explicitly models the three-dimensional structure of the scene in which it operates. VLAs inherit 2D visual features from image-text pretraining. WAMs learn dynamics from video, which is a 2D projection of 3D reality. Models trained on wearable sensor data capture force and motion, but not scene geometry. Spatial intelligence models can fill this gap by learning to reconstruct, generate, and reason about the full 3D structure of physical environments: geometry, lighting, occlusion, object relationships, and spatial layout.
The convergence across approaches is the key point. Whether representations are inherited from VLMs, learned through video co-training, or built natively from physical interaction data, the underlying foundational technology is the same: a compressed and transferable model of how the physical world works. The data flywheel for this representation is enormous and largely untapped. It includes not only internet video and robot trajectories, but also the vast range of human physical experience that wearable devices are only now beginning to capture at scale. The same representation can serve a robot learning to fold towels, an autonomous lab predicting reaction outcomes, and a neural decoder interpreting grasp plans in the motor cortex.
Architectures for Embodied Action
Representations of physics are necessary, but not sufficient. To turn understanding into reliable physical action, an architecture has to solve several problems at once: translating high-level intent into continuous motor commands, maintaining coherence across long action horizons, operating within real-time latency constraints, and improving through experience.
Dual-system hierarchical architectures have become the standard pattern. They separate a slow, powerful vision-language model responsible for scene understanding and task reasoning, System 2, from a fast, lightweight visuomotor policy responsible for real-time control, System 1. GR00T N1, Gemini Robotics, and Figure's Helix all use variations of this approach. It resolves the fundamental tension between the rich reasoning of large models and the millisecond-level control frequencies required by physical tasks. Generalist, meanwhile, takes a different approach called harmonic reasoning, in which thinking and acting happen together.
Action-generation mechanisms are also evolving quickly. Flow matching and diffusion-based action heads, pioneered by π₀, have become the mainstream way to produce smooth, high-frequency continuous actions. They are replacing discrete tokenization methods borrowed from language models. This approach treats action generation as a denoising process similar to image synthesis, producing trajectories that are physically smoother and more robust to compounding errors than autoregressive token prediction.
But the most important architectural advance may be adding reinforcement learning to pretrained VLAs: the idea that a foundation model trained from demonstrations can improve by practicing on its own, much like people refine skills through repetition and self-correction. Physical Intelligence's π₀.₆ work shows the clearest large-scale demonstration of this principle. Their method, RECAP (RL with Experience and Corrections via Advantage-conditioned Policies), addresses a problem pure imitation learning cannot solve: identifying where something went wrong in a long task. If a robot grips an espresso machine's portafilter at a slightly wrong angle, the failure may only appear several steps later when insertion fails. Imitation learning has no mechanism for assigning that failure to the earlier grasp. Reinforcement learning does. RECAP learns a value function that estimates the probability of success from any intermediate state, then conditions the VLA to choose high-advantage actions. The important point is that it integrates heterogeneous data into one training pipeline: demonstration data, autonomous experience, and data corrected by an expert through teleoperation during execution.
The results are encouraging. π₀.₆ folds 50 new clothing categories in real homes, reliably assembles boxes, and makes espresso on a professional machine, operating for hours without human intervention. On the hardest tasks, RECAP more than doubles throughput compared with imitation-only baselines and cuts failure rates by more than half. It also shows that RL post-training produces behavior qualitatively different from imitation: smoother recovery, more efficient grasping strategies, and adaptive error correction not present in the demonstration data.
These results suggest that the same compute-scaling dynamics that took LLMs from GPT-2 to GPT-4 are beginning to operate in embodied domains. The difference is that we are earlier on the curve, and the action spaces are continuous, high-dimensional, and constrained by unforgiving real-world physics.
Simulation and Synthetic Data as Scaling Infrastructure
In language, the internet solved the data problem. Trillions of naturally occurring text tokens were available for free. In the physical world, the data problem is harder by orders of magnitude. This is already well known, and the surge of startups trying to build data vendors for the physical world is evidence of it. Real-world robot trajectories are expensive to collect, risky to scale, and limited in diversity. A language model can learn from a billion conversations, but a robot cannot yet perform a billion physical interactions.
Simulation and synthetic data generation are the infrastructure layer that loosens this constraint. Their maturation is one of the key reasons physical-world AI is accelerating now rather than five years ago.
Modern simulation stacks combine physics-based simulation engines, photorealistic rendering through ray tracing, procedural environment generation, and world foundation models that create photorealistic video from simulation inputs to bridge the sim-to-real gap. The pipeline begins with neural reconstruction of real environments using only a smartphone, then moves to physically accurate placement of 3D assets and large-scale synthetic data generation with automatic annotations.
The implication of better simulation stacks is intuitive: they change the economic assumptions underlying physical-world AI. If the bottleneck in physical AI moves from real-world data collection to designing diverse virtual environments, the cost curve breaks. Simulation scales with compute, not human labor or physical hardware. It changes the economics of training physical-world AI in the same way internet-scale text data changed the economics of training language models. That means investment in simulation infrastructure has asymmetrically large leverage across the entire ecosystem.
Simulation is not a foundational technology for robotics alone. The same infrastructure applies to autonomous science, including digital twins of laboratory equipment and simulated reaction environments for pre-testing hypotheses; to new interfaces, including simulated neural environments for training BCI decoders and synthetic sensory data for calibrating new sensors; and to every other field where AI meets the physical world. Simulation is a general-purpose data engine for physical-world AI.
Expanding the Sensory Manifold
The physical world communicates through signals far richer than vision and language. Touch conveys information about material properties, grip stability, and contact geometry that cameras cannot see. Neural signals encode motor intent, cognitive state, and perceptual experience at bandwidths far beyond any existing human-computer interface. Muscle activity below the vocal cords already contains speech intent before sound is produced. The fourth foundational technology is the rapid expansion of AI's sensory access to modalities that were previously inaccessible. This is being driven not only by research, but also by an ecosystem building the devices, software, and infrastructure to capture and process these signals at consumer scale.
The most visible sign of this expansion is the emergence of new device categories. AR devices have improved substantially in user experience and form factor in recent years, and companies are building applications for both consumer and industrial use. Voice-first AI wearables follow users into the physical world and give language-based AI broader context. Over the long term, neural interfaces may open even more comprehensive interaction modes. AI has created an inflection point in computing and opened the opportunity to dramatically advance how humans interact with computers. Companies such as Sesame are building new modalities and devices for that future.
Dominant modalities such as voice create tailwinds for new means of interaction. As products such as Wispr Flow push voice as a primary input mode, taking advantage of its high information density, market dynamics around silent speech interfaces become more favorable too. Silent speech devices use various sensors to detect movements of the tongue and vocal tract and decode speech without sound, making them an even higher-density modality than voice for interacting with computers and AI.
Whether invasive or noninvasive, brain-computer interfaces represent a deeper frontier, and the commercial ecosystem continues to move forward. The signal here is progress toward clinical validation, regulatory clearance, platform integration, and institutional capital, all around a technology category that only a few years ago belonged almost entirely to academia.
Tactile sensing is entering embodied AI architectures. Some robot-learning models have begun to explicitly include touch as a core part of their approach. Olfactory interfaces are becoming real engineering artifacts as well. Wearable displays using compact odor generators with millisecond response times have been demonstrated for mixed-reality applications, and olfactory models paired with vision AI systems are being developed for chemical process monitoring.
The pattern across all these developments is that they ultimately converge on one another. AR glasses generate continuous visual and spatial data about how users interact with physical environments. EMG wristbands capture the statistics of human motor intent. Silent speech interfaces capture the mapping between subvocal articulation and language output. BCIs capture neural activity at the highest possible resolution. Tactile sensors capture the contact dynamics of physical manipulation. Each new device category is also a data-generation platform that feeds models across multiple application domains at once. A robot trained on motor-intent data extracted from EMG learns different grasping strategies than a robot trained only through teleoperation. A laboratory interface that responds to subvocal commands enables a different kind of scientist-machine interaction than a keyboard. A neural decoder trained on high-density BCI data creates representations of motor planning inaccessible through any other channel.
The spread of these devices is expanding the effective dimensionality of the data manifold available for training frontier physical-world AI systems. And because much of this expansion is being driven not only by academic labs but also by well-capitalized consumer product companies, the data flywheel can scale with market adoption.
Closed-Loop Agentic Systems
The final foundational technology is more architectural: the ability to connect perception, reasoning, and action into autonomous systems that operate continuously over long periods without human intervention. The core is not simply making one judgment and taking one action. It is building feedback loops that observe the result of an action and revise the next strategy based on that result.
A similar shift appeared in language models with the rise of agentic systems. With multi-step reasoning, tool use, and self-correcting workflows, models evolved from one-shot question-answering machines into systems that solve problems autonomously. The same transition is happening in the physical world, but the requirements are much harsher. If a language agent makes a mistake, it can often roll back at almost no cost. If a physical agent drops a beaker of reagent, rollback is not so simple.
Physical-world agent systems differ from digital agents in three ways. First, they have to be directly embedded in an experimental or operational loop. They must connect directly to raw data streams from equipment, physical state sensors, and actuation primitives, grounding their reasoning in actual physical reality rather than in a textual description of reality. Second, they need persistence over long periods. Instead of treating each task as an independent episode, they need memory, provenance tracking, safety monitoring, and recovery behaviors that maintain continuity across tasks. Third, they have to adapt based on outcomes. They must revise strategy not merely from textual feedback, but from real physical results.
This foundational technology turns individual capabilities into systems that actually work. A good world model, a stable action architecture, and a rich sensor suite are not enough if they exist separately. They need an integration layer that binds them together and lets them operate autonomously in the physical world. Only when this layer matures can the three application areas described below move beyond isolated research demonstrations into systems deployable in the real world.
Three Domains
The foundational technologies described above are a general-purpose enabling layer. But they do not by themselves determine where the most important applications will emerge. Many fields require physical action, physical measurement, or physical sensation. What separates a frontier system from a mere improvement to an existing system is how strongly model performance and scaling infrastructure compound within that domain. The important question is whether they can create qualitatively new capabilities that were previously impossible, not merely improve performance.
Robotics, AI-driven science, and new human-machine interfaces are the three domains where this compounding effect is strongest. Each combines the foundational technologies above in a different way. Each has also been constrained by longstanding limits, and those bottlenecks are now beginning to loosen because of the technologies discussed earlier. As these systems operate, they generate structured physical data that improves the same foundational technologies again. The result is a closed feedback loop that pushes the entire system upward faster.