Motion Intelligence

Movement reveals intent.

Before someone crosses a road, they shift their weight.
Before a cyclist turns, their trajectory begins to change.
Before two people collide, their movements begin to converge.

Movement contains information about what is happening now, what may happen next, and how others should respond.

Motion Intelligence is the ability to read that information.

Seeing is not understanding.

Modern perception systems are remarkably good at identifying what surrounds a machine. A person. A bicycle. A vehicle. A doorway.

Planning systems can then determine a path or action.

But real environments are dynamic. The important question is often not simply what is there, but what is it doing?

• Is the pedestrian waiting or about to cross?
• Is the approaching person passing beside me or moving into my path?
• Is the vehicle maintaining its lane or beginning to merge?

These distinctions emerge through motion.

The grammar of movement.

Human beings interpret movement almost continuously – we notice hesitation, acceleration, commitment, convergence and avoidance. Often without consciously calculating any of them.

SpatioTemporal is exploring how machines can develop an equivalent computational capability.

Rather than treating motion as a secondary consequence of visual perception, we model it directly as structured information across space and time.

The premise is simple: Motion has structure. Structure carries information. Information about movement reveals intent.

Compressing space and time into Motion Tokens.

Continuous movement is complex. SpatioTemporal converts short windows of kinematic behaviour into discrete Motion Tokens: a learned vocabulary of recurring patterns of movement.

Each token represents a small piece of motion through space and time.

A sequence of tokens becomes something richer.

Just as the meaning of a sentence emerges from relationships between words, behavioural information can emerge from relationships between movements.

Movement becomes vocabulary.
 Sequences become intent.
 Intent reveals behaviour.

Explore Motion Tokens →

Physics up, not pixels down.

Much of contemporary AI begins with pixels and attempts to learn increasingly rich representations of the physical world from them.

SpatioTemporal takes a complementary approach.

We begin with motion itself: position, direction, velocity, acceleration and how these change through time.

This creates a compact representation focused specifically on how things move, rather than reconstructing everything a machine can see.

It is a deliberately different signal, remarkably lightweight, designed to complement perception and planning systems rather than replace them.

It’s closer to instinct than deliberate thought – the reflex that lets us find a gap in traffic, pause a beat to let someone pass, or cross a busy road without consciously calculating a single trajectory.

LSTM-01, our first Large SpatioTemporal Model, learns patterns of movement from sequences of Motion Tokens. Rather than controlling a robot directly, it provides an additional source of information about movement, intent and what may happen next.

Between perception and planning.

A machine’s existing perception stack observes and tracks the world. SpatioTemporal consumes the resulting motion information and interprets its evolution through time. We then provide the existing planning system with additional context about how the scene is changing, what those changes may imply, and what may happen next.

PERCEPTION
What’s around me?
↓
SPATIOTEMPORAL INTELLIGENCE
How is it moving?
What does that movement imply?
What may happen next?
↓
PLANNING
What should I do?

This makes Motion Intelligence a parallel source of context for autonomous systems operating around people, vehicles and other moving agents.

Experience Motion Intelligence for yourself.

Motion Intelligence is easiest to understand when you move through it.

In the Motion Lab, you become the human inside a shared-space robotics simulation. Approach a robot, cross its path, hesitate or suddenly change direction, then switch Motion Intelligence on and off.

The perception and planning problem stays the same. What changes is the robot’s ability to interpret the motion between them.

Try the Motion Lab →

From motion to consequence.

Motion Intelligence is where we begin. As the representation develops, the same foundation provides a path towards richer understanding of dynamic environments:

The goal is not simply to predict trajectories. It’s to give machines a better representation of what is unfolding around them, why it matters, and what their own actions may cause.

And all in a lightweight, vector-based representation of the environment we call the World State Vector.

Motion changes behaviour.

In NVIDIA Cosmos simulation, we compared an unchanged navigation planner with the same planner augmented by Motion Intelligence.

• The baseline produced near-collisions in 24% of runs.
• With Motion Intelligence, near-collisions fell to 2%.

The planner itself was unchanged. What changed was the information available to it.

Explore the research →

One model of motion. Two directions.

Motion Intelligence can work in two directions. In the real world, it reads movement to help infer intent. In simulation, the same underlying representation can work forward, turning intent and context into movement.

This creates a different way to populate simulated worlds. Rather than selecting from a fixed library of pedestrian behaviours, a simulator could create them on the fly: a tired commuter moving slowly but purposefully towards a train, a distracted driver reacting late to a changing light, a hurried cyclist threading through traffic, or a hesitant pedestrian approaching a crossing, stepping forward, reconsidering, then committing. Each can vary continuously in pace, confidence, attention, urgency and response to the world around them.

The result is not simply more people in a simulation. It is more behavioural variation. The same crossing can play out thousands of subtly different ways, creating the unusual, ambiguous and imperfect human interactions that Physical AI will eventually encounter in the real world. A model that learns the grammar of movement can help machines read that grammar, and potentially generate new examples of it for training and evaluation.

Robots that read the room.
Cars that read the road.

Physical AI will increasingly operate in environments shaped by people: roads, workplaces, hospitals, footpaths, shops and homes.

Seeing those environments is essential. Understanding how they are changing is another problem.

Running Motion Intelligence on-device does not require another enormous robot brain. Motion Intelligence is designed as a small, fast reflex layer that can sit alongside the autonomy stack and stay awake when deeper reasoning doesn’t need to.

Motion Intelligence. A new domain of AI.