Movement reveals intent.
Before someone crosses a road, they shift their weight.
Before a cyclist turns, their trajectory begins to change.
Before two people collide, their movements begin to converge.
Movement contains information about what is happening now, what may happen next, and how others should respond.
Motion Intelligence is the ability to read that information.
Seeing is not understanding.
Modern perception systems are remarkably good at identifying what surrounds a machine. A person. A bicycle. A vehicle. A doorway.
Planning systems can then determine a path or action.
But real environments are dynamic. The important question is often not simply what is there, but what is it doing?
- Is the pedestrian waiting or about to cross?
- Is the approaching person passing beside me or moving into my path?
- Is the vehicle maintaining its lane or beginning to merge?
These distinctions emerge through motion.
The grammar of movement.
Human beings interpret movement almost continuously – we notice hesitation, acceleration, commitment, convergence and avoidance. Often without consciously calculating any of them.
SpatioTemporal is exploring how machines can develop an equivalent computational capability.
Rather than treating motion as a secondary consequence of visual perception, we model it directly as structured information across space and time.
The premise is simple: Motion has structure. Structure carries information. Information about movement reveals intent.
Compressing space and time into Motion Tokens.
Continuous movement is complex. SpatioTemporal converts short windows of kinematic behaviour into discrete Motion Tokens: a learned vocabulary of recurring patterns of movement.
Each token represents a small piece of motion through space and time.
A sequence of tokens becomes something richer.
Just as the meaning of a sentence emerges from relationships between words, behavioural information can emerge from relationships between movements.
Movement becomes vocabulary. Sequences become intent. Intent reveals behaviour.
Physics up, FTW.
Much of contemporary AI begins with pixels and attempts to learn increasingly rich representations of the physical world from them.
SpatioTemporal takes a complementary approach.
We begin with motion itself: position, direction, velocity, acceleration and how these change through time.
This creates a compact representation focused specifically on how things move, rather than reconstructing everything a machine can see.
It is a deliberately different signal, remarkably lightweight, designed to complement perception and planning systems rather than replace them.
It’s closer to instinct than deliberate thought – the reflex that lets us find a gap in traffic, pause a beat to let someone pass, or cross a busy road without consciously calculating a single trajectory.
Between perception and planning.
A machine’s existing perception stack observes and tracks the world. SpatioTemporal consumes the resulting motion information and interprets its evolution through time. We then inform the machine’s existing planning system with more colour – with more adjectives – describing the scene, with alerts and warnings where appropriate.
PERCEPTION
What’s around me?
↓
SPATIOTEMPORAL INTELLIGENCE
How is it moving?
What does that movement imply?
What may happen next?
↓
PLANNING
What should I do?
This makes Motion Intelligence a parallel source of context for autonomous systems operating around people, vehicles and other moving agents – rising to the top priority when reading the road/reading the room becomes of critical importance.
From motion to consequence.
Motion Intelligence is where we begin. As the representation develops, the same foundation provides a path towards richer understanding of dynamic environments:
Motion
↓
Intent
↓
Human Awareness
↓
World State
↓
Possible Futures
↓
Consequence
The goal is not simply to predict trajectories. It’s to give machines a better representation of what is unfolding around them, why it matters, and what their own actions may cause.
And all in a lightweight, vector-based representation of the environment we call the World State Vector.
Motion changes behaviour.
In NVIDIA Cosmos simulation, we compared an unchanged navigation planner with the same planner augmented by Motion Intelligence.
- The baseline produced near-collisions in 24% of runs.
- With Motion Intelligence, near-collisions fell to 2%.
The planner itself was unchanged. What changed was the information available to it.
Robots that read the room.
Cars that read the road.
Physical AI will increasingly operate in environments shaped by people: roads, workplaces, hospitals, footpaths, shops and homes.
Seeing those environments is essential. Understanding how they are changing is another problem. Doing that on-device, with a low-power, reflex-like model will be the new standard when robots are interacting in and around humans.
Motion Intelligence. A new domain of AI.