Safe Nonlinear and World Model Control in Microseconds via Koopman-Linearized Pre-Compilation
Summary
Abstract
Expressive models. Microsecond planning. precompiles learned Koopman dynamics into a compact, action-only optimization problem whose size is independent of the lifted state dimension. Expensive matrix operations move offline; online control uses matrix–vector products and projections, adapting to new goals and constraints without refactorization.
reduces planning latency by one to over two orders of magnitude across benchmarks, enabling sub-millisecond visual world model planning, 4096-dimensional quadruped control in under 5 ms, and collision-aware planning for 200 quadrotors at over 30 Hz. The same framework runs onboard a Crazyflie microcontroller, bringing efficient predictive control from embedded robots to high-dimensional learned models.
Method
1 / 4Precompute the latent rollout
A learned encoder lifts the measured state into linear latent dynamics. compiles these dynamics into maps from the initial state and control sequence to future outputs. The maps are built offline for a fixed model and horizon, so online prediction needs only one state encoding and matrix–vector products.
2 / 4Condense the quadratic program
Substituting the prediction maps into the objective and constraints leaves only the control sequence to optimize. The QP size depends on the horizon and input dimension, not the latent dimension. New goals change only the linear objective term; locally linearized constraints become inequalities on the controls, tightened for approximation error.
3 / 4Solve with a frozen inverse
A primal–dual solver updates constraint multipliers by projection and controls through a precomputed inverse. For fixed model, horizon, weights, and regularization, this inverse is reused as goals and constraints change. Each iteration needs only matrix–vector products and projections. This moves expensive matrix inversion and factorization offline, keeping online updates lightweight.
4 / 4Learn through differentiable control
A differentiable tracking QP recovers the actions needed to reproduce demonstrated trajectories. Comparing these actions with demonstrations trains the encoder and dynamics for control, alongside prediction losses. DAgger-style aggregation adds simulated trajectories from the tracking controller, helping the model learn on the states its own controller visits.
Experiments
Large models do not have to mean slow control. Across these benchmarks, runs 4096-dimensional quadruped models in milliseconds and 2048-dimensional visual world models in less than one millisecond. For 3D quadcopter control at a 100-step horizon, it reaches 128 μs on CPU: 19× faster than OSQP on the identical condensed QP and over 100× faster than GPU-ADMM SQP on CPU.
2 / 5Fast, Scalable Multi-Agent Planning
Four times the agents. Only 2.66× the planning time. Scaling from 50 to 200 quadrotors raises batched planning latency from 12.1 to 32.1 ms, maintaining over 30 Hz updates and above 98% goal reaching. The 150-agent case shown here takes 23.0 ms. On physical hardware, seven differential-drive robots complete a full five-round update in 18.24 ms on a modest CPU, using just batched NumPy without accelerator libraries such as JAX.
3 / 5Long Horizon (H=75) planning onboard the crazyflie MCU in 24.57 ms
A 75-step lookahead, running on the aircraft’s own microcontroller. A 128-dimensional learned model plans 1.875 seconds ahead on the Crazyflie’s Cortex–M4F, averaging 24.57 ms per solve during flight with just 8.7 kB of runtime workspace. This onboard deployment solves an unconstrained planning problem; the complete experimental loop runs at about 15 Hz, including communication and state acquisition.
4 / 5Whole-Body Quadruped Planning
4096 latent dimensions. Twelve joint torques. Just 4.47 ms. The model captures the full 61D robot state: base pose and velocity, joint states, foot positions, and ground-reaction forces. Planning directly commands all 12 joint torques; the 25-step timing includes reference generation and state encoding. Extending the horizon to 100 steps takes 13.30 ms. With whole-body obstacle filtering, the 100-step controller still runs in 28.18 ms, avoiding the obstacle crossed by the nominal policy in the evaluated rollouts.
5 / 52048D Linear World Models
A 2048-dimensional world model, planning from pixels in under a millisecond. Bimanual rope planning takes 0.725 ms with 94% success, versus 559 ms and 46% for Fast-LeWM on the evaluated tasks. Hardware cube manipulation takes 0.483 ms per solve. Differentiable QP supervision recovers the vertical control authority needed to reach the elevated target; prediction-only training falls short. The same visual planning framework also sustains about 30 Hz on Jetson hardware.
Cite
@misc{muMPC_2026,
title = {μMPC: Safe Nonlinear and World Model Control in Microseconds via Koopman-Linearized Pre-Compilation},
author = {Nath, Devesh and Chou, Glen},
note = {Preprint},
year = {2026}
}