Real-robot rollouts, side by side. Ours: our retimed policy with the predicted tempo and our controller.
Fine-tuned
Four real runs per task: Default 1× + PD, Default 1× + high-gain PD (gains ×4, velocity gains ×2, no step clipping), Default 1× + our controller, and Ours (predicted tempo + our controller). Default: DROID joint-position policy fine-tuned on the demonstrations at their recorded timing.
Trash cleanup
The high-gain PD run ends in a reflex stop (red light) by violating the arm’s kinodynamic limits. Our controller keeps every chunk within them.
Cup stacking
Times are from a single run of each policy.
Towel folding
In the high-gain PD run, the wrist chattered until an emergency stop.
Zero-shot
No task-specific fine-tuning. Default: DROID joint-position policy trained at the recorded timing of DROID.
Banana pick-and-place
Cube pick-and-place
Abstract
The pace of a robot demonstration is often determined by the teleoperator rather than by the task itself. As a result, the same motion can appear at widely different speeds across demonstrations, and policies trained on such data may inherit operator-specific timing along with task behavior. We propose to learn a speed prior directly from large-scale demonstration data. Given a task instruction, a motion path and the images along it, it predicts a distribution over execution speed along the path, from which a chosen quantile defines a consistent clock for retiming each demonstration. Existing methods derive the acceleration signal from the very data they accelerate, so each new task costs another fit, annotation, or pair of predefined rates. A single trained speed prior instead covers every task in a corpus. We fit a speed prior on both robot teleoperation data and handheld demonstrations in which a human arm, rather than a robot arm, moves the gripper, and use it to retime robot demonstrations to the handheld pace before fine-tuning a vision-language-action policy. For execution, we pair the policy with a real-time optimal-control layer that tracks predicted action chunks under robot constraints using the predicted pace. On 120 RoboLab tasks, policies fine-tuned on our learned clocks succeed more often and move with up to half the jerk compared with policies fine-tuned on the original demonstration timing and uniformly accelerated to the same speed. On a real robot, our policy stacks cups in 3.50 s, 1.72× faster than the default policy’s 6.02 s while succeeding as often, in 29 of 30 trials.
Method
Swipe to see the whole figure, or tap to enlarge.
Swipe to see the whole figure, or tap to enlarge.
Retiming demonstrations at training time
Demonstrators move at their own pace, so situations that look the same can come with action labels that run at very different speeds. Below are three demonstrations at nearly the same moment of cup stacking, just after the first cup is picked up. Laid out at one waypoint per 1/15 s, their action labels spread far apart along the same path. Retiming every demonstration to the median clock of our learned speed prior brings the same waypoints back together, so the same situation comes with the same tempo, which is easier to imitate.
Controlling the pace at inference time
Action space augmented with tempo channels
Real-robot execution
On the right is the recorded execution: the policy predicts how fast each action should run, and our controller executes that tempo within the robot's limits. On the left, the same footage is slowed down so that every action takes the default time step (1/15 s); it is a replay of the recording, not a second run. Turn on the sound: the engine speeds up when the executed tempo rises and slows down when it falls.
Engine sound: “ferrari355underhood4.mp3” by enginemusic, CC BY 3.0, modified.
One path, four clocks
The same recorded Cup motion replayed on four tempo channels. q50 (one action per 1/15 s) always runs; pick a faster channel on the scale to race it against q50. The video overlays the q50 run as a faint ghost.
tempo
quantile
All four are timing replays of one q99 rollout (dots: one action each, 1/15 s apart on the action clock). Times are measured from the policy start; done = second cup released.
Citation
@article{moon2026retiming,
title = {Retiming Robot Demonstrations with a Learned Speed Prior},
author = {Moon, Seungyong and Lee, Deokjae and Lee, Hansang and Battsengel, Munkhtselmeg and Kim, Youngin and Yun, Seonghyeon and Jeon, Sungjin and Kim, SongEun and Park, Jeongmin and Song, Hyun Oh},
journal = {arXiv preprint arXiv:2610.32676},
year = {2026}
}