Retiming Robot Demonstrations with a Learned Speed Prior

1× real-time Every video plays at actual speed. Playback is not sped up.

Real-robot rollouts, side by side. Ours: π_0.5-DROID fine-tuned on demonstrations retimed by our speed prior, executed by our pace-aware controller. Baseline: π_0.5-DROID fine-tuned on the original demonstration timing, executed with standard PD position control. The two differ in both training timing and controller; Table 2 of the paper separates the two effects.

Fine-tuned on handheld demonstrations

Both policies are fine-tuned on the same handheld demonstrations of each task.

Stacking cups

Stack three cups into a pyramid. Open video

Folding towel

Fold the towel on the table. Open video

Cleaning trash

Put the paper balls on the table into the bin. Open video

Zero-shot

Both policies are trained on DROID only, with no demonstrations of this task; ours retimes DROID to the handheld pace of FastUMI-100K and runs on its 0.9 tempo channel.

Placing bananas

Put both bananas into the tray. Open video

Abstract

The pace of a robot demonstration is often determined by the teleoperator rather than by the task itself. As a result, the same motion can appear at widely different speeds across demonstrations, and policies trained on such data may inherit operator-specific timing along with task behavior. We propose to learn a speed prior directly from large-scale demonstration data. Given a task instruction and a motion path, it predicts a distribution over execution speed along the path, from which a chosen quantile defines a consistent clock for retiming each demonstration. Existing methods derive the acceleration signal from the very data they accelerate, so each new task costs another fit, annotation, or pair of predefined rates. A single trained speed prior instead covers every task in a corpus. We fit a speed prior on both robot teleoperation data and handheld demonstrations in which a human arm, rather than a robot arm, moves the gripper, and use it to retime robot demonstrations to the handheld pace before fine-tuning a vision-language-action policy. For execution, we pair the policy with a real-time optimal-control layer that tracks predicted action chunks under robot constraints using the predicted pace. On 120 RoboLab tasks, policies fine-tuned on our learned clocks succeed more often and move with up to half the jerk compared with policies fine-tuned on the original demonstration timing and uniformly accelerated to the same speed. On a real robot, our policy stacks cups in 3.43 s, 1.65× faster than baseline's 5.67 s while succeeding in all 30 trials.

Method

Speed prior pipeline: path windows, speed prior, speed along the path, and clock — **Learning the pace.** A speed prior predicts the distribution of speed along a demonstration path; integrating a chosen quantile gives the clock used to retime the demonstration.

One execution cycle: tempo policy, then the solver's draft, splice and smooth steps — **One execution cycle.** The policy predicts an action chunk and its pace; the solver drafts a path at that pace, joins it to the running plan at the earliest point it can reach on time, and smooths it within robot limits.