Retiming Robot Demonstrations
with a Learned Speed Prior

Seungyong Moon* Deokjae Lee* Hansang Lee* Munkhtselmeg Battsengel Youngin Kim Seonghyeon Yun Sungjin Jeon SongEun Kim Jeongmin Park Hyun Oh Song†

Seoul National University

*Equal contribution   †Corresponding author

Preprint

1× real-time Every video plays at the robot’s actual speed.

Real-robot rollouts, side by side. Ours: our retimed policy with the predicted tempo and our controller.

Fine-tuned

Four real runs per task: Default 1× + PD, Default 1× + high-gain PD (gains ×4, velocity gains ×2, no step clipping), Default 1× + our controller, and Ours (predicted tempo + our controller). Default: DROID joint-position policy fine-tuned on the demonstrations at their recorded timing.

Trash cleanup

Default 1× + PD 0/15 (timeout)Default 1× + high-gain PD 0/15Default 1× + our controller 9/15 (timeout)Ours 15/15 (48.26 s)
The high-gain PD run ends in a reflex stop (red light) by violating the arm’s kinodynamic limits. Our controller keeps every chunk within them.

Cup stacking

Default 1× + PD 8.39 sDefault 1× + high-gain PD (timeout)Default 1× + our controller 4.50 sOurs 3.16 s
Times are from a single run of each policy.

Towel folding

Default 1× + PD Default 1× + high-gain PD Default 1× + our controller (timeout)Ours 4.49 s
In the high-gain PD run, the wrist chattered until an emergency stop.

Zero-shot

No task-specific fine-tuning. Default: DROID joint-position policy trained at the recorded timing of DROID.

Banana pick-and-place

Default 1× + PD 1/2 (timeout)Ours 2/2 (11.49 s)

Cube pick-and-place

Default 1× + PD 2/9 (timeout)Ours 9/9 (52.90 s)

Abstract

The pace of a robot demonstration is often determined by the teleoperator rather than by the task itself. As a result, the same motion can appear at widely different speeds across demonstrations, and policies trained on such data may inherit operator-specific timing along with task behavior. We propose to learn a speed prior directly from large-scale demonstration data. Given a task instruction, a motion path and the images along it, it predicts a distribution over execution speed along the path, from which a chosen quantile defines a consistent clock for retiming each demonstration. Existing methods derive the acceleration signal from the very data they accelerate, so each new task costs another fit, annotation, or pair of predefined rates. A single trained speed prior instead covers every task in a corpus. We fit a speed prior on both robot teleoperation data and handheld demonstrations in which a human arm, rather than a robot arm, moves the gripper, and use it to retime robot demonstrations to the handheld pace before fine-tuning a vision-language-action policy. For execution, we pair the policy with a real-time optimal-control layer that tracks predicted action chunks under robot constraints using the predicted pace. On 120 RoboLab tasks, policies fine-tuned on our learned clocks succeed more often and move with up to half the jerk compared with policies fine-tuned on the original demonstration timing and uniformly accelerated to the same speed. On a real robot, our policy stacks cups in 3.50 s, 1.72× faster than the default policy’s 6.02 s while succeeding as often, in 29 of 30 trials.

Method

Speed prior pipeline: path windows, speed prior, speed along the path, and clock

Swipe to see the whole figure, or tap to enlarge.

Learning the pace. A speed prior predicts the distribution of speed along a demonstration path; integrating a chosen quantile gives the clock used to retime the demonstration.
One execution cycle: tempo policy, then the solver's draft, splice and smooth steps

Swipe to see the whole figure, or tap to enlarge.

One execution cycle. The policy predicts an action chunk and its pace; the solver drafts a path at that pace, splices it into the running plan where it arrives on time, and smooths it within robot limits.

Retiming demonstrations at training time

Demonstrators move at their own pace, so situations that look the same can come with action labels that run at very different speeds. Below are three demonstrations at nearly the same moment of cup stacking, just after the first cup is picked up. Laid out at one waypoint per 1/15 s, their action labels spread far apart along the same path. Retiming every demonstration to the median clock of our learned speed prior brings the same waypoints back together, so the same situation comes with the same tempo, which is easier to imitate.

Similar observations wrist camera, just after picking up the first cup demo A demo B demo C Original timing raw labels: the operator's pace side view (Y–Z) · ● start distance along the path 0 mm 100 mm 200 mm 300 mm 152 mm 202 mm 262 mm per-waypoint spread 19% Learned speed prior → retime every demonstration to its q50 (median) clock Tempo-aligned labels on the learned q50 clock side view (Y–Z) · ● start distance along the path 0 mm 100 mm 200 mm 300 mm 299 mm 300 mm 303 mm per-waypoint spread 2%
Real labels from the 412-demonstration cup dataset: the same three observations and the same waypoints before and after retiming, identical scales, starts moved to a common point; grey dashed lines join the same waypoint across the three demonstrations. Spread = standard deviation / mean of the distance covered at each waypoint, averaged over the waypoints.

Controlling the pace at inference time

Action space augmented with tempo channels

action expert output one step · 32 dims ⋮ 7 joints gripper tempo ×3 dims 8–10 padding (dims 11–31) time → action (joint 2) → same 15-step action chunk 0 s 0.25 s 0.5 s 0.75 s 1 s q50 retimed action default step · 1/15 s 1.00 s q90 tempo channel 0.80 s q95 tempo channel 0.75 s q99 tempo channel 0.66 s chunk start (arm state when the chunk begins) predicted waypoints
Choose the pace as a quantile of the speed prior, i.e. of the speeds demonstrated for the same motion across the training data (q90, q95, q99), or keep the default step of 1/15 s (q50).

Real-robot execution

On the right is the recorded execution: the policy predicts how fast each action should run, and our controller executes that tempo within the robot's limits. On the left, the same footage is slowed down so that every action takes the default time step (1/15 s); it is a replay of the recording, not a second run. Turn on the sound: the engine speeds up when the executed tempo rises and slows down when it falls.

The dial on the right video shows the executed speed-up over one action per 1/15 s, live: 0% at the top, slower to the left, faster to the right. The tempo changes within the task; it is not a uniform speed-up.
Engine sound: “ferrari355underhood4.mp3” by enginemusic, CC BY 3.0, modified.

One path, four clocks

The same recorded Cup motion replayed on four tempo channels. q50 (one action per 1/15 s) always runs; pick a faster channel on the scale to race it against q50. The video overlays the q50 run as a faint ghost.

tempo
quantile

All four are timing replays of one q99 rollout (dots: one action each, 1/15 s apart on the action clock). Times are measured from the policy start; done = second cup released.

Citation

@article{moon2026retiming,
  title   = {Retiming Robot Demonstrations with a Learned Speed Prior},
  author  = {Moon, Seungyong and Lee, Deokjae and Lee, Hansang and Battsengel, Munkhtselmeg and Kim, Youngin and Yun, Seonghyeon and Jeon, Sungjin and Kim, SongEun and Park, Jeongmin and Song, Hyun Oh},
  journal = {arXiv preprint arXiv:2610.32676},
  year    = {2026}
}