Action generation

The policy predicts robot targets and independent left- and right-jaw commands. These visualizations replay the original checkpoint on recorded observations.

At three pauses, six candidate action sequences are refined over 16 DDIM iterations. Color marks future action index; the inset shows raw jaw values. The plans are new offline outputs, approximately projected into the camera view. Resuming footage shows the original recording.

Browse 23 individual action videos · Replay and projection details

Diffusion-policy outputs

The numerical explorer below uses a separate replay with the checkpoint’s original 100-iteration setting. It shows the recorded intermediate model values, including the raw jaw predictions.

Offline checkpoint replay
Wrist RGB conditioning frame
Wrist camera
Side camera 1 conditioning frame
Side camera 1
Side camera 2 conditioning frame
Side camera 2
Iteration100 / 100

16 future steps

Normalized Cartesian targets
Equal normalized scale for XYZ.
Raw jaw values Left Right
0 = closed; 1 = open after decoding.
100 / 100

EMA checkpoint, seed 42. New offline predictions on recorded inputs; video/state alignment is approximate.

Four denoising iterations with fixed-scale normalized Cartesian trajectories and raw jaw predictions
One 100-iteration replay at 20.0 s. Top: normalized XYZ with fixed axis limits. Bottom: raw jaw values; the dotted line marks the decoding threshold. PDF
Final action sequence in physical units
Final x, y, z targets in millimeters and decoded left and right jaw sequences
The same offline sample after denoising. PDF · Numerical arrays

Recorded robot and jaw commands

Left jaw—
Right jaw—
Episode time0.0 s

Logged commands from episode 14. Right closure occurs at 20.6 s and left closure at 21.1 s. These values do not measure contact or force.

Recorded Cartesian targets, measured end-effector positions, and independent jaw commands
Recorded episode 14. Initial all-zero logger rows are excluded from the target curves. PDF · Data