TRACE: Trajectory-Routed Causal Memory for Delayed-Evidence Visuomotor Imitation

TRACE stores task evidence while it is visible, routes that evidence through a path-signature memory, and reads it back when the robot reaches an ambiguous branch.

Read paper View results

Policy families Regression + Diffusion

Memory form Fixed-slot causal memory

Routing key Depth-3 path signatures

Evaluation 5 real robot tasks

Abstract

Long-horizon robot manipulation can require a robot to choose between task alternatives using evidence that is no longer visible. TRACE adds a compact online memory to visuomotor imitation policies. It writes visual-state evidence into fixed-size latent slots, organizes access with streamed path and delta signatures, and exposes the memory through lightweight adapters while leaving the policy backbone, action head, and imitation objective unchanged.

TRACE delayed-evidence overview figure — Delayed-evidence overview: the early cue is no longer visible at the branch point, so a short-history policy can choose the wrong continuation.

Problem

Current observations can match, while correct actions diverge.

01 / Early cue

The histories are still visibly different.

The robot observes the cue that will later determine the branch: origin, side, tray source, or object identity. TRACE writes this visual-state evidence before it leaves view.

History A Book begins on desk cue still visible branch not needed

History B Book begins on bed cue still visible branch not needed

TRACE stores the cue as memory content.

cue(H_t) = desk
cue(H'_t) = bed

At the beginning, the current observation can still identify the causal history.

Method

A routed memory layer around unchanged imitation policies.

TRACE signal flow figure — TRACE signal flow: visual-state evidence becomes memory content, path signatures become trajectory addresses, and the adapter returns compact conditioning to the base policy.

Write

Visual-state evidence is stored when the cue is visible.

Current RGB and proprioceptive features form the memory content, so the module stores the evidence before it disappears from the camera view.

Medicine / early tray cue A visible cue is written before the branch.

Task suite 5 real robot tasks

Physical evaluation 25 rollouts per task

Results

TRACE improves stage progress across delayed-evidence manipulation.

Average stage progress %

Main comparison

Causal memory helps when evidence has left view.

TRACE Regression reaches 69.23 average progress and TRACE Diffusion reaches 59.53, compared with 25.50 for ACT and 25.00 for Diffusion Policy.

Best reported average: 69.23 TRACE Regression
Task count: 5 physical delayed-evidence tasks

Rollout videos

Real-robot clips show the cue, shared segment, and delayed branch.

The video set collects the full Book, Laundry, Cable, and Medicine rollouts used to illustrate the delayed-evidence evaluation behind the stage-progress tables.

Book / desk origin Origin-conditioned relocation.

The book starts on the desk route, then moves through a shared carry segment before the policy must select the origin-consistent placement.

TRACE Regression: 83.00% Book progress

Book rollout results and memory diagnostics — The rollout figure shows early cue, ambiguous transit, target selection, and memory signals for the Book task. **TRACE writes the origin evidence while it is visible.** The initial book origin is visible at the start, so the memory update can store the cue before the later views become similar.

Paper Data Explorer

Effect and diagnostic results as interactive bar charts.

The controls below keep only result-bearing tables: choose a data group, switch the measured effect column, then hover any bar to inspect the exact reported value.

Table

Data group

Shown metric Metric

Implementation

The codebase mirrors the paper split: updater, adapter, and deployment path.

TRACE training and inference consistency — Training scans a masked fixed-budget causal history; deployment applies the same updater once per executed step.