khanhnd61/impact_libero_goal

IMPACT — libero_goal

Instruction-Modulated Perception + ACTion chunking, trained from scratch on the libero_goal suite of LIBERO. No robot pretraining: an ImageNet ResNet-18 and a frozen T5-small are the only pretrained weights in the model.

libero_goal is ten different goals in a single shared kitchen scene, distinguishable only by the instruction.

Result

	success rate	episodes
this checkpoint	89.0%	200

Evaluated on 10 tasks × 20 initial states = 200 rollouts, at 256×256, control_mode=relative, executing the full 50-action chunk open-loop.

This is the 25,000-step checkpoint, selected as the highest-scoring of the four checkpoints evaluated for this suite. The final checkpoint of the same run scores 84.5%; this one was picked because it scored higher on the reported rollouts, which means the 89.0% figure carries selection bias and 84.5% is the unselected number.

How it compares

Averages over all four LIBERO suites. IMPACT's 78.6% is the mean of each suite's selected checkpoint; shipping every suite's final checkpoint instead gives 76.4%.

policy	params	robot pretraining	LIBERO avg
Diffusion Policy	—	none	72.4%
Octo	93M	Open X-Embodiment	75.1%
OpenVLA	7B	Open X-Embodiment	76.5%
IMPACT	78M	none	78.6%
OpenVLA + parallel decoding & action chunking	7B	Open X-Embodiment	90.2%
π0	3B	proprietary + OXE	94.2%
OpenVLA-OFT	7B	Open X-Embodiment	97.1%

Baselines from Kim et al., Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success, Table I. IMPACT leads every policy that does not use robot pretraining, and both pretrained models of its own generation, at 78M parameters. The 90%+ tier is 3–7B models; the gap between OpenVLA at 76.5% and the same model at 90.2% comes from execution strategy, not capacity — which is the most promising thing to try here next, since IMPACT currently executes all 50 predicted actions open-loop.

Architecture

Two camera frames, an 8-D end-effector state and a natural-language instruction in; a 50-step action chunk out in one forward pass. The instruction enters twice — as 32 tokens appended to the transformer encoder sequence, and as FiLM scale/shift (1 + γ) ⊙ x + β on every ResNet stage, so language modulates perception and not only the fused memory.

backbone	ResNet-18, ImageNet init, BN folded
text tower	T5-small encoder, frozen, 32 tokens
transformer	d=512, 8 heads, ff 3200, 6 encoder / 4 decoder layers
chunk	50 actions
parameters	78M trainable, 113M total

Training

data	HuggingFaceVLA/libero, libero_goal episodes only
steps	25,000 (of a 30k run) at batch 64
lr	1e-4 trunk, 1e-5 backbone, flat
hardware	1× RTX 3090, ~8 h

The suite split resolves each episode's instruction against the LIBERO benchmark's own per-suite task lists and raises on anything unplaceable, rather than splitting on task-index ranges.

Usage

from lerobot.policies.impact.modeling_impact import IMPACTPolicy

policy = IMPACTPolicy.from_pretrained("khanhnd61/impact_libero_goal")

Requires the impact policy from LeRobot. Instructions are tokenized to a fixed 32 tokens; every LIBERO instruction fits (the longest, in libero_10, is 22 tokens), but longer instructions are silently truncated.

Limitations

• Single-suite. Trained on this suite's 10 tasks only; it is not a generalist policy.
• No held-out split. Training used every episode, so there is no validation signal separating learning from memorisation. The rollouts are the only out-of-sample evidence, and they use the same 10 tasks with different initial states.
• 200 rollouts per suite, not the 500 used by most published LIBERO tables, so sampling noise is roughly ±7 points (±2 SE).
• No FiLM ablation. Language use is measured, not assumed: on libero_goal, showing the policy another task's instruction drops it to 0.0% on the original goal over 600 episodes (control 88.5%) while it achieves the instructed goal 91.0% of the time. Which language path carries that is not isolated - shuffling perturbs FiLM and the encoder text tokens together - so use_film=false is still needed for the FiLM claim specifically.
• No test of language generalisation. Every instruction evaluated is one the policy trained on, so this is instruction following, not robustness to unseen wording.