--- license: apache-2.0 library_name: mimickit tags: - motion-tracking - deepmimic - humanoid - unitree-g1 - lafan - ppo - reinforcement-learning - isaac-lab datasets: - lvhaidong/LAFAN1_Retargeting_Dataset - ember-lab-berkeley/LAFAN-G1 base_model: [] pipeline_tag: reinforcement-learning --- # MimicKit-G1-LAFAN 5 motion-tracking policies for Unitree G1 (29-DoF) trained with [MimicKit](https://github.com/xbpeng/MimicKit) DeepMimic-style PPO on LAFAN1 retargeted slices: **fight, run, dance, jumps** (15 s each), plus a longer **30 s dance** (`dance1_subject2`) that warm-starts from the 15 s dance and holds full-horizon tracking. Single 4090 24G, ~1 h per 15 s motion, 4096 envs. **Project repo (源码 / 复现入口):** - 🧪 [vitorcen/isaaclab-experience](https://github.com/vitorcen/isaaclab-experience) — MimicKit training scripts, eval chain, USD material fix, design docs --- ## TL;DR | Motion | Episode-Length (终态 / max=15s) | Ship quality | Frames slice | |---|---|---|---| | **fight** | 14.85 s / **99.0 %** | 🟢 触顶 | LAFAN `fight1_subject2` [600:1050] | | **dance** | 14.70 s / **98.0 %** | 🟢 触顶 | LAFAN `dance1_subject1` [1746:2196] | | **jumps** | 14.70 s / **98.0 %** | 🟢 触顶 | LAFAN `jumps1_subject1` [3441:3891] | | **run** | 9.45 s / **63.0 %** | 🟡 plateau | LAFAN `run1_subject2` [3341:3791] | | **dance (30 s)** | full 30 s, `Test_Return` **244** (>15 s base 227) | 🟢 longer-horizon | LAFAN `dance1_subject2` [1521:2421] | 3/4 of the 15 s motions reach ship quality at iter 1500. `run` plateaus at ~63 %, likely needing ADD-style residual or curriculum sequencing — kept as a baseline. The **30 s dance** doubles the horizon (900 frames @ 30 fps): warm-started from the 15 s dance ckpt and run to 2500 iters, its converged `Test_Return` (244) actually exceeds the 15 s dance (227) — the discounted return saturates near the same ceiling regardless of clip length, so matching/exceeding it means full-horizon coverage held. --- ## Demos (eval, 4 envs, with restored per-link material) **30 s dance (longer-horizon, `dance1_subject2`)** — DeepMimic PPO holds the full 30 s; an AMP baseline on the same clip could not keep rhythm (see notes below). > Student renders use the **per-link material fix** (`assets/g1_textured.usd`) — the shipped MimicKit `g1.usd` collapses every MJCF `` into a single white `DefaultMaterial`. We re-parse `g1.xml`, build one `UsdPreviewSurface` per unique color, and rebind 36 mesh prims via `UsdShade.MaterialBindingAPI`. Details in the repo's `scripts/g1_usd_recolor.py`. --- ## Dataset chain | Layer | Source | What it provides | |---|---|---| | Original mocap | [ubisoft-la-forge/lafan1](https://github.com/ubisoft/ubisoft-laforge-animation-dataset) (CC-BY-NC 4.0) | Raw Ubisoft Montreal BVH | | G1 retargeting | [lvhaidong/LAFAN1_Retargeting_Dataset](https://huggingface.co/datasets/lvhaidong/LAFAN1_Retargeting_Dataset) | Unitree-style G1/H1/H1_2 retarget (community-mirrored — original unitreerobotics repo is currently offline) | | IsaacLab AMP repack | [ember-lab-berkeley/LAFAN-G1](https://huggingface.co/datasets/ember-lab-berkeley/LAFAN-G1) | Berkeley npz repackaging (CC-BY 4.0) | | **MimicKit slices (this repo)** | `scripts/lafan_g1_npz_to_mimickit.py` | 30 fps → MimicKit pkl, 450-frame center slices | Slicing rule: pick the most representative central segment of each `*1` clip (skip start/end idle), 450 frames @ 30 fps = 15 s. --- ## Files ``` MimicKit-G1-LAFAN/ ├── README.md ├── videos/ │ ├── dance1s2.mp4 (0.7 MB, 30 s dance, first 8 s) │ ├── fight.mp4 (6.9 MB) │ ├── run.mp4 (2.6 MB) │ ├── dance.mp4 (1.9 MB) │ └── jumps.mp4 (4.7 MB) ├── fight_15s/ │ ├── model_0000001500.pt # 11 MB │ ├── env.yaml # train_lafan_fight_15s_env.yaml │ └── motion.pkl # 450-frame LAFAN slice ├── run_15s/ … ├── dance_15s/ … ├── jumps_15s/ … ├── dance_30s/ # longer-horizon dance (dance1_subject2) │ ├── model.pt # 11 MB, final (2500 iters, warm-started) │ ├── env.yaml # 900-frame / 30 s slice │ └── motion.pkl └── assets/ └── g1_textured.usd # 26 MB, per-link material restored ``` --- ## Training recipe (all 4 identical) | Hyperparam | Value | |---|---| | Algorithm | DeepMimic PPO (MimicKit default) | | Optimizer | SGD, lr=1e-4, momentum=0.9 | | Num envs | 4096 (Isaac Lab, RTX 4090 24 G) | | Iters | 1500 | | Action std (Gaussian policy) | 0.05 | | Reward | DeepMimic style: pose + vel + ee + root + COM, no AMP discriminator | | Termination | head fall, knee contact, large pose error | | Wallclock | ~1 h / motion | See `/env.yaml` for the full Isaac Lab env spec and `data/agents/deepmimic_g1_ppo_agent.yaml` upstream for PPO config. --- ## Reproduce ```bash git clone https://github.com/vitorcen/isaaclab-experience cd isaaclab-experience && git submodule update --init dependencies/MimicKit # 1. Pull LAFAN G1 retargeting npz (from ember-lab-berkeley) huggingface-cli download ember-lab-berkeley/LAFAN-G1 --repo-type dataset \ --local-dir dependencies/MimicKit/data/motions/g1_extra/ember_lab # 2. Convert + slice (4 × 450-frame center clips) python scripts/lafan_g1_npz_to_mimickit.py # see script for slice ranges # 3. (optional) Restore per-link material on the student USD python scripts/g1_usd_recolor.py export MIMICKIT_G1_USD=$PWD/dependencies/MimicKit/data/assets/g1/g1_textured.usd # 4. Train all 4 motions sequentially (~4 h total on a 4090) scripts/mimickit_train_queue.sh # 5. Eval chain (sequential GUI windows) scripts/mimickit_eval_chain.sh ``` --- ## Known limitations - **`run` plateau at 63 %**: the LAFAN run clip has fast contact + slip; vanilla DeepMimic reward + fixed action std saturates here. Likely fixes: ADD residual, motion-curriculum from walk, or larger action std at start. - **Single-clip overfit**: each policy tracks one clip; no multi-motion conditioning. For composition, see ProtoMotions / OmniH2O. - **AMP vs phase-tracking on dance**: an AMP (adversarial motion prior) baseline on the full 131 s `dance1_subject2` failed — the discriminator plateaued at ~0.98 agent-accuracy and the policy could not keep the choreography's rhythm. Phase-conditioned DeepMimic tracking (used here) is the right tool for high-fidelity dance; AMP fits continuous/loopable skills that don't require exact timing. The 30 s dance is the DeepMimic answer to "longer dance." - **No sim-to-real transfer attempted**: trained in Isaac Lab with raw observations, no domain randomization, no actuator delay model. --- ## License - Code & policy weights: Apache-2.0 - LAFAN1 motion data: CC-BY-NC 4.0 (Ubisoft Montreal) — non-commercial only - G1 USD assets: Unitree ## Citation Upstream papers (cite these instead — this repo is just trained weights, not a publication): ```bibtex @inproceedings{peng2018deepmimic, title = {DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills}, author = {Peng, Xue Bin and Abbeel, Pieter and Levine, Sergey and van de Panne, Michiel}, booktitle = {ACM Trans. Graph. (SIGGRAPH)}, year = {2018} } @article{harvey2020robust, title = {Robust Motion In-betweening}, author = {Harvey, F{\'e}lix G. and Yurick, Mike and Nowrouzezahrai, Derek and Pal, Christopher}, journal = {ACM Trans. Graph. (SIGGRAPH)}, year = {2020} } ```