flex-pi/flexpi-robotwin

Flex-π — RoboTwin 2.0 (3-camera, 384×320)

Flex-π checkpoint for RoboTwin 2.0, trained jointly on all 50 tasks.

• Paper: arXiv:2608.10860
• Project page: https://flex-pi.github.io/
• Code: https://github.com/geyan21/flex-pi

Results

RoboTwin 2.0, success rate (%) over 50 tasks, as reported in the paper (Table 1). Clean and Randomized are background conditions; the two rows are inference regimes served by these same weights.

Inference regime	Clean	Randomized	Avg.
Action-only	94.5	94.6	94.6
Full joint	94.3	94.8	94.6

K = 4 Euler denoising steps throughout. Full joint additionally denoises the future-video, DINO, and pointmap streams; action-only skips them, trading them for cheaper inference at the same average success rate. Selecting a regime is an inference-time flag — no retraining, no separate weights.

Architecture

A Mixture-of-Transformers pairing a video DiT with an action DiT, coupled by HBridge. Alongside actions the model can denoise three auxiliary streams: future video, DINO features, and pointmaps.

Video expert	Wan2.2-TI2V-5B, 5.00 B params
Action expert	ActionDiT, 1.02 B params
Layers	30 (HBridge: 7 bottom / 16 middle / 7 top)
Semantic encoder	DINOv3 vit_base_patch16_dinov3.lvd1689m, 768-d, frozen
Cameras	cam_high, cam_left_wrist, cam_right_wrist @ 240×320
Composite video	384×320, 33 frames
Action / proprio	14-d each (bimanual ALOHA-AgileX), ConcatLeftAlign
Action : video rate	4:1

Training

Data	2,500 clean + 25,000 randomized demos, all 50 tasks
Epochs	6
Learning rate	1e-4
Precision	bf16

Trained with flex-joint sampling at p = 0.5 on every present and joint flag, with cross-modal prediction enabled for all three streams. That is what lets one set of weights serve any regime in the results table above.

Files

config.yaml                            # architecture + processor; autoloaded by the eval
dataset_stats.json                     # action/state normalization statistics
checkpoints/weights/step_048060.pt     # 12 GB

Keep this directory layout. The eval locates config.yaml and dataset_stats.json by walking up from the checkpoint path.

Usage

This repository holds the policy weights only. The Wan2.2 base components and the ActionDiT backbone are separate downloads, resolved through DIFFSYNTH_MODEL_BASE_PATH — see docs/INSTALL.md and docs/ROBOTWIN.md.

hf download flex-pi/flexpi-robotwin --local-dir runs/flexpi-robotwin

export DIFFSYNTH_MODEL_BASE_PATH="$(pwd)/checkpoints"   # Wan2.2 weights

Then set the checkpoint at the top of scripts/eval_flexpi_robotwin.sh:

CKPT="./runs/flexpi-robotwin/checkpoints/weights/step_048060.pt"
DATASET_STATS="./runs/flexpi-robotwin/dataset_stats.json"

and run:

bash scripts/eval_flexpi_robotwin.sh

The launcher defaults reproduce the full joint row: NUM_INFERENCE_STEPS=4, INSTRUCTION_TYPE=unseen, EVAL_NUM_EPISODES=100, and all six regime flags true. For the action-only row, set the three INFER_JOINT_* flags to false. PHASES=clean,random covers both background conditions.

License

MIT — see LICENSE.

Citation

@article{yan2026flexpi,
  title   = {Flex-$\pi$: A Multi-Stream World-Action Model with Compute Flexibility},
  author  = {Yan, Ge and Liu, Jinghao and Fan, Yuzhi and Cai, Lei and Liao, Minwen
             and Zhang, Jesse and Fox, Dieter},
  journal = {arXiv preprint arXiv:2608.10860},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.10860}
}