XiaomiRobotics/Xiaomi-Robotics-0-Calvin-ABC_D

Xiaomi-Robotics-0

Xiaomi-Robotics-0 is a state-of-the-art Vision-Language-Action (VLA) model with 4.7B parameters, specifically engineered for high-performance robotic reasoning and seamless real-time execution.

• Paper: Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution
• Project Page: https://xiaomi-robotics-0.github.io/
• Repository: https://github.com/XiaomiRobotics/Xiaomi-Robotics-0

Model Description

Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities. The model is optimized for asynchronous execution to address inference latency and is fully compatible with the Hugging Face transformers ecosystem.

Key Features:

• Strong Generalization: Pre-trained on diverse cross-embodiment trajectories.
• Real-Time Ready: Optimized for fast and smooth real-time rollouts on consumer-grade GPUs.
• Flexible Deployment: Built on top of transformers and supports Flash Attention 2.

Quick Start: Deployment

The following code demonstrates how to load the model and generate actions using multi-view observations.

import torch
from transformers import AutoModel, AutoProcessor

# 1. Load model and processor 
model_path = "XiaomiRobotics/Xiaomi-Robotics-0"
model = AutoModel.from_pretrained(
    model_path, 
    trust_remote_code=True, 
    attn_implementation="flash_attention_2", 
    dtype=torch.bfloat16
).cuda().eval()
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=False)

# 2. Construct the prompt with multi-view inputs
language_instruction = "Pick up the red block."
instruction = (
    f"<|im_start|>user
The following observations are captured from multiple views.
"
    f"# Base View
<|vision_start|><|image_pad|><|vision_end|>
"
    f"# Left-Wrist View
<|vision_start|><|image_pad|><|vision_end|>
"
    f"Generate robot actions for the task:
{language_instruction} /no_cot<|im_end|>
"
    f"<|im_start|>assistant
<cot></cot><|im_end|>
"
)

# 3. Prepare inputs
# Assuming `image_base`, `image_wrist`, and `proprio_state` (numpy array) are already loaded
inputs = processor(
    text=[instruction],
    images=[image_base, image_wrist], # [PIL.Image, PIL.Image]
    videos=None,
    padding=True,
    return_tensors="pt",
).to(model.device)

# Add proprioceptive state and action mask
robot_type = "libero" # Select based on your robot/env configuration
inputs["state"] = torch.from_numpy(proprio_state).to(model.device, model.dtype).view(1, 1, -1)
inputs["action_mask"] = processor.get_action_mask(robot_type).to(model.device, model.dtype)

# 4. Generate action 
with torch.no_grad():
    outputs = model(**inputs)
    
# Decode raw outputs into actionable control commands
action_chunk = processor.decode_action(outputs.actions, robot_type=robot_type)
print(f"Generated Action Chunk Shape: {action_chunk.shape}")

Citation

@misc{robotics2026xiaomi,
  title        = {Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution},
  author       = {Xiaomi Robotics},
  howpublished={\url{https://xiaomi-robotics-0.github.io}},
  year         = {2026},
  note={Project Website}
}