AREX2-27B-3.80bpw-fp16

This is a 3.80 bpw (bits per weight) quantized version of BAAI/AREX-2, optimized for Apple Silicon via MLX.

📋 Model Details

Property	Value
Base Model	BAAI/AREX-2
Architecture	Qwen3.5 (MoE)
Parameters	27B (3.8B active)
Quantization	3.80 bpw (per-layer, mixed precision)
Format	MLX (safetensors)
Context Length	32,768 tokens
License	Other (see base model)

🧠 Features

• Per-layer Quantization: Mixed precision (3-bit/4-bit/5-bit/8-bit) for optimal quality/size tradeoff
• MLX Native: Runs efficiently on Apple Silicon (M1/M2/M3/M4) without conversion
• Multi-modal Ready: Supports vision-language tasks (image/text-to-text)

🎨 Creative Demo: One-Shot SVG Generation

This model can generate animated SVG illustrations in a single prompt. All three examples below use the same prompt and one-shot setting, differing only in reasoning effort:

"Write an animated svg for a pelican riding a motorcycle."

🔴 High Reasoning Effort (xhigh)

Reasoning effort: xhigh — Maximum thinking, most detailed output.

🟡 Medium Reasoning Effort (medium)

Reasoning effort: medium — Balanced thinking, good quality/speed tradeoff.

🟢 Low Reasoning Effort (low)

Reasoning effort: low — Minimal thinking, fastest generation.

All three are one-shot generations — no iteration, no refinement. The model produces complete, animated SVGs with spinning wheels, bobbing rider, flapping wings, puffing exhaust, scrolling background, and dynamic lighting.

🚀 Quick Start (MLX)

# Install MLX
pip install mlx mlx-lm

# Run inference
python -m mlx_lm.generate \
    --model ~/llm_model/mlx-community/AREX2-27B-3.80bpw-fp16 \
    --prompt "Explain quantum computing in simple terms" \
    --max-tokens 512

With Chat Template

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("~/llm_model/mlx-community/AREX2-27B-3.80bpw-fp16")

messages = [
    {"role": "user", "content": "Write a haiku about coding"}
]

# Apply chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
sampler = make_sampler(temp=0.7, top_p=0.9)
response = generate(model, tokenizer, prompt=prompt, sampler=sampler, max_tokens=256)
print(response)

📦 Quantization Details

This model uses per-layer mixed-precision quantization (3.80 bpw average):

• 3-bit: Embedding tokens, most gate_proj/up_proj, some down_proj, lm_head
• 4-bit: Most attention projections (qkv/z/out/o_proj), some down_proj
• 5-bit: Selected down_proj layers (early layers 0-2, late layers 56-63)
• 8-bit: Key/Value projections every 4 layers (layers 3, 7, 11, 15... 63)

Quantization config: affine, group_size=64 (embed_tokens uses group_size=128)

Result: ~14 GB disk size vs ~54 GB for fp16, with minimal quality loss.

🔗 Related Models

• Base: BAAI/AREX-2
• Original Qwen3.5: Qwen/Qwen3.5-27B
• Other quantizations: Check NapYang's profile for more variants

⚠️ Limitations

• Inherits all limitations of the base model
• Quantization may affect edge-case reasoning
• Requires Apple Silicon + MLX for optimal performance

📜 Citation

If you use this model, please cite the base model:

@misc{arex-2,
  author = {BAAI},
  title = {AREX-2},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/BAAI/AREX-2}
}

---

Quantized and uploaded by NapYang