---
pipeline_tag: text-generation
license: other
license_name: modified-mit
license_link: https://github.com/MiniMax-AI/MiniMax-M2.7/blob/main/LICENSE
library_name: transformers
base_model: MiniMaxAI/MiniMax-M2.7
tags:
  - mxfp4
  - quantized
  - moe
  - compressed-tensors
---

# MiniMax-M2.7-MXFP4

MXFP4 quantization of [MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) (228B params, 62 layers, 256 experts/layer, top-8 sigmoid routing).

**All 15,872 MoE expert weights quantized to MXFP4.** Attention, layer norms, embeddings, and router weights kept at original precision.

| | Base (FP8) | MXFP4 |
|---|---|---|
| Size | 215 GB | 119 GB |
| Perplexity (WikiText-2) | 4.997 | 5.063 (+1.34%) |
| KL divergence | -- | 0.174 nats/tok (mean), 0.031 (median) |
| Top-1 agreement | -- | 85.8% |
| Compression | 1x | 1.81x |

## Quality Analysis

![Evaluation plots](eval_minimax_m27.png)

**KLD is heavily right-skewed**: median KLD is 0.031 nats/tok (5.6x lower than the mean). 96.6% of tokens have KLD < 1 nat. Only 69 out of 2048 eval tokens show significant divergence -- these are low-confidence positions where the model is already distributing probability across many candidates.

**Error is diffuse across experts**: per-expert quantization error analysis of all 15,872 experts shows extremely uniform error (std=0.000271, range 0.110--0.116). The 256-expert top-8 architecture is inherently quantization-tolerant -- each expert contributes ~1/8th of the output, so MXFP4 errors average out across the mixture.

## Format

MXFP4 block-32 quantization in [compressed-tensors](https://github.com/neuralmagic/compressed-tensors) format:

- `weight_packed`: uint8 `[out, in//2]` -- two 4-bit values packed per byte (even=low nibble, odd=high nibble)
- `weight_scale`: uint8 e8m0 `[out, in//32]` -- one shared exponent per block of 32 elements

Quantization is **calibration-free** (no calibration data needed). MXFP4 block-32 scaling is deterministic -- the shared exponent is derived directly from the max magnitude in each block.

Quantized with [quant4](https://github.com/olka-fi/quant4).

## Serving

### vLLM

Requires vLLM with MXFP4 compressed-tensors support and the CUTLASS FP4xFP8 kernel for Blackwell GPUs:

```bash
vllm serve /path/to/MiniMax-M2.7-MXFP4 \
    --tensor-parallel-size 2 \
    --trust-remote-code \
    --max-num-seqs 512 \
    --enable-chunked-prefill \
    --max-num-batched-tokens 16384 \
    --kv-cache-dtype fp8
```

### Memory Budget

At 119 GB, this fits on 2x DGX Spark (2x 120 GB = 240 GB total) with ~100 GB remaining for KV cache, enabling long-context or multi-session serving that would be impossible with the 215 GB FP8 original.

## Evaluation Details

Evaluated on WikiText-2 test set (2048 tokens) using layer-by-layer streaming inference with `MiniMaxLayerRunner`. Both models run identical forward passes; logits compared token-by-token.

| Metric | Value |
|---|---|
| Perplexity (ref) | 4.997 |
| Perplexity (MXFP4) | 5.063 |
| PPL degradation | +1.34% |
| KL(ref\|\|target) mean | 0.174 nats/tok |
| KL(ref\|\|target) median | 0.031 nats/tok |
| KL(ref\|\|target) P95 | 0.824 nats/tok |
| KL(ref\|\|target) P99 | 1.827 nats/tok |
| Top-1 agreement | 85.8% |
| Tokens with KLD > 1 nat | 69 / 2048 (3.4%) |

## Acknowledgments

Based on [MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) by [MiniMax](https://www.minimax.io/). Original model license applies.