eric8810/StartLux-Decision-v1.1-9B-AutoThink-Mixed-MXFP4-INT8

StartLux-Decision-v1.1-9B-AutoThink — Mixed MXFP4 + INT8

Community pre-baked mixed-precision checkpoint for StartLux-Prism. It is not an official StartLux Labs release.

This is a single tensor bundle, not a transformers or llama.cpp checkpoint. It needs StartLux-Prism's --checkpoint support (patch included).

HF may show a GGUF tag because the repo also contains the official mmproj-*.gguf vision tower; the mixed weights themselves are prism_mixed.safetensors, not a GGUF model.

What is already baked

Part	Precision
MLP gate/up, down	MXFP4 (e2m1 weights and activations, e8m0 scale per 32), SmoothQuant alpha 0.5
Attention qkv, o	W8A8, SmoothQuant alpha 0.75
GDN inp, out	W8A8, SmoothQuant alpha 0.75
Embedding and full output head	Q8_0 (int8 values, fp16 block scales)
Norms, GDN conv/state, q/k norms	fp32, unchanged

The int8 projections are already quantized in prism_mixed.safetensors. No source GGUF and no int8_calib.pt are needed at startup, and no calibration or quantization runs on the user's machine.

The output head is the full vocabulary (248320 rows), so AutoThink thought decoding works; it is not just the 26 letter rows.

Files

File	Purpose
prism_mixed.safetensors	all weights (6.36 GiB)
prism_checkpoint.json	config, source hashes, quantization description
config.json, decision_config.json, tokenizer files	model metadata and tokenizer (unchanged from the official release)
mmproj-StartLux-Decision-v1.1-9B-AutoThink-F16.gguf	vision tower for image inputs (unchanged, F16)
prism-checkpoint.patch	StartLux-Prism patch adding --checkpoint
reference/load_checkpoint.py	standalone reader / verifier that does not import StartLux-Prism
docs/checkpoint-format.md	tensor names, dtypes and loader notes
SHA256SUMS	checksums

Quick start

Required: an NVIDIA Blackwell GPU (SM120 or SM121) for the native MXFP4 kernels; Python 3.12; StartLux-Prism's normal requirements. The patch was generated against StartLux-Prism main at 7c158c9 (2026-10-25).

M=/path/to/this/repo
git clone https://github.com/StartLuxLabs/StartLux-Prism
cd StartLux-Prism
git checkout 7c158c9
patch -p1 < "$M/prism-checkpoint.patch"
pip install -r requirements.txt
pip install safetensors        # only needed for the checkpoint loader

python -m startlux_prism.server \
  --checkpoint "$M" \
  --model-dir "$M" \
  --port 8092

If the patch does not apply because upstream moved, use reference/load_checkpoint.py and docs/checkpoint-format.md to wire the bundle into the current tree; the loader code is in startlux_prism/engine/checkpoint.py in the patch.

One request:

curl -s localhost:8092/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "I was charged twice and want a refund.",
  "questions": {"intent": {"type": "choice", "instructions": "Choose the customer intent.",
    "criteria": {"billing": "A payment or refund issue", "technical": "A malfunction or setup issue"}}}
}'

Images work because the official mmproj-*.gguf is in the same folder; pass --mmproj none for text only.

Validation

• Bit parity with the GGUF + calibration build: on 6 fixed requests (all seven evaluation suites except wide33), the letter logits are identical: max |dlogit| = 0, 0 differing questions.
• Server smoke test: one-pass request and an 8-token thought both return normally with the checkpoint.
• Startup: about 2.3 s and 8.2 GiB of GPU memory at --max-tokens 2048 on an RTX 5060 Ti 16 GB.

Quality and speed are from the StartLux-Prism research notes (docs/fp4.md, 2026-10-25), on a 217-item DecisionPivot subset:

Build	Gold	1k latency (RTX 5060 Ti)
int8 (attention/GDN/MLP all W8A8)	66.8%	183 ms
this checkpoint (MLP MXFP4 + int8 attention/GDN)	66.8%	146.5 ms
int4 W4A8 (Q4_K_M)	64.1%	232 ms

The 217-item noise is about ±3 points, so the equality with int8 is not a high-confidence win; a 500–600 item run is still needed. This is a research result, not a leaderboard submission.

License

The original StartLux-Decision weights are CC BY-NC 4.0; this quantized derivative is distributed under the same license. See LICENSE and NOTICE. Changes: the projections and MLP were quantized as described above by eric8810; the tokenizer/config files and the vision tower are unchanged from the official release.

Commercial use requires a separate license from StartLux Labs (contact@startlux.com).

Citation

@misc{startlux2026decision,
  title         = {StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions},
  author        = {Zhuang, Xialie and Su, Zunhai and Pan, Jia and Wong, Ngai and Chen, Guanhua and Kuo, Chuan-Wei},
  year          = {2026},
  eprint        = {2610.17309},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.17309}
}