Aijuus/unlimited-ocr-bf16

Unlimited-OCR — bf16 .rad container for radiance

baidu/Unlimited-OCR converted, unchanged, into a single .rad container for the radiance engine: the 12-layer Llama-MoE decoder with its tied head, and the DeepEncoder vision tower (SAM, CLIP, projector, image_newline, view_seperator), at the checkpoint's own bf16 precision. The container also holds the tokenizer and the chat template, so the server needs nothing else.

No weights are quantised. This is the reference-precision model; it is the baseline the engine's tiled (gundam), single-page (base) and multi-page paths are verified against the HF implementation.

The decoder's q, k and v projections are stored fused as blk.N.attn_qkv.weight (one stacked-rows GEMM), as the unlimitedocr_bf16 plugin declares them since radiance 1.9.0 rev 859efc7. Use that revision or newer: earlier plugins expect separate blk.N.attn_q/attn_k/ attn_v entries and will refuse this container with 12 declared weights are absent.

Files

file	size
unlimited-ocr-bf16.rad	6.22 GiB	the container: decoder, vision tower, tokenizer, chat template
unlimited-ocr-bf16.recipe	3 KiB	the rad-convert recipe that produced it from the bf16 checkpoint

Download and serve

One AMD card (RDNA3/RDNA4, ROCm 7.x). The server keeps ~7 GiB of VRAM: ~6.2 GiB of weights plus the ~0.7 GiB vision tower on the card, with the fp8 KV cache.

mkdir -p /srv/models/rad && cd /srv/models/rad
curl -LC - -O https://huggingface.co/Aijuus/unlimited-ocr-bf16/resolve/main/unlimited-ocr-bf16.rad

From source:

build/bin/radiance --model /srv/models/rad/unlimited-ocr-bf16.rad \
    --tp 1 --max-num-seqs 8 --max-model-len 32768 --kv-cache-dtype fp8

With Docker, use the compose file unlimited-ocr-bf16.yaml from the radiance repository; it mounts this file at /models/unlimited-ocr-bf16.rad and passes the flags above.

Use

/v1/chat/completions with the picture as an image_url part before the text part (the reference prompt is <image> then the instruction):

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"Unlimited-OCR",
       "messages":[{"role":"user","content":[
         {"type":"image_url","image_url":{"url":"data:image/png;base64,..."}},
         {"type":"text","text":"document parsing."}]}],
       "images_config":"gundam","temperature":0,"max_tokens":2048}'

• images_config: "gundam" — one picture, cropped into 640-px tiles plus the 1024-px global view. {"base_size":1024,"image_size":640,"crop_mode":true} is the same thing spelled out.
• images_config: {"base_size":1024,"image_size":640,"crop_mode":false} — one 640-px page.
• Multi-page: send one image part per page with the same base mode; the pages are parsed in one prompt.
• The checkpoint's generation config bans repeated 35-grams over a 128-token window; the engine applies it by default. The generated-token ring (128) is the checkpoint's own attention window.

Verification

• Encoder output against the HF implementation on the reference document: rows cosine 0.9997 (bf16 kernel-order noise), tile stage cosines ≥ 0.9994.
• Tiled document parse vs HF at temperature 0: identical det blocks and text structure, det coordinates within 1 px.
• Two distinct pages vs HF infer_multi: 0.9917 text similarity, boxes within 2 px.
• Eight distinct pages: 76 response rows on each side, 64 exactly equal; the remainder are single-value misreads, the engine being correct where HF misreads a page.

License

The base checkpoint is MIT-licensed (baidu/Unlimited-OCR); this converted container is redistributed under the same license, with no weight changes.