MLX
mlx-vlm
lora
receipt-extraction

Technical report: Read paper · PDF · BibTeX · All models

Collection: OCR and document extraction

Xiaomi OCR receipt extraction adapter

GGUF: Lucasgitguy-co/Xiaomi-OCR-0-CORD-LoRA-GGUF (LoRA for llama.cpp).

Experimental MLX 4-bit LoRA adapter for mapping receipt images to CORD JSON. Requires the original Xiaomi-OCR-0 base and MLX-VLM; this is not a standalone Transformers checkpoint or a Hugging Face PEFT-format adapter.

Training recipe: supervised assistant-only loss, frozen base/vision weights, rank 8, alpha 16, learning rate 5e-5, batch size 1, gradient accumulation 8, up to two passes over the prepared training split. See selected_checkpoint.txt for the checkpoint selected for publication. Images have a maximum side of 640 pixels. Examples exceeding 1,536 processed tokens are excluded entirely. See preparation.json for retained split sizes, train.json for configuration, and requirements-lock.txt for the installed environment.

Data and attribution

Uses NAVER CLOVA's CORD v2, distributed under CC BY 4.0. Images are resized and gt_parse annotations are converted to assistant JSON targets. Official train/validation/test splits remain separate. Base model: Xiaomi-OCR-0, published under Apache 2.0. Training data are not included in this adapter repo.

Evaluation and limitations

See baseline.json, quantized-baseline.json, and finetuned.json for measured results after running evaluation. No improvement is claimed in advance. Metrics cover the resized, length-filtered test subset, not the full official CORD benchmark. The base model's pretraining overlap with CORD is unknown. Results do not establish improved general OCR. Small print can become unreadable at this image resolution; JSON fields can be missing or hallucinated.

Load with MLX-VLM

Install the environment described in RUN.md, download this adapter repository, and use:

from mlx_vlm import load
from mlx_vlm.trainer import prepare_model_for_training

model, processor = load("xiaomi-research/Xiaomi-OCR-0")
model = prepare_model_for_training(model, checkpoint_path="/path/to/adapter/weights.safetensors")
model.eval()

Keep adapter_config.json beside weights.safetensors; its metadata restores the quantized base representation. See evaluate.py for prompt and image usage.

Licence

The adapter is Apache-2.0 (see LICENSE and NOTICE). The base model Xiaomi-OCR-0 is Apache-2.0. The CORD v2 training data is CC BY 4.0, by NAVER CLOVA, and is not included.

Downloads last month

-

Downloads are not tracked for this model. How to track
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Lucasgitguy-co/Xiaomi-OCR-0-CORD-MLX-LoRA

Adapter
(2)
this model

Dataset used to train Lucasgitguy-co/Xiaomi-OCR-0-CORD-MLX-LoRA

Collections including Lucasgitguy-co/Xiaomi-OCR-0-CORD-MLX-LoRA