hugo/protocolos-clinicos-br-cpt_lora-1gen-14b

CPT via LoRA (1 generator, GPT-4.1-mini) — Qwen2.5-14B-Instruct

LoRA adapter (r=32, α=64) trained on Qwen2.5-14B-Instruct with continual pre-training on synthetic data from a single generator (GPT-4.1-mini). Ablation of full fine-tuning vs LoRA for CPT, single-generator regime.

• Base model: Qwen/Qwen2.5-14B-Instruct
• Type: LoRA adapter (PEFT)

Test-split accuracy

Benchmark	Accuracy
HealthBench-BR	67.2%
PCDT-QA	65.2%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-14B-Instruct", torch_dtype="auto", device_map="auto")
tok  = AutoTokenizer.from_pretrained("hugo/protocolos-clinicos-br-cpt_lora-1gen-14b")
model = PeftModel.from_pretrained(base, "hugo/protocolos-clinicos-br-cpt_lora-1gen-14b")

Intended use & limitations

Research model for studying domain adaptation of LLMs to Brazilian clinical guidelines. Not a certified medical device. Even at the best accuracy reported in the paper, residual errors may involve consequential details (dosages, contraindications). Use only under qualified professional supervision.

Citation

See the paper and code at the project repository:

Code & paper: https://github.com/hugoabonizio/clinical-protocols-br