NLS Advocates index-card detector (RetinaNet)

An object detector that locates the card region in a photograph of a National Library of Scotland Advocates' manuscript-catalogue index card, so the image can be cropped before downstream transcription by a multimodal model. Single foreground class (card).

Fine-tuned by NLS on its own digitised collection. It supersedes archival-index-card-detector.

Scope. These weights are trained on one collection's scans. They are a reference implementation and a starting point, not a general-purpose card detector: expect to train your own on your own material. See Limitations.

Model details

Architecture: torchvision retinanet_resnet50_fpn_v2 (RetinaNet, ResNet50-FPN v2), num_classes=2 (background + card). BSD-3-Clause.
Base model / initialisation: ResNet50 backbone initialised from torchvision's ImageNet-pretrained weights (ResNet50_Weights.IMAGENET1K_V1); the FPN and detection heads were trained from scratch.
Input: an RGB scan; the detector resizes so the longest side is 800 px.
Output: the highest-confidence card box as [x, y, w, h] in original-image pixel coordinates (confidence threshold 0.5); None if no card is found.
Size: ~146 MB (card_detector_retinanet.pt, a torchvision state_dict).
Framework: PyTorch / torchvision.

Intended use

Crop index-card scans to the card region before sending them to a multimodal model for structured transcription. In the pipeline it is paired with a deterministic, ML-free verso pre-filter that drops blank card-backs before the detector runs.

Limitations

Collection-specific. Tuned to one card collection, including its mid-collection scan-config change from a black binding band to a manilla background. Another library's cards, scanner, lighting, or card stock may produce materially worse boxes.
Not a general document-layout detector. One class, one document type.
Fails safely, not loudly. In the reference integration a missing or unusable detector falls back to a fixed-margin uniform crop rather than erroring — check your pipeline reports which path it took before trusting a large run.
Evaluation below is on held-out images from the same collection, so it measures fit to this material, not transferability.

Training data

1,358 human-verified labels (1,158 train / 200 val), drawn from across the whole collection with deliberate coverage of the late-range (image 1127+) manilla scans. Includes both card boxes (positives) and verso / booklet-end negatives.
Labels were produced by SAM2 box pre-labelling → human correction in Label Studio (SAM2 was a labelling aid only; it is not part of this model).
Source images and labels are NLS's own material.

Evaluation

Validation set (200 images, 184 ground-truth card boxes):

Metric	Value
mAP@50	100.0%
mAP@75	1.000
mAP@50–95	0.948
False positives on 16 hand-labelled negatives (versos / booklet-ends)	0

These figures are in-distribution: the validation images are held out, but come from the same collection, scanner and card stock as the training set. Read them as "fits this material well", not as an expected score on your cards. No cross-collection evaluation has been done.

Usage

Directly:

import torch
from PIL import Image
import torchvision.transforms.functional as TF
from torchvision.models.detection import retinanet_resnet50_fpn_v2

model = retinanet_resnet50_fpn_v2(weights=None, num_classes=2)
model.load_state_dict(torch.load("card_detector_retinanet.pt", map_location="cpu"))
model.eval()

img = Image.open("card.jpg").convert("RGB")
W, H = img.size
s = min(1.0, 800 / max(W, H))
im = img.resize((int(W * s), int(H * s))) if s < 1.0 else img
with torch.no_grad():
    out = model([TF.to_tensor(im)])[0]
# best box (conf > 0.5), rescaled to original coords:
i = int(out["scores"].argmax())
x1, y1, x2, y2 = (v / s for v in out["boxes"][i].tolist())

Training procedure

Trained on an AMD Strix Halo iGPU (Radeon 8060S, gfx1151) via the TheRock ROCm nightly PyTorch. 30 epochs, AdamW, lr 1e-4, batch 4, images at 800 px. Stability guards were needed for the iGPU (skip any batch with non-finite loss or gradient; best-checkpoint only). The full recipe is SAM2 pre-labelling → verso filter → Label Studio → ROCm training.

Licensing

This model (weights): Apache-2.0.
Code (pipeline + training scripts): Apache-2.0.
Training images and box labels: not published with this model.

Citation

National Library of Scotland. NLS Advocates index-card detector (RetinaNet). Trained on the NLS Advocates' manuscript-catalogue index cards.

Downloads last month: -; Downloads are not tracked for this model. How to track