nakamura196/yolov8-ndl-layout

yolov8x-ndl-layout

YOLOv8x fine-tuned for layout analysis of Japanese pre-modern printed and manuscript books (古典籍), trained on nakamura196/ndl-layout-dataset. It detects five region types per page: the overall text block, handwritten regions, typographic (printed) regions, illustrations, and seals/stamps.

⚠️ Read the Evaluation section before citing any number from this model. The evaluation split used during training leaks at the document level: the same digitised item (PID) appears in both training and evaluation. The reported scores are therefore optimistic and are not a valid estimate of generalisation to unseen documents. An earlier version of this card reported "mAP 85.4% / IoU 78.2%". Those figures have been removed: 85.4% corresponds to no measurement of this model, and "IoU 78.2%" was the mAP@50-95 value (0.781) under the wrong name.

Model Details

• Developed by: Satoru Nakamura (Historiographical Institute, The University of Tokyo)
• Model type: Object detection (YOLOv8x, 5 classes)
• Trained: 2024-05-19, Ultralytics 8.2.18, single GPU (Google Colab)
• Input size: 640 px
• Fine-tuned from: yolov8x.pt (Ultralytics COCO weights)
• License: AGPL-3.0, inherited from the Ultralytics YOLOv8 weights this model was fine-tuned from. The training data is Public Domain Mark (NDL Digital Collections); the weights are not.

Classes

id	name	rectangles in dataset	share
0	1_overall	1,219	4.7%
1	2_handwritten	13,851	53.6%
2	3_typography	9,262	35.9%
3	4_illustration	1,119	4.3%
4	5_stamp	369	1.4%

5_stamp occurs on only 194 of 1,219 pages; treat its scores as noisy.

Uses

Direct Use

• Region detection on digitised Japanese pre-modern books, as a pre-processing step for OCR (e.g. feeding 2_handwritten / 3_typography regions to a line or character recogniser)
• Bulk structural description for digital archives and IIIF pipelines

Out-of-Scope Use

• Modern printed material, newspapers, magazines, manuscripts in other scripts. The training data is exclusively NDL-DocL pre-modern books; nothing else was seen.
• Tables, figures-with-captions, headers/footers, reading order — these are not classes here.
• Real-time or low-latency use (YOLOv8x is the largest variant).

Bias, Risks, and Limitations

• Very narrow document population. Training used 1,219 pages from only 89 digitised items. Layout conventions vary by publisher, period and genre; 89 items do not cover that variety.
• Images were downscaled. Every training image had its long side reduced to 640 px (most are 640×480). The model has never seen full-resolution scans, and small elements — seals, inline annotations — may behave differently on high-resolution input.
• Class imbalance. 5_stamp is 1.4% of all rectangles.
• The published metrics are not a generalisation estimate (see below).

How to Get Started with the Model

from ultralyticsplus import YOLO, render_result
import os

# load model
model = YOLO('nakamura196/yolov8-ndl-layout')

# set model parameters
conf_threshold = 0.25  # NMS confidence threshold
iou_threshold = 0.45   # NMS IoU threshold

# set image
img = 'https://dl.ndl.go.jp/api/iiif/2534020/T0000001/full/full/0/default.jpg'

# perform inference
results = model.predict(img, conf=conf_threshold, iou=iou_threshold, device="cpu")
render = render_result(model=model, image=img, result=results[0])

os.makedirs('results', exist_ok=True)
render.save('results/1.jpg')

Training Details

Training Data

nakamura196/ndl-layout-dataset — 1,219 page images with 25,820 annotated rectangles, derived from the NDL-DocL dataset (古典籍 subset only). Images are downscaled to a 640 px long side.

Training used the dataset's original data/data.yaml split: 853 / 244 / 122 images (train / val / test), assigned randomly by page.

Training Procedure

Started from COCO-pretrained yolov8x.pt; no layers frozen.

epochs	100 (completed; no early stopping, patience=100)
image size	640
batch size	31
optimizer	auto (Ultralytics default selection), lr0=0.01
seed	0
mixed precision	AMP (fp16) enabled
augmentation	fliplr=0.5, scale=0.5, erasing=0.4, hsv_h=0.015, mosaic=0.0, degrees=0.0, flipud=0.0
Ultralytics version	8.2.18

Evaluation

The split leaks at the document level

The dataset's original split was drawn randomly over pages, not over documents. The 1,219 pages come from only 89 items (PIDs), so the same item lands in several splits:

• of the 71 PIDs in train, 34 also appear in val and 32 also appear in test
• 117 of the 122 test pages come from an item that the model saw pages of during training

Pages from the same item share paper, typeface, block layout, ruling and seals, so the model can score highly by recognising the item rather than by generalising. Every number below is inflated by that leakage.

Reported scores (page-level split — leaky)

Scores stored inside the released checkpoint, measured on the 244-page val split at the final (100th) epoch:

metric	value
precision	0.951
recall	0.899
mAP@50	0.935
mAP@50-95	0.781

Measured here on the 122-page test split (Ultralytics 8.4.7, imgsz=640, CPU):

class	P	R	mAP@50	mAP@50-95
all	0.943	0.938	0.964	0.794
1_overall	0.997	1.000	0.995	0.995
2_handwritten	0.994	0.954	0.992	0.807
3_typography	0.964	0.933	0.971	0.756
4_illustration	0.942	0.934	0.954	0.775
5_stamp	0.820	0.868	0.907	0.636

Note that the test scores come out higher than the val scores above. With 122 pages drawn from items the model was trained on, that is what leakage plus a small sample looks like — not evidence that the model is stronger than reported. 1_overall is near-perfect because it is the single page-sized block on every page, which is a trivial detection target.

Why this model was not re-evaluated on a document-disjoint split

The dataset now ships a document-disjoint split (data/data_pid.yaml, 848 / 248 / 123 images over 67 / 12 / 10 PIDs) so that future models can be measured honestly. It cannot rescue this checkpoint. Only 5 of the 1,219 pages belong to items entirely absent from this model's training and validation data — too few to measure anything. There is no held-out, document-disjoint data for these weights.

To obtain a trustworthy figure, the model has to be retrained on data_pid.yaml and evaluated on its test split. Until that is done, treat this model as a working tool and not as a benchmarked result.

Model Card Contact

Satoru Nakamura — nakamura@hi.u-tokyo.ac.jp