Aegis

Urdu span detector for personally identifiable information. This repository is the privacy-filter checkpoint. It is not the female TTS voice mahwizzzz/piper-voice-ur-aegis-female.

Purpose

Span-level detection of names, phone numbers, email addresses, addresses, dates, and account or CNIC-style identifiers in Urdu text, for assisted redaction. A demo that downloads this repository is mahwizzzz/de-identify.

Upstream attribution

finetune_summary.json records base_checkpoint as openai/privacy-filter. The same file records model_id as mahwizzzz/urdu-legal-privacy-filter, which is not this repository's id. config.json sets model_type to privacy_filter.

Span labels in config.json: O, private_person, private_phone, private_email, private_address, private_date, account_number.

Data provenance

Training data are mahwizzzz/uopf-sipar. That dataset repository is private, so the public cannot recheck it from the Hub.

Its dataset card tags the corpus synthetic and lists these train-split sources:

Source	Records
synthetic_multidomain_urdu_v1	32,000
llm_synthetic_realistic_urdu_v1	26,640
synthetic_legal_urdu_v1	15,396
synthetic_finance_urdu_v1	5,964

Those four counts sum to 80,000. The dataset card's split sizes, also repeated below, are 80,000 / 10,000 / 10,000. Generator model ids, prompts, and any human review are not in this model repository. The dataset card declares CC BY 4.0. This model card declares Apache-2.0. The model license does not relicense the dataset.

Split	Records
Train	80,000
Validation	10,000
Test	10,000

Train-domain counts stated on both cards: legal 21,986; finance 13,850; telecom 6,315; healthcare 5,410; travel 4,876; social_media 4,845; news_channel 4,818; government 3,965; ecommerce 3,923; insurance 3,382; education 3,319; hr_jobs 3,311.

Training

Figures below are copied from finetune_summary.json (updated_at 2026-05-22T08:26:21.708110Z). The checkpoint name in that file is uopf_sipar_prod_20260520_090709.

Item	Value
Epochs	3 (best epoch 2)
Batches	40,000
Tokens seen	3,677,980
Learning rate	1e-5
Batch size	2
Gradient accumulation	1
Weight decay	0.01
Max grad norm	1.0
Context window	128000
Tokenizer	o200k_base
Precision	bfloat16
Device	CUDA (GPU model not named)
Final train loss	0.000593
Final validation loss	0.001713
Best validation loss	0.001593
Validation token accuracy	0.9997

Evaluation

Two different records are in this repository. They are not the same result.

eval_metrics.json has an empty test object. Its validation block points at data/validation.jsonl and reports span-detection F1 0.9994091722694807, precision 0.9997770246164823, and recall 0.9990415905139749. Per-class span F1 in that file: private_person 0.9998982550742649, private_phone 1.0, private_email 0.9997391077484998, private_address 0.9978453383767306, private_date 1.0, account_number 0.9997052755673446. That file does not contain a base-model comparison.

The table below is retained from the previous card. It is not reproduced by eval_metrics.json. The card text described the metric as same-label overlap span F1 on a held-out test split, with exact span F1 requiring label and character offsets to match.

Metric	Base openai/privacy-filter	Fine-tuned model	Absolute gain
Overlap detection F1	0.517	0.957	+0.440
Overlap precision	0.497	0.997	+0.500
Overlap recall	0.538	0.920	+0.382
Macro F1	0.468	0.931	+0.463
Exact span F1	0.021	0.406	+0.385

Entity	Base F1	Fine-tuned F1	Absolute gain
private_person	0.509	0.966	+0.457
private_phone	0.386	0.755	+0.369
private_email	0.646	0.999	+0.353
private_address	0.015	0.986	+0.971
private_date	0.657	0.989	+0.332
account_number	0.597	0.893	+0.296

finetune_summary.json also records an evaluation_10k loss of 0.0015, 10,000 examples, 459,982 tokens, and 1004.19 inference tokens/sec. The script, split, and hardware for that timing are not named.

Limitations

• The public Hub copy of the training set cannot be inspected.
• The dataset card describes the text as synthetic. Performance on real documents is not reported in eval_metrics.json.
• The retained test table and the shipped validation JSON disagree. Treat them as separate, unreconciled records.
• The de-identify Space states that redaction is not a substitute for human review, and that Roman Urdu and mixed script are unsupported. Those statements are interface text, not rows in eval_metrics.json.
• The Space also says account_number is the weakest class. In the retained table above it is the lowest fine-tuned overlap F1. In eval_metrics.json validation span F1 it is not the lowest class.

License

Apache-2.0, as declared in the metadata. A LICENSE file was not in the repository file list retrieved for this revision.

Usage

This repository does not include an inference script or a Transformers modeling file. config.json uses model_type: privacy_filter. The de-identify Space downloads mahwizzzz/aegis and decodes BIOES labels with a Viterbi routine implemented in that Space, not in this repository. ONNX exports are under onnx/. Their input contract is not documented here.

Citation

@misc{mahwiz2026urdu_privacy_filter,
  title        = {Aegis},
  author       = {Mahwiz Khalil},
  year         = {2026},
  howpublished = {Hugging Face Model Hub},
  url          = {https://huggingface.co/mahwizzzz/aegis}
}