ndgold commited on
Commit
bd10eea
·
verified ·
1 Parent(s): 4d0c654

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +66 -4
README.md CHANGED
@@ -1,7 +1,69 @@
1
  ---
2
- language: en
3
- pipeline_tag: text-generation
4
  library_name: mlx
5
- tags:
6
- - mlx
7
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen3-1.7B
4
  library_name: mlx
5
+ tags: [mlx, text-simplification, easy-language, accessibility, multilingual, qwen3]
6
+ language: [de, fr, en, es, ar, da, et, fi, hi, it, ja, ko, nl, pt, ru, sk, sv, tr, vi, zh, uk, ro, bg, pl, hr, nb, hu, cs]
7
  ---
8
+
9
+ # Qwen3-1.7B-EasyLanguage (4-bit, MLX)
10
+
11
+ LoRA fine-tune of **Qwen/Qwen3-1.7B** that rewrites live speech transcripts into easy-to-read
12
+ registers across **29 language locales** — German (Leichte Sprache), French (FALC), English (Easy/Plain English), Spanish (Lectura Fácil), Easy-to-Read Arabic (Inclusion Europe / Information for All), Letlæst (Inclusion Europe ETR), and more — for the
13
+ [Live Linguist](https://github.com/ngoldbla/live-linguist) on-device captioner.
14
+ Each locale follows its own national / European Easy-to-Read or plain-language standard.
15
+ Quantized to 4-bit for Apple-silicon inference via MLX.
16
+
17
+ It splits run-ons into short sentences, drops disfluencies, keeps names/numbers, and
18
+ stays in the input language. Trained on a lean prompt (no few-shots) so the register is
19
+ internalized — shorter prompts, lower live-caption latency.
20
+
21
+ ## Evaluation (held-out test set; SARI = simplification quality)
22
+ | lang | SARI ft | SARI stock | chrF ft | LID ft | compliance ft |
23
+ | --- | --- | --- | --- | --- | --- |
24
+ | de | 51.15 | 41.92 | 46.62 | 1.0 | 1.0 |
25
+ | fr | 62.03 | 54.15 | 55.63 | 1.0 | 0.995 |
26
+ | es | 61.66 | 56.9 | 57.79 | 0.995 | 0.99 |
27
+ | en | 62.37 | 52.0 | 57.12 | 1.0 | 1.0 |
28
+ | ar | 52.14 | 51.05 | 50.53 | 1.0 | 0.985 |
29
+ | da | 55.56 | 49.17 | 52.76 | 0.995 | 0.985 |
30
+ | et | 48.41 | 45.59 | 50.68 | 0.995 | 0.99 |
31
+ | fi | 50.57 | 47.25 | 55.96 | 1.0 | 0.995 |
32
+ | hi | 54.54 | 49.22 | 47.64 | 0.975 | 0.845 |
33
+ | it | 56.22 | 49.53 | 51.89 | 1.0 | 0.8866 |
34
+ | ja | 8.35 | 8.36 | 45.49 | 1.0 | 0.985 |
35
+ | ko | 50.1 | 46.48 | 42.85 | 1.0 | 0.975 |
36
+ | nl | 55.07 | 51.81 | 51.44 | 0.985 | 0.94 |
37
+ | pt-BR | 57.78 | 56.87 | 53.97 | 1.0 | 0.89 |
38
+ | pt-PT | 61.2 | 54.3 | 60.99 | 1.0 | 0.965 |
39
+ | ru | 54.46 | 51.04 | 51.46 | 1.0 | 0.97 |
40
+ | sk | 51.54 | 46.89 | 47.71 | 0.995 | 0.96 |
41
+ | sv | 54.95 | 49.53 | 57.89 | 0.975 | 0.96 |
42
+ | tr | 50.57 | 46.35 | 50.6 | 1.0 | 0.99 |
43
+ | vi | 62.5 | 57.28 | 60.86 | 1.0 | 0.865 |
44
+ | zh-CN | 8.64 | 8.8 | 53.85 | 0.995 | 0.99 |
45
+ | uk | 53.12 | 51.75 | 54.79 | 0.995 | 0.995 |
46
+ | ro | 53.69 | 48.06 | 52.88 | 1.0 | 1.0 |
47
+ | bg | 52.74 | 51.7 | 55.05 | 0.99 | 0.975 |
48
+ | pl | 51.55 | 44.47 | 49.84 | 1.0 | 0.995 |
49
+ | hr | 53.55 | 45.38 | 54.55 | 0.995 | 0.995 |
50
+ | nb | 54.3 | 41.35 | 56.16 | 0.995 | 0.99 |
51
+ | hu | 50.74 | 49.82 | 52.34 | 1.0 | 0.995 |
52
+ | cs | 50.44 | 48.65 | 49.23 | 1.0 | 0.995 |
53
+
54
+ ## Usage (MLX)
55
+ ```python
56
+ from mlx_lm import load, generate
57
+ model, tok = load("ndgold/Qwen3-1.7B-EasyLanguage-4bit")
58
+ msgs = [{"role":"system","content":"<framework system prompt>"},
59
+ {"role":"user","content":"Original: <utterance>\nRewritten:"}]
60
+ p = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=False)
61
+ print(generate(model, tok, prompt=p, max_tokens=96))
62
+ ```
63
+
64
+ ## Sources & licenses
65
+ - **Base model:** Qwen3 (Apache-2.0).
66
+ - **German seed data:** [tum-nlp/German4All-Corpus](https://huggingface.co/datasets/tum-nlp/German4All-Corpus) (German Wikipedia, CC BY-SA).
67
+ - **Synthetic pairs (all other languages + de augmentation):** spoken→easy-language pairs generated by Claude (Anthropic) — Leichte Sprache (de), FALC (fr), Easy/Plain English (en), Lectura Fácil (es), and 25 further locales following each language's Easy-to-Read / plain-language standard (Inclusion Europe "Information for All", Selkokieli, Lättläst, やさしい日本語, ISO 24495-1, …). Every pair is filtered by a deterministic validator suite (per-language sentence-length caps, language-ID, fidelity anchoring, number preservation, anti-parroting).
68
+ - Intended for the **Live Linguist** on-device live-caption simplifier. Not a general chatbot.
69
+