oddadmix commited on
Commit
5933d29
·
verified ·
1 Parent(s): 4106921

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +190 -36
README.md CHANGED
@@ -1,59 +1,213 @@
1
  ---
2
- base_model: unsloth/qwen2-vl-2b-instruct-unsloth-bnb-4bit
3
- library_name: transformers
4
- model_name: Qari-OCR-0.2.2.1-VL-2B-Instruct
5
  tags:
6
- - generated_from_trainer
7
  - unsloth
 
8
  - trl
9
- - sft
10
- licence: license
 
 
 
 
 
 
 
 
11
  ---
12
 
13
- # Model Card for Qari-OCR-0.2.2.1-VL-2B-Instruct
14
 
15
- This model is a fine-tuned version of [unsloth/qwen2-vl-2b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2-vl-2b-instruct-unsloth-bnb-4bit).
16
- It has been trained using [TRL](https://github.com/huggingface/trl).
17
 
18
- ## Quick start
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
- ```python
21
- from transformers import pipeline
22
 
23
- question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
24
- generator = pipeline("text-generation", model="oddadmix/Qari-OCR-0.2.2.1-VL-2B-Instruct", device="cuda")
25
- output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
26
- print(output["generated_text"])
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ```
28
 
29
- ## Training procedure
 
 
 
 
 
 
30
 
31
-
32
 
 
 
 
 
 
 
 
 
33
 
34
- This model was trained with SFT.
 
35
 
36
- ### Framework versions
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
- - TRL: 0.14.0
39
- - Transformers: 4.49.0
40
- - Pytorch: 2.4.1
41
- - Datasets: 3.4.1
42
- - Tokenizers: 0.21.1
43
 
44
- ## Citations
 
45
 
 
46
 
 
47
 
48
- Cite TRL as:
49
-
50
- ```bibtex
51
- @misc{vonwerra2022trl,
52
- title = {{TRL: Transformer Reinforcement Learning}},
53
- author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
54
- year = 2020,
55
- journal = {GitHub repository},
56
- publisher = {GitHub},
57
- howpublished = {\url{https://github.com/huggingface/trl}}
58
  }
59
  ```
 
1
  ---
2
+ base_model:
3
+ - unsloth/Qwen2-VL-2B-Instruct-unsloth-bnb-4bit
 
4
  tags:
5
+ - transformers
6
  - unsloth
7
+ - qwen2_vl
8
  - trl
9
+ - ocr
10
+ license: apache-2.0
11
+ language:
12
+ - ar
13
+ metrics:
14
+ - bleu
15
+ - wer
16
+ - cer
17
+ pipeline_tag: image-text-to-text
18
+ library_name: peft
19
  ---
20
 
21
+ # Qari-OCR-Arabic-0.2.2.1-VL-2B-Instruct Model
22
 
 
 
23
 
24
+ ## Model Overview
25
+
26
+ This model is a fine-tuned version of [unsloth/Qwen2-VL-2B-Instruct](https://huggingface.co/unsloth/Qwen2-VL-2B-Instruct-unsloth-bnb-4bit) on an Arabic OCR dataset. It is optimized to perform Arabic Optical Character Recognition (OCR) for full-page text.
27
+
28
+
29
+ ## Key Features
30
+
31
+ - **Superior Accuracy**: Achieves state-of-the-art performance metrics for Arabic OCR
32
+ - **Diacritics Support**: Full recognition of Arabic diacritical marks (tashkeel) including fatḥah, kasrah, ḍammah, sukūn, shadda, and tanwin forms - a strength confirmed by evaluation on a primarily diacritical text dataset
33
+ - **Multiple Font Support**: Works across a variety of Arabic font styles
34
+ - **Layout Flexibility**: Handles different document layouts and formats
35
+
36
+
37
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/628f7a71dd993507cfcbe587/HuUcfziXcDT_2kwDoz5qH.png)
38
+
39
+ ## Model Details
40
+ - **Base Model**: Qwen2 VL
41
+ - **Fine-tuning Dataset**: Arabic OCR dataset
42
+ - **Objective**: Extract full-page Arabic text with high accuracy
43
+ - **Languages**: Arabic
44
+ - **Tasks**: OCR (Optical Character Recognition)
45
+ - **Dataset size**: 20,000 records
46
+ - **Epochs**: 1
47
+
48
+
49
+ ## Performance Comparison
50
+
51
+ ### Key Metrics
52
+
53
+ | Model | WER ↓ | CER ↓ | BLEU ↑ |
54
+ |-------|-------|-------|--------|
55
+ | Qari-OCR-0.1-VL-2B-Instruct | 0.90 | 0.34 | 0.019 |
56
+ | **Qari-OCR-0.2.2.1-Arabic-2B** | **0.19** | **0.05** | **0.638** |
57
+ | AIN 7B | 0.69 | 0.22 | 0.100 |
58
+ | Qwen2-VL-2B-Instruct | 3.25 | 2.13 | 0.001 |
59
+ | EasyOCR | 1.00 | 0.68 | 0.001 |
60
+ | Tesseract | 0.99 | 0.94 | 0.000002 |
61
+
62
+ *Note: For WER and CER, lower values are better (↓). For BLEU, higher values are better (↑).*
63
+
64
+ ### Performance Highlights
65
+
66
+ - **79.3% reduction** in Word Error Rate compared to our previous version
67
+ - **86.5% improvement** in character recognition accuracy
68
+ - **BLEU score 33.8x higher** than the previous version
69
+ - Outperforms all tested alternatives across all metrics
70
+
71
+
72
+ ### WER Comparison
73
+
74
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/630535e0c7fed54edfaa1a75/mfPYDrfnkFO68cygB5UmI.png" width="600px"/>
75
+
76
+ ### CER Comparison
77
+
78
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/630535e0c7fed54edfaa1a75/efxmdvwr2ArTWEeRLYx_1.png" width="600px"/>
79
 
 
 
80
 
81
+
82
+ ### BLEU Score Comparison
83
+
84
+
85
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/630535e0c7fed54edfaa1a75/zCG5-JK_LCSJq8vd9JVAw.png" width="600px"/>
86
+
87
+
88
+ ## Model Details
89
+
90
+ ### Training Data
91
+ The model was trained using the following specifications:
92
+
93
+ - **Font Sizes**: 14, 16, 18, 20, 24, 32, 40 pt
94
+ - **Page Layouts**:
95
+ - A4 (210mm × 297mm)
96
+ - Letter (216mm × 279mm)
97
+ - Small (105mm × 148mm)
98
+ - Square (1080px × 1080px)
99
+ - OneLine (210mm × 10mm)
100
+ - **Arabic Fonts Used**:
101
+ - IBM Plex Sans Arabic
102
+ - KFGQPCUthman Taha Naskh
103
+ - Scheherazade New
104
+ - Amiri
105
+ - Diwani Letter
106
+ - Tajawal
107
+ - Cairo
108
+ - Lateef
109
+ - Almarai
110
+ - AlQalam Quran
111
+ - Noto Naskh Arabic
112
+
113
+ ### Limitations
114
+ Based on the training specifications, the model has the following limitations:
115
+
116
+ 1. **Font Size Constraints**: May have reduced accuracy with very small (< 14pt) or very large (> 40pt) text
117
+ 2. **Font Coverage**: Performance may degrade on uncommon Arabic fonts not represented in the training data
118
+ 3. **Diacritics Complexity**: While the model supports diacritics (tashkeel), extremely dense or unconventional diacritical mark combinations may reduce accuracy
119
+ 4. **Layout Sensitivity**: May have difficulty with complex multi-column layouts or unconventional page formats
120
+ 4. **Handwriting Recognition**: Limited capability with handwritten text as training focused on digital fonts
121
+ 5. **Decorative Text**: May struggle with highly stylized or decorative Arabic calligraphy
122
+ 6. **Background Complexity**: Optimized for clear backgrounds; performance may degrade with complex or textured backgrounds
123
+ 7. **Text Degradation**: May have challenges with severely degraded, blurry, or low-resolution text
124
+ 8. **Non-standard Orientations**: Primarily designed for horizontally oriented text; may struggle with vertical or diagonal text
125
+
126
+ ### Evaluation Method
127
+ Evaluation was performed on a diverse dataset of Arabic text images, **primarily featuring diacritical marks (tashkeel)**, measuring:
128
+ - **Word Error Rate (WER)**: The percentage of incorrectly recognized words
129
+ - **Character Error Rate (CER)**: The percentage of incorrectly recognized characters
130
+ - **BLEU Score**: A measure of translation quality, higher scores indicate better overall text recognition
131
+
132
+
133
+ ## How to Use
134
+
135
+ [Try Qari v0.2.2 - Google Colab](https://colab.research.google.com/github/NAMAA-ORG/public-notebooks/blob/main/Qari_V0.2.2_Free_Colab.ipynb)
136
+
137
+ You can load this model using the `transformers` and `qwen_vl_utils` library:
138
+ ```
139
+ !pip install transformers qwen_vl_utils accelerate>=0.26.0 PEFT -U
140
+ !pip install -U bitsandbytes
141
  ```
142
 
143
+ ```python
144
+ from PIL import Image
145
+ from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
146
+ import torch
147
+ import os
148
+ from qwen_vl_utils import process_vision_info
149
+
150
 
 
151
 
152
+ model_name = "NAMAA-Space/Qari-OCR-0.2.2.1-Arabic-2B-Instruct"
153
+ model = Qwen2VLForConditionalGeneration.from_pretrained(
154
+ model_name,
155
+ torch_dtype="auto",
156
+ device_map="auto"
157
+ )
158
+ processor = AutoProcessor.from_pretrained(model_name)
159
+ max_tokens = 2000
160
 
161
+ prompt = "Below is the image of one page of a document, as well as some raw textual content that was previously extracted for it. Just return the plain text representation of this document as if you were reading it naturally. Do not hallucinate."
162
+ image.save("image.png")
163
 
164
+ messages = [
165
+ {
166
+ "role": "user",
167
+ "content": [
168
+ {"type": "image", "image": f"file://{src}"},
169
+ {"type": "text", "text": prompt},
170
+ ],
171
+ }
172
+ ]
173
+ text = processor.apply_chat_template(
174
+ messages, tokenize=False, add_generation_prompt=True
175
+ )
176
+ image_inputs, video_inputs = process_vision_info(messages)
177
+ inputs = processor(
178
+ text=[text],
179
+ images=image_inputs,
180
+ videos=video_inputs,
181
+ padding=True,
182
+ return_tensors="pt",
183
+ )
184
+ inputs = inputs.to("cuda")
185
+ generated_ids = model.generate(**inputs, max_new_tokens=max_tokens)
186
+ generated_ids_trimmed = [
187
+ out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
188
+ ]
189
+ output_text = processor.batch_decode(
190
+ generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
191
+ )[0]
192
+ os.remove(src)
193
+ print(output_text)
194
 
195
+ ```
 
 
 
 
196
 
197
+ ## License
198
+ This model follows the licensing terms of the original Qwen2 VL model. Please review the terms before using it commercially.
199
 
200
+ ## Citation
201
 
202
+ If you use this model in your research, please cite:
203
 
204
+ ```
205
+ @misc{QariOCR2025,
206
+ title={Qari-OCR: A High-Accuracy Model for Arabic Optical Character Recognition},
207
+ author={NAMAA},
208
+ year={2025},
209
+ publisher={Hugging Face},
210
+ howpublished={\url{https://huggingface.co/NAMAA-Space/Qari-OCR-0.2.2.1-Arabic-2B-instruct}},
211
+ note={Accessed: 2025-03-15}
 
 
212
  }
213
  ```