Image-Text-to-Text
Transformers
Safetensors
English
qwen2_5_vl
ocr
vision
qwen2.5-vl
pdf
document-understanding
conversational
text-generation-inference
Instructions to use bevaya/GutenOCR-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bevaya/GutenOCR-7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="bevaya/GutenOCR-7B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("bevaya/GutenOCR-7B") model = AutoModelForMultimodalLM.from_pretrained("bevaya/GutenOCR-7B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bevaya/GutenOCR-7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bevaya/GutenOCR-7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bevaya/GutenOCR-7B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/bevaya/GutenOCR-7B
- SGLang
How to use bevaya/GutenOCR-7B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bevaya/GutenOCR-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bevaya/GutenOCR-7B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bevaya/GutenOCR-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bevaya/GutenOCR-7B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use bevaya/GutenOCR-7B with Docker Model Runner:
docker model run hf.co/bevaya/GutenOCR-7B
Hunter Heidenreich commited on
Upload processor
Browse files- chat_template.jinja +1 -84
chat_template.jinja
CHANGED
|
@@ -1,88 +1,5 @@
|
|
| 1 |
{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
GEOMETRY:
|
| 5 |
-
- Coordinates: integer pixels; origin (0,0) top-left; [x1,y1,x2,y2] with x1<x2, y1<y2.
|
| 6 |
-
- Clip all boxes to the image bounds; drop boxes with zero/negative area.
|
| 7 |
-
- Reading order: read text in natural reading order: top-to-bottom, left-to-right.
|
| 8 |
-
- Rotated/angled text: return the axis-aligned bounding box of the minimal enclosing rectangle (no rotated boxes).
|
| 9 |
-
|
| 10 |
-
TASK TYPES:
|
| 11 |
-
- reading: a full-text reading task on the entire image.
|
| 12 |
-
- localized_reading: read text within a specified bounding box in the image.
|
| 13 |
-
- detection: detect text regions in the image without transcription.
|
| 14 |
-
- conditional_detection: detect text regions in the image based on a provided text query.
|
| 15 |
-
|
| 16 |
-
OUTPUT TYPES:
|
| 17 |
-
- TEXT: one plain string; collapse multiple spaces to one; preserve line breaks. Non-grounded output only.
|
| 18 |
-
- TEXT2D: one plain string; preserve whitespace as layout cue (spaces + `\n` only; no coordinates). Non-grounded output only.
|
| 19 |
-
- LINES: JSON array of objects, corresponding to line-by-line OCR: `{"text": string, "bbox": [x1,y1,x2,y2]}`. When locally reading, only return the text: `string`.
|
| 20 |
-
- WORDS: JSON array of objects, corresponding to word-by-word OCR: `{"text": string, "bbox": [x1,y1,x2,y2]}`.
|
| 21 |
-
- PARAGRAPHS: JSON array of objects, corresponding to paragraph-wise OCR: `{"text": string, "bbox": [x1,y1,x2,y2]}`. When locally reading, only return the text: `string`.
|
| 22 |
-
- LATEX: JSON array of objects, corresponding to LaTeX expressions: `{"text": string, "bbox": [x1,y1,x2,y2]}`. When locally reading, only return the latex: `string`.
|
| 23 |
-
- BOX: JSON array of bounding boxes only: `[ [x1,y1,x2,y2], ... ]`. For detection and conditional_detection tasks only.
|
| 24 |
-
|
| 25 |
-
OUTPUT FORMAT
|
| 26 |
-
- For non-grounded outputs, return a string:
|
| 27 |
-
|
| 28 |
-
```text
|
| 29 |
-
Recognized text goes here.
|
| 30 |
-
```
|
| 31 |
-
|
| 32 |
-
```text2d
|
| 33 |
-
ABSTRACT
|
| 34 |
-
|
| 35 |
-
Recognition of text in a 2D layout.
|
| 36 |
-
```
|
| 37 |
-
|
| 38 |
-
If the output is empty, return an empty string:
|
| 39 |
-
|
| 40 |
-
```text
|
| 41 |
-
```
|
| 42 |
-
|
| 43 |
-
```text2d
|
| 44 |
-
```
|
| 45 |
-
|
| 46 |
-
- For grounded outputs, return a JSON array of objects when performing reading tasks.
|
| 47 |
-
Each object is expected to have two keys: "text" and "bbox".
|
| 48 |
-
The "text" key is the what and the "bbox" key is the where.
|
| 49 |
-
|
| 50 |
-
```json
|
| 51 |
-
[
|
| 52 |
-
{"text": "First line of text", "bbox": [100, 200, 400, 250]},
|
| 53 |
-
{"text": "Second line of text", "bbox": [100, 500, 400, 600]}
|
| 54 |
-
]
|
| 55 |
-
```
|
| 56 |
-
|
| 57 |
-
```json
|
| 58 |
-
[
|
| 59 |
-
{"text": "\\frac{a}{b}", "bbox": [525, 558, 755, 620]}
|
| 60 |
-
]
|
| 61 |
-
```
|
| 62 |
-
|
| 63 |
-
If the output is empty, return an empty JSON array:
|
| 64 |
-
|
| 65 |
-
```json
|
| 66 |
-
[]
|
| 67 |
-
```
|
| 68 |
-
- For detection tasks, return a JSON array of bounding boxes only.
|
| 69 |
-
|
| 70 |
-
```json
|
| 71 |
-
[
|
| 72 |
-
[100, 200, 400, 250],
|
| 73 |
-
[100, 500, 400, 600]
|
| 74 |
-
]
|
| 75 |
-
```
|
| 76 |
-
- For localized reading tasks, return the recognized text within the specified bounding box.
|
| 77 |
-
|
| 78 |
-
```text
|
| 79 |
-
Recognized text within the bounding box.
|
| 80 |
-
```
|
| 81 |
-
|
| 82 |
-
If no text is recognized within the bounding box, return an empty string:
|
| 83 |
-
|
| 84 |
-
```text
|
| 85 |
-
```<|im_end|>
|
| 86 |
{% endif %}<|im_start|>{{ message['role'] }}
|
| 87 |
{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
|
| 88 |
{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
|
|
|
|
| 1 |
{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
|
| 2 |
+
You are a helpful assistant.<|im_end|>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
{% endif %}<|im_start|>{{ message['role'] }}
|
| 4 |
{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
|
| 5 |
{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
|