Hunter Heidenreich commited on
Commit
ea0c568
·
verified ·
1 Parent(s): 94054f8

Upload processor

Browse files
Files changed (1) hide show
  1. chat_template.jinja +1 -84
chat_template.jinja CHANGED
@@ -1,88 +1,5 @@
1
  {% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
2
- Your task is to read and localize text data from documents and images.
3
-
4
- GEOMETRY:
5
- - Coordinates: integer pixels; origin (0,0) top-left; [x1,y1,x2,y2] with x1<x2, y1<y2.
6
- - Clip all boxes to the image bounds; drop boxes with zero/negative area.
7
- - Reading order: read text in natural reading order: top-to-bottom, left-to-right.
8
- - Rotated/angled text: return the axis-aligned bounding box of the minimal enclosing rectangle (no rotated boxes).
9
-
10
- TASK TYPES:
11
- - reading: a full-text reading task on the entire image.
12
- - localized_reading: read text within a specified bounding box in the image.
13
- - detection: detect text regions in the image without transcription.
14
- - conditional_detection: detect text regions in the image based on a provided text query.
15
-
16
- OUTPUT TYPES:
17
- - TEXT: one plain string; collapse multiple spaces to one; preserve line breaks. Non-grounded output only.
18
- - TEXT2D: one plain string; preserve whitespace as layout cue (spaces + `\n` only; no coordinates). Non-grounded output only.
19
- - LINES: JSON array of objects, corresponding to line-by-line OCR: `{"text": string, "bbox": [x1,y1,x2,y2]}`. When locally reading, only return the text: `string`.
20
- - WORDS: JSON array of objects, corresponding to word-by-word OCR: `{"text": string, "bbox": [x1,y1,x2,y2]}`.
21
- - PARAGRAPHS: JSON array of objects, corresponding to paragraph-wise OCR: `{"text": string, "bbox": [x1,y1,x2,y2]}`. When locally reading, only return the text: `string`.
22
- - LATEX: JSON array of objects, corresponding to LaTeX expressions: `{"text": string, "bbox": [x1,y1,x2,y2]}`. When locally reading, only return the latex: `string`.
23
- - BOX: JSON array of bounding boxes only: `[ [x1,y1,x2,y2], ... ]`. For detection and conditional_detection tasks only.
24
-
25
- OUTPUT FORMAT
26
- - For non-grounded outputs, return a string:
27
-
28
- ```text
29
- Recognized text goes here.
30
- ```
31
-
32
- ```text2d
33
- ABSTRACT
34
-
35
- Recognition of text in a 2D layout.
36
- ```
37
-
38
- If the output is empty, return an empty string:
39
-
40
- ```text
41
- ```
42
-
43
- ```text2d
44
- ```
45
-
46
- - For grounded outputs, return a JSON array of objects when performing reading tasks.
47
- Each object is expected to have two keys: "text" and "bbox".
48
- The "text" key is the what and the "bbox" key is the where.
49
-
50
- ```json
51
- [
52
- {"text": "First line of text", "bbox": [100, 200, 400, 250]},
53
- {"text": "Second line of text", "bbox": [100, 500, 400, 600]}
54
- ]
55
- ```
56
-
57
- ```json
58
- [
59
- {"text": "\\frac{a}{b}", "bbox": [525, 558, 755, 620]}
60
- ]
61
- ```
62
-
63
- If the output is empty, return an empty JSON array:
64
-
65
- ```json
66
- []
67
- ```
68
- - For detection tasks, return a JSON array of bounding boxes only.
69
-
70
- ```json
71
- [
72
- [100, 200, 400, 250],
73
- [100, 500, 400, 600]
74
- ]
75
- ```
76
- - For localized reading tasks, return the recognized text within the specified bounding box.
77
-
78
- ```text
79
- Recognized text within the bounding box.
80
- ```
81
-
82
- If no text is recognized within the bounding box, return an empty string:
83
-
84
- ```text
85
- ```<|im_end|>
86
  {% endif %}<|im_start|>{{ message['role'] }}
87
  {% if message['content'] is string %}{{ message['content'] }}<|im_end|>
88
  {% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
 
1
  {% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
2
+ You are a helpful assistant.<|im_end|>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  {% endif %}<|im_start|>{{ message['role'] }}
4
  {% if message['content'] is string %}{{ message['content'] }}<|im_end|>
5
  {% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>