Text Generation
Transformers
Safetensors
French
qwen3_5
image-text-to-text
french
conversational
edge
qwen
MaxLSB jddqd commited on
Commit
a7f6141
·
0 Parent(s):

Squash history (storage cleanup); repository contents unchanged

Browse files

Co-authored-by: jddqd <jddqd@users.noreply.huggingface.co>

.gitattributes ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Qwen/Qwen3.5-2B
4
+ language:
5
+ - fr
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - french
10
+ - conversational
11
+ - edge
12
+ - qwen
13
+ license: apache-2.0
14
+ datasets:
15
+ - kurakurai/Luth-2-Post-Training-SFT
16
+ - kurakurai/Luth-2-Post-Training-RL
17
+ ---
18
+
19
+ ![luth-2-banner](https://cdn-uploads.huggingface.co/production/uploads/66beff523ae330ae8b698ee9/wso_h0n6syZf9E8YDKWYA.png)
20
+
21
+ # Luth-2-2B
22
+
23
+ **Luth-2-2B** is a 1.88B-parameter (text only) non-reasoning model, setting a new **state of the art in French** for its size across math, code, instruction following, general knowledge and tool calling. It is trained on a 3B-token French SFT mixture followed by multi-domain on-policy distillation (MOPD). The model outperforms every other model in its size class on our selected French benchmarks and stays competitive with larger models. It is small enough for efficient local and on-device deployment.
24
+
25
+ - 📄 **Blog**: [Luth-2: Pushing the French Capabilities of SLMs with MOPD](https://huggingface.co/blog/MaxLSB/luth-2)
26
+ - 🤗 **Models**: [Luth-2-0.8B](https://huggingface.co/kurakurai/Luth-2-0.8B) · [Luth-2-2B](https://huggingface.co/kurakurai/Luth-2-2B)
27
+ - 📊 **Datasets**: [SFT](https://huggingface.co/datasets/kurakurai/Luth-2-Post-Training-SFT) · [RL](https://huggingface.co/datasets/kurakurai/Luth-2-Post-Training-RL)
28
+ - 💻 **Code**: [GitHub](https://github.com/kurakurai/Luth-2)
29
+ - 🏆 **Leaderboard**: [French LLM Leaderboard](https://huggingface.co/spaces/kurakurai/llm_leaderboard_fr)
30
+
31
+ ![luth2_benchmarks_2b_portrait](https://cdn-uploads.huggingface.co/production/uploads/noauth/a3vGEQW6ACZ2U2ni5KSdF.png)
32
+
33
+ > [!NOTE]
34
+ > **Luth-2-2B** inherits the VLM architecture of Qwen3.5-2B but was not trained on vision data. We do not recommend using it for vision tasks.
35
+
36
+ ## Model variants
37
+
38
+ | Model | Description |
39
+ |---|---|
40
+ | [Luth-2-2B](https://huggingface.co/kurakurai/Luth-2-2B) | Original checkpoint in native format. Best for fine-tuning or inference with Transformers, vLLM and SGLang. |
41
+ | [Luth-2-2B-GGUF](https://huggingface.co/kurakurai/Luth-2-2B-GGUF) | Quantized format for llama.cpp and compatible tools. Optimized for CPU inference and reduced memory usage. |
42
+
43
+ ## Training
44
+
45
+ **Luth-2-2B** is post-trained from Qwen3.5-2B in two stages:
46
+
47
+ 1. **Supervised fine-tuning** on [Luth-2-Post-Training-SFT](https://huggingface.co/datasets/kurakurai/Luth-2-Post-Training-SFT), a 3B-token French mixture spanning math (37.2%), knowledge (27.9%), code (22.2%), instruction following (6.5%) and tool calling (6.3%). Prompts were translated from English SFT datasets and answers regenerated with strong open-source teachers.
48
+ 2. **Multi-domain on-policy distillation (MOPD)**. Three specialists (math, code, instruction following) are trained separately with GRPO on [Luth-2-Post-Training-RL](https://huggingface.co/datasets/kurakurai/Luth-2-Post-Training-RL), then distilled back into the SFT student.
49
+
50
+ ## Inference
51
+
52
+ **Luth-2-2B** is supported by Transformers, vLLM, SGLang and more.
53
+
54
+ Quick start with Transformers:
55
+
56
+ ```python
57
+ from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
58
+ model_id = "kurakurai/Luth-2-2B"
59
+ model = AutoModelForCausalLM.from_pretrained(
60
+ model_id,
61
+ device_map="auto",
62
+ dtype="bfloat16",
63
+ # attn_implementation="flash_attention_2" # uncomment on compatible GPU
64
+ )
65
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
66
+ streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
67
+ prompt = "Quelle est la capitale de la France?"
68
+ input_ids = tokenizer.apply_chat_template(
69
+ [{"role": "user", "content": prompt}],
70
+ add_generation_prompt=True,
71
+ return_tensors="pt",
72
+ tokenize=True,
73
+ )["input_ids"].to(model.device)
74
+ output = model.generate(
75
+ input_ids,
76
+ do_sample=True,
77
+ temperature=0.8,
78
+ top_p=0.95,
79
+ top_k=20,
80
+ max_new_tokens=512,
81
+ streamer=streamer,
82
+ )
83
+ ```
84
+
85
+ ## Evaluation
86
+
87
+ Evaluations can be reproduced using our [GitHub](https://github.com/kurakurai/Luth-2) repository. The benchmarks are French subsets or verified translations, scored with `temperature=0.6, top_p=0.95, top_k=20`, thinking disabled, averaged over 10 runs.
88
+
89
+ | French Benchmarks | Luth-2-2B | Luth-1.7B-Instruct | Qwen3.5-2B |
90
+ |---|--:|--:|--:|
91
+ | MGSM-rev2 | **86.52** | 75.72 | 64.60 |
92
+ | AIME 24 | **17.00** | 10.33 | 11.67 |
93
+ | AIME 25 | **19.00** | 7.00 | 6.67 |
94
+ | Math-500 | **81.52** | 65.26 | 65.06 |
95
+ | Global-MMLU-Lite | **64.45** | 58.00 | 58.00 |
96
+ | MMLU-ProX-Lite | **55.24** | 41.10 | 47.20 |
97
+ | GPQA-Diamond | **41.97** | 30.20 | 36.80 |
98
+ | IFEval | **75.06** | 64.47 | 61.91 |
99
+ | Multi-IF | **69.67** | 46.05 | 45.38 |
100
+ | HumanEval+ | **66.00** | 56.44 | 37.56 |
101
+ | MBPP+ | **57.62** | 56.40 | 42.12 |
102
+ | BFCL v2 | 68.91 | **71.05** | 51.78 |
103
+
104
+ See the [French LLM Leaderboard](https://huggingface.co/spaces/kurakurai/llm_leaderboard_fr) for comparisons across models.
105
+
106
+ ## Contact
107
+
108
+ Questions or feedback? Reach us on LinkedIn: [Maxence Lasbordes](https://www.linkedin.com/in/maxence-lasbordes/) and [Guillaume Pradel](https://www.linkedin.com/in/guillaume-pradel/).
109
+
110
+ ## Citation
111
+
112
+ ```bibtex
113
+ @misc{luth2,
114
+ title = {Luth-2: Pushing the French Capabilities of SLMs with MOPD},
115
+ author = {Maxence Lasbordes and Guillaume Pradel},
116
+ year = {2026},
117
+ url = {https://huggingface.co/MaxLSB/luth-2}
118
+ }
119
+ ```
chat_template.jinja ADDED
@@ -0,0 +1,174 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {#- Qwen3.5's stock chat template, with {% generation %} markers so TRL's
2
+ assistant_only_loss can build the loss mask. Three deviations from stock:
3
+ - the assistant turn's opening tag and its <think> prefix stay OUTSIDE the
4
+ mask, because add_generation_prompt already emits them at inference time;
5
+ - the target — reasoning (if any), content, tool calls, <|im_end|> — is
6
+ wrapped in {% generation %};
7
+ - non-thinking is forced: the generation prompt always prefills a closed
8
+ '<think>\n\n</think>\n\n' and enable_thinking is ignored (see the end of
9
+ this file). Stock opens a bare '<think>\n' when enable_thinking is true.
10
+ Everything else, tool calling included, is byte-identical to stock. #}
11
+ {%- set image_count = namespace(value=0) %}
12
+ {%- set video_count = namespace(value=0) %}
13
+ {%- macro render_content(content, do_vision_count, is_system_content=false) %}
14
+ {%- if content is string %}
15
+ {{- content }}
16
+ {%- elif content is iterable and content is not mapping %}
17
+ {%- for item in content %}
18
+ {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
19
+ {%- if is_system_content %}
20
+ {{- raise_exception('System message cannot contain images.') }}
21
+ {%- endif %}
22
+ {%- if do_vision_count %}
23
+ {%- set image_count.value = image_count.value + 1 %}
24
+ {%- endif %}
25
+ {%- if add_vision_id %}
26
+ {{- 'Picture ' ~ image_count.value ~ ': ' }}
27
+ {%- endif %}
28
+ {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
29
+ {%- elif 'video' in item or item.type == 'video' %}
30
+ {%- if is_system_content %}
31
+ {{- raise_exception('System message cannot contain videos.') }}
32
+ {%- endif %}
33
+ {%- if do_vision_count %}
34
+ {%- set video_count.value = video_count.value + 1 %}
35
+ {%- endif %}
36
+ {%- if add_vision_id %}
37
+ {{- 'Video ' ~ video_count.value ~ ': ' }}
38
+ {%- endif %}
39
+ {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
40
+ {%- elif 'text' in item %}
41
+ {{- item.text }}
42
+ {%- else %}
43
+ {{- raise_exception('Unexpected item type in content.') }}
44
+ {%- endif %}
45
+ {%- endfor %}
46
+ {%- elif content is none or content is undefined %}
47
+ {{- '' }}
48
+ {%- else %}
49
+ {{- raise_exception('Unexpected content type.') }}
50
+ {%- endif %}
51
+ {%- endmacro %}
52
+ {%- if not messages %}
53
+ {{- raise_exception('No messages provided.') }}
54
+ {%- endif %}
55
+ {%- if tools and tools is iterable and tools is not mapping %}
56
+ {{- '<|im_start|>system\n' }}
57
+ {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
58
+ {%- for tool in tools %}
59
+ {{- "\n" }}
60
+ {{- tool | tojson }}
61
+ {%- endfor %}
62
+ {{- "\n</tools>" }}
63
+ {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
64
+ {%- if messages[0].role == 'system' %}
65
+ {%- set content = render_content(messages[0].content, false, true)|trim %}
66
+ {%- if content %}
67
+ {{- '\n\n' + content }}
68
+ {%- endif %}
69
+ {%- endif %}
70
+ {{- '<|im_end|>\n' }}
71
+ {%- else %}
72
+ {%- if messages[0].role == 'system' %}
73
+ {%- set content = render_content(messages[0].content, false, true)|trim %}
74
+ {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
75
+ {%- endif %}
76
+ {%- endif %}
77
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
78
+ {%- for message in messages[::-1] %}
79
+ {%- set index = (messages|length - 1) - loop.index0 %}
80
+ {%- if ns.multi_step_tool and message.role == "user" %}
81
+ {%- set content = render_content(message.content, false)|trim %}
82
+ {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
83
+ {%- set ns.multi_step_tool = false %}
84
+ {%- set ns.last_query_index = index %}
85
+ {%- endif %}
86
+ {%- endif %}
87
+ {%- endfor %}
88
+ {%- if ns.multi_step_tool %}
89
+ {{- raise_exception('No user query found in messages.') }}
90
+ {%- endif %}
91
+ {%- for message in messages %}
92
+ {%- set content = render_content(message.content, true)|trim %}
93
+ {%- if message.role == "system" %}
94
+ {%- if not loop.first %}
95
+ {{- raise_exception('System message must be at the beginning.') }}
96
+ {%- endif %}
97
+ {%- elif message.role == "user" %}
98
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
99
+ {%- elif message.role == "assistant" %}
100
+ {%- set reasoning_content = '' %}
101
+ {%- if message.reasoning_content is string %}
102
+ {%- set reasoning_content = message.reasoning_content %}
103
+ {%- else %}
104
+ {%- if '</think>' in content %}
105
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
106
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
107
+ {%- endif %}
108
+ {%- endif %}
109
+ {%- set reasoning_content = reasoning_content|trim %}
110
+ {#- Emit the unmasked prefix (what add_generation_prompt would produce),
111
+ and defer the rest of the <think> block to the masked target. #}
112
+ {%- set target_prefix = '' %}
113
+ {%- if loop.index0 > ns.last_query_index %}
114
+ {%- if reasoning_content %}
115
+ {{- '<|im_start|>' + message.role + '\n<think>\n' }}
116
+ {%- set target_prefix = reasoning_content + '\n</think>\n\n' %}
117
+ {%- else %}
118
+ {{- '<|im_start|>' + message.role + '\n<think>\n\n</think>\n\n' }}
119
+ {%- endif %}
120
+ {%- else %}
121
+ {{- '<|im_start|>' + message.role + '\n' }}
122
+ {%- endif %}
123
+ {%- generation %}
124
+ {{- target_prefix }}
125
+ {{- content }}
126
+ {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
127
+ {%- for tool_call in message.tool_calls %}
128
+ {%- if tool_call.function is defined %}
129
+ {%- set tool_call = tool_call.function %}
130
+ {%- endif %}
131
+ {%- if loop.first %}
132
+ {%- if content|trim %}
133
+ {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
134
+ {%- else %}
135
+ {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
136
+ {%- endif %}
137
+ {%- else %}
138
+ {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
139
+ {%- endif %}
140
+ {%- if tool_call.arguments is defined %}
141
+ {%- for args_name, args_value in tool_call.arguments|items %}
142
+ {{- '<parameter=' + args_name + '>\n' }}
143
+ {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
144
+ {{- args_value }}
145
+ {{- '\n</parameter>\n' }}
146
+ {%- endfor %}
147
+ {%- endif %}
148
+ {{- '</function>\n</tool_call>' }}
149
+ {%- endfor %}
150
+ {%- endif %}
151
+ {{- '<|im_end|>\n' }}
152
+ {%- endgeneration %}
153
+ {%- elif message.role == "tool" %}
154
+ {%- if loop.previtem and loop.previtem.role != "tool" %}
155
+ {{- '<|im_start|>user' }}
156
+ {%- endif %}
157
+ {{- '\n<tool_response>\n' }}
158
+ {{- content }}
159
+ {{- '\n</tool_response>' }}
160
+ {%- if not loop.last and loop.nextitem.role != "tool" %}
161
+ {{- '<|im_end|>\n' }}
162
+ {%- elif loop.last %}
163
+ {{- '<|im_end|>\n' }}
164
+ {%- endif %}
165
+ {%- else %}
166
+ {{- raise_exception('Unexpected message role.') }}
167
+ {%- endif %}
168
+ {%- endfor %}
169
+ {#- Non-thinking only: the think block is always prefilled closed. `enable_thinking` is ignored
170
+ (accepted, no effect) — this model was never tuned for thinking mode. #}
171
+ {%- if add_generation_prompt %}
172
+ {{- '<|im_start|>assistant\n' }}
173
+ {{- '<think>\n\n</think>\n\n' }}
174
+ {%- endif %}
config.json ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3_5ForConditionalGeneration"
4
+ ],
5
+ "dtype": "float32",
6
+ "eos_token_id": 248046,
7
+ "fp8": false,
8
+ "image_token_id": 248056,
9
+ "model_type": "qwen3_5",
10
+ "pad_token_id": 248044,
11
+ "text_config": {
12
+ "attention_bias": false,
13
+ "attention_dropout": 0.0,
14
+ "attn_output_gate": true,
15
+ "bos_token_id": null,
16
+ "dtype": "float32",
17
+ "eos_token_id": 248044,
18
+ "full_attention_interval": 4,
19
+ "head_dim": 256,
20
+ "hidden_act": "silu",
21
+ "hidden_size": 2048,
22
+ "initializer_range": 0.02,
23
+ "intermediate_size": 6144,
24
+ "layer_types": [
25
+ "linear_attention",
26
+ "linear_attention",
27
+ "linear_attention",
28
+ "full_attention",
29
+ "linear_attention",
30
+ "linear_attention",
31
+ "linear_attention",
32
+ "full_attention",
33
+ "linear_attention",
34
+ "linear_attention",
35
+ "linear_attention",
36
+ "full_attention",
37
+ "linear_attention",
38
+ "linear_attention",
39
+ "linear_attention",
40
+ "full_attention",
41
+ "linear_attention",
42
+ "linear_attention",
43
+ "linear_attention",
44
+ "full_attention",
45
+ "linear_attention",
46
+ "linear_attention",
47
+ "linear_attention",
48
+ "full_attention"
49
+ ],
50
+ "linear_conv_kernel_dim": 4,
51
+ "linear_key_head_dim": 128,
52
+ "linear_num_key_heads": 16,
53
+ "linear_num_value_heads": 16,
54
+ "linear_value_head_dim": 128,
55
+ "mamba_ssm_dtype": "float32",
56
+ "max_position_embeddings": 262144,
57
+ "mlp_only_layers": [],
58
+ "model_type": "qwen3_5_text",
59
+ "mtp_num_hidden_layers": 1,
60
+ "mtp_use_dedicated_embeddings": false,
61
+ "num_attention_heads": 8,
62
+ "num_hidden_layers": 24,
63
+ "num_key_value_heads": 2,
64
+ "pad_token_id": null,
65
+ "partial_rotary_factor": 0.25,
66
+ "rms_norm_eps": 1e-06,
67
+ "rope_parameters": {
68
+ "mrope_interleaved": true,
69
+ "mrope_section": [
70
+ 11,
71
+ 11,
72
+ 10
73
+ ],
74
+ "partial_rotary_factor": 0.25,
75
+ "rope_theta": 10000000,
76
+ "rope_type": "default"
77
+ },
78
+ "tie_word_embeddings": true,
79
+ "use_cache": false,
80
+ "vocab_size": 248320
81
+ },
82
+ "tie_word_embeddings": true,
83
+ "transformers_version": "5.6.2",
84
+ "use_cache": false,
85
+ "use_grouped_mm": true,
86
+ "video_token_id": 248057,
87
+ "vision_config": {
88
+ "deepstack_visual_indexes": [],
89
+ "depth": 24,
90
+ "dtype": "float32",
91
+ "hidden_act": "gelu_pytorch_tanh",
92
+ "hidden_size": 1024,
93
+ "in_channels": 3,
94
+ "initializer_range": 0.02,
95
+ "intermediate_size": 4096,
96
+ "model_type": "qwen3_5_vision",
97
+ "num_heads": 16,
98
+ "num_position_embeddings": 2304,
99
+ "out_hidden_size": 2048,
100
+ "patch_size": 16,
101
+ "spatial_merge_size": 2,
102
+ "temporal_patch_size": 2
103
+ },
104
+ "vision_end_token_id": 248054,
105
+ "vision_start_token_id": 248053
106
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 248046,
5
+ 248044
6
+ ],
7
+ "pad_token_id": 248044,
8
+ "temperature": 0.8,
9
+ "top_k": 20,
10
+ "top_p": 0.95,
11
+ "transformers_version": "5.6.2",
12
+ "use_cache": true
13
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de1af1d1c346231e6fc5ec1cd764b75d04fde30181cb066dd45168fa2d72dfa8
3
+ size 4426558864
preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "size": {
3
+ "longest_edge": 16777216,
4
+ "shortest_edge": 65536
5
+ },
6
+ "patch_size": 16,
7
+ "temporal_patch_size": 2,
8
+ "merge_size": 2,
9
+ "image_mean": [
10
+ 0.5,
11
+ 0.5,
12
+ 0.5
13
+ ],
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "processor_class": "Qwen3VLProcessor",
20
+ "image_processor_type": "Qwen2VLImageProcessorFast"
21
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
3
+ size 19989325
tokenizer_config.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "audio_bos_token": "<|audio_start|>",
4
+ "audio_eos_token": "<|audio_end|>",
5
+ "audio_token": "<|audio_pad|>",
6
+ "backend": "tokenizers",
7
+ "bos_token": null,
8
+ "clean_up_tokenization_spaces": false,
9
+ "eos_token": "<|im_end|>",
10
+ "errors": "replace",
11
+ "image_token": "<|image_pad|>",
12
+ "is_local": false,
13
+ "local_files_only": true,
14
+ "model_max_length": 262144,
15
+ "model_specific_special_tokens": {
16
+ "audio_bos_token": "<|audio_start|>",
17
+ "audio_eos_token": "<|audio_end|>",
18
+ "audio_token": "<|audio_pad|>",
19
+ "image_token": "<|image_pad|>",
20
+ "video_token": "<|video_pad|>",
21
+ "vision_bos_token": "<|vision_start|>",
22
+ "vision_eos_token": "<|vision_end|>"
23
+ },
24
+ "pad_token": "<|im_end|>",
25
+ "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
26
+ "split_special_tokens": false,
27
+ "tokenizer_class": "Qwen2Tokenizer",
28
+ "unk_token": null,
29
+ "video_token": "<|video_pad|>",
30
+ "vision_bos_token": "<|vision_start|>",
31
+ "vision_eos_token": "<|vision_end|>"
32
+ }
video_preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "size": {
3
+ "longest_edge": 25165824,
4
+ "shortest_edge": 4096
5
+ },
6
+ "patch_size": 16,
7
+ "temporal_patch_size": 2,
8
+ "merge_size": 2,
9
+ "image_mean": [
10
+ 0.5,
11
+ 0.5,
12
+ 0.5
13
+ ],
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "processor_class": "Qwen3VLProcessor",
20
+ "video_processor_type": "Qwen3VLVideoProcessor"
21
+ }