Upload Fast-dVLM 3B W8A8 FP8 quantized weights (language + vision)

README.md

@@ -1,3 +1,130 @@
----
-license: mit
----
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- Efficient-Large-Model/Fast_dVLM_3B
+tags:
+- diffusion
+- vlm
+- block-diffusion
+- parallel-decoding
+- quantization
+- fp8
+- smoothquant
+---
+
+# Fast-dVLM (3B) — W8A8 FP8 Quantized
+
+[[Paper](https://arxiv.org/abs/2604.06832)] [[Project Page](https://nvlabs.github.io/Fast-dLLM/fast_dvlm/)] [[Code](https://github.com/NVlabs/Fast-dLLM)] [[BF16 Base](https://huggingface.co/Efficient-Large-Model/Fast_dVLM_3B)]
+
+## Introduction
+
+This repository hosts the **W8A8 FP8 quantized** version of [`Fast_dVLM_3B`](https://huggingface.co/Efficient-Large-Model/Fast_dVLM_3B), produced via SmoothQuant offline calibration.
+
+Both the language tower **and** the vision encoder are quantized to **FP8 (E4M3)** weights with per-channel static scales and per-token dynamic FP8 activations. Combined with SGLang block-diffusion serving, this checkpoint reaches **350 TPS (6.18× over the AR baseline)** while keeping MMMU-Pro-V within 0.3 of the BF16 result.
+
+### Key Highlights
+
+- **6.18× Speedup** over the Qwen2.5-VL-3B AR baseline (350 TPS vs 56.7 TPS).
+- **Near-Lossless Quality**: 23.8 vs 24.1 MMMU-Pro-V vs the BF16 block-diffusion model.
+- **Full-Model FP8**: Language + vision both quantized; only norms, embeddings, and `lm_head` remain in BF16.
+
+---
+
+## Model Overview
+
+| Property | Value |
+|---|---|
+| **Type** | Block Diffusion Vision-Language Model (FP8 quantized) |
+| **Base Model** | [`Efficient-Large-Model/Fast_dVLM_3B`](https://huggingface.co/Efficient-Large-Model/Fast_dVLM_3B) |
+| **Quantization** | SmoothQuant W8A8 FP8 (E4M3), per-channel static weight / per-token dynamic activation |
+| **Calibration** | 512 samples × 1024 tokens on C4, SmoothQuant α=0.5 |
+| **Text Layers** | 36 |
+| **Vision Depth** | 32 |
+| **Text Hidden Size** | 2048 |
+| **Attention Heads** | 16 (Q), 2 (KV, GQA) |
+| **Block Diffusion Size** | 32 |
+
+**Hardware requirement**: NVIDIA GPU with **SM89+ (Compute Capability ≥ 8.9)** — RTX 4090, L40, H100, H200. FP8 tensor cores are not available on A100 (SM80) or older.
+
+---
+
+## Quickstart (SGLang)
+
+Load this checkpoint with the customized SGLang shipped in the [Fast-dLLM repo](https://github.com/NVlabs/Fast-dLLM):
+
+```bash
+# Install the customized SGLang (one-time)
+git clone https://github.com/NVlabs/Fast-dLLM
+cd Fast-dLLM/fast_dvlm/sglang/python
+pip install -e .
+
+# Run the chatbot with FP8 quantization
+cd ../..
+python run_chatbot_sglang.py \
+    --algorithm spec \
+    --model-path Sensen02/Fast_dVLM_3B_W8A8_FP8 \
+    --quantization w8a8_fp8 \
+    --prompt "Describe this image." \
+    --image path/to/image.jpg
+```
+
+SGLang reads the `quantization_config` in `config.json` automatically:
+
+```json
+"quantization_config": {
+    "quant_method": "w8a8_fp8",
+    "is_dynamic": false,
+    "ignore": []
+}
+```
+
+The `ignore` list is empty because every linear layer — including the vision encoder — is FP8 quantized in this release.
+
+---
+
+## Benchmark Results
+
+### Quality (MMMU-Pro-V)
+
+| Variant | MMMU-Pro-V |
+|---|---|
+| Qwen2.5-VL-3B (AR baseline) | 26.3 |
+| Fast-dVLM (MDM, τ=0.9) | 21.4 |
+| Fast-dVLM (spec.) — BF16 | 24.1 |
+| **Fast-dVLM (spec.) — W8A8 FP8 (this repo)** | **23.8** |
+
+### Inference Acceleration
+
+| Setting | MMMU-Pro-V | TPS | SpeedUp |
+|---|---|---|---|
+| AR baseline | 26.3 | 56.7 | 1.00x |
+| Fast-dVLM (MDM, τ=0.9) | 21.4 | 82.2 | 1.45x |
+| + Spec. decoding (linear) | 24.6 | 112.7 | 1.98x |
+| + SGLang serving | 24.1 | 319.0 | 5.63x |
+| **+ SmoothQuant-W8A8 (FP8)** | **23.8** | **350.3** | **6.18x** |
+
+---
+
+## Citation
+
+If you use Fast-dVLM in your research, please cite:
+
+```bibtex
+@misc{wu2026fastdvlmefficientblockdiffusionvlm,
+      title={Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM},
+      author={Chengyue Wu and Shiyi Lan and Yonggan Fu and Sensen Gao and Jin Wang and Jincheng Yu and Jose M. Alvarez and Pavlo Molchanov and Ping Luo and Song Han and Ligeng Zhu and Enze Xie},
+      year={2026},
+      eprint={2604.06832},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2604.06832},
+}
+```
+
+---
+
+## License
+
+Released under **Apache 2.0**, following the base Qwen2.5-VL license.

added_tokens.json

@@ -0,0 +1,25 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652,
+  "|<MASK>|": 151665
+}

chat_template.jinja

@@ -0,0 +1,7 @@
+{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
+You are a helpful assistant.<|im_end|>
+{% endif %}<|im_start|>{{ message['role'] }}
+{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
+{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
+{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

config.json

@@ -0,0 +1,176 @@
+{
+  "always_mask_im_end": true,
+  "anneal_block_size": true,
+  "architectures": [
+    "Fast_dVLMForConditionalGeneration"
+  ],
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration.Fast_dVLMConfig",
+    "AutoModel": "modeling.Fast_dVLMForConditionalGeneration",
+    "AutoModelForCausalLM": "modeling.Fast_dVLMForConditionalGeneration"
+  },
+  "bd_size": 32,
+  "block_causal_no_dynamic": false,
+  "complementary_mask": true,
+  "dtype": "bfloat16",
+  "enable_efficient_vision_embed": false,
+  "entropy_loss": false,
+  "entropy_loss_weight": 1.0,
+  "eos_token_id": 151645,
+  "flexible_bd_size": false,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "image_token_id": 151655,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 128000,
+  "max_window_layers": 70,
+  "minimum_noise_level": 0.001,
+  "model_type": "fast_dvlm",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 2,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": {
+    "mrope_section": [
+      16,
+      24,
+      24
+    ],
+    "rope_type": "default",
+    "type": "default"
+  },
+  "rope_theta": 1000000.0,
+  "sliding_window": 32768,
+  "text_config": {
+    "architectures": [
+      "Fast_dVLMForConditionalGeneration"
+    ],
+    "attention_dropout": 0.0,
+    "auto_map": {
+      "AutoConfig": "configuration.Fast_dVLMConfig",
+      "AutoModel": "modeling.Fast_dVLMForConditionalGeneration",
+      "AutoModelForCausalLM": "modeling.Fast_dVLMForConditionalGeneration"
+    },
+    "bd_size": 8,
+    "block_causal_no_dynamic": false,
+    "bos_token_id": 151643,
+    "complementary_mask": true,
+    "dtype": "float32",
+    "entropy_loss": false,
+    "entropy_loss_weight": 1.0,
+    "eos_token_id": 151645,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "image_token_id": null,
+    "initializer_range": 0.02,
+    "intermediate_size": 11008,
+    "layer_types": [
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 128000,
+    "max_window_layers": 70,
+    "minimum_noise_level": 0.001,
+    "model_type": "fast_dvlm_for_causal_lm",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 36,
+    "num_key_value_heads": 2,
+    "rms_norm_eps": 1e-06,
+    "rope_scaling": {
+      "mrope_section": [
+        16,
+        24,
+        24
+      ],
+      "rope_type": "default",
+      "type": "default"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": null,
+    "tie_word_embeddings": true,
+    "use_block_causal_mask": false,
+    "use_cache": true,
+    "use_sliding_window": false,
+    "video_token_id": null,
+    "vision_end_token_id": 151653,
+    "vision_start_token_id": 151652,
+    "vision_token_id": 151654,
+    "vocab_size": 151936
+  },
+  "transformers_version": "4.57.1",
+  "use_block_causal_mask": true,
+  "use_cache": true,
+  "use_sliding_window": false,
+  "video_token_id": 151656,
+  "vision_config": {
+    "depth": 32,
+    "dtype": "float32",
+    "fullatt_block_indexes": [
+      7,
+      15,
+      23,
+      31
+    ],
+    "hidden_act": "silu",
+    "hidden_size": 1280,
+    "in_channels": 3,
+    "in_chans": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 3420,
+    "model_type": "fast_dvlm",
+    "num_heads": 16,
+    "out_hidden_size": 2048,
+    "patch_size": 14,
+    "spatial_merge_size": 2,
+    "spatial_patch_size": 14,
+    "temporal_patch_size": 2,
+    "tokens_per_second": 2,
+    "window_size": 112
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652,
+  "vision_token_id": 151654,
+  "vocab_size": 151936,
+  "quantization_config": {
+    "quant_method": "w8a8_fp8",
+    "is_dynamic": false,
+    "ignore": []
+  }
+}

configuration.py

@@ -0,0 +1,190 @@
+
+# limitations under the License.
+from transformers.configuration_utils import PretrainedConfig, layer_type_validation
+from transformers.modeling_rope_utils import rope_config_validation
+
+
+class Fast_dVLMVisionConfig(PretrainedConfig):
+    model_type = "fast_dvlm"
+    base_config_key = "vision_config"
+
+    def __init__(
+        self,
+        depth=32,
+        hidden_size=3584,
+        hidden_act="silu",
+        intermediate_size=3420,
+        num_heads=16,
+        in_channels=3,
+        patch_size=14,
+        spatial_merge_size=2,
+        temporal_patch_size=2,
+        tokens_per_second=4,
+        window_size=112,
+        out_hidden_size=3584,
+        fullatt_block_indexes=[7, 15, 23, 31],
+        initializer_range=0.02,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+
+        self.depth = depth
+        self.hidden_size = hidden_size
+        self.hidden_act = hidden_act
+        self.intermediate_size = intermediate_size
+        self.num_heads = num_heads
+        self.in_channels = in_channels
+        self.patch_size = patch_size
+        self.spatial_merge_size = spatial_merge_size
+        self.temporal_patch_size = temporal_patch_size
+        self.tokens_per_second = tokens_per_second
+        self.window_size = window_size
+        self.fullatt_block_indexes = fullatt_block_indexes
+        self.out_hidden_size = out_hidden_size
+        self.initializer_range = initializer_range
+
+
+class Fast_dVLMTextConfig(PretrainedConfig):
+
+    model_type = "fast_dvlm_for_causal_lm"
+    base_config_key = "text_config"
+    keys_to_ignore_at_inference = ["past_key_values"]
+    base_model_tp_plan = {
+        "layers.*.self_attn.q_proj": "colwise",
+        "layers.*.self_attn.k_proj": "colwise",
+        "layers.*.self_attn.v_proj": "colwise",
+        "layers.*.self_attn.o_proj": "rowwise",
+        "layers.*.mlp.gate_proj": "colwise",
+        "layers.*.mlp.up_proj": "colwise",
+        "layers.*.mlp.down_proj": "rowwise",
+    }
+    base_model_pp_plan = {
+        "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+        "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+        "norm": (["hidden_states"], ["hidden_states"]),
+    }
+
+    def __init__(
+        self,
+        vocab_size=152064,
+        hidden_size=8192,
+        intermediate_size=29568,
+        num_hidden_layers=80,
+        num_attention_heads=64,
+        num_key_value_heads=8,
+        hidden_act="silu",
+        max_position_embeddings=32768,
+        initializer_range=0.02,
+        rms_norm_eps=1e-05,
+        use_cache=True,
+        tie_word_embeddings=False,
+        rope_theta=1000000.0,
+        use_sliding_window=False,
+        sliding_window=4096,
+        max_window_layers=80,
+        layer_types=None,
+        attention_dropout=0.0,
+        rope_scaling=None,
+        image_token_id=None,
+        video_token_id=None,
+        bd_size=8,
+        self_spec_inference_mode=None,
+        block_length=None,
+        use_block_causal_mask=False,
+        complementary_mask=True,
+        minimum_noise_level=1e-3,
+        entropy_loss=False,
+        entropy_loss_weight=1.0,
+        block_causal_no_dynamic=False,
+        **kwargs,
+    ):
+        self.vocab_size = vocab_size
+        self.max_position_embeddings = max_position_embeddings
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.use_sliding_window = use_sliding_window
+        self.sliding_window = sliding_window if self.use_sliding_window else None
+        self.max_window_layers = max_window_layers
+
+        # for backward compatibility
+        if num_key_value_heads is None:
+            num_key_value_heads = num_attention_heads
+
+        self.num_key_value_heads = num_key_value_heads
+        self.hidden_act = hidden_act
+        self.initializer_range = initializer_range
+        self.rms_norm_eps = rms_norm_eps
+        self.use_cache = use_cache
+        self.rope_theta = rope_theta
+        self.attention_dropout = attention_dropout
+        self.rope_scaling = rope_scaling
+        self.bd_size = bd_size
+        self.layer_types = layer_types
+        self.use_block_causal_mask = use_block_causal_mask
+        self.complementary_mask = complementary_mask
+        self.minimum_noise_level = minimum_noise_level
+        self.entropy_loss = entropy_loss
+        self.entropy_loss_weight = entropy_loss_weight
+        self.block_causal_no_dynamic = block_causal_no_dynamic
+        self.self_spec_inference_mode = self_spec_inference_mode
+        self.block_length = block_length
+        if self.layer_types is None:
+            self.layer_types = [
+                "sliding_attention"
+                if self.sliding_window is not None and i >= self.max_window_layers
+                else "full_attention"
+                for i in range(self.num_hidden_layers)
+            ]
+        layer_type_validation(self.layer_types)
+
+        # Validate the correctness of rotary position embeddings parameters
+        # BC: if there is a 'type' field, move it to 'rope_type'.
+        # and change type from 'mrope' to 'default' because `mrope` does default RoPE calculations
+        # one can set it to "linear"/"dynamic" etc. to have scaled RoPE
+        # TODO: @raushan update config in the hub
+        if self.rope_scaling is not None and "type" in self.rope_scaling:
+            if self.rope_scaling["type"] == "mrope":
+                self.rope_scaling["type"] = "default"
+            self.rope_scaling["rope_type"] = self.rope_scaling["type"]
+        rope_config_validation(self, ignore_keys={"mrope_section"})
+        self.image_token_id = image_token_id
+        self.video_token_id = video_token_id
+        super().__init__(tie_word_embeddings=tie_word_embeddings, **kwargs)
+
+
+class Fast_dVLMConfig(PretrainedConfig):
+
+    model_type = "fast_dvlm"
+    sub_configs = {"vision_config": Fast_dVLMVisionConfig, "text_config": Fast_dVLMTextConfig}
+    keys_to_ignore_at_inference = ["past_key_values"]
+
+    def __init__(
+        self,
+        text_config=None,
+        vision_config=None,
+        image_token_id=151655,
+        video_token_id=151656,
+        enable_efficient_vision_embed=False,
+        **kwargs,
+    ):
+        if isinstance(vision_config, dict):
+            self.vision_config = self.sub_configs["vision_config"](**vision_config)
+        elif vision_config is None:
+            self.vision_config = self.sub_configs["vision_config"]()
+
+        if isinstance(text_config, dict):
+            self.text_config = self.sub_configs["text_config"](**text_config)
+        elif text_config is None:
+            # For BC use all kwargs to init `TextConfig`
+            self.text_config = self.sub_configs["text_config"](**kwargs)
+
+        self.image_token_id = image_token_id
+        self.video_token_id = video_token_id
+        self.enable_efficient_vision_embed = enable_efficient_vision_embed
+
+        super().__init__(**kwargs)
+
+
+__all__ = ["Fast_dVLMConfig", "Fast_dVLMTextConfig"]

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 1e-06,
+  "transformers_version": "4.57.1"
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f5b214663a8acabe37b0087f5bdd8968a4b2d9e53ea585d2e8c23efcd282e23
+size 2816274784

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ce63fa22bac05f0e81e9a008154deb2371f586aa71b7e162485f7885fa8b095
+size 1880108072

preprocessor_config.json

@@ -0,0 +1,19 @@
+{
+  "min_pixels": 3136,
+  "max_pixels": 12845056,
+  "patch_size": 14,
+  "temporal_patch_size": 2,
+  "merge_size": 2,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "image_processor_type": "Qwen2VLImageProcessor",
+  "processor_class": "Qwen2_5_VLProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,25 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "|<MASK>|",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb2105b66192c5a532e2a098dc899df86eca233b4faa48461211e4312c8b3568
+size 11422081

tokenizer_config.json

@@ -0,0 +1,204 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "|<MASK>|",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "|<MASK>|"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}