Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,109 @@
+---
+license: mit
+base_model: nvidia/GLM-5.2-NVFP4
+base_model_relation: quantized
+pipeline_tag: text-generation
+tags:
+  - moe
+  - reap
+  - pruning
+  - expert-pruning
+  - nvfp4
+  - glm
+  - glm-5.2
+---
+
+# GLM-5.2-504B-Nvidia — REAP keep-168 of NVIDIA's official NVFP4 (no retraining)
+
+> A **34%-expert-pruned GLM-5.2**, cut directly from **[`nvidia/GLM-5.2-NVFP4`](https://huggingface.co/nvidia/GLM-5.2-NVFP4)**
+> using the **exact same REAP keep-168 expert selection** as [`0xSero/GLM-5.2-504B`](https://huggingface.co/0xSero/GLM-5.2-504B).
+> Whole-expert copy — **NVIDIA's NVFP4 weights and scales are preserved bit-for-bit**; no re-quantization, no fine-tuning.
+
+This is a **pure structural prune** of NVIDIA's NVFP4 checkpoint. It keeps the highest-saliency 168 of
+256 routed experts per layer and drops the rest. Unlike [`0xSero/GLM-5.2-504B`](https://huggingface.co/0xSero/GLM-5.2-504B),
+it is **not** Router-KD recovered — the router gates here are the original rows sliced down to the kept
+experts. Use the recommended sampler guardrail below (it fully recovers pruning-induced looping for free).
+
+---
+
+## What it is
+
+GLM-5.2 is a `GlmMoeDsaForCausalLM` MoE — **78 layers** (3 dense + 75 MoE) + **1 MTP** layer,
+**256 routed experts** per layer (top-8) + 1 shared expert, DeepSeek-style **MLA attention with a DSA
+sparse "indexer,"** hidden size 6144.
+
+This model keeps **168 of the 256 routed experts per layer** (≈**504B params**, down from ~744–763B),
+**uniformly across every MoE layer and the MTP layer** (`n_routed_experts: 168`), so it loads and
+serves cleanly in vLLM.
+
+| | |
+|---|---|
+| **Source** | [`nvidia/GLM-5.2-NVFP4`](https://huggingface.co/nvidia/GLM-5.2-NVFP4) (NVIDIA modelopt NVFP4) |
+| **Prune method** | **REAP** — saliency = `gate_weight × ‖expert_output‖`, top-168 kept per layer |
+| **Expert selection** | **identical** to [`0xSero/GLM-5.2-504B`](https://huggingface.co/0xSero/GLM-5.2-504B) (same REAP plan) |
+| **Recovery** | **none** — raw structural prune (gates sliced, not KD-retrained) |
+| **Quantization** | **NVFP4** on routed experts (3–77) + **FP8 KV cache**, preserved verbatim from NVIDIA; MTP layer 78 BF16 |
+| **Params** | ~504B (34% of routed experts pruned) |
+
+## How it was made
+
+A direct safetensors transform — **no GPU, no training, no re-quantization**:
+
+1. For each MoE layer (3–78), keep the top-168 experts from the REAP saliency plan; **copy each kept
+   expert whole** so its NVFP4 packing (`weight`, `weight_scale`, `weight_scale_2`, `input_scale`)
+   travels intact. NVIDIA's quantized values are bit-identical to the source.
+2. Renumber surviving experts `0..167` and **slice the router** (`gate.weight`,
+   `gate.e_score_correction_bias`) to the same 168 rows.
+3. Everything else — attention, shared expert, norms, embeddings, the MTP block — is copied verbatim.
+   `n_routed_experts` is set to 168.
+
+The MTP (next-token / multi-token-prediction) layer is preserved at 168 experts, so **self-speculative
+decoding still works**.
+
+## Serving (vLLM)
+
+```bash
+vllm serve 0xSero/GLM-5.2-504B-Nvidia \
+  --tensor-parallel-size 8 \
+  --quantization modelopt_fp4 \
+  --kv-cache-dtype fp8 \
+  --trust-remote-code \
+  --max-model-len 262144
+```
+
+### Recommended sampler guardrail — recover the loop cost for free
+
+REAP pruning roughly doubles GLM-5.2's tendency to fall into repeat / `</think>`-restart loops (the
+dominant agent-use failure mode). As established in the [GLM-5.2-504B report](https://huggingface.co/0xSero/GLM-5.2-504B),
+this is **fully recoverable at serving time with no retraining** via a light sampler guardrail
+(measured at n=2000 on the keep-168 cut):
+
+- **`min_p=0.05, repetition_penalty=1.05`** → gentle, safe default.
+- **`min_p=0.05, repetition_penalty=1.10`** → drops looping to ~2.3% (below the unpruned teacher's raw
+  3.6%). Start at 1.05; go to 1.10 if you see loops.
+
+Because this variant is **not** Router-KD recovered (the gates were not retrained), the guardrail is
+**strongly recommended** rather than optional. For the KD-recovered variant with the full evaluation,
+use [`0xSero/GLM-5.2-504B`](https://huggingface.co/0xSero/GLM-5.2-504B).
+
+## Relationship to the GLM-5.2 REAP series
+
+| model | source | recovery | use when |
+|---|---|---|---|
+| [`0xSero/GLM-5.2-504B`](https://huggingface.co/0xSero/GLM-5.2-504B) | 0xSero NVFP4 | Router-KD | you want the evaluated, recovered flagship |
+| **this model** | **NVIDIA NVFP4** | none | you want NVIDIA's exact quantization, pruned |
+| [`0xSero/GLM-5.2-REAP-504B-GGUF`](https://huggingface.co/0xSero/GLM-5.2-REAP-504B-GGUF) | BF16 | — | llama.cpp / CPU / Metal |
+
+> 📄 **Method, evaluation, and the honest accounting of pruning cost:** see the
+> [GLM-5.2-504B technical report](https://huggingface.co/0xSero/GLM-5.2-504B/blob/main/REPORT.md).
+
+## Provenance & honesty
+
+- Expert selection is the **same REAP keep-168 plan** that produced `0xSero/GLM-5.2-504B`.
+- NVFP4 routed-expert weights/scales are **NVIDIA's, unmodified** (whole-expert copy; no re-quant).
+- This specific variant was **not separately re-evaluated**; the behavioral numbers cited above come
+  from the keep-168 cut in the linked report and are referenced as guidance, not as a fresh measurement
+  of this checkpoint. It is **not** Router-KD recovered.
+
+---
+*REAP expert-pruning applied to NVIDIA's NVFP4 GLM-5.2. Quantization by NVIDIA; pruning recipe from the GLM-5.2 REAP study.*

chat_template.jinja

@@ -0,0 +1,119 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = 'high' if reasoning_effort is defined and reasoning_effort == 'high' else 'max' -%}
+{%- if (enable_thinking is not defined or enable_thinking) and effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- elif item is mapping and item.type in ['image', 'image_url', 'video', 'video_url', 'audio', 'audio_url', 'input_audio'] -%}
+                {%- set media_type = item.type | replace('_url', '') | replace('input_', '') -%}
+                {{- "<reminder>You are unable to process this " ~ media_type ~ " because you don't have multi-modal input ability. Try different methods.</reminder>" }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if ((clear_thinking is defined and not clear_thinking) or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' + tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+{%- endif %}
+{%- if m.content is string -%}
+    {{- '<tool_response>' + m.content + '</tool_response>' -}}
+{%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0.type == "tool_reference" -%}
+    {{- '<tool_response><tools>\n' -}}
+    {% for tr in m.content %}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0 is mapping and m.content.0.output is defined -%}
+    {%- for tr in m.content -%}
+        {{- '<tool_response>' + tr.output + '</tool_response>' -}}
+    {%- endfor -%}
+{%- else -%}
+    {{- '<tool_response>' + visible_text(m.content) + '</tool_response>' -}}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think></think>' if (enable_thinking is defined and not enable_thinking) else '<think>' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,496 @@
+{
+  "architectures": [
+    "GlmMoeDsaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "ep_size": 1,
+  "first_k_dense_replace": 3,
+  "head_dim": 192,
+  "hidden_act": "silu",
+  "hidden_size": 6144,
+  "index_head_dim": 128,
+  "index_n_heads": 32,
+  "index_share_for_mtp_iteration": true,
+  "index_skip_topk_offset": 3,
+  "index_topk": 2048,
+  "index_topk_freq": 4,
+  "index_topk_pattern": null,
+  "indexer_rope_interleave": true,
+  "indexer_types": [
+    "full",
+    "full",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared"
+  ],
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "kv_lora_rank": 512,
+  "layer_types": [
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention",
+    "deepseek_sparse_attention"
+  ],
+  "max_position_embeddings": 1048576,
+  "mlp_bias": false,
+  "mlp_layer_types": [
+    "dense",
+    "dense",
+    "dense",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse"
+  ],
+  "model_type": "glm_moe_dsa",
+  "moe_intermediate_size": 2048,
+  "moe_layer_freq": 1,
+  "n_group": 1,
+  "n_routed_experts": 168,
+  "n_shared_experts": 1,
+  "norm_topk_prob": true,
+  "num_attention_heads": 64,
+  "num_experts": 256,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 78,
+  "num_key_value_heads": 64,
+  "num_nextn_predict_layers": 1,
+  "pad_token_id": 154820,
+  "pretraining_tp": 1,
+  "q_lora_rank": 2048,
+  "qk_head_dim": 256,
+  "qk_nope_head_dim": 192,
+  "qk_rope_head_dim": 64,
+  "rms_norm_eps": 1e-05,
+  "rope_interleave": true,
+  "rope_parameters": {
+    "rope_theta": 8000000,
+    "rope_type": "default"
+  },
+  "routed_scaling_factor": 2.5,
+  "scoring_func": "sigmoid",
+  "tie_word_embeddings": false,
+  "topk_group": 1,
+  "topk_method": "noaux_tc",
+  "transformers_version": "5.11.0",
+  "use_cache": true,
+  "v_head_dim": 256,
+  "vocab_size": 154880,
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "weights": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "targets": [
+          "Linear"
+        ]
+      }
+    },
+    "ignore": [
+      "lm_head",
+      "model.embed_tokens",
+      "model.layers.0*",
+      "model.layers.1.*",
+      "model.layers.10.mlp.shared_experts*",
+      "model.layers.10.self_attn*",
+      "model.layers.11.mlp.shared_experts*",
+      "model.layers.11.self_attn*",
+      "model.layers.12.mlp.shared_experts*",
+      "model.layers.12.self_attn*",
+      "model.layers.13.mlp.shared_experts*",
+      "model.layers.13.self_attn*",
+      "model.layers.14.mlp.shared_experts*",
+      "model.layers.14.self_attn*",
+      "model.layers.15.mlp.shared_experts*",
+      "model.layers.15.self_attn*",
+      "model.layers.16.mlp.shared_experts*",
+      "model.layers.16.self_attn*",
+      "model.layers.17.mlp.shared_experts*",
+      "model.layers.17.self_attn*",
+      "model.layers.18.mlp.shared_experts*",
+      "model.layers.18.self_attn*",
+      "model.layers.19.mlp.shared_experts*",
+      "model.layers.19.self_attn*",
+      "model.layers.2.*",
+      "model.layers.20.mlp.shared_experts*",
+      "model.layers.20.self_attn*",
+      "model.layers.21.mlp.shared_experts*",
+      "model.layers.21.self_attn*",
+      "model.layers.22.mlp.shared_experts*",
+      "model.layers.22.self_attn*",
+      "model.layers.23.mlp.shared_experts*",
+      "model.layers.23.self_attn*",
+      "model.layers.24.mlp.shared_experts*",
+      "model.layers.24.self_attn*",
+      "model.layers.25.mlp.shared_experts*",
+      "model.layers.25.self_attn*",
+      "model.layers.26.mlp.shared_experts*",
+      "model.layers.26.self_attn*",
+      "model.layers.27.mlp.shared_experts*",
+      "model.layers.27.self_attn*",
+      "model.layers.28.mlp.shared_experts*",
+      "model.layers.28.self_attn*",
+      "model.layers.29.mlp.shared_experts*",
+      "model.layers.29.self_attn*",
+      "model.layers.3.mlp.shared_experts*",
+      "model.layers.3.self_attn*",
+      "model.layers.30.mlp.shared_experts*",
+      "model.layers.30.self_attn*",
+      "model.layers.31.mlp.shared_experts*",
+      "model.layers.31.self_attn*",
+      "model.layers.32.mlp.shared_experts*",
+      "model.layers.32.self_attn*",
+      "model.layers.33.mlp.shared_experts*",
+      "model.layers.33.self_attn*",
+      "model.layers.34.mlp.shared_experts*",
+      "model.layers.34.self_attn*",
+      "model.layers.35.mlp.shared_experts*",
+      "model.layers.35.self_attn*",
+      "model.layers.36.mlp.shared_experts*",
+      "model.layers.36.self_attn*",
+      "model.layers.37.mlp.shared_experts*",
+      "model.layers.37.self_attn*",
+      "model.layers.38.mlp.shared_experts*",
+      "model.layers.38.self_attn*",
+      "model.layers.39.mlp.shared_experts*",
+      "model.layers.39.self_attn*",
+      "model.layers.4.mlp.shared_experts*",
+      "model.layers.4.self_attn*",
+      "model.layers.40.mlp.shared_experts*",
+      "model.layers.40.self_attn*",
+      "model.layers.41.mlp.shared_experts*",
+      "model.layers.41.self_attn*",
+      "model.layers.42.mlp.shared_experts*",
+      "model.layers.42.self_attn*",
+      "model.layers.43.mlp.shared_experts*",
+      "model.layers.43.self_attn*",
+      "model.layers.44.mlp.shared_experts*",
+      "model.layers.44.self_attn*",
+      "model.layers.45.mlp.shared_experts*",
+      "model.layers.45.self_attn*",
+      "model.layers.46.mlp.shared_experts*",
+      "model.layers.46.self_attn*",
+      "model.layers.47.mlp.shared_experts*",
+      "model.layers.47.self_attn*",
+      "model.layers.48.mlp.shared_experts*",
+      "model.layers.48.self_attn*",
+      "model.layers.49.mlp.shared_experts*",
+      "model.layers.49.self_attn*",
+      "model.layers.5.mlp.shared_experts*",
+      "model.layers.5.self_attn*",
+      "model.layers.50.mlp.shared_experts*",
+      "model.layers.50.self_attn*",
+      "model.layers.51.mlp.shared_experts*",
+      "model.layers.51.self_attn*",
+      "model.layers.52.mlp.shared_experts*",
+      "model.layers.52.self_attn*",
+      "model.layers.53.mlp.shared_experts*",
+      "model.layers.53.self_attn*",
+      "model.layers.54.mlp.shared_experts*",
+      "model.layers.54.self_attn*",
+      "model.layers.55.mlp.shared_experts*",
+      "model.layers.55.self_attn*",
+      "model.layers.56.mlp.shared_experts*",
+      "model.layers.56.self_attn*",
+      "model.layers.57.mlp.shared_experts*",
+      "model.layers.57.self_attn*",
+      "model.layers.58.mlp.shared_experts*",
+      "model.layers.58.self_attn*",
+      "model.layers.59.mlp.shared_experts*",
+      "model.layers.59.self_attn*",
+      "model.layers.6.mlp.shared_experts*",
+      "model.layers.6.self_attn*",
+      "model.layers.60.mlp.shared_experts*",
+      "model.layers.60.self_attn*",
+      "model.layers.61.mlp.shared_experts*",
+      "model.layers.61.self_attn*",
+      "model.layers.62.mlp.shared_experts*",
+      "model.layers.62.self_attn*",
+      "model.layers.63.mlp.shared_experts*",
+      "model.layers.63.self_attn*",
+      "model.layers.64.mlp.shared_experts*",
+      "model.layers.64.self_attn*",
+      "model.layers.65.mlp.shared_experts*",
+      "model.layers.65.self_attn*",
+      "model.layers.66.mlp.shared_experts*",
+      "model.layers.66.self_attn*",
+      "model.layers.67.mlp.shared_experts*",
+      "model.layers.67.self_attn*",
+      "model.layers.68.mlp.shared_experts*",
+      "model.layers.68.self_attn*",
+      "model.layers.69.mlp.shared_experts*",
+      "model.layers.69.self_attn*",
+      "model.layers.7.mlp.shared_experts*",
+      "model.layers.7.self_attn*",
+      "model.layers.70.mlp.shared_experts*",
+      "model.layers.70.self_attn*",
+      "model.layers.71.mlp.shared_experts*",
+      "model.layers.71.self_attn*",
+      "model.layers.72.mlp.shared_experts*",
+      "model.layers.72.self_attn*",
+      "model.layers.73.mlp.shared_experts*",
+      "model.layers.73.self_attn*",
+      "model.layers.74.mlp.shared_experts*",
+      "model.layers.74.self_attn*",
+      "model.layers.75.mlp.shared_experts*",
+      "model.layers.75.self_attn*",
+      "model.layers.76.mlp.shared_experts*",
+      "model.layers.76.self_attn*",
+      "model.layers.77.mlp.shared_experts*",
+      "model.layers.77.self_attn*",
+      "model.layers.8.mlp.shared_experts*",
+      "model.layers.8.self_attn*",
+      "model.layers.9.mlp.shared_experts*",
+      "model.layers.9.self_attn*",
+      "model.layers.78*"
+    ],
+    "quant_algo": "NVFP4",
+    "kv_cache_scheme": {
+      "dynamic": false,
+      "num_bits": 8,
+      "type": "float"
+    },
+    "producer": {
+      "name": "modelopt",
+      "version": "0.46.0.dev65+g977d34dc3"
+    },
+    "quant_method": "modelopt"
+  }
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "do_sample": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.11.0"
+}

hf_quant_config.json

@@ -0,0 +1,169 @@
+{
+    "producer": {
+        "name": "modelopt",
+        "version": "0.46.0.dev65+g977d34dc3"
+    },
+    "quantization": {
+        "quant_algo": "NVFP4",
+        "kv_cache_quant_algo": "FP8",
+        "group_size": 16,
+        "exclude_modules": [
+            "lm_head",
+            "model.embed_tokens",
+            "model.layers.0*",
+            "model.layers.1.*",
+            "model.layers.10.mlp.shared_experts*",
+            "model.layers.10.self_attn*",
+            "model.layers.11.mlp.shared_experts*",
+            "model.layers.11.self_attn*",
+            "model.layers.12.mlp.shared_experts*",
+            "model.layers.12.self_attn*",
+            "model.layers.13.mlp.shared_experts*",
+            "model.layers.13.self_attn*",
+            "model.layers.14.mlp.shared_experts*",
+            "model.layers.14.self_attn*",
+            "model.layers.15.mlp.shared_experts*",
+            "model.layers.15.self_attn*",
+            "model.layers.16.mlp.shared_experts*",
+            "model.layers.16.self_attn*",
+            "model.layers.17.mlp.shared_experts*",
+            "model.layers.17.self_attn*",
+            "model.layers.18.mlp.shared_experts*",
+            "model.layers.18.self_attn*",
+            "model.layers.19.mlp.shared_experts*",
+            "model.layers.19.self_attn*",
+            "model.layers.2.*",
+            "model.layers.20.mlp.shared_experts*",
+            "model.layers.20.self_attn*",
+            "model.layers.21.mlp.shared_experts*",
+            "model.layers.21.self_attn*",
+            "model.layers.22.mlp.shared_experts*",
+            "model.layers.22.self_attn*",
+            "model.layers.23.mlp.shared_experts*",
+            "model.layers.23.self_attn*",
+            "model.layers.24.mlp.shared_experts*",
+            "model.layers.24.self_attn*",
+            "model.layers.25.mlp.shared_experts*",
+            "model.layers.25.self_attn*",
+            "model.layers.26.mlp.shared_experts*",
+            "model.layers.26.self_attn*",
+            "model.layers.27.mlp.shared_experts*",
+            "model.layers.27.self_attn*",
+            "model.layers.28.mlp.shared_experts*",
+            "model.layers.28.self_attn*",
+            "model.layers.29.mlp.shared_experts*",
+            "model.layers.29.self_attn*",
+            "model.layers.3.mlp.shared_experts*",
+            "model.layers.3.self_attn*",
+            "model.layers.30.mlp.shared_experts*",
+            "model.layers.30.self_attn*",
+            "model.layers.31.mlp.shared_experts*",
+            "model.layers.31.self_attn*",
+            "model.layers.32.mlp.shared_experts*",
+            "model.layers.32.self_attn*",
+            "model.layers.33.mlp.shared_experts*",
+            "model.layers.33.self_attn*",
+            "model.layers.34.mlp.shared_experts*",
+            "model.layers.34.self_attn*",
+            "model.layers.35.mlp.shared_experts*",
+            "model.layers.35.self_attn*",
+            "model.layers.36.mlp.shared_experts*",
+            "model.layers.36.self_attn*",
+            "model.layers.37.mlp.shared_experts*",
+            "model.layers.37.self_attn*",
+            "model.layers.38.mlp.shared_experts*",
+            "model.layers.38.self_attn*",
+            "model.layers.39.mlp.shared_experts*",
+            "model.layers.39.self_attn*",
+            "model.layers.4.mlp.shared_experts*",
+            "model.layers.4.self_attn*",
+            "model.layers.40.mlp.shared_experts*",
+            "model.layers.40.self_attn*",
+            "model.layers.41.mlp.shared_experts*",
+            "model.layers.41.self_attn*",
+            "model.layers.42.mlp.shared_experts*",
+            "model.layers.42.self_attn*",
+            "model.layers.43.mlp.shared_experts*",
+            "model.layers.43.self_attn*",
+            "model.layers.44.mlp.shared_experts*",
+            "model.layers.44.self_attn*",
+            "model.layers.45.mlp.shared_experts*",
+            "model.layers.45.self_attn*",
+            "model.layers.46.mlp.shared_experts*",
+            "model.layers.46.self_attn*",
+            "model.layers.47.mlp.shared_experts*",
+            "model.layers.47.self_attn*",
+            "model.layers.48.mlp.shared_experts*",
+            "model.layers.48.self_attn*",
+            "model.layers.49.mlp.shared_experts*",
+            "model.layers.49.self_attn*",
+            "model.layers.5.mlp.shared_experts*",
+            "model.layers.5.self_attn*",
+            "model.layers.50.mlp.shared_experts*",
+            "model.layers.50.self_attn*",
+            "model.layers.51.mlp.shared_experts*",
+            "model.layers.51.self_attn*",
+            "model.layers.52.mlp.shared_experts*",
+            "model.layers.52.self_attn*",
+            "model.layers.53.mlp.shared_experts*",
+            "model.layers.53.self_attn*",
+            "model.layers.54.mlp.shared_experts*",
+            "model.layers.54.self_attn*",
+            "model.layers.55.mlp.shared_experts*",
+            "model.layers.55.self_attn*",
+            "model.layers.56.mlp.shared_experts*",
+            "model.layers.56.self_attn*",
+            "model.layers.57.mlp.shared_experts*",
+            "model.layers.57.self_attn*",
+            "model.layers.58.mlp.shared_experts*",
+            "model.layers.58.self_attn*",
+            "model.layers.59.mlp.shared_experts*",
+            "model.layers.59.self_attn*",
+            "model.layers.6.mlp.shared_experts*",
+            "model.layers.6.self_attn*",
+            "model.layers.60.mlp.shared_experts*",
+            "model.layers.60.self_attn*",
+            "model.layers.61.mlp.shared_experts*",
+            "model.layers.61.self_attn*",
+            "model.layers.62.mlp.shared_experts*",
+            "model.layers.62.self_attn*",
+            "model.layers.63.mlp.shared_experts*",
+            "model.layers.63.self_attn*",
+            "model.layers.64.mlp.shared_experts*",
+            "model.layers.64.self_attn*",
+            "model.layers.65.mlp.shared_experts*",
+            "model.layers.65.self_attn*",
+            "model.layers.66.mlp.shared_experts*",
+            "model.layers.66.self_attn*",
+            "model.layers.67.mlp.shared_experts*",
+            "model.layers.67.self_attn*",
+            "model.layers.68.mlp.shared_experts*",
+            "model.layers.68.self_attn*",
+            "model.layers.69.mlp.shared_experts*",
+            "model.layers.69.self_attn*",
+            "model.layers.7.mlp.shared_experts*",
+            "model.layers.7.self_attn*",
+            "model.layers.70.mlp.shared_experts*",
+            "model.layers.70.self_attn*",
+            "model.layers.71.mlp.shared_experts*",
+            "model.layers.71.self_attn*",
+            "model.layers.72.mlp.shared_experts*",
+            "model.layers.72.self_attn*",
+            "model.layers.73.mlp.shared_experts*",
+            "model.layers.73.self_attn*",
+            "model.layers.74.mlp.shared_experts*",
+            "model.layers.74.self_attn*",
+            "model.layers.75.mlp.shared_experts*",
+            "model.layers.75.self_attn*",
+            "model.layers.76.mlp.shared_experts*",
+            "model.layers.76.self_attn*",
+            "model.layers.77.mlp.shared_experts*",
+            "model.layers.77.self_attn*",
+            "model.layers.8.mlp.shared_experts*",
+            "model.layers.8.self_attn*",
+            "model.layers.9.mlp.shared_experts*",
+            "model.layers.9.self_attn*",
+            "model.layers.78*"
+        ]
+    }
+}

model-00000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc9b6557d1a737e62890dba5359699abd25f133d85ed6053de5fe2debf9d496d
+size 5001457484

model-00001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:494c5a1b86722444e1df2a722465033279d94d5c46b2cf7f724dfdce6e6d89d2
+size 5001505840

model-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1cdd465db311505f383c0d51187ada3d78ca94880ee98eacd7a327b74ac657dd
+size 5000979180

model-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39dd7242eec9b65c7cc8235d800ff89db8f3836458f99931615041da90e8f014
+size 5006368128

model-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7c53a25c9079fe1c4ea62977d80effe150574681977ca714e4df6b42b52a848
+size 5000935528

model-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:001119736cb111cdec98d50ba6dabb338db350d44e15d8198a9af19032bcc1ec
+size 5001508136

model-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad8263f6d67022eb1192ac3c55a15b7e69afc9869c9f7ec2898fbfbf8dc75a18
+size 5003445816

model-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cfd6a8e50a223d0f45972691154d84d78f7bc08da26c1e928c9aaff448d3b7a
+size 5001508240

model-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:671107f5ea50e419939a7611a4f7d0cc291f36c2df53304e2ffde6e54c170eac
+size 5003445520

model-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:752d6b2fc4b4c53873cf54e4e67d169b97bc0b26f9df258d0fa1e59ae4d0a0d8
+size 5005650336

model-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5ee2157a65077e917ce4947e51734173e59da77072a75a9d107e2eb0b212c49
+size 5006095872

model-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d6f6acb413dc22ebaa232523ab2b5c98e21fbc8355df8b2e22b4769c005ddac
+size 5004134504

model-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7de3ad207a80ade664284d1c4ca85df095b8261e186eb9fbf7e7aaccb001d73
+size 5003159712

model-00020.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:630f50dcd3a01d63c21dca14b167386f2d250a0787ef3e022886ac89cf476492
+size 5001508320

model-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c864c9fd65377d2e0bdcc9cee10ca852e4d381415f91496b96352d956d14994f
+size 5005936104

model-00024.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d73092a8e321dbfb8d788620c059115d405df60b3d20a46a15687a7d8284f615
+size 5002803872

model-00025.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2abfd4f384cef42a25fadd8fc7e0e256e7dbadbe1684657e04e6463f7a5212d
+size 5000961880

model-00026.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d89b884ab707ac49ada071dcc899dce2a8622b613dfe0854f5534288507ce2c1
+size 5005309728

model-00027.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10f9f3491a9f38b4fcf2c87fe10c8c0725cb625ac168d2f676b16bec7aff41a2
+size 5005662808

model-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa22a97b8527611c74b2302576f4a8c5a2dc221602294c51c00f6cb08b0acd63
+size 5004121744

model-00030.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2e0ee8bcff43b3f9ca21890133025ab6275787bf4027638cd32b4bd6ce83a5e
+size 5003056084

model-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bd3fc612b1211ff5cb36a00f6e9b6d9be8c7b77c830d5f91ae47eeeaeaf6922
+size 5003445696

model-00034.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d78233134153921ee02aa03a49826334a7b32275c8f8d33f793f6480ee69bedd
+size 5084050724

model-00035.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba3c4cd9574e15f472962bd183d5ce248f7f8f6eb186a77d5d81f7e91df3ea0f
+size 5005569508

model-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d32d9477eb64ad9f0b196694a4d4ff3db8762061ac6f2a95af71129fcf243813
+size 5005936264

model-00037.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c040d7e29e274c25fb3c25f5a6abbf93a00c22ef1240b6dac27a45b9ca5102d0
+size 5004134528

model-00039.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b50c921139ece85ec6eca3e4f686e48f05b0721fa60a504f914643a0a1be4768
+size 5001508000

model-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d39329df4cb9184f36c4d42a51591c6fda04411bfce8f82a5bb58ccecace151
+size 5084312236

model-00041.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d1b016dee32292a751e372cb5f8d297098c2d289cc72618ede56c975462085a
+size 5003620628

model-00042.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1def347e91cdfd81721a08479232f1810e6f732546a8257ebea51582ce423c0
+size 5003159656

model-00043.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad6399c080b44851a2dee122a95c571d26d0e9b5883d7a8889bf1121d65eeb1b
+size 5001508192

model-00044.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e464f67964b874303c4f4bb8bfdd2822342df69e53f2a3e1b7a0314668e03822
+size 5001483256

model-00045.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de4386a69be80a137229bd468fd25b344d171c415d8189a155fd1bfc23e40430
+size 5006096320

model-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86d5c3c89a50ff333a5008f5a23e811bb468f770a28ef04a74e7dd35b2f2de7e
+size 5004121576

model-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfd9752a90bd281cc0132b739e0c42fe2efa7f6fae9e6e718d4a0f4a7a80ab30
+size 5001508472

model-00050.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0779e0ca348069ae9173d33b75c069b2e6dfeee99408172c31c5fcd0bbf2667
+size 5005662776

model-00053.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d91b982160d99aeef4ac98dc6fe962ee14d53547ff5e7a21557e42a71c9f7cae
+size 5001508200

model-00054.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6798ce66409ba255570031edccce42942dce87d2082dea602a144915d78e9b6
+size 5003172128

model-00058.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62c45b5e2ad9e412a7e760547646195f7b0b3a95a40d92668a422477fd6bd349
+size 5001863952

model-00059.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7889df89bae2af8a9449772717ed73f0f845dc3c38495164e40d75cc310d198b
+size 5084312220

model-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28cf2df47c4b7c8dc079c64813bd55f48d8914709d953d067b23799b2c283181
+size 5008036416

model-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06a120258a580e4d7c7de23fda1f4cca61fe720c0e6ce12b1f7ff16c939f24ba
+size 5008023872

tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 1048576,
+  "model_specific_special_tokens": {},
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "TokenizersBackend"
+}