Hugging Face's logo

FINAL-Bench
/
POCKET-35B-GGUF

Text Generation
GGUF
llama.cpp
conversational
on-device
mobile
iphone
android
cpu
local-llm
edge
mixture-of-experts
Mixture of Experts
quantized
pocket
vidraft
qwen3_5_moe
darwin
imatrix
Model card Files
xet
 Community
3
SeaWolf-AI commited on
Commit
de43f6e
Β·
1 Parent(s): 625b959

Unified POCKET header + badges + em-dash cleanup (#3)

Browse files

- Unified POCKET header + badges + em-dash cleanup (79b7b29f8b822b0ee3fe6a86231cefce735f25e0)

Files changed (1) hide show
  1. README.md +55 -47
README.md CHANGED
@@ -24,35 +24,40 @@ tags:
24
  - darwin
25
  ---
26
 
27
- > ### ⭐ Latest and flagship models
28
- > [![VKORE-46M-NPU](https://img.shields.io/badge/%F0%9F%86%95_VKORE--46M--NPU-pretraining_on_NPU-6c3fd1?style=for-the-badge)](https://huggingface.co/FINAL-Bench/VKORE-46M-NPU) [![Darwin-27B-ZTC-v5](https://img.shields.io/badge/Darwin--27B--ZTC--v5-S1MB_%231_EN+JP-ff6b8b?style=for-the-badge)](https://huggingface.co/FINAL-Bench/Darwin-27B-ZTC-v5)
29
- > [![POCKET family](https://img.shields.io/badge/%F0%9F%A4%97-POCKET_family-ffb020)](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6) [![Darwin family](https://img.shields.io/badge/%F0%9F%A4%97-Darwin_family-ffb020)](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) [![All models](https://img.shields.io/badge/%F0%9F%A4%97-FINAL--Bench-ffb020)](https://huggingface.co/FINAL-Bench) [![Website](https://img.shields.io/badge/%F0%9F%8C%90-vidraft.net-4ea8ff)](https://vidraft.net) [![THE VC](https://img.shields.io/badge/%F0%9F%93%8A-THE_VC-0a66c2)](https://thevc.kr/vidraft)
30
-
31
- > ### πŸ†• **[POCKET-Qwen3.8-Flash-Next](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF)** β€” a **180B** model running on a **laptop with 8 GB VRAM + 32 GB RAM** Β· **4.17 tok/s measured**.
 
 
 
 
 
 
32
  > [![New](https://img.shields.io/badge/πŸ†•_NEW-POCKET--180B_on_a_laptop-6c3fd1)](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) [![VRAM](https://img.shields.io/badge/VRAM-8_GB-1baf7a)]() [![RAM](https://img.shields.io/badge/RAM-32_GB-1baf7a)]() [![Speed](https://img.shields.io/badge/measured-4.17_tok%2Fs-243456)]() <!-- POCKET-FLASHNEXT-BADGE -->
33
 
34
- > ### πŸ†• **[POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)** β€” photoreal **images** in **46 s** on a **CPU only**. No GPU, no CUDA, no Python.
35
  > [![New](https://img.shields.io/badge/πŸ†•_NEW-POCKET--Zimage--CPU-ff6b35)](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) [![Space](https://img.shields.io/badge/πŸ€—_Space-try_on_CPU-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU) [![RAM](https://img.shields.io/badge/RAM-6.4_GB-1baf7a)]() <!-- POCKET-Zimage-CPU-BADGE -->
36
 
37
  > ### πŸ“š Collections
38
- > **β–Ά [POCKET Models](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)** β€” this family (on-device, no GPU)
39
  > [Darwin Family](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) Β· [Aether Foundation](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a) Β· [VKAE Accelerated](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a)
40
 
41
- > ### πŸ“Š **[VIDRAFT κΈ°μ—…μ •λ³΄Β·νˆ¬μž (THE VC) β†—](https://thevc.kr/vidraft)** Β· νˆ¬μžΒ·κΈ°μ—…κ°€μΉ˜Β·μ΅œμ‹  λ‰΄μŠ€
42
 
43
  ![POCKET](./pocket_hero.svg)
44
 
45
  # POCKET-35B-GGUF
46
 
47
- ### A **35B** model that runs on your **PC with no GPU** β€” and on your phone. Just stock `llama.cpp`. **No fork**, no CUDA, no cloud.
48
 
49
- > πŸš€ **Try it live, no install β†’** [![POCKET-35B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--35B_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) [![POCKET-26B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--26B_CPU_chat-0f9d6e)](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) β€” both answering on a **CPU-only** box (no GPU). POCKET-26B is Gemma4-based.
50
 
51
  [![License](https://img.shields.io/badge/License-Apache_2.0-0f6e56)](https://www.apache.org/licenses/LICENSE-2.0) [![Runtime](https://img.shields.io/badge/runtime-stock_llama.cpp-f0992a)](https://github.com/ggml-org/llama.cpp) [![No GPU](https://img.shields.io/badge/GPU-not_required-1baf7a)]() [![Base](https://img.shields.io/badge/base-Qwen3.5--35B--A3B-185fa5)]()
52
 
53
  **Pick your build β†’** [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![26B](https://img.shields.io/badge/POCKET--26B-GGUF-2a9d8f)](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) [![KR GGUF](https://img.shields.io/badge/POCKET--KR-GGUF-7A1F3D)](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-MLX_(iPhone)-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN GGUF](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) [![180B laptop](https://img.shields.io/badge/POCKET--180B-Qwen3.8_Flash--Next_Β·_laptop-6c3fd1)](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) [![Image NF4](https://img.shields.io/badge/POCKET--Image-Z--Image_NF4-d35400)](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) [![Image CPU](https://img.shields.io/badge/POCKET--Zimage-CPU-ff6b35)](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)
54
 
55
- ## The POCKET lineup β€” pick by your device
56
 
57
  | Repo | File | Size | Runs on | Best for | Korean PPL* |
58
  |---|---|---|---|---|---|
@@ -61,24 +66,24 @@ tags:
61
  | **POCKET-35B-GGUF** | `IQ1_M` | 8.2 GB | 16 GB RAM box | smallest full model | 9.69 |
62
  | **POCKET-KR-GGUF** | `IQ2_M` | 5.1 GB | Android 8 GB+ | πŸ‡°πŸ‡· Korean phone | 7.95 |
63
  | **POCKET-KR-MLX** | 2-bit | 5.1 GB | 🍎 **iPhone / iPad / Mac** | πŸ‡°πŸ‡· Korean, Apple-native | 7.95 |
64
- | **POCKET-EN-GGUF** | `iPhone-mix` | 5.3 GB | 🍎 iPhone (PocketPal) | 🌍 English phone | β€” |
65
- | **POCKET-EN-GGUF** | `PC-mix` | 6.8 GB | PC / Android | 🌍 English, best quality | β€” |
66
  | **POCKET-Qwen3.8-Flash-Next-GGUF** | `Q4_K_M` | 111 GiB | πŸ’» **laptop, 8 GB VRAM + 32 GB RAM** | **180B on a laptop** | 6.03† |
67
 
68
  *Wikipedia-Korean perplexity, lower is better. `Q4_K_M` = 5.79 baseline. English builds are tuned on English; see each repo.
69
- †Separate 80-chunk run (40,960 tokens) on a different model β€” compare within a model, not across rows.
70
 
71
- > 🍎 **Why MLX for Korean but GGUF for English on iPhone?** Apple-native MLX only does uniform quantization. Korean survives it; English needs our proprietary quantization, which only GGUF supports β€” so the English iPhone build ships as a GGUF you run with [PocketPal](https://github.com/a-ghorbani/pocketpal-ai). Honest, not lazy.
72
 
73
- > πŸ†• **POCKET-26B** β€” a **Gemma4-26B-A4B**-based sibling that loads in **any app today** (Ollama Β· LM Studio Β· PocketPal Β· MLX), no bleeding-edge runtime needed: **[GGUF](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF)** (`Q2_K` 11 GB Β· `Q4_K_M` 17 GB Β· GPQA-Diamond **67%**). Universal compatibility for 12 GB phones, PC, and browser.
74
 
75
  ![Speed vs Bonsai](./pocket_speed.svg)
76
 
77
- ## Benchmarks β€” what is measured, what is not
78
 
79
  **We measure Bonsai on the same machine with the same stock `llama.cpp`, and we tell you where we lose.**
80
 
81
- `[measured]` Generation speed β€” POCKET wins on both CPU and GPU:
82
 
83
  | | POCKET-35B IQ1_M | Bonsai-27B Q1_0 | |
84
  |---|---|---|---|
@@ -87,7 +92,7 @@ tags:
87
  | GPU prompt (H100) | 753 | **1816** | πŸ”΄ 0.41Γ— |
88
  | Quality (HellaSwag, 400q) | 61.0% | 60.0% | βšͺ tie (CI overlaps) |
89
 
90
- `[measured on a MacBook M3 Pro, 18 GB]` β€” and on a laptop, POCKET wins **every** axis, including prompt processing:
91
 
92
  | | POCKET-35B IQ1_M | Bonsai-27B Q1_0 | |
93
  |---|---|---|---|
@@ -96,9 +101,9 @@ tags:
96
  | Metal prompt (pp128) | **240.7 tok/s** | 73.4 | 🟒 3.28Γ— |
97
  | CPU prompt (pp128) | **45.5 tok/s** | 9.6 | 🟒 4.75Γ— |
98
 
99
- On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 is gone, so MoE sparsity wins across the board. `POCKET-35B-Q2_K` runs on the M3 Pro's CPU at **19.5 tok/s** β€” on an 18 GB Mac, run Q2_K on CPU (`-ngl 0`); its 13 GB exceeds the recommended Metal budget.
100
 
101
- `[measured β€” GPQA Diamond, 198q, greedy]` reasoning quality vs quantization:
102
 
103
  | Model | GPQA-Diamond (greedy) |
104
  |---|---|
@@ -106,9 +111,9 @@ On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 i
106
  | POCKET-35B Q4_K_M | 68.7% |
107
  | POCKET-35B Q2_K | 60.1% |
108
 
109
- `[pending β€” community reports welcome]` on-device **iPhone** and **Strix Halo** throughput. We publish only what we ran ourselves; help us fill the rest.
110
 
111
- > The same-size rival `Ternary-Bonsai-27B-Q2_0` (7.2 GB) **fails to load in upstream llama.cpp** β€” it needs the PrismML fork. POCKET runs on the tools you already have.
112
 
113
  ## Files in this repo
114
  | File | Size | bpw | Runs on | Korean PPL |
@@ -118,31 +123,31 @@ On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 i
118
  | **`POCKET-35B-Q2_K.gguf`** ⭐ | 13 GB | 2.6 | mini-PC 16–24 GB | 6.49 (**best value**) |
119
  | `POCKET-35B-IQ1_M.gguf` | 8.2 GB | 1.9 | 16 GB RAM | 9.69 (smallest) |
120
 
121
- ## Quickstart β€” no fork needed
122
  ```bash
123
- # any recent llama.cpp β€” brew / winget / apt, or LM Studio / Ollama
124
  llama-cli -m POCKET-35B-Q2_K.gguf -p "μ•ˆλ…•ν•˜μ„Έμš”" -ngl 0 -t 8
125
  # reproduce our CPU numbers:
126
  llama-bench -m POCKET-35B-IQ1_M.gguf -p 128 -n 64 -ngl 0 -t 16
127
  ```
128
- Use physical-core count for `-t` (max ~32). Do **not** pass all threads β€” it can slow down sharply.
129
 
130
- ## Lineage β€” where POCKET comes from
131
 
132
- POCKET is quantized from **[Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus)**, VIDRAFT's flagship β€” a model bred and evolved over several generations on the **Darwin platform** (crossbreeding, healing, expert surgery). Darwin-36B-Opus itself traces back to a Qwen3.5-family MoE architecture.
133
 
134
  | Component | Origin |
135
  |---|---|
136
- | **Starting checkpoint** | **[Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus)** β€” VIDRAFT, multi-generation Darwin evolution |
137
  | Base architecture | Qwen3.5-family MoE (256 experts, top-8), unchanged |
138
- | Quantization (`Q4_K_M`…`IQ1_M`) | stock llama.cpp β€” no custom format |
139
- | Runtime | upstream llama.cpp / Apple MLX β€” unmodified |
140
  | Proprietary language-specific tuning (KR/EN builds) | **ours** (VIDRAFT) |
141
 
142
- The CPU/GPU speed comes from the sparse-MoE architecture plus ordinary quantization β€” reproducible with the same base and the same tools. What we add is the Darwin-evolved weights, the honest measurement, the Korean tuning, and the pruning that makes the 5 GB phone builds.
143
 
144
  ## Limitations
145
- - The iPhone/Mac speed is **not yet measured by us** β€” community reports welcome.
146
  - Extreme quants (`IQ1_M`) hurt Korean ~2.8Γ— more than English; use `Q2_K` or larger for quality.
147
  - English phone builds trade quality for size; the PC build (`PC-mix`) is much closer to full quality.
148
 
@@ -153,33 +158,36 @@ Apache-2.0.
153
  *POCKET is a VIDRAFT model family. 35B, in your pocket. No GPU.*
154
 
155
  ## Learn more
156
- - On-device LLMs without a GPU β€” and how POCKET measures up: [Can you run a large LLM without a GPU?](https://vidraft.net/insights/on-device-llm-without-gpu.html)
157
  - What model quantization is, and why a 4-bit model stays smart: [What is model quantization?](https://vidraft.net/insights/what-is-quantization-llm.html)
158
 
159
  <!-- POCKET-FAMILY -->
160
 
161
  ---
162
 
163
- ## 🧩 The POCKET Family β€” On-device AI by VIDRAFT
164
 
165
  *Big models, small hardware. No GPU, no cloud.*
166
 
167
  **Models**
168
- - πŸ“¦ [POCKET-35B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) β€” flagship, PC / server, no GPU
169
- - πŸ“¦ [POCKET-26B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) β€” compact 26B
170
- - πŸ‡°πŸ‡· [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) β€” Korean, Android
171
- - 🍎 [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) β€” Korean, iPhone / Mac
172
- - 🌍 [POCKET-EN-GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) β€” English, phone / PC
173
- - πŸ’» [POCKET-Qwen3.8-Flash-Next-GGUF](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) β€” **180B on a laptop** (8 GB VRAM + 32 GB RAM)
174
- - πŸ–ΌοΈ [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) β€” character-perfect text in any image
175
- - πŸ–₯️ [POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) β€” photoreal images on a CPU only
176
 
177
  **Demos & tools (Spaces)**
178
- - 🎨 [POCKET-Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio) β€” text-in-image, generate in-page
179
- - πŸ–₯️ [POCKET-35B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) β€” 35B answering on a CPU
180
- - πŸ–₯️ [POCKET-26B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) β€” 26B on a CPU
181
- - πŸ–ΌοΈ [POCKET-Zimage-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU) β€” image generation on a CPU
182
 
183
  πŸ“š [Full POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)
184
 
185
  <!-- /POCKET-FAMILY -->
 
 
 
 
24
  - darwin
25
  ---
26
 
27
+ > ### ⭐ VIDRAFT · POCKET family
28
+ > [![Website](https://img.shields.io/badge/Website-vidraft.net-111?style=for-the-badge&logo=firefoxbrowser&logoColor=white)](https://vidraft.net) [![LinkedIn](https://img.shields.io/badge/LinkedIn-VIDRAFT-0A66C2?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/vidraft/) [![Threads](https://img.shields.io/badge/Threads-@vidraft__lab-111?style=for-the-badge&logo=threads&logoColor=white)](https://www.threads.com/@vidraft_lab) [![X](https://img.shields.io/badge/X-@VIDRAFT__ai-111?style=for-the-badge&logo=x&logoColor=white)](https://x.com/VIDRAFT_ai)
29
+ >
30
+ > [![POCKET-35B-GGUF](https://img.shields.io/badge/β–Ά_POCKET--35B-THIS-16a34a?style=for-the-badge)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![POCKET-Darwin-180B-GGUF](https://img.shields.io/badge/POCKET--Darwin--180B-FLAGSHIP-7c3aed?style=for-the-badge)](https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF) [![POCKET-26B-GGUF](https://img.shields.io/badge/POCKET--26B-LIGHTWEIGHT-2563eb?style=for-the-badge)](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF)
31
+ >
32
+ > [![S1MB English #1](https://img.shields.io/badge/S1MB_English-%231_of_197_Β·_Borda_90.38-ff5a1f?style=for-the-badge)](https://huggingface.co/FINAL-Bench/Darwin-27B-ZTC-v9) [![Fast Gemma](https://img.shields.io/badge/Google%C3%97HF_Fast_Gemma-%231_verified_record-16a34a?style=for-the-badge)](https://vidraft.net) [![Live Demo](https://img.shields.io/badge/β–Ά_Live_Demo-Aether_Sovereign_AI-000?style=for-the-badge&logo=huggingface)](https://huggingface.co/spaces/FINAL-Bench/Aether-Sovereign-AI)
33
+ >
34
+ > [![POCKET family](https://img.shields.io/badge/%F0%9F%A4%97-POCKET_family-ffb020)](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6) [![Darwin family](https://img.shields.io/badge/%F0%9F%A4%97-Darwin_family-ffb020)](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) [![All models](https://img.shields.io/badge/%F0%9F%A4%97-FINAL--Bench-ffb020)](https://huggingface.co/FINAL-Bench)
35
+ > [![VKORE-46M-NPU](https://img.shields.io/badge/VKORE--46M--NPU-pretraining_on_NPU-6c3fd1)](https://huggingface.co/FINAL-Bench/VKORE-46M-NPU)
36
+
37
+ > ### πŸ†• **[POCKET-Qwen3.8-Flash-Next](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF)**: a **180B** model running on a **laptop with 8 GB VRAM + 32 GB RAM** Β· **4.17 tok/s measured**.
38
  > [![New](https://img.shields.io/badge/πŸ†•_NEW-POCKET--180B_on_a_laptop-6c3fd1)](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) [![VRAM](https://img.shields.io/badge/VRAM-8_GB-1baf7a)]() [![RAM](https://img.shields.io/badge/RAM-32_GB-1baf7a)]() [![Speed](https://img.shields.io/badge/measured-4.17_tok%2Fs-243456)]() <!-- POCKET-FLASHNEXT-BADGE -->
39
 
40
+ > ### πŸ†• **[POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)**: photoreal **images** in **46 s** on a **CPU only**. No GPU, no CUDA, no Python.
41
  > [![New](https://img.shields.io/badge/πŸ†•_NEW-POCKET--Zimage--CPU-ff6b35)](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) [![Space](https://img.shields.io/badge/πŸ€—_Space-try_on_CPU-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU) [![RAM](https://img.shields.io/badge/RAM-6.4_GB-1baf7a)]() <!-- POCKET-Zimage-CPU-BADGE -->
42
 
43
  > ### πŸ“š Collections
44
+ > **β–Ά [POCKET Models](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)**: this family (on-device, no GPU)
45
  > [Darwin Family](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) Β· [Aether Foundation](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a) Β· [VKAE Accelerated](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a)
46
 
 
47
 
48
  ![POCKET](./pocket_hero.svg)
49
 
50
  # POCKET-35B-GGUF
51
 
52
+ ### A **35B** model that runs on your **PC with no GPU**, and on your phone. Just stock `llama.cpp`. **No fork**, no CUDA, no cloud.
53
 
54
+ > πŸš€ **Try it live, no install β†’** [![POCKET-35B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--35B_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) [![POCKET-26B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--26B_CPU_chat-0f9d6e)](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU): both answering on a **CPU-only** box (no GPU). POCKET-26B is Gemma4-based.
55
 
56
  [![License](https://img.shields.io/badge/License-Apache_2.0-0f6e56)](https://www.apache.org/licenses/LICENSE-2.0) [![Runtime](https://img.shields.io/badge/runtime-stock_llama.cpp-f0992a)](https://github.com/ggml-org/llama.cpp) [![No GPU](https://img.shields.io/badge/GPU-not_required-1baf7a)]() [![Base](https://img.shields.io/badge/base-Qwen3.5--35B--A3B-185fa5)]()
57
 
58
  **Pick your build β†’** [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![26B](https://img.shields.io/badge/POCKET--26B-GGUF-2a9d8f)](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) [![KR GGUF](https://img.shields.io/badge/POCKET--KR-GGUF-7A1F3D)](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-MLX_(iPhone)-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN GGUF](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) [![180B laptop](https://img.shields.io/badge/POCKET--180B-Qwen3.8_Flash--Next_Β·_laptop-6c3fd1)](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) [![Image NF4](https://img.shields.io/badge/POCKET--Image-Z--Image_NF4-d35400)](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) [![Image CPU](https://img.shields.io/badge/POCKET--Zimage-CPU-ff6b35)](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)
59
 
60
+ ## The POCKET lineup, pick by your device
61
 
62
  | Repo | File | Size | Runs on | Best for | Korean PPL* |
63
  |---|---|---|---|---|---|
 
66
  | **POCKET-35B-GGUF** | `IQ1_M` | 8.2 GB | 16 GB RAM box | smallest full model | 9.69 |
67
  | **POCKET-KR-GGUF** | `IQ2_M` | 5.1 GB | Android 8 GB+ | πŸ‡°πŸ‡· Korean phone | 7.95 |
68
  | **POCKET-KR-MLX** | 2-bit | 5.1 GB | 🍎 **iPhone / iPad / Mac** | πŸ‡°πŸ‡· Korean, Apple-native | 7.95 |
69
+ | **POCKET-EN-GGUF** | `iPhone-mix` | 5.3 GB | 🍎 iPhone (PocketPal) | 🌍 English phone | - |
70
+ | **POCKET-EN-GGUF** | `PC-mix` | 6.8 GB | PC / Android | 🌍 English, best quality | - |
71
  | **POCKET-Qwen3.8-Flash-Next-GGUF** | `Q4_K_M` | 111 GiB | πŸ’» **laptop, 8 GB VRAM + 32 GB RAM** | **180B on a laptop** | 6.03† |
72
 
73
  *Wikipedia-Korean perplexity, lower is better. `Q4_K_M` = 5.79 baseline. English builds are tuned on English; see each repo.
74
+ †Separate 80-chunk run (40,960 tokens) on a different model, compare within a model, not across rows.
75
 
76
+ > 🍎 **Why MLX for Korean but GGUF for English on iPhone?** Apple-native MLX only does uniform quantization. Korean survives it; English needs our proprietary quantization, which only GGUF supports, so the English iPhone build ships as a GGUF you run with [PocketPal](https://github.com/a-ghorbani/pocketpal-ai). Honest, not lazy.
77
 
78
+ > πŸ†• **POCKET-26B**: a **Gemma4-26B-A4B**-based sibling that loads in **any app today** (Ollama Β· LM Studio Β· PocketPal Β· MLX), no bleeding-edge runtime needed: **[GGUF](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF)** (`Q2_K` 11 GB Β· `Q4_K_M` 17 GB Β· GPQA-Diamond **67%**). Universal compatibility for 12 GB phones, PC, and browser.
79
 
80
  ![Speed vs Bonsai](./pocket_speed.svg)
81
 
82
+ ## Benchmarks, what is measured, what is not
83
 
84
  **We measure Bonsai on the same machine with the same stock `llama.cpp`, and we tell you where we lose.**
85
 
86
+ `[measured]` Generation speed, POCKET wins on both CPU and GPU:
87
 
88
  | | POCKET-35B IQ1_M | Bonsai-27B Q1_0 | |
89
  |---|---|---|---|
 
92
  | GPU prompt (H100) | 753 | **1816** | πŸ”΄ 0.41Γ— |
93
  | Quality (HellaSwag, 400q) | 61.0% | 60.0% | βšͺ tie (CI overlaps) |
94
 
95
+ `[measured on a MacBook M3 Pro, 18 GB]`, and on a laptop, POCKET wins **every** axis, including prompt processing:
96
 
97
  | | POCKET-35B IQ1_M | Bonsai-27B Q1_0 | |
98
  |---|---|---|---|
 
101
  | Metal prompt (pp128) | **240.7 tok/s** | 73.4 | 🟒 3.28Γ— |
102
  | CPU prompt (pp128) | **45.5 tok/s** | 9.6 | 🟒 4.75Γ— |
103
 
104
+ On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 is gone, so MoE sparsity wins across the board. `POCKET-35B-Q2_K` runs on the M3 Pro's CPU at **19.5 tok/s**: on an 18 GB Mac, run Q2_K on CPU (`-ngl 0`); its 13 GB exceeds the recommended Metal budget.
105
 
106
+ `[measured, GPQA Diamond, 198q, greedy]` reasoning quality vs quantization:
107
 
108
  | Model | GPQA-Diamond (greedy) |
109
  |---|---|
 
111
  | POCKET-35B Q4_K_M | 68.7% |
112
  | POCKET-35B Q2_K | 60.1% |
113
 
114
+ `[pending, community reports welcome]` on-device **iPhone** and **Strix Halo** throughput. We publish only what we ran ourselves; help us fill the rest.
115
 
116
+ > The same-size rival `Ternary-Bonsai-27B-Q2_0` (7.2 GB) **fails to load in upstream llama.cpp**: it needs the PrismML fork. POCKET runs on the tools you already have.
117
 
118
  ## Files in this repo
119
  | File | Size | bpw | Runs on | Korean PPL |
 
123
  | **`POCKET-35B-Q2_K.gguf`** ⭐ | 13 GB | 2.6 | mini-PC 16–24 GB | 6.49 (**best value**) |
124
  | `POCKET-35B-IQ1_M.gguf` | 8.2 GB | 1.9 | 16 GB RAM | 9.69 (smallest) |
125
 
126
+ ## Quickstart, no fork needed
127
  ```bash
128
+ # any recent llama.cpp, brew / winget / apt, or LM Studio / Ollama
129
  llama-cli -m POCKET-35B-Q2_K.gguf -p "μ•ˆλ…•ν•˜μ„Έμš”" -ngl 0 -t 8
130
  # reproduce our CPU numbers:
131
  llama-bench -m POCKET-35B-IQ1_M.gguf -p 128 -n 64 -ngl 0 -t 16
132
  ```
133
+ Use physical-core count for `-t` (max ~32). Do **not** pass all threads, it can slow down sharply.
134
 
135
+ ## Lineage, where POCKET comes from
136
 
137
+ POCKET is quantized from **[Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus)**, VIDRAFT's flagship, a model bred and evolved over several generations on the **Darwin platform** (crossbreeding, healing, expert surgery). Darwin-36B-Opus itself traces back to a Qwen3.5-family MoE architecture.
138
 
139
  | Component | Origin |
140
  |---|---|
141
+ | **Starting checkpoint** | **[Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus)**: VIDRAFT, multi-generation Darwin evolution |
142
  | Base architecture | Qwen3.5-family MoE (256 experts, top-8), unchanged |
143
+ | Quantization (`Q4_K_M`…`IQ1_M`) | stock llama.cpp, no custom format |
144
+ | Runtime | upstream llama.cpp / Apple MLX, unmodified |
145
  | Proprietary language-specific tuning (KR/EN builds) | **ours** (VIDRAFT) |
146
 
147
+ The CPU/GPU speed comes from the sparse-MoE architecture plus ordinary quantization, reproducible with the same base and the same tools. What we add is the Darwin-evolved weights, the honest measurement, the Korean tuning, and the pruning that makes the 5 GB phone builds.
148
 
149
  ## Limitations
150
+ - The iPhone/Mac speed is **not yet measured by us**: community reports welcome.
151
  - Extreme quants (`IQ1_M`) hurt Korean ~2.8Γ— more than English; use `Q2_K` or larger for quality.
152
  - English phone builds trade quality for size; the PC build (`PC-mix`) is much closer to full quality.
153
 
 
158
  *POCKET is a VIDRAFT model family. 35B, in your pocket. No GPU.*
159
 
160
  ## Learn more
161
+ - On-device LLMs without a GPU, and how POCKET measures up: [Can you run a large LLM without a GPU?](https://vidraft.net/insights/on-device-llm-without-gpu.html)
162
  - What model quantization is, and why a 4-bit model stays smart: [What is model quantization?](https://vidraft.net/insights/what-is-quantization-llm.html)
163
 
164
  <!-- POCKET-FAMILY -->
165
 
166
  ---
167
 
168
+ ## 🧩 The POCKET Family, On-device AI by VIDRAFT
169
 
170
  *Big models, small hardware. No GPU, no cloud.*
171
 
172
  **Models**
173
+ - πŸ“¦ [POCKET-35B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF): flagship, PC / server, no GPU
174
+ - πŸ“¦ [POCKET-26B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF): compact 26B
175
+ - πŸ‡°πŸ‡· [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF): Korean, Android
176
+ - 🍎 [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX): Korean, iPhone / Mac
177
+ - 🌍 [POCKET-EN-GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF): English, phone / PC
178
+ - πŸ’» [POCKET-Qwen3.8-Flash-Next-GGUF](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF): **180B on a laptop** (8 GB VRAM + 32 GB RAM)
179
+ - πŸ–ΌοΈ [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage): character-perfect text in any image
180
+ - πŸ–₯️ [POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU): photoreal images on a CPU only
181
 
182
  **Demos & tools (Spaces)**
183
+ - 🎨 [POCKET-Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio): text-in-image, generate in-page
184
+ - πŸ–₯️ [POCKET-35B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU): 35B answering on a CPU
185
+ - πŸ–₯️ [POCKET-26B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU): 26B on a CPU
186
+ - πŸ–ΌοΈ [POCKET-Zimage-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU): image generation on a CPU
187
 
188
  πŸ“š [Full POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)
189
 
190
  <!-- /POCKET-FAMILY -->
191
+
192
+ ---
193
+ [VIDRAFT κΈ°μ—…μ •λ³΄Β·νˆ¬μž (THE VC)](https://thevc.kr/vidraft) Β· [vidraft.net](https://vidraft.net)