Blame for README.md

---
license: apache-2.0
library_name: llama.cpp
pipeline_tag: text-generation
base_model:
- FINAL-Bench/Darwin-36B-Opus
tags:
- gguf
- llama.cpp
- conversational
- on-device
- mobile
- iphone
- android
- cpu
- local-llm
- edge
- mixture-of-experts
- moe
- quantized
- pocket
- vidraft
- qwen3_5_moe
- darwin
---
<!-- POCKET-27B-DISCOVERY:START -->
<p align="center">
  <a href="https://huggingface.co/FINAL-Bench/POCKET-27B-GGUF"><img src="https://img.shields.io/badge/NEW%20POCKET--27B-Q6__K%20%C2%B7%2022.43%20GB-55cbb4?style=for-the-badge" alt="New POCKET-27B: Qwen3.8-27B, Q6_K, 22.43 GB - model card and downloads"></a>
</p>
<p align="center"><b>Meet POCKET-27B.</b> Qwen3.8-27B in ARCHsloth Q6_K GGUF. <a href="https://huggingface.co/FINAL-Bench/POCKET-27B-GGUF/tree/main">Choose files &amp; download</a> · <a href="https://huggingface.co/FINAL-Bench/POCKET-27B-GGUF#quality-and-evaluation">Quality &amp; limits</a></p>
<!-- POCKET-27B-DISCOVERY:END -->

<!-- SUPERCLUE-2026-11:START -->
## 🏆 SuperCLUE November 2026: #1 open-weight model in math and science reasoning

[![SuperCLUE Nov 2026: Darwin-180B-RSI #1 open-weight, #2 overall in math and science reasoning](https://huggingface.co/datasets/VIDraft/press-assets/resolve/main/press/superclue_banner_darwin_2026-11.png)](https://www.superclueai.com/homepage)

**From the VIDRAFT Darwin family.** Our flagship reasoning model, [Darwin-180B-RSI-R3](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI-R3), was scored by [SuperCLUE](https://www.superclueai.com/homepage), China's most widely cited LLM benchmark, on its private November 2026 test set: **73.68 in mathematical and scientific reasoning, #1 among open-weight models and #2 overall** among 19 models, behind only OpenAI's GPT-6.1-Sol and ahead of Gemini-4-Argon, DeepSeek-V4.1-Flash, Grok-4.7 and Qwen3.8-Max. The same recursive self-improvement line and the same team built the POCKET on-device builds.

| Model | Developer | Open weights | Math & science reasoning |
|---|---|---|---|
| GPT-6.1-Sol | OpenAI | No | 76.32 |
| **Darwin-180B-RSI-R3** | **VIDRAFT** | **Yes** | **73.68** |
| Gemini-4-Argon | Google | No | 71.05 |
| DeepSeek-V4.1-Flash | DeepSeek | Yes | 71.05 |
| Grok-4.7 | xAI | No | 68.42 |
| Qwen3.8-Max-0902 | Alibaba | No | 68.42 |
| GLM-5.3 | Zhipu AI | Yes | 65.79 |
| Kimi-K3.1-Pro | Moonshot | Yes | 63.16 |
| Qwen3.8-Flash (parent of Darwin-180B) | Alibaba | Yes | 57.89 |

Darwin-180B-RSI inherited its expert layers from Qwen3.8-Flash-Next and rebuilt only the attention and shared-expert layers through recursive self-improvement. On the same board the parent, Qwen3.8-Flash, placed 15th overall (39.81) with 57.89 in math and science reasoning; Darwin placed 9th (43.08) with 73.68.

<details><summary>Overall intelligence index, November 2026 (screenshot)</summary>

![SuperCLUE Nov 2026 intelligence index](https://huggingface.co/datasets/VIDraft/press-assets/resolve/main/press/superclue_intelligence_index_2026-11.png)

</details>

Source: [SuperCLUE general leaderboard, November 2026](https://www.superclueai.com/homepage) (screenshots taken November 5, 2026). Scores are SuperCLUE's, not VIDRAFT's.
<!-- SUPERCLUE-2026-11:END -->


# POCKET-35B-GGUF

Korean-tuned 35B local AI for PCs: choose a full-model GGUF build and run without a GPU.

![POCKET-35B overview: full-model PC builds, Q2_K recommended, CPU-only option](./pocket-overview.svg)

[Download files](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/tree/main) · [Setup guide](https://vidraft.net/p/pocket.html?model=pocket_35b&utm_source=huggingface&utm_medium=referral&utm_campaign=pocket_2026q4&utm_content=pocket_35b_model_card_top) · [Benchmarks](#quality-and-evaluation)

[POCKET-35B](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) · [POCKET-26B](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) · [POCKET-Darwin-180B](https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF)

## At a glance

| Item | Details |
|---|---|
| Artifact | Full 34.66B-parameter sparse MoE, approximately 3B active parameters per token; GGUF |
| Starting checkpoint | [FINAL-Bench/Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus) |
| Recommended first build | Q2_K, approximately 13 GB |
| Hardware | PC with sufficient RAM; a CPU-only execution path is available |
| Runtime | llama.cpp with qwen35moe architecture support; check the installed build |
| License | Apache-2.0; see [License and attribution](#license-and-attribution) |
| Evidence status | Publisher measurements with device-specific conditions below |

This repository contains the **full-model PC builds**, not the approximately 5 GB phone derivatives. For a phone, use the separate [Korean GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF), [Korean MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX), or [English GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) repository and its own requirements.

## Choose your build

| Build | File size | Memory guidance | Best for |
|---|---|---|---|
| [Q2_K — recommended](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/POCKET-35B-Q2_K.gguf) | About 13 GB | Publisher guidance: 16–24 GB PC RAM; allow OS and context headroom | Balanced daily use |
| [Q4_K_M](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/POCKET-35B-Q4_K_M.gguf) | About 21 GB | Publisher guidance: 32 GB PC RAM | Highest measured Korean quality among these builds |
| [Q3_K_M](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/POCKET-35B-Q3_K_M.gguf) | About 16 GB | Publisher guidance: 24 GB PC RAM | Intermediate size and quality |
| [IQ1_M](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/POCKET-35B-IQ1_M.gguf) | About 8.2 GB | Publisher guidance: 16 GB PC RAM | Smallest full-model file; greater quality loss |

File sizes are rounded decimal GB, not process-memory measurements. Context, runtime buffers, the operating system and other applications need additional memory. The approximately 8.2 GB IQ1_M file is approximately 7.7 GiB; do not interpret those units as different builds.

## Download and run

Install the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) and a [llama.cpp build](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md) that supports qwen35moe. The example downloads only the recommended file from the audited revision; it does not clone or download the whole repository.

**1. Download the recommended build.**

```bash
hf download FINAL-Bench/POCKET-35B-GGUF POCKET-35B-Q2_K.gguf --revision 3870039a87406fb90764b27e3984c8b1a36b269e --local-dir ./POCKET-35B
```

**2. Run on the CPU.**

```bash
llama-cli -m ./POCKET-35B/POCKET-35B-Q2_K.gguf --device none -ngl 0 -t 8 -c 4096 -n 256 -p "Explain why local AI can be useful."
```

The expected result is generated text in the terminal, not a fixed benchmark answer. These command examples were syntax-reviewed but not executed during this review.

Commands use the standalone llama.cpp executable names. Some distributions expose a unified `llama` launcher; consult that distribution's help and the [official model-loading documentation](https://github.com/ggml-org/llama.cpp/blob/master/docs/models.md). For `-t`, start with the physical-core count rather than all logical threads; more threads can reduce throughput.

For installation, memory errors and app-specific help, use the [model-specific setup guide](https://vidraft.net/p/pocket.html?model=pocket_35b&utm_source=huggingface&utm_medium=referral&utm_campaign=pocket_2026q4&utm_content=pocket_35b_quickstart_help). Model files remain directly available from Hugging Face without a VIDRAFT login.

## Runtime compatibility

| Runtime | Requirement | Evidence and limits |
|---|---|---|
| llama.cpp | A build with qwen35moe support | The original card reports upstream llama.cpp use; an exact build/hash was not supplied for every measurement |
| Ollama | A version supporting this model architecture and GGUF | Generic [Hugging Face GGUF integration](https://huggingface.co/docs/hub/ollama) exists; an exact file/version compatibility matrix is not supplied |
| LM Studio | A bundled runtime supporting this model architecture | Paste the repository URL into model search; check the app's compatibility information and available memory |
| PocketPal and phone runtimes | Use a separately supplied phone derivative and its requirements | No performance guarantee for these full-model PC files on phones |
| Apple MLX | A separately supplied MLX artifact, not this GGUF file | See [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) |

The same file format does not guarantee support in every app or version. A dated, exact-version compatibility matrix remains a community contribution opportunity.

## Hardware and performance

These are publisher-reported measurements, not promised speeds for arbitrary PCs. The CPU-only quickstart explicitly disables GPU acceleration with `--device none`; `-ngl 0` alone is not the strongest no-GPU guarantee in every backend.

| Device or mode | Configuration | Generation | Evidence and limits |
|---|---|---|---|
| Server CPU | Xeon, 16 threads; IQ1_M | 27.0 tok/s | Same-machine comparison below; exact CPU SKU/build and run repetitions were not supplied |
| Datacenter GPU | H100; IQ1_M | 197 tok/s | GPU result, not CPU-only performance |
| Mac CPU | MacBook M3 Pro, 18 GB, 8 threads; IQ1_M | 13.8 tok/s | Measured in the original card |
| Mac Metal | MacBook M3 Pro, 18 GB; IQ1_M, tg64 | 25.4 tok/s | Metal uses the integrated GPU |
| Mac CPU | MacBook M3 Pro, 18 GB; Q2_K | 19.5 tok/s | Different quant from the headline IQ1_M measurement |
| iPhone / Strix Halo | No matching throughput measurement supplied | Not measured | Do not infer speed from file size or a related platform |

On the 18 GB M3 Pro, the original card recommends Q2_K on CPU because its approximately 13 GB file exceeds the recommended Metal memory budget. This is a device-specific recommendation, not a universal Apple-memory limit.

<details>
<summary>Same-machine Bonsai comparison and benchmark command</summary>

| Measurement | POCKET-35B IQ1_M | Bonsai-27B Q1_0 | Conditions |
|---|---:|---:|---|
| Server CPU generation | 27.0 tok/s | 10.1 tok/s | Xeon, 16 threads |
| H100 generation | 197 tok/s | 89 tok/s | Same stock llama.cpp comparison |
| H100 prompt processing | 753 tok/s | 1,816 tok/s | Bonsai is faster on this measurement |
| M3 Pro Metal generation | 25.4 tok/s | 12.8 tok/s | tg64, 18 GB MacBook |
| M3 Pro CPU generation | 13.8 tok/s | 4.4 tok/s | 8 threads |
| M3 Pro Metal prompt processing | 240.7 tok/s | 73.4 tok/s | pp128 |
| M3 Pro CPU prompt processing | 45.5 tok/s | 9.6 tok/s | pp128 |

The configurations do not establish equal quality, equal file size or a universal architecture advantage. The original card separately reports that Ternary-Bonsai-27B-Q2_0 required a PrismML fork; compatibility can change with releases.

To inspect the original CPU benchmark shape, first download IQ1_M instead of Q2_K, then run:

```bash
hf download FINAL-Bench/POCKET-35B-GGUF POCKET-35B-IQ1_M.gguf --revision 3870039a87406fb90764b27e3984c8b1a36b269e --local-dir ./POCKET-35B
llama-bench -m ./POCKET-35B/POCKET-35B-IQ1_M.gguf -p 128 -n 64 --device none -ngl 0 -t 16
```

This is a reproduction starting point, not proof of the published result on a different computer or runtime.

</details>

## Quality and evaluation

Publisher-reported results. Each table has its own task, model and measurement conditions.

| Build | Evaluation | Result | Conditions and limits |
|---|---|---|---|
| Q4_K_M | Korean Wikipedia perplexity | 5.79 | Lower is better; reference within this model |
| Q3_K_M | Korean Wikipedia perplexity | 6.06 | Same reported Korean comparison |
| Q2_K | Korean Wikipedia perplexity | 6.49 | Size/quality compromise |
| IQ1_M | Korean Wikipedia perplexity | 9.69 | Greater Korean degradation |

| Build | Evaluation | Result | Conditions and limits |
|---|---|---|---|
| Qwen3.6-35B-A3B reference | GPQA Diamond | 73.2% | 198 questions, greedy; a distinct reference model |
| POCKET-35B Q4_K_M | GPQA Diamond | 68.7% | 198 questions, greedy |
| POCKET-35B Q2_K | GPQA Diamond | 60.1% | 198 questions, greedy |
| POCKET-35B IQ1_M | HellaSwag | 61.0% | 400 questions |
| Bonsai-27B Q1_0 reference | HellaSwag | 60.0% | Same reported 400-question comparison; confidence intervals overlap |

The Qwen3.6 reference is not a same-checkpoint Darwin BF16 baseline. Do not interpret its difference as quantization-only loss. Small benchmark subsets and overlapping confidence intervals do not establish general quality equivalence.

<details>
<summary>Lineage and language-specific derivatives</summary>

POCKET is quantized from [Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus), VIDRAFT's multi-generation Darwin model. The original card describes a Qwen3.5-family sparse MoE architecture with 256 experts and top-8 routing; the distinct Qwen3.6 model above is an evaluation reference, not a replacement starting-checkpoint declaration.

| Component | Attribution |
|---|---|
| Starting checkpoint | Darwin-36B-Opus by VIDRAFT |
| GGUF quantization and runtime | Upstream llama.cpp; standard GGUF types |
| Full-model files in this repository | Q4_K_M, Q3_K_M, Q2_K and IQ1_M |
| Language-specific tuning and phone derivatives | Separate VIDRAFT KR/EN builds, with their own quality and compatibility trade-offs |

The original card reports stronger Korean degradation at extreme precision and different Korean/English phone trade-offs. Do not transfer a language-specific derivative's quality or memory results to the full-model files in this repository.

</details>

## Intended uses

- Local text assistance, experimentation and Korean-language workflows on a PC with suitable memory.
- Comparing quantization size, Korean perplexity and throughput on a clearly identified device.
- CPU-only workflows where local execution is preferred; no cloud service is required by the supplied command.

## Limitations

- The smallest file in this repository is about 8.2 GB, not a 5 GB phone build.
- IQ1_M trades quality for size; Q2_K or a larger build is preferable when quality matters.
- MacBook M3 Pro measurements are supplied above. iPhone and Strix Halo throughput remain unmeasured in the supplied evidence.
- File size is not a complete RAM budget; longer context increases memory requirements.
- Published speed and benchmark results are device-, build-, quant- and protocol-specific, not universal guarantees.

## License and attribution

This model card declares **Apache-2.0**, consistent with the current immediate starting checkpoint. Read the [Apache-2.0 license](https://www.apache.org/licenses/LICENSE-2.0) and the [starting checkpoint's license](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus/blob/main/LICENSE) before redistribution.

Starting checkpoint: [FINAL-Bench/Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus). POCKET quantization, language tuning and documentation: VIDRAFT / FINAL-Bench. Runtime tooling: the llama.cpp contributors. Keep required notices and attribution when redistributing.

## Support and updates

For device selection and installation help, visit the [VIDRAFT POCKET setup guide](https://vidraft.net/p/pocket.html?model=pocket_35b&utm_source=huggingface&utm_medium=referral&utm_campaign=pocket_2026q4&utm_content=pocket_35b_model_card_footer). Report reproducible issues in the [model discussions](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/discussions), including the filename, runtime version, device, memory and command; do not post private prompts or credentials.

A browser-accessible [POCKET-35B CPU demo](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) is a separate hosted service; its availability and latency are not local-device measurements. No local-inference telemetry or website gate is required to obtain these public files.

## POCKET family

| Model | Role | Start here |
|---|---|---|
| POCKET-35B-GGUF | Full 35B PC builds; Q2_K recommended first | [Model card](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) |
| POCKET-26B-GGUF | Korean-tuned Gemma4 GGUF; 11 GB and 17 GB options | [Model card](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) |
| POCKET-Darwin-180B-GGUF | Text-only RSI reasoning; four-shard download | [Model card](https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF) |

Explore the [POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6). For phone-oriented derivatives, see [KR GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF), [KR MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) and [EN GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF).

<details>
<summary>More POCKET projects and research</summary>

[Qwen3.8 Flash-Next GGUF](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) is a separate parent-model package, distinct from the POCKET-Darwin-180B RSI-R3 weights. Image projects are also separate: [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage), [POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) and [Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio).

Research collections: [Darwin](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193), [Aether](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a), [VKAE](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a).

</details>

## References

- [Source card and publisher measurements, revision 3870039](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/blob/3870039a87406fb90764b27e3984c8b1a36b269e/README.md).
- [Hugging Face CLI download documentation](https://huggingface.co/docs/huggingface_hub/guides/cli).
- [llama.cpp build and CPU-backend guidance](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#notes-about-gpu-accelerated-backends).
- [llama.cpp model loading](https://github.com/ggml-org/llama.cpp/blob/master/docs/models.md).
- [Hugging Face with Ollama](https://huggingface.co/docs/hub/ollama) and [LM Studio](https://huggingface.co/docs/hub/lmstudio).
- [VIDRAFT: on-device LLMs without a GPU](https://vidraft.net/insights/on-device-llm-without-gpu.html) and [model quantization](https://vidraft.net/insights/what-is-quantization-llm.html).

Documentation reviewed on 2026-11-04.