---
license: apache-2.0
language:
- en
base_model:
- hexgrad/Kokoro-82M
pipeline_tag: text-to-speech
tags:
- coreml
- kokoro
- apple-silicon
- ane
---

# Kokoro ANE v3 — hybrid ANE + GPU

An opt-in Kokoro Core ML pipeline (English in this directory; Japanese in `ja/`,
Mandarin in `zh/`) for **macOS 15 / iOS 18 or newer**.
It combines fixed quantized Albert models, a masked ANE decoder, deterministic
native Accelerate source/STFT, and Matt Mireles's optimized GPU generator adapted
to our parameters. It is a hybrid implementation; the generator is GPU-based.
Existing model paths in this repository remain available to older clients.

The base model and pretrained weights are **[hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)**, created by [hexgrad](https://huggingface.co/hexgrad). This variant converts and optimizes those weights for Core ML.

## FluidAudio SDK v0.17.6

Use `KokoroAneManager(version: .v3)` for English,
`KokoroAneManager(variant: .japanese, version: .v3)` for Japanese, or
`KokoroAneManager(variant: .mandarin, version: .v3)` for Mandarin.
All three accept native text and variant voices. Existing callers retain
`.legacy`. See [SDK usage, voices and platform limits](https://github.com/FluidInference/FluidAudio/blob/d7931323eebea4e39874b0d2e75d1b0e90e13084/Documentation/TTS/KokoroAne.md).
Requires macOS 15 / iOS 18; physical iOS inference remains unvalidated.

The SDK pins model release `6c0750dd02ef9d981fb82f147d784a3f171755ee` and base
revision `006395f65025af251858b1ab0a7178a6a1e73f9f`, adopting fp32
`KokoroProsody_v2` and COLA-corrected `KokoroTail_v2`. The bundled standalone
conversion runtime and its historical measurements below are separate from this
SDK integration. This documentation update does not change model bytes or pins.

### SDK compute routing and selected measurements

Hybrid is **not `.all` or ANE-only**. Albert, PostAlbert, Alignment, Prosody and
masked decoder request `.cpuAndNeuralEngine`; the fast generator requests
`.cpuAndGPU`; native source/STFT runs on CPU/Accelerate. Fallback Vocoder requests
CPU+ANE, Noise/Tail CPU+GPU. Core ML chooses actual placement within each policy.

| Policy | English | Japanese | Mandarin |
| --- | ---: | ---: | ---: |
| Current hybrid | **49.45 ms** | **35.85 ms** | **53.97 ms** |
| CPU+ANE throughout | 511.40 ms | 474.45 ms | 584.19 ms |
| `.all` throughout | SIGABRT | SIGABRT | SIGABRT |
| CPU+GPU throughout | SIGABRT | SIGABRT | SIGABRT |

Apple M5 Pro, 24 GiB, macOS 27.0 (26A428); October 7, 2026 (Toronto).
Median of five calls after two warmups on one selected demo input per language;
3.350 / 3.050 / 3.675 seconds of output. Timer covers prepared tokens/style to
complete PCM, excluding text processing, loading and WAV IO. Identical inputs
per policy, serial isolated processes, native DSP unchanged. These are local
exploratory checks, not corpus or cross-device results. Global overrides were
applied in a local harness; the public v3 API retains its fixed hybrid routing.
Both `.all` and CPU+GPU aborted during Prosody on the second warmup with Apple's
`GPURNNOps: JIT not supported`. CPU+ANE permits CPU fallback; it is not ANE-only.

Live-demo screenshots instead time raw text through complete audio and showed
57.8–79.8 ms in selected serial runs. Their 1.88× / 5.29× / 2.25× speedups versus
legacy are individual observations, not a benchmark average. Concurrent demo
mode also includes shared-resource contention.
See [inputs, ranges, protocol and raw records](https://github.com/FluidInference/FluidAudio/blob/d7931323eebea4e39874b0d2e75d1b0e90e13084/Documentation/TTS/Benchmarks.md#kokoro-ane-v3-selected-m5-pro-measurements).

Compute plans prefer ANE for static Albert/decoder and GPU for the generator,
but plan estimates and system-wide asitop power samples do not establish actual
per-process execution. See [ANE profiling and telemetry limits](https://github.com/FluidInference/FluidAudio/blob/d7931323eebea4e39874b0d2e75d1b0e90e13084/Documentation/ANE_Profiler.md#kokoro-ane-v3).

## Initial standalone Japanese and Mandarin extension

[Japanese](ja/README.md) and [Mandarin](zh/README.md) bundles include their voices,
vocabularies, conversion sources and real WAV samples. Use `--language ja` or
`--language zh` with the downloader and the commit containing those bundles.
They measured 24–28 ms on four short M5 Pro inputs versus 111–138 ms baseline.

The shared native STFT now uses Accelerate SGEMM to match reference convolution
summation. English model bytes are unchanged. With this runtime, English warm
medians are **22.36 / 32.72 ms**, versus **98.15 / 151.94 ms** baseline; isolated
native-source/generator relative RMSE is **0.07246 / 0.07080**, correlation
**0.99739 / 0.99750**. Its five real-input integration checks still pass.
The comparison and fidelity figures below describe the initial English release.

## Initial English latency evidence

| Warm synthesis | Published seven-stage baseline | This variant | Matt Mireles, M5 routing fallback |
|---|---:|---:|---:|
| Hello world, 1.625 s output | 96.54 ms | **22.05 ms** | 27.47 ms |
| Fox sentence, 3 s output | 151.19 ms | **32.99 ms** | 36.01 ms |

Apple M5 Pro, 24 GiB RAM, macOS 27.0; optimized Swift; three discarded warmups
and five measurements per input/model with rotating serial model order. Timer:
tokens/style through complete 24 kHz PCM, excluding G2P, model loading and file IO.
Matt's duration and pitch routing was changed to CPU+ANE because its default
CPU+GPU execution aborted on this host. These two-clip developer-machine results
are exploratory; they do not establish performance on other devices or lengths.

## Contents

- `fast/`: Albert at 32/64 tokens, fused source/generator/iSTFT, native DSP
  parameters and real validation fixtures.
- `decoder/`: masked decoder models at 120/200 acoustic frames (3/5 seconds).
- `.mlpackage` source models and `.mlmodelc` compiled models are both included.
  Compiled models were built on macOS 27.0; use source packages and compile on
  the deployment platform when needed.
- `runtime/`: standalone SwiftPM library, sanity checker and bounded benchmark.
- `conversion/`: the exporters and validation code used for this variant.
- `validation/`: raw timing/fidelity evidence and selected extra fixtures.
- `samples/hello.wav`, `samples/fox.wav`: actual output from the validated runtime.
- `manifest.json`: SHA-256/size manifest and pinned fallback dependency.

The downloader retrieves the seven original fallback models from this same
repository at revision `006395f65025af251858b1ab0a7178a6a1e73f9f`. Requests beyond
five seconds use that original vocoder instead of being truncated. Requests
above 64 tokens use the original dynamic Albert. The runtime accepts at most
2,000 predicted acoustic frames. Inputs are token IDs and a 256-value voice style;
text normalization, G2P and voice selection belong to the caller.

## Download and run

Install Python with `huggingface_hub` and Apple's Swift/Core ML tools. Pin the
40-character Hugging Face commit containing this release as `KOKORO_REVISION`.
After downloading this `ANE-v3` directory, run from it:

```sh
python tools/download-fast-pipeline.py \
  --revision "$KOKORO_REVISION" --output-dir assets
swift run --package-path runtime -c release kokoro-fast-check \
  assets/base assets/fast assets/decoder \
  assets/validation/kokoro-fast-extra-fixtures.json

# Two selected fixtures; 3 warmups + 5 recorded calls per fixture.
swift run --package-path runtime -c release kokoro-fast-benchmark \
  assets/base assets/fast assets/decoder assets/fast/fixtures.json > latency.json
```

Use `--format mlpackage` in the downloader to get source packages instead of
precompiled models. The Swift runtime compiles packages when no compiled model
is present. Numerical validation additionally requires coremltools 9.0, numpy
and soundfile:

```sh
python tools/validate-fast-pipeline.py --models-dir assets/fast
```

The checker emits the native spectra and waveform files consumed by that check.
Cache `KokoroFastPipeline(baseDirectory:fastDirectory:decoderDirectory:)` and call
`synthesize(inputIds:style:speed:)`. Its result contains 24 kHz PCM, stage timings
and the `usedFastVocoder` flag. Each instance serializes synthesis calls.

## Fidelity and scope

With reference decoder features held fixed, native source plus the adapted
GPU graph has waveform correlation 0.99789 / 0.99848 and relative RMSE
0.06489 / 0.05506 on the two clips. Full-chain correlation versus the published
baseline is 0.95688 / 0.92139; log1p-mel correlation is 0.99844 / 0.99872.
The isolated score is not a full-chain or perceptual equivalence claim. Full-chain
correlation versus the unquantized PyTorch reference is approximately 0.75.

Real-input checks cover both measured clips, “Good morning,” a slowed fox
sentence using the 200-frame decoder, and a six-second repeated sentence using
the original fallback. Invalid inputs are rejected. A pinned Whisper base.en
smoke check correctly transcribed six selected baseline/candidate clips without
a supplied transcript prompt; it is not a formal quality evaluation.

## Provenance and licenses

- Kokoro weights/voice: `hexgrad/Kokoro-82M`, revision
  `f3ff3571791e39611d31c381e3a41a3af07b4987` (Apache-2.0).
- Published quantized Albert: `FluidInference/kokoro-82m-coreml`, revision
  `006395f65025af251858b1ab0a7178a6a1e73f9f`.
- Adapted GPU graph: [mattmireles/kokoro-coreml](https://huggingface.co/mattmireles/kokoro-coreml/tree/9183566f294ca3f31a63fd2767cf5848bfebfd50),
  revision `9183566f294ca3f31a63fd2767cf5848bfebfd50` (Apache-2.0).
- Source inspiration: [Matt's repository](https://github.com/mattmireles/kokoro-coreml/tree/a3f1ff27b1d8683efa9976704b46cb1d96da1a4b).
- Original seven-stage conversion: laishere/kokoro-coreml (MIT).

The adapter checks 202 convolution/linear parameters, replaces 23 small
folded-bias differences and two inverse-STFT kernels with our export's values,
and preserves the optimized upstream graph. Per-parameter hashes and changed
constant names are recorded in `fast/generator-provenance.json`. This is an
explicit upstream graph adaptation. Retain `NOTICE`, `mattmireles-LICENSE` and
`conversion-LICENSE` when redistributing it.