HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs
HARP is a neural audio codec that partitions residual vector quantization (RVQ) across harmonically meaningful frequency bands, giving high-quality, variable-bitrate audio compression from a single model. A harmonic-aware partitioning distributes codebook capacity across perceptually meaningful bands, so one model serves multiple bitrates by decoding a growing number of codebook groups.
- 📄 Paper: HARP (Interspeech 2026, Oral)
- 💻 Code: https://github.com/QiaoyuYang/harp-codec
Model details
- Architecture: convolutional encoder/decoder with grouped RVQ; multi-period / multi-scale STFT discriminator (training only).
- Codebooks: 9 codebooks × 1024 entries (10 bits each), codebook dim 8.
- Band groups: 4 groups with a 3-2-2-2 codebook split, ordered by perceptual
importance:
Group Band Codebooks 0 Bass (~0–1 kHz) 3 1 Low-mid (~1–4 kHz) 2 2 High-mid (~4–10 kHz) 2 3 Treble (~10–22 kHz) 2
The bass group is always decoded; adding groups raises quality and bitrate.
Bitrate tiers
With 9 codebooks @ 1024 entries and a ~86 Hz frame rate:
| Groups | Codebooks | Approx. bitrate |
|---|---|---|
| 1 | 3 | ~2.6 kbps |
| 2 | 5 | ~4.3 kbps |
| 3 | 7 | ~6.0 kbps |
| 4 | 9 | ~7.7 kbps (full) |
Files
harp.ckpt— slim, inference-ready weights (~293 MB, weights only).
Usage
Set up the GitHub repo (uv sync),
download this checkpoint, and reconstruct audio:
uv pip install "huggingface_hub[cli]"
hf download KelvinYang/harp-codec harp.ckpt --local-dir checkpoints
python entry.py -i --input audio.wav --output recon.wav # full rate
python entry.py -i --input audio.wav --n-groups 2 --output out.wav # lower bitrate (1..4)
Each run reports SI-SDR, multi-scale mel loss, LSD, and SNR.
Citation
@inproceedings{harp2026,
title = {HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs},
author = {Yang, Qiaoyu and He, Lixing and Deng, Binyue and Zhao, Weifeng},
booktitle = {Interspeech},
year = {2026}
}
License
MIT