feat: add nemotron voice input

Co-authored-by: Codex <noreply@openai.com>

DESIGN.md

@@ -1,6 +1,6 @@
 # Build Small Hackathon Advisor — Design & Implementation Notes
 
-> A **small-model agent** (text-first; voice is a later bonus) that investigates what other people have already built
+> A **small-model agent** with text and voice input that investigates what other people have already built
 > for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon) and brainstorms an original new design
 > *with you*. Output = streaming text + live visuals (no TTS). All models small, open-weight, run locally.
 >
@@ -21,13 +21,13 @@ authoritative decision log; the rest of the doc is written to be consistent with
    archivist divines a fate-page; ink **bleeds and cites the real Spaces** you overlap (page 47, page 112…), or
    **blooms gold + sprouts a leaf** when it's unwritten. Engine unchanged underneath (crawl → whitespace/originality →
    score). The dry "advisor" stays under the hood. Full spec + de-risking grafts in §2.
-2. **Text-first.** Ship a fully demoable text-input (type / button) vertical slice first. Voice (push-to-talk → batch
-   ASR) is a later BONUS layer. Real-time streaming + in-browser turn detection are **deferred** (see §13 appendix).
+2. **Text-first with voice input.** The core workflow remains typed/editable text. Voice records or uploads a note,
+   transcribes it with batch ASR, and places the transcript in the same idea box. Real-time streaming + in-browser turn
+   detection are **deferred**.
 3. **Add a 🎯 Well-Tuned fine-tune** — a small LoRA (MiniCPM5 advisor persona / tool-calling), trained on Modal,
    published to the Hub → 6/6 badges → strong shot at 🎖️ Bonus Quest Champion ($2,000).
-4. **ASR = Nemotron primary (batch) + Parakeet fallback.** `nvidia/nemotron-speech-streaming-en-0.6b` used in BATCH
-   per-turn mode (`transcribe([wav])` — simple API). Day-1 spike confirms it builds + runs in one `@spaces.GPU` call;
-   if the ZeroGPU *environment* fights it, drop in `nvidia/parakeet-tdt-0.6b-v3` (native transformers, same batch API).
+4. **ASR = Nemotron batch.** `nvidia/nemotron-speech-streaming-en-0.6b` runs through NVIDIA NeMo in a ZeroGPU function.
+   Audio is normalized to mono WAV before calling `transcribe([wav])`.
 
 **Verified corrections:**
 - **Drop SGLang.** It needs a persistent GPU process → incompatible with ZeroGPU (same root cause as vLLM). Run
@@ -58,7 +58,7 @@ authoritative decision log; the rest of the doc is written to be consistent with
 - Trivial `@spaces.GPU` hello-cuda build GREEN on torch 2.8+, deps pinned, heavy deps added one at a time.
 - `gr.Server` minimal: static `index.html` + one same-origin `/api/agent-turn` NDJSON stream, plus the retained
   `@app.api()` generator for external clients, on the real ZeroGPU Space.
-- Nemotron `nemo_toolkit[asr]` install + one batch `transcribe()` inside `@spaces.GPU` (decision #4; else Parakeet).
+- Nemotron `nemo_toolkit[asr]` install + one batch `transcribe()` inside `@spaces.GPU` (decision #4).
 
 ---
 
@@ -110,7 +110,7 @@ UNWRITTEN." People compile draws into a "chapter" and dare friends to get a page
 
 **Defaults (revisit if time):** single-page artifact first (chapter compiler later); page-numbers visible, real titles
 on hover (keep the burn aimed at the idea, not a named builder); seal animation = safe typewriter + static-stamp floor
-first, bespoke ink-reveal last (graceful degradation); voice input stays text-first, a post-MVP flourish.
+first, bespoke ink-reveal last. Voice input is batch ASR that fills the same idea box before the user presses Ink.
 
 Input is **text-first**; the experience is fully delightful with typed input alone.
 
@@ -124,18 +124,16 @@ investigate → ideate → score loop — the experience collapses without the m
 
 | Role | Model | Params | Runtime | License | Prize hook |
 |---|---|---|---|---|---|
-| STT (batch, voice-later) | **`nvidia/nemotron-speech-streaming-en-0.6b`** (used in batch) | 0.6B | NeMo, GPU+CUDA | NVIDIA Open Model (commercial OK) | 🟩 NVIDIA Nemotron Quest |
-| ↳ fallback | `nvidia/parakeet-tdt-0.6b-v3` | 0.6B | **transformers** (no NeMo) | CC-BY-4.0 | 🟩 (Quest brand — verify, §5.1) |
+| STT (batch voice input) | **`nvidia/nemotron-speech-streaming-en-0.6b`** | 0.6B | NeMo, GPU+CUDA | NVIDIA Open Model (commercial OK) | 🟩 NVIDIA Nemotron Quest |
 | LLM brain | **`openbmb/MiniCPM5-1B`** ("OpenCPM5") | 1.08B | **transformers** (self-parse XML) / llama.cpp | **Apache-2.0** | 🏮 OpenBMB |
-| Turn detection (voice-later) | **`pipecat-ai/smart-turn-v3`** | ~8M | ONNX Runtime (browser) | BSD-2 | (natural voice UX) |
 | Embedder | **`ggml-org/embeddinggemma-300M-qat-q4_0-GGUF`** | ~300M | llama.cpp / llama-cpp-python | Gemma | 🔌 Off the Grid · 🦙 Llama Champion · 🟢 Modal |
 | Fine-tune | LoRA on MiniCPM5 → published to Hub | — | PEFT / HF Jobs | — | 🎯 Well-Tuned |
 
-**Total ≈ 1.9B params → ≤4B → 🐜 Tiny Titan eligible.** All open-weight, all runnable locally → 🔌 Off the Grid.
+**Total ≈ 1.98B params → ≤4B → 🐜 Tiny Titan eligible.** All open-weight, all runnable locally → 🔌 Off the Grid.
 
 > Naming: "OpenCPM5 1B" = `openbmb/MiniCPM5-1B` (MiniCPM 5.0, ~May 2026). "EmbeddingGemma 270M" =
 > `google/embeddinggemma-300m` (308M total; 270M = non-embedding transformer params). **SGLang dropped** (ZeroGPU
-> incompatible). STT used in **batch per-turn** mode, not streaming.
+> incompatible). STT is used in **batch voice-note** mode, not a persistent stream.
 
 ---
 
@@ -148,21 +146,21 @@ With **text-first + batch ASR**, the old "streaming ASR vs ZeroGPU" Config A/B t
 - **Text-first runtime loop:** user types → custom `/api/agent-turn` NDJSON endpoint → one `@spaces.GPU` call runs
   MiniCPM5 (tool loop, in `transformers`) → streamed text tokens + live visual updates. The `@app.api()` endpoint
   remains as the Gradio-client contract for external checks.
-- **Voice (later bonus):** push-to-talk records an utterance → POST blob → the same `@spaces.GPU` call also runs
-  Nemotron/Parakeet ASR (batch) before the brain. No persistent stream, no WebRTC, **no TURN server**.
+- **Voice input:** push-to-talk records an utterance or uploads a voice note → `/api/transcribe` normalizes audio with
+  ffmpeg → one `@spaces.GPU` call runs Nemotron ASR through NeMo → transcript fills the idea box. No persistent stream,
+  no WebRTC, **no TURN server**.
 - **Modal (build-time only):** crawl the org + build the llama.cpp EmbeddingGemma vector index offline; the Space ships
   with checked-in project vectors. Runtime never calls Modal → 🔌 Off the Grid holds (see §10).
 
 > Off the Grid = no proprietary cloud inference APIs. Open weights on an HF GPU Space / local box / Modal all qualify.
 
-**Deferred (voice-later appendix, §13):** real-time streaming ASR (`conformer_stream_step`), in-browser Smart Turn +
-Silero VAD turn detection, FastRTC. Documented but not on the text-first critical path.
+**Deferred:** real-time streaming ASR and turn detection are not part of the shipped app.
 
 ---
 
 ## 5. Per-model implementation notes
 
-### 5.1 ASR — `nvidia/nemotron-speech-streaming-en-0.6b` (batch) · fallback `parakeet-tdt-0.6b-v3`
+### 5.1 ASR — `nvidia/nemotron-speech-streaming-en-0.6b` (batch)
 
 - **Primary, batch usage (simple):**
   ```python
@@ -170,20 +168,10 @@ Silero VAD turn detection, FastRTC. Documented but not on the text-first critica
   asr = nemo_asr.models.ASRModel.from_pretrained("nvidia/nemotron-speech-streaming-en-0.6b")
   text = asr.transcribe(["utterance.wav"])      # 16 kHz mono WAV in; punctuated EN text out
   ```
-  Install (heavy; CUDA): `apt-get install -y libsndfile1 ffmpeg` + `pip install Cython packaging` +
-  `pip install "git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]"` (NeMo ≥ 25.11).
-  **Day-1 spike** must confirm this installs + runs in one `@spaces.GPU` call (load with `map_location="cpu"` at module
-  level, `.to("cuda")` inside the decorator; watch NeMo CUDA-at-import + the RNNT CUDA-graphs path).
-- **Fallback (if the ZeroGPU env fights NeMo):** `nvidia/parakeet-tdt-0.6b-v3` has **native transformers** support —
-  ```python
-  from transformers import pipeline
-  asr = pipeline("automatic-speech-recognition", model="nvidia/parakeet-tdt-0.6b-v3")
-  text = asr("utterance.wav")
-  ```
-  Same 0.6B, CC-BY-4.0, 25 languages, one clean `@spaces.GPU` batch call, no NeMo/git build.
-- **Quest caveat:** the award says "standout **Nemotron** builds." Parakeet is the encoder *inside* the Nemotron-Speech
-  family but is branded "Parakeet" → if we ship Parakeet, **confirm Quest eligibility with organizers**, or keep
-  Nemotron documented as the Quest narrative. Hosted NVIDIA NIM API would break Off the Grid — never in the badge demo.
+  Runtime install: `packages.txt` provides `ffmpeg` and `libsndfile1`; `requirements.txt` pins
+  `nemo_toolkit[asr]==2.7.3` plus Cython and packaging. The app records or uploads audio, normalizes it to mono
+  16 kHz WAV, runs NeMo in a ZeroGPU function, then returns the transcript to the idea box. Hosted NVIDIA NIM API would
+  break Off the Grid, so it is not used.
 
 ### 5.2 MiniCPM5-1B brain — `openbmb/MiniCPM5-1B` (transformers, self-parsed XML)
 
@@ -203,24 +191,6 @@ Silero VAD turn detection, FastRTC. Documented but not on the text-first critica
   tool-calling is a pending PR — verify before relying on it for the badge runtime.)
 - **1B discipline:** small tool schemas, few params each, clear descriptions, low temp, single-hop tool calls.
 
-### 5.3 Smart Turn v3 — `pipecat-ai/smart-turn-v3` (voice-later)
-
-- Whisper-tiny encoder + linear head, ~8M params, ONNX (`smart-turn-v3.1.onnx`), 8 MB int8 / 32 MB fp32, BSD-2.
-- Input: 16 kHz mono PCM float32, ≤8 s (front-padded). Output: sigmoid prob; **>0.5 = "user finished."** ~12 ms CPU.
-  ```python
-  import numpy as np, onnxruntime as ort
-  from transformers import WhisperFeatureExtractor
-  fe = WhisperFeatureExtractor(chunk_length=8); sess = ort.InferenceSession("smart-turn-v3.1.onnx")
-  def turn_complete(audio):
-      x = fe(audio, sampling_rate=16000, return_tensors="np", padding="max_length",
-             max_length=8*16000, truncation=True, do_normalize=True).input_features.astype("float32")
-      return sess.run(None, {"input_features": x})[0][0].item() > 0.5
-  ```
-- **Browser path (voice-later):** runs via ONNX Runtime Web / Transformers.js. The `.onnx` needs Whisper **log-mel
-  input_features** (128 mel bins, 8 s, NOT whisper-tiny's 80-mel) — no upstream JS example exists, so a small POC is
-  required before relying on it; fallbacks: port pipecat's numpy-only mel extractor to JS, or do feature-extraction +
-  onnx **server-side** per posted blob. Pair with `@ricky0123/vad-web` (Silero) for the speech start/stop gate.
-
 ### 5.4 EmbeddingGemma GGUF — `ggml-org/embeddinggemma-300M-qat-q4_0-GGUF`
 
 - Active retrieval model: `embeddinggemma-300M-qat-Q4_0.gguf`, 768-dimensional normalized embeddings.
@@ -239,8 +209,7 @@ llama.cpp on Modal, and runtime query embeddings use the same llama.cpp path.
 |---|---|---|---|
 | `openbmb/MiniCPM5-1B` | ✅ planned only | llama.cpp / Ollama | Not used for deployed tool-calling; Transformers + LoRA is the deployed brain. |
 | `ggml-org/embeddinggemma-300M-qat-q4_0-GGUF` | ✅ active | llama.cpp / llama-cpp-python | Builds project vectors on Modal and embeds runtime queries in the Space. |
-| ASR (Nemotron / Parakeet) | ❌ | NeMo / transformers | FastConformer-RNNT |
-| `pipecat-ai/smart-turn-v3` | ❌ | ONNX Runtime | Whisper encoder + classifier head |
+| ASR (Nemotron) | ❌ | NeMo | FastConformer-RNNT |
 
 If retrieval quality becomes the bottleneck, compare Q4_0 against Q8_0, but do not keep two runtime retrieval paths.
 
@@ -393,12 +362,12 @@ open grimoire as the hero component.
 | Target | How it's earned |
 |---|---|
 | 🍄 Thousand Token Wood | **The Unwritten Almanac** (§2) — the bleed-citation wow IS the engine rendered; AI load-bearing; original |
-| 🐜 Tiny Titan (special, $1.5k) | total ~1.9B, every model ≤4B; largest single = MiniCPM5 1.08B |
+| 🐜 Tiny Titan (special, $1.5k) | total ~1.98B, every model ≤4B; largest single = MiniCPM5 1.08B |
 | 🔌 Off the Grid (badge) | all open weights run locally; offline index; no cloud inference at runtime |
 | 🎯 Well-Tuned (badge) | published LoRA fine-tune of MiniCPM5 on the Hub (§10) → **6/6 badges** |
 | 🎨 Off-Brand (badge + $1.5k) | `gr.Server` custom UI is the agent's output surface |
 | 🏮 OpenBMB ($10k) | brain = MiniCPM5-1B ("OpenBMB pick") |
-| 🟩 NVIDIA Quest (2× RTX 5080) | ASR = Nemotron (verify if Parakeet qualifies, §5.1) |
+| 🟩 NVIDIA Quest (2× RTX 5080) | ASR = Nemotron (§5.1) |
 | 🦙 Llama Champion (badge) | EmbeddingGemma GGUF retrieval index and runtime query embeddings run through llama.cpp (§5.5) |
 | 📡 Sharing is Caring (badge) | publish the agent's tool-call trace to the Hub |
 | 📓 Field Notes (badge) | this DESIGN.md → a build blog post |
@@ -417,22 +386,20 @@ lever is §11 custom-UI polish.
 
 ## 13. Risks / open items
 
-1. **Day-1 spikes are go/no-go** (§1): ZeroGPU hello-cuda build; gr.Server same-origin NDJSON browser streaming;
-   Nemotron batch in `@spaces.GPU` (else Parakeet). Do these before feature work.
+1. **Deployment smoke tests are mandatory:** ZeroGPU Space build, same-origin NDJSON browser streaming, and Nemotron
+   batch ASR in `@spaces.GPU` must be verified after every runtime dependency change.
 2. **EmbeddingGemma is gated** — accept Gemma terms + `HF_TOKEN` before any crawl/build.
 3. **MiniCPM5 tool-call reliability at 1B** — covered by the degradation ladder (§8); validate name+args in code.
-4. **NVIDIA Quest brand** — Parakeet is not "Nemotron"-branded; confirm eligibility with organizers or keep Nemotron
-   primary (§5.1).
-5. **Concept skin** — **chosen: The Unwritten Almanac** (§2). Make-or-break is the bleed/bloom hero animation; build the
+4. **Concept skin** — **chosen: The Unwritten Almanac** (§2). Make-or-break is the bleed/bloom hero animation; build the
    safe typewriter + static-stamp floor first (graceful degradation), upgrade ink last. Watch the thin-org echo
    threshold + the dry-but-benevolent tone (real cited Spaces, never punch at a named builder).
-6. **Param-budget claim** — document the 1.9B total in the README/Space card for Tiny Titan judging.
+5. **Param-budget claim** — document the 1.98B total in the README/Space card for Tiny Titan judging.
 
 ---
 
 ## 14. Build order
 
-**Text-first vertical slice first; voice as a bonus layer.** Always keep a demoable artifact.
+**Text-first vertical slice first; voice input is now part of the app.** Always keep a demoable artifact.
 
 0. **Day-1 spikes** (§1) — get the three go/no-go builds green.
 1. **`crawler.py` + Modal index** — crawl the org, embed with EmbeddingGemma, build the local index. *You immediately
@@ -442,27 +409,24 @@ lever is §11 custom-UI polish.
 4. **`app.py`** — `gr.Server` custom frontend (idea board, project/whitespace wall, streaming text), called via
    first-party `/api/...` endpoints; concept skin applied.
 5. **Well-Tuned LoRA** — small fine-tune on Modal → publish to Hub (→ 6/6 badges).
-6. **Polish + submission** — demo video + social post (Best Demo / Community Choice), publish agent trace (📡),
+6. **Voice input** — push-to-talk record and voice-note upload through Nemotron batch ASR in `/api/transcribe`.
+7. **Polish + submission** — demo video + social post (Best Demo / Community Choice), publish agent trace (📡),
    write up Field Notes (📓).
 
-**Voice bonus (only if time):** push-to-talk record → batch ASR (Nemotron/Parakeet) in the existing `@spaces.GPU` call.
-**Deferred:** real-time streaming ASR, in-browser Smart Turn + Silero VAD, FastRTC + TURN. (Original streaming design is
-preserved in git history if revisited.)
+**Deferred:** real-time streaming ASR and turn detection. The shipped path stays batch audio → transcript → editable idea.
 
 ---
 
 ## 15. Sources
 
 **Models:** [nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b) ·
-[parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) ·
-[parakeet transformers doc](https://huggingface.co/docs/transformers/en/model_doc/parakeet) ·
 [MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B) · [MiniCPM5-1B-GGUF](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) ·
-[smart-turn-v3](https://huggingface.co/pipecat-ai/smart-turn-v3) · [embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m)
+[embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m)
 
 **Platforms:** [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) ·
 [Introducing gradio.Server](https://huggingface.co/blog/introducing-gradio-server) · [Gradio Server Mode guide](https://www.gradio.app/guides/server-mode) ·
 [Modal GPU](https://modal.com/docs/guide/gpu) · [Modal model weights](https://modal.com/docs/guide/model-weights) · [Modal pricing](https://modal.com/pricing) ·
 [Build Small Hackathon](https://huggingface.co/build-small-hackathon)
 
-*Verify-before-ship: Nemotron-in-ZeroGPU (Day-1 spike); MiniCPM5 license on the live card; NVIDIA Quest eligibility for
-Parakeet; Smart Turn in-browser feature extraction (voice-later); llama.cpp MiniCPM5 tool-calling (pending PR).*
+*Verify-before-ship: Nemotron-in-ZeroGPU after dependency changes; MiniCPM5 license on the live card; llama.cpp MiniCPM5
+tool-calling remains planned only and is not used by the deployed brain.*

README.md

@@ -29,6 +29,7 @@ The current milestone is a deployed ZeroGPU + MiniCPM5 LoRA advisor:
 
 - Local snapshot of public `build-small-hackathon` Spaces.
 - Modal-built EmbeddingGemma GGUF retrieval index, with runtime query embeddings computed through llama.cpp.
+- Nemotron Speech Streaming voice input through NVIDIA NeMo ASR on ZeroGPU.
 - Jargon correction for hackathon/model terms.
 - MiniCPM5 tool-call planning with a published PEFT LoRA adapter, plus deterministic local rules for tests and CPU-only
   development.
@@ -88,6 +89,13 @@ Users can also click any Idea Board page to make it current before pressing `Pla
 If the board is empty, `Plan` and `Compare` do not create placeholder pages; they prompt the user to write an idea or
 press `Gap` first.
 
+## Voice Input
+
+The `Speak` and `Voice note` controls send audio to `/api/transcribe`. The backend normalizes the uploaded audio with
+ffmpeg, then transcribes it with `nvidia/nemotron-speech-streaming-en-0.6b` through NVIDIA NeMo inside the same ZeroGPU
+runtime used by the advisor. The transcript is placed back in the idea box so the user can edit it before pressing
+`Ink`.
+
 ## Gap Exploration
 
 The `Gap` command walks through unused whitespace candidates instead of repeating the same first suggestion. Each chosen
@@ -180,12 +188,13 @@ ADVISOR_ADAPTER_ID=build-small-hackathon/hackathon-advisor-minicpm5-lora
 ADVISOR_ADAPTER_REVISION=25de69bcde397e1bcdd852923b56a42f10222650
 ADVISOR_EMBEDDING_MODEL_REPO=ggml-org/embeddinggemma-300M-qat-q4_0-GGUF
 ADVISOR_EMBEDDING_MODEL_FILE=embeddinggemma-300M-qat-Q4_0.gguf
+ADVISOR_ASR_MODEL_ID=nvidia/nemotron-speech-streaming-en-0.6b
 ```
 
 `agent_turn` wraps the engine call with `spaces.GPU` when `ADVISOR_ZERO_GPU=1`, so model loading and generation run on
 the ZeroGPU allocation. The retrieval query embedder downloads the GGUF model through `huggingface_hub` unless
-`ADVISOR_EMBEDDING_MODEL_PATH` points to a local file. Local tests and CPU-only development still default to
-`ADVISOR_MODEL_BACKEND=rules`.
+`ADVISOR_EMBEDDING_MODEL_PATH` points to a local file. `/api/transcribe` uses the same ZeroGPU wrapper for Nemotron ASR.
+Local tests and CPU-only development still default to `ADVISOR_MODEL_BACKEND=rules`.
 
 ## Test
 

app.py

@@ -3,14 +3,16 @@ from __future__ import annotations
 import json
 import os
 from pathlib import Path
+import tempfile
 from typing import Any, Iterator
 
-from fastapi import Body
+from fastapi import Body, File, HTTPException, UploadFile
 from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response, StreamingResponse
 from gradio import Server
 
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.artifact_bundle import BUNDLE_FILENAME, build_demo_bundle_zip
+from hackathon_advisor.asr_runtime import create_asr_transcriber
 from hackathon_advisor.chapter import build_chapter_markdown
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.demo_rehearsal import build_demo_rehearsal
@@ -31,9 +33,11 @@ STATIC_DIR = ROOT / "static"
 DATA_PATH = ROOT / "data" / "projects.json"
 INDEX_PATH = ROOT / "data" / "project_index.json"
 PROFILE_FIELDS = ["skills", "time", "preferences", "constraints"]
+MAX_AUDIO_UPLOAD_BYTES = 25 * 1024 * 1024
 
 index = ProjectIndex.from_files(DATA_PATH, INDEX_PATH)
 engine = AdvisorEngine(index)
+voice_transcriber = create_asr_transcriber()
 app = Server()
 
 
@@ -46,6 +50,11 @@ def _engine_turn(message: str, session: dict[str, Any]):
     return engine.turn(message, session)
 
 
+@gpu_task
+def _transcribe_voice(audio_path: str) -> dict[str, Any]:
+    return voice_transcriber.transcribe(Path(audio_path)).to_dict()
+
+
 def _session_from_json(session_json: str = "{}") -> dict[str, Any]:
     try:
         session = json.loads(session_json or "{}")
@@ -107,6 +116,7 @@ def health() -> dict:
         "ok": True,
         "projects": len(index.projects),
         "runtime": engine.runtime_status(),
+        "voice": voice_transcriber.status().to_dict(),
         **trace_metadata(index),
     }
 
@@ -117,6 +127,7 @@ def bootstrap() -> dict:
     return {
         "project_count": len(index.projects),
         "runtime": runtime_status,
+        "voice": voice_transcriber.status().to_dict(),
         **trace_metadata(index),
         "top_projects": [project.to_public_dict() for project in index.top_projects(limit=8)],
         "whitespace": [item.to_dict() for item in index.find_whitespace(limit=5)],
@@ -134,7 +145,7 @@ def runtime() -> dict:
 
 @app.get("/api/prize-ledger")
 def prize_ledger_endpoint() -> dict:
-    return prize_ledger(engine.runtime_status(), trace_metadata(index))
+    return prize_ledger(engine.runtime_status(), trace_metadata(index), voice_transcriber.status().to_dict())
 
 
 @app.get("/api/tool-contracts")
@@ -153,7 +164,7 @@ def demo_session() -> dict:
 @app.get("/api/demo-bundle.zip")
 def demo_bundle() -> Response:
     runtime_status = engine.runtime_status()
-    ledger = prize_ledger(runtime_status, trace_metadata(index))
+    ledger = prize_ledger(runtime_status, trace_metadata(index), voice_transcriber.status().to_dict())
     metadata = {
         **trace_metadata(index),
         "project_count": len(index.projects),
@@ -190,6 +201,34 @@ def agent_turn_stream(payload: dict[str, Any] | None = Body(default=None)) -> St
     return StreamingResponse(stream(), media_type="application/x-ndjson")
 
 
+@app.post("/api/transcribe")
+async def transcribe_audio(audio: UploadFile = File(...)) -> dict[str, Any]:
+    content_type = str(audio.content_type or "")
+    if content_type and not content_type.startswith("audio/"):
+        raise HTTPException(status_code=415, detail="Voice input must be an audio file.")
+    with tempfile.TemporaryDirectory(prefix="advisor-upload-") as directory:
+        filename = Path(str(audio.filename or "voice-note")).name
+        suffix = Path(filename).suffix or ".audio"
+        source = Path(directory) / f"voice{suffix}"
+        await _save_audio_upload(audio, source)
+        return _transcribe_voice(str(source))
+
+
+async def _save_audio_upload(upload: UploadFile, target: Path) -> None:
+    total = 0
+    with target.open("wb") as handle:
+        while True:
+            chunk = await upload.read(1024 * 1024)
+            if not chunk:
+                break
+            total += len(chunk)
+            if total > MAX_AUDIO_UPLOAD_BYTES:
+                raise HTTPException(status_code=413, detail="Voice note is too large.")
+            handle.write(chunk)
+    if total == 0:
+        raise HTTPException(status_code=400, detail="Voice note is empty.")
+
+
 @app.post("/api/field-notes")
 def field_notes_api(payload: dict[str, Any] | None = Body(default=None)) -> Response:
     session = _session_from_payload(payload)
@@ -219,7 +258,7 @@ def chapter_api(payload: dict[str, Any] | None = Body(default=None)) -> Response
 @app.get("/api/lora-training-kit.zip")
 def lora_training_kit() -> Response:
     runtime_status = engine.runtime_status()
-    ledger = prize_ledger(runtime_status, trace_metadata(index))
+    ledger = prize_ledger(runtime_status, trace_metadata(index), voice_transcriber.status().to_dict())
     metadata = {
         **trace_metadata(index),
         "project_count": len(index.projects),
@@ -291,7 +330,7 @@ def submission_packet_artifact(session_json: str = "{}") -> str:
             **trace_metadata(index),
             "project_count": len(index.projects),
         },
-        prize_ledger(runtime_status, trace_metadata(index)),
+        prize_ledger(runtime_status, trace_metadata(index), voice_transcriber.status().to_dict()),
     )
 
 

hackathon_advisor/asr_runtime.py

@@ -0,0 +1,159 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import tempfile
+from typing import Any
+
+
+DEFAULT_ASR_MODEL_ID = "nvidia/nemotron-speech-streaming-en-0.6b"
+DEFAULT_ASR_BACKEND = "nemo-asr"
+DEFAULT_ASR_SAMPLE_RATE = 16_000
+
+
+@dataclass(frozen=True)
+class AsrTranscript:
+    transcript: str
+    model_id: str
+    backend: str
+    sample_rate: int
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "transcript": self.transcript,
+            "model_id": self.model_id,
+            "backend": self.backend,
+            "sample_rate": self.sample_rate,
+        }
+
+
+@dataclass(frozen=True)
+class AsrStatus:
+    backend: str
+    model_id: str
+    loaded: bool
+    sample_rate: int
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "backend": self.backend,
+            "model_id": self.model_id,
+            "loaded": self.loaded,
+            "sample_rate": self.sample_rate,
+        }
+
+
+class NemotronAsrTranscriber:
+    backend = DEFAULT_ASR_BACKEND
+
+    def __init__(
+        self,
+        model_id: str = DEFAULT_ASR_MODEL_ID,
+        sample_rate: int = DEFAULT_ASR_SAMPLE_RATE,
+    ) -> None:
+        self.model_id = model_id.strip() or DEFAULT_ASR_MODEL_ID
+        self.sample_rate = sample_rate
+        self._model = None
+
+    def status(self) -> AsrStatus:
+        return AsrStatus(
+            backend=self.backend,
+            model_id=self.model_id,
+            loaded=self._model is not None,
+            sample_rate=self.sample_rate,
+        )
+
+    def transcribe(self, audio_path: Path) -> AsrTranscript:
+        source = Path(audio_path)
+        if not source.is_file():
+            raise RuntimeError("Voice note was not saved before transcription.")
+        self._ensure_loaded()
+        with tempfile.TemporaryDirectory(prefix="advisor-asr-") as directory:
+            wav_path = Path(directory) / "voice.wav"
+            normalize_audio_for_asr(source, wav_path, self.sample_rate)
+            outputs = self._model.transcribe([str(wav_path)], batch_size=1)
+        transcript = extract_transcript(outputs).strip()
+        if not transcript:
+            raise RuntimeError("Nemotron ASR returned an empty transcript.")
+        return AsrTranscript(
+            transcript=transcript,
+            model_id=self.model_id,
+            backend=self.backend,
+            sample_rate=self.sample_rate,
+        )
+
+    def _ensure_loaded(self) -> None:
+        if self._model is not None:
+            return
+        try:
+            import torch
+            import nemo.collections.asr as nemo_asr
+        except ImportError as error:
+            raise RuntimeError(
+                "Nemotron voice input requires NVIDIA NeMo ASR. Install `nemo_toolkit[asr]` "
+                "before enabling voice transcription."
+            ) from error
+        model = nemo_asr.models.ASRModel.from_pretrained(model_name=self.model_id)
+        device = os.environ.get("ADVISOR_ASR_DEVICE", "").strip()
+        if not device:
+            device = "cuda" if torch.cuda.is_available() else "cpu"
+        model.to(device)
+        model.eval()
+        self._model = model
+
+
+def create_asr_transcriber() -> NemotronAsrTranscriber:
+    sample_rate = int(os.environ.get("ADVISOR_ASR_SAMPLE_RATE", str(DEFAULT_ASR_SAMPLE_RATE)))
+    if sample_rate <= 0:
+        raise RuntimeError("ADVISOR_ASR_SAMPLE_RATE must be a positive integer.")
+    return NemotronAsrTranscriber(
+        model_id=os.environ.get("ADVISOR_ASR_MODEL_ID", DEFAULT_ASR_MODEL_ID),
+        sample_rate=sample_rate,
+    )
+
+
+def normalize_audio_for_asr(source: Path, target: Path, sample_rate: int = DEFAULT_ASR_SAMPLE_RATE) -> None:
+    ffmpeg = shutil.which("ffmpeg")
+    if not ffmpeg:
+        raise RuntimeError("Voice transcription requires ffmpeg to normalize audio.")
+    command = [
+        ffmpeg,
+        "-hide_banner",
+        "-loglevel",
+        "error",
+        "-y",
+        "-i",
+        str(source),
+        "-ac",
+        "1",
+        "-ar",
+        str(sample_rate),
+        "-sample_fmt",
+        "s16",
+        str(target),
+    ]
+    completed = subprocess.run(command, check=False, capture_output=True, text=True)
+    if completed.returncode != 0:
+        message = completed.stderr.strip() or "ffmpeg could not read this audio file."
+        raise RuntimeError(message)
+
+
+def extract_transcript(outputs: Any) -> str:
+    if isinstance(outputs, str):
+        return outputs
+    if isinstance(outputs, dict):
+        return str(outputs.get("text") or outputs.get("transcript") or "")
+    if isinstance(outputs, (list, tuple)):
+        if not outputs:
+            return ""
+        return extract_transcript(outputs[0])
+    text = getattr(outputs, "text", None)
+    if text is not None:
+        return str(text)
+    transcript = getattr(outputs, "transcript", None)
+    if transcript is not None:
+        return str(transcript)
+    return str(outputs)

hackathon_advisor/prize_ledger.py

@@ -20,11 +20,11 @@ MODEL_STACK = [
         "runtime": "Modal-built llama.cpp GGUF index + runtime llama.cpp query embeddings",
     },
     {
-        "role": "Voice bonus",
+        "role": "Voice input",
         "model": "nvidia/nemotron-speech-streaming-en-0.6b",
         "params_b": 0.60,
-        "status": "deferred bonus",
-        "runtime": "batch ASR",
+        "status": "deployed",
+        "runtime": "ZeroGPU + NVIDIA NeMo ASR",
     },
 ]
 
@@ -87,12 +87,17 @@ TRAINING_ARTIFACTS = [
 ]
 
 
-def prize_ledger(runtime: dict[str, Any], index_metadata: dict[str, Any] | None = None) -> dict[str, Any]:
+def prize_ledger(
+    runtime: dict[str, Any],
+    index_metadata: dict[str, Any] | None = None,
+    voice_metadata: dict[str, Any] | None = None,
+) -> dict[str, Any]:
     total_params = round(sum(float(item["params_b"]) for item in MODEL_STACK), 2)
     largest = max(MODEL_STACK, key=lambda item: float(item["params_b"]))
     return {
         "runtime": runtime,
         "retrieval_index": index_metadata or {},
+        "voice": voice_metadata or {},
         "model_stack": MODEL_STACK,
         "total_params_b": total_params,
         "largest_model": {

packages.txt

@@ -0,0 +1,2 @@
+ffmpeg
+libsndfile1

pyproject.toml

@@ -8,9 +8,12 @@ license = { text = "MIT" }
 authors = [{ name = "Jacob LinCool" }]
 dependencies = [
   "accelerate>=1.0,<2",
+  "Cython>=3,<4",
   "gradio>=6.16.0,<7",
   "huggingface-hub>=0.36,<1",
   "llama-cpp-python>=0.3.26,<1",
+  "nemo_toolkit[asr]==2.7.3",
+  "packaging>=24,<26",
   "peft>=0.13,<1",
   "pillow>=10,<13",
   "spaces>=0.50,<1",
@@ -24,7 +27,10 @@ dev = [
 ]
 model = [
   "accelerate>=1.0,<2",
+  "Cython>=3,<4",
   "llama-cpp-python>=0.3.26,<1",
+  "nemo_toolkit[asr]==2.7.3",
+  "packaging>=24,<26",
   "peft>=0.13,<1",
   "spaces>=0.50,<1",
   "torch>=2.8,<3",

requirements.txt

@@ -1,9 +1,12 @@
 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
 
 accelerate>=1.0,<2
+Cython>=3,<4
 gradio>=6.16.0,<7
 huggingface-hub>=0.36,<1
 llama-cpp-python==0.3.26
+nemo_toolkit[asr]==2.7.3
+packaging>=24,<26
 peft>=0.13,<1
 pillow>=10,<13
 spaces>=0.50,<1

static/app.js

@@ -26,6 +26,9 @@ const exportButton = document.querySelector("#export-artifact");
 const exportNotesButton = document.querySelector("#export-notes");
 const exportChapterButton = document.querySelector("#export-chapter");
 const resetButton = document.querySelector("#reset-session");
+const recordVoiceButton = document.querySelector("#record-voice");
+const uploadVoiceButton = document.querySelector("#upload-voice");
+const voiceFileInput = document.querySelector("#voice-file");
 
 const SESSION_STORAGE_KEY = "hackathon-advisor-session-v2";
 const FIELD_NOTES_FILENAME = "hackathon-advisor-field-notes.md";
@@ -43,6 +46,10 @@ let sawTurnToken = false;
 let bootstrapData = null;
 let sessionRevision = 0;
 let sessionControlsLocked = false;
+let voiceBusy = false;
+let voiceRecorder = null;
+let voiceStream = null;
+let voiceChunks = [];
 
 bootstrap().catch(handleBootstrapError);
 
@@ -87,6 +94,22 @@ resetButton.addEventListener("click", () => {
   resetSession();
 });
 
+recordVoiceButton.addEventListener("click", async () => {
+  await toggleVoiceRecording();
+});
+
+uploadVoiceButton.addEventListener("click", () => {
+  if (uploadVoiceButton.disabled || voiceBusy || sessionControlsLocked) return;
+  voiceFileInput.click();
+});
+
+voiceFileInput.addEventListener("change", async () => {
+  const file = voiceFileInput.files?.[0] || null;
+  voiceFileInput.value = "";
+  if (!file) return;
+  await transcribeVoiceBlob(file, file.name || "voice-note.audio");
+});
+
 goalsEl.addEventListener("change", (event) => {
   const target = event.target;
   if (!(target instanceof HTMLInputElement) || !target.dataset.goal) return;
@@ -210,6 +233,121 @@ async function runCommand(command) {
   await runTurn(command);
 }
 
+async function toggleVoiceRecording() {
+  if (voiceRecorder?.state === "recording") {
+    voiceRecorder.stop();
+    return;
+  }
+  await startVoiceRecording();
+}
+
+async function startVoiceRecording() {
+  if (sessionControlsLocked || voiceBusy) return;
+  if (!navigator.mediaDevices?.getUserMedia || !window.MediaRecorder) {
+    setSessionStatus("Voice recording is not available in this browser. Upload a voice note instead.");
+    return;
+  }
+  try {
+    voiceStream = await navigator.mediaDevices.getUserMedia({ audio: true });
+    voiceChunks = [];
+    const mimeType = recordingMimeType();
+    voiceRecorder = new MediaRecorder(voiceStream, mimeType ? { mimeType } : undefined);
+    voiceRecorder.addEventListener("dataavailable", (event) => {
+      if (event.data?.size) voiceChunks.push(event.data);
+    });
+    voiceRecorder.addEventListener("stop", () => {
+      const recorderMimeType = voiceRecorder?.mimeType || mimeType || "audio/webm";
+      stopVoiceStream();
+      const extension = recorderMimeType.includes("mp4")
+        ? "m4a"
+        : recorderMimeType.includes("ogg")
+          ? "ogg"
+          : "webm";
+      const blob = new Blob(voiceChunks, { type: recorderMimeType });
+      voiceRecorder = null;
+      voiceChunks = [];
+      setActionButtonLabel(recordVoiceButton, "Speak");
+      setVoiceControlsDisabled(false);
+      transcribeVoiceBlob(blob, `recorded-idea.${extension}`);
+    });
+    voiceRecorder.start();
+    setActionButtonLabel(recordVoiceButton, "Stop");
+    setVoiceControlsDisabled(false);
+    setSessionStatus("Listening. Press Stop when your idea is ready.");
+  } catch (error) {
+    stopVoiceStream();
+    voiceRecorder = null;
+    setActionButtonLabel(recordVoiceButton, "Speak");
+    setVoiceControlsDisabled(false);
+    setSessionStatus(`Voice recording could not start: ${error.message}`);
+  }
+}
+
+function recordingMimeType() {
+  const candidates = ["audio/webm;codecs=opus", "audio/webm", "audio/ogg;codecs=opus", "audio/mp4"];
+  return candidates.find((type) => MediaRecorder.isTypeSupported(type)) || "";
+}
+
+function stopVoiceStream() {
+  if (!voiceStream) return;
+  voiceStream.getTracks().forEach((track) => track.stop());
+  voiceStream = null;
+}
+
+async function transcribeVoiceBlob(blob, filename) {
+  if (sessionControlsLocked || voiceBusy) return false;
+  if (!blob?.size) {
+    setSessionStatus("Voice note is empty.");
+    return false;
+  }
+  const revision = bumpSessionRevision();
+  voiceBusy = true;
+  submit.disabled = true;
+  input.disabled = true;
+  setCommandDisabled(true);
+  setSessionControlsDisabled(true);
+  setVoiceControlsDisabled(true);
+  setActionButtonLabel(recordVoiceButton, "Hearing...");
+  setSessionStatus("Transcribing voice note.");
+  try {
+    const formData = new FormData();
+    formData.append("audio", blob, filename || "voice-note.audio");
+    const response = await fetch("/api/transcribe", {
+      method: "POST",
+      body: formData,
+    });
+    if (!response.ok) throw new Error(`voice note failed with ${response.status}`);
+    const data = await response.json();
+    const transcript = String(data.transcript || "").trim();
+    if (!transcript) throw new Error("empty transcript");
+    if (!isCurrentSessionRevision(revision)) return false;
+    input.value = mergeDraftWithTranscript(input.value, transcript);
+    session.ui_status = "Voice note transcribed. Edit the draft or press Ink.";
+    corrections.textContent = session.ui_status;
+    saveSession();
+    return true;
+  } catch (error) {
+    if (isCurrentSessionRevision(revision)) setSessionStatus(`Voice note could not be transcribed: ${error.message}`);
+    return false;
+  } finally {
+    voiceBusy = false;
+    setActionButtonLabel(recordVoiceButton, "Speak");
+    if (isCurrentSessionRevision(revision)) {
+      submit.disabled = false;
+      input.disabled = false;
+      setSessionControlsDisabled(false);
+      setCommandDisabled(false);
+      setVoiceControlsDisabled(false);
+      input.focus();
+    }
+  }
+}
+
+function mergeDraftWithTranscript(draft, transcript) {
+  const current = String(draft || "").trim();
+  return current ? `${current}\n${transcript}` : transcript;
+}
+
 async function bootstrap() {
   const response = await fetch("/api/bootstrap");
   if (!response.ok) throw new Error(`project index failed with ${response.status}`);
@@ -344,6 +482,13 @@ function setSessionControlsDisabled(disabled) {
   whitespaceEl.querySelectorAll("button[data-gap-prompt]").forEach((gap) => {
     gap.disabled = disabled;
   });
+  setVoiceControlsDisabled(disabled);
+}
+
+function setVoiceControlsDisabled(disabled) {
+  const recording = voiceRecorder?.state === "recording";
+  recordVoiceButton.disabled = voiceBusy || (disabled && !recording) || !bootstrapData;
+  uploadVoiceButton.disabled = voiceBusy || recording || disabled || !bootstrapData;
 }
 
 function resetSession() {

static/index.html

@@ -36,6 +36,13 @@
       <symbol id="icon-check" viewBox="0 0 24 24">
         <path d="M5 12l4 4 10-11" />
       </symbol>
+      <symbol id="icon-mic" viewBox="0 0 24 24">
+        <path d="M12 3a3 3 0 0 0-3 3v6a3 3 0 0 0 6 0V6a3 3 0 0 0-3-3z" />
+        <path d="M5 11a7 7 0 0 0 14 0M12 18v3M9 21h6" />
+      </symbol>
+      <symbol id="icon-upload" viewBox="0 0 24 24">
+        <path d="M12 17V4M7 9l5-5 5 5M5 20h14" />
+      </symbol>
     </svg>
 
     <div class="desk-glow"></div>
@@ -112,6 +119,18 @@
               ></textarea>
               <div class="underline"></div>
 
+              <div class="toolbar voice-row" aria-label="Voice input">
+                <button id="record-voice" class="btn btn-voice" type="button" title="Record an idea">
+                  <svg class="icon"><use href="#icon-mic"></use></svg>
+                  Speak
+                </button>
+                <button id="upload-voice" class="btn btn-ghost" type="button" title="Transcribe a voice note">
+                  <svg class="icon"><use href="#icon-upload"></use></svg>
+                  Voice note
+                </button>
+                <input id="voice-file" type="file" accept="audio/*" hidden />
+              </div>
+
               <div class="toolbar command-row" aria-label="Advisor commands">
                 <button id="submit" class="btn btn-ink" type="submit" title="Score this idea">
                   <svg class="icon"><use href="#icon-quill"></use></svg>

static/styles.css

@@ -302,6 +302,10 @@ textarea:disabled {
   flex: 1 1 20px;
 }
 
+.voice-row {
+  margin-top: 10px;
+}
+
 .btn {
   display: inline-flex;
   min-height: 36px;
@@ -349,6 +353,17 @@ textarea:disabled {
   border-color: #38260f;
 }
 
+.btn-voice {
+  color: #f6ecd2;
+  background: var(--leaf);
+  border-color: var(--leaf);
+}
+
+.btn-voice:hover:not(:disabled) {
+  background: #265935;
+  border-color: #265935;
+}
+
 .btn-ghost {
   color: var(--ink-soft);
   background: transparent;

tests/test_app.py

@@ -21,6 +21,7 @@ from app import (
     prize_ledger_endpoint,
     runtime,
     submission_packet_artifact,
+    transcribe_audio,
     tool_contract_check,
     tool_contracts,
     trace_artifact,
@@ -34,6 +35,28 @@ async def _read_streaming_response(response) -> str:
     return "".join(chunks)
 
 
+class DummyUpload:
+    def __init__(
+        self,
+        content: bytes,
+        filename: str = "voice.wav",
+        content_type: str = "audio/wav",
+    ) -> None:
+        self._content = content
+        self._offset = 0
+        self.filename = filename
+        self.content_type = content_type
+
+    async def read(self, size: int = -1) -> bytes:
+        if self._offset >= len(self._content):
+            return b""
+        if size is None or size < 0:
+            size = len(self._content) - self._offset
+        start = self._offset
+        self._offset = min(len(self._content), self._offset + size)
+        return self._content[start : self._offset]
+
+
 def test_health_exposes_index_metadata() -> None:
     payload = health()
 
@@ -41,6 +64,7 @@ def test_health_exposes_index_metadata() -> None:
     assert payload["projects"] == len(index.projects)
     assert payload["index_algorithm"] == "llama-cpp-embedding-v1"
     assert payload["runtime"]["backend"] == "rules"
+    assert payload["voice"]["model_id"] == "nvidia/nemotron-speech-streaming-en-0.6b"
     assert len(payload["snapshot_digest"]) == 64
 
 
@@ -51,6 +75,7 @@ def test_bootstrap_exposes_index_metadata() -> None:
     assert payload["index_generated_at"]
     assert payload["snapshot_digest"]
     assert payload["runtime"]["tool_count"] >= 8
+    assert payload["voice"]["backend"] == "nemo-asr"
     assert payload["top_projects"]
     assert payload["default_goals"] == payload["goal_options"][:3]
     assert [goal["id"] for goal in payload["goal_profiles"]] == payload["goal_options"]
@@ -78,6 +103,48 @@ def test_agent_turn_stream_endpoint_exports_ndjson_events() -> None:
     assert lines[-1]["state"]["ideas"]
 
 
+def test_transcribe_audio_endpoint_saves_audio(monkeypatch) -> None:
+    captured = {}
+
+    def fake_transcribe(path: str) -> dict:
+        captured["path"] = path
+        return {
+            "transcript": "A local-first memory archive.",
+            "model_id": "nvidia/nemotron-speech-streaming-en-0.6b",
+            "backend": "nemo-asr",
+            "sample_rate": 16000,
+        }
+
+    monkeypatch.setattr("app._transcribe_voice", fake_transcribe)
+
+    payload = asyncio.run(transcribe_audio(DummyUpload(b"RIFF....WAVE")))
+
+    assert payload["transcript"] == "A local-first memory archive."
+    assert captured["path"].endswith(".wav")
+
+
+def test_transcribe_audio_endpoint_rejects_non_audio() -> None:
+    upload = DummyUpload(b"hello", filename="note.txt", content_type="text/plain")
+
+    try:
+        asyncio.run(transcribe_audio(upload))
+    except Exception as error:
+        assert getattr(error, "status_code", None) == 415
+    else:
+        raise AssertionError("non-audio upload should fail")
+
+
+def test_transcribe_audio_endpoint_rejects_empty_audio() -> None:
+    upload = DummyUpload(b"", filename="empty.wav", content_type="audio/wav")
+
+    try:
+        asyncio.run(transcribe_audio(upload))
+    except Exception as error:
+        assert getattr(error, "status_code", None) == 400
+    else:
+        raise AssertionError("empty audio upload should fail")
+
+
 def test_markdown_api_endpoints_return_plain_markdown() -> None:
     state = engine.turn("A local-first archive cartographer for family photos", {}).state
 
@@ -246,8 +313,10 @@ def test_prize_ledger_endpoint_reports_submission_evidence() -> None:
 
     assert payload["runtime"]["backend"] == "rules"
     assert payload["tiny_titan_eligible"] is True
+    assert payload["voice"]["model_id"] == "nvidia/nemotron-speech-streaming-en-0.6b"
     assert any(badge["name"] == "Sharing is Caring" for badge in payload["badges"])
     assert {badge["name"]: badge["status"] for badge in payload["badges"]}["Llama Champion"] == "ready"
+    assert {item["role"]: item["status"] for item in payload["model_stack"]}["Voice input"] == "deployed"
     assert payload["retrieval_index"]["index_algorithm"] == "llama-cpp-embedding-v1"
     assert payload["retrieval_index"]["embedding_runtime"] == "llama.cpp via llama-cpp-python"
     assert payload["training_artifacts"][0]["endpoint"] == "lora_dataset"

tests/test_asr_runtime.py

@@ -0,0 +1,29 @@
+from dataclasses import dataclass
+
+from hackathon_advisor.asr_runtime import (
+    DEFAULT_ASR_MODEL_ID,
+    NemotronAsrTranscriber,
+    extract_transcript,
+)
+
+
+@dataclass
+class Hypothesis:
+    text: str
+
+
+def test_nemotron_transcriber_status_is_lazy() -> None:
+    transcriber = NemotronAsrTranscriber()
+
+    status = transcriber.status().to_dict()
+
+    assert status["backend"] == "nemo-asr"
+    assert status["model_id"] == DEFAULT_ASR_MODEL_ID
+    assert status["loaded"] is False
+    assert status["sample_rate"] == 16_000
+
+
+def test_extract_transcript_accepts_nemo_output_shapes() -> None:
+    assert extract_transcript(["A spoken idea."]) == "A spoken idea."
+    assert extract_transcript([{"text": "A mapped archive."}]) == "A mapped archive."
+    assert extract_transcript([Hypothesis("A private timeline.")]) == "A private timeline."

tests/test_frontend_copy.py

@@ -13,6 +13,8 @@ def test_main_interface_copy_is_builder_facing() -> None:
     assert "Example idea board loaded with a plan and share page." in app_js
     assert "/api/artifact.png" in app_js
     assert "/api/agent-turn" in app_js
+    assert "/api/transcribe" in app_js
+    assert "MediaRecorder" in app_js
     assert "readNdjson" in app_js
     assert "@gradio/client" not in app_js
     assert "renderArtifactCanvas" not in app_js
@@ -20,6 +22,8 @@ def test_main_interface_copy_is_builder_facing() -> None:
     assert 'aria-label="Export build notes"' in html
     assert 'aria-label="Export idea-board chapter"' in html
     assert 'aria-label="Export current page as PNG"' in html
+    assert 'aria-label="Voice input"' in html
+    assert "Voice note" in html
     assert "ideaCardAriaLabel" in app_js
     assert "Select idea:" in app_js