Voice Coach

A self-hosted PWA for singing practice: play the original recording of a song from your Navidrome library, listen to your voice via the phone mic in real time, and get live in-key / sharp / flat feedback against the song's original vocal melody.

---

Architecture

Two halves, deliberately separated:

/
├── client/   # React PWA — library browsing, queueing, the singing screen
├── server/   # Node/Fastify API — Navidrome/Subsonic proxy, job queue, pitch-curve cache
├── worker/   # Python GPU worker — Demucs + torchcrepe pipeline
└── docker-compose.yml

1. Offline/background processing (server + worker, GPU)

• The client asks the server to queue a song (by Navidrome song id).
• The server writes a row to a songs table (Postgres) with status = queued. This table is the job queue — see "Why no Redis/BullMQ" below.
• The worker polls Postgres, claims the oldest queued song with SELECT ... FOR UPDATE SKIP LOCKED (atomic even with multiple workers), and runs it through:
  1. Fetch — downloads the source audio from Navidrome via the Subsonic API.
  2. Demucs (htdemucs) — isolates the vocal stem. This is an intermediate artifact only, discarded after step 3 — never served to the client, never heard by the user.
  3. torchcrepe — extracts a time-aligned f0 (pitch) curve from the isolated vocal, at a fixed hop size (10ms).
  4. Writes the curve to a disk cache keyed by song id ({PITCH_CACHE_DIR}/{songId}.json, shared volume) and marks the row ready. A song is only ever processed once; re-queueing a ready song is a no-op, and only a failed song can be re-queued.
• No pre-batching of the library — everything is on-demand.

2. Real-time client (PWA, phone)

• Browse/search the Navidrome library, queue a song, poll status.
• Singing screen: plays the original track (streamed straight from Navidrome — unmodified, includes the original vocalist) while an AudioWorklet taps the mic and runs a client-side YIN pitch detector. Feedback is a scrolling pitch timeline (client/src/components/PitchTimeline.tsx) rather than a discrete in-tune/sharp/flat readout — a fixed vertical playhead with the reference melody drawn both ahead of and behind it (the whole curve is already known up front, so upcoming notes are visible like a karaoke prompter), and the user's own sung pitch scrolling underneath it, colored green/amber/blue per point based on how close it is to the target at that instant. This sidesteps an earlier design entirely: an initial discrete "Listening… / In tune / Sharp / Flat" state flickered between states on every ~45ms audio block that failed the silence/clarity gates (a consonant, a breath, a borderline clarity dip) — a continuous graph has no such state to flicker; a dropout just reads as a gap in the line.

---

Why no Redis/BullMQ for the job queue

Single GPU worker, on-demand processing, low volume (a home user queueing songs one at a time). A songs.status column plus FOR UPDATE SKIP LOCKED on the existing Postgres instance gives atomic claim semantics for free, with zero extra infrastructure. If this ever needs multiple concurrent workers or priority scheduling, revisit — for v1 it would be overengineering.

Why torchcrepe instead of crepe

The original CREPE package is TensorFlow-based. Demucs is PyTorch-based. Running both frameworks in the same GPU worker means carrying two deep learning stacks (and their CUDA/cuDNN version constraints) for no benefit. torchcrepe is a PyTorch port of the same model, so the worker only needs one framework.

---

Bleed tolerance (Risk #1)

The user sings along to the original track, vocals included — some headphone bleed into the mic is expected even with good headphones. Two layers handle this, deliberately without real-time source separation or echo cancellation beyond what the browser gives for free:

1. getUserMedia constraints (client/src/pitch/useLivePitch.ts): request echoCancellation: true, noiseSuppression: true. This is a free first pass — the browser's own AEC is built for exactly this speaker-bleeding- into-mic scenario (it's the same mechanism used for video calls).
2. Reference-biased octave disambiguation (client/src/pitch/yin.ts, selectBestTau): YIN normally resolves pitch ambiguity by picking the shortest plausible period (highest frequency) it finds. If bleed from the original vocal introduces a second plausible periodicity — e.g. the singer's note and the original vocalist's note at different pitches or octaves — classic YIN's "take the first dip" heuristic can pick the wrong one. Since we already know what note should be sung at this instant (the cached reference curve), we instead pick whichever candidate period lands closest in cents to the reference pitch. The reference acts as a prior on "which periodicity is ours."
3. Silence/low-confidence gating (client/src/pitch/useLivePitch.ts): an RMS gate skips analysis on very quiet input (bleed-only, no active singing), and a clarity threshold (YIN's own confidence measure) skips recording a point rather than record an unreliable one — this shows up as a gap in the scrolling sung-pitch line rather than a fabricated reading.

Limitations: this is not source separation. If the user sings very quietly relative to loud headphone leakage, or sings a harmony far from the reference note, the bias can still be defeated. It has been unit-tested against synthetic bleed (a sine wave plus ~22 dB quieter broadband noise, see yin.test.ts) but not against a real recording with real headphone bleed — that needs on-device verification (see "What I could not test" below).

Firefox for Android PWA install (Risk #2)

Verified via Mozilla's bug tracker and current documentation rather than a physical device (see "What I could not test"):

• Modern Firefox for Android (Fenix) does detect a valid web app manifest and offers "Install" (not just a bookmark) from its menu — using the manifest's name, icon, and start_url. This is a real improvement over the old Fennec-era behavior where "Add to Home Screen" silently failed or fell back to a bare bookmark.
• However, Firefox does not honor display: standalone the way Chromium browsers do — this is a long-standing, still-open Mozilla issue (bug 1285858). The installed icon launches the app with Firefox's normal toolbar/address bar visible, not a chrome-less window.
• Conclusion: the manifest is built correctly (client/vite.config.ts — proper display: standalone, icons, theme colors) so Chromium-based browsers and any future Firefox fix get the full experience for free. On Firefox for Android today, "Install" gets you a correctly-icon'd, correctly- landing-page home screen shortcut, just with the address bar still showing. Per the brief, this isn't worth fighting — the in-tab/in-toolbar experience is fully responsive and mobile-optimized regardless, so nothing is lost by using it that way.

Bluetooth HFP downgrade (Risk #3)

When a phone has both mic capture (getUserMedia) and audio playback active over the same Bluetooth device, Android can only satisfy simultaneous mic input over the HFP/HSP profile (phone-call quality: mono, low bitrate, ~16kHz), because the higher-quality A2DP profile is playback-only and has no input path. This is an OS/Bluetooth-stack limitation, not something a web app can negotiate around.

Mitigation: none attempted in-app (out of scope per the brief). The README and the in-app error copy (SingingPage.tsx) recommend wired headphones with a mic as the default setup — wired analog audio has no separate input/output profile to renegotiate, so playback quality is unaffected by mic capture.

---

What I could not test

This was built and unit-tested in a headless dev sandbox with no browser, no microphone, no speakers, and no Android device attached. Everything that depends on real browser/OS audio behavior is implemented to spec and carefully reasoned about, but not run against real hardware from here — real on-device testing (thank you) has already caught and fixed several bugs this couldn't: the AudioWorklet processor shipping as unparsed TypeScript, audio.play() losing its user-gesture window when called after the mic permission prompt, a stream-URL redirect pointing at an internal-only Docker hostname, and pitch-detection thresholds tuned too strictly for a real mic. Still not verified on-device:

• The scrolling pitch timeline (PitchTimeline.tsx) — the coordinate math (time↔pixel, MIDI↔pixel, vocal-range detection, gap segmentation) is fully unit-tested (timeline.test.ts), but the actual canvas rendering — whether it reads clearly at a glance, whether 60fps requestAnimationFrame scrolling is smooth on real phone hardware, whether the colors are legible in daylight — can't be verified without a real screen.
• The reference-bias octave disambiguation (bleed tolerance, above) is unit-tested against synthetic signals but has not been validated against a real recording of a real person singing along to a real song with real headphone bleed.
• Firefox-for-Android "Install" behavior — documented above from Mozilla's own bug tracker/docs, not observed firsthand.
• Bluetooth HFP downgrade — well-documented Android platform behavior, not reproduced on a physical device here.

Before relying on this for real practice, please: install it on your phone, do one full sing-through with wired headphones, and separately try Bluetooth headphones to confirm the quality drop described above actually happens on your hardware.

---

Assumptions and shortcuts (please review)

• Auth: a single shared API_PASSWORD env var gates mutating requests (queueing a song), sent as Authorization: Bearer <password> from localStorage. No login flow — this is a single-user home app behind whatever network-level access control you already run (Traefik/authentik in your homelab). Reads (search, status, pitch curve, stream) are open, since they don't cost GPU time or write anything.
• Stem retention: the isolated vocal stem is deleted after pitch extraction (per-job scratch directory, removed on success or failure). If you later want key transposition (explicitly out of scope for v1), you'd need to either keep the stem or re-run Demucs.
• In-tune tolerance: ±50 cents (a quarter-tone) by default (compareToReference, client/src/pitch/comparison.ts). Easy to make configurable later; hardcoded for v1.
• Pitch confidence gating: mic frames below 0.85 YIN clarity or below an RMS silence threshold show "Listening…" instead of a verdict (useLivePitch.ts). These thresholds are reasonable starting points, not tuned against real recordings.
• No octave-correctness guarantee for the reference curve itself: torchcrepe can still make its own octave errors on the isolated vocal stem (e.g. on breathy or heavily processed vocals) — there's no smoothing/ post-processing pass on the extracted reference curve in v1. Worth revisiting if reference curves look spiky in practice.
• Worker base image: pytorch/pytorch:2.4.0-cuda12.1-cudnn9-runtime in worker/Dockerfile — verify this tag is still current and matches your host's NVIDIA driver before building (I have no internet/registry access from this sandbox to confirm the tag exists today). demucs may also pin a torch version that conflicts with the base image's; if pip install in the worker build complains, pin versions explicitly.
• Stream proxying: /songs/:id/stream proxies audio bytes through the server (SubsonicClient.fetchStream, routes/songs.ts) rather than redirecting the browser to a signed Subsonic URL. It wasn't originally built this way — a 302 redirect is simpler and avoids piping large files through Node — but two real bugs on real deployments forced the change:
  1. A redirect requires NAVIDROME_URL to be reachable from the browser, not just the server, since the browser follows it directly. The first deploy set it to an internal-only Docker address, causing "the media resource ... was not suitable" errors.
  2. After fixing that (with a NAVIDROME_PUBLIC_URL override — still used for coverArtUrl, which nothing calls yet), the same error came back. The cause: Navidrome was configured with a short ND_AUTHWINDOWLENGTH (auth tokens expire quickly), and a redirect hands the browser a signed URL without any control over when it actually gets fetched — initial buffering, seeking, or resuming after a pause can all issue the real request well after the token's window has closed. Proxying means the server generates and uses the token in the same instant, every time, regardless of browser timing. The incoming Range header (used for seeking) is forwarded upstream, and Navidrome's status/headers/body are relayed back as-is — see SubsonicClient.fetchStream and its tests for the exact contract.
• Placeholder PWA icons: client/public/icon-192.png / icon-512.png are solid-color placeholders generated for this build, not real branding — swap them before shipping to a home screen you'll look at daily.
• /api/ proxy re-resolves the server container's address: nginx (client/default.conf.template) proxies /api/ to the server container by hostname. A literal proxy_pass hostname is resolved once, at nginx startup, and cached for the container's whole lifetime — if server is ever recreated (redeploy, crash restart) it gets a new IP on the Docker network, and nginx keeps sending requests to the old, now-dead address until nginx itself restarts, surfacing as intermittent 404s on /api/* that clear up on a manual docker restart of the client container. Fixed by adding resolver 127.0.0.11 valid=10s; (Docker's embedded DNS) and routing proxy_pass through a variable, which is what actually makes nginx consult resolver instead of caching the first answer forever. That switch to a variable has its own side effect: a literal proxy_pass URI ending in / makes nginx replace the matched /api/ location prefix automatically, but a variable-based one forwards the request URI unchanged — so without an explicit rewrite ^/api/(.*)$ /$1 break; first, the backend sees /api/songs/search (no such route) instead of /songs/search and 404s on every request, not just occasionally.

---

Local development

cp .env.example .env   # fill in NAVIDROME_URL / NAVIDROME_USER / NAVIDROME_PASSWORD
docker compose -f docker-compose.dev.yml up

• Client: http://localhost:5173
• Server: http://localhost:3000
• Worker: polls Postgres in the background, logs to its container's stdout

Run tests without Docker:

make test            # server (vitest) + client (vitest) + worker (pytest)

The worker's tests only need pytest (see worker/requirements-dev.txt) — none of the Fake/Null implementations import torch, demucs, or torchcrepe, so you don't need a GPU or those heavy packages installed just to run the test suite. worker/requirements.txt (the real, GPU-dependent packages) is only needed inside the worker's Docker image.

Production deploy

cp .env.example .env   # fill in credentials
docker compose up -d --build

The worker requires an NVIDIA GPU with the NVIDIA Container Toolkit installed on the host (docker-compose.yml requests a GPU device reservation).