Pied Piper — Audio Fingerprinting & Song Recognition

Index (offline, CLI): Audio → spectrogram → spectral peaks → 32-bit fingerprint hashes → SQLite Identify (live, web): Mic recording → WAV → same fingerprinting → hash lookup → score by time-offset consistency → ranked matches

• •Preprocessing: normalize to 16-bit PCM at 44.1 kHz, apply a 5 kHz low-pass filter, then downsample 4× to 11.025 kHz — most identifying musical energy lives below 5 kHz, so this cuts data volume without losing signal.
• •Time–frequency analysis: split into 1024-sample frames with a 512-sample hop (50% overlap, ~93 ms per frame), apply a Hamming window to reduce spectral leakage, then a Cooley–Tukey FFT, keeping the positive-frequency half of the magnitude spectrum.
• •Feature extraction: divide frequency bins into logarithmic bands and keep only the strongest peak per band above a threshold, producing a sparse constellation map of (time, frequency) landmarks that survives noise and compression.
• •Fingerprinting: pair each anchor peak with up to 5 subsequent target peaks and pack each pair into a 32-bit hash — 9 bits of anchor frequency, 9 bits of target frequency, and 14 bits of time delta. Hashing *relationships* rather than absolute values is what makes the fingerprint robust to where the recording started.

• •Every query hash is looked up in an indexed table mapping hash → (songID, anchorTimeMs), yielding candidate songs with matched time pairs.
• •For each match, compute the relative offset dbTime - queryTime and bucket it into 100 ms windows to tolerate timing drift.
• •Score each candidate by the size of its strongest offset cluster. A correct match produces many fingerprints piling into a single bucket; a wrong one scatters offsets with no consensus.
• •Rank by score and return the best match only if it clears a confidence threshold, otherwise return no match.

• •Ingest CLI: pulls audio via yt-dlp, converts through FFmpeg, fingerprints it, and writes to SQLite.
• •Server: Go HTTP server with Socket.IO handlers streaming the identification lifecycle back to the browser.
• •DSP package: spectrogram, FFT, peak extraction, fingerprinting, and matching.
• •Client: React 18 + Vite frontend with microphone capture, a live waveform visualizer, and match results.

• •Implementing the FFT and windowing correctly — an off-by-one in framing or a missing window silently degrades every downstream stage.
• •Choosing peak-picking thresholds sparse enough to stay noise-resistant but dense enough to still match short clips.
• •Packing frequency and time-delta into 32 bits without collisions that flood the candidate set.
• •Recognizing that hash equality only *generates* candidates — time-offset consistency is what actually identifies the song.
• •Managing fingerprint database growth, since a full track produces millions of rows.

• •Digital signal processing fundamentals: sampling, aliasing, windowing, spectral leakage, and the time–frequency tradeoff.
• •Why combinatorial hashing of peak pairs beats matching raw spectra.
• •Designing a compact binary hash format under hard bit budgets.
• •Consistency-based scoring as a general technique for matching under unknown offset.
• •Streaming a multi-stage backend pipeline's progress to a browser over Socket.IO.

• •Built a complete, working audio recognition system — ingestion, DSP, fingerprint storage, matching, and a live web client — with the signal processing written from first principles.
• •Reproduced the core algorithm behind industrial audio identification systems end to end.