(de)Engineering
Using LLMs to Simulate Wild Thinking and Convergence Visualizing Song Structure with Self-Similarity Matrices Mapping Consonants as Percussion: A Small Experiment with Whisper and Audio Analysis Semantic Self-Similarity or How I Split a Conversation into Scenes Using Language Models Modeling the Noise: Building a Tinnitus Generator in Python Making A Humble OpenGL Rotating Cube Semantic Analysis Applied to "Sent From My Telephone" by Voice Actor Longitudinal Sentiment Analysis of Personal Chat Logs Filtering spam using cheap heuristics A Two-Stage x86 Bootloader That Draws a Penguin Looking for structure in "Red is a Slow Colour"

Mapping Consonants as Percussion: A Small Experiment with Whisper and Audio Analysis

Sunday, August 24th 2025 — Audio Analysis, Creative AI, Music Tech, Plosive Consonants, Vocal Percussion, Whisper

I've been playing with treating consonant sounds in vocals as if they were percussion hits. Many singers don't just carry melody and semantics with their voices. They, knowingly or not, also use the attack of consonants (p, t, k, ch) as rhythmic accents that become part of the drumming texture.

The experiment was simple:

  1. Take a track (The World Backwards - Broadcast (UK)) and run it through Whisper, the FOSS speech recognition model from OpenAI.
  2. From the transcript, ignore the accuracy or meaning of words altogether and only pay attention to their plosive consonants. Note: If the model suggests "feel" instead of "fear", or "pattern" instead of "patent" we don't care, as we're not looking for semantic accuracy, just the plosives.
  3. Place those sounds on a timeline aligned with a bar/beat grid in 3/4 at 188 BPM (which is the song's tempo).

What comes out is a kind of map of vocal percussion, a scatter of markers where the singer's consonants align with or play against the beat.

Markers show plosive consonants plotted against a 3/4 time grid at 188 BPM
and plosives align quite well to the grid 👌

Why bother with this? Maybe because lyrics aren't just semantics or poetry, but also audio events that interact with rhythm, just like a hi-hat or a snare. By visualizing them this way, you can see how the human voice is simultaneously melody, meaning, and percussion.

The Technical Side

The Python implementation was straightforward:

  • Parse Whisper's JSON output with word-level timestamps
  • Filter for words containing plosive consonants (p, b, t, d, k, g, c, q, ch)
  • Plot them against calculated beat markers based on BPM
# Simple plosive detection
plosives = ["p", "b", "t", "d", "k", "g", "c", "q", "ch"]
for word in transcript:
    if any(plosive in word.lower() for plosive in plosives):
        events.append(word.timestamp)

Unexpected Insights

It's a tiny project, but it sparked some observations:

  • Machine transcription tools aren't just for accessibility or subtitles.
  • Rhythm isn't only in drums and basslines.
  • You can strip away meaning and listen to the raw mechanics of sound.

What's Next?

The logical next step? What if we applied this to glossolalia or constructed languages like Simlish? Whisper would still attempt to guess and fit phonetic patterns to real words, potentially revealing the "envelope" or rhythmic skeleton of vocals.

Sometimes you should try to use tools for purposes they weren't designed for 😸

Show All