Visualizing how we hear — from spectrograms to perception
PerceptoMap is an open-source audio plugin (VST3) that visualizes psychoacoustic features of audio signals in real time. Built with JUCE, it's designed to help you see how we perceive sound — not just how it looks on a frequency plot.
Unlike typical spectrum or spectrogram analyzers, it supports perceptual visualizations such as Mel spectrograms and Mel-frequency cepstral coefficients (MFCCs), and soon also Chroma, Tempogram etc., offering insight into how humans perceive sound.
🎧 If you're the kind of creator who trusts your ears above all — you might not need this.
But if you're curious about how your audio measures up to what humans actually hear… welcome aboard.
- Real-time Mel Spectrogram display with perceptual frequency scaling
- Real-time Mel-frequency cepstral coefficients (MFCCs) representing timbral texture and spectral envelope
- Real-time Spectral Centroid tracking to visualize spectral brightness (center of mass of STFT spectrum)
- Visual analysis of Chroma, Tempogram, and other psychoacoustic features (planned)
- Configurable color maps
- Optional dB scaling, log or linear frequency axis for classic linear STFT spectrogram
- Freeze frame mode and interactive mouse hover to inspect frequency and loudness at any point
Plugin User Interface in Ableton Live 12
Resizable GUI Window can shrink to fit minimal layout |
Resizable GUI Window expands for detailed viewing |
Classic Default high-contrast mapping |
Grayscale Neutral luminance-based display |
Magma Perceptually uniform, dark background |
Hover ReadoutDisplays precise frequency, dB level, and corresponding MIDI note (C4 = Middle C) [added in v0.3] under the mouse pointer. |
Adjustable dB Floor SliderControls the minimum dB threshold for color brightness, helping visualize low-level signals. [added in v0.3] |
Linear STFT spectrogram with linear frequency axis: Displays physical frequency content directly. |
Linear STFT spectrogram with log frequency axis: Approximates human pitch perception. Emphasizes low-frequency resolution and compresses high-frequency bands. |
Mel-scaled STFT spectrogram: Reflects the nonlinear frequency resolution of human hearing. Provides a more perceptually accurate representation than simple log-scaling. |
Mel-frequency cepstral coefficients (MFCCs): Captures the spectral envelope using Discrete Cosine Transform (DCT) over Mel energies. Represents timbral texture and vocal tract shape. [added in v0.2] |
Spectral Centroid: Indicates the "center of mass" of the spectrum. Tracks brightness and perceptual sharpness by showing where the dominant frequencies are concentrated over time. [added in v0.3] |
|
| Feature | Status | Description | Implementation Details |
|---|---|---|---|
| Linear STFT Spectrogram | ✅ Done (v0.1) | Classic time–frequency analysis | FFT size = 2048, Hann window, with log/linear frequency axis display |
| Mel-Spectrogram | ✅ Done (v0.1) | Nonlinear frequency scaling approximating human pitch perception | 128 bands, Slaney-style: 2595 * log10(1 + f / 700) |
| MFCC | ✅ Done (v0.2) | Mel frequency cepstral coefficients, compact representation of timbre based on perceptual log-mel spectrum | DCT-II on log-mel spectrum, 20 coefficients, no liftering, values clipped to [−100, 100] and normalized to [0, 1] for display |
| Spectral Centroid (STFT-based) | ✅ Done (v0.3) | Tracks the "center of mass" of the spectrum; correlates with brightness and sharpness | Computed from linear STFT magnitude spectrum with smoothing (Exponential Moving Average), overlaid as a curve on the STFT spectrogram |
| Chroma | ⏳ Planned | Pitch class mapping, projection of spectral energy onto 12 pitch classes (C, C#, D…) | - |
| Tempogram / Rhythm Map | ⏳ Planned | Visualizes perceived tempo and rhythmic periodicities over time | - |
| Spectral Flatness / Contrast | ⏳ Planned | Measures of timbral characteristics | - |
In the fields of psychoacoustics, and machine learning, perceptually inspired representations such as Mel spectrograms and MFCCs are widely used — for example, in music genre classification, emotion recognition, or detecting AI-generated audio.
As a frequent user of tools like librosa in Python, while learning about DAWs, I was surprised to find that most DAWs seem to lack real-time, perceptually grounded visualization tools.
So I decided to build one — a lightweight, JUCE-based plugin that brings these powerful analysis tools directly into the DAW environment, where musicians, sound designers, and researchers can explore them interactively.
You can download the latest version of PerceptoMap from the Releases page.
- VST3 (
.vst3)
- Download the plugin
.zipfile from the Releases - Unzip the file
- Copy
.vst3plugin folder to the default system VST3 directory:C:\Program Files\Common Files\VST3\
Note: If you use a custom VST3 plugin path, copy it there instead.
- Launch your DAW and run a plugin rescan if necessary
- Then, you should be able to find the plugin under
hqrrr - PerceptoMap
Not yet prebuilt — If you're a developer or familiar with JUCE/Xcode, please build from source, see instructions for developers below.
- Download the plugin
.zipfile from the Releases - Unzip the file
- Copy
.vst3plugin folder to your VST3 directory
Special thanks to @FranklyFlawless from the KVR Audio forum for building and testing v0.3 on Debian 12 Bookworm (DAW: Ardour 8.12.0).
After installation, restart your DAW and run a plugin rescan if needed.
- JUCE 8.x (automatically fetched via CMake)
- C++17 compatible compiler, e.g. Visual Studio 2022 (Windows)
- CMake 3.22+
Based on JUCE CMake Plugin Template.
- Open the project root in Visual Studio (choose "Open a local folder").
- Visual Studio will automatically detect the CMakeLists.txt.
- Select a CMake target configuration (e.g. x64-Release).
- In the CMake Targets View:
PerceptoMap/PerceptoMap Project/Targets, right-clickPerceptoMap_VST3and click Build. - The plugin binary will be placed in the build output directory:
out/build/x64-Release/VST3/PerceptoMap.vst3
cd path\to\PerceptoMap
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release
If using Ninja
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build
PerceptoMap/
├── _pics/ -> Screenshots and images for documentation
├── Source/ -> Main plugin source code
├── CMakeLists.txt -> Main build configuration (CMake-based)
├── CMakeSettings.json -> (Optional) Visual Studio CMake config
├── README.md -> Project documentation
└── LICENSE -> AGPLv3 license file (required for JUCE open-source usage)
PerceptoMap is proudly open-source and completely free to use, modify, and redistribute under the terms of the GNU AGPLv3 License.
There are no hidden fees, paid versions, or limitations — the plugin is intended to be a community-driven tool for perceptual audio analysis and creative exploration.
I do not ask for donations — what matters more is your feedback, feature ideas, or even better: your involvement in development.
Ways you can contribute:
- 🐞 Report bugs or issues you encounter
- 💡 Suggest improvements or new perceptual features
- 🔧 Submit pull requests to improve code or documentation
- 📢 Share the plugin with others who may find it useful
Feel free to leave a comment — bug reports, feature ideas, or just thoughts are always welcome.