How I built an audio-reactive ASCII visualizer for my DJ sets

I wanted visuals for the sets I post on YouTube. I got the idea from this reel by 303.mtl, an audio-reactive ASCII camera made in TouchDesigner. I built my own version so I could get the video out quickly.

Me redrawn in blue ASCII characters on a black background

You can try it in your browser or get the code on GitHub.

Webcam to ASCII

Each webcam frame is shrunk to a small grid, one pixel per character. Each pixel’s brightness picks a character from a ramp, from a space for the darkest pixels to @ for the brightest.

const CHARS = " .:-=+*#%@"; // dark -> bright

const lum = (0.299 * r + 0.587 * g + 0.114 * b) / 255;
line += CHARS[Math.floor(lum * (CHARS.length - 1))];

Each frame’s brightness range is stretched across the full ramp, so the image uses all the characters in any lighting.

Cutting out the background

MediaPipe’s selfie segmentation finds the person in the frame. It’s a small model from Google that runs in the browser, and it gives each pixel a score for how likely it is to be a person. Any cell below a cutoff is drawn as a space.

segmenter.segmentForVideo(video, now, (result) => {
  mask = result.confidenceMasks[0].getAsFloat32Array();
});

// for each character cell
if (mask[i] < CUTOFF) line += " ";
const isMe =
  average >= CUTOFF ||
  (peak >= MOTION_CUTOFF && moving > MOTION_AMOUNT);

Reacting to the music

You drag your mix file onto the page. The browser plays it and analyzes it at the same time. The mic isn’t used.

It follows the kick drum. When the low end of the audio jumps above its recent average, that counts as a kick:

if (bass > bassAvg * 1.25 && bass > 0.3) flash = 1;
flash *= 0.88; // fades out before the next kick

On each kick, the characters get brighter and the colors change faster.

Colors

The colors come from a photo I took at a rave: dark blue, bright blue, light cyan, laser green, and orange. The visualizer fades through them in order.

A blue-lit rave crowd with laser beams overhead, next to the color swatches taken from it

const PALETTE = [
  [42, 60, 255],   // dark blue
  [61, 120, 255],  // bright blue
  [140, 215, 255], // light cyan
  [90, 255, 80],   // laser green
  [255, 95, 45],   // orange
  // ...
];

Three frames from my set: me in blue, green, and orange ASCII

Recording

The browser records the canvas directly, and the mix’s audio is added to the same recording:

const stream = canvas.captureStream(60);
mixAudio.getAudioTracks().forEach((t) => stream.addTrack(t));
const recorder = new MediaRecorder(stream, { videoBitsPerSecond: 16e6 });

It records at 60fps. Recording happens in real time, so keep the tab open and visible the whole time.

The result

Here’s my set from Bourbon on Division in Chicago with the visualizer:

All the settings, like the colors, character size, and how strongly it reacts, are at the top of the file.


I DJ as Max Control. Hear my mixes on SoundCloud.