DJ VJ

Why I built this

At home the projector used to loop YouTube videos that ignored whatever I was playing. I wanted the wall to follow the mix, calmer in a breakdown and bigger at a drop, without my touching anything mid-set: at a house party both hands are on the controller. The idea came on the evening of 30 September. On 3 October about ten friends watched it run all evening.

What it does

rekordbox sends a copy of the master mix to a virtual audio device, and a fullscreen browser page on the projector listens to that copy. The speakers, headphones and controller never pass through it, so if the visuals crash, the music carries on.

• Main: geometric light scenes and psychedelic presets, changing every 64 bars on a bar line
• Peak: a heavy sub-bass held for a minute brings the psychedelic presets
• Breakdown: eight bars without a kick fades to slow organic shapes
• Drop: a hard cut on the returning kick, back to the preset from before the breakdown
• Surge: when a track gets louder or brighter, a geometric scene gives way to a preset
• Ambient interlude: smooth music gets its own slow pool, with 15-second crossfades
• Silence: a dim drift, never a frozen frame or the desktop

There are no live controls, by decision. The only keys are quit and a tuning overlay that is off by default.

Architecture

Everything runs on one MacBook Air, alongside rekordbox. The engine only listens: nothing it does can reach the speakers.

The Air has no fan and throttles under sustained load, so every scene is judged on frame time as well as looks. If frames run slow for ten seconds, the compositor drops the render size a step, and never steps back up during a night, so it can’t oscillate.

Scenes take time from the session clock and randomness from a seed, never from the system clock. That is what will let a recorded set be re-rendered to video exactly as the wall showed it.

Scenes

Seven original shaders, written from scratch rather than adapted from Shadertoy, whose default licence bars commercial reuse. Each takes its shape from a seed (counts, symmetry, size, palette), so a scene that comes round again looks different, not just recoloured. Alongside them play 46 MilkDrop presets through Butterchurn, kept from 100 by watching each one on the wall.

Stills rendered straight from the scene code, on a simulated kick. The presets aren’t shown here; they are other people’s work.

Hearing the music

I play three kinds of section: deep (sparse and spacious), driving (a continuous kick with bright tops) and peak (weight: a heavy low end, tension and release). The first plan was one energy score across all three. On a labelled 29-minute set of mine it got peak right 0% of the time, because my peaks are darker, not louder: loudness differed by about 2 dB across all three.

Only peak showed up in the audio at all, as the sub-bass (20–60 Hz) outweighing the bass above it. Deep and driving overlapped on every measure tried. So the model became two characters, main and peak, plus events. On parts of that set held out from tuning, it read peak correctly 79% of the time and the rest as main 90–94% of the time.

Then my own habits got in the way. A low-pass filter sweep cuts the bass but not the sub, which fakes a peak for as long as it lasts, and I ride the filter in most blends. So peak has to hold for a full minute before the wall changes, and ends the moment it fades.

Safe for a room of people

Psychedelic presets can strobe, and guests at a party haven’t signed up for that. A limiter in the compositor applies the web accessibility rule for flashing: no more than three flashes a second. It checks each frame as an 8 × 6 grid of tiles, because on a wall a local flash matters long before the whole-frame average moves. No scene, mine or borrowed, is trusted to limit itself.

The first version counted only jumps between consecutive frames, so a preset pulsing smoothly at hi-hat speed passed untouched at up to 13 changes a second. Counting every jump would have damped an ordinary kick instead. It now counts pairs of opposite changes, as the standard does. The cost shows up as stutter, since a held frame looks frozen, so I dimmed the twelve presets it held most rather than drop them. Most now play smoothly.

Tuned on real sets

The tests use synthetic audio; whether it looks right took real sets. I played four on 2 October, and each one changed the director:

• Set 1: the same three geometric scenes 85% of the time. Presets got a third of the main picks (later over half), and each scene now takes its shape from a seed
• Set 2: better, but restless. On a slow track with a soft kick the wall changed 18 times in 3.5 minutes. A breakdown now needs eight bars without a kick, and a switch between main and peak waits until the scene has shown for a minute
• Set 3: calm but dull through loud, rough sections. A surge now lifts the wall when a track gets louder or brighter
• Set 4: an hour of dark minimal techno that “all worked alright”. One change: the drop now cuts hard instead of fading

Where it’s at

V1 passed its acceptance test on 3 October: a 3.5-hour house party for about ten friends, with me playing back to back with another DJ.

• No crash, no gap in the frame log across 12,683 seconds, and full render resolution all night
• About 17 minutes of silence in 13 stretches while guests arrived; the wall picked the music back up every time
• 55 different scenes shown; presets were on screen 74% of the night
• Still open: a new preset drops one frame as it starts, which only a patch to Butterchurn would remove
• Still unproven: whether it finds my peaks beyond the set it was tuned on. That needs a set with the times written down

What’s next

Recording a set and rendering it to video afterwards, replaying the logged decisions rather than deciding again, so the video matches the night. Butterchurn calls Math.random() in 29 places, so the first test is whether its presets can be made to replay exactly. Most of my sets are streamed, which Spotify’s terms don’t allow recording, so recording will be opt-in and for downloaded tracks only.

Later, a new scene when a new track comes in, read from the controller, and scene changes locked to the true bar through Ableton Link. Link stays read-only: a peer that sets the tempo would retime the deck that’s playing.