Ray Kiln
Shader

Pulse Synth

A small FM synthesizer and the oscilloscope watching it. An arpeggio plays, each note one sine wave whose phase is bent by another, shaped by an attack, decay, sustain and release. The picture calls the sound's own function and draws its samples, left and right, as two glowing traces a few cents apart, with the note's envelope above them and the sidebands the bending makes below.

· Sound & Rhythm · Fragment shader

Teaches FM synthesis and envelopes, a picture drawn from its own waveform

Builds on Kaleidoscope (a note of the scale), A Shader That Keeps the Beat (whole-number hash), Murmuration (segment distance)

#FM synthesis#envelope#oscilloscope#Bessel functions#procedural sound
Watch how it is builtSee the variations

Watch

The idea

Every Ray Kiln shader has a sound function beside its picture, and in A Shader That Keeps the Beat the two kept time by working out the same clock. This one goes further: the picture is the sound. synth(t, p) returns the stereo sample at time t, and shade calls it, seven times for every pixel, to draw the last few milliseconds of it the way an oscilloscope does. Whatever you change in the sound, the trace changes with it, because there is only one function.

The sound is a small synthesizer of the kind that filled the 1980s: frequency modulation, found by John Chowning in 1973. One sine wave, the note, has its phase pushed back and forth by a second sine, the modulator. Two sines make a whole spectrum of partials, the sidebands, and how hard the push is decides how bright the note sounds.

It is built in four steps, and each step is a shader you can run on its own.

Stage 1 · A wave you can see

One sine at pitch (220 Hz to start), the same on both sides, faded in over a second and a half. synth returns it at half scale; sound plays it at a third of that, since a tone that never stops is loud.

The picture is a scope. Across the middle 92% of the width it draws 9 milliseconds of synth, so a 220 Hz wave shows about two cycles and an 880 Hz one about eight. A scope that started each sweep at an arbitrary moment would show the wave rolling; this one starts at the last whole cycle, floor(t × pitch) / pitch, which is what a scope's trigger does, and the wave stands still.

To draw a curve known only by its samples, traceDistance takes seven of them, three pixels apart around the pixel being drawn, joins them with straight segments and measures the distance to the nearest one. The line is solid within two pixels and has an exp glow beyond. A faint graticule, a line every tenth of the height, makes it a screen.

stage-1-wave.metal
// Stage 1 · A wave you can see
// One sine wave, heard and drawn. The picture is an oscilloscope: it calls the same synth() the sound plays, for 9
// milliseconds of samples, and draws them across the width. Starting each sweep where a cycle begins holds the wave
// still, as a scope's trigger does.

// @param pitch 220 110 880 The note's frequency, in hertz: more cycles fit across
// @param volume 0.8 0 1 Loudness of the sound

constant float TAU = 6.2831853;

// The kiln palette from Flowing Color Field.
float3 kiln(float t) {
    float3 c0 = float3(0.020, 0.012, 0.020);
    float3 c1 = float3(0.280, 0.040, 0.030);
    float3 c2 = float3(0.880, 0.260, 0.050);
    float3 c3 = float3(1.000, 0.680, 0.200);
    float3 c4 = float3(1.000, 0.970, 0.840);
    t = clamp(t, 0.0, 1.0);
    float3 c = mix(c0, c1, smoothstep(0.00, 0.22, t));
    c = mix(c, c2, smoothstep(0.22, 0.50, t));
    c = mix(c, c3, smoothstep(0.50, 0.76, t));
    return mix(c, c4, smoothstep(0.76, 1.00, t));
}

// The sound before its volume: a sine at `pitch`, the same on both sides,
// faded in over the first second and a half.
float2 synth(float t, constant Params& p) {
    float fadeIn = smoothstep(0.0, 1.5, t);
    return float2(sin(TAU * p.pitch * t)) * 0.5 * fadeIn;
}

// Distance from point a to the segment from b to c.
float segment(float2 a, float2 b, float2 c) {
    float2 ba = a - b, cb = c - b;
    float h = clamp(dot(ba, cb) / dot(cb, cb), 0.0, 1.0);
    return length(ba - cb * h);
}

// 1 on a line d away, fading out over w; and 1 when v is between a and b.
float lineAt(float d, float w) {
    return 1.0 - smoothstep(0.0, w, d);
}

float within(float v, float a, float b) {
    return step(a, v) * step(v, b);
}

// The scope's graticule: faint lines every tenth of the height, the middle
// one brighter.
float graticule(float2 uv, float2 q, float px) {
    float2 cell = abs(fract(q / 0.1 + 0.5) - 0.5) * 0.1;
    float grid = lineAt(min(cell.x, cell.y), 1.5 * px);
    float heat = 0.05 * grid * within(uv.y, 0.2, 0.8);
    return heat + 0.06 * lineAt(abs(uv.y - 0.5), 1.5 * px);
}

// How far q is from each trace, left and right: 9 ms of synth() from
// `start`, across the width. Seven samples 3 pixels apart, joined by lines.
float2 traceDistance(float2 q, float start, float aspect, float px,
                     constant Params& p) {
    float2 off = float2(1.0);
    float2 last = float2(0.0);
    for (int i = -3; i <= 3; i++) {
        float x = q.x + float(i) * 3.0 * px;
        float share = clamp((x / aspect - 0.04) / 0.92, 0.0, 1.0);
        float2 s = synth(start + share * 0.009, p) * 0.6 + 0.5;
        if (i > -3) {
            float x0 = x - 3.0 * px;
            off.x = min(off.x, segment(q, float2(x0, last.x), float2(x, s.x)));
            off.y = min(off.y, segment(q, float2(x0, last.y), float2(x, s.y)));
        }
        last = s;
    }
    return off;
}

float4 shade(float2 uv, constant Uniforms& u, constant Params& p) {
    float t = u.time;
    float aspect = u.resolution.x / u.resolution.y;
    float2 q = float2(uv.x * aspect, uv.y);
    float px = 1.0 / u.resolution.y;
    float heat = 0.03 + graticule(uv, q, px);

    // The sweep starts at the last whole cycle, so the wave stands still.
    float start = floor(t * p.pitch) / p.pitch;
    float off = traceDistance(q, start, aspect, px, p).x;
    float inside = within(uv.x, 0.04, 0.96);
    heat += (0.55 * lineAt(off, 2.0 * px) + 0.18 * exp(-off / (8.0 * px)))
            * inside;
    return float4(kiln(heat), 1.0);
}

// ---- Sound -----------------------------------------------------------------------------------------------------
// What you hear is synth(), the same samples the picture draws, at the volume the slider sets.

// A tone that never stops is loud: it plays at a third.
float2 sound(float t, constant Params& p) {
    return synth(t, p) * 0.3 * p.volume;
}

Stage 2 · Notes and envelopes

Now the sine plays notes: eighths, so a step lasts 30 / tempo seconds (0.3 s at 100), eight to a bar. Each bar arpeggiates a chord of A minor pentatonic: its root (A3, E4, D4, then C4) and the same shape over the scale above it. The first note of a bar always plays and about one in five of the others rest, chosen by a hash of the step, so the phrase repeats every four bars.

Each note is shaped by an ADSR envelope (adsr). The key is held for 60% of a step. While it is down the level rises in a straight line over attack seconds, then falls towards sustain with a time constant of decay; once it is let go, it falls from wherever it was towards silence with a time constant of release. The mix is every note of the last twelve steps, each times its envelope, and a note twelve steps old is faded out over its last three so dropping it is never a click. tanh keeps the sum under full scale.

The trace now grows on each strike and shrinks as the note dies, and where one note rings into the next, the scope shows the two waves added: a sine bent out of shape. Above it, envelopePanel draws the latest note's envelope from strike to silence, a mark where the key is let go, and a dot riding it at the moment you hear.

stage-2-envelope.metal94 new lines since stage 1
// Stage 2 · Notes and envelopes
// Now the sine plays notes: an arpeggio in eighths, each note shaped by an envelope. It rises (attack), falls to a
// level it holds (decay, sustain), and when the key is let go it fades (release). The trace grows and shrinks with
// it, and the envelope is drawn above.

// @param tempo 100 60 160 Beats a minute; the notes are eighths
// @param attack 0.01 0.003 0.3 Seconds each note takes to rise
// @param decay 0.25 0.05 1 Seconds, roughly, to fall from the peak to the sustain
// @param sustain 0.4 0 1 The level a note holds while its key is down
// @param release 0.3 0.02 0.6 Seconds, roughly, a note takes to fade once let go
// @param volume 0.8 0 1 Loudness of the sound

constant float TAU = 6.2831853;

// The kiln palette from Flowing Color Field.
float3 kiln(float t) {
    float3 c0 = float3(0.020, 0.012, 0.020);
    float3 c1 = float3(0.280, 0.040, 0.030);
    float3 c2 = float3(0.880, 0.260, 0.050);
    float3 c3 = float3(1.000, 0.680, 0.200);
    float3 c4 = float3(1.000, 0.970, 0.840);
    t = clamp(t, 0.0, 1.0);
    float3 c = mix(c0, c1, smoothstep(0.00, 0.22, t));
    c = mix(c, c2, smoothstep(0.22, 0.50, t));
    c = mix(c, c3, smoothstep(0.50, 0.76, t));
    return mix(c, c4, smoothstep(0.76, 1.00, t));
}

uint hashBits(uint x) {
    x ^= x >> 16;
    x *= 0x7feb352du;
    x ^= x >> 15;
    x *= 0x846ca68bu;
    x ^= x >> 16;
    return x;
}

// ---- The sequencer -------------------------------------------------------------------------------------------------
// Eighth notes, eight to a bar; each bar arpeggiates a chord of A minor pentatonic.

float noteLength(constant Params& p) {
    return 30.0 / max(p.tempo, 1.0);
}

// A minor pentatonic from A2: three octaves.
float scaleNote(int i) {
    const float notes[15] = {110.00, 130.81, 146.83, 164.81, 196.00,
                             220.00, 261.63, 293.66, 329.63, 392.00,
                             440.00, 523.25, 587.33, 659.25, 783.99};
    return notes[clamp(i, 0, 14)];
}

// Step n's pitch: the bar's chord, then the arpeggio's shape.
float noteFrequency(int n) {
    const int roots[4] = {5, 8, 7, 6};
    const int shape[8] = {0, 2, 4, 5, 7, 5, 4, 2};
    return scaleNote(roots[(n / 8) % 4] + shape[n % 8]);
}

// The first note of a bar always plays; about one in five others rests.
bool notePlays(int n) {
    if (n < 0) return false;
    if (n % 8 == 0) return true;
    return (hashBits(uint(n % 32) + 777u) & 255u) > 50u;
}

// ---- The envelope --------------------------------------------------------------------------------------------------
// Attack, decay, sustain, release. The key is held for 60% of a step (the gate): the level rises in a straight line,
// falls towards the sustain, and once let go falls from wherever it was towards silence.

float gateLength(constant Params& p) {
    return 0.6 * noteLength(p);
}

float heldLevel(float since, constant Params& p) {
    float a = max(p.attack, 0.003);
    if (since < a) return since / a;
    float fall = exp(-(since - a) / max(p.decay, 0.01));
    return mix(p.sustain, 1.0, fall);
}

float adsr(float since, constant Params& p) {
    if (since < 0.0) return 0.0;
    float g = gateLength(p);
    if (since < g) return heldLevel(since, p);
    return heldLevel(g, p) * exp(-(since - g) / max(p.release, 0.01));
}

// ---- The voice -----------------------------------------------------------------------------------------------------
// A plain sine, `since` seconds into a note of frequency f, times its envelope.

float sineVoice(float f, float since, constant Params& p) {
    return sin(TAU * f * since) * adsr(since, p);
}

// ---- The mix -------------------------------------------------------------------------------------------------
// Every note of the last twelve steps, summed; the oldest let go over three steps, so dropping it never clicks.

float2 synth(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    float sum = 0.0;
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (!notePlays(n)) continue;
        float since = t - float(n) * len;
        float held = 1.0 - smoothstep(9.0, 12.0, since / len);
        sum += held * sineVoice(noteFrequency(n), since, p);
    }
    float fadeIn = smoothstep(0.0, 1.5, t);
    return float2(tanh(sum * 0.25)) * fadeIn;
}

// ---- The picture ---------------------------------------------------------------------------------------------------

// The latest note to have started: its start time and frequency.
float2 latestNote(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (notePlays(n)) return float2(float(n) * len, noteFrequency(n));
    }
    return float2(0.0, noteFrequency(0));
}

// Distance from point a to the segment from b to c.
float segment(float2 a, float2 b, float2 c) {
    float2 ba = a - b, cb = c - b;
    float h = clamp(dot(ba, cb) / dot(cb, cb), 0.0, 1.0);
    return length(ba - cb * h);
}

// 1 on a line d away, fading out over w; and 1 when v is between a and b.
float lineAt(float d, float w) {
    return 1.0 - smoothstep(0.0, w, d);
}

float within(float v, float a, float b) {
    return step(a, v) * step(v, b);
}

// The scope's graticule: faint lines every tenth of the height, the middle
// one brighter.
float graticule(float2 uv, float2 q, float px) {
    float2 cell = abs(fract(q / 0.1 + 0.5) - 0.5) * 0.1;
    float grid = lineAt(min(cell.x, cell.y), 1.5 * px);
    float heat = 0.05 * grid * within(uv.y, 0.2, 0.8);
    return heat + 0.06 * lineAt(abs(uv.y - 0.5), 1.5 * px);
}

// How far q is from each trace, left and right: 9 ms of synth() from
// `start`, across the width. Seven samples 3 pixels apart, joined by lines.
float2 traceDistance(float2 q, float start, float aspect, float px,
                     constant Params& p) {
    float2 off = float2(1.0);
    float2 last = float2(0.0);
    for (int i = -3; i <= 3; i++) {
        float x = q.x + float(i) * 3.0 * px;
        float share = clamp((x / aspect - 0.04) / 0.92, 0.0, 1.0);
        float2 s = synth(start + share * 0.009, p) * 0.6 + 0.5;
        if (i > -3) {
            float x0 = x - 3.0 * px;
            off.x = min(off.x, segment(q, float2(x0, last.x), float2(x, s.x)));
            off.y = min(off.y, segment(q, float2(x0, last.y), float2(x, s.y)));
        }
        last = s;
    }
    return off;
}

// The envelope, top left: the latest note's shape from strike to silence,
// the floor, a mark where the key is let go, and a dot where it is now.
float envelopePanel(float2 q, float since, float aspect, float px,
                    constant Params& p) {
    float2 box = float2(0.04 * aspect, 0.84);
    float2 size = float2(0.36, 0.11);
    float span = gateLength(p) + 3.0 * p.release;
    float ex = (q.x - box.x) / size.x;
    float on = within(ex, 0.0, 1.0);
    float heat = 0.0;
    if (q.y > box.y - 0.02 && q.y < box.y + size.y + 0.02) {
        float ea = clamp(ex, 0.0, 1.0), eb = ea + px / size.x;
        float2 a = box + float2(ea, adsr(ea * span, p)) * size;
        float2 b = box + float2(eb, adsr(eb * span, p)) * size;
        heat += 0.5 * lineAt(segment(q, a, b), 1.5 * px) * on;
        heat += 0.08 * lineAt(abs(q.y - box.y), px) * on;
        float gx = box.x + gateLength(p) / span * size.x;
        float tall = within(q.y, box.y, box.y + size.y);
        heat += 0.1 * lineAt(abs(q.x - gx), px) * tall;
    }
    float2 now = float2(clamp(since / span, 0.0, 1.0), adsr(since, p));
    float d = length(q - (box + now * size));
    return heat + 0.9 * (1.0 - smoothstep(4.0 * px, 5.5 * px, d));
}

float4 shade(float2 uv, constant Uniforms& u, constant Params& p) {
    float t = u.time;
    float aspect = u.resolution.x / u.resolution.y;
    float2 q = float2(uv.x * aspect, uv.y);
    float px = 1.0 / u.resolution.y;
    float2 note = latestNote(t, p);
    float since = t - note.x;
    float heat = 0.03 + graticule(uv, q, px);

    // The sweep starts at the latest note's last whole cycle.
    float start = note.x + floor(since * note.y) / note.y;
    float off = traceDistance(q, start, aspect, px, p).x;
    float inside = within(uv.x, 0.04, 0.96);
    heat += (0.55 * lineAt(off, 2.0 * px) + 0.18 * exp(-off / (8.0 * px)))
            * inside;

    heat += envelopePanel(q, since, aspect, px, p);
    return float4(kiln(heat), 1.0);
}

// ---- Sound -----------------------------------------------------------------------------------------------------
// What you hear is synth(), the same samples the picture draws, at the volume the slider sets.

float2 sound(float t, constant Params& p) {
    return synth(t, p) * p.volume;
}
3 lines from stage 1 that are gone
    return float2(sin(TAU * p.pitch * t)) * 0.5 * fadeIn;
⋯
    float start = floor(t * p.pitch) / p.pitch;
⋯
    return synth(t, p) * 0.3 * p.volume;

Stage 3 · Frequency modulation

The voice, fmVoice, becomes sin(2π f t + I sin(2π · ratio · f t)): the modulator, at ratio times the note's frequency, pushes the note's phase by up to I radians. I is index at the strike, settling to 30% of it with exp(-6t), so each note starts bright and mellows, as a struck string does.

What that does to the spectrum is exact: the sound now has partials at the note's frequency plus and minus every whole multiple of the modulator's, and the one k steps away is as big as the Bessel function J_k(I). besselJ sums its series, and sidebands draws a bar for each k from −8 to 8 along the bottom, 0 to 4 kHz across, as tall as |J_k(I)| and as bright as the note is loud. A sideband that lands below zero folds back to the positive side, as it does in the sound. Watch a strike: the bars spread out, then pull in towards the note as the index settles. With ratio at a whole number the bars line up on harmonics and the scope's wave repeats; at 1.41 they fall between, the sound turns to a bell, and the trace never quite stands still.

stage-3-fm.metal42 new lines since stage 2
// Stage 3 · Frequency modulation
// The sine's phase is now pushed back and forth by a second sine, the modulator: FM, as John Chowning found it in
// 1973. Two sines make a whole set of partials, the sidebands, drawn along the bottom from Bessel functions.

// @param tempo 100 60 160 Beats a minute; the notes are eighths
// @param ratio 2 0.5 5 The modulator's frequency, as a multiple of the note's
// @param index 3 0 6 How hard the modulator bends the note: 0 is a pure sine, more is brighter
// @param attack 0.01 0.003 0.3 Seconds each note takes to rise
// @param decay 0.25 0.05 1 Seconds, roughly, to fall from the peak to the sustain
// @param sustain 0.4 0 1 The level a note holds while its key is down
// @param release 0.3 0.02 0.6 Seconds, roughly, a note takes to fade once let go
// @param volume 0.8 0 1 Loudness of the sound

constant float TAU = 6.2831853;

// The kiln palette from Flowing Color Field.
float3 kiln(float t) {
    float3 c0 = float3(0.020, 0.012, 0.020);
    float3 c1 = float3(0.280, 0.040, 0.030);
    float3 c2 = float3(0.880, 0.260, 0.050);
    float3 c3 = float3(1.000, 0.680, 0.200);
    float3 c4 = float3(1.000, 0.970, 0.840);
    t = clamp(t, 0.0, 1.0);
    float3 c = mix(c0, c1, smoothstep(0.00, 0.22, t));
    c = mix(c, c2, smoothstep(0.22, 0.50, t));
    c = mix(c, c3, smoothstep(0.50, 0.76, t));
    return mix(c, c4, smoothstep(0.76, 1.00, t));
}

uint hashBits(uint x) {
    x ^= x >> 16;
    x *= 0x7feb352du;
    x ^= x >> 15;
    x *= 0x846ca68bu;
    x ^= x >> 16;
    return x;
}

// ---- The sequencer -------------------------------------------------------------------------------------------------
// Eighth notes, eight to a bar; each bar arpeggiates a chord of A minor pentatonic.

float noteLength(constant Params& p) {
    return 30.0 / max(p.tempo, 1.0);
}

// A minor pentatonic from A2: three octaves.
float scaleNote(int i) {
    const float notes[15] = {110.00, 130.81, 146.83, 164.81, 196.00,
                             220.00, 261.63, 293.66, 329.63, 392.00,
                             440.00, 523.25, 587.33, 659.25, 783.99};
    return notes[clamp(i, 0, 14)];
}

// Step n's pitch: the bar's chord, then the arpeggio's shape.
float noteFrequency(int n) {
    const int roots[4] = {5, 8, 7, 6};
    const int shape[8] = {0, 2, 4, 5, 7, 5, 4, 2};
    return scaleNote(roots[(n / 8) % 4] + shape[n % 8]);
}

// The first note of a bar always plays; about one in five others rests.
bool notePlays(int n) {
    if (n < 0) return false;
    if (n % 8 == 0) return true;
    return (hashBits(uint(n % 32) + 777u) & 255u) > 50u;
}

// ---- The envelope --------------------------------------------------------------------------------------------------
// Attack, decay, sustain, release. The key is held for 60% of a step (the gate): the level rises in a straight line,
// falls towards the sustain, and once let go falls from wherever it was towards silence.

float gateLength(constant Params& p) {
    return 0.6 * noteLength(p);
}

float heldLevel(float since, constant Params& p) {
    float a = max(p.attack, 0.003);
    if (since < a) return since / a;
    float fall = exp(-(since - a) / max(p.decay, 0.01));
    return mix(p.sustain, 1.0, fall);
}

float adsr(float since, constant Params& p) {
    if (since < 0.0) return 0.0;
    float g = gateLength(p);
    if (since < g) return heldLevel(since, p);
    return heldLevel(g, p) * exp(-(since - g) / max(p.release, 0.01));
}

// ---- The voice -----------------------------------------------------------------------------------------------------
// Frequency modulation (John Chowning, 1973): a sine whose phase is pushed back and forth by a second sine, the
// modulator, at `ratio` times its frequency, by up to `index` radians: strongest at the strike, then 30%.

float modIndex(float since, constant Params& p) {
    return p.index * (0.3 + 0.7 * exp(-since * 6.0));
}

float fmVoice(float f, float since, constant Params& p) {
    float phase = f * since;
    float bend = modIndex(since, p) * sin(TAU * p.ratio * phase);
    return sin(TAU * phase + bend) * adsr(since, p);
}

// ---- The mix -------------------------------------------------------------------------------------------------
// Every note of the last twelve steps, summed; the oldest let go over three steps, so dropping it never clicks.

float2 synth(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    float sum = 0.0;
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (!notePlays(n)) continue;
        float since = t - float(n) * len;
        float held = 1.0 - smoothstep(9.0, 12.0, since / len);
        sum += held * fmVoice(noteFrequency(n), since, p);
    }
    float fadeIn = smoothstep(0.0, 1.5, t);
    return float2(tanh(sum * 0.25)) * fadeIn;
}

// ---- The picture ---------------------------------------------------------------------------------------------------

// The Bessel function J_n(x), from its series: the sum over m of (-1)^m (x/2)^(2m+n) / (m! (m+n)!). The sideband
// k modulator steps from the note is |J_k(index)| big.
float besselJ(int n, float x) {
    float h = 0.5 * x;
    float term = 1.0;
    for (int i = 1; i <= n; i++) term *= h / float(i);
    float sum = term;
    for (int m = 1; m < 16; m++) {
        term *= -h * h / (float(m) * float(m + n));
        sum += term;
    }
    return sum;
}

// The latest note to have started: its start time and frequency.
float2 latestNote(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (notePlays(n)) return float2(float(n) * len, noteFrequency(n));
    }
    return float2(0.0, noteFrequency(0));
}

// Distance from point a to the segment from b to c.
float segment(float2 a, float2 b, float2 c) {
    float2 ba = a - b, cb = c - b;
    float h = clamp(dot(ba, cb) / dot(cb, cb), 0.0, 1.0);
    return length(ba - cb * h);
}

// 1 on a line d away, fading out over w; and 1 when v is between a and b.
float lineAt(float d, float w) {
    return 1.0 - smoothstep(0.0, w, d);
}

float within(float v, float a, float b) {
    return step(a, v) * step(v, b);
}

// The scope's graticule: faint lines every tenth of the height, the middle
// one brighter.
float graticule(float2 uv, float2 q, float px) {
    float2 cell = abs(fract(q / 0.1 + 0.5) - 0.5) * 0.1;
    float grid = lineAt(min(cell.x, cell.y), 1.5 * px);
    float heat = 0.05 * grid * within(uv.y, 0.2, 0.8);
    return heat + 0.06 * lineAt(abs(uv.y - 0.5), 1.5 * px);
}

// How far q is from each trace, left and right: 9 ms of synth() from
// `start`, across the width. Seven samples 3 pixels apart, joined by lines.
float2 traceDistance(float2 q, float start, float aspect, float px,
                     constant Params& p) {
    float2 off = float2(1.0);
    float2 last = float2(0.0);
    for (int i = -3; i <= 3; i++) {
        float x = q.x + float(i) * 3.0 * px;
        float share = clamp((x / aspect - 0.04) / 0.92, 0.0, 1.0);
        float2 s = synth(start + share * 0.009, p) * 0.6 + 0.5;
        if (i > -3) {
            float x0 = x - 3.0 * px;
            off.x = min(off.x, segment(q, float2(x0, last.x), float2(x, s.x)));
            off.y = min(off.y, segment(q, float2(x0, last.y), float2(x, s.y)));
        }
        last = s;
    }
    return off;
}

// The envelope, top left: the latest note's shape from strike to silence,
// the floor, a mark where the key is let go, and a dot where it is now.
float envelopePanel(float2 q, float since, float aspect, float px,
                    constant Params& p) {
    float2 box = float2(0.04 * aspect, 0.84);
    float2 size = float2(0.36, 0.11);
    float span = gateLength(p) + 3.0 * p.release;
    float ex = (q.x - box.x) / size.x;
    float on = within(ex, 0.0, 1.0);
    float heat = 0.0;
    if (q.y > box.y - 0.02 && q.y < box.y + size.y + 0.02) {
        float ea = clamp(ex, 0.0, 1.0), eb = ea + px / size.x;
        float2 a = box + float2(ea, adsr(ea * span, p)) * size;
        float2 b = box + float2(eb, adsr(eb * span, p)) * size;
        heat += 0.5 * lineAt(segment(q, a, b), 1.5 * px) * on;
        heat += 0.08 * lineAt(abs(q.y - box.y), px) * on;
        float gx = box.x + gateLength(p) / span * size.x;
        float tall = within(q.y, box.y, box.y + size.y);
        heat += 0.1 * lineAt(abs(q.x - gx), px) * tall;
    }
    float2 now = float2(clamp(since / span, 0.0, 1.0), adsr(since, p));
    float d = length(q - (box + now * size));
    return heat + 0.9 * (1.0 - smoothstep(4.0 * px, 5.5 * px, d));
}

// The sidebands, along the bottom: energy at the note's frequency plus and
// minus whole multiples of the modulator's, each |J_k(index)| tall, as
// bright as the note is loud. 0 to 4 kHz across; below zero folds back.
float sidebands(float2 uv, float2 note, float since, float aspect, float px,
                constant Params& p) {
    float floorY = 0.04, tall = 0.18;
    if (uv.y < floorY - 0.01 || uv.y > floorY + tall + 0.01) return 0.0;
    if (uv.x < 0.03 || uv.x > 0.97) return 0.0;
    float hz = (uv.x - 0.04) / 0.92 * 4000.0;
    float I = modIndex(since, p);
    float level = adsr(since, p);
    float bars = 0.0;
    for (int k = -8; k <= 8; k++) {
        float f = abs(note.y + float(k) * p.ratio * note.y);
        float w = abs(hz - f) / 4000.0 * 0.92 * aspect;
        if (w > 3.0 * px) continue;
        float a = min(abs(besselJ(abs(k), I)), 1.0);
        float bar = 1.0 - smoothstep(px, 2.5 * px, w);
        bar *= within(uv.y, floorY, floorY + tall * a);
        float bright = 0.3 + 0.5 * level + 0.15 * float(k == 0);
        bars = max(bars, bar * bright);
    }
    float axis = lineAt(abs(uv.y - floorY), px) * within(uv.x, 0.04, 0.96);
    return bars + 0.08 * axis;
}

float4 shade(float2 uv, constant Uniforms& u, constant Params& p) {
    float t = u.time;
    float aspect = u.resolution.x / u.resolution.y;
    float2 q = float2(uv.x * aspect, uv.y);
    float px = 1.0 / u.resolution.y;
    float2 note = latestNote(t, p);
    float since = t - note.x;
    float heat = 0.03 + graticule(uv, q, px);

    // The sweep starts at the latest note's last whole cycle.
    float start = note.x + floor(since * note.y) / note.y;
    float off = traceDistance(q, start, aspect, px, p).x;
    float inside = within(uv.x, 0.04, 0.96);
    heat += (0.55 * lineAt(off, 2.0 * px) + 0.18 * exp(-off / (8.0 * px)))
            * inside;

    heat += envelopePanel(q, since, aspect, px, p);
    heat += sidebands(uv, note, since, aspect, px, p);
    return float4(kiln(heat), 1.0);
}

// ---- Sound -----------------------------------------------------------------------------------------------------
// What you hear is synth(), the same samples the picture draws, at the volume the slider sets.

float2 sound(float t, constant Params& p) {
    return synth(t, p) * p.volume;
}
3 lines from stage 2 that are gone
float sineVoice(float f, float since, constant Params& p) {
    return sin(TAU * f * since) * adsr(since, p);
⋯
        sum += held * sineVoice(noteFrequency(n), since, p);

Stage 4 · Pulse Synth

The finished shader is stereo. The right side plays every note detune × 8 cents sharp (2 cents at the default), so the two sides drift in and out of step, the slow shimmer of a chorus. traceDistance already measured both: now the right trace is drawn in ember under the left in gold, and where they agree they add up to white. Since the trigger follows the left side, the right trace slides along under it as the note goes on, further for older notes. A soft haze round the middle line warms with the note.

The picture draws synth, the sound before its volume, so the volume slider changes what you hear and not the picture.

the kiln palette and shade(), from the finished shader
// The kiln palette from Flowing Color Field.
float3 kiln(float t) {
    float3 c0 = float3(0.020, 0.012, 0.020);
    float3 c1 = float3(0.280, 0.040, 0.030);
    float3 c2 = float3(0.880, 0.260, 0.050);
    float3 c3 = float3(1.000, 0.680, 0.200);
    float3 c4 = float3(1.000, 0.970, 0.840);
    t = clamp(t, 0.0, 1.0);
    float3 c = mix(c0, c1, smoothstep(0.00, 0.22, t));
    c = mix(c, c2, smoothstep(0.22, 0.50, t));
    c = mix(c, c3, smoothstep(0.50, 0.76, t));
    return mix(c, c4, smoothstep(0.76, 1.00, t));
}

float4 shade(float2 uv, constant Uniforms& u, constant Params& p) {
    float t = u.time;
    float aspect = u.resolution.x / u.resolution.y;
    float2 q = float2(uv.x * aspect, uv.y);
    float px = 1.0 / u.resolution.y;
    float2 note = latestNote(t, p);
    float since = t - note.x;
    float heat = 0.03 + graticule(uv, q, px);

    // The traces start where the latest note's wave last began a cycle, so
    // a steady wave stands still, as a scope's trigger holds it.
    float start = note.x + floor(since * note.y) / note.y;
    float2 off = traceDistance(q, start, aspect, px, p);
    float inside = within(uv.x, 0.04, 0.96);
    float2 core = (1.0 - smoothstep(0.0, 2.0 * px, off)) * inside;
    float2 glow = exp(-off / (8.0 * px)) * inside;
    // Right in ember, left over it in gold: where they agree, they burn white.
    heat += 0.45 * core.y + 0.14 * glow.y;
    heat += 0.55 * core.x + 0.18 * glow.x;

    heat += envelopePanel(q, since, aspect, px, p);
    heat += sidebands(uv, note, since, aspect, px, p);

    // The whole picture warms a little with the note.
    float level = adsr(since, p);
    heat += 0.06 * level * exp(-abs(uv.y - 0.5) * 4.0);
    return float4(kiln(heat), 1.0);
}
pulse-synth.metal, the whole file13 new lines since stage 3
// Ray Kiln · Pulse Synth
//
// @episode Pulse Synth
// @length 60
// @variation 30
// @variations 2,3,4,7,8,9,10,11
// @still 9.64
// @teaches FM synthesis and envelopes, a picture drawn from its own waveform
// @category Fragment
// @tags FM synthesis, envelope, oscilloscope, Bessel functions, procedural sound
// @sweeps index, ratio, detune
// @final Pulse Synth | Two channels a few cents apart, drawn as two glowing traces that slide past each other, over the envelope and the sidebands.
//
// A small synthesizer and the oscilloscope watching it. The sound plays an arpeggio, every note one sine wave whose
// phase is bent by another (frequency modulation), its loudness shaped by an envelope. The picture calls the sound's
// own function: the traces across the middle are the samples you hear, left and right, a few milliseconds of them.
// Above them is the note's envelope, and below them the sidebands the bending makes, from the same numbers.
// Each @param line below becomes a slider in the host app and a row in the page's parameters table:
//   // @param name default min max description

// @param tempo 100 60 160 Beats a minute; the notes are eighths
// @param ratio 2 0.5 5 The modulator's frequency, as a multiple of the note's
// @param index 3 0 6 How hard the modulator bends the note: 0 is a pure sine, more is brighter
// @param attack 0.01 0.003 0.3 Seconds each note takes to rise
// @param decay 0.25 0.05 1 Seconds, roughly, to fall from the peak to the sustain
// @param sustain 0.4 0 1 The level a note holds while its key is down
// @param release 0.3 0.02 0.6 Seconds, roughly, a note takes to fade once let go
// @param detune 0.25 0 1 How far apart left and right are, up to 8 cents
// @param volume 0.8 0 1 Loudness of the sound

constant float TAU = 6.2831853;

// The kiln palette from Flowing Color Field.
float3 kiln(float t) {
    float3 c0 = float3(0.020, 0.012, 0.020);
    float3 c1 = float3(0.280, 0.040, 0.030);
    float3 c2 = float3(0.880, 0.260, 0.050);
    float3 c3 = float3(1.000, 0.680, 0.200);
    float3 c4 = float3(1.000, 0.970, 0.840);
    t = clamp(t, 0.0, 1.0);
    float3 c = mix(c0, c1, smoothstep(0.00, 0.22, t));
    c = mix(c, c2, smoothstep(0.22, 0.50, t));
    c = mix(c, c3, smoothstep(0.50, 0.76, t));
    return mix(c, c4, smoothstep(0.76, 1.00, t));
}

uint hashBits(uint x) {
    x ^= x >> 16;
    x *= 0x7feb352du;
    x ^= x >> 15;
    x *= 0x846ca68bu;
    x ^= x >> 16;
    return x;
}

// ---- The sequencer -------------------------------------------------------------------------------------------------
// Eighth notes, eight to a bar; each bar arpeggiates a chord of A minor pentatonic.

float noteLength(constant Params& p) {
    return 30.0 / max(p.tempo, 1.0);
}

// A minor pentatonic from A2: three octaves.
float scaleNote(int i) {
    const float notes[15] = {110.00, 130.81, 146.83, 164.81, 196.00,
                             220.00, 261.63, 293.66, 329.63, 392.00,
                             440.00, 523.25, 587.33, 659.25, 783.99};
    return notes[clamp(i, 0, 14)];
}

// Step n's pitch: the bar's chord, then the arpeggio's shape.
float noteFrequency(int n) {
    const int roots[4] = {5, 8, 7, 6};
    const int shape[8] = {0, 2, 4, 5, 7, 5, 4, 2};
    return scaleNote(roots[(n / 8) % 4] + shape[n % 8]);
}

// The first note of a bar always plays; about one in five others rests.
bool notePlays(int n) {
    if (n < 0) return false;
    if (n % 8 == 0) return true;
    return (hashBits(uint(n % 32) + 777u) & 255u) > 50u;
}

// ---- The envelope --------------------------------------------------------------------------------------------------
// Attack, decay, sustain, release. The key is held for 60% of a step (the gate): the level rises in a straight line,
// falls towards the sustain, and once let go falls from wherever it was towards silence.

float gateLength(constant Params& p) {
    return 0.6 * noteLength(p);
}

float heldLevel(float since, constant Params& p) {
    float a = max(p.attack, 0.003);
    if (since < a) return since / a;
    float fall = exp(-(since - a) / max(p.decay, 0.01));
    return mix(p.sustain, 1.0, fall);
}

float adsr(float since, constant Params& p) {
    if (since < 0.0) return 0.0;
    float g = gateLength(p);
    if (since < g) return heldLevel(since, p);
    return heldLevel(g, p) * exp(-(since - g) / max(p.release, 0.01));
}

// ---- The voice -----------------------------------------------------------------------------------------------------
// Frequency modulation (John Chowning, 1973): a sine whose phase is pushed back and forth by a second sine, the
// modulator, at `ratio` times its frequency, by up to `index` radians: strongest at the strike, then 30%.

float modIndex(float since, constant Params& p) {
    return p.index * (0.3 + 0.7 * exp(-since * 6.0));
}

float fmVoice(float f, float since, constant Params& p) {
    float phase = f * since;
    float bend = modIndex(since, p) * sin(TAU * p.ratio * phase);
    return sin(TAU * phase + bend) * adsr(since, p);
}

// ---- The mix -------------------------------------------------------------------------------------------------
// Every note of the last twelve steps, summed; the oldest let go over three steps, so it never clicks. The right
// side plays a few cents sharp, so the two drift in and out of step: a chorus. synth() leaves the volume out.

float2 synth(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    // The right side plays each note a few cents sharp.
    float sharp = exp2(p.detune * 8.0 / 1200.0);
    float2 sum = float2(0.0);
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (!notePlays(n)) continue;
        float since = t - float(n) * len;
        float held = 1.0 - smoothstep(9.0, 12.0, since / len);
        float f = noteFrequency(n);
        float2 pair = float2(fmVoice(f, since, p), fmVoice(f * sharp, since, p));
        sum += held * pair;
    }
    float fadeIn = smoothstep(0.0, 1.5, t);
    return tanh(sum * 0.25) * fadeIn;
}

// ---- The picture ---------------------------------------------------------------------------------------------------

// The Bessel function J_n(x), from its series: the sum over m of (-1)^m (x/2)^(2m+n) / (m! (m+n)!). The sideband
// k modulator steps from the note is |J_k(index)| big.
float besselJ(int n, float x) {
    float h = 0.5 * x;
    float term = 1.0;
    for (int i = 1; i <= n; i++) term *= h / float(i);
    float sum = term;
    for (int m = 1; m < 16; m++) {
        term *= -h * h / (float(m) * float(m + n));
        sum += term;
    }
    return sum;
}

// The latest note to have started: its start time and frequency.
float2 latestNote(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (notePlays(n)) return float2(float(n) * len, noteFrequency(n));
    }
    return float2(0.0, noteFrequency(0));
}

// Distance from point a to the segment from b to c.
float segment(float2 a, float2 b, float2 c) {
    float2 ba = a - b, cb = c - b;
    float h = clamp(dot(ba, cb) / dot(cb, cb), 0.0, 1.0);
    return length(ba - cb * h);
}

// 1 on a line d away, fading out over w; and 1 when v is between a and b.
float lineAt(float d, float w) {
    return 1.0 - smoothstep(0.0, w, d);
}

float within(float v, float a, float b) {
    return step(a, v) * step(v, b);
}

// The scope's graticule: faint lines every tenth of the height, the middle
// one brighter.
float graticule(float2 uv, float2 q, float px) {
    float2 cell = abs(fract(q / 0.1 + 0.5) - 0.5) * 0.1;
    float grid = lineAt(min(cell.x, cell.y), 1.5 * px);
    float heat = 0.05 * grid * within(uv.y, 0.2, 0.8);
    return heat + 0.06 * lineAt(abs(uv.y - 0.5), 1.5 * px);
}

// How far q is from each trace, left and right: 9 ms of synth() from
// `start`, across the width. Seven samples 3 pixels apart, joined by lines.
float2 traceDistance(float2 q, float start, float aspect, float px,
                     constant Params& p) {
    float2 off = float2(1.0);
    float2 last = float2(0.0);
    for (int i = -3; i <= 3; i++) {
        float x = q.x + float(i) * 3.0 * px;
        float share = clamp((x / aspect - 0.04) / 0.92, 0.0, 1.0);
        float2 s = synth(start + share * 0.009, p) * 0.6 + 0.5;
        if (i > -3) {
            float x0 = x - 3.0 * px;
            off.x = min(off.x, segment(q, float2(x0, last.x), float2(x, s.x)));
            off.y = min(off.y, segment(q, float2(x0, last.y), float2(x, s.y)));
        }
        last = s;
    }
    return off;
}

// The envelope, top left: the latest note's shape from strike to silence,
// the floor, a mark where the key is let go, and a dot where it is now.
float envelopePanel(float2 q, float since, float aspect, float px,
                    constant Params& p) {
    float2 box = float2(0.04 * aspect, 0.84);
    float2 size = float2(0.36, 0.11);
    float span = gateLength(p) + 3.0 * p.release;
    float ex = (q.x - box.x) / size.x;
    float on = within(ex, 0.0, 1.0);
    float heat = 0.0;
    if (q.y > box.y - 0.02 && q.y < box.y + size.y + 0.02) {
        float ea = clamp(ex, 0.0, 1.0), eb = ea + px / size.x;
        float2 a = box + float2(ea, adsr(ea * span, p)) * size;
        float2 b = box + float2(eb, adsr(eb * span, p)) * size;
        heat += 0.5 * lineAt(segment(q, a, b), 1.5 * px) * on;
        heat += 0.08 * lineAt(abs(q.y - box.y), px) * on;
        float gx = box.x + gateLength(p) / span * size.x;
        float tall = within(q.y, box.y, box.y + size.y);
        heat += 0.1 * lineAt(abs(q.x - gx), px) * tall;
    }
    float2 now = float2(clamp(since / span, 0.0, 1.0), adsr(since, p));
    float d = length(q - (box + now * size));
    return heat + 0.9 * (1.0 - smoothstep(4.0 * px, 5.5 * px, d));
}

// The sidebands, along the bottom: energy at the note's frequency plus and
// minus whole multiples of the modulator's, each |J_k(index)| tall, as
// bright as the note is loud. 0 to 4 kHz across; below zero folds back.
float sidebands(float2 uv, float2 note, float since, float aspect, float px,
                constant Params& p) {
    float floorY = 0.04, tall = 0.18;
    if (uv.y < floorY - 0.01 || uv.y > floorY + tall + 0.01) return 0.0;
    if (uv.x < 0.03 || uv.x > 0.97) return 0.0;
    float hz = (uv.x - 0.04) / 0.92 * 4000.0;
    float I = modIndex(since, p);
    float level = adsr(since, p);
    float bars = 0.0;
    for (int k = -8; k <= 8; k++) {
        float f = abs(note.y + float(k) * p.ratio * note.y);
        float w = abs(hz - f) / 4000.0 * 0.92 * aspect;
        if (w > 3.0 * px) continue;
        float a = min(abs(besselJ(abs(k), I)), 1.0);
        float bar = 1.0 - smoothstep(px, 2.5 * px, w);
        bar *= within(uv.y, floorY, floorY + tall * a);
        float bright = 0.3 + 0.5 * level + 0.15 * float(k == 0);
        bars = max(bars, bar * bright);
    }
    float axis = lineAt(abs(uv.y - floorY), px) * within(uv.x, 0.04, 0.96);
    return bars + 0.08 * axis;
}

float4 shade(float2 uv, constant Uniforms& u, constant Params& p) {
    float t = u.time;
    float aspect = u.resolution.x / u.resolution.y;
    float2 q = float2(uv.x * aspect, uv.y);
    float px = 1.0 / u.resolution.y;
    float2 note = latestNote(t, p);
    float since = t - note.x;
    float heat = 0.03 + graticule(uv, q, px);

    // The traces start where the latest note's wave last began a cycle, so
    // a steady wave stands still, as a scope's trigger holds it.
    float start = note.x + floor(since * note.y) / note.y;
    float2 off = traceDistance(q, start, aspect, px, p);
    float inside = within(uv.x, 0.04, 0.96);
    float2 core = (1.0 - smoothstep(0.0, 2.0 * px, off)) * inside;
    float2 glow = exp(-off / (8.0 * px)) * inside;
    // Right in ember, left over it in gold: where they agree, they burn white.
    heat += 0.45 * core.y + 0.14 * glow.y;
    heat += 0.55 * core.x + 0.18 * glow.x;

    heat += envelopePanel(q, since, aspect, px, p);
    heat += sidebands(uv, note, since, aspect, px, p);

    // The whole picture warms a little with the note.
    float level = adsr(since, p);
    heat += 0.06 * level * exp(-abs(uv.y - 0.5) * 4.0);
    return float4(kiln(heat), 1.0);
}

// ---- Sound -----------------------------------------------------------------------------------------------------
// What you hear is synth(), the same samples the picture draws, at the volume the slider sets.

float2 sound(float t, constant Params& p) {
    return synth(t, p) * p.volume;
}
6 lines from stage 3 that are gone
    float sum = 0.0;
⋯
        sum += held * fmVoice(noteFrequency(n), since, p);
⋯
    return float2(tanh(sum * 0.25)) * fadeIn;
⋯
    float off = traceDistance(q, start, aspect, px, p).x;
⋯
    heat += (0.55 * lineAt(off, 2.0 * px) + 0.18 * exp(-off / (8.0 * px)))
            * inside;

The sound

The sound is the arpeggio of the stages, through the FM voice and the envelope, the right side a little sharp. At the defaults (100 beats a minute, ratio 2, index 3) its partials sit on odd harmonics of each note at the strike, which gives the hollow, reedy tone of a clarinet, and soften towards a plain sine as the index settles. It fades in over the first second and a half. At the defaults the peak is about −9.8 dBFS and the average (RMS) about −19.4 dBFS; at the extremes of the sliders the peak stays between −8 and −10 dBFS. The 60-second video ends mid-phrase, at a sample of about 0.024 (−32 dBFS).

New in Ray Kiln

Save the sound. The playground's Export now offers a third choice beside a picture and a video: The sound alone (WAV). It renders the shader's sound on your computer from the time on the clock, with the sliders as they are, and downloads it as a 16-bit, 48 kHz stereo WAV file that any audio editor opens. Unless you set Seconds, it holds the clock's range: from the clock to the end of the scrubber, the shader's @length, up to ten minutes. The samples are the very ones an MP4 export encodes as AAC, and when it is done the dialog gives their peak and RMS in dBFS, measured as raykiln sound measures them. It is free, like the playground's other exports, and needs no audio encoder, so it works in a browser that can't put sound in a video.

Open Pulse Synth in the playground, press Export, pick The sound alone (WAV) and Download: you get the arpeggio as a file to drop into a track, and the levels the dialog reports are the ones under The sound above.

Try this

  • Set index to 0: the bending stops, every sideband but the note itself vanishes, and the trace is a plain sine.
  • Set ratio to 1.41: the sidebands fall between the harmonics, the notes ring like bells, and the trace never stands still.
  • Set sustain to 1 and release to 0.6: the notes hold and overlap, and the scope shows them adding into one wave.
  • Set attack to 0.3: each note swells in, longer than the key is held, so it never reaches full level.
  • Set detune to 1: the two traces part faster, and the chorus is wider.
  • In modIndex, change 0.3 + 0.7 * exp(-since * 6.0) to 1.0: the strike's brightness never fades, and the sidebands stay spread.
  • Export the sound as a WAV twice, index at 0 and at 6, and compare them in an audio editor's spectrum view with the bars along the bottom.

Parameters

ParameterDefaultRangeWhat it does
tempo10060 to 160Beats a minute; the notes are eighths
ratio20.5 to 5The modulator's frequency, as a multiple of the note's
index30 to 6How hard the modulator bends the note: 0 is a pure sine, more is brighter
attack0.010.003 to 0.3Seconds each note takes to rise
decay0.250.05 to 1Seconds, roughly, to fall from the peak to the sustain
sustain0.40 to 1The level a note holds while its key is down
release0.30.02 to 0.6Seconds, roughly, a note takes to fade once let go
detune0.250 to 1How far apart left and right are, up to 8 cents
volume0.80 to 1Loudness of the sound

More from this shader

The same shader 8 more ways: each from a different moment, with different settings, one after another. Each chapter below says which settings moved most, then lists them all.

Wallpapers

A still from the shader, rendered at full size for a desktop or a phone screen. Free to use as your wallpaper.

The desktop wallpaper of Pulse Synth Desktop3840 × 2160 · 0.6 MB JPEG The phone wallpaper of Pulse Synth Phone1320 × 2868 · 0.4 MB JPEG

Clips to post

Fifteen seconds of the shader with its sound, sized for posting: square for feeds, vertical for stories and short videos. Free to post, ideally with a link to this page.

SquareFor feeds · 1080 × 1080 · 15 MB MP4VerticalFor stories and reels · 1080 × 1920 · 15 MB MP4

The full source

This is the whole episode: the picture, the parameters and the sound.

Download the shader Download the standalone versionOpen in Ray KilnCheck in the linter
pulse-synth.metal
// Ray Kiln · Pulse Synth
//
// @episode Pulse Synth
// @length 60
// @variation 30
// @variations 2,3,4,7,8,9,10,11
// @still 9.64
// @teaches FM synthesis and envelopes, a picture drawn from its own waveform
// @category Fragment
// @tags FM synthesis, envelope, oscilloscope, Bessel functions, procedural sound
// @sweeps index, ratio, detune
// @final Pulse Synth | Two channels a few cents apart, drawn as two glowing traces that slide past each other, over the envelope and the sidebands.
//
// A small synthesizer and the oscilloscope watching it. The sound plays an arpeggio, every note one sine wave whose
// phase is bent by another (frequency modulation), its loudness shaped by an envelope. The picture calls the sound's
// own function: the traces across the middle are the samples you hear, left and right, a few milliseconds of them.
// Above them is the note's envelope, and below them the sidebands the bending makes, from the same numbers.
// Each @param line below becomes a slider in the host app and a row in the page's parameters table:
//   // @param name default min max description

// @param tempo 100 60 160 Beats a minute; the notes are eighths
// @param ratio 2 0.5 5 The modulator's frequency, as a multiple of the note's
// @param index 3 0 6 How hard the modulator bends the note: 0 is a pure sine, more is brighter
// @param attack 0.01 0.003 0.3 Seconds each note takes to rise
// @param decay 0.25 0.05 1 Seconds, roughly, to fall from the peak to the sustain
// @param sustain 0.4 0 1 The level a note holds while its key is down
// @param release 0.3 0.02 0.6 Seconds, roughly, a note takes to fade once let go
// @param detune 0.25 0 1 How far apart left and right are, up to 8 cents
// @param volume 0.8 0 1 Loudness of the sound

constant float TAU = 6.2831853;

// The kiln palette from Flowing Color Field.
float3 kiln(float t) {
    float3 c0 = float3(0.020, 0.012, 0.020);
    float3 c1 = float3(0.280, 0.040, 0.030);
    float3 c2 = float3(0.880, 0.260, 0.050);
    float3 c3 = float3(1.000, 0.680, 0.200);
    float3 c4 = float3(1.000, 0.970, 0.840);
    t = clamp(t, 0.0, 1.0);
    float3 c = mix(c0, c1, smoothstep(0.00, 0.22, t));
    c = mix(c, c2, smoothstep(0.22, 0.50, t));
    c = mix(c, c3, smoothstep(0.50, 0.76, t));
    return mix(c, c4, smoothstep(0.76, 1.00, t));
}

uint hashBits(uint x) {
    x ^= x >> 16;
    x *= 0x7feb352du;
    x ^= x >> 15;
    x *= 0x846ca68bu;
    x ^= x >> 16;
    return x;
}

// ---- The sequencer -------------------------------------------------------------------------------------------------
// Eighth notes, eight to a bar; each bar arpeggiates a chord of A minor pentatonic.

float noteLength(constant Params& p) {
    return 30.0 / max(p.tempo, 1.0);
}

// A minor pentatonic from A2: three octaves.
float scaleNote(int i) {
    const float notes[15] = {110.00, 130.81, 146.83, 164.81, 196.00,
                             220.00, 261.63, 293.66, 329.63, 392.00,
                             440.00, 523.25, 587.33, 659.25, 783.99};
    return notes[clamp(i, 0, 14)];
}

// Step n's pitch: the bar's chord, then the arpeggio's shape.
float noteFrequency(int n) {
    const int roots[4] = {5, 8, 7, 6};
    const int shape[8] = {0, 2, 4, 5, 7, 5, 4, 2};
    return scaleNote(roots[(n / 8) % 4] + shape[n % 8]);
}

// The first note of a bar always plays; about one in five others rests.
bool notePlays(int n) {
    if (n < 0) return false;
    if (n % 8 == 0) return true;
    return (hashBits(uint(n % 32) + 777u) & 255u) > 50u;
}

// ---- The envelope --------------------------------------------------------------------------------------------------
// Attack, decay, sustain, release. The key is held for 60% of a step (the gate): the level rises in a straight line,
// falls towards the sustain, and once let go falls from wherever it was towards silence.

float gateLength(constant Params& p) {
    return 0.6 * noteLength(p);
}

float heldLevel(float since, constant Params& p) {
    float a = max(p.attack, 0.003);
    if (since < a) return since / a;
    float fall = exp(-(since - a) / max(p.decay, 0.01));
    return mix(p.sustain, 1.0, fall);
}

float adsr(float since, constant Params& p) {
    if (since < 0.0) return 0.0;
    float g = gateLength(p);
    if (since < g) return heldLevel(since, p);
    return heldLevel(g, p) * exp(-(since - g) / max(p.release, 0.01));
}

// ---- The voice -----------------------------------------------------------------------------------------------------
// Frequency modulation (John Chowning, 1973): a sine whose phase is pushed back and forth by a second sine, the
// modulator, at `ratio` times its frequency, by up to `index` radians: strongest at the strike, then 30%.

float modIndex(float since, constant Params& p) {
    return p.index * (0.3 + 0.7 * exp(-since * 6.0));
}

float fmVoice(float f, float since, constant Params& p) {
    float phase = f * since;
    float bend = modIndex(since, p) * sin(TAU * p.ratio * phase);
    return sin(TAU * phase + bend) * adsr(since, p);
}

// ---- The mix -------------------------------------------------------------------------------------------------
// Every note of the last twelve steps, summed; the oldest let go over three steps, so it never clicks. The right
// side plays a few cents sharp, so the two drift in and out of step: a chorus. synth() leaves the volume out.

float2 synth(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    // The right side plays each note a few cents sharp.
    float sharp = exp2(p.detune * 8.0 / 1200.0);
    float2 sum = float2(0.0);
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (!notePlays(n)) continue;
        float since = t - float(n) * len;
        float held = 1.0 - smoothstep(9.0, 12.0, since / len);
        float f = noteFrequency(n);
        float2 pair = float2(fmVoice(f, since, p), fmVoice(f * sharp, since, p));
        sum += held * pair;
    }
    float fadeIn = smoothstep(0.0, 1.5, t);
    return tanh(sum * 0.25) * fadeIn;
}

// ---- The picture ---------------------------------------------------------------------------------------------------

// The Bessel function J_n(x), from its series: the sum over m of (-1)^m (x/2)^(2m+n) / (m! (m+n)!). The sideband
// k modulator steps from the note is |J_k(index)| big.
float besselJ(int n, float x) {
    float h = 0.5 * x;
    float term = 1.0;
    for (int i = 1; i <= n; i++) term *= h / float(i);
    float sum = term;
    for (int m = 1; m < 16; m++) {
        term *= -h * h / (float(m) * float(m + n));
        sum += term;
    }
    return sum;
}

// The latest note to have started: its start time and frequency.
float2 latestNote(float t, constant Params& p) {
    float len = noteLength(p);
    int now = int(floor(t / len));
    for (int back = 0; back < 12; back++) {
        int n = now - back;
        if (n < 0) break;
        if (notePlays(n)) return float2(float(n) * len, noteFrequency(n));
    }
    return float2(0.0, noteFrequency(0));
}

// Distance from point a to the segment from b to c.
float segment(float2 a, float2 b, float2 c) {
    float2 ba = a - b, cb = c - b;
    float h = clamp(dot(ba, cb) / dot(cb, cb), 0.0, 1.0);
    return length(ba - cb * h);
}

// 1 on a line d away, fading out over w; and 1 when v is between a and b.
float lineAt(float d, float w) {
    return 1.0 - smoothstep(0.0, w, d);
}

float within(float v, float a, float b) {
    return step(a, v) * step(v, b);
}

// The scope's graticule: faint lines every tenth of the height, the middle
// one brighter.
float graticule(float2 uv, float2 q, float px) {
    float2 cell = abs(fract(q / 0.1 + 0.5) - 0.5) * 0.1;
    float grid = lineAt(min(cell.x, cell.y), 1.5 * px);
    float heat = 0.05 * grid * within(uv.y, 0.2, 0.8);
    return heat + 0.06 * lineAt(abs(uv.y - 0.5), 1.5 * px);
}

// How far q is from each trace, left and right: 9 ms of synth() from
// `start`, across the width. Seven samples 3 pixels apart, joined by lines.
float2 traceDistance(float2 q, float start, float aspect, float px,
                     constant Params& p) {
    float2 off = float2(1.0);
    float2 last = float2(0.0);
    for (int i = -3; i <= 3; i++) {
        float x = q.x + float(i) * 3.0 * px;
        float share = clamp((x / aspect - 0.04) / 0.92, 0.0, 1.0);
        float2 s = synth(start + share * 0.009, p) * 0.6 + 0.5;
        if (i > -3) {
            float x0 = x - 3.0 * px;
            off.x = min(off.x, segment(q, float2(x0, last.x), float2(x, s.x)));
            off.y = min(off.y, segment(q, float2(x0, last.y), float2(x, s.y)));
        }
        last = s;
    }
    return off;
}

// The envelope, top left: the latest note's shape from strike to silence,
// the floor, a mark where the key is let go, and a dot where it is now.
float envelopePanel(float2 q, float since, float aspect, float px,
                    constant Params& p) {
    float2 box = float2(0.04 * aspect, 0.84);
    float2 size = float2(0.36, 0.11);
    float span = gateLength(p) + 3.0 * p.release;
    float ex = (q.x - box.x) / size.x;
    float on = within(ex, 0.0, 1.0);
    float heat = 0.0;
    if (q.y > box.y - 0.02 && q.y < box.y + size.y + 0.02) {
        float ea = clamp(ex, 0.0, 1.0), eb = ea + px / size.x;
        float2 a = box + float2(ea, adsr(ea * span, p)) * size;
        float2 b = box + float2(eb, adsr(eb * span, p)) * size;
        heat += 0.5 * lineAt(segment(q, a, b), 1.5 * px) * on;
        heat += 0.08 * lineAt(abs(q.y - box.y), px) * on;
        float gx = box.x + gateLength(p) / span * size.x;
        float tall = within(q.y, box.y, box.y + size.y);
        heat += 0.1 * lineAt(abs(q.x - gx), px) * tall;
    }
    float2 now = float2(clamp(since / span, 0.0, 1.0), adsr(since, p));
    float d = length(q - (box + now * size));
    return heat + 0.9 * (1.0 - smoothstep(4.0 * px, 5.5 * px, d));
}

// The sidebands, along the bottom: energy at the note's frequency plus and
// minus whole multiples of the modulator's, each |J_k(index)| tall, as
// bright as the note is loud. 0 to 4 kHz across; below zero folds back.
float sidebands(float2 uv, float2 note, float since, float aspect, float px,
                constant Params& p) {
    float floorY = 0.04, tall = 0.18;
    if (uv.y < floorY - 0.01 || uv.y > floorY + tall + 0.01) return 0.0;
    if (uv.x < 0.03 || uv.x > 0.97) return 0.0;
    float hz = (uv.x - 0.04) / 0.92 * 4000.0;
    float I = modIndex(since, p);
    float level = adsr(since, p);
    float bars = 0.0;
    for (int k = -8; k <= 8; k++) {
        float f = abs(note.y + float(k) * p.ratio * note.y);
        float w = abs(hz - f) / 4000.0 * 0.92 * aspect;
        if (w > 3.0 * px) continue;
        float a = min(abs(besselJ(abs(k), I)), 1.0);
        float bar = 1.0 - smoothstep(px, 2.5 * px, w);
        bar *= within(uv.y, floorY, floorY + tall * a);
        float bright = 0.3 + 0.5 * level + 0.15 * float(k == 0);
        bars = max(bars, bar * bright);
    }
    float axis = lineAt(abs(uv.y - floorY), px) * within(uv.x, 0.04, 0.96);
    return bars + 0.08 * axis;
}

float4 shade(float2 uv, constant Uniforms& u, constant Params& p) {
    float t = u.time;
    float aspect = u.resolution.x / u.resolution.y;
    float2 q = float2(uv.x * aspect, uv.y);
    float px = 1.0 / u.resolution.y;
    float2 note = latestNote(t, p);
    float since = t - note.x;
    float heat = 0.03 + graticule(uv, q, px);

    // The traces start where the latest note's wave last began a cycle, so
    // a steady wave stands still, as a scope's trigger holds it.
    float start = note.x + floor(since * note.y) / note.y;
    float2 off = traceDistance(q, start, aspect, px, p);
    float inside = within(uv.x, 0.04, 0.96);
    float2 core = (1.0 - smoothstep(0.0, 2.0 * px, off)) * inside;
    float2 glow = exp(-off / (8.0 * px)) * inside;
    // Right in ember, left over it in gold: where they agree, they burn white.
    heat += 0.45 * core.y + 0.14 * glow.y;
    heat += 0.55 * core.x + 0.18 * glow.x;

    heat += envelopePanel(q, since, aspect, px, p);
    heat += sidebands(uv, note, since, aspect, px, p);

    // The whole picture warms a little with the note.
    float level = adsr(since, p);
    heat += 0.06 * level * exp(-abs(uv.y - 0.5) * 4.0);
    return float4(kiln(heat), 1.0);
}

// ---- Sound -----------------------------------------------------------------------------------------------------
// What you hear is synth(), the same samples the picture draws, at the volume the slider sets.

float2 sound(float t, constant Params& p) {
    return synth(t, p) * p.volume;
}

Running it in your own project

Every Ray Kiln shader is the same shape. The standalone download above is the shader with the small wrapper around it that the Ray Kiln host adds, so it compiles with the ordinary Metal compiler and runs in your own app. It defines three entry points: rk_vertex (a full-screen triangle), rk_fragment (calls the shader's shade) and, for shaders with sound, the compute kernel rk_sound.

To draw it, pass the uniforms at fragment buffer 0 and the parameter values at buffer 1, in the order they are declared:

Drawing it, in Swift
struct Uniforms {
    var resolution: SIMD2<Float>
    var time: Float
    var timeDelta: Float
    var mouse: SIMD2<Float>
    var frame: UInt32
    var pad: UInt32 = 0
}

let library = try device.makeLibrary(source: standaloneSource, options: nil)
let descriptor = MTLRenderPipelineDescriptor()
descriptor.vertexFunction = library.makeFunction(name: "rk_vertex")
descriptor.fragmentFunction = library.makeFunction(name: "rk_fragment")
descriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
let pipeline = try device.makeRenderPipelineState(descriptor: descriptor)

// Each frame, inside a render pass:
var uniforms = Uniforms(resolution: size, time: time, timeDelta: dt, mouse: mouse, frame: frame)
var params: [Float] = [100, 2, 3, 0.01, 0.25, 0.4, 0.3, 0.25, 0.8]     // the @param defaults, in the order they are declared
encoder.setRenderPipelineState(pipeline)
encoder.setFragmentBytes(&uniforms, length: MemoryLayout<Uniforms>.stride, index: 0)
encoder.setFragmentBytes(&params, length: params.count * MemoryLayout<Float>.stride, index: 1)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)

Time is in seconds, mouse is in pixels from the bottom left, and uv in the shader runs 0 to 1 with (0, 0) at the bottom left. Write the output as sRGB: the shader's numbers go to the screen as they are.

Built on later

More in Sound & Rhythm

A still from A Shader That Keeps the Beat

A Shader That Keeps the Beat

A sequencer in the sound, the picture synced to it