Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions agent_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@
- [x] **Refactor NoteSelector Focus Management:** Verified `NoteSelector` robustness with new test suite `src/components/__tests__/NoteSelector.test.tsx` mocking `requestAnimationFrame`.
- [x] **Step-Sequenced Drive:** Allow individual sequencer steps to override the global Drive/Distortion amount for aggressive, rhythmic vocal accents.

- [x] **Glissando/Portamento Curves:** Allow users to draw custom pitch curves or select between Linear and Exponential glide types between steps. (Implemented Exponential Glide in `SingingVoice.ts`!)
- [x] **Per-Step Breath Intensity:** Allow sequence steps to override global breathiness for rhythmic breathing and whisper effects. (Implemented in `useAudioEngine.ts`!)
- [ ] **Custom Sample Slicing UI:** Add a waveform view to `SamplerPanel` that allows users to manually add, move, and remove transient markers for slicing a custom WAV file instead of just auto-slicing by phoneme.

### Domain C: Accessibility & Mobile
- [x] **Touch Targets:** Audit `Sequencer.tsx` click listeners to ensure mobile drag-to-create works smoothly.
- [x] **A11y Colors:** Verify high-contrast separation between `synth-1` (Chords) and `synth-2` (Lead) notes.
Expand All @@ -48,7 +52,6 @@
* [x] **Idea:** "Spectral Granulator" - Add a granular synthesis mode to the sampler that uses FFT to freeze and smear TTS phonemes over time. (Implemented in Sampler and RubberBandProcessor via a 100ms looping Hann window!)
* [x] **Idea:** "Chord Evolving" - Allow drawing automation curves for the chord inversions or voicings used by `VoiceManager` in Polyphonic Synth A. (Implemented via automation track logic in `useStepHandler.ts` and `App.tsx`!)
* **Idea:** "Step-Sequenced Formant Shifts" - Allow users to pitch shift the formants of the TTS engine independently of the fundamental frequency per step.
* **Idea:** "Custom Sample Slicing UI" - Add a waveform view to `SamplerPanel` that allows users to manually add, move, and remove transient markers for slicing a custom WAV file instead of just auto-slicing by phoneme.
*These are concepts to be fleshed out by the agent during "Architect Mode".*

* [x] **Idea:** "Lyric Track" - A global text input that automatically distributes syllables across selected MIDI notes. (Implemented via global Lyric Track lane and `sliceIndex` auto-mapping!)
Expand All @@ -60,23 +63,23 @@
* [x] **Idea:** "Vocal Envelope Shaper" - Add granular attack/decay ADSR shaping explicitly for TTS syllables to create sharp plucks or smooth pads from any word. (Implemented in ExpressiveVoiceProcessor and exposed to SamplerPanel!)
* [x] **Idea:** "Dynamic Tremolo" - Expose Tremolo Rate and Depth to UI to pulse vocals. (Implemented!)
* [x] **Idea:** "LFO to Freeze Amount" - Automate the Freeze parameter with an LFO to create rhythmic pulsing granular clouds. (Implemented in RubberBandProcessor and exposed to SamplerPanel!)
* **Idea:** "LFO to Freeze Amount" - Automate the Freeze parameter with an LFO to create rhythmic pulsing granular clouds.
* [x] **Idea:** "Per-Step Filter & Resonance" - Allow sequence steps to override cutoff and resonance for rhythmic acid-style filtering of TTS samples. (Implemented!)
* [x] **Idea:** "Filter Envelope Mod" - Allow the sequence steps to have an envelope mod amount that specifically shapes the filter envelope per step. (Implemented!)
* [x] **Idea:** "Vocal Formant LFO" - Introduce a Formant LFO with rate and depth controls for dynamic rhythmic Wah-Wah effects on TTS vowels. (Implemented in FormantShifter and SamplerPanel!)
* [x] **Idea:** "Step-Sequenced Formant LFO" - Allow individual steps to override the global Formant LFO rate and depth for highly articulated rhythmic sequences.
* **Idea:** "Custom Waveform LFO" - Allow users to draw custom LFO shapes for formant and freeze modulation.
* **Idea:** "Glissando/Portamento Curve Drawing" - Allow users to draw custom pitch curves between steps, rather than just a linear glide.
* [x] **Idea:** "Glissando/Portamento Curve Drawing" - Allow users to draw custom pitch curves between steps, rather than just a linear glide. (Implemented!)
* [x] **Idea:** "Phoneme-Aware Velocity" - Automatically adjust the amplitude envelope attack/decay based on the phoneme type (e.g., plosives get faster attack, vowels get smoother attack). (Implemented via dynamic envelope overrides in `SingingVoice.ts`!)
* [x] **Idea:** "Spectral Morphing" - Implement functionality to morph spectrally between two different TTS phonemes or samples over a sequence of steps. (Implemented via FormantShifter Voice Character Morphing and step-sequenced automation!)
* [x] **Idea:** "Phoneme-Aware Velocity" - Automatically adjust the amplitude envelope attack/decay based on the phoneme type (e.g., plosives get faster attack, vowels get smoother attack). (Implemented via dynamic envelope scaling in `triggerSlice`!)
* [x] **Idea:** "Dynamic Reverb" - Allow users to draw automation curves for reverb send per step. (Implemented!)
* [x] **Idea:** "Global Saturation / Tape Warmth" - Add a master channel saturation unit to glue the mix together. (Implemented via WaveShaperNode!)
* **Idea:** "AI Auto-Mix Assistant" - Automatically adjusts levels, panning, and EQ based on track content to maintain a balanced mix.
* **Idea:** "Per-Step Breath Intensity" - Allow sequence steps to override global breathiness for rhythmic breathing and whisper effects. (Implemented!)

---

## 📜 Changelog
* [2026-06-20] - Implemented Glissando/Portamento Curves & Per-Step Breath Intensity: Added `slideType` parameter (Linear/Exponential) to allow musical variations of pitch glides in TTS, and allowed individual steps to override global breath noise via `breathIntensity`. Added Custom Sample Slicing UI to Active Backlog.
* [2026-06-19] - Implemented Global Saturation: Added a master channel `WaveShaperNode` with a variable distortion curve mapped to a "Warmth" (Saturation) slider in the top utility UI. Routed the entire master mix through it to add glue and presence. Added new idea: "AI Auto-Mix Assistant".
* [2026-06-18] - Implemented Dynamic Reverb: Added a `ConvolverNode` hooked up to the master output with a generated exponential decay noise impulse response. Mapped `reverbSend` from individual sequence steps in `NoteSelector` to send audio from the TTS `SingingVoice` into the new global reverb bus. Added new idea: "Global Saturation / Tape Warmth".
* [2026-06-17] - Implemented Spectral Morphing: Added `characterMorph` and `morphTarget` to allow smooth interpolation between voice characters (e.g., male to female) per sequence step using `FormantShifter.ts`. Added Morph knob and target selector to `SamplerPanel.tsx` and override controls to `NoteSelector.tsx`.
Expand Down
7 changes: 3 additions & 4 deletions src/components/AISongModal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -1153,11 +1153,10 @@ export function AISongModal({ isOpen, onClose, onImport, onShowToast, audioEngin
})
) : null}
</div>
<div className="space-y-1.5">
{trackStatisticsRows.length > 0 && (trackStatisticsRows as React.ReactNode)}
</div>
</div>
<div className="space-y-1.5">
{trackStatisticsRows.length > 0 && (trackStatisticsRows as React.ReactNode)}
</div>
</div>
) : null}

{/* Automation Visualization */}
Expand Down
7 changes: 2 additions & 5 deletions src/components/SamplerVoicePanel.tsx
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
import { type HarmonizerConfig } from '../engines/Harmonizer';
import React, { useState, useRef } from 'react';
import { type HarmonizerConfig, type HarmonyType, HARMONIZE_PRESETS } from '../engines/Harmonizer';
import React, { useState, useRef, useCallback } from 'react';
import { HardwareModule, type KnobConfig } from './HardwareModule';
import { LadderButton } from './sampler/LadderButton';
import { VerticalKnob } from './sampler/VerticalKnob';
import { HSlider } from './sampler/HSlider';
import { HarmonizerPopover } from './sampler/HarmonizerPopover';

interface SamplerVoicePanelProps {
title: string;
Expand Down
31 changes: 31 additions & 0 deletions src/engines/SingingVoice.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,17 @@ export class SingingVoice {
}
}

/**
* Exponentially ramp the pitch scale ratio to the given target.
* @param ratio Target pitch multiplier
* @param time Time to reach the target ratio
*/
exponentialRampToPitch(ratio: number, time: number): void {
if (this.workletNode) {
this.workletNode.parameters.get('pitchScale')!.exponentialRampToValueAtTime(ratio, time);
}
}

/**
* Linearly ramp the pitch from current value to the target MIDI note.
*/
Expand All @@ -275,6 +286,26 @@ export class SingingVoice {
this.linearRampToPitch(pitchRatio, time || this.audioContext.currentTime);
}

/**
* Exponentially ramp the pitch from current value to the target MIDI note.
*/
exponentialRampPitchFromMidi(targetMidiNote: number, baseMidiNote?: number, time?: number, coarseTune?: number, fineTune?: number): void {
const effectiveBaseNote = baseMidiNote ?? this.rootNote;
const effectiveCoarse = coarseTune ?? this.coarseTune;
const effectiveFine = fineTune ?? this.fineTune;

const totalSemitoneOffset = effectiveCoarse + (effectiveFine / 100);
const adjustedTargetMidi = targetMidiNote + totalSemitoneOffset;

const targetFreq = midiToFreq(adjustedTargetMidi);
const baseFreq = midiToFreq(effectiveBaseNote);

let pitchRatio = targetFreq / baseFreq;
pitchRatio = Math.max(PITCH_RATIO_LIMITS.MIN, Math.min(PITCH_RATIO_LIMITS.MAX, pitchRatio));

this.exponentialRampToPitch(pitchRatio, time || this.audioContext.currentTime);
}

/**
* Set pitch from MIDI note number relative to base note.
* Uses stored rootNote, coarseTune, and fineTune values.
Expand Down
21 changes: 17 additions & 4 deletions src/hooks/useAudioEngine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,7 @@ export const useAudioEngine = (pyodide: unknown) => {
sliceIndex?: number,
retrigger?: number,
slideFromMidi?: number,
slideType?: 'linear' | 'exponential',
phonemes?: PhonemeData[],
freeze?: number,
filterCutoff?: number,
Expand All @@ -281,7 +282,8 @@ export const useAudioEngine = (pyodide: unknown) => {
vibratoDepth?: number,
reverbSend?: number,
drive?: number,
characterMorph?: number
characterMorph?: number,
breathIntensity?: number
},
pitchOffsetSemitones: number = 0
) => {
Expand Down Expand Up @@ -383,7 +385,11 @@ export const useAudioEngine = (pyodide: unknown) => {
}
if (params.tremoloDepth !== undefined) voice.setTremoloDepth(params.tremoloDepth, triggerTime);
if (params.tremoloRate !== undefined) voice.setTremoloRate(params.tremoloRate, triggerTime);
if (params.breathIntensity !== undefined) voice.setBreathIntensity(params.breathIntensity, triggerTime);
if (noteParams?.breathIntensity !== undefined) {
voice.setBreathIntensity(noteParams.breathIntensity, triggerTime);
} else if (params.breathIntensity !== undefined) {
voice.setBreathIntensity(params.breathIntensity, triggerTime);
}
if (params.attack !== undefined) voice.setAttack(params.attack, triggerTime);
if (params.decay !== undefined) voice.setDecay(params.decay, triggerTime);
if (params.sustain !== undefined) voice.setSustain(params.sustain, triggerTime);
Expand Down Expand Up @@ -444,7 +450,12 @@ export const useAudioEngine = (pyodide: unknown) => {
voice.setPitchFromMidi(startMidi + pitchOffset, 60, triggerTime);
// Glide over half the target duration or a minimum of 0.15s, bounded by actual duration
const glideDuration = Math.min(Math.max(targetDuration * 0.5, 0.15), targetDuration);
voice.linearRampPitchFromMidi(targetMidi + pitchOffset, 60, triggerTime + glideDuration);

if (noteParams?.slideType === 'exponential' || params.portamentoType === 'exponential') {
voice.exponentialRampPitchFromMidi(targetMidi + pitchOffset, 60, triggerTime + glideDuration);
} else {
voice.linearRampPitchFromMidi(targetMidi + pitchOffset, 60, triggerTime + glideDuration);
}
} else {
voice.setPitchFromMidi(targetMidi + pitchOffset, 60, triggerTime);
}
Expand Down Expand Up @@ -619,14 +630,16 @@ export const useAudioEngine = (pyodide: unknown) => {
sliceIndex?: number,
retrigger?: number,
slideFromMidi?: number,
slideType?: 'linear' | 'exponential',
phonemes?: PhonemeData[],
freeze?: number,
filterCutoff?: number,
filterResonance?: number,
vibratoDepth?: number,
reverbSend?: number,
drive?: number,
characterMorph?: number
characterMorph?: number,
breathIntensity?: number
}
) => {
// Harmonize support - if harmonizer is active, generate multiple harmony voices
Expand Down
3 changes: 3 additions & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ export interface SamplerBankParams {
choir?: number; // Choir effect amount (0-1) - Detuned side voices
glitchChance?: number; // Probability of glitch/stutter effect (0-1)
freeze?: number; // Freeze/smear amount (0-1)
portamentoType?: 'linear' | 'exponential'; // Glide curve shape
freezeLfoRate?: number; // Freeze LFO rate (Hz)
freezeLfoDepth?: number; // Freeze LFO depth (0-1)
formantLfoRate?: number; // Formant LFO rate (Hz)
Expand Down Expand Up @@ -166,6 +167,8 @@ export interface Note {
vibratoDepth?: number; // 0-100%, vibrato depth override (Sampler only)
reverbSend?: number; // 0-1, amount sent to reverb bus
drive?: number; // 0-1, distortion/drive amount override (Sampler only)
slideType?: 'linear' | 'exponential'; // Shape of portamento curve
breathIntensity?: number; // 0-1, override breath noise amount (Sampler only)

// Phase 2: Melodic Lyric Mode - Per-step pitch control
pitch?: number; // MIDI note number for sampler melodic mode (default: 60 = C4)
Expand Down
Loading