Repository navigation
Description
Classification
RESEARCH / P2 BACKLOG / REUSE-FIRST / KILL-FIRST / NOT ADOPTED / NO PRODUCTION CHANGES AUTHORIZED. 2026-11-22.
Research question
Does an optional, higher-fidelity guitar audition/rendering backend measurably improve human selection of Griff's generated swancore guitar riffs versus the existing playback, and what is the least costly way to obtain it? Compare SoundFont/SF2, SFZ, and (only where justified) SuperCollider rather than assuming a synthesizer is automatically a better guitar.
This issue is not about changing the generator: Griff's canonical Guitar Pro/Score is the source of truth; MIDI is a lossy interchange adapter. The S10 CLAP target stays MIDI-out only with no audio code path.
Grounded current state
- SPEC.md: symbolic riff generator, not an audio synthesizer; Guitar Pro notation retains strings/frets/techniques that MIDI cannot.
- cockpit/src/audio.rs: desktop routes through
midirto selected external MIDI output; browser plays a sustained sine per pitch using Web Audio. - ui-core/src/playback.rs: shared
Player → PlaybackSinkevent seam handles note-on/off, loops, seeking, All-Notes-Off and master tempo; itsPianoRollViewcurrently drops per-note velocity and usesDEFAULT_VELOCITY = 96. Adding an instrument alone will not restore dynamic detail. - S8: already has native/web audition, candidate A/B, history and global-chain audition. Avoid introducing another event scheduler.
- S10: MIDI-out only.
- Related #216: CLMR audio embeddings vs symbolic riff similarity, which requires pinned, comparable offline audio renders; it does not authorize a new production audio backend.
Candidates and intended boundaries
| Candidate | Use | Limitation |
|---|---|---|
| Existing native MIDI out / browser Web Audio sine | Baseline, no new dependency | Web audition lacks instrument-like timbre; per-note dynamics currently flattened in view projection |
| FluidSynth + lawful SF2/SF3 guitar patch | Simple offline benchmark renderer and perhaps a preview option | Guitar realism/articulations depend on patch and available expressive controls; native/browser deployment must be assessed |
| sfizz + lawful SFZ guitar instrument | Sample-based guitar articulation test, optional authoring backend | Instrument and playback control mapping are nontrivial; assess integration, asset size and licensing |
| SuperCollider core UGens, Pluck, NRT | Offline experimental physical-modeling, procedural tone or batch WAV for #216 | A plucked-string model is not a realistic electric-guitar renderer or TTS. NRT is not a realtime cockpit backend. Avoid sc3-plugins/FluCoMa/FaustGen until evidence requires them |
Discovery catalogue: awesome-supercollider (archived 2022, Quarks references may be stale). Research note: Awesome SuperCollider Page. Optional related video-editor-only hypothesis: video-autoedit-research #3 (not a shared implementation proposal).
Kill-first protocol (no integration first)
- Characterize the current playback contract first using tests/fixtures: note onset/offset, overlaps, note-off/all-notes-off, tempo changes, loop boundaries, A/B switch and per-note velocity. Document exactly what the canonical Score retains versus what
PianoRollView/PlaybackSinkloses. Do not falsely claim an instrument reproduces unavailable articulation information. - Freeze 5–10 short, legally usable representative swancore phrases (clean lead, fast legato/tapping notated where present, chords, palm-muted pattern, dynamic variation); stable Score/MIDI digests and exact render parameters.
- A = existing browser sine / native selected MIDI port; B = offline FluidSynth + a pinned, licensed SF2 patch; C = offline sfizz + a pinned, licensed SFZ guitar instrument; D = SuperCollider basic
Pluck+ optional distortion/filter, offline NRT only if the prior candidates fail a specified tonal/experimental need. - Normalize loudness fairly and blind-listen to the same symbolic candidates: can two auditioners distinguish useful musical differences, assess phrasing, and select a better candidate? Ask separately about timbre and about riff quality; never infer playability from synthesized tone.
- For a comparable offline render fixture, pin versions, exact preset/instrument file hashes, note-to-articulation mapping, tempo, sample rate, processing, seeds, render time, failure rate, output hashes, size and licenses. Check reproducibility. Reuse the fixture/renderer in Research: benchmark CLMR audio embeddings against Griff symbolic riff similarity #216 only if it meets Research: benchmark CLMR audio embeddings against Griff symbolic riff similarity #216's timbre-confound design, without conflating the two issues.
- Produce a compact A/B table (quality/selection usefulness, fidelity gaps, latency, WASM/mobile feasibility, dependency/asset weight, integration cost), logs/sample renders where legally sharable and one GO / CHANGE / PARK / KILL recommendation per use case: interactive audition versus offline analysis.
Acceptance / stop conditions
- Current data loss is documented (especially velocity, techniques/fret/string metadata) and the baseline is measured.
- Blind A/B against a cheap patch-based solution before adding SuperCollider or an engine to the cockpit.
- Mobile/browser constraints, licenses and redistribution rights for instrument samples are explicit.
- If a backend is later proposed, it reuses the existing
Player → PlaybackSinkseam, not a second scheduler, and keepsgriff-core, Swang, Score and S10 CLAP semantics unchanged. - PARK/KILL if a cheap SF2/SFZ sound suffices, added synthesis makes riff ranking no more reliable, instrument timbre masks compositional differences, mobile cost is excessive, or the implementation requires expanding Griff into an audio workstation.
- No implementation from this issue alone: research and evidence first; only open a narrow follow-up implementation issue if a reproducible, measurable advantage exists.
Non-goals
No production audio synthesis, DSP framework integration, new realtime engine, full guitar-technique modeling, neural music generation, changes to S6/S7/S9 ranking, CLMR adoption, CLAP audio output or wholesale refactor of the shared playback system.
Activity