Skip to content

Research / AUDIO-AUDITION-01: guitar preview fidelity (SF2/SFZ vs SuperCollider NRT) #218

Open

Description

@PhysShell

Classification

RESEARCH / P2 BACKLOG / REUSE-FIRST / KILL-FIRST / NOT ADOPTED / NO PRODUCTION CHANGES AUTHORIZED. 2026-11-22.

Research question

Does an optional, higher-fidelity guitar audition/rendering backend measurably improve human selection of Griff's generated swancore guitar riffs versus the existing playback, and what is the least costly way to obtain it? Compare SoundFont/SF2, SFZ, and (only where justified) SuperCollider rather than assuming a synthesizer is automatically a better guitar.

This issue is not about changing the generator: Griff's canonical Guitar Pro/Score is the source of truth; MIDI is a lossy interchange adapter. The S10 CLAP target stays MIDI-out only with no audio code path.

Grounded current state

  • SPEC.md: symbolic riff generator, not an audio synthesizer; Guitar Pro notation retains strings/frets/techniques that MIDI cannot.
  • cockpit/src/audio.rs: desktop routes through midir to selected external MIDI output; browser plays a sustained sine per pitch using Web Audio.
  • ui-core/src/playback.rs: shared Player → PlaybackSink event seam handles note-on/off, loops, seeking, All-Notes-Off and master tempo; its PianoRollView currently drops per-note velocity and uses DEFAULT_VELOCITY = 96. Adding an instrument alone will not restore dynamic detail.
  • S8: already has native/web audition, candidate A/B, history and global-chain audition. Avoid introducing another event scheduler.
  • S10: MIDI-out only.
  • Related #216: CLMR audio embeddings vs symbolic riff similarity, which requires pinned, comparable offline audio renders; it does not authorize a new production audio backend.

Candidates and intended boundaries

Candidate Use Limitation
Existing native MIDI out / browser Web Audio sine Baseline, no new dependency Web audition lacks instrument-like timbre; per-note dynamics currently flattened in view projection
FluidSynth + lawful SF2/SF3 guitar patch Simple offline benchmark renderer and perhaps a preview option Guitar realism/articulations depend on patch and available expressive controls; native/browser deployment must be assessed
sfizz + lawful SFZ guitar instrument Sample-based guitar articulation test, optional authoring backend Instrument and playback control mapping are nontrivial; assess integration, asset size and licensing
SuperCollider core UGens, Pluck, NRT Offline experimental physical-modeling, procedural tone or batch WAV for #216 A plucked-string model is not a realistic electric-guitar renderer or TTS. NRT is not a realtime cockpit backend. Avoid sc3-plugins/FluCoMa/FaustGen until evidence requires them

Discovery catalogue: awesome-supercollider (archived 2022, Quarks references may be stale). Research note: Awesome SuperCollider Page. Optional related video-editor-only hypothesis: video-autoedit-research #3 (not a shared implementation proposal).

Kill-first protocol (no integration first)

  1. Characterize the current playback contract first using tests/fixtures: note onset/offset, overlaps, note-off/all-notes-off, tempo changes, loop boundaries, A/B switch and per-note velocity. Document exactly what the canonical Score retains versus what PianoRollView / PlaybackSink loses. Do not falsely claim an instrument reproduces unavailable articulation information.
  2. Freeze 5–10 short, legally usable representative swancore phrases (clean lead, fast legato/tapping notated where present, chords, palm-muted pattern, dynamic variation); stable Score/MIDI digests and exact render parameters.
  3. A = existing browser sine / native selected MIDI port; B = offline FluidSynth + a pinned, licensed SF2 patch; C = offline sfizz + a pinned, licensed SFZ guitar instrument; D = SuperCollider basic Pluck + optional distortion/filter, offline NRT only if the prior candidates fail a specified tonal/experimental need.
  4. Normalize loudness fairly and blind-listen to the same symbolic candidates: can two auditioners distinguish useful musical differences, assess phrasing, and select a better candidate? Ask separately about timbre and about riff quality; never infer playability from synthesized tone.
  5. For a comparable offline render fixture, pin versions, exact preset/instrument file hashes, note-to-articulation mapping, tempo, sample rate, processing, seeds, render time, failure rate, output hashes, size and licenses. Check reproducibility. Reuse the fixture/renderer in Research: benchmark CLMR audio embeddings against Griff symbolic riff similarity #216 only if it meets Research: benchmark CLMR audio embeddings against Griff symbolic riff similarity #216's timbre-confound design, without conflating the two issues.
  6. Produce a compact A/B table (quality/selection usefulness, fidelity gaps, latency, WASM/mobile feasibility, dependency/asset weight, integration cost), logs/sample renders where legally sharable and one GO / CHANGE / PARK / KILL recommendation per use case: interactive audition versus offline analysis.

Acceptance / stop conditions

  • Current data loss is documented (especially velocity, techniques/fret/string metadata) and the baseline is measured.
  • Blind A/B against a cheap patch-based solution before adding SuperCollider or an engine to the cockpit.
  • Mobile/browser constraints, licenses and redistribution rights for instrument samples are explicit.
  • If a backend is later proposed, it reuses the existing Player → PlaybackSink seam, not a second scheduler, and keeps griff-core, Swang, Score and S10 CLAP semantics unchanged.
  • PARK/KILL if a cheap SF2/SFZ sound suffices, added synthesis makes riff ranking no more reliable, instrument timbre masks compositional differences, mobile cost is excessive, or the implementation requires expanding Griff into an audio workstation.
  • No implementation from this issue alone: research and evidence first; only open a narrow follow-up implementation issue if a reproducible, measurable advantage exists.

Non-goals

No production audio synthesis, DSP framework integration, new realtime engine, full guitar-technique modeling, neural music generation, changes to S6/S7/S9 ranking, CLMR adoption, CLAP audio output or wholesale refactor of the shared playback system.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions