日本語 README はこちら / Japanese README → README.ja.md
A real-time pitch-shifting / time-stretching C++ library, engineered so that harmonic purity, transient sharpness, natural formants, and bit-exact reproducibility all hold at once — none traded off against the others.
- C++17 static library, zero external runtime dependencies (pffft and Signalsmith Linear are build dependencies; see the license notices)
- Streaming pull API (
feed→available→read,finishto flush) for real-time inserts and offline rendering alike - Pitch ±24 semitones, time-stretch 0.25×–4× (wider values are accepted and clamped internally), independent and combinable
- Automatable while streaming:
setPitchSemitones()/setTimeStretch()may be called betweenfeed()calls, so a host can follow a pitch envelope without splitting the signal into segments — the read position is re-anchored at the change, and the output-side ratio is applied where that audio actually reaches the output, so a pitch ramp does not drag the material out of time - Formant preservation via a True Envelope class spectral-envelope engine
- Multi-resolution time-stretch engine (offline) — band-dependent window sizing with content-adaptive layout, which removes the phase-vocoder "chorusing"/watery artifact on dense mixes, pads, and held vowels while keeping transients sharp
- Deterministic: same input + same settings ⇒ bit-identical output, regardless of host buffer size, and across re-renders
Status: under active development (pre-1.0). The streaming API shown below is the stable surface.
- Why pbshift
- Measured results
- Feature details
- Algorithm overview
- Public API
- CLI usage
- Building
- Tests and quality gates
- Benchmark methodology
- Determinism guarantee
- Latency
- Repository layout
- Roadmap
- License and credits
Time-scale modification classically forces a trade-off: harmonic purity or sharp transients or natural formants or reproducible renders. pbshift is engineered so that none of these degrade the others:
- The native paths share an analysis front-end. Extreme speech processing also uses a synchronized auxiliary path with phase alignment and a gradual handover.
- Transients are handled as sparse, time-domain events rather than being smeared through the spectral pipeline.
- All stochastic components are position-seeded, so randomness never breaks reproducibility.
- The bypass path (stretch 1.0, pitch 0) is held to a strict null-test gate from the first commit — the engine cannot silently color the signal.
pbshift's own objective measurements on a deterministic synthetic/TTS corpus (see Benchmark methodology). All numbers are reproducible with the scripts in benchmarks/.
| Metric (median unless noted) | pbshift |
|---|---|
| Harmonic purity (HNR, tonal signals) | 134 dB |
| Pitch accuracy (f0 error) | 0.000 cents |
| Stereo coherence drift | 0.000 |
| Attack preservation ratio, percussion (1.00 = perfect) | 1.06 |
| Formant envelope distance, voice, mean over ±pitch conditions | 2.73 dB |
| Bypass / identity reconstruction null | −138 dB |
Notes:
- HNR (harmonic-to-interharmonic-noise ratio) measures "phasiness"/metallic noise on tonal material; higher is cleaner.
- Attack preservation ratio compares 10–90 % rise times of matched onsets in output vs. input; values above 1.0 indicate smearing, so 1.06 is near-perfect attack retention.
- Formant envelope distance is the log-spectral distance between input and output spectral envelopes on time-mapped frames.
- These are internal measurements on the corpus and conditions described below, not listening tests. The harness is engine-agnostic — point it at any engine that exposes the CLI convention and score it for yourself.
- Range: −24 to +24 semitones (continuous, fractional values allowed).
- Implemented with an engine-internal Kaiser-windowed sinc resampler combined with the spectral stretch engine.
- Direction-adaptive resampler placement: for downward shifts the resampler runs before the engine (slowing articulation into the analysis), for upward shifts it runs after the engine (avoiding temporal compression of articulation). This placement is chosen automatically per shift direction and measurably improves consonant/attack naturalness.
- Range: 0.25× to 4× (ratio = output duration / input duration). Wider values are accepted and clamped by the engine.
- Pitch and stretch are fully independent and can be combined (e.g. +7 st at 1.5×).
- At deep compression the analysis hop is automatically shortened so magnitude evolution is never temporally aliased (no content skipped between frames).
In the Live and StudioRT tiers, Music (CLI --multi) and Rhythm (CLI --rhythm) are streaming paths tuned for mixes and broadband/percussive material. Both use an N/8 fine frame grid, reconstruct sharp single (non-doubled) transients and chorus-free sustain, and avoid rate-pinning or onset phase-reset scheduling tricks that can leave a flam/echo on stretched drums:
- Long analysis window, content-adaptive synthesis support. The long window keeps frequency resolution.
Musicretains the full synthesis window on steady tonal frames and switches to a half-length4*hsdual window on detected event frames, preserving harmonic purity and stereo stability without spreading attacks.Rhythmuses the short window continuously for the sharpest dense percussion. Dualsig/normWOLA reconstruction stays normalized through either window. - Spectrum-wide identity-phase-locked coherence on every frame (Laroche & Dolson 1999), driven by the reassignment group delay from the analysis front-end: each valley-to-valley partial region is rebuilt as a rigid body about its peak's time-propagated phase, so overlapping grains reinforce into a single impulse instead of combing.
- Energy-rising partial suppression (cf. transient processing, Röbel 2003): a broadband hit spikes many bins; partials whose energy jumps sharply over the previous frame are merged onto the few strongest coherent centres, so the whole click is governed by a handful of phase centres = one crisp attack rather than a fan of detuned copies.
- Unbroken phase continuity — no onset reset — so all overlapping grains place a transient at one instant. Stereo uses the same verbatim inter-channel phase copy as the default path, so the image is preserved.
In those real-time tiers, select them explicitly with --multi / Config::Mode::Music, or --rhythm / Config::Mode::Rhythm.
The hardest time-stretch material — dense mixes, sustained pads, held vocal vowels — exposes the fundamental limit of a single analysis window. A short window keeps transients crisp but lacks the frequency resolution to separate closely-spaced partials, so their phases beat against the overlap-add grid and produce the classic phase-vocoder "chorusing" / watery artifact. A long window separates the partials cleanly but smears attacks. The Offline multi-resolution engine resolves this the way high-end stretchers do — by processing different frequency bands with a window sized for each:
- Low band, long window — high frequency resolution keeps dense partials cleanly separated, which is what removes the chorusing/flutter on sustained and complex material.
- High band, short window — high time resolution keeps attacks sharp.
- Bands are split in the STFT magnitude domain at the same frame timing, so they stay phase-aligned across the boundary (no time-domain crossover "flam").
On top of the band split:
- Content-adaptive layout. The engine inspects the signal (spectral flatness + pitch periodicity) and picks the layout automatically:
- broadband percussion → a single short window, no crossover (so isolated clicks stay razor-sharp — a band split would spread a click's low-frequency energy across the long window and dull the attack);
- a pure sustained tone or held vowel → a single long window (every harmonic stays phase-coherent; splitting dense voice harmonics across a crossover would decorrelate them);
- everything else (tonal mixes, polyphony, speech) → the multi-resolution split.
- Ratio-adaptive windows and overlap. Window length and synthesis overlap adapt to the stretch ratio — a longer window under expansion (tracks slowly-evolving harmonics), higher overlap under compression (keeps the analysis hop fine) — to hold quality across the whole 0.25×–4× range.
- Transient pinning. A broadband-onset detector pins the local rate to 1.0 across each attack so it is never time-spread, repaying the timing debt over the following ~100 ms. Gating on a broadband rise (not any spectral flux) keeps it from firing on tonal vibrato, which would otherwise inject amplitude modulation.
- Stereo phase lock. The loudest channel drives phase/onset decisions and correlated channels retain their inter-channel phase difference. An extremely quiet, decorrelated tonal bed is rendered independently so broadband energy in another channel cannot modulate it. The result remains deterministic and host-buffer independent.
On pbshift's internal objective chorusing metric (1–25 Hz amplitude-modulation depth on sustained partials, plus attack-sharpness crest on transients), the multi-resolution engine drives the chorusing measure down across the large majority of signal-class × ratio combinations, with the largest reductions at moderate-to-extreme stretch (2×–4×) — exactly where the chorusing artifact is most audible — and holds sustained tonal, voice, and pure-tone material near the noise floor at every ratio. Reproduce with benchmarks/ratio_eval.py.
Status: default Offline Music path. The multi-resolution engine is whole-signal (buffered) and is selected by Config::Tier::Offline plus Config::Mode::Music (the CLI combination --tier offline --multi). PBSHIFT_MULTIRES still opts other Offline modes into it for experiments. Live/StudioRT streaming is unchanged, and formant-preserving requests transparently fall back to the streaming engine.
- Optional (
setFormantPreserve(true)), designed for pitch-shifted voice and other formant-critical material. - Uses a True Envelope class cepstral spectral-envelope estimator (max-filter subsampling with adaptive iterative refinement) and applies the correction as a spectral pre-warp with per-bin gain clamping.
- Envelope order is tuned per FFT size; a mode gate automatically suppresses transient pinning in consonant-like contexts, which measurably improved voice quality.
| Mode | Material it is for | CLI | Current state |
|---|---|---|---|
Auto |
unknown / mixed input | (default) | picks a layout from spectral flatness and pitch periodicity |
Voice |
speech, vocals | --voice |
pitch-synchronous time-domain engine (src/pb_voice.h): real pitch-period grains, streaming and causal, ~46 ms input latency at 48 kHz |
Music |
mixes, polyphony | --multi |
event-adaptive synthesis windows; the mix-oriented default |
Rhythm |
drums, percussion | --rhythm |
continuously short synthesis window; the sharpest attacks we render |
Config exposes a mode hint (Auto / Voice / Rhythm / Music) and a latency tier (Live / StudioRT / Offline). StudioRT is the reference tier. Music uses event-adaptive synthesis windows; Rhythm keeps the short window active for percussion. Live is the implemented low-latency tier.
Voice does not run through the spectral engine at all. A phase vocoder buys length by re-printing spectral frames, and on speech that reads as a fine "delay echo" — copies of the same glottal pulse landing one synthesis hop apart. src/pb_voice.h carries real pitch periods instead, so the waveform is never resynthesised: that comb has no way to form, and the spectral valleys between harmonics stay as clean as the input's.
Three mechanisms, each aimed at one audible defect:
- Consistent-polarity pitch marks. Every mark is refined to the local extremum of one globally chosen excitation polarity. Marks that flip sign between periods splice opposite-going waveforms and click.
- Wide correlation search on the low band (±0.9 period). When a period must be reused to buy time, a narrow search leaves the copy sitting almost exactly one period from its original — the deepest possible comb, heard as doubling. A wide search lets the reuse land where the waveform actually matches, which costs neither chorus (nothing is blended) nor noise (nothing is synthesised).
- De-doubled high band. Above 3 kHz the content is aspiration and sibilance, where a duplicate is heard as doubled hiss. Grains are periodically time-reversed, which decorrelates a grain from its own repeat while preserving its magnitude spectrum exactly. The reversal period tracks the stretch ratio (
round(alpha)): at 2× plain alternation, at 4× every fourth grain. This is not a detail — using every third grain at 2× fills the spectral valleys by 3.5 dB and fails the noise gate outright.
It is causal and streaming. Pitch comes from a YIN tracker running per hop rather than from a whole-signal analysis, the crossover is a linear-phase complementary FIR rather than a zero-phase filter, and the grain search reaches only into a bounded look-ahead. Reported latency is frame/2 + 1.9·Tmax + 128 input samples — about 46 ms at 48 kHz, below the Live tier's spectral path. The analysis frame grows with the sample rate so the pitch floor does not drift upward (a fixed 2048-point frame caps the lag search at 94 Hz once you reach 96 kHz).
Every decision is a function of absolute sample position and that fixed look-ahead, never of how the host chunked its buffers, so the output is bit-identical at any block size from 1 sample upward. Getting there took closing three separate holes, all of which an offline prototype hides by construction: the grain schedule anchored on whatever the tracker had reached rather than on its first estimate, synthesis read the newest period estimate instead of the one belonging to the position it was rendering, and a grain's alignment search saw more input the longer it took to run.
Real-time behaviour. Both the pitch tracker and the grain search are spread across calls rather than run to completion the moment their audio arrives: a frame of YIN is around maxLag × frameLen multiply-adds and a grain's search a few hundred thousand more, so doing either in one block put a 14× spike on top of a low average. Each call now does a share of that work proportional to the audio it was handed — the arithmetic and its order are unchanged, so the output is identical; only when the work happens moves.
The budget matters in both directions. It has to exceed the schedule's own rate (about 1.3 alignment candidates per output sample), or synthesis falls a little further behind every block until the source position it still needs has been recycled out of the ring and it starts correlating against unrelated audio — a silent, gradual corruption rather than an obvious failure.
Measured on the corpus generator's stereo signal at 48 kHz (median of five 30-second runs, worst block in steady state, against the block's own real-time budget):
| Host block | 2× | 0.5× | 2× with +7 st |
|---|---|---|---|
| 128 | 63 % | 61 % | — |
| 256 | 48 % | — | — |
| 512 | 40 % | — | 77 % |
| 1024 | — | — | 52 % (+12 st) |
At a 64-sample block (a 1.3 ms budget) the engine sits at roughly 100 % and is not recommended; since its own latency is ~46 ms, a buffer that small buys nothing here. The native Voice path uses about 12 MB for 48 kHz stereo regardless of stream length — every buffer is a ring sized once at configure(), and nothing allocates on the audio thread.
For pure time stretch from 0.97× through 1.05×, Offline Auto/Voice/Music uses the quality-first multi-resolution renderer. Explicit Offline Rhythm uses an endpoint-anchored, absolute-grid WSOLA renderer for broadband and transient material; if any audible channel is sustained/tonal it safely falls back to multi-resolution. Non-trivial clips shorter than 50 ms also use multi-resolution in every mode, avoiding the pitch shift caused by endpoint interpolation. WSOLA analysis positions cannot accumulate timing drift, transients stay on the uniform map, the head and tail are protected, and every audible channel contributes a separately normalized similarity score. Exactly 1.0× is a bit-identical copy, requested output length is exact, and whole-signal output becomes available after finish().
- Arbitrary channel count (
Config::channels); stereo material gets dedicated coherence handling (see below). - Sample rate is configurable; internal window sizes are derived from it (48 kHz is the reference used in tests and benchmarks).
- Audio I/O is planar (non-interleaved) float:
float* const buffers[channel][frame]. - Each
Stretcherinstance is fully independent — no global state.
pbshift is a from-scratch, patent-conscious implementation built only from published research and our own measurements. The signal path:
input
→ [formant pre-warp (True Envelope class), optional]
→ [pitch resampler (Kaiser-sinc) — placed pre-engine for pitch-down]
→ shared analysis front-end (one STFT per hop):
magnitude, phase,
per-bin instantaneous frequency (reassignment operator, time-derivative window)
per-bin group delay (reassignment operator, time-weighted window)
→ single synthesis loop:
harmonic regions : phase-gradient integration over the reassignment
estimates with identity phase locking inside each
spectral region (below-tolerance bins get
position-seeded random phase)
transients : hold → fire → pin engine (see below)
→ stereo: phase decisions on a reference channel; input inter-channel
phase deltas copied verbatim
→ [pitch resampler — placed post-engine for pitch-up]
→ WOLA (weighted overlap-add) synthesis
→ output
Instead of estimating frequencies by differencing phases between frames (the classic phase-vocoder approach, which is noisy and hop-dependent), the front-end computes reassignment operators from a single frame: per-bin instantaneous frequency and per-bin group delay, obtained from auxiliary windowed transforms. Verified accuracy (see gates below): 0.0002 cents frequency error near a spectral peak and sub-sample group-delay error on impulses.
Output phase is produced by integrating the measured phase gradients (instantaneous frequency in time, group delay in frequency) across the time-frequency plane, processing the most reliable (highest-magnitude) bins first. Bins inside one spectral region stay phase-locked to each other exactly as in the input — vertical coherence is preserved by construction, which is what drives the 134 dB harmonic-purity result. Bins below the reliability tolerance receive deterministic, position-seeded random phase.
Attacks are detected with a spectral-flux broadband-rise detector (with a refractory period so a single event can't double-trigger). Each detected attack is then:
- Hold — the engine briefly holds frames while the attack approaches, instead of smearing it across the stretch grid;
- Fire — the attack is emitted at its correctly mapped output position (
t_out = ratio · t_in) with its original duration and decay; - Pin — the local stretch ratio is pinned to 1 around the event, and the timing debt this creates is repaid smoothly over the following frames.
Because input frames consumed by an event are marked as consumed, transient doubling is structurally impossible. Result: attack-preservation ratio 1.06 on the percussion tests (1.00 = perfect).
For correlated stereo/multichannel material, phase decisions are made once on the loudest reference channel and the input inter-channel phase differences are copied to the output. Measured coherence drift on the correlated stereo gate: 0.000. The only whole-signal exception is an extremely quiet, decorrelated tonal bed underneath a broadband reference; it is rendered independently to prevent the unrelated broadband phase trajectory from modulating that tonal channel.
Every place the algorithm needs randomness (below-tolerance phase, noise excitation) uses a PRNG seeded from the absolute stream position, never from time, pointers, or call patterns. Combined with deterministic internal scheduling, this yields bit-identical output regardless of how the host slices its buffers. See Determinism guarantee.
The entire public surface is one header: include/pbshift/pbshift.h.
namespace pbshift {
struct Config {
int sampleRate = 48000;
int channels = 2;
enum class Mode { Auto, Voice, Rhythm, Music };
Mode mode = Mode::Auto;
enum class Tier { Live, StudioRT, Offline };
Tier tier = Tier::StudioRT;
};
class Stretcher {
public:
void configure(const Config& cfg);
void reset();
void setTimeStretch(double ratio); // output/input duration, [0.25, 4]
void setPitchSemitones(double semitones); // [-24, +24]
void setFormantPreserve(bool enable);
void feed(const float* const* in, int frames); // planar, non-interleaved
void finish(); // end of input; flush tail
int available() const; // finalized output frames
int read(float* const* out, int frames); // returns frames written
int inputLatency() const; // samples, for host PDC
int outputLatency() const;
};
} // namespace pbshiftStretcher is a pull-model streaming processor:
feed()any number of input frames (any chunk size — 1 sample or 1 million, output bits are identical);available()reports how many finalized output frames are ready;read()retrieves them;- when the input ends, call
finish()once — the remaining tail becomes available; reset()returns the instance to a clean state for a new stream.
Offline pure time stretch in the 0.97×–1.05× near-unity range is intentionally whole-signal: available() remains zero until finish(). The API still accepts any feed() size. Oversized calls on every path are internally admitted safely and cannot wrap either the input ring or unread WOLA output.
#include "pbshift/pbshift.h"
#include <vector>
void render(const std::vector<std::vector<float>>& in, // [ch][frames]
std::vector<std::vector<float>>& out, // [ch][target]
int sampleRate, double stretch, double pitchSemitones,
bool preserveFormants) {
const int ch = (int)in.size();
const long long nIn = (long long)in[0].size();
const long long target = (long long)llround(nIn * stretch);
for (auto& c : out) c.assign(target, 0.0f);
pbshift::Config cfg;
cfg.sampleRate = sampleRate;
cfg.channels = ch;
pbshift::Stretcher st;
st.configure(cfg);
st.setTimeStretch(stretch);
st.setPitchSemitones(pitchSemitones);
st.setFormantPreserve(preserveFormants);
std::vector<const float*> ip(ch);
std::vector<float*> op(ch);
long long fed = 0, got = 0;
const int CHUNK = 8192; // any size works
while (fed < nIn) {
const int k = (int)std::min<long long>(CHUNK, nIn - fed);
for (int c = 0; c < ch; ++c) ip[c] = in[c].data() + fed;
st.feed(ip.data(), k);
fed += k;
int avail = st.available();
if (avail > 0 && got < target) {
const int want = (int)std::min<long long>(avail, target - got);
for (int c = 0; c < ch; ++c) op[c] = out[c].data() + got;
got += st.read(op.data(), want);
}
}
st.finish(); // flush the tail
while (got < target && st.available() > 0) {
const int want = (int)std::min<long long>(st.available(), target - got);
for (int c = 0; c < ch; ++c) op[c] = out[c].data() + got;
const int r = st.read(op.data(), want);
if (r <= 0) break;
got += r;
}
}For a plugin/insert context, call feed() with each host block and read() whatever available() reports; report inputLatency() / outputLatency() to the host for plugin delay compensation. tests/rt_bench.cpp demonstrates exactly this pattern and measures per-block cost against the real-time budget.
The repository builds an offline command-line renderer (tools/bin/pbshift[.exe]) that doubles as the benchmark-harness adapter:
pbshift in.wav out.wav [--pitch <semitones>] [--stretch <ratio>] [--formant] [--voice|--multi|--rhythm] [--tier live|offline]
| Option | Meaning | Range |
|---|---|---|
--pitch <st> |
Pitch shift in semitones (fractional OK) | −24 … +24 |
--stretch <r> |
Time-stretch ratio = output / input duration | 0.25 … 4 |
--formant |
Enable formant preservation | flag |
--voice |
Voice mode: shape-invariant harmonic-locked phase for speech/vocals | flag |
--multi |
Music mode: mix-oriented path; short synthesis windows only on detected events | flag |
--rhythm |
Rhythm mode: percussion-oriented path with continuously short synthesis windows | flag |
--tier |
Latency/quality tier: live (~64 ms), default (~128 ms), offline (~256 ms) |
enum |
Examples:
# One octave up, formant-preserved (voice)
pbshift vocal.wav vocal_up12.wav --pitch 12 --formant
# Half-speed (2x duration), pitch unchanged
pbshift drums.wav drums_2x.wav --stretch 2.0
# Combined: +7 semitones at 1.5x duration
pbshift mix.wav mix_p7_s15.wav --pitch 7 --stretch 1.5
# Content-adaptive mix stretch / percussion-priority drum stretch
pbshift mix.wav mix_2x.wav --stretch 2 --multi
pbshift drums.wav drums_2x.wav --stretch 2 --rhythmThe mode flags are mutually exclusive; specify at most one of --voice, --multi, and --rhythm.
Input: WAV (the bundled reader in tools/common/wav_io.h). Output: 32-bit float WAV at the input sample rate.
The multi-resolution time-stretch engine (above) also ships as a standalone offline renderer, tools/bin/multires[.exe]:
multires in.wav out.wav --stretch <ratio> [--voice] [--nopin] [--scales "N:lo:hi,..."]
| Option | Meaning |
|---|---|
--stretch <r> |
Time-stretch ratio = output / input duration |
--voice |
Force the sustained-vowel (single long-window) layout; otherwise the layout is content-adaptive |
--nopin |
Disable transient pinning |
--scales "N:lo:hi,…" |
Manual band layout: comma-separated fftSize:loHz:hiHz scales (for experimentation) |
# content-adaptive multi-resolution 2× stretch (percussion, tonal, voice auto-routed)
multires mix.wav mix_2x.wav --stretch 2.0
# a held sung vowel, forced to the long-window voiced layout
multires vowel.wav vowel_slow.wav --stretch 1.5 --voiceThe same engine is the default for Config::Tier::Offline + Config::Mode::Music. Setting PBSHIFT_MULTIRES before configure() opts other Offline modes into it; Config::Mode::Voice then selects the voiced layout. Formant-preserving requests fall back to the streaming path. Pitch shifting still runs through the internal resamplers.
Upward speech pitch processing no longer leaves an entire weakly voiced frame at its original pitch. It estimates spectral phase coherence, transposes the coherent component, and preserves the complementary aperiodic component for consonants and breath. Peak mapping uses instantaneous frequency to reduce multiplied frequency errors. Both components use the original full-spectrum formant envelope. This adds computation and history, but no lookahead or change to reported latency.
At +24 semitones, an independent 96-condition noisy-harmonic backend test reduced original-key detections from 77 to 16, with no newly failing cases; seven cases still missed the strict target-line criterion. This is a reduction in residual original pitch, not a guarantee of artifact-free conversion. All 3,032 compatibility renders were unchanged outside ten eligible upward Auto/formant-preserving cases. Those ten cases had no new verified pbACC failures and mean FLMOS increased from 2.109 to 2.116. Raw source-reference similarity and unverified noise scores can decrease: pbACC marks source-reference FR-QUAL as judged=false for pitch changes and NISQA-NOI as unverified in that domain. They remain recorded as advisory metrics; known pitch-matched references are required to judge target similarity. Legacy raw acceptance results are not relabeled as passes.
The upward speech path uses 16 overlapping analysis windows, conservative voiced-frame detection, and a finer formant envelope. Once speech is recognized, its classification persists through short phonetic changes and resets after 200 ms below the observed peak energy by 40 dB. Per-frame pitch reliability and attack protection still control routing.
In Auto mode with formant preservation, 44.1 and 48 kHz speech can use an auxiliary processor when the native analysis window is at least 2048 samples. Routing requires strong periodicity, changing pitch and spectral shape, and resolved source harmonics; it is determined by the audio content.
For large upward shifts at stretch 1, a modified Signalsmith Stretch backend preserves unvoiced components and uses a longer effective analysis window than the native pitch path. Eligibility requires at least twice the native frequency resolution, including the sample-rate scaling. At time stretch 3.5×–4× and pitch 0, phase-aligned pitch-synchronous waveform fitting can recover sections where the native output loses periodicity. Recovery requires an independently periodic alternative. Other sections continue through the native path. Transitions use a 20 ms fade with shared channel timing.
Auto speech processing reserves additional working buffers. Queued audio, control history, and unread pitch metadata can grow when input is supplied faster than output is read; drain output regularly for real-time use. The auxiliary path keeps the existing latency and output length. Changing pitch, stretch, or formant controls abandons the auxiliary path for the remainder of the stream; reset() permits fresh classification. A source pitch contour remains available until the corresponding output is read, including when a host supplies a long input before reading.
- CMake ≥ 3.20
- A C++17 compiler. Verified: MinGW-w64 GCC 15.1 on Windows. (MSVC configuration paths exist in the CMake script; MinGW is the tested toolchain.)
- Build dependencies: pffft (BSD-like license) and Signalsmith Linear (MIT), placed under
third_party/(not tracked in git). A modified MIT-licensed Signalsmith Stretch implementation is included insrc/pb_speech_spectrum.h:
cd third_party
git clone --depth 1 https://github.com/marton78/pffft.git
git clone https://github.com/Signalsmith-Audio/linear.git signalsmith-linear
git -C signalsmith-linear checkout de55e6a50ffcf6f8f43f649692d94691c7025151mkdir build && cd build
cmake -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release ..
cmake --build . -jTargets produced:
| Target | What it is |
|---|---|
pbshift |
The static library (src/pb_engine.cpp + headers in include/) |
pbshift_cli |
Offline CLI renderer → tools/bin/pbshift[.exe] (statically linked on MinGW) |
test_core |
Core correctness gates (see below) |
test_streaming |
Streaming determinism gates |
test_speech_state |
Long-input pitch-contour retention, release, and reset |
rt_bench |
Real-time viability benchmark |
Notes:
- pffft is compiled with
-mavx(andPFFFT_STATIC_DEFINE) on non-MSVC toolchains. - To link against the library from your own project: add
include/to your include path and link thepbshiftstatic library — that is all.
cd build
ctest # runs core, streaming, voice, real-time insert, and speech state
# or directly:
./test_core
./test_streaming
./rt_bench [block] [stretch] [pitch] [mode] # e.g. ./rt_bench 512 2.0 0 musicEvery gate below is enforced by an executable test in tests/ and was passing at the time of writing, with the measured value shown.
| Gate | Threshold | Measured |
|---|---|---|
| WOLA identity reconstruction (bypass null) | < −120 dB | −138 dB |
| Instantaneous-frequency accuracy near a spectral peak (pure sine) | < 0.5 cents | 0.0002 cents |
| Group-delay accuracy (offset impulse) | < 1 sample | sub-sample |
Chunk-size independence (64 vs 8192 samples per feed) |
0 differing samples | 0 (bit-identical) |
| Re-render determinism (two runs, same settings) | 0 differing samples | 0 (bit-identical) |
tests/test_core.cpp— gates 1–3: the analysis/synthesis skeleton must reconstruct a multi-tone input below −120 dB, and the reassignment operators must hit their accuracy targets.tests/test_voice.cpp— gates the Voice engine: chunk-independence at five ratios (1-sample, 37-sample, 64, 8192 and one-shot feeds all byte-compared), exact output length including ratios outside the tuned range, a ratio change mid-stream, degenerate inputs (100 samples, pure silence, an empty stream), and 44.1/96 kHz operation.tests/test_streaming.cpp— gates 4–5: renders the same stereo signal through four Auto conditions, two Music conditions, Rhythm 2×, and Voice 2× with 64-sample and 8192-sample feeds, then byte-compares outputs and re-renders. It also requires complete target-length output, preserves the Music/Rhythm fine hop across setters, and verifies compression-safe hop recalculation after reset/reconfigure.tests/rt_bench.cpp— streams stereo audio in host-sized blocks for 30 s and fails if the worst-case block cost exceeds the real-time budget; the optional fourth argument selectsauto,music,rhythm, orvoiceand the result includes reported latency.
Everything needed to reproduce our numbers lives in benchmarks/. The harness is engine-agnostic: any engine that exposes the shared CLI convention can be rendered and scored alongside pbshift. Point it at whatever engines you want to compare — the repository ships only pbshift.
A fully deterministic (fixed-seed) 48 kHz float32 corpus following time-scale-modification evaluation practice, covering every signal class that is historically hard for time-scale modification:
| Class | Signals |
|---|---|
| Voice | real TTS speech (English + Japanese, generated locally via the OS speech API — reproducible and license-free), synthesized sung vowel with vibrato, glide, and /a/→/i/ formant morph |
| Percussive | 118 BPM drum loop (kick/snare/hats, with ground-truth onset times), sparse castanet-style clicks (the classic transient torture test) |
| Plucked / keys | plucked guitar chord + strum, piano chords with inharmonicity and hammer noise |
| Pad / mix | detuned stereo string pad, full stereo mix (bass + drums + arpeggio) |
| Analytic | 440 Hz sine, rich harmonic tone (A2), log sweep, 4 Hz AM tone (modulation-spectrum reference) |
Ground-truth onset times are written to corpus/ground_truth.json for the percussion metrics.
14 conditions per signal per engine:
- Stretch-only: 0.25×, 0.5×, 0.8×, 1.25×, 2.0×, 4.0×
- Pitch-only: −24, −12, −5, +5, +12, +24 semitones
- Combined: +7 st @ 1.5× and −7 st @ 0.75×
Formant preservation is switched on for voice signals with nonzero pitch shift, on every engine that supports it.
There is no time-aligned reference for stretched output, so we measure artifact-specific quantities:
| Metric | What it detects |
|---|---|
duration_err |
output length vs. expected (stretch × input) |
onset_recall |
missed onsets = transient smearing |
onset_precision |
spurious onsets = doubling / stuttering |
attack_ratio |
output/input 10–90 % rise time at matched onsets (> 1 = smeared) |
hnr_db |
harmonic-to-interharmonic-noise ratio (drop = phasiness / metallic noise) |
f0_err_cents |
measured output f0 vs. expected f0 |
envelope_lsd_db |
log-spectral distance of spectral envelopes on time-mapped frames (formant quality) |
ltas_dist_db |
long-term average spectrum distance (stretch-only) |
warble_db |
spurious 2–16 Hz amplitude modulation on steady tones |
stereo_coh_drift |
loss of inter-channel coherence vs. input |
Renders corpus × conditions through every engine executable found in tools/bin/, then aggregates per-engine medians and writes benchmarks/out/results.csv. Any engine can participate by exposing the shared CLI convention:
engine in.wav out.wav --pitch <semitones> --stretch <out/in ratio> [--formant]
The corpus runner scores; this one prepares material a human can actually judge.
It renders one signal across a set of ratios through every pbshift mode (and any
reference engines named in PBSHIFT_REFERENCES), sweeping both directions —
lengthening and shortening fail differently, and an engine that holds together
at 2× can still come apart at 0.5×.
Two rules keep such a comparison honest, and both were added after they changed a conclusion:
- Headroom. A full-scale master is attenuated to −6 dBFS before it reaches any engine. Stretching a 0 dBFS mix can overshoot; without this, an engine that clips internally is scored on the source level rather than on its algorithm.
- Loudness. Every render is normalised to one integrated loudness. A listening comparison is decided by level long before it is decided by quality.
python benchmarks/ratio_sweep.py path/to/signal.wav 0.5 0.8 1.25 2.0 4.0
PBSHIFT_REFERENCES="other=/path/other.exe" python benchmarks/ratio_sweep.py signal.wav# Python 3.11+, numpy / scipy / soundfile / pyloudnorm
cd benchmarks
python make_corpus.py # deterministic corpus -> corpus/*.wav
python run_bench.py # all engines in tools/bin -> out/results.csv + summary
python run_bench.py --quick --engines pb # fast sanity pass, our engine onlyTwo of the metrics above carry most of the weight on rhythmic material, and it is worth being explicit about what they correspond to perceptually:
onset_precision(spurious onsets). A correct time stretch contains the same number of hits as its input; every extra detected onset is a hit that got printed twice. This is what a listener describes as an echo, a flam, or simply "you can hear that it was stretched". Count onsets in the source with the refractory window scaled by the stretch ratio, or the comparison flatters whichever engine was measured at the coarser spacing.attack_ratio(10–90 % rise time). This is the honest transient measure. A peak-to-preceding-RMS ratio looks like a transient metric and is not: under expansion the decay before a hit is stretched louder too, so the ratio falls even when the attack itself is untouched, and any engine that leaves gaps in the signal scores well for the wrong reason.
pbshift guarantees: same input samples + same settings ⇒ bit-identical output. Concretely:
- Host-buffer independence. Feeding 64 samples at a time or 8192 samples at a time produces byte-for-byte identical output. Your bounce is identical to your real-time pass.
- Re-render identity. Rendering the same material twice produces identical bits — a duplicated, phase-inverted track nulls to digital silence. (Verified in
test_streamingand in the benchmark matrix.) - No hidden entropy. Every stochastic element is seeded from the absolute stream position; nothing depends on wall-clock time, memory addresses, or thread timing.
This property is a hard CI gate, not a best-effort behavior — your offline bounce nulls against your real-time pass to digital silence, every time.
- Latency is fixed for a given configuration and reported in samples via
inputLatency()/outputLatency()— feed these to your host's plugin delay compensation. - The default Studio-RT tier uses a 4096-point analysis window at 48 kHz.
Auto/Voiceuse a 1024-sample synthesis hop;Music/Rhythmuse the finer 512-sample hop. - Spectral PDC is
input = N/2. Auto without formant preservation retainsoutput = N/2 + 2*hsfor attack pinning. Other spectral paths useoutput = N/2 + hs + 128, including a reserve for sinc resampling. At 48 kHz, Auto with formant preservation reports 109.3 ms in StudioRT and 56 ms in Live; StudioRT Music/Rhythm reports 98.7 ms. Auto without formant preservation remains 128 ms / 64 ms. These values use the standard hop; compression can refine it. The time-domain Voice path reports its own latency. available()counts samples the resampler can actually produce, including lookahead, queued ratio changes, and the end of the signal. For a fixed-rate insert, prime with the reported input-plus-output latency in silent input frames and retain the queued output. Time stretching uses the pull API because transient handling can vary output availability within a stream.- Designed tiers (
Config::Tier):
| Tier | Design point | Status |
|---|---|---|
Live |
2048-point window at 48 kHz; lowest streaming latency | implemented |
StudioRT |
4096-point window; reference real-time quality | current default |
Offline |
Exact-length WSOLA/MultiRes near unity; Music defaults to whole-signal multi-resolution; other processing uses the 8192-point path | implemented |
rt_benchprints average / worst-case per-block cost against the real-time budget along with the reported latencies for any block size.
include/pbshift/pbshift.h public API (the only header you include)
src/ engine implementation
pb_engine.cpp streaming scheduler + engine core
pb_stft.h shared analysis front-end (reassignment operators) + WOLA
pb_pghi.h phase-gradient integration engine
pb_envelope.h True Envelope class formant engine
pb_multires.h multi-resolution time-stretch engine (offline)
pb_voice.h pitch-synchronous voice engine (streaming, causal)
pb_speech_*.h content-gated extreme-speech processing and spectrum backend
pb_prepared_queue.h retained analysis/control queues
pb_resampler.h Kaiser-windowed sinc pitch resampler
pb_fft.h, pb_window.h FFT wrapper (pffft), window sets
tools/pbshift_cli/ offline CLI renderer
tools/multires_cli/ standalone multi-resolution renderer
tools/common/wav_io.h minimal WAV reader/writer
tests/ quality gates + real-time benchmark
benchmarks/ corpus generator, metrics, benchmark runner
third_party/ vendored dependencies (cloned locally, not tracked)
- Broader multi-resolution defaults. Offline Music now uses the content-adaptive multi-resolution engine by default; expanding that default to more modes remains subject to listening tests.
- Streaming multi-resolution. The multi-resolution path is currently whole-signal (offline). A block-streaming version would bring the chorusing-free quality to real-time inserts.
- Per-region content adaptation. Layout is currently chosen once per signal; per-region (time-varying) detection would let a single track switch layout as it moves between, e.g., a drum fill and a sustained pad.
- Voice pitch shifting. The pitch-synchronous engine currently serves time stretch; pitch shifts still reach it through the resamplers rather than by resampling the grains themselves, which is the more direct route for large shifts.
- Noise-component morphing — dedicated resynthesis of the noise/ambience component so pads, reverb tails, and textures stay natural at large stretch ratios instead of turning metallic.
- Real-time load smoothing — split-computation scheduling to flatten worst-case per-block cost during pitch shifting.
Livelatency tier — asymmetric analysis/synthesis window pair for sub-60 ms operation.- VST3 wrapper — a reference real-time insert plugin on top of the streaming API.
- License: pbshift is open-source software released under the MIT License.
- Third-party dependency: pffft (BSD-like FFTPACK-derived license) is used for FFTs. It retains its own copyright and license; see Third-party notices. The extreme-speech pitch path includes a modified Signalsmith Stretch 1.3.2 implementation (MIT, commit
a670068d9aeb64913331d5cc29337b19a457a7df) and uses Signalsmith Linear (MIT). - The native engine is based on published research and our own measurements; the attributed speech spectrum backend is derived from Signalsmith Stretch. Full third-party license texts are retained.
Streams started with formant preservation off in Auto, Music or Rhythm use an interpolation cutoff at 93.6% of Nyquist (previously 92%). The 64 one-sided taps and Kaiser beta 12 stay the same; decimation retains its existing anti-alias filter. This preserves more high-frequency detail on downward pitch shifts without adding lookahead. The profile is selected before the stream buffers input and stays fixed through automation; reset selects it again from the current settings. Voice and formant-preserving streams retain their existing profile.
The change passed the fixed pbACC per-case guards on the 308-case primary matrix, 528-case secondary matrix, 460-case external matrix and the additional 200-case pitch/time grid, including downward steps from -0.5 to -24 semitones. Eight unused utterances from known speakers at 44.1/48 kHz added 224 safety checks; they do not demonstrate an independent improvement. A separate 512-case coherent-sine audit measured unwanted interpolation images below -120 dB; an overly wide filter was rejected by the same audit. This update improves native downward-pitch processing; further time-stretch quality work is ongoing.
Music mode in the Offline tier retimes short, isolated decaying sounds by moving their peak anchors and retaining the original samples and inter-channel timing. It preserves a compact hit's attack and decay rather than depositing it through several overlapping spectral frames. This applies to pure time stretching (pitch = 0) in the existing whole-signal renderer.
The complete signal must have at least 50% exact digital silence. Events separated by less than 12 ms are grouped; each group must last at most 80 ms, reach its peak within 20 ms, and, for groups longer than 10 ms, have the last fifth's RMS no greater than one quarter of the peak. A continuous background, sustained/long event, target collision or boundary truncation selects the regular multi-resolution renderer. Quiet background samples are retained, without noise-floor removal, gain adjustment or a limiter.
A separate 540-condition test covered six sample rates (8–48 kHz), mono/stereo signals, four transient families and six kinds of compatibility control, at length ratios 0.25, 0.5, 0.75, 0.9, 1, 1.1, 1.25, 2 and 4. All 258 admitted outputs matched their known waveform targets sample for sample; all 282 protected cases matched the previous renderer's WAV. With the validated pbACC clock, mean FR-QUAL on the 270 known-event targets increased from 4.275 to 4.695 without a new per-case regression. The remaining cases are compatibility checks, not subjective-quality references. This does not certify arbitrary music or speech time stretching.
Use pbshift input.wav output.wav --multi --tier offline --stretch 4. Reproduce the controlled comparison with python benchmarks/transient_quality_check.py --exe path/to/new/pbshift.exe --baseline path/to/previous/pbshift.exe --pbacc path/to/pbacc.exe --out local/transient-check (NumPy and SoundFile). Every source, target, raw result and compatibility case is retained.