Repository navigation
feat(macos): capture system audio with a Core Audio tap — no screen recording permission - #16
Merged
Merged
Conversation
Replace the ScreenCaptureKit-only system lane with a Core Audio process tap (macOS 14.2+), which needs no screen-recording TCC grant: a private global stereo tap excluding LCT itself, wrapped in a private aggregate device and read through an IO proc on a serial queue, with buffers converted to 16 kHz mono via MicrophoneFormatConverter. ScreenCaptureKit stays as the fallback and only then checks the screen-recording permission; the active backend and the tap failure reason are recorded for the UI. The IO block is built in a nonisolated context so it never inherits MainActor isolation on Core Audio's realtime thread, and non-interleaved buffer lists that the zero-copy wrap rejects fall back to a per-channel copy.
start() no longer checks the screen-recording permission before enabling the system lane — the Core Audio tap needs none. The old mic-only fallback warning is gone; when the tap fails and the lane runs on ScreenCaptureKit a sticky warning names the failing step and OSStatus, and only that fallback can raise the screen-recording error notice. The capture service and speech engine are now injectable into the view model so this flow is testable offline.
The welcome flow lists only microphone and speech recognition — the screen-recording row, restart hint and resume-store usage are gone, and AppDelegate no longer preflights screen capture at launch, so a first run never triggers the screen-recording prompt. OnboardingResumeStore stays for its tests but nothing writes to it.
Add NSAudioCaptureUsageDescription for the process tap, annotate the diagnostics report (screen recording is only needed for the fallback, system audio runs on the Core Audio tap), rewrite diagnose_permissions.sh for the new permission model, and update the user docs to say system audio capture needs no extra permission.
…ect denial The Core Audio tap reports success on every setup step even when macOS withholds the system-audio consent — a denied tap simply never delivers an IO callback, while an authorized one calls back continuously, even in silence. Use that first-callback signal twice: - Onboarding gains an optional System Audio row whose Allow button runs SystemAudioPermissionProbe: start a tap off the main thread (start() blocks while the TCC prompt is up), watch for the first callback, then always stop it. Granted shows a check, denied offers the Settings pane. - AudioCaptureService starts a 3s first-callback watchdog after every tap start. On silence it stops and drops the tap (never falling back to ScreenCaptureKit), keeps the microphone lane running, and the ViewModel surfaces an error with an Open Settings action; a system-only session winds down to idle. The last verdict (granted/denied) is persisted and shown as a status line in the diagnostics report.
…enial On a real Mac, a tap whose IO proc was created while the consent prompt was on screen delivered no callbacks after the user clicked Allow; the next tap worked. Both the capture watchdog and the onboarding probe now rebuild a silent tap once, so allowing access is no longer misreported as denied. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
System audio no longer needs Screen Recording. It is captured with a Core Audio process tap (macOS 14.2+; LCT requires 15), whose consent is a normal in-app Allow prompt — no trip to System Settings, no relaunch, no periodic re-confirmation of screen recording.
SystemAudioTap: private global stereo tap excluding LCT's own process → private aggregate device → IO proc on a dedicated queue (builtnonisolated; a closure formed in a@MainActorcontext traps on Core Audio's IO thread) →MicrophoneFormatConverterto 16 kHz mono. Idempotent teardown in Core Audio order; every step's OSStatus logged. Core Audio calls sit behindSystemAudioTapHardwareso tests never create a real tap.AudioCaptureService: tap first; the ScreenCaptureKit stream is kept only as a fallback when the tap cannot be created (only that path checks screen recording, with a sticky warning).SCShareableContentcall removed; new System Audio · Allow row runsSystemAudioPermissionProbeoff the main thread so the consent prompt appears during setup instead of on the first Start.NSAudioCaptureUsageDescriptionadded; diagnostics and user docs updated.Findings from real-Mac testing (macOS 27)
CGPreflightScreenCaptureAccess() == false) captured system audio: 2837 callbacks in 30 s, peak RMS 0.37.AudioDeviceCreateIOProcIDWithBlockblocks (UI showsstarting…) — hence the onboarding row.Test results
./Scripts/run_tests.sh: 444 tests, 0 failures, no new warnings; CLT build clean. Tests use fake taps only — no real tap, no consent prompt.大家好，今天我们来聊一聊大语言模型…→ "Hello everyone, today we'd like to discuss how large language models are trained…"); a 1.5 h dual-lane session produced 701 system-lane results; watchdog verdict "authorized" 22 ms after start; second start: tap 0.35 s, whole start 3.8 s; onboarding shows Microphone ✓ / Speech ✓ / System Audio [Allow] → probegranted; no screen-recording prompt anywhere.Code by Qoder (Kimi-K3) from task briefs plus a Claude-written spike; the silent-tap rebuild, review and GUI verification by Claude.
🤖 Generated with Claude Code