Skip to content

feat(android): add a prototype Android app with a floating dictation bubble - #262

Draft
m-ods wants to merge 3 commits into
mainfrom
android-prototype
Draft

feat(android): add a prototype Android app with a floating dictation bubble#262
m-ods wants to merge 3 commits into
mainfrom
android-prototype

Conversation

@m-ods

@m-ods m-ods commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown

What & why

A first Android version of Blurt in App/BlurtAndroid (Kotlin, Gradle), modelled on Wispr Flow's Android bubble. It floats over other apps while a keyboard is up: tap it, speak, tap again, and the words land in the focused field.

  • Same request as the Mac and iPhone apps: one streamed POST to dictation.assemblyai.com/v1/transcribe/live. The config part goes first, then 16 kHz mono PCM frames as they're recorded, with the text before the cursor as stt_prompt and the engine's cleanup instruction as llm_instruction. DictationClient.kt mirrors AssemblyAITranscriber.swift.
  • Typing into other apps: an accessibility service finds the focused field across windows, inserts with the spacing rule, and reports whether a keyboard is on screen, so the bubble only shows while typing. A field that reports no cursor gets a paste instead: Telegram's empty message box reports its "Message" placeholder as text with selection -1.
  • Bubble: a microphone foreground service started from the app, since Android won't let a service open the mic from the background otherwise. The window is not focusable, so tapping it doesn't steal the field's focus. Drag has haptic ticks and glides to the nearer edge. Long-press cancels a dictation, or opens settings when idle. It re-snaps on fold and unfold.
  • Styles: None, Professional, Casual and Ultracasual (copied verbatim from StylePreset.swift), plus Custom, appended with the engine's style preamble.
  • API key: pasted in, or minted by Sign in with AssemblyAI, and encrypted under an Android Keystore key.
  • Sign in with AssemblyAI: a port of Sources/AssemblyAI/OAuth and the engine's AssemblyAIAccount: PKCE on the consent page in a Custom Tab, the access token traded for the connection's API key, renewal, and a sign-out that revokes server-side. It reuses the Mac's redirect, com.assemblyai.blurt://oauth/callback. It's off unless local.properties sets blurt.oauth.* (the Mac's SignIn.local.xcconfig values), the same way the Mac's release builds leave it off. Android can't keep a renewal timer running, so it renews when the app opens, when a dictation starts, and at once when dictation reports the key refused. A sandbox sign-in dictates against its own host.
  • Icons: built from the b icon in the Blurt Figma file (Mac Os › 256x256).

Two files copy Swift text verbatim and must be kept in sync by hand: CleanupInstruction.kt and the presets in Styles.kt.

AGENTS.md gains a repo-map entry. Prettier also realigns one existing table row there (AVAudioEngine), which already failed prettier --check on main.

Not done yet: key terms, text shortcuts, release signing, tests.

How it was tested

  • Built and installed with ./gradlew installDebug, then used on a Galaxy Z Fold (Android 17): dictation into Telegram and other apps, styles, drag and snap, keyboard-only visibility.

  • Sign in with AssemblyAI is untested end to end: the sandbox values weren't available. With no blurt.oauth.* set, the setup screen is unchanged (paste-a-key only).

  • Android Lint: no issues beyond dependency-version and backup notices.

  • scripts/check.sh --portable passes for the files this touches. Several linters weren't installed locally, so they were skipped. Nothing in CI builds or checks the Android app yet.

  • scripts/check.sh passes (or CI will, if I'm not on a Mac)

  • I read AGENTS.md and this doesn't reintroduce anything deliberately removed

  • Docs updated if behavior changed

🤖 Generated with Claude Code

m-ods and others added 3 commits October 23, 2026 09:38
@m-ods @claude
feat(android): add a prototype Android app with a floating dictation …
e0964ab 
…bubble

A Kotlin app in App/BlurtAndroid, modelled on Wispr Flow's Android bubble:
it floats over other apps while a keyboard is up; tap to dictate, tap
again, and the words land in the focused field.

- One streamed POST to the dictation route, the same request the Mac and
  iPhone make: config first, then 16 kHz PCM frames as they're recorded,
  the text before the cursor as stt_prompt, and the engine's cleanup
  instruction (copied verbatim) as llm_instruction.
- An accessibility service finds the focused field across windows, inserts
  with the spacing rule, and reports whether a keyboard is on screen. A
  field that reports no cursor gets a paste: Telegram's empty box reports
  its "Message" placeholder as text with selection -1.
- The iPhone's style presets (copied verbatim) plus a custom style.
- The API key is encrypted under an Android Keystore key.
- Icons are built from the b icon in the Blurt Figma file.

Not built or checked by CI. AGENTS.md gains a repo-map entry; prettier
also realigns one existing table row there that already failed
`prettier --check` on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@m-ods @claude
feat(android): add Sign in with AssemblyAI
1b6e7c3 
A port of the Mac's sign-in (Sources/AssemblyAI/OAuth and the engine's
AssemblyAIAccount): authorization code with PKCE on the dashboard's consent
page in a Custom Tab, the access token traded for the connection's API key,
renewal, and a sign-out that revokes server-side.

- Off unless local.properties sets blurt.oauth.* (the Mac's
  SignIn.local.xcconfig values), so builds without them show only the
  paste-a-key path, as the Mac's release builds do.
- Reuses the Mac's redirect, com.assemblyai.blurt://oauth/callback, which
  the Connected App already lists; MainActivity is singleTask so the
  redirect lands on the open setup screen.
- The minted key goes where a pasted one does; the session sits beside it,
  encrypted under the same Keystore key (SecureStore).
- No renewal timer on Android: renews when the app opens and when the
  bubble starts a dictation, and at once (at most every 15 min) when
  dictation reports the key refused.
- A sandbox sign-in dictates against its own host (dictationOrigin).

Untested end to end: the sandbox values weren't available.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@m-ods @claude
feat(android): prepare for Google Play internal testing
a67f0d6 
- Target Android 16 (API 36), which Play requires of new apps from
  2026-08-31; AGP 8.10.1 for compileSdk 36.
- Play's prominent disclosure for the Accessibility API: step 4's "Open
  settings" first shows what Blurt typing reads, what goes to AssemblyAI,
  and that password fields are never read; only "Agree and continue" opens
  the settings. The service declares isAccessibilityTool="false".
- Release signing with an upload key kept outside the repo: its path in
  local.properties, its password from BLURT_UPLOAD_PASSWORD or the macOS
  Keychain. `./gradlew bundleRelease` builds the .aab Play takes.

Built and signature-verified; not yet run on a device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@m-ods