Skip to content

fw: stop clipping the start of replies, 1.3.5-dev.1102.1 (dotty-stackchan #200) - #4

Open
BrettKinny wants to merge 3 commits into
dottyfrom
agent/fw-reply-head-clip
Open

BrettKinny wants to merge 3 commits into
dottyfrom
agent/fw-reply-head-clip

Conversation

@BrettKinny

Copy link
Copy Markdown
Owner

What

The Speaking transition ends with ResetDecoder(). Since the Dotty patch moved that transition from tts start to sentence_start, audio the server sent right behind sentence_start was already queued and got thrown away, clipping the start of a reply. This moves the reset to tts start, before any reply audio exists. Version bumps to 1.3.5-dev.1102.1.

Refs BrettKinny/dotty-stackchan#200 (part 2).

Evidence

On the robot, from the serial log:

Build ResetDecoder dropping unplayed audio Replies / speaking turns
v2 b423afa 45 183 replies
this build b846d9f 0 23 speaking turns

Also 0 panics, 0 task watchdog hits and 0 allocation failures over the session.

The 0 comes from that log line, which proves the drop no longer happens. Nobody has scored the audio itself yet for clipping.

Not yet verified: please read before merging

  • Do not merge until take_photo passes on this build. It's the low-memory canary, and it hasn't run here because Kid Mode blocked the requests.
  • Free SRAM: lowest was 3043 B, against about 7 KB on 1.3.4 overnight and a 3 KB regression limit set for this candidate line. Steady free was about 20 KB.
  • V02 unbiased hasn't been rerun.
  • Unexplained: the WebSocket dropped once (TCP receive failed) about 8 s after a head pet, and rejoined in 3 s. Not known to be related.
  • #210 wake latency is on this firmware line and isn't changed here.

AI-assisted: written by Claude Opus 5.5; needs human review before merge.

🤖 Generated with Claude Code

BrettKinny and others added 2 commits November 2, 2026 15:17
…200)

On v2 'ResetDecoder dropping unplayed audio' fired 45 times in 183 replies, every
time at the start of a reply (decode=0-5, playback=0-4 packets, 60 ms each), never at
tts stop. The Dotty patch moved the Speaking transition from 'tts start' to
'sentence_start', and the Speaking entry (display, face detector, StateManager hooks,
~130 ms) ends with ResetDecoder(). Audio the server sent right behind sentence_start
was already queued and got thrown away, clipping the head of about a quarter of the
replies. Reset on 'tts start' instead, before any audio of the reply exists.

Built only; not flashed. Untested on the robot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@BrettKinny

Copy link
Copy Markdown
Owner Author

Update 2026-11-02 16:30: lowest free SRAM on this build has reached 1219 B, below the 3 KB limit, with no allocation failure or panic so far. The session also ran about 160 background captures in an hour, against 14-16 in about two hours overnight, so the captures may be the cause, but that isn't proven. Do not merge until this is explained.

AI-assisted: posted by Claude Opus 5.5.

@BrettKinny

Copy link
Copy Markdown
Owner Author

Update 2026-11-02 17:00, after a power cycle with background captures blocked (dotty-stackchan #215): lowest free SRAM 10.2 KB over about 7.5 minutes, set at boot and unchanged through 5 voice turns and a long story. The previous session had already reached 3.3 KB at the same age. The captures look like the main cause, but this was a short run in Kid Mode.

New: 3 ResetDecoder dropping lines in 8 speaking turns, all on a reply right after idle -> listening; 0 on later turns in the same connection. Hypothesis, not proven: EnableVoiceProcessing(true) also resets the decoder (audio_service.cc:595), and that reply was already arriving (tts start 23 ms after the state change, drop about 490 ms later, 4 playback packets). This PR doesn't cover that path, so it fixes reply turns but not the first reply after idle.

AI-assisted: posted by Claude Opus 5.5.

…t (#200)

The server bursts 5 packets right behind sentence_start, but the Speaking
transition is scheduled and lands 90-300 ms later, and OnIncomingAudio
discarded anything outside Speaking without logging. The ResetDecoder count
from the last fix could not see this, so its 0 drops did not prove whole
replies. Open a reply on the network thread at sentence_start, and keep the
entry-to-Listening and late tts-start resets from clearing it. The privacy
gate still blocks audio.

tts stop now logs early and discarded packet counts. Version 1.3.5-dev.1102.2.
Built only; not flashed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@BrettKinny

Copy link
Copy Markdown
Owner Author

Pushed e2096af (1.3.5-dev.1102.2), built, not flashed. The earlier 0-drop count only covered ResetDecoder: OnIncomingAudio also discarded reply audio without logging until Speaking began, and the server bursts 5 packets right after sentence_start. This commit accepts reply audio from sentence_start, stops the two resets clearing an open reply, and logs early= / discarded= counts at tts stop. The privacy gate still blocks audio.

AI-assisted: posted by Claude Opus 5.5.

@BrettKinny

Copy link
Copy Markdown
Owner Author

Robot results on e2096af (1.3.5-dev.1102.2), 7 replies including one after idle -> listening: every tts stop logged early=0 discarded=0, so no reply audio was lost before Speaking and the 300 ms gap I described didn't happen in this sample. The sentence_start change is defensive for now. The one ResetDecoder dropping (decode=3 playback=4) was the scheduled tts-start reset after a face wake, before any sentence_start, so most likely the wake chime, not reply audio. Lowest free SRAM 10.9 KB (Kid Mode, no background captures), no panics or allocation failures. Still open before merge: take_photo with Kid Mode off.

AI-assisted: posted by Claude Opus 5.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant