Skip to content

runner: diagnose missing streaming wakeword measurements - #193

Draft
rohanbanerjeee wants to merge 1 commit into
mlcommons:masterfrom
rohanbanerjeee:codex/sww-empty-measurement-diagnostics
Draft

rohanbanerjeee wants to merge 1 commit into
mlcommons:masterfrom
rohanbanerjeee:codex/sww-empty-measurement-diagnostics

Conversation

@rohanbanerjeee

Copy link
Copy Markdown

When the streaming runner receives an empty interface response or fewer than two complete processing pulses, it can fail with an IndexError or a NumPy zero-size reduction error. This change gives those failures measurement-specific diagnostics and points to the I2S, PROCESSING, and WW_DETECTED signal paths.

It also explains the reference firmware's initial detection synchronization pulse and adds hardware-free regression tests for response parsing, incomplete pulse captures, valid timing and detection results, and the actual streaming script step with simulated board responses. Valid measurement calculations and detection scoring are unchanged.

Related to #175. This is a runner diagnostics contribution; it does not reproduce or resolve the reported DMA callback failure. The original hardware issue still requires a board-based reproduction.

Validation:

  • python -m unittest discover -s benchmark/runner/tests -v: all 17 tests pass with Python 3.9 and NumPy 1.26.4. Against the original runner code, the same suite produced 6 errors and 2 failures.
  • git diff --check: passes.
  • No physical board, audio delivery, or energy measurement validation was performed.

@github-actions

Copy link
Copy Markdown

MLCommons CLA bot:
Thank you very much for your submission; we really appreciate it. Before we can accept your contribution,
we ask that you sign the MLCommons CLA (Apache 2). Please submit your GitHub ID to our onboarding form to initiate
authorization. If you are from a MLCommons member organization, we will request that you be added to the CLA.
If you are not from a member organization, we will email you a CLA to sign. For any questions, please contact
support@mlcommons.org.
0 out of 1 committers have signed the MLCommons CLA.
❌ @rohanbanerjeee
You can retrigger this bot by commenting recheck in this Pull Request

@rohanbanerjeee

Copy link
Copy Markdown
Author

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant