Skip to content

Characterize usage reporting's settings and client failure paths - #55

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/characterize-usage-reporting-failures
Oct 31, 2026
Merged

dmccoystephenson merged 1 commit into
mainfrom
feature/characterize-usage-reporting-failures

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

Stage B (unit-test expansion) cycle. Characterization tests are added for the three error paths in usage_reporting.py that no test reached: coverage showed lines 94, 108–109 and 137–139 unexecuted. No production code is changed.

  • test_settings_file_that_is_not_an_object_disables_reporting_without_raising: a settings.json that holds valid JSON but not an object (["usage_reporting"]) makes loadSettings return None and log settings file is not a JSON object. The file is left untouched. This matches the README's statement that such a file "is left untouched, and nothing is reported for that run".
  • test_unwritable_settings_file_still_returns_the_defaults_and_warns_once: when the settings file cannot be written (its directory does not exist), the default block is still returned, the first-run notice is followed by a Could not write …; the notice above will be shown again next time. message, and no file is created.
  • test_settings_the_client_rejects_disable_reporting_without_raising: when TraceClient raises ValueError, buildClient logs Could not configure usage reporting (…) and returns a disabled client. Two real inputs trigger this: a whitespace-only endpoint, which is truthy and so bypasses the DEFAULT_ENDPOINT fallback, and a TRACE_INSTALL_ID longer than 255 characters. No installation-ID file is created in either case.

Every path is triggered with real input rather than a mock of the code under test.

Test plan

  • python3 -m unittest discover -s tests: 159 tests ran, OK (156 before, plus 3 new)
  • python3 -m py_compile on every module listed in the CI workflow passes
  • Mutation check: changing the isinstance(settings, dict) guard to if False:, and narrowing each of the two except clauses to except KeyError, made all three new tests error (4 subtest-level errors). Restoring the source returned them to green.
  • The interactive anchor (python3 main.py 10 --exit-after-create against a live Viron) is UNVERIFIED-not-applicable: no rendering or Viron-service code is touched.

Deferred issues

The tests stand without a tracking issue; the gap was found by coverage during triage. Open issues were not picked for these reasons:

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

🤖 Generated with Claude Code


drafted by Claude on behalf of Daniel Stephenson

Cover the three branches no test reached: a settings file that holds
valid JSON but not an object, a settings file that cannot be written,
and settings that TraceClient rejects with ValueError (a blank endpoint
or an oversized TRACE_INSTALL_ID). Each is triggered with real input
rather than a mock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric (anchored on CI run 41176547278, which passed, and on a local run of 159 tests, all OK):

  • Scope: PASS. The only file changed is tests/test_usage_reporting.py (+43 lines, no deletions). No production code, formatting or renames are included.
  • Tests-new: PASS. No new public functions were added. The three previously unexecuted branches (usage_reporting.py lines 94, 108–109 and 137–139 on main) each now have a test.
  • Tests-fix: N/A, with a mutation check in its place. No bug is fixed. As a substitute, the isinstance guard was disabled and each except clause was narrowed to KeyError. All three new tests then errored, and they went green again once the source was restored.
  • Sibling structure: PASS. The new tests sit in TestUsageReportingSettings, reuse self.log/self.logged/readSettingsFile(), and follow the existing test_<behaviour>_without_raising naming (compare test_unreadable_settings_file_disables_reporting_without_raising).
  • Sibling renames: N/A. Nothing is renamed.
  • Docs: PASS. README.md lines 267–269 say a file that "does not hold a JSON object" is left untouched and nothing is reported, which the first test now asserts. The README does not cover the unwritable-file or rejected-settings paths. Since it makes no contrary claim, this is an omission rather than drift, and it is not expanded under a test cycle. The .bat scripts are unaffected.
  • Issue resolution: N/A. There is no Closes reference; the gap was found by coverage during triage, as the PR body states.
  • Manual validation: PASS for the unit suite; UNVERIFIED-not-applicable for the interactive Viron run, since no rendering or service-call code is touched.

Inline notes:

  • tests/test_usage_reporting.py:150: the unwritable case uses a settings path inside a missing directory, not a read-only file. A permission-based setup would not fail when run as root (as both CI and this sandbox run), while a missing parent directory fails for every user. That makes the test deterministic across environments.
  • tests/test_usage_reporting.py:163: the blank-endpoint case records a slightly surprising current behaviour. A whitespace-only endpoint is truthy, so it bypasses the DEFAULT_ENDPOINT fallback and disables reporting instead of falling back. This is characterized as-is and not changed. Whether " " should fall back like a missing endpoint is a possible follow-up for the owner, not a defect this PR claims.

The diff is review-ready. No do-not-auto-merge path is touched.

This review comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson

Copy link
Copy Markdown
Member Author

Correction to the self-review above: the two inline notes cite the wrong lines. The missing-directory setup is at tests/test_usage_reporting.py:163 (not 150), and the blank-endpoint case is at tests/test_usage_reporting.py:176 (not 163). The findings themselves are unchanged.

This comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit 1ccbffc into main Oct 31, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the feature/characterize-usage-reporting-failures branch October 31, 2026 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant