Skip to content

fix(chronicle): prevent response data loss in nl_search and search_udm when :udmSearch returns stats - #228

Open
dandye wants to merge 2 commits into
google:mainfrom
dandye:chronicle__search-udm-stats
Open

dandye wants to merge 2 commits into
google:mainfrom
dandye:chronicle__search-udm-stats

Conversation

@dandye

@dandye dandye commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #227

Summary

This stability fix prevents HTTP 200 response data loss and false-negative empty results in ChronicleClient.nl_search, ChronicleClient.search_udm, and secops search --nl-query.

When a caller runs a natural language prompt such as "Count failed user logins by user in the last 24 hours" via client.nl_search(...) or secops search --nl-query, Chronicle's :translateUdmQuery endpoint emits a Chronicle SIEM YARA-L 2.0 query with match: and outcome: sections. nl_search passes that translated query directly to client.search_udm(...), and secops stats has no --nl-query flag. GET /v1alpha/{instance}:udmSearch returns HTTP 200 with result["stats"] and omits events. Previously, search_udm discarded result["stats"] and returned {"events": [], "total_events": 0, "more_data_available": False}, reporting zero matches even when the API returned matching security records. Passing translate_nl_to_udm(...) output to search_udm(...) as shown in README.md hit the same bug.

Changes

  • When as_list=False and isinstance(result.get("stats"), dict) holds, search_udm sets response["stats"] = process_stats_results(raw_stats) using the existing helper in secops.chronicle.stats, matching the exact columns, rows, and total_rows dictionary structure of ChronicleClient.get_stats.
  • Folds raw_stats.get("tooManyResults", False) into the existing more_data_available boolean so callers still detect truncated statistical results without extra schema fields.
  • Updates docstrings in search.py, nl_search.py, client.py, and README.md, noting that ChronicleClient.get_stats remains the dedicated method for direct statistical queries.

Backward compatibility

  • No signature or return-type changes. search_udm, ChronicleClient.search_udm, nl_search, ChronicleClient.nl_search, ChronicleClient.get_stats, and secops search keep identical parameters, defaults, and type annotations.
  • Plain UDM event filter queries unchanged. When :udmSearch omits stats, search_udm returns the exact 3-key dictionary {"events": [...], "total_events": int, "more_data_available": bool} with no stats key added.
  • as_list=True unchanged. search_udm with as_list=True continues returning list[dict[str, Any]].
  • Additive stats key on YARA-L 2.0 outcome: queries. Existing top-level keys events, total_events, and more_data_available remain present with identical types, while response["stats"] is purely additive and matches ChronicleClient.get_stats 1:1.
  • CLI formatters compatible. secops search passes the returned dictionary directly to output_formatter, which serializes json and text output cleanly without schema assumptions.

Maintenance mode alignment

This PR is a backward-compatible stability fix for HTTP 200 response data loss in existing SDK and CLI methods. It adds no new endpoints, methods, parameters, or dependencies.

Testing

  • Added unit tests in tests/chronicle/test_search.py covering stats normalization, more_data_available propagation from tooManyResults, zero-row stats payloads, non-dict stats guards, as_list=True behavior on stats responses, and absence of stats on standard event responses.
  • Added unit test in tests/chronicle/test_nl_search.py verifying end-to-end nl_search preservation of stats when :translateUdmQuery emits a YARA-L 2.0 outcome: query.
  • Verified pytest -m "not integration" --cov=secops and pylint --rcfile=pylintrc src/secops.

Attach normalized stats results via process_stats_results along with too_many_results when :udmSearch returns a stats payload on Chronicle SIEM YARA-L 2.0 statistical queries. Plain event filter queries without a stats payload continue returning only events, total_events, and more_data_available.

Fixes google#227
@dandye
dandye requested a review from prachib29 as a code owner November 6, 2026 20:34
Match ChronicleClient.get_stats() 3-key stats dict, fold tooManyResults
into more_data_available, guard non-dict stats payloads, and clarify
search_udm and nl_search docstrings.
@dandye dandye changed the title fix(chronicle): preserve stats results in search_udm and nl_search fix(chronicle): prevent response data loss in nl_search and search_udm when :udmSearch returns stats Nov 6, 2026
@dandye

dandye commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

@prachib29 Could you please review this backward-compatible stability fix when you have a chance? It resolves HTTP 200 response data loss in nl_search and search_udm when :udmSearch returns statistical results, and all CI checks are green. Thank you.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(chronicle): nl_search and search_udm drop :udmSearch stats payload and misreport 0 results

1 participant