[al3xanndru]

Problem

PC-5227: a pre-execution PII guardrail check made the agent run fail. All five POST agentsruntime_/api/execution/guardrails/validate attempts ended in httpx.ReadTimeout at about 30.05 s each, and the run failed with AGENT_RUNTIME.UNEXPECTED_ERROR after 168 s. This happened in prd, paid.eus: job fda68d58-abb3-4c44-8d76-21cf1e2c425a on 2026-10-13 and job 81cda36c on 2026-10-12. It was found by the triage of SRE-679237.

On the server, the same ValidateGuardrail calls took 30.1, 120.1, 120.1, 91.6 (200) and 30.0 s while LLMOps PII detection was returning 429s (LLMOPS-3371). So the server did return a verdict, but the client had already given up.

Root cause

• GuardrailsService.evaluate_guardrail passes a longer timeout (60 s) only when attachments are present. A text-only check uses the client's default timeout=30.0 from common/_http_config.py, which is shorter than the server's 120 s budget.
• BaseService.request retries every httpx.TimeoutException up to MAX_RETRY_ATTEMPTS = 5. That includes ReadTimeout, where the request was already sent and the server is still working on it. Each retry sends the same input again and starts a new server-side PII chain. That adds load to an already throttled quota and times out again for the same reason.

Change

• _guardrails_service.py: every validate call now uses httpx.Timeout(30.0, read=150.0). Text-only checks, empty-attachment checks and attachment checks all get it. Connect, write and pool keep the 30 s default, and the read budget is above the server's 120 s. This replaces _ATTACHMENT_VALIDATE_TIMEOUT_SECONDS = 60.0, so attachment checks also go from 60 s to 150 s.
• The validate call goes through BaseService.request.retry_with(...). It keeps request's wait and stop strategy but swaps the predicate for the new is_retryable_platform_exception_except_read_timeout.
• common/retry.py: adds is_retryable_platform_exception_except_read_timeout. It behaves like is_retryable_platform_exception but returns False for ReadTimeout. ConnectError, ConnectTimeout, WriteTimeout, PoolTimeout, the retryable statuses (408, 429, 502, 503, 504, 524) and 5xx responses are all still retried.
• Bumps uipath-platform 0.2.41 → 0.2.42 and updates the two lockfile entries, following the repo's convention.

The worst-case wait is about the same as before: one read of up to 150 s, against five reads of 30 s plus backoff today. The difference is that a server answer that takes 30 to 120 s now arrives in time.

Tests

• tests/services/test_guardrails_service.py: TestGuardrailAttachmentTiming is replaced by TestGuardrailValidateTiming:
  • The read timeout is 150 s and the connect timeout is 30 s for text-only, empty-attachment and attachment checks (parametrized).
  • A ReadTimeout raises after exactly one request.
  • A ConnectError is still retried, and the second request succeeds.
• tests/services/test_retry.py: tests for the new predicate cover ReadTimeout, ConnectTimeout, ConnectError, 429 and 400.
• Without the source change, the new timeout tests (all three parametrizations) and the no-resend test fail. The connect-retry test passes with and without it, as intended.
• In packages/uipath-platform: pytest → 1898 passed, 11 skipped. ruff check . and ruff format --check . are clean. mypy src tests reports no issues in 233 files.

Risks / open questions

• The 120 s server budget was observed, not read from code. Two server-side ValidateGuardrail requests ended at 120.1 s with status 0. I couldn't find the limit in the Agents backend (it may be an ingress or gateway timeout). If the real cap is different, _VALIDATE_TIMEOUT should be adjusted.
• retry_with is a tenacity attribute that tenacity doesn't type, hence one # type: ignore[attr-defined]. The alternative was a per-call retry switch on the shared BaseService.request, which seemed too broad for this fix.
• The guardrail node still fails closed on infrastructure errors. That is intentional for PII and unchanged.
• Not in this PR:
  • uipath-agents-python's exception_mapper has no httpx.TimeoutException branch, so a timeout is still reported as Unknown rather than System.
  • In uipath-langchain 0.16.22, the guardrail node calls the sync client from inside the async node, which blocks the event loop. That is already fixed in 0.18.9.

---

Drafted by the wabbit-dri draft-fixes loop for PC-5227 (draft #1), for a human to review: not ready to merge as is.

🤖 Generated with Claude Code

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud