Skip to content

fix(server): downgrade expected connection teardown errors to debug - #1369

Merged
drew merged 1 commit into
NVIDIA:mainfrom
russellb:fix/server-connection-teardown-logs
May 14, 2026
Merged

drew merged 1 commit into
NVIDIA:mainfrom
russellb:fix/server-connection-teardown-logs

Conversation

@russellb

Copy link
Copy Markdown
Contributor

Summary

h2 connection errors during sandbox deletion are expected — the supervisor gRPC stream is torn down when the sandbox is removed. Downgrade these from ERROR/WARN to DEBUG so they don't pollute gateway logs during normal operations.

Related Issue

None — discovered during nftables migration e2e testing (#1335).

Changes

  • Add is_benign_connection_close() helper in lib.rs that matches expected h2/connection teardown error messages
  • Downgrade connection error logs from error! to debug! for both TLS and plaintext paths when the error matches a benign connection close
  • Downgrade ForwardTcp: inbound stream failed from warn! to debug! in grpc/sandbox.rs — stream teardown during sandbox deletion is expected

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated — verified gateway logs no longer show ERROR/WARN during normal sandbox lifecycle (create → exec → delete)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

@copy-pr-bot

copy-pr-bot Bot commented May 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@drew

drew commented May 14, 2026

Copy link
Copy Markdown
Collaborator

/ok to test 1d988d9

@drew

drew commented May 14, 2026

Copy link
Copy Markdown
Collaborator

tests look good, but needs a rebase

h2 connection errors during sandbox deletion are expected — the
supervisor gRPC stream is torn down when the sandbox is removed.
Downgrade these from ERROR/WARN to DEBUG so they don't pollute
gateway logs during normal operations.

Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
@russellb
russellb force-pushed the fix/server-connection-teardown-logs branch from 1d988d9 to 33449bb Compare May 14, 2026 15:34
@russellb

Copy link
Copy Markdown
Contributor Author

tests look good, but needs a rebase

thanks! rebase complete

@drew

drew commented May 14, 2026

Copy link
Copy Markdown
Collaborator

/ok test 33449bb

@drew
drew merged commit 94025d8 into NVIDIA:main May 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants