Skip to content

feat(server): implement grpc.health.v1.Health/Watch streaming RPC #2569

Open

Description

@bmahabirbu

Problem Statement

Clients that need to observe gateway health changes (e.g., Kaiden, potential Podman Desktop extensions, dashboards, operators) currently have no event-driven mechanism. The only options are:

  • openshell status -o json — polls the unary Health RPC, returns connected/disconnected
  • openshell gateway info -o json — polls GetGatewayInfo, returns healthy/degraded/unhealthy
  • HTTP /readyz — poll-based, and disabled by default for local gateways

All of these require polling. For integrations that need to react to gateway state transitions (healthy → degraded, connected → disconnected), polling adds latency and unnecessary load.

The standard gRPC health checking protocol defines a Watch streaming RPC for exactly this purpose.

Proposed Design

Implement the standard grpc.health.v1.Health/Watch server-streaming RPC using tonic-health.

What's already in place:

  • The /grpc.health. path prefix is already in the unauthenticated bypass list (crates/openshell-server/src/auth/oidc.rs:33), so no auth changes are needed.
  • The ServiceStatus enum (Healthy, Degraded, Unhealthy) already exists in the proto and SDK.
  • The DatabaseHealthMonitor in crates/openshell-server/src/readiness.rs already maintains a tokio::sync::watch channel with health state — this can drive the tonic-health reporter.

Implementation outline:

  1. Add tonic-health dependency to openshell-server.
  2. Register the grpc.health.v1.Health service on the gRPC server alongside the existing OpenShell service.
  3. Wire the DatabaseHealthMonitor's watch channel to the tonic-health reporter so that Watch streams reflect actual readiness state (not just "up = healthy").
  4. Expose the Watch RPC in the SDK client as a streaming health subscription.
  5. Optionally surface it in the CLI as openshell status --watch or openshell status --follow.

Behavior:

  • Watch("") (empty service) returns overall gateway health.
  • Status transitions (Serving → NotServing) are pushed to subscribers when the database health monitor detects a change.
  • Existing unary Health and GetGatewayInfo RPCs remain unchanged.

Alternatives Considered

  • Polling openshell status — works but adds latency (seconds) and CPU cost for tight polling intervals. Acceptable as a fallback but not ideal for reactive integrations.
  • WebSocket health endpoint — non-standard, requires a separate HTTP upgrade path, harder for gRPC-native clients to consume.
  • Custom streaming RPC on the OpenShell service — unnecessary when the standard grpc.health.v1 protocol already exists and tooling supports it.

Agent Investigation

  • Confirmed the grpc.health.v1.Health/Watch RPC is not implemented — no tonic-health dependency exists in the project.
  • Confirmed the auth bypass for /grpc.health. is already configured at crates/openshell-server/src/auth/oidc.rs:33 with a test at line 416.
  • Confirmed the DatabaseHealthMonitor (crates/openshell-server/src/readiness.rs) already maintains real-time health state via tokio::sync::watch that could drive the reporter.
  • The current unary Health handler (crates/openshell-server/src/grpc/mod.rs:224-232) unconditionally returns Healthy — this proposal would also improve that by connecting it to actual readiness state.

Activity

  1. github-actions commented on Oct 18, 2026

    @github-actions

    This issue has had no activity for 14 days and is now marked stale. It may be closed in 14 days if there is no further activity. Comment or remove the state:stale label to keep it open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:staleInactive item at risk of automatic closure.state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions