feat(providers): add gateway-owned credential refresh for short-lived provider tokens

Problem Statement

OpenShell needs first-class gateway-owned credential refresh for providers that use short-lived access tokens. The current provider model can inject static credentials into sandbox traffic, and recent provider work added provider profiles, attach/detach, custom profiles, and provider environment revision polling for running sandboxes. However, OpenShell does not yet own the lifecycle for refresh material such as OAuth refresh tokens, client secrets, or service-account private keys.

This blocks long-running sandboxes that depend on providers such as Microsoft Graph / Azure / Entra-backed APIs, Google Workspace, Google Vertex AI, IBM watsonx, ChatGPT/OpenAI OAuth-style integrations, and other enterprise SSO-bound APIs. Access tokens commonly expire during a sandbox session. Users should not have to restart sandboxes, expose refresh material inside the sandbox, or build one-off host refresh daemons.

Refresh material must stay in the OpenShell control plane / gateway. Sandboxes should continue to see only placeholder values or proxy-injected short-lived access tokens, and should observe refreshed access tokens through the existing provider environment revision path.

Proposed Design

Add gateway-owned provider credential refresh while keeping placeholder-based credential injection as the delivery path for this first chunk.

The core model should be:

Store refresh material in a separate non-injectable refresh-state object type in the existing objects table.
Keep Provider.credentials as the injectable current credential map used by GetSandboxProviderEnvironment.
Add refresh metadata to provider profile credential declarations so profiles can describe how a credential is renewed.
Add a gateway refresh worker that reads refresh-state objects, refreshes due credentials, writes the resulting short-lived access token into the owning provider's injectable credential key, and updates the provider object so the provider environment revision changes.
Rely on the existing sandbox provider-env revision polling and ProviderCredentialState mechanics so running sandboxes observe refreshed credentials without restarting.
Keep refresh tokens, client secrets, service-account JSON, private keys, and equivalent refresh material out of Provider.credentials and out of GetSandboxProviderEnvironment.

Initial refresh strategies should include:

static: no active minting. Provider credential updates are picked up through the current provider environment revision path.
external: OpenShell does not mint tokens. An external process updates the provider credential store, and running sandboxes pick up the update through the revision path.
oauth2_refresh_token: gateway refreshes an access token using stored refresh-token material.
oauth2_client_credentials: gateway mints access tokens using client credentials, covering Microsoft S2S / Entra-style flows.
google_service_account_jwt: gateway signs a JWT assertion using stored service-account material and exchanges it for a Google access token, covering Vertex/GCP service-account use cases.

The first implementation should prioritize Microsoft S2S / Entra, Google OAuth refresh-token, and Google service-account / Vertex flows. The design should leave room for IBM watsonx IAM, AWS STS, OIDC variants, and other future refresh strategies without coupling them to the first implementation.

User-facing UX should include a way to inspect refresh status, force rotation, and update refresh configuration without exposing refresh material in normal provider output. Candidate CLI shape:

openshell provider refresh-status <provider>
openshell provider rotate <provider> [--credential <name>]
openshell provider refresh-config <provider> ...

Provider create/update should accept refresh material separately from injectable credentials, so users cannot accidentally place refresh tokens or service-account keys in the sandbox-visible credential map.

Related Requirements

Enhanced Provider Management #896 tracks the overall provider v2 design, including declarative refresh strategies, max lifetime, lead time, fail-closed expiry, status/rotate UX, and refresh telemetry.
feat(sandbox): supervisor should periodically refresh provider credentials for running sandboxes #1264 tracks the running-sandbox last-mile refresh requirement. Current code already has provider environment revisions and supervisor polling; this issue should use that path.
Provider-managed OAuth credential refresh #1066 proposed provider-managed OAuth credential refresh with refresh material staying outside the sandbox.
feat: credential refresh trait for short-lived provider tokens (Vertex AI first implementation) #472 proposed a gateway refresh trait for short-lived inference provider tokens, including Google Vertex, Azure OpenAI / Entra ID, and IBM watsonx.
NemoClaw issues OpenAI/ChatGPT OAuth is not a viable NemoClaw provider path without manual host-side OpenClaw gateway bridging NemoClaw#1793, [Linux][Brev]Ollama-local provider returns HTTP 401 Unauthorized via inference.local — wizard caches OPENAI_API_KEY env var into credentials.json and not refreshable via env unset NemoClaw#2519, ollama proxy token diverges from stored token after re-onboard, causing persistent HTTP 401 on inference NemoClaw#2553, [Onboard] upgrade-sandboxes preflight requires stale credential env after provider switch from remote to local NemoClaw#2625, and [Ollama-local] Gateway routing proxy does not inject NEMOCLAW_OLLAMA_PROXY_TOKEN as Bearer on :11435 — silent hang on clean machine (continuation of #2519) NemoClaw#3198 show adjacent credential-state problems: stale cached credentials, local/no-auth providers being treated as remote auth, and runtime injection mismatches. This OpenShell design should explicitly model no-auth, static, external-refresh, and gateway-refreshing credentials so stale or non-applicable credentials do not keep affecting providers.

Alternatives Considered

Store refresh tokens directly in Provider.credentials. Rejected because Provider.credentials is the injectable map returned through GetSandboxProviderEnvironment; refresh material must not enter the sandbox.
Wait for profile-side/proxy-side credential injection before refresh. Rejected because placeholder-based injection can already deliver refreshed access tokens, and short-lived token demand is immediate.
Require host-side refresh daemons. Rejected because this duplicates provider-specific logic, fails for long-running autonomous sandboxes, and moves lifecycle ownership outside OpenShell.
Add a new database table for refresh state. Rejected for this scope; profile and provider follow-up work has kept provider storage in the existing objects table.
Implement a single provider-specific refresher first. Rejected because Microsoft S2S, Google OAuth, and Google service-account flows are already known requirements and need a shared scheduler/state model.

Agent Investigation

The current repo already has much of the sandbox-side propagation path needed for refreshed credentials:

Provider.credentials is the current secret map on provider records.
GetSandboxProviderEnvironment resolves attached providers into an environment map and returns provider_env_revision.
crates/openshell-sandbox/src/provider_credentials.rs maintains revision-scoped credential snapshots and keeps recent generations so existing placeholders continue to resolve.
The sandbox settings poll loop refreshes provider environment when provider_env_revision changes and installs the new environment into ProviderCredentialState.
The proxy resolves placeholders per request through SecretResolver, so updated provider credential values can affect later proxied requests without mutating already-running process environments.
Provider profiles currently define credentials, endpoints, binaries, category, and inference capability, but they do not yet define refresh metadata.
Custom provider profiles are stored in the existing objects table; this issue should use the same persistence boundary for refresh-state objects rather than adding new tables.
Existing OpenShell CLI OIDC login code already contains OAuth refresh-token helper logic for gateway authentication, but provider refresh needs separate gateway-side storage, scheduling, status, and provider credential updates.

Non-Goals

Do not replace placeholder-based credential injection in this issue.
Do not implement profile-side/proxy-side credential injection in this issue.
Do not expose refresh tokens, client secrets, service-account JSON, or private keys through provider env resolution.
Do not add new database tables.
Do not make already-running process environments mutable.
Do not implement every possible provider strategy in the first PR; keep the strategy interface extensible.

🏗️ build-plan

Implementation Plan

Issue type: feat
Complexity: High
Confidence: Medium-High — the storage, refresh ownership, external strategy, and profile metadata boundaries are now decided; the remaining risk is implementation breadth across gateway, CLI, scheduler, and tests.

Summary

Add gateway-owned refresh for short-lived provider tokens by keeping Provider.credentials as the current injectable access-token map and moving refresh material into a separate non-injectable object stored in the existing objects table. The gateway refresh worker should update only the provider record on successful refresh so the existing GetSandboxProviderEnvironment revision flow, sandbox ProviderCredentialState, and inference route resolution continue to work without sandbox-side redesign.

Resolved Design Decisions

external refresh is push-only: OpenShell does not fetch from external brokers or run scripts. External systems update current credentials through the provider update path.
provider update remains the path for externally rotated current credentials, with a new general expiry option such as --credential-expires-at KEY=TIMESTAMP.
Credential expiry metadata is general for any provider credential and must apply to static, external, and gateway-refreshed credentials.
provider rotate means “force gateway-owned refresh now” and applies to gateway-managed strategies, not normal external refresh pushes.
Refresh metadata lives on ProviderProfileCredential, because refresh lifecycle is per credential rather than per provider profile.
User-facing refresh commands address credentials by injectable credential key, for example MS_GRAPH_ACCESS_TOKEN, not by profile-internal credential name such as access_token.
The gateway maps an injectable key back to the matching ProviderProfileCredential by loading the provider profile and finding the credential whose env_vars contains that key.
A new provider credential refresh-state object owns runtime refresh material/status and links back to the provider through objects.scope = provider_id; the payload also stores provider_id, provider_name, and credential_key for self-description.
objects.scope is a logical/application-enforced FK to the provider instance, not a database FK.
Expiry must be enforced during credential resolution, not only displayed in status. Expired refresh-managed credentials should fail closed even if old sandbox placeholder generations still exist.

Codebase Mapping

proto/datamodel.proto, proto/openshell.proto: add the new stored provider-refresh object, refresh status/config RPCs, provider profile refresh metadata, and per-credential expiry metadata surfaces.
crates/openshell-core/src/metadata.rs: register object traits for the new stored refresh object.
crates/openshell-providers/src/profiles.rs, providers/*.yaml: extend profile schema and validation with ProviderProfileCredential.refresh; seed Microsoft and Google defaults first.
crates/openshell-server/src/grpc/provider.rs, crates/openshell-server/src/grpc/mod.rs, new crates/openshell-server/src/provider_refresh.rs, crates/openshell-server/src/lib.rs: persist refresh material/status, expose refresh UX RPCs, run the scheduler, and write refreshed access tokens back into Provider.credentials.
crates/openshell-server/src/grpc/policy.rs, crates/openshell-server/src/inference.rs: reuse existing provider-env and inference resolution; add expiry-aware credential resolution and regression coverage proving provider updates trigger new revisions and refreshed inference credentials.
crates/openshell-cli/src/main.rs, crates/openshell-cli/src/run.rs, crates/openshell-cli/tests/*: add refresh-status, rotate, refresh-config, and provider update expiry support while keeping refresh material separate from normal injectable credentials.

Implementation Steps

Define the data model and API surface: add a dedicated non-injectable provider-refresh object in the existing objects table, add refresh strategy metadata to ProviderProfileCredential, add per-credential expiry metadata, and add minimal RPCs for refresh-status, refresh-config, and rotate.
Implement gateway persistence and validation: store refresh material and status separately from Provider.credentials, use objects.scope = provider_id for refresh-state ownership, preserve existing provider read contracts, and keep CreateProvider/UpdateProvider scoped to injectable current credentials plus general credential expiry metadata.
Add provider update expiry support: allow current credentials to be updated with optional credential_key -> expires_at metadata, and ensure expiry metadata can be set for any credential without changing the Provider.credentials map shape.
Add the refresh engine: introduce a background gateway worker that scans configured refresh objects, schedules refresh before expiry, serializes refresh per provider credential, and implements static, external, oauth2_refresh_token, oauth2_client_credentials, and google_service_account_jwt, with first-pass coverage for Microsoft S2S/Entra, Google OAuth refresh-token, and Google service-account/Vertex.
Reuse the existing credential propagation path: on successful gateway-owned refresh, update Provider.credentials[credential_key] and expiry metadata so existing provider env revision polling, sandbox placeholder rotation, and inference-route resolution observe new credentials without mutating running process envs.
Enforce expiry at resolution time: expired credentials should not resolve into sandbox requests, including retained old placeholder generations, and should return clear fail-closed diagnostics.
Wire CLI UX: add openshell provider refresh-status, openshell provider refresh-config, and openshell provider rotate; add provider update --credential-expires-at KEY=TIMESTAMP; use injectable credential keys in user-facing command arguments.
Finish with targeted profile and docs updates: add refresh metadata to prioritized profiles first, then document the new refresh lifecycle and user-facing commands.

Test Plan

Unit tests: profile serde/validation for refresh metadata; new object metadata/accessors; strategy-specific refresh helpers; scheduler timing/backoff/state transitions; redaction tests proving refresh material never appears in provider read APIs.
Integration tests: gateway provider tests for update vs refresh-config separation, successful refresh updating Provider.credentials, failed refresh preserving last-known-good credentials until expiry, expiry fail-closed behavior, compute_provider_env_revision changing when the provider record changes, and inference route resolution seeing rotated provider tokens.
CLI integration tests: parsing and execution of new provider refresh commands, provider update --credential-expires-at, injectable-key-based credential selection, and clear error cases for invalid refresh-config/material input.
E2E tests: one running sandbox attached to a refresh-managed provider, backed by a fake token issuer, proving the gateway refresh updates the provider record and the sandbox observes the new provider_env_revision through the existing poll loop; add inference-route coverage if the refreshed provider is used for managed routes.

Docs Impact

Update architecture/gateway.md with the provider refresh ownership boundary and background worker lifecycle.
Update docs/sandboxes/manage-providers.mdx for refresh configuration, status, rotation, external push refresh, expiry metadata, and supported strategies.
Update docs/sandboxes/inference-routing.mdx if refreshed provider credentials become part of the supported inference path.

Security Risks

CWE-200: refresh tokens, client secrets, and service-account keys must stay non-injectable, redacted from all read APIs, and excluded from logs and OCSF messages.
CWE-918: if token endpoints or issuer URLs are configurable, validate them against profile metadata or an explicit allowlist so the gateway refresh worker cannot become an SSRF primitive.
CWE-362: refresh must be single-writer per provider credential; failed or concurrent refreshes must not clobber the current credential map or cause unnecessary revision churn.
Expiry enforcement must account for retained sandbox credential generations so stale tokens do not remain usable after their recorded expiry.

Revision 2 — resolved external strategy, ProviderProfileCredential metadata, provider update expiry, refresh-state ownership, and expiry enforcement decisions
Revision 1 — initial plan

We are hitting this in a downstream integration (a desktop app that manages OpenShell sandboxes in WSL), adding a data point and a question on the build plan.

The #1349 foundation works well for us (gateway-owned rotation of Azure DevOps OAuth tokens via ConfigureProviderRefresh). The gap we still see on v0.0.90 is that after the gateway rotates a credential, long-lived processes inside the sandbox (VS Code server, open PTYs) hold their originally captured placeholder and their git operations fail from then on, the egress proxy drops the TLS connection, so users see opaque connection failures mid-session.
Our workaround is a "kill VS Code server" button so users can restart it to re-capture placeholders.

One question on the plan, after a successful refresh, does a placeholder captured at sandbox start keep resolving to the current credential indefinitely? i.e. is recognition of old markers unbounded (keyed per credential) or a bounded recent-generations list? Our tokens rotate roughly hourly and editor/PTY processes could live for days, so if old markers either stay bound to their original token value or eventually fall off a bounded retention window, processes started before a rotation still break, just later.

@tarabishy2020

@tarabishy2020 looks like your issue should have been solved by #2777 - will close out this issue and let us know if you see the issue again.

Navigation Menu

Repository navigation

feat(providers): add gateway-owned credential refresh for short-lived provider tokens #1306

Problem Statement

Proposed Design

Related Requirements

Alternatives Considered

Agent Investigation

Non-Goals

Implementation Plan

Summary

Resolved Design Decisions

Codebase Mapping

Implementation Steps

Test Plan

Docs Impact

Security Risks

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

feat(providers): add gateway-owned credential refresh for short-lived provider tokens #1306

Description

Problem Statement

Proposed Design

Related Requirements

Alternatives Considered

Agent Investigation

Non-Goals

Activity

johntmyers commented on May 11, 2026

Implementation Plan

Summary

Resolved Design Decisions

Codebase Mapping

Implementation Steps

Test Plan

Docs Impact

Security Risks

tarabishy2020 commented on Jul 28, 2026

johntmyers commented on Aug 19, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions