Repository navigation
feat(ci): automated proto drift detection and SDK sync notifications #2825
Description
User Story
As a contributor changing proto definitions, I want to know immediately if my changes affect any SDK, so that I can regenerate the stubs myself or trust that SDK maintainers will be notified automatically. Contributors shouldn't need to understand every SDK's internals to make a proto change.
As an SDK maintainer, I want to be notified when proto changes introduce fields or RPCs that my SDK's converter layer doesn't yet expose, enabling planned and prioritized SDK update work without manually tracking every proto change.
Problem Statement
OpenShell maintains four SDKs (Go, TypeScript, Python, Rust) across different teams. Proto file changes have varying impacts per SDK:
Go is unique in committing generated proto stubs to the repository. Go's source-level dependency model (go get clones and compiles sources directly) necessitates this. When protos change, committed stubs must be regenerated and re-committed. The existing go:proto:check CI task verifies stub consistency but is a simple pass/fail gate without notifications.
TypeScript, Python, and Rust all generate stubs at build time and gitignore them. TypeScript uses buf generate (output to src/gen/, gitignored). Python runs grpc_tools.protoc via a python:proto task before builds. Rust uses tonic-prost-build in a build.rs script at compile time. Stub drift is structurally impossible for these three SDKs.
All four SDKs share a deeper problem: each has a hand-written domain type layer (converters, typed wrappers, From impls) mapping proto types into idiomatic language constructs. A new proto field compiles and tests pass, but the field is silently absent from the SDK's public API. SDK consumers discover missing fields only through trial and error. This "converter coverage gap" grows silently as proto definitions evolve.
| SDK | Stubs Committed? | Generation Tool | Domain Type Layer | Stub Drift Risk |
|---|---|---|---|---|
| Go | Yes (source-level deps require it) | buf generate |
types/ + converter/ |
Yes |
| TypeScript | No (src/gen/ gitignored) |
buf generate |
Curated API + raw escape hatch |
No |
| Python | No (_proto/ gitignored) |
grpc_tools.protoc |
sandbox.py dataclass wrappers |
No |
| Rust | No (compile-time build.rs) |
tonic-prost-build |
types.rs with From impls |
No |
Impact / Why This Matters
Current behavior: Proto changes land without any notification to SDK maintainers. For Go, contributors must know to regenerate and commit stubs. For all four SDKs, new proto fields are silently ignored by the domain type layer.
Current workaround: SDK maintainers manually monitor proto changes by watching the commit log. This is error-prone and doesn't scale.
Why the workaround is insufficient: As the contributor base grows, proto changes come from many people who may not be aware of SDK implications. SDK consumers depend on proto fields being exposed through the SDK. Silent gaps create downstream surprises.
Proposed Design
Stub drift detection + issue lifecycle
Scope: Go SDK (the only SDK committing generated stubs). Infrastructure is designed to be extensible if other SDKs' build models change.
- Per-SDK mise tasks (
go:proto:drift) that regenerate stubs in a temp directory and compare against committed files, outputting structured JSON - A PR-triggered GitHub workflow annotating PRs with drift warnings (non-blocking, informational). Contributors see messages like: "Your proto change affects SDK stubs. The Go SDK stubs are 3 files behind."
- A daily cron workflow that:
- Runs drift detection for each SDK
- If drift is found, runs a build verification pipeline (regenerate, build, test)
- Creates or updates a GitHub issue per SDK (deduplicated by
sdk:{sdk}:synclabel, matching the repo's existingarea:sdk:{language}convention) - Auto-closes issues when drift resolves
- A Python script for issue body generation and lifecycle management via
gh
This makes contributions transparent without blocking anyone. Contributors see warnings; SDK maintainers get actionable issues with fix commands.
Extensibility
The system should use a config-driven approach where adding a new SDK requires only:
- A drift detection mise task for that language
- A config entry with display name, source directories, and build/test tasks
This pattern scales to all current and future SDKs.
Future extensions
Converter coverage analysis — compare proto message fields against what each SDK's converter/domain-type layer actually maps, reporting fields present in proto but absent from SDK type mappings. This would detect the deeper "business logic gap" that stub comparison cannot catch. Scope would cover all four SDKs.
Acceptance Criteria
- PR-triggered workflow annotates proto-touching PRs with SDK drift warnings (Go; extensible)
- Daily cron creates issues when drift is detected, with fix commands and affected file lists
- Issues auto-close when drift resolves
- Currently covers Go SDK stub drift (only SDK that commits stubs); matrix design supports future SDKs
- Adding a new SDK requires only a mise task + config entry
- Unit tests cover issue body generation and lifecycle management
Alternatives Considered
Fail CI on proto drift: Rejected. This would block proto contributors until every affected SDK is updated. Contributors should be able to land proto changes without understanding SDK internals. SDK maintenance is the SDK maintainer's responsibility, triggered by notifications.
Wiki dashboard: Considered and deferred. Wiki pages have low discoverability. PR annotations and auto-filed issues are more visible to those who need to act.
Integrate into existing branch-checks.yml: Worth discussing. A "Proto Drift" row alongside existing checks increases visibility. A separate workflow is easier to iterate on initially and can be promoted later.
Extend stub drift to TypeScript/Python/Rust: Not needed. All three generate stubs at build time and gitignore them, making stub drift structurally impossible. Go is the only SDK where stubs are committed. All four SDKs would benefit from the converter coverage analysis extension.
Activity
elezar commentedon Aug 20, 2026
Ygnas commentedon Aug 20, 2026
Scope
This proposal only covers in-tree SDKs. Right now that means Go and TypeScript. Python and Rust regenerate stubs at build time, so they don't need stub drift detection. Adding a new SDK takes a mise task and a config entry.
Triage questions
1. What versioned contract does an SDK pull?
The contract is
mainHEAD. All the in-tree SDKs live in the same repo as the proto definitions, so the daily cron and PR checks both compare against whateverproto/looks like on the branch ormain.2. How does an SDK record which contract revision it supports?
For Go, the committed stubs (
sdk/go/proto/*.pb.go) are the record. They're a snapshot of the last-synced proto revision. TypeScript generates stubs at build time and gitignores them, so the record is implicit: whatevermainHEAD compiles against.3. How does the SDK detect stub and converter gaps?
Two layers, both on this branch.
Go is the only SDK that commits stubs, so it's the only one with actual stub drift. The
go:proto:driftmise task regenerates to a temp directory and diffs against committed files. If something changed, that's drift.For both Go and TypeScript, there's also build verification: regenerate bindings, then run build and test. If the build fails, the converter layer has a gap. The
go:proto:build-checkandsdk:ts:proto:build-checktasks handle this in the cron workflow. TypeScript is a bit different here because stubs are gitignored, so its drift detection is really just build verification. Thesdk:ts:proto:drifttask regenerates protos and typechecks the handwritten SDK code against them.Converter coverage analysis (comparing proto message fields against what the SDK actually maps) is a future extension. That would catch silent gaps where the build passes but a new field isn't in the SDK's public API.
4. How does the SDK create, update, and resolve findings?
The cron workflow manages issues in the OpenShell repo using
ghCLI withGITHUB_TOKEN. Issues are deduplicated bysdk:{name}:synclabels and auto-closed when drift resolves. Everything stays in one repo, no cross-repo credentials needed.5. Should
go:proto:checkstay?Yes.
go:proto:checkis a CI gate that prevents merging stale committed stubs. It answers "are the.pb.gofiles current?" The drift notification workflow answers a different question: "has proto evolved past what the SDK exposes?" They serve different purposes and both checks are needed independently.Ygnas commentedon Aug 24, 2026
@elezar I answered all five triage questions in my comment above. Is something else needed to move this from needs-info? If anything is not clear I am happy to explain more.
elezar commentedon Aug 27, 2026
Thanks, that clarifies the scope.
Given the follow-up, I think this can move out of
state:needs-infoif we frame the issue as an in-tree SDK automation task rather than a general compatibility contract for out-of-tree SDKs.The accepted scope should be:
go:proto:checkas the existing blocking producer-side integrity check.I would leave out-of-tree SDK compatibility and converter field-coverage analysis as follow-up issues/RFC material. They are related, but they require a different contract model than this issue needs to implement.
rhuss commentedon Sep 3, 2026
Additional coverage layers: closing gaps in Go SDK proto-to-API chain
During the review of #3123, we traced the Go SDK's coverage chain and found gaps worth capturing for the converter coverage analysis phase.
What exists today:
coverage_test.gouses protobuf reflection to verify that all fields on specific message types are handled by converters. This catches new fields added to existing messages (e.g., a newpause_timefield onSandboxStatusfails the test). However, each message type needs a hand-written test function with ahandledfield set. There is no auto-discovery.What falls through:
coverage_test.gofails)This means a PR that adds a new RPC (e.g.,
PauseSandboxwith new request/response messages) passes CI without any converter or curated client method. The raw gRPC method exists in the generated stubs, but the SDK's public API silently omits it.Proposed: three-layer reflection-based coverage chain
coverage_test.go), but per-message, not auto-discoveredprotoreflectto scan service descriptors, compare againstClientInterfacemethodsreflectto scan client interface signatures for domain type referencesLayer 2 would also close the new-message-type gap: if every RPC must have a curated client method, and the client method must reference domain types, then the message types used by that RPC are forced into the converter layer.
All three layers could follow the same self-maintaining pattern as the existing field coverage test: reflection-based scanning with a
skippedset for intentional exceptions. No manual lists to keep in sync. Failures surface at test time.This extends the "converter coverage analysis" described in the original issue. Worth a dedicated issue if it's in scope.
rhuss commentedon Sep 3, 2026
Design constraint: Go stubs make hard vs. soft gating non-trivial
Following up on the coverage layers comment above. There's an important design constraint to consider before implementing RPC-level coverage checks.
The problem: The Go SDK commits generated stubs to the repo. Any proto-changing PR regenerates
sdk/go/proto/*.pb.go, which meanssdk/go/is always touched. This forces all Go SDK tests to run on every proto-changing PR, includingcoverage_test.go.For field-level coverage (the existing
coverage_test.go), this is fine and intentional: if you add a field toSandboxStatus, the converter must handle it or CI breaks. The scope is narrow and the fix is mechanical.But if we extend this to RPC-level coverage (every new RPC must have a curated client method + converter), the same mechanism would block every proto-changing PR until the full Go SDK implementation is done. A platform developer adding a new RPC would be forced to also write the Go SDK wrapper, converter, and domain types, or their PR fails. This is the exact friction the original issue rejected for stub drift.
Proposed approach: split the checks by enforcement level
coverage_test.goThe soft checks could be implemented as either:
continue-on-error: true(visible on PRs but non-blocking)This preserves the existing behavior (field coverage is hard, proto contributors aren't blocked by SDK completeness) while adding visibility for the deeper coverage gaps.
Ygnas commentedon Oct 5, 2026
Scope decision for #3123: we are removing the daily issue-management workflow and keeping the PR-time drift feedback.
Required CI already catches stale Go bindings and regenerates and type-checks TypeScript bindings. We have not identified a normal-path drift that the daily scan would catch uniquely, and automated issue creation could add low-signal backlog noise.
The daily issue lifecycle is intentionally deferred. Semantic SDK-surface coverage will be tracked as follow-up work. PR #3123 keeps
Closes #2825for the agreed initial scope of contributor-facing drift feedback.Ygnas commentedon Oct 13, 2026
Closing this issue for now. Required CI already catches mechanical proto drift, so the scheduled drift/issue-creation workflow would add maintenance without covering a gap CI misses. Semantic SDK coverage, such as proto RPCs not exposed by the curated SDK is separate follow-up work tracked in #3162.