Skip to content

docs(adr): durable session retention + graceful drain (0070) + ArgoCD GitOps (0071) - #608

Merged
delete-cloud merged 4 commits into
mainfrom
docs/adr-session-durability-gitops
Jun 16, 2026
Merged

delete-cloud merged 4 commits into
mainfrom
docs/adr-session-durability-gitops

Conversation

@delete-cloud

@delete-cloud delete-cloud commented Jun 16, 2026 •

Copy link
Copy Markdown
Owner

Two Proposed ADRs. Revised after an independent + codex code review corrected ADR-0070's root cause.

ADR-0070 — Durable session retention and graceful drain

Corrected root cause: the REVISION 6 rollout appeared to lose sessions, but create-time persistence already exists (POST /sessions persists synchronously at session_manager.py:2754) and neither shutdown nor startup deletes rows. The real deleter is the 30-min idle GC: cleanup_idle_sessions → close_session → _remove_session_async → delete_session destructively deletes durable sessions, restart-independent.

Decision:

  • D1 (primary): idle GC must only shut down runtime + release lease, never delete durable metadata/tape/runs/checkpoints. Destructive delete only on explicit user/admin close.
  • D2: graceful drain (SIGTERM/preStop/terminationGracePeriodSeconds) as in-flight-turn hardening — not the row-loss cause.
  • D3 (optional): runtime rehydration + best-effort interrupted-turn resume (ADR-0055); no promise to recover a killed turn process.
  • Dropped the original create-time-persistence proposal (already implemented).

ADR-0071 — ArgoCD GitOps deployment

Unchanged in substance (pure ArgoCD, private values repo, immutable :<sha>, manual sync). Minor fixes: dependency wording → "D1 idle-GC fix + D2 graceful drain"; replaced a nonexistent test name with the real apply-mode placeholder-rejection tests.

Implementation will land as separate PRs after this docs PR merges — a #607 runtime-config atomicity hotfix (separate), then D1, then D2, then ArgoCD (after D1/D2 are live). This PR is docs only.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Architecture documentation updated for session lifecycle management during system updates.
    • GitOps-based deployment strategy documented for automated release consistency.
  • Infrastructure & Reliability

    • Session persistence improved to prevent data loss during rollouts.

ADR-0070: wire the existing durable layer (ADR-0055/0067/0068) into a
restart-recovery closed loop so single-replica live HTTP sessions survive a
process restart — create-time persistence (P1), graceful drain (P2), boot
rehydration + interrupted-turn resume (P3). Multi-replica HA explicitly out
of scope. Motivated by live session loss on REVISION 6 rollout + the #606
config-checksum auto-roll putting restart on the hot path.

ADR-0071: adopt ArgoCD pull-based GitOps as the target deploy mechanism
(private values repo, immutable :sha promotion via Image Updater, manual
sync with diff preview), replacing manual SSH helm upgrade. Includes the
deployment contract. P2 graceful drain is the prerequisite.

Both Proposed; implementation pending.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 16, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 584366f3-db6b-4558-ad5b-0d18606aeb39

📥 Commits

Reviewing files that changed from the base of the PR and between 70060e9 and bc0b468.

📒 Files selected for processing (2)
  • coding-agent/docs/adr/0070-restart-safe-live-sessions.md
  • coding-agent/docs/adr/0071-argocd-gitops-deployment.md
✅ Files skipped from review due to trivial changes (2)
  • coding-agent/docs/adr/0071-argocd-gitops-deployment.md
  • coding-agent/docs/adr/0070-restart-safe-live-sessions.md

📝 Walkthrough

Walkthrough

Two Architecture Decision Records are updated and added to coding-agent/docs/adr/. ADR-0070 corrects the root cause of lost sessions from missing persistence to destructive idle-GC deletion of durable metadata, and reframes the decision into three slices: D1 (preserve metadata), D2 (SIGTERM drain hardening), and D3 (optional lazy rehydration). ADR-0071 specifies replacing manual helm upgrade with ArgoCD pull-based GitOps using immutable image tags and a private values repository.

Changes

ADR-0070: Restart-Safe Live Sessions

Layer / File(s) Summary
Root cause diagnosis and D1–D3 decision framework
coding-agent/docs/adr/0070-restart-safe-live-sessions.md
Pinpoints the actual mechanism: idle GC periodically deletes durable session metadata rows once sessions exceed the configured idle threshold, not missing create-time persistence. Reframes the decision from P1–P3 to D1–D3: D1 preserves durable rows/metadata (release runtime/owner-lease only); D2 adds SIGTERM drain hardening and Helm terminationGracePeriodSeconds/preStop configuration; D3 treats rehydration and interrupted-turn resume as optional, leveraging existing store-backed listing and lazy hydration.
Alternatives, acceptance criteria, and implementation roadmap
coding-agent/docs/adr/0070-restart-safe-live-sessions.md
Updates rejected alternatives to reflect the corrected premise (create-time persistence already exists; the contract violation is destructive idle deletion). Adjusts acceptance criteria to mark D1 and D2 as pending separate PRs after this docs change. Enumerates concrete test cases for idle-cleanup preservation, shutdown + lease release, lazy rebuild, explicit deletion, SIGTERM drain, and Helm drain config. Updates references with exact code/Helm artifact locations for idle GC, session lifecycle, and drain behavior.

ADR-0071: ArgoCD GitOps Deployment

Layer / File(s) Summary
Full ArgoCD GitOps ADR
coding-agent/docs/adr/0071-argocd-gitops-deployment.md
Documents the end-to-end deployment decision: ArgoCD Application with --take-ownership Helm release adoption, private Git repository as the source of truth for real Helm values, ArgoCD Image Updater for immutable :<sha> tag promotion and commit-back, manual sync workflow with diff/preview before apply, secret-by-name-only constraints, environment-specific values requirements, and a binding deployment contract referencing ADR-0070 D1/D2 as pre-deploy safety gates. Enumerates rejected alternatives and acceptance criteria covering sync/health status, image/tag correctness, rollback restoration, and required existing Helm chart and deploy-script test/guard suites.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

  • delete-cloud/Coding-Agent-Loop#161: Defines durable remote session and workspace retention semantics that directly overlap with the restart-safe session persistence model and root-cause analysis in ADR-0070.

Poem

🐇 Two scrolls unveil the path once lost,
Idle GC was the hidden cost—
Sessions persist through restarts anew,
ArgoCD pulls what's right and true,
With SHA tags and graceful drains,
The rabbit skips through safer lanes! 🌿

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: two ADRs documenting durable session retention with graceful drain (0070) and ArgoCD GitOps deployment (0071).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/adr-session-durability-gitops

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
coding-agent/docs/adr/0071-argocd-gitops-deployment.md (1)

1-105: ⚠️ Potential issue | 🟠 Major

Correct release/namespace name in acceptance criteria and fix incomplete criterion formatting.

The ADR structure and decision rationale are sound, but two acceptance-criteria issues need correction:

  1. Line 84: Incorrect release name. The criterion hardcodes coding-agent-deepseek, but the deploy script defaults to coding-agent (both HELM_RELEASE and K8S_NAMESPACE default to coding-agent, not coding-agent-deepseek). Update line 84 to use coding-agent, or clarify if coding-agent-deepseek is an o6n-prod-specific override with different environment-specific naming.

  2. Line 92–93: Incomplete acceptance criterion. The final criterion is a bare test command. Rephrase it as a criterion description, e.g., "Existing deploy-script and helm-chart tests pass: uv run pytest tests/deploy/test_deploy_script.py tests/deploy/test_helm_chart.py -q".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/docs/adr/0071-argocd-gitops-deployment.md` around lines 1 - 105,
In the Acceptance Criteria section, two corrections are needed: First, on line
84, replace the hardcoded release name `coding-agent-deepseek` with
`coding-agent` to match the defaults in the deploy script (where both
HELM_RELEASE and K8S_NAMESPACE default to `coding-agent`), unless this is an
environment-specific override that should be clarified in the criterion text.
Second, on lines 92-93, the final acceptance criterion is incomplete and
consists only of a bare test command; rephrase it as a proper criterion
description (e.g., starting with an action verb and outcome) that includes the
test command, such as "Existing deploy-script and helm-chart tests pass: `uv run
pytest tests/deploy/test_deploy_script.py tests/deploy/test_helm_chart.py -q`".
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@coding-agent/docs/adr/0070-restart-safe-live-sessions.md`:
- Around line 84-100: The pytest command at line 100 in the acceptance criteria
has incorrect file paths that lack the `coding-agent/` prefix required for this
monorepo structure. Update the command to use the correct paths relative to the
repository root: change `tests/coding_agent/test_session_restart_durability.py
tests/deploy/test_helm_chart.py` to
`coding-agent/tests/coding_agent/test_session_restart_durability.py
coding-agent/tests/deploy/test_helm_chart.py`. Alternatively, you may preface
the command with an instruction to execute from the `coding-agent/` directory.
The `-k "restart or drain or rehydrate"` filter pattern remains correct and
safe.

---

Outside diff comments:
In `@coding-agent/docs/adr/0071-argocd-gitops-deployment.md`:
- Around line 1-105: In the Acceptance Criteria section, two corrections are
needed: First, on line 84, replace the hardcoded release name
`coding-agent-deepseek` with `coding-agent` to match the defaults in the deploy
script (where both HELM_RELEASE and K8S_NAMESPACE default to `coding-agent`),
unless this is an environment-specific override that should be clarified in the
criterion text. Second, on lines 92-93, the final acceptance criterion is
incomplete and consists only of a bare test command; rephrase it as a proper
criterion description (e.g., starting with an action verb and outcome) that
includes the test command, such as "Existing deploy-script and helm-chart tests
pass: `uv run pytest tests/deploy/test_deploy_script.py
tests/deploy/test_helm_chart.py -q`".
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 463ab193-5337-4a7e-9183-63248ec63296

📥 Commits

Reviewing files that changed from the base of the PR and between 154d638 and 70060e9.

📒 Files selected for processing (2)
  • coding-agent/docs/adr/0070-restart-safe-live-sessions.md
  • coding-agent/docs/adr/0071-argocd-gitops-deployment.md

Comment thread coding-agent/docs/adr/0070-restart-safe-live-sessions.md Outdated
Independent code review (and codex re-review) found ADR-0070's original P1
premise wrong: POST /sessions already persists synchronously at create
(session_manager.py:2754), and neither shutdown nor startup deletes session
rows. The real cause of the lost durable session row is the 30-min idle GC:
cleanup_idle_sessions -> close_session -> _remove_session_async -> delete_session
destructively deletes durable sessions. Restart-independent.

Rewrite 0070 around that: D1 (primary) idle GC must only shut down runtime +
release lease, never delete durable metadata/tape/runs/checkpoints — destructive
delete only on explicit user/admin close. P2 kept as in-flight-turn graceful-
drain hardening (not the row-loss cause). P3 downgraded to optional runtime
rehydration/resume (no promise to recover a killed turn process). Drop the
create-time-persistence proposal (already implemented).

Minor 0071 fixes: dependency wording -> 'D1 idle-GC fix + D2 graceful drain';
replace the nonexistent test_deploy_script_fails_closed_on_placeholder_values
with the real apply-mode placeholder-rejection test names.

Docs only; no runtime code changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@delete-cloud delete-cloud changed the title docs(adr): restart-safe live sessions (0070) + ArgoCD GitOps deploy (0071) docs(adr): durable session retention + graceful drain (0070) + ArgoCD GitOps (0071) Jun 16, 2026
delete-cloud and others added 2 commits June 16, 2026 15:09
- 0070: restart surfaces (not immediately triggers) the loss — cleanup task
  sleeps 60s before first pass.
- 0070 D1: add ownership note — if idle cleanup releases the owner lease, the
  next prompt must reacquire (current path only _assert_owners).
- 0070 D3: resume is best-effort from durable history/context per ADR-0055, not
  a checkpoint rollback (was overstated).
- 0070 acceptance: fix test path tests/coding_agent -> tests/ui.
- 0071: separate the idle-GC row-loss cause (D1) from the drain gap (D2);
  drop the claim that manual SSH 'caused' the session loss (idle GC did).

Docs only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Clarify the ArgoCD sync namespace: coding-agent-deepseek is the o6n-prod
  override (chart/deploy-script default is coding-agent), not a hardcoding error.
- Rephrase the bare pytest command into a full acceptance-criterion sentence.

Docs only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@delete-cloud
delete-cloud merged commit f89f44a into main Jun 16, 2026
2 checks passed
@delete-cloud
delete-cloud deleted the docs/adr-session-durability-gitops branch June 16, 2026 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant