Skip to content

refactor(layout): batch1 low-risk relocations - #591

Merged
delete-cloud merged 3 commits into
mainfrom
refactor/layout-batch1
Jun 14, 2026
Merged

delete-cloud merged 3 commits into
mainfrom
refactor/layout-batch1

Conversation

@delete-cloud

@delete-cloud delete-cloud commented Jun 13, 2026 •

Copy link
Copy Markdown
Owner

What

Batch 1 of the package-layout consolidation: relocate 6 low-risk (≤2 refs) top-level modules in src/coding_agent/ into their cohesive subpackages. Old import paths are preserved via deprecation shims — no call-site changes, so this is a pure move.

old path new path
local_durable_store.py stores/durable_local.py
pg_durable_store.py stores/durable_pg.py
storage_migration.py stores/migration.py
external_executor.py executors/external.py
workspace_archive.py environment/archive.py
scheduled_runs.py runs/scheduled.py

Compat mechanism

Each old path is now a shim that aliases the moved module via sys.modules[__name__] = _moved, preserving every public/private name and isinstance identity. from coding_agent.<old> import X keeps working.

One eager old-path import in environment/docker_workspace_provider.py was repointed to .archive to break a circular import (otherwise the workspace_archive shim could not use the standard alias form).

Verification

  • uv run pytest -q: 3725 passed, 8 skipped, 20 xfailed
  • Fresh-process import check passes for all 6 old paths.
  • coding_agent.workspace_archive is coding_agent.environment.archive → True in both import orderings.

Spec / task packet: .opencode/prompts/tasks/package-layout-consolidation-task-packet.md. This is batch 1 of 4.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added support for external executors including Docker, Kubernetes, and Argo Workflow integrations.
    • Added durable scheduling system with automatic trigger planning and proactive signal support.
    • Added workspace archiving utilities for packaging and restoring workspace state.
  • Chores

    • Reorganized storage modules to improve modularity and maintainability.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR reorganizes storage persistence, execution, and scheduling subsystems into dedicated modules with comprehensive durable backing. It introduces workspace archive utilities, external executor adapters with PostgreSQL persistence, scheduled-run planning with proactive signals, and owner-fenced SQLite/PostgreSQL durable stores, while converting legacy modules into backward-compatible shims.

Changes

Storage, Execution, and Scheduling Refactoring

Layer / File(s) Summary
Workspace Archive Utilities
coding-agent/src/coding_agent/environment/archive.py, coding-agent/src/coding_agent/environment/docker_workspace_provider.py
Archive creation packs workspace files into gzip-compressed tar with size/member limits and symlink rejection; extraction validates base64, decompresses with byte-counting, rejects path traversal, and reconstructs with preserved modes/timestamps. Path exclusion filters preserved root names and unsafe paths.
External Executor System and PostgreSQL Run Store
coding-agent/src/coding_agent/executors/external.py
ExternalExecutor and ExecutorRunStore protocols with adapters: LocalExecutorAdapter executes local runners; DockerExecutorAdapter, KubernetesJobExecutorAdapter, ArgoWorkflowExecutorAdapter render manifests and map external status to internal results. Immutable dataclasses enforce validation. PGExecutorRunStore provides SQL-driven run CRUD, filtering, pagination, and evidence serialization. Plan construction, authorization signatures, workspace hashing, and extensive validation/sanitization helpers included.
Scheduled Run Planning and PostgreSQL Store
coding-agent/src/coding_agent/runs/scheduled.py
ScheduleRecord, ScheduleTriggerRecord, ProactiveSignalRecord dataclasses with validation. ScheduledRunPlanner fetches due schedules, creates planned triggers, advances status with cadence handling. ProactiveSignalPlanner applies cooldown/skip logic. ScheduledRunLaunchPreparer resolves or creates topics and constructs run metadata. PGScheduledRunStore lazily creates tables, provides async CRUD/upsert for schedules/triggers/signals with row conversion and comprehensive validation including safe-label enforcement and recursive metadata checks.
SQLite Durable Storage with Owner Fencing
coding-agent/src/coding_agent/stores/durable_local.py
SQLiteLocalDurableStore wraps SQLite tables with owner-lease authorization and fencing tokens. Session persistence validates JSON payloads and enforces consistency. Tape binding prevents rebinding; entry append/load/truncate use sequence tracking. Runtime durability enforces session ownership on run/event/snapshot/interaction operations. Executor run claiming filters by metadata and transitions to claimed status. Checkpoint save/restore clears and reinserts tape entries. FencedSQLiteTapeStore, FencedSQLiteCheckpointStore, FencedSQLiteRuntimeStore wrappers derive authority and delegate to durable layer.
PostgreSQL Durable Storage with Owner Fencing
coding-agent/src/coding_agent/stores/durable_pg.py
PGDurableStore adds row-level locking and lease/epoch validation via SQL constants. Lazy schema initialization creates required tables. Session operations include tape binding and payload upsert. Tape append/truncate verify owner and stable tape. Runtime operations (create/update runs, append events, save snapshots, create/resolve interactions) enforce ownership via locking queries. Executor run claiming atomically transitions status while merging metadata. Checkpoint save/delete/restore includes tape truncation and entry reinsertion. Centralized ownership helpers and database validators included. FencedPG*Store wrappers provide authority resolution.
Legacy Storage Migration Utilities
coding-agent/src/coding_agent/stores/migration.py
StoreMigrationReport and LegacySQLiteMigrationReport dataclasses track scanned/migrated/skipped counts. migrate_legacy_storage_to_sqlite orchestrates tape (JSONL) and checkpoint (FS) migrations. migrate_jsonl_tapes_to_sqlite reads *.jsonl, compares to SQLite to skip unchanged, conditionally replaces via truncation. migrate_fs_checkpoints_to_sqlite loads FS snapshots, compares to SQLite, saves new items. _read_jsonl_objects parses UTF-8 JSONL with dict validation and contextual errors. dry_run mode suppresses writes.
Module Deprecation Shims for Backward Compatibility
coding-agent/src/coding_agent/external_executor.py, coding-agent/src/coding_agent/local_durable_store.py, coding-agent/src/coding_agent/pg_durable_store.py, coding-agent/src/coding_agent/scheduled_runs.py, coding-agent/src/coding_agent/storage_migration.py, coding-agent/src/coding_agent/workspace_archive.py
Each legacy module now imports its new location and reassigns sys.modules[__name__] to redirect all former imports, preserving backward compatibility.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • delete-cloud/Coding-Agent-Loop#349: Refactors external executor protocols and PostgreSQL run store into coding_agent/executors/external.py with external_executor.py becoming a shim; includes related executor model and store tests.
  • delete-cloud/Coding-Agent-Loop#413: Adds SQLite migration implementation and CLI command for legacy storage migration, directly extending the storage migration module refactoring in this PR.

🐰 A rabbit hops through code so grand,
Storage, schedules, executors planned!
With fences and shims, the refactor's sublime,
New modules dance, old paths realigned. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'refactor(layout): batch1 low-risk relocations' directly summarizes the main change: moving six modules into subpackages as batch 1 of a layout consolidation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/layout-batch1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (2)
coding-agent/src/coding_agent/stores/durable_pg.py (1)

727-738: 💤 Low value

Unused for_update parameter.

The for_update parameter is accepted but immediately deleted and ignored—the query always uses FOR UPDATE regardless. Either remove the parameter or implement the conditional behavior.

♻️ Proposed fix to remove unused parameter
     async def _session_id_for_run(
         self,
         connection: Any,
         run_id: str,
-        *,
-        for_update: bool,
     ) -> str | None:
-        del for_update
         row = await connection.fetchrow(self._SELECT_RUN_SESSION_FOR_UPDATE_SQL, run_id)
         if row is None:
             return None
         return _required_str(dict(row), "session_id")

Also update the caller at line 746-749:

         session_id = await self._session_id_for_run(
             connection,
             run_id,
-            for_update=True,
         )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/src/coding_agent/stores/durable_pg.py` around lines 727 - 738,
The method _session_id_for_run currently accepts a for_update parameter but
immediately deletes it and always executes _SELECT_RUN_SESSION_FOR_UPDATE_SQL
(which implies FOR UPDATE); either remove the unused for_update parameter from
_session_id_for_run and update all callers to stop passing that argument (and
optionally rename _SELECT_RUN_SESSION_FOR_UPDATE_SQL to remove the “FOR_UPDATE”
wording), or implement the conditional behavior by keeping for_update and
switching the query used (e.g., choose between
_SELECT_RUN_SESSION_FOR_UPDATE_SQL and a non-locking _SELECT_RUN_SESSION_SQL) so
the SQL reflects the for_update value; locate and update any callers that pass
the extra argument to match the chosen fix.
coding-agent/src/coding_agent/executors/external.py (1)

477-485: 💤 Low value

Intentional broad exception catch – consider adding a comment.

The bare Exception catch (flagged by Ruff BLE001) is appropriate here: the runner may raise arbitrary exceptions from user code, and sanitizing these prevents leaking sensitive information. A brief comment would clarify the intent.

📝 Suggested comment
-        except Exception as exc:
+        except Exception as exc:  # Catch-all: sanitize unexpected runner errors
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/src/coding_agent/executors/external.py` around lines 477 - 485,
Add a short inline comment above the "except Exception as exc:" block explaining
that the broad Exception catch is intentional to safely capture and sanitize any
errors coming from user code (Ruff BLE001 is acceptable here) and prevent
leaking sensitive details; mention that the handler returns a sanitized
ExecutorResult (see the ExecutorResult(...) construction with sanitized_summary
and metadata "phase": "local_runner_failed") and therefore no raw exception info
is propagated.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@coding-agent/src/coding_agent/environment/archive.py`:
- Around line 36-46: The archive path checks use check-then-use filesystem calls
(path.is_symlink(), path.is_file(), path.stat(), path.read_bytes()) which are
vulnerable to TOCTOU if the workspace is mutated concurrently; fix by acquiring
the exclusive workspace lock around the entire walk-and-read sequence (or pause
the container) and, for each file, perform no-follow checks and revalidation
before IO: use lstat/no-follow semantics to detect symlinks, verify the file's
resolved realpath remains inside the expected workspace_root (e.g., compare
Path.resolve(strict=True) or os.fstat on an opened FD) immediately before read,
open or read with follow_symlinks=False (or via an FD opened with O_NOFOLLOW) to
prevent symlink races, and keep calling _raise_if_archive_too_large as before;
apply the same locking + no-follow/revalidate pattern to the analogous block at
lines 147-157.
- Around line 31-50: The archive creation loop doesn't enforce the member-count
cap (_MAX_WORKSPACE_ARCHIVE_MEMBERS), so many tiny files can produce an archive
that later fails extraction; add a member counter (e.g., member_count = 0 before
the loop), increment it each time you add a file (where total_input_bytes is
updated / before archive.addfile), and if member_count >
_MAX_WORKSPACE_ARCHIVE_MEMBERS raise the same error used on extract (or raise
ValueError) to stop creation; reference the constant
_MAX_WORKSPACE_ARCHIVE_MEMBERS and ensure the check is performed in the loop
alongside the existing total_input_bytes/_raise_if_archive_too_large check.

In `@coding-agent/src/coding_agent/runs/scheduled.py`:
- Around line 163-175: The summary field is declared as required (summary: str)
but __post_init__ calls _require_optional_display_text which allows None; change
the validation to enforce non-None by replacing the call to
_require_optional_display_text("summary", self.summary) with a required
validator (e.g., _require_display_text("summary", self.summary)) or add an
explicit check like if self.summary is None: raise ValueError(...) before
calling the existing validator so that Summary in runs/scheduled.py (inside
__post_init__) cannot be None at runtime.
- Around line 77-84: The _SECRET_VALUE_MARKERS tuple in runs/scheduled.py is
missing several markers and treats "sk-" differently from the logic used
elsewhere; update _SECRET_VALUE_MARKERS to include "bearer ", "gho_", "ghp_",
"kubeconfig", and "private key" and change the detection logic used when
scanning folded values to use substring matching (e.g., treat any marker found
via "marker in folded" rather than only folded.startswith("sk-")), so the
behavior of the _SECRET_VALUE_MARKERS constant and its usage matches the
executor path (or alternatively add a clear comment documenting the intentional
difference if you must keep distinct behavior). Ensure you modify the scanning
code that uses _SECRET_VALUE_MARKERS in runs/scheduled.py to use the same
membership check as executors/external.py (and update tests/comments
accordingly).

In `@coding-agent/src/coding_agent/stores/migration.py`:
- Around line 130-140: The checkpoint migration currently overwrites existing
checkpoints whenever existing != snapshot; update the migration logic in the
function that iterates checkpoints (references: target_store.load,
target_store.save, SQLiteCheckpointStore, variables existing and snapshot) to
respect a new boolean parameter replace_checkpoints (default false) or, if you
prefer the minimal change, emit a warning log and skip overwriting when
replace_checkpoints is false; specifically, add the replace_checkpoints
parameter to the migration call site and signature, check it before calling
await target_store.save(snapshot) (only save when replace_checkpoints is true or
existing is None), and when skipping an overwrite emit a warning with
identifying checkpoint_id so behavior matches the tape migration semantics.

---

Nitpick comments:
In `@coding-agent/src/coding_agent/executors/external.py`:
- Around line 477-485: Add a short inline comment above the "except Exception as
exc:" block explaining that the broad Exception catch is intentional to safely
capture and sanitize any errors coming from user code (Ruff BLE001 is acceptable
here) and prevent leaking sensitive details; mention that the handler returns a
sanitized ExecutorResult (see the ExecutorResult(...) construction with
sanitized_summary and metadata "phase": "local_runner_failed") and therefore no
raw exception info is propagated.

In `@coding-agent/src/coding_agent/stores/durable_pg.py`:
- Around line 727-738: The method _session_id_for_run currently accepts a
for_update parameter but immediately deletes it and always executes
_SELECT_RUN_SESSION_FOR_UPDATE_SQL (which implies FOR UPDATE); either remove the
unused for_update parameter from _session_id_for_run and update all callers to
stop passing that argument (and optionally rename
_SELECT_RUN_SESSION_FOR_UPDATE_SQL to remove the “FOR_UPDATE” wording), or
implement the conditional behavior by keeping for_update and switching the query
used (e.g., choose between _SELECT_RUN_SESSION_FOR_UPDATE_SQL and a non-locking
_SELECT_RUN_SESSION_SQL) so the SQL reflects the for_update value; locate and
update any callers that pass the extra argument to match the chosen fix.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 86b81e17-2231-4884-904f-2c05b4ebc9de

📥 Commits

Reviewing files that changed from the base of the PR and between cf07d79 and 75b613b.

📒 Files selected for processing (13)
  • coding-agent/src/coding_agent/environment/archive.py
  • coding-agent/src/coding_agent/environment/docker_workspace_provider.py
  • coding-agent/src/coding_agent/executors/external.py
  • coding-agent/src/coding_agent/external_executor.py
  • coding-agent/src/coding_agent/local_durable_store.py
  • coding-agent/src/coding_agent/pg_durable_store.py
  • coding-agent/src/coding_agent/runs/scheduled.py
  • coding-agent/src/coding_agent/scheduled_runs.py
  • coding-agent/src/coding_agent/storage_migration.py
  • coding-agent/src/coding_agent/stores/durable_local.py
  • coding-agent/src/coding_agent/stores/durable_pg.py
  • coding-agent/src/coding_agent/stores/migration.py
  • coding-agent/src/coding_agent/workspace_archive.py

@delete-cloud
delete-cloud merged commit 771ef0a into main Jun 14, 2026
2 checks passed
@delete-cloud
delete-cloud deleted the refactor/layout-batch1 branch June 14, 2026 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant