Skip to content

feat: migrate legacy storage to sqlite - #413

Merged
delete-cloud merged 1 commit into
mainfrom
feat/sqlite-storage-migration
Jun 1, 2026
Merged

delete-cloud merged 1 commit into
mainfrom
feat/sqlite-storage-migration

Conversation

@delete-cloud

@delete-cloud delete-cloud commented Jun 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • add a legacy storage migration module for JSONL tape files and FS checkpoints
  • add python -m coding_agent storage migrate-sqlite with default /data-style paths and explicit override options
  • make migration idempotent: matching existing SQLite rows are skipped, mismatched tapes fail unless --replace-tapes is passed, and --dry-run does not create SQLite files

Verification

  • uv run pytest tests/coding_agent/test_sqlite_storage_migration.py tests/cli/test_entrypoint_contract.py tests/agentkit/storage/test_sqlite.py tests/coding_agent/plugins/test_storage.py tests/ui/test_session_manager_public_api.py -v -> 112 passed
  • uv run ruff check <changed python files> --output-format=concise -> All checks passed
  • uv run ruff format --check <changed python files> -> 4 files already formatted
  • git diff --check -> passed

Summary by CodeRabbit

  • New Features
    • New storage migration command for migrating legacy data to SQLite format
    • Configurable data directories and dry-run mode for testing migrations
    • Option to replace existing data during migration
    • Reports migration statistics including scanned, migrated, and skipped item counts

@coderabbitai

coderabbitai Bot commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR adds a complete storage migration feature enabling users to migrate legacy JSONL-based tapes and filesystem checkpoint snapshots into SQLite-backed stores. The migration module includes data models, orchestration logic, two specialized migration paths, and a CLI command with comprehensive test coverage.

Changes

Legacy Storage to SQLite Migration

Layer / File(s) Summary
Migration report models and orchestration
coding-agent/src/coding_agent/storage_migration.py
StoreMigrationReport and LegacySQLiteMigrationReport dataclasses track scan/migrated/skipped counts. migrate_legacy_storage_to_sqlite coordinator wires tape and checkpoint migrations together, resolves default paths from AGENT_DATA_DIR, and forwards replace_tapes and dry_run options.
Tape and checkpoint migration logic
coding-agent/src/coding_agent/storage_migration.py
migrate_jsonl_tapes_to_sqlite scans *.jsonl files in sorted order, parses entries, detects conflicts, supports conflict resolution via replace, and tracks counters. migrate_fs_checkpoints_to_sqlite iterates *.meta.json files, loads FSCheckpointStore snapshots, compares to existing SQLite entries, and persists with counters. _read_jsonl_objects provides strict JSONL parsing with UTF-8 decoding, blank-line skipping, and contextual error reporting.
CLI storage command group and migrate-sqlite subcommand
coding-agent/src/coding_agent/cli/main.py
New storage group with migrate-sqlite subcommand accepts filesystem/SQLite path options, --replace-tapes and --dry-run flags. Resolves default data directory from AGENT_DATA_DIR (defaulting to ./data), executes migration via asyncio.run, and prints per-category scanned/migrated/skipped statistics.
End-to-end migration module tests
coding-agent/tests/coding_agent/test_sqlite_storage_migration.py
Tests verify tape idempotency (second run skips unchanged entries), tape conflict detection with ValueError, tape dry-run with no file creation, checkpoint round-trip preservation, checkpoint dry-run behavior, and legacy storage migration with default paths loading both tapes and checkpoints correctly.
CLI integration and contract tests
coding-agent/tests/cli/test_entrypoint_contract.py
CLI help text updated to expect storage subcommand. New test for storage migrate-sqlite mocks the migration function, verifies output includes count statistics, and confirms the function is invoked with correct argument mappings from flags.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A rabbit hops through storage files,
Old tapes and snapshots in a trail,
To SQLite they migrate with smiles,
No data lost along the way!
CLI wires it all with care. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding functionality to migrate legacy storage to SQLite.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/sqlite-storage-migration

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@delete-cloud
delete-cloud merged commit 398c175 into main Jun 1, 2026
1 check was pending
@delete-cloud
delete-cloud deleted the feat/sqlite-storage-migration branch June 1, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant