Skip to content

refactor(table): extract file store factories - #516

Merged
lxy-9602 merged 2 commits into
apache:mainfrom
lucasfang:codex/table-store
Nov 6, 2026
Merged

lxy-9602 merged 2 commits into
apache:mainfrom
lucasfang:codex/table-store

Conversation

@lucasfang

Copy link
Copy Markdown
Collaborator

Purpose

Related issue: #459. First split from #447; this PR contains only the component-factory extraction.

Managed-table scan, read, write and commit construction repeats path, snapshot, manifest and index setup inside table.cpp. Extract that setup into an internal, operation-local FileStore, so the existing entry points delegate component construction without changing their loading or execution flow.

Scope: 4 files, 267 insertions and 131 deletions:

  • Add src/paimon/core/table/file_store.h/.cpp with factories for paths, snapshot managers, manifest lists/files, index manifests/handlers and partition computers.
  • Replace the corresponding construction blocks in table.cpp, preserving the selected branch, bucket mode, physical format, filesystem, cache and resource arguments.
  • Register the new implementation in CMake.

FileStore receives already resolved options and is local to each operation. It does not load schemas, parse options, cache managers, or become a member of Table. The existing nine CoreOptions::FromMap calls and six schema-validation calls remain unchanged and in the same order. Catalog snapshot loaders retain shared ownership of their catalog.

This PR does not include the Table hierarchy, Context/API changes, schema getter rename, Copy/time-travel fixes, or the proposed ExecutionContext/CatalogEnvironment design.

Tests

Local Debug/AddressSanitizer build with REST, Lance, Mosaic, ORC, Avro and Vortex enabled:

  • Full configured build passed.
  • All 155 focused Table/scan/read/commit tests passed.
  • All 2,626 core tests passed in the full CTest run.
  • Initial full CTest run: 34/36 targets passed. Append and primary-key compaction functional assertions passed, but LeakSanitizer reported Lance/Tokio allocations at process exit.
  • Both failed targets passed on rerun (2/2) with unchanged code and sanitizer configuration. Thus all 36 targets passed across the initial run and rerun; this was not a clean 36/36 first run.
  • The initial exit leaks were not stably reproduced. Main-code comparison runs also passed, so they are not claimed to be confirmed baseline defects. Their root cause remains unresolved; no suppression or test changes are included.
  • Changed-file pre-commit checks and git diff --check passed.

Commands:

cmake --build build -j 48
./build/debug/paimon-core-test --gtest_filter='TableTest.*:TableScanTest.*:TableReadTest.*:FileStoreCommitImplTest.*'
ctest --test-dir build --output-on-failure -j 12
ctest --test-dir build --output-on-failure -j 2 -R 'paimon-(append|pk)-compaction-inte-test'
pre-commit run --files src/paimon/core/table/file_store.h src/paimon/core/table/file_store.cpp src/paimon/core/table/table.cpp src/paimon/CMakeLists.txt
git diff --check

Existing behavior tests are retained without caller migration; no implementation-mirroring factory tests are added.

API and Format

No public API, storage format or wire protocol changes. Table remains concrete, existing Contexts and static factory signatures remain unchanged, and format/system-table dispatch stays in place. Option parsing, validation, schema loading, commit selection and Copy behavior are outside this extraction.

Documentation

No user-facing feature is introduced. The new internal class documents its operation-local factory role; user documentation does not change.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-6)

Centralize path, snapshot, manifest, index and partition component construction in an operation-local FileStore. Preserve existing option parsing, validation, dispatch and commit behavior.
Comment thread src/paimon/core/table/file_store.cpp Resolved
Comment thread src/paimon/core/table/table.cpp Resolved
Comment thread src/paimon/core/table/table.cpp Resolved
Comment thread src/paimon/core/table/table.cpp Resolved

@lszskye lszskye left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602 lxy-9602 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602
lxy-9602 merged commit caba0ff into apache:main Nov 6, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants