Repository navigation
Conversation
Target: 50% instruction / 40% branch coverage via two-phase approach (command/function audit + JaCoCo gap fill) with ratchet schedule. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…leanup Bucketization failed permanently for several clin-hg19-prd tables. The temp dictionary copy is named ".<table>.temp.bucketizing.<rand>.gord", and opening it as a table makes TableInfoBase derive a metadata folder of "..<table>.temp.bucketizing.<rand>" (the name is already dot-prefixed). TableLifeCycleSupport then creates a history/ subdir inside it, so the folder is non-empty. cleanTempBucketData deleted every root entry matching the temp prefix with FileReader.delete, which is Files.deleteIfExists underneath and throws DirectoryNotEmptyException on that folder. The exception is retried until the handler gives up and bucketize aborts, leaving another temp folder behind on every run. Branch on entry type and use the recursive deleteDirectory for folders, the same way TableLifeCycleSupport.delete() already does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
|
|
||
| from __future__ import annotations | ||
|
|
||
| import os |
There was a problem hiding this comment.
Not addressed here: this file was never part of the intended change. The branch was cut from a commit that is not on origin/main, so unrelated work was included by mistake. This PR is closed in favour of #163, which contains only the BucketManager fix and its test.
🤖 Addressed by Claude Code
| """ | ||
|
|
||
| from __future__ import annotations | ||
| import re, sys |
There was a problem hiding this comment.
Not addressed here: this file was never part of the intended change. The branch was cut from a commit that is not on origin/main, so unrelated work was included by mistake. This PR is closed in favour of #163, which contains only the BucketManager fix and its test.
🤖 Addressed by Claude Code
Junit Tests - Summary2 012 tests - 2 756 1 916 ✅ - 2 681 10m 39s ⏱️ - 7m 45s Results for commit db2e7df. ± Comparison against base commit 36bd22a. This pull request removes 2756 tests. |
Problem
Bucketization can fail permanently for a table, with no self-recovery: every run aborts while cleaning temp data, and buckets referenced in the dictionary are never finished.
Root cause
BucketManager.createTempTablewrites a temp dictionary copy named.<table>.temp.bucketizing.<rand>.gord— one leading dot.TableInfoBasesetsname = removeExtension(fileName)andfolderPath = resolve(rootUri, "." + name). The name is already dot-prefixed, so the metadata folder becomes..<table>.temp.bucketizing.<rand>— the double dot above.TableLifeCycleSupportcreates that folder plus ahistory/subdir, so it is a non-empty directory.cleanTempBucketDatalists the root path and callsfileReader.delete(...)on every entry containing the temp prefix. That matches both the.gordfile and the..directory.FileSource.deleteusesFiles.deleteIfExists, which throwsDirectoryNotEmptyExceptionon a non-empty directory. The exception is marked.retry(), the retry handler exhausts its attempts, andbucketizeaborts.Each failed run leaves another temp directory behind, so later runs fail the same way.
Fix
Branch on entry type in
cleanTempBucketDataand use the recursiveFileReader.deleteDirectoryfor folders — the same handlingTableLifeCycleSupport.delete()already does.Test
testCleaningOfOldTempDirsplants a temp metadata folder containinghistory/old.gord, then bucketizes. Before the fix it failed with the stack above (FileSource.delete<-cleanTempBucketData<-bucketize); after it passes. FullUTestBucketManageris green (17/17,:gortools:slowTest).Notes
VERSION5.11.3; line numbers differ on later branches...*.temp.bucketizing.*directories on disk. Those must be removed before bucketization can succeed again — not covered by this PR.GorResourceException.fromIOExceptiononly rewrites the message forFileNotFoundException, so this failure logs as a bare path with no explanation. Worth improving separately.ENGKNOW-4142
🤖 Generated with Claude Code