Skip to content
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,16 @@ change the default engine. See [installation, input validation and regression
instructions](docs/VERTEBRAE_HAO.md). Enable with `--vertebrae_engine shapekit_hao`
and `--vertebrae_only`; start with `--cpu_count 1` and a new output directory.

# Drift-Fix Vertebrae Engine (ShapeKit-Driftfix)

An optional `shapekit_driftfix` backend cleans up the 24 vertebra masks in five stages (specks and
far debris, detached fragments, label drift at the discs, a lost-level repair, and an order check that
only flags). The CT is optional; without it the engine runs mask-only and says so in its report. It does
not change the default engine. All five stages run by default; set
`vertebrae_driftfix_relabel: false` to skip the two that can change a vertebra's level (safer on spines with an L6 or T13). See
[usage, options and known limits](docs/VERTEBRAE_DRIFTFIX.md). Enable with
`--vertebrae_engine shapekit_driftfix` and `--vertebrae_only`, with a new output directory.

# Evidence-Gated Vertebrae Engine (ShapeKit-Pro)

The default vertebrae module works from the masks alone. ShapeKit can now
Expand Down
8 changes: 8 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,10 @@ if_save_combined_label: True
# requires matching calibrated CT and target_organs [vertebrae].
# Fails on grid mismatch/missing CT; no legacy fallback.
# See docs/VERTEBRAE_HAO.md for CLI overrides and regression tests.
# shapekit_driftfix - opt-in five-stage cleanup of 24 vertebra masks; CT optional
# (mask-only path if missing); requires target_organs [vertebrae].
# vertebrae_driftfix_relabel (below) can turn off the level-changing repairs.
# See docs/VERTEBRAE_DRIFTFIX.md.
# shapekit - default mask-based module (no CT needed)
# shapekit_songlin - VerSe-inspired iterative refinement loop (no CT needed)
# shapekit_pro - evidence-gated repair against the case CT: recolors
Expand All @@ -109,3 +113,7 @@ vertebrae_engine: shapekit_songlin # use iterative refinement as default option
# <ct_root>/<case_id>/<ct_file_name> when ct_root is set.
ct_file_name: ct.nii.gz
# ct_root: /path/to/original/ct/cases

# shapekit_driftfix only: true (default) runs all five stages, as scored on held-out scans.
# false skips the two stages that can change a vertebra's level (safer on spines with L6 or T13).
vertebrae_driftfix_relabel: true
164 changes: 164 additions & 0 deletions docs/VERTEBRAE_DRIFTFIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
# Drift-fix vertebrae post-processing (`shapekit_driftfix`)

An optional engine that cleans up 24-level vertebra masks (L5 ... C1) and flags what still looks wrong.
It is not the default and changes nothing else in ShapeKit. The algorithm is a vendored script,
`utils/vertebrae_driftfix_engine.py`, kept byte-for-byte as it was written. `utils/vertebrae_driftfix.py`
is a thin adapter that connects it to ShapeKit.

## What it does (five stages)

1. **Load and crop.** Read the labels (and the CT if there is one), cut away the empty space around the spine.
2. **Clean-up.** Delete tiny specks, decide what happens to detached fragments and far-away debris
(for example hip bone labelled C1), and drop label blobs that are far off the spine.
3. **Label drift.** If label boundaries cut through vertebral bodies instead of the discs, split the bone
at the discs and give each vertebra one label. Needs the CT.
4. **Sacrum and lost level.** Find the sacrum below the lowest vertebra. If it confirms L5 and the chain looks
one level too low above a duplicated label, restore the lost level.
5. **Order check.** Look at the vertebra centres along the head-foot axis and flag inverted or oversized steps.
This stage never changes a voxel.

Every change and every flag is written to the JSON report with its reason.

## Install

No new packages: it needs numpy, scipy, nibabel (and PyYAML for the config key), all already in
`requirements-vertebrae-hao.txt`. No separate requirements file is added.

## Input

```text
predictions/CASE/segmentations/vertebrae_L5.nii.gz
...
predictions/CASE/segmentations/vertebrae_C1.nii.gz
ct_root/CASE/ct.nii.gz (optional)
```

Masks must be binary, 3D, on one grid, and must not overlap. A missing level file is treated as empty and
recorded. If a CT is given and its grid differs from the masks, the run stops with an error. Any orientation is
accepted (the engine works in millimetres through the affine). No resampling is done.

If the CT is **missing**, the engine runs its mask-only path: the HU checks and the sacrum anchor are skipped,
and the report says `"ct_used": false` and `inputs.ct_status` starts with `missing`. This differs
from `shapekit_hao` and `shapekit_ct_refinement`, which raise an error. Check `ct_used` in the report.

## Run

Run from the ShapeKit root. Use a new output folder.

```sh
python main.py \
--vertebrae_engine shapekit_driftfix \
--vertebrae_only \
--input_folder /path/to/predictions \
--ct_root /path/to/ct_root \
--output_folder /path/to/new_results \
--case CASE \
--cpu_count 1 \
--log_folder /path/to/logs
```

`--vertebrae_only` is required, as for the other case engines.

## Options

One option, in `config.yaml`:

```yaml
vertebrae_driftfix_relabel: true # default
```

- The key is read from `config.yaml` in the current working directory, like the rest of ShapeKit's settings. If the
file is not found there, the default (`true`) is used.
- `true` (default): all five stages run. This is the configuration behind every held-out number on this page.
- `false`: stages 3 and 4, the two that can change which level a voxel belongs to, are skipped. This is the
script's `--no_relabel`. Clean-up and the order check still run. It has no held-out numbers of its own.

When to use false: on spines with an extra vertebra (L6 or T13) the lost-level repair can move many
vertebrae to the wrong level (see Known limits). If your data may contain such spines and nobody reviews
the flags, false is the safer setting. From Python you can also pass `relabel=True` or `False`
to `process_case`.

## Output

- All 24 binary masks (empty ones too) with the header, qform and sform of the input.
- `combined_labels.nii.gz` with ShapeKit ids (26 = L5 ... 49 = C1). The engine's own numbering is
1 = L5 ... 24 = C1 (`id = label + 25`).
- `vertebrae_driftfix_report.json`: the engine log (`changes`, `order_flags`, `qa_before`, `qa_after`,
`lost_level`, `label_drift`, `sacrum`, `timing_s`), plus `review_flags` (the entries that need a person to
look), `ct_used`, `relabel`, `label_map`, `missing_input_masks` and `engine_sha256_lf`.
- Other files in the input segmentation folder are not copied.
- The case is written into `.incomplete` and renamed at the end. An existing output folder is refused.

## Runtime and memory

Measured on an 8 GB Windows laptop, one worker, on the cropped demo case BDMAP_00000006
(278 x 322 x 290 voxels): about 25 s inside the adapter, about 28 s through `main.py`.
The author's notes record about 5 s for the full case 06 and about 30 s for the large case 31 on a larger
machine, and a median of 1.6 s per case (maximum 19 s) on a cloud CPU. Those times are for the standalone script on
one combined label file. Through ShapeKit the median was 14 s per scan in the benchmark below, because the adapter also
reads and writes the 24 per-level mask files. Peak memory was not measured.
The adapter holds the 24 masks as one uint8 volume and the engine works on a crop around the spine, so memory is
roughly a few copies of the full uint8 volume plus the CT as int16. Large 0.7 mm whole-spine scans were not
run.

## Known limits

Numbers below come from the author's held-out test of 100 CTs with ground truth (frozen script, scored once).
The write-up (`REPORT.md`, `eval/heldout2_results.md`) is in the author's own project, not in this repository.

- On 90 spines with typical anatomy: Dice 0.880 to 0.888 (gain +0.009, 95% CI +0.003 to +0.016).
Order violations per case fell from 12.4 to 3.9. The Dice gain is small; most of the gain is anatomical validity.
- On 10 spines with an L6 or T13: Dice fell from 0.577 to 0.506. Almost all of that is one scan where the
lost-level repair took the lowest vertebra to be L5 and moved 14 vertebrae one level the wrong way.
So **the lost-level repair can wreck a spine with an L6**. The label-drift repair also hurt one L6 scan.
- The model has only 24 classes, so L6 and T13 cannot be written.
- A simple rule that keeps only the largest piece of each label scored about as well on Dice (0.876 on the
90 typical spines, against 0.888 here); this engine leaves fewer order violations but more small pieces.
- On the two demo cases there is no ground truth. Checks there are about consistency (volumes, pieces, order),
not accuracy.
- On one scan a 21-voxel edge piece of a vertebra was removed as a speck, which took that level's Dice to 0.
- Not tested: large 0.7 mm scans; Python versions other than 3.13 locally (the benchmark ran on Kaggle's Python).

## Validation done for this integration

- `tests/test_vertebrae_driftfix.py`: label round trip, synthetic phantom, missing CT, existing output,
grid mismatch, overlap, non-binary, and voxel-for-voxel equality with the engine run directly, on the
phantom and on a cropped demo case (that test is skipped if `data/shapekit_demo` is not present).
- BDMAP_00000006 (cropped) through `main.py`: 794 voxels changed, 1 review flag. BDMAP_00000031 was not run
(too large for the 8 GB laptop).

These checks show the integration reproduces the standalone script. They are not an accuracy test.

## Benchmark against other engines

70 CT scans from TotalSegmentator v3.0.0 with public vertebra masks, none of them in SuPreM's training, validation or
test lists. Input to every engine: the raw SuPreM vertebra prediction (default inference command). Each engine was run
through `main.py` at upstream commit `ddfa8d9` on a Kaggle CPU machine (4 vCPU), then scored against the public masks.
Dice is the mean over the vertebrae present in the ground truth, averaged over scans. The interval is a 95% paired
bootstrap over scans (engine minus raw).

| engine | scans done | mean Dice | change vs raw [95% interval] | scans better / same / worse | level-identity rate | scans under 0.7 Dice | order violations | median s/scan |
|---|---|---|---|---|---|---|---|---|
| raw SuPreM | 70 | 0.8805 | - | - | 0.938 | 7 | 964 | - |
| shapekit | 70 / 70 | 0.8769 | -0.0036 [-0.0111, +0.0032] | 48 / 0 / 22 | 0.931 | 9 | 970 | 105 |
| shapekit_songlin | 70 / 70 | 0.8460 | -0.0345 [-0.0603, -0.0124] | 47 / 0 / 23 | 0.894 | 13 | 983 | 296 |
| shapekit_hao | 70 / 70 | 0.8809 | +0.0004 [-0.0016, +0.0023] | 55 / 0 / 15 | 0.942 | 7 | 957 | 28 |
| shapekit_ct_refinement | 70 / 70 | 0.8905 | +0.0100 [+0.0038, +0.0175] | 39 / 30 / 1 | 0.940 | 7 | 962 | 22 |
| shapekit_driftfix | 70 / 70 | 0.8892 | +0.0087 [+0.0017, +0.0176] | 59 / 2 / 9 | 0.944 | 6 | 287 | 14 |
| shapekit_pro (first 10 scans only) | 4 / 10 | 0.8561 (raw 0.8557 on the same 4) | +0.0005 [-0.0213, +0.0215] | 2 / 0 / 2 | 0.902 (raw 0.958) | 0 | 30 (raw 40) | 27 |

How to read it:

- `shapekit_ct_refinement` has the highest mean Dice and makes almost no scan worse (1 of 70). This engine is second
on Dice and makes 9 scans worse. The two intervals overlap, so the Dice order between them is not settled.
- This engine is the only one that removes most order violations (964 to 287). The others leave them almost unchanged.
- `shapekit_songlin` lowers Dice on this set, mostly through a few scans that get much worse (13 scans under 0.7).
- `shapekit_pro` was run on the first 10 scans by case id because of its documented run time. It crashed on 6 of them
with `IndexError` at `utils/vertebrae_pro_engine.py:815` (`s_axis[c]` with `c` equal to the array length, when a band
reaches the top of the cropped spine). Four scans are too few to rank it.
- The `shapekit_driftfix` output equals the standalone script's output voxel for voxel on all 70 scans.

Limits of this benchmark: one dataset, one segmentation model, 1.5 mm scans, and no scan with a sixth lumbar or a
thirteenth thoracic vertebra (on 10 such scans this engine lowers Dice, see "Known limits"). These 70 scans were scored
once before with the standalone script, after it was frozen; no engine was tuned on them. Peak memory was not measured.
Kernel scripts, logs and the scorer are available on request.
10 changes: 6 additions & 4 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from utils.vertebrae_pro import postprocessing_vertebrae_pro
from utils.vertebrae_hao import process_case as process_hao_case
from utils.vertebrae_ct_refinement import process_case as process_ct_refinement_case
from utils.vertebrae_driftfix import process_case as process_driftfix_case
import logging
import yaml
import traceback
Expand Down Expand Up @@ -228,15 +229,16 @@ def main(input_path, input_folder_name, output_path=None):
input_path: the folder path
"""

if vertebrae_engine in ('shapekit_hao', 'shapekit_ct_refinement'):
if vertebrae_engine in ('shapekit_hao', 'shapekit_ct_refinement', 'shapekit_driftfix'):
# Isolate the evaluated algorithm from legacy organ cleanup and IO.
# The strict adapter validates each mask/CT grid and writes zero masks.
if target_organs != {'vertebrae'}:
raise ValueError(f'{vertebrae_engine} requires --vertebrae_only')
ct_path = os.path.join(input_path, ct_file_name)
if not os.path.exists(ct_path) and ct_root is not None:
ct_path = os.path.join(ct_root, input_folder_name, ct_file_name)
processor = process_ct_refinement_case if vertebrae_engine == 'shapekit_ct_refinement' else process_hao_case
processor = {'shapekit_ct_refinement': process_ct_refinement_case,
'shapekit_driftfix': process_driftfix_case}.get(vertebrae_engine, process_hao_case)
return processor(
input_path, os.path.join(output_path, input_folder_name),
ct_path, class_map, subfolder_name=subfolder_name)
Expand Down Expand Up @@ -348,7 +350,7 @@ def run_in_parallel(sub_folders, input_folder, output_folder, max_workers=4, tqd
parser.add_argument('--cpu_count', type=int, default=cpu_count(), help='Number of CPU cores to use for parallel processing (default: system max)')
parser.add_argument('--continue_prediction', action="store_true", help='If continue from last processing record')
parser.add_argument('--tqdm_ncols', type=int, default=100, help='Width of tqdm progress bar in characters')
parser.add_argument('--vertebrae_engine', choices=['shapekit', 'shapekit_songlin', 'shapekit_pro', 'shapekit_hao', 'shapekit_ct_refinement'],
parser.add_argument('--vertebrae_engine', choices=['shapekit', 'shapekit_songlin', 'shapekit_pro', 'shapekit_hao', 'shapekit_ct_refinement', 'shapekit_driftfix'],
help='Override config.yaml vertebrae engine for this run')
parser.add_argument('--vertebrae_only', action='store_true', help='Process vertebrae only')
parser.add_argument('--ct_root', help='Override external case CT root for this run')
Expand Down Expand Up @@ -397,7 +399,7 @@ def run_in_parallel(sub_folders, input_folder, output_folder, max_workers=4, tqd
parser.error('--input_folder and --output_folder are required')
if args.cpu_count < 1:
parser.error('--cpu_count must be at least 1')
if vertebrae_engine in ('shapekit_hao', 'shapekit_ct_refinement'):
if vertebrae_engine in ('shapekit_hao', 'shapekit_ct_refinement', 'shapekit_driftfix'):
if target_organs != {'vertebrae'}:
parser.error(f'{vertebrae_engine} requires --vertebrae_only')
if args.continue_prediction:
Expand Down
Loading