Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
7e140cf
added a2 dev
dmitriy-myz Mar 3, 2021
d2dc7ea
added a2 dev
dmitriy-myz Mar 3, 2021
7ddfe80
06cb:00a2 work: this will recognize already added fingers; but how mi…
dmitriy-myz May 31, 2021
f303917
Support for 06cb:00a2 (enroll on host, match on chip)
dmitriy-myz May 20, 2026
647704b
Merge remote-tracking branch 'origin/master' into moh-opencv-cleanup
dmitriy-myz Jun 2, 2026
14cb813
fix for multiple finger enrollment
dmitriy-myz Jun 2, 2026
6a47a2f
reset blob reverted
dmitriy-myz Jun 2, 2026
67ebd21
correct init_hardcoded_clean_slate
dmitriy-myz Jun 2, 2026
c54e166
fix reset flow
dmitriy-myz Jun 2, 2026
862cb11
Refactor MoH enrollment out of sensor.py; merge moh_extract into moh_…
dmitriy-myz Jun 6, 2026
997c154
refactor(moh_native): remove unused code
dmitriy-myz Jun 6, 2026
9a7fbf5
Merge pull request #2 from dmitriy-myz/moh-native-remove-unused
dmitriy-myz Jun 6, 2026
048fe75
refactor(moh_native): native float pipeline (non-byte-exact experiment)
dmitriy-myz Jun 11, 2026
f7fbe9a
test(moh_native): smoke tests + review cleanups
dmitriy-myz Jun 11, 2026
480788c
Merge pull request #3 from dmitriy-myz/moh-native-float-experiment
dmitriy-myz Jun 11, 2026
3e870c2
build: declare numpy runtime dependency
dmitriy-myz Jun 11, 2026
ba8e1ee
refactor(enroll_moh_chip): drop optional cv2 resize path
dmitriy-myz Jun 11, 2026
3ee5e1c
Merge pull request #4 from dmitriy-myz/moh-native-float-experiment
dmitriy-myz Jun 11, 2026
504fe87
review cleanups: drop dev artifacts, align frame count, document capt…
dmitriy-myz Jun 11, 2026
377d233
enroll_moh: capture 6 frames by default, fill sections with the best 4
dmitriy-myz Jun 11, 2026
8451429
enroll_moh: document that frame selection approximates the DLL
dmitriy-myz Jun 11, 2026
4841f24
enroll_moh: rank frames by uncapped pool size; gate frames that can't…
dmitriy-myz Jun 12, 2026
769094f
Fix review findings on MoH enrollment (PR #261)
dmitriy-myz Sep 7, 2026
840cfbe
MoH cleanups: share envelope/store/assembly code, align default subtype
dmitriy-myz Sep 7, 2026
cae4f41
Merge pull request #5 from dmitriy-myz/moh-review-fixes
dmitriy-myz Oct 27, 2026
0d133a7
MoH: ignore stale Cancel, report full template partition, settle befo…
dmitriy-myz Oct 27, 2026
81d98fd
Merge remote-tracking branch 'origin/moh-review-fixes' into moh-openc…
dmitriy-myz Oct 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions debian/control
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Depends: ${python3:Depends},
python3-dbus,
python3-usb,
python3-yaml,
python3-numpy,
dbus,
open-fprintd (>= 0.6~),
innoextract (>= 1.6~)
Expand Down
88 changes: 88 additions & 0 deletions dev/FRAME-QUALITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Frame-quality metric for native MoH enrollment (06cb:00a2)

**Question** (2026-06-12): `enroll_moh` captured N frames and kept the "best 4
by keypoint count" — but on hardware every frame returns the maximum 250
keypoints, so the selection was a no-op. Is a quality threshold needed, and on
what metric?

**Answer: yes — but on the *uncapped* detector pool size, not the capped
count.** Implemented in `moh_enrollment.enroll_moh` (rank by `n_pool`, gate at
`min_frame_pool = FRAME_KP_CAP = 250` with per-frame recapture).

## Why the capped count cannot rank frames

`extract_frame_native` caps its keypoint pool at `FRAME_KP_CAP = 250`
(matching the DLL's `sub_18000AAB0` 0xfa cap). Synthetic degradations of a
real captured frame (`dev/analyze_frame_quality.py`, frame from
`scripts/enroll_moh_chip.py --dry-run`):

| variant | n_pool | capped | cap_score | med_score | tiles |
|--------------------|-------:|-------:|----------:|----------:|------:|
| original | 495 | 250 | 37894 | 72677 | 9 |
| half-right-empty | 250 | 250 | 757 | 45782 | 6 |
| two-thirds-empty | 168 | 168 | 0 | 39979 | 3 |
| bottom-half-empty | 258 | 250 | 985 | 24753 | 6 |
| blur-r1 | 417 | 250 | 17404 | 39653 | 9 |
| blur-r2 | 308 | 250 | 2904 | 12452 | 9 |
| blur-r3 | 213 | 213 | 0 | 2895 | 9 |
| contrast-0.5 | 455 | 250 | 9531 | 18160 | 9 |
| contrast-0.25 | 382 | 250 | 2357 | 4522 | 9 |
| contrast-0.1 | 146 | 146 | 0 | 927 | 8 |
| empty | 0 | 0 | 0 | 0 | 0 |

Even a **half-empty** or **25%-contrast** frame still saturates the cap. The
capped count only drops once the frame is unusable.

- `n_pool` (uncapped pool) falls monotonically with degradation.
- `cap_score` (|resp| of the weakest *kept* keypoint) is the most sensitive
discriminator (37894 → 757 at half-empty); it is 0 iff `n_pool < 250`.
- `med_score` also works but conflates "partial but sharp" with "weak".

## Real-capture calibration

`dev/analyze_real_captures.py` runs our detector front-end on the raw working
image tiles captured (gdb, `GDB_DUMP_F250`) from a real 8-placement Windows
driver enrollment (`/media/sf_vbox-rw/finger/frida_dumps/f250_raw_tile_*`,
session 1780170*):

| frame | n_pool | cap_score | med_score |
|------:|-------:|----------:|----------:|
| 0–7 | 493–516 | 47290–59479 | 76175–89021 |

The healthy band is tight: **pool ≈ 500 ≈ 2× the cap**. A frame that cannot
even fill its 250 template slots is therefore severely degraded (cf. the
synthetic table: the <250 cluster is ⅔-empty / heavy blur / 10% contrast).

The DLL's own minutia tables (`minutia_table_*_250.bin`, 3 sessions × 8–9
frames) confirm the DLL never ranks by count either: after its cross-frame
consensus cull it keeps only **108–150 active** keypoints per frame, and frame
selection uses a learned quality regression (`sub_180008980`, score clamped to
[0, 3000], gate `0x699 = 1689`) whose inputs are cross-frame inlier counts —
not portable to our single-frame pipeline.

## What was implemented (branch `moh-frame-quality`)

1. `moh_native.extract_frame_native(..., stats=None)` — optional dict out:
`n_pool`, `cap_score`, `med_score` (computed in phase 2, before the cap).
2. `moh_enrollment.enroll_moh`:
- ranks frames by `n_pool` (was: saturated `len(kps)`);
- new `min_frame_pool` gate (default `FRAME_KP_CAP`): a frame with
`n_pool < 250` is recaptured within the per-frame retry budget; on
exhaustion it is kept (ranking deprioritizes it) rather than failing
the enrollment;
- logs `pool/cap_score/med_score` per frame so future hardware runs keep
producing calibration data.

## Threshold choice & caveats

- `min_frame_pool = 250` is deliberately conservative: zero false rejects on
the real healthy band (~500), and it has a structural meaning ("can fill
the v30 section"). The gap between degraded (≤308) and healthy (≥493)
synthetic/real clusters would tolerate up to ~400, but that is calibrated
on ONE real finger — collect more `pool=` log lines from hardware before
raising it.
- The degradation study uses synthetic perturbations of one real frame; real
bad placements (dry finger, edge-of-sensor) should be spot-checked on
hardware via the new per-frame log line.
- If a stronger discriminator is ever wanted, `cap_score` is the candidate
(most dynamic range); thresholding it needs more real bad-frame data.
123 changes: 123 additions & 0 deletions dev/analyze_frame_quality.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
"""Frame-quality metric study for native MoH enrollment (06cb:00a2).

Question: enroll_moh ranks frames by len(extract_frame_native(...)) to pick
the best 4 of N captures, but on real hardware every frame saturates the
250-keypoint cap (FRAME_KP_CAP), so the ranking is a no-op. Is a threshold
(or a different metric) needed?

Method: take a real captured frame (scripts/enroll_moh_chip.py --dry-run
saves one to /tmp/native_capture_112x112.bin) and degrade it the way bad
placements degrade (partial contact, blur/wet, light press, no finger).
For each variant run the detector front-end (tile -> DoH -> NMS -> subpix
-> edge cull, i.e. extract_frame_native minus the descriptor stages) and
report candidate quality metrics:

n_pool uncapped keypoint count (pre-250-cap pool size)
n_capped min(n_pool, 250) = what enroll_moh ranks by today
cap_score |resp| of the 250th-strongest keypoint (0 if n_pool < 250)
= "how strong is the weakest keypoint we keep"
med_score median |resp| of the kept (top-250) keypoints
tiles number of 3x3 tiles contributing >= 5 keypoints (coverage)

Usage: ./.venv-poc/bin/python dev/analyze_frame_quality.py [frame.bin ...]
"""
import sys

import numpy as np

sys.path.insert(0, '.')
from validitysensor.moh_native import (FRAME_KP_CAP, GRID, tile_image,
tile_origin, doh, nms,
subpix_refine_kp,
_a960_passes_global_edge)


def detect_pool(image_q16, h=112, w=112):
"""Phases 1-2 of extract_frame_native, uncapped: the scored keypoint pool
[(|resp|, tile_id, gx, gy)] after subpix refine + global edge cull."""
pool = []
for ti, tj, tile in tile_image(np.asarray(image_q16, dtype=np.int64)):
_, _, _, resp = doh(tile)
oy, ox = tile_origin(ti, tj, h, w)
for score, lx, ly in nms(resp):
r = subpix_refine_kp(resp, lx, ly)
if r is None:
continue
sx_q16, sy_q16 = r
if not _a960_passes_global_edge(sx_q16, sy_q16, oy, ox, h, w, ti, tj):
continue
pool.append((score, ti * GRID + tj,
((ox << 16) + sx_q16) >> 16,
((oy << 16) + sy_q16) >> 16))
pool.sort(key=lambda r: -r[0])
return pool


def metrics(pool):
kept = pool[:FRAME_KP_CAP]
scores = [s for s, *_ in kept]
per_tile = np.bincount([t for _, t, *_ in kept], minlength=GRID * GRID)
return {
'n_pool': len(pool),
'n_capped': len(kept),
'cap_score': scores[FRAME_KP_CAP - 1] if len(pool) >= FRAME_KP_CAP else 0.0,
'med_score': float(np.median(scores)) if scores else 0.0,
'tiles': int((per_tile >= 5).sum()),
}


def degrade(img):
"""Yield (name, variant) uint8 images simulating bad placements."""
rng = np.random.default_rng(42)
yield 'original', img

# Partial contact: region without finger reads near-uniform background.
bg = float(np.median(img))
for name, sl in (('half-right-empty', np.s_[:, 56:]),
('two-thirds-empty', np.s_[:, 38:]),
('bottom-half-empty', np.s_[56:, :])):
v = img.copy().astype(np.float64)
region = v[sl]
v[sl] = bg + rng.normal(0, 2, region.shape)
yield name, np.clip(v, 0, 255).astype(np.uint8)

# Blur (wet finger / smear): box blur, growing radius.
for r in (1, 2, 3):
v = img.astype(np.float64)
k = 2 * r + 1
ker = np.ones(k) / k
for ax in (0, 1):
v = np.apply_along_axis(
lambda m: np.convolve(m, ker, mode='same'), ax, v)
yield f'blur-r{r}', np.clip(v, 0, 255).astype(np.uint8)

# Light press: contrast collapse around the mean.
m = img.mean()
for c in (0.5, 0.25, 0.1):
v = (img.astype(np.float64) - m) * c + m
yield f'contrast-{c}', np.clip(v, 0, 255).astype(np.uint8)

# No finger at all: background + sensor noise.
yield 'empty', np.clip(
bg + rng.normal(0, 2, img.shape), 0, 255).astype(np.uint8)


def main():
paths = sys.argv[1:] or ['/tmp/native_capture_112x112.bin']
hdr = f"{'variant':>18} {'n_pool':>6} {'capped':>6} {'cap_score':>9} " \
f"{'med_score':>9} {'tiles':>5}"
for path in paths:
img = np.fromfile(path, dtype=np.uint8).reshape(112, 112)
print(f'== {path} (min={img.min()} max={img.max()} '
f'mean={img.mean():.1f} std={img.std():.1f})')
print(hdr)
for name, v in degrade(img):
mm = metrics(detect_pool(v.astype(np.int32) << 16))
print(f"{name:>18} {mm['n_pool']:>6} {mm['n_capped']:>6} "
f"{mm['cap_score']:>9.0f} {mm['med_score']:>9.0f} "
f"{mm['tiles']:>5}")
print()


if __name__ == '__main__':
main()
117 changes: 117 additions & 0 deletions dev/analyze_real_captures.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
"""Frame-quality metrics on REAL enrollment captures (06cb:00a2).

Companion to dev/analyze_frame_quality.py (which uses one real frame +
synthetic degradations). This one mines the gdb capture dumps from real
Wine-driver enrollments in /media/sf_vbox-rw/finger/frida_dumps:

1. f250_raw_tile_<ts>_call<NNN>_<H>x<W>.bin — the raw Q16 working-image
tiles at the pre-smooth entry. 18 calls per placement (the pre-smooth
runs twice per tile: DoH chain + descriptor-gradient chain), 9 tiles
per frame, column-major (i = call%3, j = call//3). 144 calls = one
real 8-placement enrollment. We run OUR detector front-end per tile
and report the same metrics as analyze_frame_quality.py — this is the
real-data answer to "do healthy frames always saturate the 250 cap,
and what does the pool/score distribution look like?"

2. minutia_table_<ts>_250.bin — the DLL's in-memory 250 x 32-byte
keypoint table per frame (3 sessions x 8-9 frames). Fields (RE'd):
+0x8 active flag, +0x9 tile id, +0xa quality byte. We report the
DLL's own active-count per frame for comparison.

Usage: ./.venv-poc/bin/python dev/analyze_real_captures.py [dump_dir]
"""
import glob
import os
import re
import sys
from collections import defaultdict

import numpy as np

sys.path.insert(0, '.')
from validitysensor.moh_native import FRAME_KP_CAP
from dev.analyze_frame_quality import metrics

from validitysensor.moh_native import (GRID, tile_origin, doh, nms,
subpix_refine_kp,
_a960_passes_global_edge)

DUMPS = '/media/sf_vbox-rw/finger/frida_dumps'


def tile_pool(tile_q16, i, j, h=112, w=112):
"""detect_pool for ONE captured tile (origin-aware edge cull)."""
_, _, _, resp = doh(tile_q16)
oy, ox = tile_origin(i, j, h, w)
out = []
for score, lx, ly in nms(resp):
r = subpix_refine_kp(resp, lx, ly)
if r is None:
continue
sx_q16, sy_q16 = r
if not _a960_passes_global_edge(sx_q16, sy_q16, oy, ox, h, w, i, j):
continue
out.append((score, i * GRID + j,
((ox << 16) + sx_q16) >> 16,
((oy << 16) + sy_q16) >> 16))
return out


def analyze_tiles(dump_dir):
pat = re.compile(r'f250_raw_tile_(\d+)_call(\d+)_(\d+)x(\d+)\.bin$')
frames = defaultdict(dict) # frame_no -> {(i,j): tile}
for p in sorted(glob.glob(os.path.join(dump_dir, 'f250_raw_tile_*.bin'))):
m = pat.search(p)
if not m:
continue
_, call, hh, ww = (int(g) for g in m.groups())
idx = call % 18
if idx >= 9: # 2nd pre-smooth invocation: same tile data
continue
i, j = idx % 3, idx // 3 # column-major capture order
tile = np.fromfile(p, dtype=np.int32).reshape(hh, ww)
frames[call // 18][(i, j)] = tile

if not frames:
print('no f250_raw_tile dumps found')
return
print(f'== real enrollment tiles: {len(frames)} frames '
f'({dump_dir}/f250_raw_tile_*)')
print(f"{'frame':>5} {'n_pool':>6} {'capped':>6} {'cap_score':>9} "
f"{'med_score':>9} {'tiles':>5}")
for fno in sorted(frames):
pool = []
for (i, j), tile in sorted(frames[fno].items()):
pool.extend(tile_pool(tile, i, j))
pool.sort(key=lambda r: -r[0])
mm = metrics(pool)
print(f"{fno:>5} {mm['n_pool']:>6} {mm['n_capped']:>6} "
f"{mm['cap_score']:>9.0f} {mm['med_score']:>9.0f} "
f"{mm['tiles']:>5}")
print()


def analyze_minutia_tables(dump_dir):
files = sorted(glob.glob(os.path.join(dump_dir, 'minutia_table_*_250.bin')))
if not files:
print('no minutia_table dumps found')
return
print(f'== DLL minutia tables ({len(files)} frames, 3 sessions): '
'per-frame ACTIVE counts + quality')
print(f"{'session':>10} {'frame_ts':>13} {'active':>6} "
f"{'qual med':>8} {'qual max':>8}")
for p in files:
ts = int(re.search(r'minutia_table_(\d+)_250', p).group(1))
recs = np.fromfile(p, dtype=np.uint8).reshape(250, 32)
active = recs[:, 0x8] == 1
qual = recs[active, 0xa]
print(f"{str(ts)[:6]:>10} {ts:>13} {int(active.sum()):>6} "
f"{float(np.median(qual)) if qual.size else 0:>8.0f} "
f"{int(qual.max()) if qual.size else 0:>8}")
print()


if __name__ == '__main__':
d = sys.argv[1] if len(sys.argv) > 1 else DUMPS
analyze_minutia_tables(d)
analyze_tiles(d)
Loading