Skip to content

VectorIndex: re-resolve the Java index by name after REBUILD INDEX - #117

Merged
tae898 merged 3 commits into
mainfrom
fix/116-stale-vector-index
Oct 28, 2026
Merged

tae898 merged 3 commits into
mainfrom
fix/116-stale-vector-index

Conversation

@tae898

@tae898 tae898 commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #116.

A VectorIndex wrapper held across a SQL REBUILD INDEX now keeps serving. Before, the old Java index object (replaced by the rebuild, isValid() false) answered searches with no rows and no error: on arcadedb-embedded 26.10.2.post1 with 2,000 vectors of dimension 16, find_nearest went from 5 rows to 0 and totalVectors from 2000 to 0, while a wrapper re-fetched with db.schema.get_vector_index returned 5 and 2000.

The fix is in src/arcadedb/vector.py: a new _check_live() runs at the start of every public method that touches the Java index (find_nearest, find_nearest_by_key, find_nearest_approximate, get_size, get_metadata, get_stats, get_quantization, warm_up, build_graph_now). When the cached index is no longer valid it looks the index up by name in the schema, replaces _java_index, and resets _lsm_indexes and _pq_ready. If the name no longer resolves (the index was dropped), it raises ArcadeDBError naming the index, never an empty result.

Hot path cost: one added JPype crossing per public call, isValid() on the cached object. The lookup by name runs only after an invalidation. find_nearest_by_key and find_nearest_approximate check once and then call internal helpers (_find_nearest_live, _get_quantization_live), so neither pays the crossing twice. find_nearest therefore adds exactly 1 crossing per call.

Native build: NativeVectorIndex names the index in every statement, so it has no cached engine object to go stale. _check_live is a no-op there. One related change: get_stats now reads the bucket index name afresh, because a rebuild builds new bucket indexes and the memoized name can be gone.

Tests, in tests/test_vector_index_introspection.py (shared by both backends, no Java object used): test_old_wrapper_serves_after_rebuild_index and test_old_wrapper_follows_a_second_rebuild hold the wrapper across REBUILD INDEX and assert 5 rows from find_nearest and the right totalVectors, size, and metadata; test_dropped_index_raises_and_never_answers_empty drops the index and asserts ArcadeDBError, with the index name in the message on the JVM build. On the old code the rebuild tests fail by the measurement above (0 rows instead of 5, totalVectors 0 instead of the row count), and the drop test fails because the old wrapper returned an empty list instead of raising. I did not run the suite locally; CI does.

Engine side, filed upstream as ArcadeData/arcadedb#10897: the old bucket LSMVectorIndex answers findNeighborsFromVector, get, countEntries and getStats with 0 instead of throwing (JDK 21 and 25, main c3af6d9028), while the old TypeIndex throws. This PR does not depend on that fix: the wrapper checks isValid() itself.

🤖 Generated with Claude Code

tae898 and others added 2 commits October 29, 2026 00:04
)

REBUILD INDEX replaces the engine's index object. The wrapper kept the old one,
whose isValid() is false but whose search path does not check it, so a held
wrapper answered find_nearest with 0 rows and totalVectors 0 and no error.
Every public method now calls _check_live() first: one isValid() crossing on
the cached object; when it is false, the index is looked up by name, and
_lsm_indexes and _pq_ready are reset. A name that no longer resolves raises
ArcadeDBError naming the index. find_nearest_by_key and find_nearest_approximate
check once and call internal helpers, so a call never pays the crossing twice.
The native wrapper resolves by name per statement: _check_live is a no-op there,
and get_stats reads the bucket index name afresh since a rebuild renames it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…index

# Conflicts:
#	tests/test_vector_index_introspection.py
… NONE

create_vector_index defaults to quantization="INT8", so the hard-coded "NONE" failed on
the JVM legs and native parity (assert 'INT8' == 'NONE', the only failure: 1832 passed JVM,
1572 passed native). The test now reads get_quantization() before REBUILD INDEX and asserts the
held wrapper reports the same value after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 57f37d9 into main Oct 28, 2026
36 checks passed
@tae898
tae898 deleted the fix/116-stale-vector-index branch October 28, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VectorIndex wrapper silently returns no results after REBUILD INDEX

1 participant