Repository navigation
VectorIndex: re-resolve the Java index by name after REBUILD INDEX - #117
Merged
Merged
Conversation
) REBUILD INDEX replaces the engine's index object. The wrapper kept the old one, whose isValid() is false but whose search path does not check it, so a held wrapper answered find_nearest with 0 rows and totalVectors 0 and no error. Every public method now calls _check_live() first: one isValid() crossing on the cached object; when it is false, the index is looked up by name, and _lsm_indexes and _pq_ready are reset. A name that no longer resolves raises ArcadeDBError naming the index. find_nearest_by_key and find_nearest_approximate check once and call internal helpers, so a call never pays the crossing twice. The native wrapper resolves by name per statement: _check_live is a no-op there, and get_stats reads the bucket index name afresh since a rebuild renames it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…index # Conflicts: # tests/test_vector_index_introspection.py
… NONE create_vector_index defaults to quantization="INT8", so the hard-coded "NONE" failed on the JVM legs and native parity (assert 'INT8' == 'NONE', the only failure: 1832 passed JVM, 1572 passed native). The test now reads get_quantization() before REBUILD INDEX and asserts the held wrapper reports the same value after it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #116.
A
VectorIndexwrapper held across a SQLREBUILD INDEXnow keeps serving. Before, the old Java index object (replaced by the rebuild,isValid()false) answered searches with no rows and no error: onarcadedb-embedded 26.10.2.post1with 2,000 vectors of dimension 16,find_nearestwent from 5 rows to 0 andtotalVectorsfrom 2000 to 0, while a wrapper re-fetched withdb.schema.get_vector_indexreturned 5 and 2000.The fix is in
src/arcadedb/vector.py: a new_check_live()runs at the start of every public method that touches the Java index (find_nearest,find_nearest_by_key,find_nearest_approximate,get_size,get_metadata,get_stats,get_quantization,warm_up,build_graph_now). When the cached index is no longer valid it looks the index up by name in the schema, replaces_java_index, and resets_lsm_indexesand_pq_ready. If the name no longer resolves (the index was dropped), it raisesArcadeDBErrornaming the index, never an empty result.Hot path cost: one added JPype crossing per public call,
isValid()on the cached object. The lookup by name runs only after an invalidation.find_nearest_by_keyandfind_nearest_approximatecheck once and then call internal helpers (_find_nearest_live,_get_quantization_live), so neither pays the crossing twice.find_nearesttherefore adds exactly 1 crossing per call.Native build:
NativeVectorIndexnames the index in every statement, so it has no cached engine object to go stale._check_liveis a no-op there. One related change:get_statsnow reads the bucket index name afresh, because a rebuild builds new bucket indexes and the memoized name can be gone.Tests, in
tests/test_vector_index_introspection.py(shared by both backends, no Java object used):test_old_wrapper_serves_after_rebuild_indexandtest_old_wrapper_follows_a_second_rebuildhold the wrapper acrossREBUILD INDEXand assert 5 rows fromfind_nearestand the righttotalVectors, size, and metadata;test_dropped_index_raises_and_never_answers_emptydrops the index and assertsArcadeDBError, with the index name in the message on the JVM build. On the old code the rebuild tests fail by the measurement above (0 rows instead of 5,totalVectors0 instead of the row count), and the drop test fails because the old wrapper returned an empty list instead of raising. I did not run the suite locally; CI does.Engine side, filed upstream as ArcadeData/arcadedb#10897: the old bucket
LSMVectorIndexanswersfindNeighborsFromVector,get,countEntriesandgetStatswith 0 instead of throwing (JDK 21 and 25, mainc3af6d9028), while the oldTypeIndexthrows. This PR does not depend on that fix: the wrapper checksisValid()itself.🤖 Generated with Claude Code