Repository navigation
Native: release the callback ids of every async executor call - #80
Merged
Merged
Conversation
Only AsyncExecutor.transaction gave its callback ids back. create_record, query, command, new_edge, new_edge_by_keys and scan_type registered an id per callback in extras._callbacks and never released it, so the table grew without bound over a long-lived executor, and on_ok / on_error added two more each time they were set. All calls now go through one _queue helper. A queued call puts its ids on the executor's list after the library accepted it; wait_completion releases the ids that were queued before it started (a call queued by another thread during the wait keeps its callbacks, and a timed-out or interrupted wait puts the ids back); close releases everything. A call the library refuses gives its ids back at once. on_ok and on_error keep one id each until close, and a new handler replaces the old id. Python only: no change to the library. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
8 of 10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every native async call that takes a callback registers a Python function in
extras._callbacksand hands the library its id. OnlyAsyncExecutor.transactionever gave the ids back (#67, #69).create_record,query,command,new_edge,new_edge_by_keysandscan_typeleaked one to three ids per call, andon_ok/on_errorleaked two more each time they were set, so the table grew without bound over a long-lived executor.What changes
All async calls now go through one
_queue(op, **args)helper inNativeAsyncExecutor, which is the ruletransactionalready used, applied to every op:cb,ok,err) on the executor's list after the library accepted it.wait_completionreleases the ids that were queued before it started, so a call that another thread queues during the wait keeps its callbacks. A wait that times out or is interrupted puts the ids back.closereleases everything the executor holds, including calls that were never waited for. Closing the database closes its executors, so it releases them too.on_okandon_errorare executor-wide, so a wait does not drop them. They keep one id each untilclose, and a new handler replaces the old id (the engine replaces the handler, and a test pins that).new_edgeand_statementnow build their arguments before they register callbacks, so a bad argument cannot leave an id behind.Python only. The library is unchanged.
Verification
New
tests/test_native_async_callbacks.py(native only, listed incollect_ignorefor the JVM build). Against the unfixed code, six of the first seven tests failed with growth (for example 71 ids against a baseline of 21 after one close).wait_completion: the table size returns to the baseline after each round, and every kind ran its callback.wait_completion(0)that finds the writers busy keeps the ids.closewithout a wait, and closing the database, release the ids.on_ok/on_error: two slots no matter how often they are set, kept across a wait, released on close, and the replacedon_okdoes not fire.wait_completion: all 200 callbacks run and the table returns to the baseline.Run on the native build (
ARCADEDB_BACKEND=native, library image: the newest local build,native-lib/winlogof 2026-10-26; no Java change is needed):test_async_executor,test_async_transaction,test_native_async_callbacks,test_python_callbacks,test_concurrencyandtest_coregive 118 passed. Bandit (-c pyproject.toml -r src tests --severity-level low --confidence-level low), isort--profile black, black andscripts/check_no_embedded_name.pyare clean.🤖 Generated with Claude Code