Repository navigation
Windows: engine logs go to the file only, so redirecting fd 1/2 cannot put log text in database files - #78
Merged
Merged
Conversation
…se files On Windows the runtime caches the OS handle of stdout and stderr at start-up. A host that redirects fd 1 or 2 afterwards (pytest's default fd capture, a service, an embedding application) closes that handle; a database file opened later gets the same handle value and the next console log line is written into it. The native Windows legs of tests/test_vector.py failed 1-5 times in 50 runs per test (Invalid vector id, Invalid position N (size=65536), no result after a reopen), and the failing database snapshots held engine log text. - _logdir: the generated logging config has no ConsoleHandler on Windows by default. ARCADEDB_LOG_CONSOLE=1/0 turns the console log on or off on every platform; Linux and macOS keep the console log by default. The file-only config has its own file name (logging-file-only.properties) so processes with different settings never replace each other's file. A user's own java.util.logging.config.file still wins. - native glue: NativeExtras.swallowed and LibraryLoader log through java.util.logging instead of System.err. - tests/test_log_redirect.py: in a fresh child process, dup2 a file onto fd 1 and 2 after the library started, build a vector index and its graph, reopen, query, and assert that no database file holds a log line. Runs on every platform; on POSIX it passes trivially, on Windows it is the real check. - parity_wheel.py keeps pytest's default capture on Windows (now the regression check); test-jvm.yml keeps --capture=sys until a default-capture run has passed there. - docs: how-to/logs.adoc describes the console log and ARCADEDB_LOG_CONSOLE. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On native Windows, engine log lines were written into database files when the host process redirected fd 1 or 2 after the library started. The runtime caches the OS handle of stdout and stderr at start-up. pytest's default fd capture (any
os.dup2onto fd 1 or 2, a service, an embedding application) closes that handle, a database file opened later gets the same handle value, and the next console log line lands in that file. POSIX is not affected, because writes go to fd 1 and 2 by number. This is the same mechanism as humemai/arcadedb-embedded-python#10, whichtest-jvm.ymlworks around with--capture=syson Windows.Evidence from the temporary repro workflow (run 40226704699 on branch
win-vector-persistence-repro, native Windows, the wholetests/test_vector.pyper process):Invalid vector id -N,Invalid position N (size=65536), 0 results after the reopen), bothtest_lsm_persistenceandtest_lsm_vector_search_approximate_persistence.--capture=sys: 0 failures in 400 runs. JVM control with--capture=sys: 0.Doc_0.1.65536.v0.bucketstarts with2026-10-25 19:37:03.759 INFO [LSMVectorIndex] Discovered and loaded graph file ..., and.lsmvecidxfiles containGraph build validating ....Fix
src/arcadedb/_logdir.py): the generatedjava.util.loggingconfig has noConsoleHandleron Windows by default, so the log goes to the per-user log file only.ARCADEDB_LOG_CONSOLE=1(ortrue,yes,on) turns the console log on andARCADEDB_LOG_CONSOLE=0turns it off, on every platform. Ajava.util.logging.config.filethe user sets still wins. The file-only config is written tologging-file-only.properties(the config with the console stayslogging.properties), so two processes with different settings that share a log directory never replace each other's file between the write and the engine's first log call.NativeExtras.swallowed(callback failures of async writers) andLibraryLoader(jvector library load failures) now log throughjava.util.logginginstead ofSystem.err, so on Windows they go to the log file like the rest. Nothing else innative/srcorsrc/javawrites toSystem.outorSystem.err; the only remaining use is the jextract downcall trace inNativeSimdOps, which runs only with-Djextract.trace.downcalls=true. The JVM build's glue (src/java/com/arcadedb/python) never printed.native/wheel/parity_wheel.pykeeps pytest's default capture on Windows; it is now the regression check. A comment says so.test-jvm.ymlkeeps--capture=sysfor now (comment updated) and can drop it once a default-capture run has passed on its Windows legs.how-to/logs.adochas a "Console output" section (where the log goes,ARCADEDB_LOG_CONSOLE, one paragraph on the Windows handle reuse).Tests
tests/test_log_redirect.pyruns on every platform. In a fresh child process it starts the library, thenos.dup2s a file onto fd 1 and 2 and closes the original, creates a vector index (PRODUCT quantization, 259 vectors), builds the graph, closes, reopens and queries (the same work astest_lsm_persistenceandtest_lsm_vector_search_approximate_persistence). It asserts the reopen query is correct and that no file in the database directory contains an engine log line (date, time with milliseconds, level), and that the work did log to the file. On POSIX it passes trivially; on Windows it is the real check. Further tests cover the console opt-in and opt-out (a child that does not redirect, stderr captured by the parent), the platform and env defaults, the rendered config and the config file name that both builds hand to the engine. Existing assertions intest_log_dir.pythat namedlogging.propertiesnow use the name for the setting.Run locally on Linux against a freshly built native image:
test_log_redirect.py,test_log_dir.py,test_native_log_dir.py,test_core.py,test_check_test_skips.pypass. bandit, black, isort,check_no_embedded_name.pyand actionlint are clean. The Windows legs oftest-native.ymlandtest-jvm.yml, and a rerun of the repro workflow against this branch's native wheel with pytest's default fd capture, are the verification on the platform that has the bug.🤖 Generated with Claude Code