Skip to content

Native and JVM builds: switch off the JEP 498 sun.misc.Unsafe warning at its source (Windows data corruption) - #94

Merged
tae898 merged 1 commit into
mainfrom
native-jep498-warning
Oct 26, 2026
Merged

tae898 merged 1 commit into
mainfrom
native-jep498-warning

Conversation

@tae898

@tae898 tae898 commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What

The native arcadedb wheel (GraalVM CE native image) can write the JDK's JEP 498 warning into a database file on Windows. The release run 40291551544 (tag 26.11.1.dev1) failed the parity job "windows-x64, Python 3.11" with tests/test_log_redirect.py::test_redirect_with_the_console_off_sends_nothing_to_the_redirect_target: db0\Doc_0_485397029700.4.262144.v0.lsmvecidx started with A terminally deprecated method in sun.misc.Unsafe has been called. The Python 3.12 and 3.13 Windows legs passed, because it is the handle-reuse race of #78 and #88. This PR switches the warning off at its source in both packages, so no stream has it to write.

Where the warning comes from in the native image

jvector's SimpleMappedReader$Supplier calls sun.misc.Unsafe.invokeCleaner when it unmaps a graph file, on the first reopen of a vector index. In JDK 25, sun.misc.Unsafe.beforeMemoryAccess() then prints the four-line warning with VM.initialErr(), which is System.initialErr: in the image, the stream over the stderr handle that the runtime cached at start-up. #78 moved the log and System.out/System.err away from that handle, and #88 did the same for System.initialErr in the JVM build, but the native image has no System.setErr hook for it, and nothing in the glue code redirects it. After the host's dup2 onto fd 2 closes the cached handle, a database file opened later can get the same handle value, and the warning lands in it. On Linux the same text goes to the host's redirect target (checked on the dev0 native wheel: ARCADEDB_LOG_CONSOLE=0, fd 2 redirected, four lines in the target).

The option is a build-time constant in the image: sun.misc.Unsafe.MEMORY_ACCESS_OPTION is a static final that is read from VM.getSavedProperty("sun.misc.unsafe.memory.access") when the class is initialised, and the image builder folds it into the image. ARCADEDB_NATIVE_ISOLATE_ARGS=-Dsun.misc.unsafe.memory.access=allow (and =deny) at run time did nothing on the existing wheel.

The fix

  • Native: JavaArgs = -Dsun.misc.unsafe.memory.access=allow in the glue jar's native-image.properties. JavaArgs goes to the builder's JVM, so every way of building the glue jar gets it. I tested four ways on a minimal image that calls Unsafe.invokeCleaner (GraalVM CE 25.4.4.1.1, P1=1 NEED_GB=13 JOB_MEM_GB=14 quiet_build.sh): without the option the program prints the four lines; -J-Dsun.misc.unsafe.memory.access=allow, -J--sun-misc-unsafe-memory-access=allow and JavaArgs in a jar's native-image.properties print nothing; a -Dsun.misc.unsafe.memory.access=allow at run time on the first image prints the four lines.
  • JVM: --sun-misc-unsafe-memory-access=allow is a default of _build_jvm_args, next to --enable-native-access; a value of the caller's own in jvm_args or ARCADEDB_JVM_ARGS wins. This is belt and braces next to JVM build: Java System.out/System.err go to a file the library owns when the console is off (Windows data corruption) #88's System.initialErr redirect, which stays for every other text of that kind. The option reaches HotSpot through JPype (checked: with it the four lines are gone from arcadedb-stdio.log).
  • docs/modules/ROOT/pages/how-to/logs.adoc says that neither package lets the warning appear, and that the native value is fixed in the image.

Tests

  • tests/test_log_redirect.py: on both builds the warning must be in no database file, not in the redirect target, not on the console and not in arcadedb-stdio.log (the native build was exempt from the warning assertions before). The JVM build also runs the console-off redirect once more with ARCADEDB_JVM_ARGS=--sun-misc-unsafe-memory-access=warn, so JVM build: Java System.out/System.err go to a file the library owns when the console is off (Windows data corruption) #88's redirect of System.initialErr stays tested: the warning must then be in arcadedb-stdio.log and nowhere else.
  • tests/test_jvm_args.py: the default, and that a caller's value wins.
  • Negative control: the new test run against the existing pre-fix native wheel (Linux, ARCADEDB_REDIRECT_REPEATS=2) fails on the console-off redirect, console-on and console-off tests, with the four lines in the redirect target and on the console. On the JVM build (Linux, 35 tests in test_log_redirect.py and test_jvm_args.py) all pass.

Windows proof

Temporary workflow on the branch tmp-native-win-loop (no PR, deleted after the run), run 40301549781: it builds the Windows native image and wheel from this branch (GraalVM CE 25.4.4.1.1, MSVC, engine 26.10.2), installs the wheel with uv run --with and loops. Each iteration starts a new pytest process (default fd capture) for test_redirecting_fd_1_and_2_leaves_the_database_files_clean, test_redirect_with_the_console_off_sends_nothing_to_the_redirect_target and test_console_log_can_be_turned_off on Windows, with ARCADEDB_REDIRECT_REPEATS repeats of the create, build, close, reopen and query cycle per child.

  • fixed-r1-b: 100 iterations at 1 repeat(s) per child (300 test runs), 0 failures
  • fixed-r10-a: 40 iterations at 10 repeat(s) per child (120 test runs), 0 failures
  • fixed-r10-b: 40 iterations at 10 repeat(s) per child (120 test runs), 0 failures
  • fixed-r1-a: 100 iterations at 1 repeat(s) per child (300 test runs), 0 failures

With the fix: 280 iterations (840 test runs), 0 failures.
Control, the Windows wheel of release run 40291551544 (tag 26.11.1.dev1, no fix) under the same tests: 40 iterations at 10 repeats per child (120 test runs), 40 failures.

CI of this PR: Test native run 40301477031: success; Test JVM run 40301476995: success.

🤖 Generated with Claude Code

… at its source

The native image prints jvector's sun.misc.Unsafe warning (the first reopen of a vector index) to the stderr handle it cached at start-up; on Windows a database file opened after the host's dup2 onto fd 2 can own that handle value and got the four lines (release run 40291551544, a .lsmvecidx file). The image keeps sun.misc.Unsafe's option as a build-time constant, so JavaArgs in the glue jar's native-image.properties sets -Dsun.misc.unsafe.memory.access=allow for the builder. The JVM build gets --sun-misc-unsafe-memory-access=allow as a default (a caller's value wins), next to the System.initialErr redirect of #88. test_log_redirect.py now checks the warning is on no stream and in no file on both builds, and keeps the initialErr redirect covered with the option set back to warn.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 662851c into main Oct 26, 2026
32 checks passed
@tae898
tae898 deleted the native-jep498-warning branch October 26, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant