Skip to content

write_memory errors when running test_https_local.py #97

Closed

Description

@Armitage64

Several errors show up in the console when running the test_https_local.py script. On the C64 screen, the following text is displayed but the test does not complete successfully:

SYS16896
CH
SH
HK1
KEYS
ENC1
RX

at this point, the machine appears to lock up and become unresponsive.

Console output is below:

Enabling UCI...
Setting turbo to 48 MHz (from Off/ 1)...
Resetting machine...
run_prg(PRG)...
net_initialized = $01
Starting U64 6510 debug stream -> 192.168.1.212:11002
WARNING: debug capture failed to start: PUT http://192.168.1.233/v1/streams/debug:start?ip=192.168.1.212%3A11002 returned HTTP 404: {
"errors" : [ "Network Host Resolve Error" ]
}
Sending 'Q' to exit PRG main_loop...
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:410: UserWarning: write_memory($0277, 2 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:414: UserWarning: write_memory($00C6, 1 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1487: UserWarning: write_memory($4200, 64 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1487: UserWarning: write_memory($4240, 53 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1491: UserWarning: write_memory($4300, 32 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(HOST_STR_ADDR, host_str.ljust(32, b"\x00"))
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1492: UserWarning: write_memory($4340, 8 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(PATH_STR_ADDR, path_str.ljust(8, b"\x00"))
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1495: UserWarning: write_memory($4380, 16 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(SENTINEL_ADDR, bytes(16))
Triggering: sys16896
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:410: UserWarning: write_memory($0277, 9 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
[ 0.5s] progress=0x03

Activity

  1. JC-000 commented on Aug 14, 2026

    @JC-000
    Owner

    Thanks — and this one is a genuinely useful report, because the screen text you captured is the diagnostic.

    The screen tells us where it stopped

    SYS16896  CH  SH  HK1  KEYS  ENC1  RX
    

    That is: ClientHello sent, ServerHello received, handshake keys derived, first encrypted record received... and then nothing. The next marker you should see is GOT2. Its absence, with everything before it healthy, matches a signature we have hit before and documented:

    every TLS handshake stalling deterministically at the first encrypted record, screen ending KEYS ENC1 RX, never reaching GOT2, at any clock speed — while DHCP, TCP and plaintext HTTP all keep working.

    On our bench that turned out to be transient device state, not client code. Soft resets did not clear it; a hard power cycle at the wall did. Firmware was unchanged throughout, and a known-good older commit failed identically while the device was in that state — which is why we wrote it down rather than bisecting again.

    So the first thing worth trying is the cheapest: power the Ultimate fully off at the wall, not a menu reset, and re-run. If it then completes, that is the whole story.

    If it survives a power cycle

    Then it is worth diagnosing properly, and two values would settle it quickly. Both are printed in the failure dump, or readable from the artifact directory the script writes under /tmp/uci_https_debug/<timestamp>/:

    • net_last_error — $86 is UCI_ERR_READ_FAIL (SOCKET_READ returned an error bit), which is what we saw in the wedge case
    • tls_recv_sub_progress — $02 in that same case

    Please paste those, plus tls_state / tls_last_state, and I will take it from there. A baseline run of python3 tools/uci/boot_check.py and python3 tools/uci/phase3_tcp_echo.py is also informative: if plain TCP echo works while TLS stalls at the same place every time, that points away from your setup and toward the device.

    The console output is noise, not errors

    The UserWarning: write_memory(...): no safe_regions declared lines are benign and expected on every run, including passing ones. They come from the harness's memory-policy guard, which is deliberately set to warn rather than deny for these scripts. They are not related to your failure. They should not be shouting at you on a normal run, though — that they read as errors is our problem, and I have noted it as something to quieten.

    The debug capture failed to start ... Network Host Resolve Error 404 is also harmless: that is the optional 6510 bus-trace stream trying to reach 192.168.1.212:11002, which is a host/port your Ultimate could not resolve. The test proceeds without it. You can silence it with DEBUG_CAPTURE=0.

    One thing that would help me confirm the match: does it fail at the same point every time, and does it also fail at TURBO_MHZ=1? Deterministic-at-any-clock was the distinguishing feature last time.

  2. Armitage64 commented on Aug 14, 2026

    @Armitage64
    Author

    The test always fails with "RX" on the screen, even after a cold power cycle. There are no artifacts in the log directory at the time the test locks up with " [ 0.6s] progress=0x03" displayed on the console.

    Here is the output of the boot_check.py script:

    % python3 tools/uci/boot_check.py
    Loaded 62977 bytes from /Users/armitage/git/c64-https/build/c64-https.prg
    Expected backend: uci ('UCI NETWORKING')

    --- PRG image checks ---
    [PASS] image carries uci banner: expected bytes 'UCI NETWORKING' in the PRG
    [PASS] image free of foreign banner: none present

    Acquired DeviceLock(192.168.1.237)
    Enabling UCI (Command Interface)...
    Resetting machine...
    run_prg(PRG)...
    Waiting up to 60s for the main menu...
    main menu reached

    --- screen RAM decoded (non-empty lines) ---
    00: c64-https client v0.1
    02: tls 1.3 / chacha20-poly1305
    03: uci networking
    05: initializing network...
    06: network ok
    07: requesting dhcp...
    08: dhcp ok - ip: 192.168.1.237
    09: i=init h=http g=https q=quit
    --- end screen ---

    --- boot checks ---
    [PASS] common banner: expected 'C64-HTTPS CLIENT V0.1'
    [PASS] uci backend banner: expected 'UCI NETWORKING'
    [PASS] no foreign backend banner: none present
    [PASS] no FAILED on screen: no failure message
    [PASS] main menu reached: expected 'Q=QUIT'

    PASS: uci PRG booted cleanly to the menu
    Disabling UCI...
    Released DeviceLock(192.168.1.237)

    Here is the output of phase3_tcp_echo.py:

    % python3 tools/uci/phase3_tcp_echo.py
    net_init = $2000
    net_dhcp_acquire = $21CA
    net_dns_resolve = $2497
    net_tcp_connect = $2259
    net_tcp_send = $232B
    net_tcp_close = $245A
    net_poll = $2044
    net_recv_byte = $2536
    net_send_len = $B9B1
    net_tcp_state = $B9B0
    net_last_error = $B9AF
    uci_host_buf = $6760
    MemoryPolicy reserved 7 region(s); arbiter allocations:
    $4200-$42FF trampoline
    $4300-$433F host_buf
    $4340-$435F test_string
    $4360-$43DF result_buf
    $43E0-$43E0 sentinel
    $43E1-$43E1 progress
    $43E2-$43E2 connect_carry
    $43E3-$43E3 send_carry
    $43E4-$43E4 result_len
    $43E5-$43E5 poll_count
    Dev host LAN IP : 192.168.1.212
    Echo port : 7777
    Test string : b'HELLO UCI'
    Echo server listening on 192.168.1.212:7777
    Routine size : 175 bytes @ $4200
    Acquired DeviceLock(192.168.1.237)
    Enabling UCI (Command Interface)...
    Resetting machine...
    run_prg(PRG)...
    net_initialized = $01
    net_tcp_state = $00
    Sending 'Q' to exit PRG main_loop...
    /Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:410: UserWarning: write_memory($0277, 2 B): no safe_regions declared and unknown_policy is not 'allow'
    self.write_memory(
    /Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:414: UserWarning: write_memory($00C6, 1 B): no safe_regions declared and unknown_policy is not 'allow'
    self.write_memory(
    /Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:428: UserWarning: write_memory($4200, 64 B): no safe_regions declared and unknown_policy is not 'allow'
    transport.write_memory(
    /Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:428: UserWarning: write_memory($4240, 64 B): no safe_regions declared and unknown_policy is not 'allow'
    transport.write_memory(
    /Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:428: UserWarning: write_memory($4280, 47 B): no safe_regions declared and unknown_policy is not 'allow'
    transport.write_memory(
    /Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:432: UserWarning: write_memory($4300, 32 B): no safe_regions declared and unknown_policy is not 'allow'
    transport.write_memory(HOST_BUF_ADDR, host_bytes)
    /Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:433: UserWarning: write_memory($4340, 9 B): no safe_regions declared and unknown_policy is not 'allow'
    transport.write_memory(TEST_STRING_ADDR, TEST_STRING)
    Disabling UCI...
    Released DeviceLock(192.168.1.237)
    Traceback (most recent call last):
    File "/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py", line 537, in
    raise SystemExit(main())
    File "/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py", line 438, in main
    transport.write_memory(labels["uci_host_buf"], host_bytes)
    File "/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py", line 158, in write_memory
    self._memory_policy.check_write(addr, len(data), override=override)
    File "/Users/armitage/git/c64-test-harness/src/c64_test_harness/memory_policy.py", line 428, in check_write
    raise MemoryPolicyError(
    c64_test_harness.memory_policy.MemoryPolicyError: write_memory($6760, 32 B) → $677F blocked: overlaps reserved region $6000-$9FFF (segment:CRYPTO_HOT). Pass override="" to bypass for a single call, or update the harness MemoryPolicy.

  3. JC-000 commented on Aug 14, 2026

    @JC-000
    Owner

    My last answer was wrong. Sorry — that power cycle was wasted effort.

    You cold-cycled the machine and got the identical failure, so the wedge I pointed you at is ruled out: that one is transient device state and a hard power cycle clears it. Yours is deterministic, which is the opposite signature. Retracting it.

    Two smaller corrections from that same comment, because both sent you looking in the wrong place:

    It isn't locked up — it's spinning. tls_recv_encrypted (src/tls13.s) prints ENC1, then RX, then loops net_poll + tls_record_recv_and_decrypt up to 65,536 times before giving up. A UCI net_poll is a full firmware command round-trip at ~40 ms, so that loop runs for about 44 minutes. progress=0x03 only means "entered http_get, hasn't returned yet" — where it sits for the entire handshake. The console has nothing new to print.

    That's also why there were no artifacts. The script writes its artifact dir when it finishes or when it hits SENTINEL_POLL_TIMEOUT, default 600 s. Interrupting before that leaves nothing behind. Run it with SENTINEL_POLL_TIMEOUT=180 and you'll get /tmp/uci_https_debug/<timestamp>/ including tls_state_dump.json.

    We reproduced your failure, by turning the REU off

    Not a hypothesis — a reproduction. On our U64E, same REU-profile PRG, two runs back to back at 48 MHz, the only variable being RAM Expansion Unit:

    REU result
    Enabled PASS in 83.4 s, http_status=200, body decrypted
    Disabled stalls forever at RX, progress=0x03

    And the failing screen is character-for-character yours:

    SYS16896
    CH
    SH
    HK1
    KEYS
    ENC1
    RX
    

    Why

    The default make BACKEND=uci build — the 62,977-byte PRG you have — is the REU profile. In it, X25519's field multiply (src/crypto/fe25519.s) fetches its multiplication rows from the REU by DMA on every call.

    With no REU the DMA silently does nothing. It doesn't fault; the rows just stay at whatever reu_mul_init left behind, so every multiply returns a wrong-but-consistent result. The consequence lines up exactly with your screen:

    • CH, SH — plaintext, so they work regardless. Your ServerHello arrives fine.
    • HK1, KEYS — handshake keys get derived, but from a wrong ECDH shared secret.
    • ENC1, RX — the first encrypted record arrives and fails its AEAD tag. Forever.

    Corroborating detail from our failing run: tcp_recv_head and tcp_recv_tail both sat at $02B2, meaning the full ~690-byte server flight arrived and was consumed. The bytes are all there. The C64 just can't decrypt them.

    What to try

    Run the onchip image instead. It needs no REU at all — it generates those rows on the CPU:

    c64-https-uci-onchip.prg (also as .d64)

    Point the test at it with C64_PRG=/path/to/c64-https-uci-onchip.prg, or build it yourself with make clean && make BACKEND=uci USE_NISTCURVES_ONCHIP=1.

    We have run that image on a U64E with the REU explicitly disabled and it completes the full handshake — 44.9 s at 48 MHz, 262 s at 8 MHz, 2131 s at stock 1 MHz. So if the REU is your problem, this image should just work, unchanged.

    The other fix is to enable the REU: Settings → C64 and Cartridge Settings → RAM Expansion Unit. Worth knowing that the C64 Ultimate ships with it Disabled out of the box, and that setting is exactly what this build needs. Above roughly 18 MHz the onchip build is the faster of the two anyway, so it's a reasonable default even with an REU present.

    Two things that would confirm it

    1. net_last_error. This is the discriminator, and it's why I'm asking rather than deflecting — the two failure modes look identical on screen and differ only here:

      • 0x00 → network is healthy, the failure is in the crypto → REU. That's what our REU-less reproduction produced.
      • 0x86 → UCI_ERR_READ_FAIL, which is the device wedge I wrongly sent you after.

      Note tls_recv_sub_progress is 0x02 in both cases, so it can't tell them apart — which is part of why I misread yours.

    2. Whether your Ultimate has the REU enabled (the menu path above).

    Both are in tls_state_dump.json once you let the script reach its timeout; net_last_error is also printed directly in the failure dump.

    Your phase3_tcp_echo.py crash is our bug, and it's fixed

    MemoryPolicyError: write_memory($6760, 32 B) → $677F blocked: overlaps
    reserved region $6000-$9FFF (segment:CRYPTO_HOT)
    

    Nothing to do with your setup — that script cannot complete on any device. It DMA-writes uci_host_buf to pre-seed the hostname, and uci_host_buf sits at $6760, inside a region the harness's memory guard reserves wholesale. The write is legitimate; it was just missing the override= marker that says so. Fix in #105.

    Thanks for pushing back on the first answer instead of accepting it. The cold power cycle is what made this diagnosable.

  4. added a commit that references this issue on Aug 14, 2026
  5. Armitage64 commented on Aug 14, 2026

    @Armitage64
    Author

    Confirmed that phase3_tcp_echo.py now runs successfully.

    Confirmed that test_https_local.py now runs successfully with 16 MB REU enabled. Note that the readme.md does not indicate the REU configuration is required in the section labelled "Ultimate 64 Elite Hardware Tests (UCI backend)". Either add this to the Readme, or update the test to enable and configure the REU via the command interface. A test to verify the REU is enabled at the beginning of the script would be useful.

  6. JC-000 commented on Aug 14, 2026

    @JC-000
    Owner

    Your diagnosis was right, and your close-out request is now the fix

    Confirmed on both counts. And you were right that a check at the start of the script is the better answer than a README paragraph — a prerequisite a program can verify should not be left as prose. That is what shipped: #107, the check first and the docs second.

    What changed

    tools/uci/_reu_preflight.py, called under the DeviceLock the script already holds, right after enable_uci and before the reset that starts the long run:

    • REU-profile build meets a device with the REU disabled → exits 4 in about 2 seconds with a message naming both remedies (enable it via Settings → C64 and Cartridge Settings → RAM Expansion Unit, or build USE_NISTCURVES_ONCHIP=1, which needs no REU). Not after 44 minutes.
    • On-chip build → let through untouched, with no REST call at all. That case mattered most: a blanket check would have broken the exact configuration we recommend to people without an REU. The profile is detected from build/labels.txt using four independent markers checked as a union, so no single upstream rename can misclassify an on-chip build and block it.
    • C64_SKIP_REU_PREFLIGHT=1 if you ever want it out of the way.

    Guarded: test_https_local.py (and the test_https_print_body.py / test_https_local_p384.py wrappers that delegate to it), test_https_bad_finished.py, bench_ecdsa_u64e.py. Not guarded, on purpose: boot_check, phase2_check, phase3_tcp_echo, test_http_local, test_http_live — none of them touches the REU.

    Measured on our U64E at 48 MHz, all three cases, REU setting restored afterwards:

    build device REU result wall
    REU Disabled exit 4, actionable message 2.08 s
    REU Enabled PASS, HTTP 200 128.42 s
    on-chip Disabled PASS, HTTP 200 90.99 s

    The check itself costs 0.073–0.081 s on the REU path and 0.001–0.002 s on the on-chip path.

    One thing it deliberately does not do is enable the REU for you, which you offered as the other option. Our U64E is queue-shared across several projects and its config writes persist until the next power cycle, so a test that silently reconfigures the hardware trades a legible error for a mystery two runs later on somebody else's branch. The message tells you the menu path instead.

    The README's "Ultimate 64 Elite Hardware Tests (UCI backend)" section now states the requirement and that the preflight enforces it.

    A correction to something I told you earlier

    In the diagnosis comment I suggested pointing the test at the on-chip image with C64_PRG=…. That only works for boot_check.py. The other scripts read build/c64-https.prg and build/labels.txt from the same link and DMA to addresses out of those labels, so a mismatched pair would misbehave in confusing ways. The right way is to build it: make clean && make BACKEND=uci USE_NISTCURVES_ONCHIP=1. Sorry for the loose advice.

    The wedge note that sent you on the wasted power cycle has also been fixed at the source (#106). That screen — CH / SH / HK1 / KEYS / ENC1 / RX, never GOT2 — has two causes, and our notes only recorded one, which is why I confidently gave you the wrong one. It now carries the discriminator (net_last_error: $86 = device wedge, $00 = no REU behind a REU-profile build) and an explicit instruction not to act on the screen alone. Your report is what exposed that gap.

    Thank you — properly

    Across #90, #93 and #97 you have found three real gaps and caught one genuine misdiagnosis of ours:

    Every one of those is a defect that only shows up for someone arriving from outside the bench it was developed on, and every report was precise enough to act on directly — the screen capture in particular is what made the REU diagnosis possible, and pushing back on my first answer instead of accepting it is what got us there. That is genuinely valuable, and we would not have found any of it on our own.

    Closing this. Please keep filing.

  7. added a commit that references this issue on Aug 14, 2026
    bf927eb
  8. added 2 commits that reference this issue on Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions