diff --git a/CLAUDE.md b/CLAUDE.md index 580600d4..d6d2c6e2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -184,7 +184,13 @@ drops a symbol fails the link by name on both backends. Surface: - Error codes: ip65 `$40-$7F` (`ip65_errors.inc`, `NET_ERR_IP65_*`), UCI `$80-$BF` (`uci_errors.inc`, `UCI_ERR_*`). The UCI range is ONE namespace shared with c64-wireguard — `$8A UCI_ERR_LONG_READ` is theirs - and reserved here; allocate new codes in SPEC §13.2's table first. + and datagram-only. **`$FFFF` is the SOCKET_READ no-data sentinel on both + transports** (idle polls answer it) and must be excluded before any + over-claim test — both fleet adapters misfiled it independently (#140). + `net_poll` caps the copy at the request and never emits `$8A`; a header + above the request that is NOT `$FFFF` leaves `$8B UCI_ERR_BAD_READ_HDR` + (C=0, stream continues) — the stream-family counterpart of `$8A`. Allocate new codes + in SPEC §13.2's table first — `$8B` was the first one allocated that way. - Gone, per §13.1: `net_tcp_set_recv_cb` (stub), `net_recv_ready`, `net_dhcp` (alias), and `net_print_ip` — IP printing is consumer UI and is now `print_local_ip` in `boot.s`, one copy for both backends. @@ -272,7 +278,10 @@ it never writes device config. `C64_SKIP_REU_PREFLIGHT=1` bypasses. wall) and a REU-profile build with no REU (`net_last_error=$00` — the row-fetch DMA no-ops, the X25519 secret is wrong, the first AEAD tag fails, and it *spins* for ~44 min). Check `net_last_error`; the REU half - is now excluded by the preflight above. + is now excluded by the preflight above. `$86` discriminates only *within* + that stall: it is set on any SOCKET_READ error bit, including the one + that ends a normal fetch, so it is present in PASSING runs too + (github.com HTTP 200, 2026-08-28) and is never on its own a wedge. - **Lease poisoning**: resetting the C64 with a live firmware socket makes `GET_IPADDR` return 0.0.0.0 forever ("REQUESTING DHCP" loop). Only a wall power cycle clears it. Rigs therefore let fetches finish and send diff --git a/docs/engineering-notes.md b/docs/engineering-notes.md index a7931020..c304d928 100644 --- a/docs/engineering-notes.md +++ b/docs/engineering-notes.md @@ -985,6 +985,14 @@ Disabled ⇒ stalls at RX (issue #97). enable the REU, or use an onchip build (c64-https-uci-onchip.prg needs none) + `$86` is a discriminator only *inside* this stall, never on its own. + `net_poll` sets it on any SOCKET_READ error bit, and one such bit ends + an ordinary fetch: the live github.com run of 2026-08-28 finished + `http_status=200`, `http_get` C=0, and still dumped + `net_last_error=$86` with `net_tcp_state=$00` (the `$86` path forces + `NET_TCP_ERROR`, then `net_tcp_close` returns it to CLOSED). Reading a + bare `$86` as a wedge is the #97 mistake in a new costume. + `tls_recv_sub_progress` is `$02` in BOTH and distinguishes nothing. Do not act on the screen alone — that mistake was made on #97 and cost an outside contributor a wasted cold power cycle. @@ -2839,3 +2847,129 @@ region ends); hashes changed, as they must. 2026-08-24 and only a cross-repo diff noticed. - c64-wireguard#48 keeps its own items (UDP-family renames, manifests); `d9cd021`'s ring clamp is flagged there as §13.3 context for wg#46. + +### #140 — the SOCKET_READ header is not a length (same day) + +Filed by the c64-wireguard lane while reviewing the above: `net_poll` took +the response header straight into `uci_poll_rem` and copied; a wild +header was survived only because the copy loop re-checks ring-full and +DATA_AV per byte. They had learned it the hard way — c64-wireguard PR #62 +trusted the header and copied ~18 KB through `$D000`, leaving packet +bytes in the VIC registers. + +**First cut, and why it was wrong.** Ported their UDP check verbatim: +compare the header to `uci_req_len`, and on an over-claim drop the +response, emit `$8A UCI_ERR_LONG_READ`, mark the socket `NET_TCP_ERROR`. +It fired on **every** real github.com session, at the first poll, with +the ring empty (`tls_last_state=2`). Adding the pair to the rig's +post-mortem dump gave the numbers: + + uci_req_len $0200 + uci_read_hdr $FFFF + +fw 3.14d answers a 512 B TCP `SOCKET_READ` with `$FFFF` — the same +sentinel c64-wireguard measured for a 1500 B UDP request. On TCP the +header is not a delivered count; the FIFO simply delivers what it has +(bounded by the request) and `DATA_AV` drops. "Over-claim" is therefore +routine on this path and the wrong frame for a code; a drop would abort +every handshake, which is exactly what it did. + +**What landed instead**: the copy count is capped at `uci_req_len` — a +memory-safety bound, not an error. Bytes beyond the request were never +delivered (they stay in the firmware for the next poll), so the cap loses +nothing. `$8A` stays reserved with c64-wireguard's UDP meaning (datagram +length exceeded the request → datagram dropped) and this adapter never +emits it; the registry row in c64-lib-contract#139 is being reworded to +scope the meaning by transport, because two UCI-family adapters were +about to give one byte two operational meanings — the `$88` failure one +row down, caught by the cross-repo view a second time in one release +cycle. + +The reverse port — our ring-free clamp into c64-wireguard — does NOT +apply either: their inbound is a flat interlocked buffer with no wrap, +and on 3.14d any request size other than 512 breaks their read path +outright (GideonZ/1541ultimate#802). + +**Detector kept.** A bare cap would also have normalised a third case — +a header above the request that is *not* `$FFFF`, which the firmware has +no documented mode for — silently. The review on c64-lib-contract#139 +asked for that to stay distinguishable, and it does: `$8B +UCI_ERR_BAD_READ_HDR`, a best-effort breadcrumb on the §13.1 short-write +pattern (copy still capped, C=0, stream continues, code left in +`net_last_error`). Allocated in SPEC §13.2's table before the adapter +emitted it — the first code to go through the rule the table exists for. + +**Correction, same evening — `$FFFF` is the no-data sentinel.** The +c64-wireguard lane measured every *idle* UDP `SOCKET_READ` returning +header `$FFFF`, `rx_len` 0, across 11 sizes. Re-reading our own capture: +`tls_last_state=2` means the first polls after ClientHello, while +ServerHello is still in flight — idle polls. "~690 B queued" above was +an inference, not a measurement, and the mechanism note warned against +exactly that. So `$FFFF` is the firmware's no-data sentinel on both +transports; the drop-and-error cut aborted on an empty read, and +c64-wireguard's "`$8A` fires routinely during healthy sessions" runbook +line was the same misfiling, documented as a firmware quirk for four +days. The shipped code is unaffected (cap, then the `DATA_AV` check +copies zero bytes) but every comment that said "more than you asked" +was wrong and is corrected. Registry rows now pair `$8A` (datagram: +drop, terminal) with `$8B` (stream: cap, advisory) as one observation +with family-specific disposition, and require excluding `$FFFF` first. + +**And the founding incident retires too.** The contract reviewer went +back to c64-wireguard PR #62 (`6fecc97`), the commit `$8A` was born +on: its own message says the copy count came straight from the header +into a loop whose SMC store bumps its own high byte, from a buffer at +`$89D4`. `$FFFF` = 65535 as a copy count from an idle poll writes +straight through `$D000` — the ~18 KB, the VIC damage, all of it, with +no firmware over-claim involved. So after excluding the sentinel there +is **no measured non-sentinel over-claim on either transport**; `$8A` +and `$8B` are reserved defensively, and the registry says so. The rule +that travels: exclude the sentinel before any length arithmetic, and +never let a device-supplied count be the only bound on a store loop — +which is what the cap in `net_poll` now guarantees regardless of what +the header says. + + +**The UDP header, measured (c64-wireguard lane, 2026-08-25, fresh power +cycle, each row repeated):** 600/768/1024/1280 B datagrams against a +512 B request all return `rx_len` 512 and header `$0200` — the delivered +count, never the true size, never above the request. So on 3.14d the +over-claim condition has **no observable trigger on either transport**: +TCP's header is a sentinel on idle polls, UDP's is the delivered count. +The two transports are not uniform, and each adapter had generalised +from its own to the other. Consequences recorded in c64-lib-contract#139: +`$8A`/`$8B` stay allocated but the "adapter drops it" clause is gone — +a truncated 1280 B datagram is byte-identical to a complete 512 B one in +every readable register, so truncation is silent and the consumer's MTU +pin is the sole defence (now stated in SPEC §13.3). For c64-https the +TCP cap in `net_poll` (#143) remains the right bound and is unaffected. + +**Correction (c64-wireguard lane, 2026-08-26): the 512 cap was theirs, and +ours.** `UCI_READ_CHUNK_MAX = 512` was c64-wireguard's Phase 3 MVP +constant; having only ever asked for 512 they measured only ever +receiving 512 and reported it as a firmware ceiling — and our +`uci_errors.inc` says the same "conservative MVP choice". The real cap is +**894** = `CMD_MAX_REPLY_LEN` (896) − 2 (GideonZ/1541ultimate#802, +root-caused by chrisgleissner, re-measured on a U64E: 512-893 B datagrams +arrive whole with the header reporting the TRUE length). A 1024 request +is rejected with `82,PARAMETER(S) OUT OF RANGE` on STATUS — which our +adapter, like theirs, drains without reading. **Never request exactly +894 on 3.14d**: it fills the response queue exactly and the FPGA repeats +it forever. So "the header never reports the true length" was an +artifact of always truncating; the indistinguishability problem is real +only for genuinely oversized datagrams. Unchanged: no non-sentinel +over-claim observed; `$FFFF` is `lwip_recv()` returning −1 stored as +signed 16-bit. + +Two consequences here. (1) Our clamp had the same latent trap the +wireguard lane hit when their cap moved to `$037D`: `cmp #>MAX / bcc / +bne / lda lo / beq` accepts the equal-high-byte case only when the low +byte is zero — correct for `$0200`, wrong for any other cap, and in our +direction it would have *raised* a sub-cap request to the cap, i.e. the +wikipedia-stall class. Rewritten as a full 16-bit compare in #143 so +the constant can move. (2) Raising `UCI_READ_CHUNK_MAX` toward 893 is a +plausible throughput win on TCP (fewer ~40 ms round-trips per flight) +but every byte read carries a fence (~5.45 ms at 1 MHz), so it is a +measured follow-up, not a free change — and any test around it should +find the accepted length by bisection, not hard-code it. + diff --git a/src/net/uci/net.s b/src/net/uci/net.s index 2e3590ec..514cf95d 100644 --- a/src/net/uci/net.s +++ b/src/net/uci/net.s @@ -10,15 +10,13 @@ ; GET_IPADDR command. Does NOT run DHCP ourselves — ; the firmware already did that before the PRG started. ; -; Exports two symbol families: -; -; (a) net_abi.inc contract — the long-term public names. -; (b) legacy caller names currently imported by boot.s / http.s / -; tls_record_io.s — kept as thin aliases until those callers are -; migrated onto net_abi.inc in a later phase. -; -; Also publishes `net_banner_str`, the backend-specific banner line -; consumed by boot.s's startup print. +; Exports exactly the surface src/net_abi.inc imports (c64-lib-contract +; SPEC §13 core/TCP/DNS families, issue #70) plus the two c64-https +; extensions declared there: net_recv_byte (the drain entry) and +; net_banner_str (the backend-specific banner line boot.s prints). +; Error codes live in uci_errors.inc, in the §13.2 UCI range $80-$BF, +; which is ONE namespace shared with c64-wireguard — allocate in SPEC +; §13.2's table first. .include "uci_regs.inc" .include "uci_errors.inc" @@ -168,14 +166,18 @@ net_poll: bne :+ rts ; ring effectively full — poll again later : - ; clamp to UCI_READ_CHUNK_MAX (512 = $0200) - lda uci_req_len+1 - cmp #>UCI_READ_CHUNK_MAX - bcc @len_clamped ; high < 2 -> under 512, keep - bne @len_use_max ; high > 2 -> over, clamp + ; clamp to UCI_READ_CHUNK_MAX — full 16-bit compare. The previous + ; shape (cmp #>MAX / bcc / bne / lda lo / beq) accepted the + ; equal-high-byte case only when the low byte was zero: correct + ; for $0200, silently wrong for any other cap (c64-wireguard hit + ; exactly that at $037D and read the result as firmware + ; misbehaviour). Written cap-shape-independently so raising the + ; constant cannot reintroduce it. lda uci_req_len+0 - beq @len_clamped ; exactly 512, keep -@len_use_max: + cmp #UCI_READ_CHUNK_MAX ; C=1 iff req >= MAX + bcc @len_clamped ; req < MAX -> keep lda #UCI_READ_CHUNK_MAX @@ -292,6 +294,46 @@ net_poll: rts @have_data: + ; --- Bound the copy by the request (#140, SPEC §13.3) -------------- + ; The response header is not a delivered-byte count. On fw 3.14d + ; $FFFF is the NO-DATA sentinel on both transports (c64-wireguard: + ; every idle UDP poll; here: the idle polls after ClientHello while + ; ServerHello is still in flight answer $FFFF to a 512 B request). + ; The copy below survived it only because it re-checks ring-full + ; and DATA_AV per byte and so copied zero bytes; a refactor of the + ; loop would have turned the header into a runaway copy. So the + ; count is capped at uci_req_len — what this poll actually asked + ; for, which the ring clamp above may have made smaller than + ; UCI_READ_CHUNK_MAX. Bytes beyond the request were never delivered + ; (they stay queued for the next poll), so the cap loses nothing. + ; A drop-and-error here (the first cut, emitting UCI_ERR_LONG_READ) + ; aborted every real handshake on its first idle poll — the same + ; misfiling c64-wireguard carried for four days as a "firmware + ; quirk". $FFFF MUST be excluded before any over-claim test. + lda uci_req_len+0 + cmp uci_poll_rem+0 + lda uci_req_len+1 + sbc uci_poll_rem+1 ; C=1 iff req >= header (16-bit) + bcs @len_bounded + ; header > request. $FFFF is the no-data sentinel (routine, copied + ; as zero bytes by the DATA_AV check); anything else above the + ; request is a header the firmware has no documented mode for. + ; Leave a breadcrumb for that case ($8B, best-effort: C=0, stream + ; continues — the cap below makes it safe either way) so a + ; post-mortem can tell the two apart. Allocated in c64-lib-contract + ; SPEC §13.2 before use; the datagram-family counterpart is $8A. + lda uci_poll_rem+0 + and uci_poll_rem+1 + cmp #$FF + beq @len_cap ; $FFFF sentinel — routine, no breadcrumb + lda #UCI_ERR_BAD_READ_HDR + sta net_last_error +@len_cap: + lda uci_req_len+0 + sta uci_poll_rem+0 + lda uci_req_len+1 + sta uci_poll_rem+1 +@len_bounded: ; Copy exactly (uci_poll_rem) bytes from UCI_RESP_DATA into the ; ring at tcp_recv_buf + tcp_recv_tail, advancing the masked tail. ; The store uses SMC on @rb_store so we can hit the full 4 KB diff --git a/src/net/uci/uci_errors.inc b/src/net/uci/uci_errors.inc index 3107962b..d353b73a 100644 --- a/src/net/uci/uci_errors.inc +++ b/src/net/uci/uci_errors.inc @@ -21,12 +21,28 @@ UCI_ERR_READ_FAIL = $86 ; SOCKET_READ returned an error bit UCI_ERR_SHORT_WRITE = $87 ; SOCKET_WRITE wrote fewer bytes than requested UCI_ERR_NO_SOCKET = $88 ; socket-open response yielded no socket_id UCI_ERR_WAIT_TIMEOUT = $89 ; a bounded wait exceeded its wall-clock budget -UCI_ERR_LONG_READ = $8A ; SOCKET_READ claimed more bytes than we asked - ; for. Allocated by c64-wireguard (2026-08-24); - ; reserved here with the same meaning. This - ; adapter does not emit it yet — net_poll - ; stops at DATA_AV rather than checking the - ; claimed length against the request. +UCI_ERR_LONG_READ = $8A ; Datagram family (c64-wireguard): the + ; SOCKET_READ header claimed more than the + ; request and was NOT the $FFFF no-data + ; sentinel; the datagram is dropped. Reserved + ; here, NEVER emitted: on a stream the same + ; observation is $8B below. $FFFF is the + ; firmware's no-data sentinel on BOTH + ; transports and must be excluded before any + ; length arithmetic (#140; both adapters got + ; this wrong independently). No non-sentinel + ; over-claim has been observed on either + ; transport; both codes are reserved + ; defensively. +UCI_ERR_BAD_READ_HDR = $8B ; Stream family: SOCKET_READ header exceeded + ; the request and was NOT the $FFFF no-data + ; sentinel — a value the firmware has no + ; documented mode for. Best- + ; effort breadcrumb (§13.1 short-write pattern): + ; the copy is still capped at the request and + ; net_poll returns normally; the code stays in + ; net_last_error for post-mortems. Allocated in + ; SPEC §13.2's table first (c64-lib-contract#139). ; TCP state values stored in net_tcp_state. The NET_TCP_* spellings in ; net_states.inc are the contract's (§13.1); these aliases keep the @@ -40,6 +56,27 @@ UCI_TCP_CONNECT_FAIL = NET_TCP_CONNECT_FAIL UCI_DATA_QUEUE_MAX = 800 ; SOCKET_READ max-length cap we ask for per net_poll cycle. -; 512 bytes is well under the 4 KB ring and the firmware data queue, -; and matches the conservative Phase 3 MVP choice. +; +; 512 is a CONSUMER choice, not the firmware's limit — it was c64-wireguard's +; Phase 3 MVP constant, and both repos carried it as a hardware ceiling +; because nothing larger had ever been requested. The FIRMWARE ceiling is +; version-dependent; 893 is OURS on every version. +; +; fw 3.14d 894 = CMD_MAX_REPLY_LEN 896 - the 2-byte header +; (GideonZ/1541ultimate#802, re-measured on a U64E: +; 512-893 B arrive whole in one poll). NEVER request +; exactly 894: the reply exactly fills the response queue +; and the FPGA repeats it forever with DATA_AV asserted. +; fw 3.15 + #806 accepts up to 1472 and the 894 hang is gone, but a reply +; over 893 payload bytes comes back SPLIT across response +; blocks (the command interface's Data More mechanism), and +; the 2-byte header is then the total for the whole command +; rather than the count in the first block. net_poll reads +; ONE block, so a request above 893 would silently drop the +; continuation. +; +; So 893 is the number to raise toward on either firmware, and going past it +; needs a multi-block reader first. Not a free win either: every byte read +; carries a uci_fence (~5.45 ms at 1 MHz). net_poll's clamp is written +; cap-shape-independently so the change is one constant. UCI_READ_CHUNK_MAX = 512 diff --git a/tools/uci/rig_https_live.py b/tools/uci/rig_https_live.py index dca7a39d..3df5a856 100644 --- a/tools/uci/rig_https_live.py +++ b/tools/uci/rig_https_live.py @@ -338,6 +338,7 @@ def main() -> int: "net_init", "net_initialized", "uci_socket_id", "net_last_error", "net_tcp_state", "tcp_recv_head", "tcp_recv_tail", + "uci_req_len", "uci_read_hdr", # #140: SOCKET_READ request vs claim "http_resp_buf", "http_resp_len", "http_status", "tls_state", "tls_last_state", ] diff --git a/tools/uci/rig_https_local.py b/tools/uci/rig_https_local.py index 8ce272fe..bd74b3f9 100644 --- a/tools/uci/rig_https_local.py +++ b/tools/uci/rig_https_local.py @@ -715,7 +715,7 @@ def r16(name: str) -> int: "tls_recv_progress", "tls_recv_sub_progress"): if name in labels: print(f" {name:22s} : 0x{r8(name):02X}") - for name in ("tcp_recv_head", "tcp_recv_tail", + for name in ("tcp_recv_head", "tcp_recv_tail", "uci_req_len", "uci_read_hdr", "http_status", "http_resp_len"): if name in labels: print(f" {name:22s} : ${r16(name):04X}")