Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,10 +143,10 @@ Variables:
both exist for uci only, so there is no ip65 comb build. Confirm from
the `ld65 -C` line, not from the profile name.
- `PQ_HYBRID=1` — links `libs/mlkem` (c64-mlkem v0.6.0, ABI 3) for the
X25519MLKEM768 hybrid (P4: `tls_connect` runs KeyGen and offers 11EC;
a 11EC ServerHello is read and Decapsed, but the **key schedule is not
hybrid yet** (P5), so against an 11EC server the handshake fails at the
first encrypted record; a 001D server completes as before). uci
X25519MLKEM768 hybrid (`tls_connect` runs KeyGen and offers 11EC; an 11EC
ServerHello is read and Decapsed and the handshake secret is extracted
from `mlkem_ss || x25519_ss` (P5), so an 11EC server completes HTTPS; a
001D server completes as before). uci
+ `USE_NISTCURVES_ONCHIP=1` or `_COMB=1`, with or without `TRUST_STORE`;
`$(error)` on ip65, uci-m3 and the uci REU profile. Unset = byte-identical
PRG on every profile. Layout in Memory layout; asserts in
Expand Down Expand Up @@ -587,6 +587,12 @@ never exercised:
during the multi-minute crypto stall.
- Cloudflare is out of scope: ignores MFL and enforces a ~15 s deadline.

PQ_HYBRID (P5): X25519MLKEM768 handshakes complete on the U64E against a local
OpenSSL 3.6.5 `s_server`, onchip and comb images. Live servers, comb image only:
browserleaks.com (HTTP 200) and en.wikipedia.org (HTTP 301 at `/`); github.com and
lwn.net do not offer 11EC (handshake_failure to a hybrid-only hello). Image hashes,
make lines and timings: the P5 section of engineering-notes.

Handshake flow, both backends: CH → SH (X25519) → EE, Certificate,
CertificateVerify (P-256 via sibling) → server Finished verified → client
Finished → traffic keys, seq counters reset (§5.3) → GET → response.
Expand Down Expand Up @@ -693,7 +699,7 @@ already refused a step later, as `DF_ERR_TYPE = $04`). Test:

UCI CH is sectioned (`CH_SECTIONED`, follows `TLS_STREAM_DEFRAME`): `tls_ch_ks_ptr/len` (16-bit, BSS, one-shot) name a bulk key-share that `tls_ch_send_sections` streams from its own buffer between the head and tail staged in `tls_rec_buf`, as ONE record; 0 = classic X25519-only hello (every build without PQ_HYBRID). Under PQ_HYBRID=1 `tls_connect` arms it itself (`pq_connect_prep`, `src/crypto/pq_hybrid.s`: DRBG seed, KeyGen, dk zeroed at once, ek in `cert_buf`; 11EC = ek || x25519, then 001D also offered); no caller arms anything. The sender clears it on every exit. HRR random -> `tls_sh_err=$01` (both backends). Test: `test_tls_handshake.py` [2d-2k], [3d].

PQ_HYBRID ServerHello (P4): `pq_sh_feed` streams the SH across any number of plaintext records (`pq_hybrid.s`; `tls_parse_server_hello` is assembled out), 11EC needs exactly 1,120 B = ct 1,088 (into `cert_buf`) || x25519 32; ct must be Decapsed (`pq_sh_decaps`, before the X25519 mult) before anything writes `cert_buf`. unsolicited SH extensions and a malformed/oversize record header while waiting for the SH are refused (fail fast, `$0D`/`$0E`); KeyGen and Decaps end with `pq_mlkem_scrub` (polys + `$B100` window zeroed); `tls_sh_err` codes `$01-$0E` are in `src/constants.inc`; `tls_kex_group` ($1D/$EC) and `tls_mlkem_ss` are P5's inputs; seed + ss are one scrub span. Tests: `tools/test_pq_sh.py` (fast), `tools/test_pq_kem.py` (KeyGen/Decaps/tls_connect), oracle `tools/pq_keylog_check.py` (hardware, vs `openssl s_server -keylogfile`).
PQ_HYBRID ServerHello (P4): `pq_sh_feed` streams the SH across any number of plaintext records (`pq_hybrid.s`; `tls_parse_server_hello` is assembled out), 11EC needs exactly 1,120 B = ct 1,088 (into `cert_buf`) || x25519 32; ct must be Decapsed (`pq_sh_decaps`, before the X25519 mult) before anything writes `cert_buf`. unsolicited SH extensions and a malformed/oversize record header while waiting for the SH are refused (fail fast, `$0D`/`$0E`); KeyGen and Decaps end with `pq_mlkem_scrub` (polys + `$B100` window zeroed); `tls_sh_err` codes `$01-$0E` are in `src/constants.inc`; `tls_kex_group` ($1D/$EC) picks the key-schedule IKM in `tls_derive_handshake_keys` (P5: $EC = the 64 B `tls_mlkem_ss || tls_shared_secret` concatenated in `input_buffer`, ML-KEM first; else the 32 B x25519 secret) and both secrets, the concatenation and `hkdf_extract`'s copy are zeroed right after the extract; seed + ss are one scrub span. Tests: `tools/test_pq_sh.py` (fast), `tools/test_pq_kem.py` (KeyGen/Decaps/tls_connect), `tools/test_pq_keyschedule.py` (hybrid IKM vs host HKDF, zeroing), oracle `tools/pq_keylog_check.py --state` (hardware: the C64's application traffic secrets vs `openssl s_server -keylogfile`'s `*_TRAFFIC_SECRET_0`, gated on `tls_reached_connected`).

### VIC-II blanking — worth 6.3%, not the fleet's "20-25%"

Expand Down
79 changes: 78 additions & 1 deletion docs/engineering-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -4235,10 +4235,87 @@ Decisions and findings:
cannot READ such a hello. A ChangeCipherSpec record between SH fragments
is skipped by the record layer, as before.

Oracle (hardware): the C64 cannot read the first encrypted record yet, so
Oracle (hardware, P4; superseded by the P5 section below): the C64 cannot read the first encrypted record yet, so
`tools/pq_keylog_check.py` runs the RFC 8446 schedule offline on the C64's
`tls_mlkem_ss || tls_shared_secret` and `tls_transcript` (read with the CPU
parked and all RAM banked in: `PQ_DUMP=<dir>` in `rig_https_local.py`) and
compares with `s_server -keylogfile`. A match proves Decaps agreed with the
server's Encaps and that both sides hashed the same CH || SH. A run that did
not reach Decaps is INCONCLUSIVE (exit 78), never a verdict.

## PQ_HYBRID P5: hybrid key schedule, first X25519MLKEM768 handshake (2026-10-17)

`tls_derive_handshake_keys` (src/tls_keyschedule.s, `.ifdef PQ_HYBRID`) now takes
the handshake-secret IKM from the group the ServerHello chose: `tls_kex_group`
= `$EC` gives HKDF-Extract(salt = derived, IKM = `tls_mlkem_ss || tls_shared_secret`)
(draft-ietf-tls-ecdhe-mlkem: ML-KEM first, then X25519, 64 B), `$1D` (or 0, a test
that calls the routine bare) the unchanged 32 B x25519 secret. Everything after
the extract is the existing schedule; the transcript is the same SHA-256.

Decisions and findings:

- **A small concat, not a contiguous span.** `tls_mlkem_ss` sits in
NET_BSS_TAIL (below `$4200`) and `tls_shared_secret` in data.s BSS, so the
64 B IKM is copied into `input_buffer` (256 B scratch, dead since the
early-secret step) and the HKDF IKM pointer/length retargeted. NET_CODE
free drops by 61 B on the onchip profiles (166 -> 105) and 63 B on the comb
profiles (335 -> 272, 197 -> 134): about 63 B of TLS_CODE, and on onchip
2 B come back from `pq_sh_decaps` (on comb its PQ_GLUE sits in CRYPTO_HOT,
+2 there). `tools/measure_margins.py`. `hkdf_extract`
copies the IKM into `hmac_data_buf` (97 B) and HMACs it as the message, so 64
is within its limit; nothing there treats it as a key.
- **Zeroed at the extract, not at 'Q'.** Right after `hkdf_extract` the two
32 B secrets, the 64 B concatenation (`input_buffer`) and `hkdf_extract`'s
copy (`hmac_data_buf[0:64]`) are zeroed, for BOTH groups. Later HKDF calls
reuse `hmac_data_buf` for public info strings, so the copy is checked by
content (no 8-byte window of the IKM survives), and the span matters: with the
`hmac_data_buf` zeroing removed (mutant K11) IKM bytes around offset 55-63
were still there after the schedule ran. Not zeroed (unchanged, same on classic builds):
`tls_ecdhe_privkey` (until 'Q'), and SHA-256's internal block/schedule
buffers (not inspected; the following HMACs of the same routine rewrite them). Only PQ builds pay for the zeroing; the seven
knob-off PRGs are byte-identical.
- **The oracle changed with it.** With the secrets gone, `pq_keylog_check.py
--state` compares what stays: `tls_c_hs_secret` / `tls_s_hs_secret` are reused
for the APPLICATION traffic secrets after `tls_derive_traffic_keys`, which are
a pure function of the hybrid handshake secret and the transcript through the
server Finished, and `s_server -keylogfile` logs them as `CLIENT_TRAFFIC_SECRET_0`
/ `SERVER_TRAFFIC_SECRET_0`. Gate: `tls_reached_connected == 7` (#204; a run
that connected and then called `tls_close` reads `tls_state = 0`), `tls_kex_group
== $EC`, `tls_sh_err == 0`, both secrets non-zero; a non-zero seed/ss/shared
secret in a connected run is a RESIDUE failure, not a pass. The P4 offline mode
(explicit secrets as arguments) is unchanged.
- **pq_sh_decaps** lost its redundant outer `php`/`plp` (`pq_mlkem_scrub` saves
and restores P itself); 2 B back in PQ_GLUE.
- **Which-group evidence on hardware:** `openssl_listener.py --trace` prints the
ServerHello key_share group (`X25519MLKEM768 (4588)` = 0x11EC); `--groups
default` runs `s_server` without `-groups`, and OpenSSL 3.6.5 then picks
X25519MLKEM768 for our hello (it prefers it).
- **Trap: `HTTPS_SNI=` on a live run.** A PQ image built with `HTTPS_SNI=www.foo.invalid`
(for the local self-signed listener) fails a LIVE host with `df_last_err = $05`
(DF_ERR_CERT_FMT: the SAN check runs against the baked name). It looked like a
hybrid defect for one run; the same image without the override passes. Build
live images without `HTTPS_SNI`.
- **Real servers.** browserleaks.com and en.wikipedia.org negotiate 11EC (they
complete `openssl s_client -groups X25519MLKEM768 -maxfraglen 512`); github.com
and lwn.net answer `handshake_failure` to a hybrid-only hello (they would pick
001D for our dual hello).

Hardware record (U64E 10.43.23.81, 48 MHz, DeviceLock, OpenSSL 3.6.5 `s_server`;
all runs full HTTPS PASS, tls_kex_group $EC unless noted):

- onchip, `make BACKEND=uci USE_NISTCURVES_ONCHIP=1 PQ_HYBRID=1
HTTPS_SNI=www.foo.invalid`, sha256
`ab85014cdc8bb074fc9cb99180c90dc8a74ac4d8ed9f68cfda32e7d311cf5e0f`:
rig wall-clock 108 s (`-groups X25519MLKEM768`), 107 s (`-groups X25519`,
$1D), 109 s (default groups, OpenSSL picks 11EC).
- comb (REU 16 MB, `C64_INIT_WAIT=90`), `make BACKEND=uci
USE_NISTCURVES_ONCHIP_COMB=1 PQ_HYBRID=1 HTTPS_SNI=www.foo.invalid`, sha256
`265c4c583c2b5a2430daf3a7ee2a3e546d99a8b7c1c4173476e8455097cfa042`: 165 s /
164 s ($1D) / 166 s. With `PHASE_TIMING=1` (1 s poll, n=1 each) trigger to
client Finished 43.7 s hybrid vs 42.0 s X25519-only: about +1.7 s.
- live (`tools/uci/rig_https_live.py`), comb image WITHOUT `HTTPS_SNI`, `make
BACKEND=uci USE_NISTCURVES_ONCHIP_COMB=1 PQ_HYBRID=1`, sha256
`b8677dd79968fb57e475baaa0995c8979d670b159ae753b207a947d5e8be8d61`:
browserleaks.com HTTP 200 (Finished at 44.1 s; the non-PQ comb image
`ad85797b...` 42.3 s), en.wikipedia.org HTTP 301 at `/` (39.8 s). Onchip was
not run against live servers.
13 changes: 6 additions & 7 deletions src/crypto/pq_hybrid.s
Original file line number Diff line number Diff line change
Expand Up @@ -219,21 +219,20 @@ pq_connect_prep:
; pq_sh_decaps - Decaps the ServerHello's ct if the server chose 11EC.
; The args are rewritten here: nothing may sit between them and the call.
; The seed is zeroed on every successful exit (no longer needed) and by
; tls_connect's error exit; tls_mlkem_ss is kept like tls_shared_secret (the
; hardware oracle reads it) and scrubbed by 'Q' (src/session_scrub.s) or by
; the next connection's pq_connect_prep.
; tls_connect's error exit. tls_mlkem_ss is zeroed by tls_derive_handshake_keys
; right after the handshake secret is extracted (P5); on a connect that fails
; BEFORE that point it is kept like tls_shared_secret and scrubbed by 'Q'
; (src/session_scrub.s) or by the next connection's pq_connect_prep.
; -----------------------------------------------------------------------------
pq_sh_decaps:
lda tls_kex_group
cmp #$EC
bne @done
jsr pq_set_args
jsr pq_mlkem_decaps_seed ; A = status; C=1/A=$FF refused
php
pha
jsr pq_mlkem_scrub ; y/e1/e2 in $D000, m'/K'/r' in the window, any outcome
pla
plp
jsr pq_mlkem_scrub ; y/e1/e2 in $D000, m'/K'/r' in the window, any outcome (preserves C)
pla ; pla leaves C alone
bcs @fail
cmp #0
bne @fail
Expand Down
3 changes: 2 additions & 1 deletion src/tls13.s
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,8 @@ tls_connect:
@error:
.ifdef PQ_HYBRID
jsr pq_seed_zero ; the d||z seed never outlives a failed connect (the
; ML-KEM secret stays, like tls_shared_secret: 'Q' scrubs)
; ML-KEM secret stays if the key schedule never ran, like
; tls_shared_secret: 'Q' scrubs; P5 zeroes both after the extract)
.endif
lda tls_state ; preserve last attempted state
bmi :+ ; already ERROR: the record layer aborted
Expand Down
50 changes: 50 additions & 0 deletions src/tls_keyschedule.s
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@

; TLS state / buffers (data.asm)
.import tls_shared_secret
.ifdef PQ_HYBRID
.import tls_mlkem_ss, tls_kex_group
.endif
.import tls_transcript
.import tls_early_secret
.import tls_handshake_secret
Expand Down Expand Up @@ -71,7 +74,12 @@
;
; Input: tls_shared_secret (32 bytes) = x25519 result
; tls_transcript (32 bytes) = hash of ClientHello || ServerHello
; PQ_HYBRID only: tls_kex_group = $EC makes the step-3 IKM the 64 B
; tls_mlkem_ss || tls_shared_secret (ML-KEM first, copied into
; input_buffer); any other value keeps the 32 B x25519 IKM.
; Output: tls_hs_write_key/iv, tls_hs_read_key/iv filled
; PQ_HYBRID only: tls_shared_secret, tls_mlkem_ss, input_buffer[0:64]
; and hmac_data_buf[0:64] are zeroed right after step 3 (both groups)
; tls_handshake_secret saved for later use
; tls_early_secret saved for reference
;
Expand Down Expand Up @@ -172,8 +180,50 @@ tls_derive_handshake_keys:
lda #32
sta hkdf_ikm_len

.ifdef PQ_HYBRID
; X25519MLKEM768 (draft-ietf-tls-ecdhe-mlkem): the shared secret is
; mlkem_ss || x25519_ss (ML-KEM first), 64 B. Concatenate into
; input_buffer (dead scratch since step 1) and point the IKM at it.
lda tls_kex_group
cmp #$EC
bne @dhk_ikm_ready
ldx #31
@dhk_cat:
lda tls_mlkem_ss,x
sta input_buffer,x
lda tls_shared_secret,x
sta input_buffer+32,x
dex
bpl @dhk_cat
lda #<input_buffer
sta hkdf_ikm_ptr
lda #>input_buffer
sta hkdf_ikm_ptr+1
lda #64
sta hkdf_ikm_len
@dhk_ikm_ready:
.endif

jsr hkdf_extract

.ifdef PQ_HYBRID
; Both shared secrets (and the concatenation and hkdf_extract's copy
; of it in hmac_data_buf) are dead once the handshake secret exists.
lda #0
ldx #63
@dhk_zero:
sta input_buffer,x
sta hmac_data_buf,x
dex
bpl @dhk_zero
ldx #31
@dhk_zss:
sta tls_shared_secret,x
sta tls_mlkem_ss,x
dex
bpl @dhk_zss
.endif

; Copy hkdf_prk → tls_handshake_secret
ldx #31
@dhk_c3:
Expand Down
10 changes: 9 additions & 1 deletion tools/https_e2e/openssl_listener.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

openssl_listener.py --groups X25519 --port 4433 --keylog /tmp/k.log
openssl_listener.py --groups X25519MLKEM768 ...
openssl_listener.py --groups default --trace ... # OpenSSL's own group list

Prints "LISTENING <port> <openssl version>" once the server accepts, then runs
until killed. Group selection is the server's whole policy: under
Expand Down Expand Up @@ -57,14 +58,21 @@ def main(argv=None):
ap.add_argument("--port", type=int, default=4433)
ap.add_argument("--keylog", required=True)
ap.add_argument("--bind", default="0.0.0.0")
ap.add_argument("--trace", action="store_true",
help="s_server -trace: the console shows each message, including the "
"negotiated key_share group (the which-group evidence for P5 runs)")
a = ap.parse_args(argv)
from ensure_certs import ensure_certs
cert, key = ensure_certs("p256")
exe, ver = find_openssl()
Path(a.keylog).unlink(missing_ok=True)
cmd = [exe, "s_server", "-accept", f"{a.bind}:{a.port}", "-cert", str(cert),
"-key", str(key), "-tls1_3", "-ciphersuites", "TLS_CHACHA20_POLY1305_SHA256",
"-groups", a.groups, "-keylogfile", a.keylog, "-num_tickets", "0", "-state"]
"-keylogfile", a.keylog, "-num_tickets", "0", "-state"]
if a.groups != "default": # "default" = no -groups: OpenSSL's own list and preference
cmd += ["-groups", a.groups]
if a.trace:
cmd.append("-trace")
# A pty, not pipes: s_server block-buffers stdout on a pipe, and the request
# line would sit in its buffer instead of reaching us.
master, slave = pty.openpty()
Expand Down
Loading