Repository navigation
Description
Observed
The same commit and the same make arguments give different PRGs depending on the build environment's locale.
- Commit:
ded2247(the v0.5.1 release, https://github.com/JC-000/c64-https/releases/tag/v0.5.1). - Arguments:
make BACKEND=uci USE_NISTCURVES_ONCHIP=1 TRUST_STORE=1 TRUST_RELEASE=1, the shippeduci-onchiparguments. - Machine: macOS 27.0.1. GNU Make 3.81 (
/usr/bin/make).ca65 V2.18andld65 V2.18from/opt/homebrew/bin.
| environment | PRG sha256 |
|---|---|
interactive shell, LANG=en_US.UTF-8 (what make package used) |
ffa47fc32e43a16b995faee0d85d3d9312e1791a568820fedf7c0446bd27dfa6 |
env -i PATH="$PATH" HOME="$HOME" make … (no LANG, so the C locale) |
98a3d882ddb7008f6f3aa646d1cf984279840fd22e22b3dd0f80967b53cb730d |
env -i PATH="$PATH" HOME="$HOME" LANG=en_US.UTF-8 make … |
ffa47fc3…, the same as the first row |
Independent reproduction (adv-296, reported to the lane, not re-run by me): under env -i, the other two wildcard-built products also move. ip65-onchip gives 1c377d34… (released 810f1158…) and uci-comb gives 6b95c329… (released 67cb0048…).
The two PRGs differ in 4,600 bytes, the first at byte 402 (1-based, as cmp -l counts). build/flags.stamp and build/https_host.inc are identical between the two builds, so the stamp does not capture the difference.
The only difference in the ld65 map's module list is the order of two objects:
$ diff <(modules in C-locale map) <(modules in en_US map)
17d16
< tls13.o:
23d21
< tls_record.o:
24a23
> tls_record.o:
25a25
> tls13.o:
That matches how the two locales order the source file names (ls src/tls*.s):
en_US.UTF-8:tls_cert … tls_record_io tls_record tls_transcript tls13. The underscore is ignored when collating.C:tls13 tls_cert … tls_record tls_record_io tls_transcript, in byte order.
Where the order comes from (from the Makefile)
Makefile:271TOP_SRCS := $(wildcard src/*.s), with no$(sort …).Makefile:478TOP_OBJS := $(patsubst src/%.s,build/%.o,$(TOP_SRCS)).Makefile:483ALL_OBJS := $(TOP_OBJS) ….Makefile:522$(LD65) $(LD65FLAGS) -o $@ $(ALL_OBJS) …. The link order is therefore the wildcard's order.Makefile:282(src/crypto/*.s) andMakefile:286(src/crypto/shared/*.s) use the same unsorted pattern. They are not implicated by this build's diff.- uci-m3 is not affected. Its
TOP_SRCSandNET_SRCSare explicit lists (Makefile:466-470), and the M3 demo PRG916d4162…was identical under both environments.
Suspected, not verified: GNU Make 3.81's $(wildcard) returns names in an order that depends on the locale's collation (glob sorting with strcoll). The map diff and the ls orders are consistent with that. I have not read make's source to confirm it, and other make versions (4.x sorts wildcard results again) may behave differently.
Impact
- "The PRG is deterministic; its sha256 is the check" (CLAUDE.md, Build) holds only for a fixed locale, and possibly a fixed make version.
build_m3_demo.shbuilds in a scrubbed environment (env -i, so the C locale).build_prgs.sh(make package) inherits the caller's locale.- A reviewer who rebuilds a release with a different
LANGgets different bytes for every image whose sources come from a wildcard (ip65, uci-onchip, uci-comb).
Possible fix (for discussion)
- Wrap the three wildcards in
$(sort …). make'ssortcompares bytes, independent of locale. This changes the link order, and so the bytes, of every wildcard-built image relative to v0.5.1, so it belongs in a release that expects new hashes. - Alternatively, pin
LC_ALL=Cin the Makefile, or in the package scripts. - Either way, add a test: build one profile under two locales and require equal PRG hashes.
Found while staging v0.5.1. Its notes promise this issue. The v0.5.1 assets were built under LANG=en_US.UTF-8, and their hashes reproduce under that locale.
Activity
make: put the object list in flags.stamp; pin cs_CZ and the stale-ord…
CLAUDE.md: rewrap the #301 sentence
test/Makefile: per-test certifies text, fail-closed stale-order preco…
make: sort source wildcards so the PRG is locale-independent (#301)
make: put the object list in flags.stamp; pin cs_CZ and the stale-ord…
CLAUDE.md: rewrap the #301 sentence
test/Makefile: per-test certifies text, fail-closed stale-order preco…
Merge pull request #311 from JC-000/fix/301-locale-link-order