Avoid retaining three times the encoded payload in Haxe 5 Bytes.ofString #405

Haxe 5's Bytes.ofString can retain almost three times the encoded payload size when a mostly ASCII string contains one non-ASCII character.

Reproduced against current main at 5f7cfef66ed4d9543c6c1a2763bdc2148433435c, using Haxe nightly 5.0.0-preview.1+fe63b35 with runtime API 500, MSVC 19.44.35213 on Windows x64 and GCC 13.3 on Linux x86-64 under WSL. Builds use HXCPP_M64, default GC settings and smart strings.

The test creates 1,048,576 ASCII characters followed by U+0100, converts the string with Bytes.ofString, and keeps the result through three full collections. It reads the backing array's capacity directly.

The encoded contents are correct. The extra 2,097,153 bytes remain allocated for the lifetime of the result, even after full collections.

In Utf8::encode(const String&), the non-ASCII path allocates string.length * 3 bytes, fills the encoded prefix, then calls out->resize(k). That changes the visible length while keeping the worst-case backing capacity. The Haxe 5 Bytes.ofString implementation uses this overload for non-ASCII strings.

Allocate from the exact encoded byte count, or trim substantial excess capacity before returning. Compare the extra counting pass with the allocation and retained-memory costs, and preserve the ASCII fast path. This can be fixed in libhxcpp without changing the compiler or standard library.

A conventional reproduction is prepared in test/regression/Utf8EncodeCapacity/{Main.hx,build.hxml,stdout.txt}. Expected output has all three checks true. Actual output:

UTF8 encoding preserves the complete payload: true
retained UTF8 storage stays close to the output length: false
ASCII control storage stays close to the output length: true

class Main {
	static function main() {
		var correct = true;
		var bounded = true;
		var asciiExact = true;
		#if (haxe_ver >= 5 && !haxe_5_preview_1)
		var plain = haxe.io.Bytes.alloc(1 << 20);
		plain.fill(0, plain.length, 97);
		var ascii = plain.toString();
		var text = ascii + "\u0100";
		var encoded = haxe.io.Bytes.ofString(text);
		var data = encoded.getData();
		correct = encoded.length == plain.length + 2 && encoded.toString() == text;
		for (_ in 0...3) cpp.vm.Gc.run(true);
		var capacity:Int = untyped __cpp__("{0}->capacity()", data);
		bounded = capacity <= encoded.length + 65536;
		var control = haxe.io.Bytes.ofString(ascii);
		var controlData = control.getData();
		var controlCapacity:Int = untyped __cpp__("{0}->capacity()", controlData);
		asciiExact = controlCapacity <= control.length + 65536 && control.toString() == ascii;
		if (Sys.args().length > 0)
			Sys.println(haxe.Json.stringify({length: encoded.length, capacity: capacity, asciiLength: control.length, asciiCapacity: controlCapacity}));
		#end
		Sys.println("UTF8 encoding preserves the complete payload: " + correct);
		Sys.println("retained UTF8 storage stays close to the output length: " + bounded);
		Sys.println("ASCII control storage stays close to the output length: " + asciiExact);
	}
}

Build with the Haxe nightly using haxe -main Main -cpp bin -D HXCPP_M64, then run bin/Main (bin/Main.exe on Windows). Pass any argument to print the lengths and capacities. This exercises API 500; older compiler configurations skip the new encoding API.