Repository navigation
Description
The ordinary CFFI wrappers generated by DEFINE_PRIME corrupt Unicode string arguments and results, even when direct Prime calls to the same exported functions work correctly.
Confirmed on fe67557, Windows x64, Haxe 4.3.7, MSVC 19.44, smart strings, default collector. This uses a real separately built native DLL loaded through the normal APIs, not a copied adapter or a replacement runtime implementation.
Native fixture:
#define IMPLEMENT_API
#include <hx/CFFIPrime.h>
HxString auditEcho(HxString value) { return value; }
DEFINE_PRIME1(auditEcho);
HxString auditMakeText() {
return alloc_hxs_utf8("\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e", 9);
}
DEFINE_PRIME0(auditMakeText);Compare the same export through:
var fallback = cpp.Lib.load("auditprime", "auditEcho", 1);
var direct = cpp.Prime.load("auditprime", "auditEcho", "ss", false);| Input | Direct Prime | Ordinary CFFI wrapper |
|---|---|---|
| plain | preserved | preserved |
| cafe with U+00E9 as its last character | preserved | last character becomes U+FFFD |
| Japanese U+65E5 U+672C U+8A9E | preserved | only U+65E5 remains |
| U+1F600 | preserved | U+FFFD |
| empty / null | preserved | preserved |
The no-argument native Unicode-return function also passes through direct Prime and fails through the ordinary wrapper. That independently covers result conversion, rather than only echoing an already damaged argument.
Two related assumptions in include/hx/CFFIPrime.h cause this:
- AutoValue::operator HxString combines val_string's UTF-8 pointer with val_strlen's Haxe string length, which is a UTF-16 code-unit count for wide strings. The resulting view has the wrong byte length.
- ToValue(HxString) passes c_str() and size() straight to alloc_string_len as UTF-8, although an HxString returned by alloc_hxs_utf8 on a smart-string host may have UTF-16 backing. The HxString ABI is not an unconditional UTF-8 byte view.
Make these adapters preserve both encoding and length in each direction. Do not change val_strlen globally without considering existing native consumers and val_wstring. Cover supplementary characters and embedded NULs with explicit lengths, preserve null/empty distinctions, and keep the direct Prime path working. This belongs in libhxcpp's CFFI headers/runtime bridge and does not require Haxe compiler or standard-library changes.
Added a local conventional regression before implementation work: test/regression/CffiPrimeUnicodeFallback, with Main.hx/build.hxml/stdout.txt, a tiny native fixture and a build macro that honors the requested native architecture. It builds and loads the DLL, compares both call paths, and reproduces all the failures above. No runtime implementation changes are included.
Activity
Keep Unicode strings in the ordinary wrapper of a Prime function
dimensionscape commentedon Oct 25, 2026
Fixed in
1e6417d697476a07e25f10ff13b89e0255098b52. An argument to the ordinary wrapper is now made again from its UTF-8 when it is not ASCII, as a direct Prime call has it, and a result that holds UTF-16 is converted to UTF-8 before alloc_string_len. Both use only CFFI functions an ndll already calls, so an ndll built with the new headers still loads in an older host. A string held as bytes still passes through as it is, embedded NULs included; in one that is not ASCII, a NUL ends it on this path, as it does val_string. Test: CffiPrimeUnicodeFallback, which builds its own ndll; run-generated.sh now builds it again where the tests run, and it is skipped on Emscripten.