Skip to content

Keep UTF-16 string construction inside the supplied native view #412

Open

Description

@dimensionscape

String::create(View<char16_t>) can include a code unit beyond the supplied view when its last visible unit is a high surrogate.

Reproduced on main at 5f7cfef66ed4d9543c6c1a2763bdc2148433435c, using Haxe nightly 5.0.0-preview.1+fe63b35, runtime API 500, Windows x64 with MSVC 19.44.35213 and Linux x86-64 under WSL with GCC 13.3. Both builds use HXCPP_M64, default GC settings and smart strings.

The native helper has an allocated array containing D83D DE00 0000, but supplies a view with length one. The constructor returns the complete two-unit surrogate pair. It should either preserve only the unit exposed by the view or reject the incomplete sequence. It must not consume the adjacent unit to complete it.

The UTF-16 view constructor checks start < end, then calls the unbounded Char16Advance(start). That decoder can advance twice for a surrogate pair. The constructor calculates its copy length from the advanced pointer, so it subsequently copies two units from a one-unit view.

Use a bounded UTF-16 scan and keep the computed copy length within the view. Preserve the existing behavior for complete pairs and NUL-terminated native strings. This is a runtime-only fix.

A conventional regression is prepared in test/regression/StringUtf16ViewBounds/{Main.hx,build.hxml,stdout.txt}. The underlying test array is fully allocated; the test checks the smaller view's extent. Expected output has all three checks true. Actual output on both platforms:

UTF16 construction stays inside the supplied view: false
a complete surrogate pair is preserved: true
a NUL terminator ends native string construction: true
#if (haxe_ver >= 5 && !haxe_5_preview_1)
@:cppFileCode('
#include <cpp/marshal/View.hpp>
static bool boundedUtf16Prefix() {
	char16_t data[3] = {0xD83D, 0xDE00, 0};
	try {
		String value = String::create(cpp::marshal::View<char16_t>(data, 1));
		return value.length <= 1;
	} catch (...) { return true; }
}
static bool completeUtf16Pair() {
	char16_t data[3] = {0xD83D, 0xDE00, 0};
	String value = String::create(cpp::marshal::View<char16_t>(data, 2));
	return value.length == 2 && value.cca(0) == 0xD83D && value.cca(1) == 0xDE00;
}
static bool terminatedUtf16View() {
	char16_t data[4] = {0x0100, 0, 65, 0};
	String value = String::create(cpp::marshal::View<char16_t>(data, 3));
	return value.length == 1 && value.cca(0) == 0x0100;
}
')
#end
class Main {
	static function main() {
		var bounded = true;
		var pair = true;
		var terminated = true;
		#if (haxe_ver >= 5 && !haxe_5_preview_1)
		bounded = untyped __cpp__("boundedUtf16Prefix()");
		pair = untyped __cpp__("completeUtf16Pair()");
		terminated = untyped __cpp__("terminatedUtf16View()");
		#end
		Sys.println("UTF16 construction stays inside the supplied view: " + bounded);
		Sys.println("a complete surrogate pair is preserved: " + pair);
		Sys.println("a NUL terminator ends native string construction: " + terminated);
	}
}

Build with the Haxe nightly using haxe -main Main -cpp bin -D HXCPP_M64, then run bin/Main (bin/Main.exe on Windows). API 500 is required to exercise this constructor; older compiler configurations skip it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions