Repository navigation
Raise InternalError for non-CPU constant storage in the emitter - #24202
Conversation
_tensor_spec_to_evalue reads a constant's bytes by casting spec.storage.data_ptr() to a host pointer. For CUDA storage that is a device address, so to_executorch() segfaults with no Python exception (pytorch#21545). Check the storage device first and raise an InternalError that names the device and asks the user to move the constants to CPU.
Constants consumed by a delegate do not reach this check, so telling every user to move the model to CPU is wrong for flows that delegate to a device backend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24202
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated FailureAs of commit 5e310ec with merge base ec56cfa ( NEW FAILURE - The following job has failed:
CANCELLED JOB - The following job was cancelled. Please retry:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The red That job runs The same job is failing on |
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gasoonjia
left a comment
There was a problem hiding this comment.
thx lgtm i will stamp it after ci passes
Summary
_tensor_spec_to_evaluereads a constant's bytes by castingspec.storage.data_ptr()to a host pointer. When the storage is on CUDA, that pointer is a device address, soto_executorch()segfaults (exit 139) with no Python exception. This PR checks the storage device first and raises anInternalError, as @shoumikhin suggested in #21545, that names the device and says how to fix the export. As discussed in the issue, lowering a CUDA module onto the portable path is not supported; only the way it fails changes. The check applies only to non-empty constants that the emitter serializes itself and whose storage is not on CPU, which are exactly the cases that reach the host read of a device pointer.Fixes #21545
Test plan
New test
test_emit_non_cpu_constant_raisesinexir/emit/test/test_emit.py. It usesmetastorage, which is not host-readable either and needs no accelerator. Without the guard, that case already fails with a wrapped "NULL pointer access"InternalError, so the test matches on the device name in the message.Built and installed from this branch after merging main at d6abe23 (
pip install . --no-build-isolationon Ubuntu 24.04, Python 3.12, torch 2.14.0+cpu):70 passed, 2 skipped, as before the merge. With
_emitter.pyreplaced by main's, the new test fails on the "NULL pointer access" message.lintrunner --skip MYPYreports no issues on the two files.I also ran the reproducer from #21545 on CUDA (RTX 3060 under WSL), with the same package and torch replaced by the cu130 build of the same version (2.14.0+cu130). With main's
_emitter.pyit segfaults in_tensor_spec_to_evalue(exit 139). With this branch it raisesInternalError: ... Constant tensor storage is on device 'cuda:0', ...(exit 1).I used Claude Code to write this PR.
cc @JacobSzwejbka @angelayi @nil-is-all @Gasoonjia @digantdesai