Skip to content

Propagate extension metadata errors when serializing scalars - #11397

Merged
robert3005 merged 1 commit into
vortex-data:developfrom
ump45nose:fix/11392-scalar-serialize-errors
Nov 12, 2026
Merged

robert3005 merged 1 commit into
vortex-data:developfrom
ump45nose:fix/11392-scalar-serialize-errors

Conversation

@ump45nose

Copy link
Copy Markdown
Contributor

Summary

Literal::serialize and PreparedSet's serializer converted their scalar through the infallible From<&Scalar> for pb::Scalar, which calls vortex_expect on the fallible dtype serializer. An extension dtype whose metadata serializer returns an error (for example a plugin extension) therefore panicked through the scalar-function serializers' VortexResult boundary instead of returning the error.

Changes

  • Add a fallible Scalar::to_proto(&self) -> VortexResult<pb::Scalar> next to the existing from_proto. The From<&Scalar> impl now delegates to it, so existing callers keep their behaviour.
  • Use Scalar::to_proto in Literal::serialize and PreparedSet::serialize, so the extension metadata error is propagated.
  • Add regression tests in vortex-array/src/scalar/proto.rs with a test extension type whose serialize_metadata always fails, checking that Scalar::to_proto, Literal::serialize, and PreparedSet::serialize all return the error rather than panicking.

Not covered here: a variant value nested inside a ScalarValue still goes through the infallible conversion (ScalarValue to pb::ScalarValue), since changing that would ripple into the to_proto/to_proto_bytes helpers; happy to follow up if you want that path made fallible too.

Tested with cargo test -p vortex-array --lib (5887 passed) and cargo clippy -p vortex-array --all-targets (clean).

AI assistance: this change was prepared with help from an AI coding assistant (agentic), and reviewed and submitted by me.

Literal and PreparedSet serialized their scalars through the infallible
`From<&Scalar> for pb::Scalar`, which calls `vortex_expect` on the
fallible dtype serializer. An extension dtype whose metadata serializer
returns an error therefore panicked through the scalar-function
serializers' `VortexResult` boundary.

Add a fallible `Scalar::to_proto`, have the `From` impl delegate to it,
and use it from both serializers so the error is returned instead.

Fixes vortex-data#11392

Signed-off-by: Vincent <1729065730@qq.com>
@robert3005 robert3005 added the changelog/fix A bug fix label Nov 12, 2026
@codspeed

codspeed Bot commented Nov 12, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 3.18%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 2 improved benchmarks
❌ 2 regressed benchmarks
✅ 1917 untouched benchmarks
⏩ 50 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation random_i8[0.01] 100.4 µs 125.4 µs -19.94%
❌ CPU Simulation filter_random_by_random[10000] 6.8 µs 7.6 µs -10.35%
⚡ Walltime add_constant_shapes_neon[(32768, PerRowConstant)] 15.7 µs 14.1 µs +10.73%
⚡ CPU Simulation encode_primitives[f32, (2000, 512)] 272.9 µs 246.8 µs +10.57%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ump45nose:fix/11392-scalar-serialize-errors (5c5d6ea) with develop (bc3d422)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@robert3005
robert3005 merged commit 824c922 into vortex-data:develop Nov 12, 2026
97 of 99 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants