Repository navigation
Propagate extension metadata errors when serializing scalars - #11397
Conversation
Literal and PreparedSet serialized their scalars through the infallible `From<&Scalar> for pb::Scalar`, which calls `vortex_expect` on the fallible dtype serializer. An extension dtype whose metadata serializer returns an error therefore panicked through the scalar-function serializers' `VortexResult` boundary. Add a fallible `Scalar::to_proto`, have the `From` impl delegate to it, and use it from both serializers so the error is returned instead. Fixes vortex-data#11392 Signed-off-by: Vincent <1729065730@qq.com>
Merging this PR will degrade performance by 3.18%2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
Summary
Literal::serializeandPreparedSet's serializer converted their scalar through the infallibleFrom<&Scalar> for pb::Scalar, which callsvortex_expecton the fallible dtype serializer. An extension dtype whose metadata serializer returns an error (for example a plugin extension) therefore panicked through the scalar-function serializers'VortexResultboundary instead of returning the error.Changes
Scalar::to_proto(&self) -> VortexResult<pb::Scalar>next to the existingfrom_proto. TheFrom<&Scalar>impl now delegates to it, so existing callers keep their behaviour.Scalar::to_protoinLiteral::serializeandPreparedSet::serialize, so the extension metadata error is propagated.vortex-array/src/scalar/proto.rswith a test extension type whoseserialize_metadataalways fails, checking thatScalar::to_proto,Literal::serialize, andPreparedSet::serializeall return the error rather than panicking.Not covered here: a variant value nested inside a
ScalarValuestill goes through the infallible conversion (ScalarValuetopb::ScalarValue), since changing that would ripple into theto_proto/to_proto_byteshelpers; happy to follow up if you want that path made fallible too.Tested with
cargo test -p vortex-array --lib(5887 passed) andcargo clippy -p vortex-array --all-targets(clean).AI assistance: this change was prepared with help from an AI coding assistant (agentic), and reviewed and submitted by me.