Skip to content

Add parse_json and format_json conversions - #11693

Open
connortsui20 wants to merge 1 commit into
developfrom
ct/json-text-conversions
Open

connortsui20 wants to merge 1 commit into
developfrom
ct/json-text-conversions

Conversation

@connortsui20

@connortsui20 connortsui20 commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Vortex has no way to turn Utf8 text into a Json value, and ext_storage returns the text of a Json value without checking it. A conversion that changes what a value means belongs to the type and not to the generic cast (#11599), so the JSON crate registers parse_json and format_json as named functions.

Both directions check that each value is one complete JSON document and keep its original text, so number notation, key order, and whitespace survive a round trip. Formatting checks too, because arrays don't check extension values when they are built yet (#11601), so a stored value that isn't valid JSON returns an error instead of text.

The options schema lives in a new vortex-json/proto/json.proto and not next to JsonToVariantOpts in vortex-array, because a schema belongs to the crate that owns the function.

Like other functions that can fail, both can fail on a RunEnd run that no row references, because RunEnd pushes a function into all of its values. #11690 fixes this for every function.

Prepared with AI assistance.

@connortsui20 connortsui20 added the changelog/feature A new feature label Nov 20, 2026
Comment thread vortex-json/src/text.rs

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be split into 2 separate scalar functions?

@connortsui20
connortsui20 force-pushed the ct/json-text-conversions branch 2 times, most recently from 4f56b1e to 489ae9f Compare November 20, 2026 18:47
@connortsui20
connortsui20 marked this pull request as ready for review November 20, 2026 18:47
@connortsui20
connortsui20 enabled auto-merge (squash) November 20, 2026 18:48
@codspeed

codspeed Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 1982 untouched benchmarks
⏩ 58 skipped benchmarks1


Comparing ct/json-text-conversions (a41646f) with develop (967bcb6)

Open in CodSpeed

Footnotes

  1. 58 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

A conversion that changes what a value means belongs to the type, not to the generic cast, so the JSON crate registers named functions that convert Utf8 to Json and back. Both validate each value as one JSON document and keep its original text.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/json-text-conversions branch from 262343a to a41646f Compare November 20, 2026 20:41
Comment thread vortex-json/src/proto.rs

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what the robots generated, is this the correct way to do things?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant