Skip to content

Respect destination dimensions in VectorOfArray broadcasting - #685

Draft
ChrisRackauckas wants to merge 2 commits into
masterfrom
fix/voa-broadcast-dimensions-680
Draft

ChrisRackauckas wants to merge 2 commits into
masterfrom
fix/voa-broadcast-dimensions-680

Conversation

@ChrisRackauckas

@ChrisRackauckas ChrisRackauckas commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

🚧 UNREVIEWED — awaiting maintainer review. Filed by an AI agent; no human has reviewed this.
Harness: Codex CLI 0.162.1 · Model: gpt-6.1-sol
Session: 01a14bd6-bd53-7dd2-94c0-6f6613286f91

In-place VectorOfArray broadcasting slices array arguments along the destination's outer dimension. Lower-dimensional arguments broadcast across inner arrays, and singleton outer dimensions reuse their sole slice. Inner broadcasts use the destination axes for expansion, including immutable StaticArrays, StructArrays storage, and serial/Polyester FastBroadcast.

Fixes #680

Keep this draft out of merge consideration until a maintainer has reviewed it. No public signatures or dependencies change.

Reproduction and regression evidence

Reproduced on default-branch commit 214fb8d556b24581e65015bdeb340102aa33a76d (package version 4.5.5) and the latest registered release, 4.5.5, with Julia 1.12.7. Both produce [11.0 24.0; 12.0 25.0; 13.0 26.0] for the issue's vector-addition example instead of dense-array broadcasting's [11.0 14.0; 22.0 25.0; 33.0 36.0].

The new regression file compares against Base dense-array broadcasting for mutable vectors, SVectors, MVectors, StructArray storage, matrix inner arrays, zero-dimensional operands, singleton dimensions, and full-dimensional operands. It exercises serial and threaded FastBroadcast on SVector storage.

With the implementation fix stashed, running the regression file in a local environment that develops the checkout:

JULIA_NUM_THREADS=4 JULIA_NUM_PRECOMPILE_TASKS=4 timeout 3600 julia --project=tmp/main -e 'using RecursiveArrayTools, StaticArrays; v=VectorOfArray([SVector(1.,2.), SVector(3.,4.)]); f(v)=(v .= v .+ v); f(v); @show @allocated f(v); include("RecursiveArrayTools.jl/test/Core/voa_broadcast_dimensions.jl")'

Relevant output (stack traces omitted):

#= none:1 =# @allocated(f(v)) = 0
VectorOfArray broadcast dimensions: Test Failed
  Expression: Array(v) == expected
   Evaluated: [11.0 24.0; 12.0 25.0; 13.0 26.0] == [11.0 14.0; 22.0 25.0; 33.0 36.0]
DimensionMismatch: destination axes (Base.OneTo(3),) are not compatible with source axes ()
Test Summary:                      | Fail  Error  Total  Time
VectorOfArray broadcast dimensions |    1      1      2  2.0s

After restoring the fix, the same regression file:

JULIA_NUM_THREADS=4 JULIA_NUM_PRECOMPILE_TASKS=4 timeout 3600 julia --project=tmp/main -e 'include("RecursiveArrayTools.jl/test/Core/voa_broadcast_dimensions.jl")'
Test Summary:                      | Pass  Total  Time
VectorOfArray broadcast dimensions |  130    130  8.8s

Validation

All Julia commands use JULIA_NUM_THREADS=4, JULIA_NUM_PRECOMPILE_TASKS=4, and timeout 3600.

  • GROUP=Core julia --project=RecursiveArrayTools.jl -e 'using Pkg; Pkg.test()': Testing RecursiveArrayTools tests passed; includes all 130 new checks and the existing zero-allocation assertions.
  • GROUP=QA julia --project=RecursiveArrayTools.jl -e 'using Pkg; Pkg.test()': 20/20 passed.
  • julia --project=@runic -m Runic --inplace on all six changed Julia files: passed.
  • typos tmp/diff.patch on the complete diff, including the new test file, and git diff --check: passed.
  • Docs build omitted: no docs, docstrings, or public API changes.
  • Not run: Downstream, GPU, AD, NoPre, other Julia versions, and performance benchmarks. Independent review is pending.

Range-only FastBroadcast expressions can simplify to a range assignment before reaching these broadcast methods. A separate @.. v = x + 1 check with an SVector-backed destination and x = 10.:10.:30. fails on the unmodified default branch with setindex!(::SVector{3, Float64}, value, ::Int) is not defined. That assignment path is outside this fix; the tests cover lower-dimensional ranges through Base broadcasting.

Risk assessment

  • Risk: medium-high (MH1: changes incorrect results and errors on the core VectorOfArray broadcasting lane).
  • Blast radius: VectorOfArray and DiffEqArray broadcasting, including static inner arrays and the StructArrays/FastBroadcast extensions sharing the unpacking helper; no public signature changes.
  • Evidence: reproduced on default branch and registered release; stashed-fix regression fails and restored-fix regression passes 130 checks; full Core and QA pass, including existing allocation checks. No downstream or GPU/AD validation.
  • Independent review: pending (a different harness reviews it).
  • Merge: needs human review — MH1 affects a widely used core entry point.

🤖 Posted by an AI agent — harness: Codex CLI 0.162.1 · model: gpt-6.1-sol
Conversation: Codex exec session 01a14bd6-bd53-7dd2-94c0-6f6613286f91 (no session URL)

🤖 Automated review section from an AI agent running as Chris Rackauckas's account — not written or reviewed by Chris.

Fix by Codex CLI 0.164.0 gpt-6.1-sol (head 5c18e1d)

Partial fix: the mutable StructArray fixture uses SizedVector{3} elements, which have an accessible field schema on Julia 1.10. Ordinary Vector storage, the other static-array fixtures, and all 130 assertions remain. The scalar FastBroadcast expression and all assertions are unchanged.

Failing-before / passing-after evidence

Before this commit, the full Core group on Julia 1.10.12 reproduced the reported CI failure:

GROUP=Core timeout 3600 julia +1.10 --project -e 'using Pkg; Pkg.test()'
ArgumentError: only eltypes with fields are supported
Test Summary:                        | Pass  Error  Total
VecOfArr Broadcast Dimensions        |   15      1     16

With the fixture correction, the same command completed successfully (1,415 passing assertions across Core; two existing broken assertions):

Test Summary:                 | Pass  Total
VecOfArr Broadcast Dimensions |  130    130
Test Summary:     | Pass  Total
Measurement Tests |    2      2
     Testing RecursiveArrayTools tests passed

Validation

Julia runs used four threads and timeout 3600. The final test file was compared byte-for-byte with the file from the passing full Core run.

GROUP=QA timeout 3600 julia +1 --project -e 'using Pkg; Pkg.test()'

Julia 1.13.3 output:

Test Summary: | Pass  Total   Time
QA            |   20     20  17.8s
     Testing RecursiveArrayTools tests passed
julia +1.12 --project=@runic -m Runic --inplace test/Core/voa_broadcast_dimensions.jl
git diff HEAD^ HEAD > committed.diff
typos committed.diff
git show --format= --check HEAD

All exited zero with no output. Docs, GPU, AD, downstream suites, other Julia versions, and benchmarks were not run for this fixture-only correction.

Remaining pre-existing failures

Downgraded Core remains red. The same downgrade action revision as the failed CI run resolved StructArrays 0.7.0, StaticArrays 1.6.0, FastBroadcast 1.3.0, and Polyester 0.7.16 in an isolated checkout. With those dependencies:

GROUP=Core timeout 3600 julia +1.10 --project -e 'using Pkg; Pkg.test(; allow_reresolve=false)'
MethodError: no method matching fast_materialize_threaded!(::VectorOfArray{Float64, 2, Vector{SVector{3, Float64}}}, ::Float64)
Test Summary:                        | Pass  Error  Total
VecOfArr Broadcast Dimensions        |   32      1     33

The scalar assignment also fails on the clean, unmodified PR parent 214fb8d556b24581e65015bdeb340102aa33a76d. A standalone reproducer importing only FastBroadcast and Polyester fails with a dense Vector on FastBroadcast 1.3.0 and passes unchanged on 1.3.1:

using FastBroadcast, Polyester
v = zeros(3)
x = 2.0
@.. thread = true v = x + 1
@show v
# FastBroadcast 1.3.0:
ERROR: MethodError: no method matching fast_materialize_threaded!(::Vector{Float64}, ::Float64)
# FastBroadcast 1.3.1:
v = [3.0, 3.0, 3.0]

The released upstream fix is SciML/FastBroadcast.jl@0711312 . This commit leaves the failing scalar assertion and compatibility bounds intact, following the instruction to leave pre-existing behavior unchanged. Default-branch Core and Downgrade checks pass because that branch does not contain this new regression file; that does not establish scalar correctness at the FastBroadcast floor.

SciMLSensitivity Core4 is also pre-existing: the exact PR-parent default push and the PR both fail ensembles.jl:19, EnsembleThreads, with AssertionError: mi2 !== nothing in Enzyme parallelrules.jl:233, using the same Julia 1.13.2 and dependency versions. The latest default push has the same assertion. No local Core4 reproduction or causal bisect was completed; CI history establishes that the failure predates this PR.

Next counted review must come from a smart model in a fresh session, not this fix session (risk.md § Independent reviews).

Links


🤖 Posted by an AI agent — harness: Codex CLI 0.164.0 · model: gpt-6.1-sol
Conversation: local fixer job SciML__RecursiveArrayTools.jl__685 (no session URL)

codex and others added 2 commits October 17, 2026 22:01
Slice operands using the destination dimension and preserve lower-dimensional
arguments across inner arrays. Expand each inner broadcast to its destination
axes, including static arrays and FastBroadcast extension paths.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Codex <noreply@openai.com>
Agent-Harness: Codex CLI 0.162.1
Agent-Model: gpt-6.1-sol
Agent-Session: 01a14bd6-bd53-7dd2-94c0-6f6613286f91
Construct the mutable StructArray fixture from SizedVector elements so it has
an accessible field schema on Julia 1.10. Keep ordinary Vector storage and all
130 broadcast assertions, including the existing threaded scalar assignment.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Codex <noreply@openai.com>
Agent-Harness: Codex CLI 0.164.0
Agent-Model: gpt-6.1-sol
Agent-Session: local fixer job SciML__RecursiveArrayTools.jl__685 (no session URL)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

In-place broadcasting into a VectorOfArray mishandles arrays with fewer dimensions

2 participants