Skip to content

internal/loopir, internal/rewrite: vectorize stores at a loop-invariant offset - #19

Open
thesyncim wants to merge 1 commit into
inline-overlap-checkfrom
offset-store-on-windows
Open

thesyncim wants to merge 1 commit into
inline-overlap-checkfrom
offset-store-on-windows

Conversation

@thesyncim

@thesyncim thesyncim commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Loads can already be at i+e, e+i or i-e for a loop-invariant e. This allows the same for stores, which covers row-wise code like:

for j := 0; j < n; j++ { b[i*ldb+j] = a[i*lda+j] }        // gonum dlacpy
for j := 0; j < k; j++ { c[i*ldc+j] -= work[i*ldwork+j] } // gonum dlarfb
for x := range w { line[off+x] = v }

Every store to a given slice has to use the same offset, so each iteration still writes its own element. Two different offsets are rejected with a new reason. For loads of a stored slice, the distance is now measured from the store offset instead of from i. The same offset means the same element. A constant positive distance in a forward loop is loaded before the chunk's stores, as today. A negative distance, a non-constant one, or a loop that counts down is rejected. The store gets its own window, cut at the offset, so the vector loop never writes an element the scalar loop doesn't; the length check and the overlap window include the offset. The if-select and copy-builtin paths take the offset too, and a constant offset that puts the first stored index below zero is rejected, as mauri870#2 does for loads.

With loopvec -json -methods (I read every added loop), std goes from 28 to 29 vectorized loops (an image/jpeg fill), gonum from 161 to 177 (dlacpy, dlaset, dlascl and dlarfb rows, plus some testlapack helpers), and goav1 from 81 to 86. TSVC s423 and s424 now vectorize and match their goldens. The generated code for loops that were already vectorized doesn't change.

offset_store.txt runs 9 shapes over 14 sizes, plus short slices that must panic and aliased views that must fall back, and both builds have to print the same output. offset_store_no_rewrite.txt pins the rejected cases.

…nt offset

A loop that stores at an offset from the index, such as a row of a 2-D
slice (b[i*ldb+j] = a[i*lda+j], c[row+j] += alpha*x[j]) or a fill that
starts part way into a slice (line[off+x] = v), was rejected: loads could
be at an offset, but every store had to be at exactly the loop index.

A store may now be at the same offsets a load may: i+e, e+i and i-e for a
loop-invariant integer e. Every store to one slice must be at the same
offset, so each iteration still writes its own element; two offsets are
rejected with their own reason. A load of the stored slice at that offset
is the same element, ordered after the stores before it as at the index.
The dependence test for a load at another offset measures the distance
from the store offset instead of from the index, so a constant positive
distance in a forward loop is still loaded before any store of the chunk,
and a negative or variable distance is rejected as before. An if that
selects between stores, and the copy builtin, take the offset too.

The emitted store is capped at the limit plus the offset, so the vector
loop never writes an element the scalar loop does not, and the length
check and the overlap window cover the elements stored. A constant offset
that puts every stored index below zero is left alone, as for a load.

The vectorized set gains one loop in std (an image/jpeg fill), sixteen in
gonum (dlacpy, dlaset, dlascl and dlarfb rows, and testlapack helpers)
and five in github.com/thesyncim/goav1; TSVC s423 and s424 now vectorize
and match their goldens.
@thesyncim
thesyncim removed this pull request from stack #20 November 4, 2026 15:50
@thesyncim
thesyncim force-pushed the offset-store-on-windows branch from 9dd4eb0 to 796e0a8 Compare November 4, 2026 15:50
@thesyncim
thesyncim added this pull request to stack #21 November 4, 2026 15:50
mauri870 pushed a commit that referenced this pull request Nov 4, 2026
The runners have moved to ubuntu-26.04, so this is the qemu part of #1
you said you'd take when that happened. `qemu-user-static` has no
installation candidate there, so every PR now fails at that step and the
arm64 runs, the golden check, fmt and lint get skipped. You can see it
on #7, #9 and #19, where Build, Test and the amd64 tsvc run all pass
first. master's last run predates the move, so the next push will hit it
too.

`qemu-user` has the static emulators on 26.04, just without the
`-static` names, so this installs that and links `qemu-aarch64-static`
when it's missing, since the Makefile runs that name. It's been passing
on my fork's ubuntu-26.04 runners, and this PR's own CI runs it too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant