Repository navigation
ci: add focused PR benchmark labels - #9018
Conversation
89eebfa to
77a7e5d
Compare
|
note that we probably dont want all of these labels, but I feel like a bit more than what we have on develop now would be nice? |
|
actually Im going to make them all disjoint, lets see how that works |
77a7e5d to
9cbc6da
Compare
2d3d02c to
c5173aa
Compare
Merging this PR will improve performance by 47.71%
Performance Changes
Tip Curious why this is faster? Comment Comparing Footnotes
|
|
Let's use bench-random-access, ra isn't straight obvious |
c5173aa to
0b4eb99
Compare
0b4eb99 to
dac3fb7
Compare
| # v4 (Postgres) ingest -- REQUIRED (see develop-bench.yml rationale). Empty records: | ||
| # post-ingest.py --postgres upserts the commit row only. Gated on the | ||
| # ingest-role ARN var (the assume-role input that MUST exist for OIDC to | ||
| # succeed). |
There was a problem hiding this comment.
Probably need a action/bench-gpu-compress label for GPU compress?
There was a problem hiding this comment.
yeah I am going to run this once this merges:
gh label edit "action/benchmark-gpu-compress" --name "action/bench-gpu-compress" \
--description "Run only the GPU compression benchmark on this PR"
gh label delete action/benchmark --yes
gh label delete action/benchmark-sql --yes
gh label delete action/benchmark-sql-full --yes
| "pr-compact": COMPACT, | ||
| "pr-full": FULL_PR, | ||
| "develop": FULL_LOCAL, | ||
| }, |
There was a problem hiding this comment.
is there any label reaching pr full?
There was a problem hiding this comment.
no since I think we should just keep everything disjoint, if someone wants to run more benchmarks I think they should just add all 4 (or 5 for gpu) labels.
4bbe37b to
fb5f8aa
Compare
| cat comment.md >> "$GITHUB_STEP_SUMMARY" | ||
| - name: Comment PR | ||
| if: github.event.pull_request.head.repo.fork == false | ||
| if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false |
There was a problem hiding this comment.
i think this is a qol change
fb5f8aa to
e332145
Compare
1247c00 to
b1b5ca5
Compare
joseph-isaacs
left a comment
There was a problem hiding this comment.
can we just move some stuff around?
|
ok codex is finding other issues that are tangentially related to this (not caused by this PR) but I think we might as well fix them now |
|
@AdamGS there are a few more changes in here now that you might want to take a look at, happy to try and split out the changes (though that will be kind of painful) |
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
5bd1630 to
3d127d1
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
3d127d1 to
14bcea0
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
| @@ -0,0 +1,20 @@ | |||
| # Runs the string encoding benchmark for a pull request. | |||
There was a problem hiding this comment.
why is this one different from random-access and compression?
There was a problem hiding this comment.
It feels strange that we would put random access, compression, and string encoding all in the same bucket? I also mainly wanted to split out compression because its by far the slowest benchmark we have
AdamGS
left a comment
There was a problem hiding this comment.
just one question re the new strings benchmark.
overall LGTM, once this merges, can you send a message in the slack channel so its more visible?
Benchmarks: String Encoding 📖vortex / vortex-file-compressed / ms (1.010x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
|
Adds focused PR benchmark triggers:
action/bench-random-accessaction/bench-compressaction/bench-string(benchmarks: add string compression benchmark #9060)action/bench-sql(excludes Vortex Compact)action/bench-sql-compact(Vortex Compact plus Parquet controls)action/bench-all(random access, compression, string encoding, and SQL, while GPU remains separate)Each trigger dispatches a role-specific reusable workflow. PR benchmarks check out
github.event.pull_request.head.sha. SQL coverage stays in the Python catalog and is resolved into a matrix at runtime. The all-CPU path uses a singlepr-allpreset (the exact union ofprandpr-compact), so shared SQL jobs run once instead of twice.Benchmark reporting now:
pr-compactto distinguish code changes from runner drift.TODO: After merge, delete the legacy
action/benchmark*labels oncedevelopuses the new dispatcher.