Repository navigation
Add TinyStories to string-bench - #11306
Conversation
Add the first 33,121 rows of the TinyStories train split as `tinystories/text`. This is the size of one file that the NeMo Curator text IO benchmark writes. Signed-off-by: Robert Kruszewski <github@robertk.io>
f10523e to
ebeda56
Compare
Merging this PR will not alter performance2 warnings
Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: String Encoding 📖Commits: PR vortex / vortex-file-compressed / ns (0.999x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
|
joseph-isaacs
left a comment
There was a problem hiding this comment.
shall we hook it up in ci?
Signed-off-by: Robert Kruszewski <github@robertk.io>
Add
tinystories/textto the string-bench input catalog: the first 33,121 rows of the TinyStories train split (about 28 MiB). This is the size of one file that the NeMo Curator text IO benchmark writes.