Skip to content

Avoid untrusted definite-array preallocation in CBOR decoder - #37

Open
OskarEichler wants to merge 2 commits into
cabo:masterfrom
OskarEichler:codex/preallocation-cap
Open

OskarEichler wants to merge 2 commits into
cabo:masterfrom
OskarEichler:codex/preallocation-cap

Conversation

@OskarEichler

@OskarEichler OskarEichler commented Aug 29, 2026 •

Copy link
Copy Markdown

Problem

The definite-array decoder allocated Ruby array capacity from the CBOR-declared element count before reading any elements. A short sequence of nested array headers can therefore reserve substantial capacity even when the input ends immediately. The existing change in this PR corrects the cap calculation, but a per-array cap alone does not limit the cumulative reservation across nested arrays.

On Ruby 4.0.6, 32 nested three-byte headers (96 input bytes) reserved 16,778,240 bytes of Ruby Array capacity before EOFError. This is an allocation measurement, not a demonstrated resident-memory increase or process crash.

Fix

Create definite arrays empty and let rb_ary_push grow them as elements arrive. Retain the capped preallocation for raw strings. Decoded values, streaming behavior, and parser errors remain the same; valid large arrays may do additional proportional growth.

Verification

  • rbenv exec rake compile and rbenv exec rake spec: 171 examples, 0 failures on Ruby 4.0.6, arm64 macOS.
  • The 96-byte malformed input reserved 1,280 bytes of Array capacity after this change. A 160-byte variant using five-byte length headers also reserved 1,280 bytes.
  • A valid 70,000-element array decoded correctly in one shot and through incremental feeding.
  • The new MRI regression example checks the malformed nested-array allocation and preserves EOFError.

@OskarEichler OskarEichler changed the title Cap container preallocation instead of wrapping Avoid untrusted definite-array preallocation in CBOR decoder Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant