Skip to content

FIX Reject negative and non-integer counts in SeedDataset.get_values and get_random_values - #3416

Open
Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
microsoft:mainfrom
ShivamSharma43:fix/seed-dataset-negative-counts
Open

Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
microsoft:mainfrom
ShivamSharma43:fix/seed-dataset-negative-counts

Conversation

@ShivamSharma43

Copy link
Copy Markdown
Contributor

Description

Fixes #3415.

SeedDataset.get_values(first=..., last=...) and get_random_values(number=...) annotated their counts as PositiveInt, but that type was only imported under TYPE_CHECKING, so it was never enforced. A negative count was used directly as a slice bound: first=-2 returned every prompt except the last two, last=-2 every prompt except the first two, and first=2, last=-1 returned duplicates (p0, p1, p1, p2, ...). get_random_values(number=-1) failed inside random.sample with an unrelated message, and True was accepted as 1.

Changes:

  • A small _validate_count helper raises ValueError("<name> must be a non-negative integer, got ...") for bools, non-integers and negative values. get_values checks first and last when they are given, and get_random_values checks number.
  • 0 stays accepted, because it is already used as "none" (the existing test_seed_dataset_get_values calls get_values(last=0) and get_values(first=2, last=0)). The annotations now say NonNegativeInt to match.
  • The docstrings list the new ValueError.

Valid counts behave exactly as before, including the overlap case (first + last >= len) and counts larger than the dataset in get_random_values.

Tests and Documentation

  • New tests in tests/unit/models/test_seed_dataset.py: get_values rejects first=-2, last=-2, first=2, last=-1, first=True and last=1.5; get_random_values rejects -1, True, 1.5 and "2"; zero counts still return the expected results.
  • On main all 9 new rejection cases fail (the calls return sliced or duplicated prompts, or fail with the random.sample error); the zero-count tests pass on both. With this change they all pass.
  • pytest tests/unit/models tests/unit/datasets: 7353 passed.
  • ruff check and ruff format pass on the changed files; git diff --check is clean.
  • No documentation changes beyond the docstrings.

🤖 Generated with Claude Code

…and get_random_values

The PositiveInt annotations were only imported under TYPE_CHECKING, so they
were never enforced. A negative first or last sliced from the other end of the
prompt list (first=-2 dropped the last two prompts) and mixing a valid and a
negative count returned duplicates. get_random_values(number=-1) failed inside
random.sample with an unrelated message, and True was accepted as 1.

Both methods now raise ValueError for counts that are not non-negative
integers. Zero stays accepted (the existing tests use last=0), so the
annotations now say NonNegativeInt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 05:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: SeedDataset.get_values and get_random_values accept negative counts and return the wrong or duplicated prompts

2 participants