[Jacqueline122]

Description

Fixes #3413.

The OBJECTIVE CSV loader previously converted human labels using bool(int(score)) without validating their original values. This silently changed labels such as 2 and -1 to True, and 0.9 to False.

This change validates every human-score column before conversion and raises a ValueError identifying the column when a label is outside 0/1.

Valid integer, float, and boolean representations of 0/1 remain supported. HARM dataset scoring is unchanged.

OpenAI Codex assisted with reproduction, implementation, regression tests, and local validation.

Tests and Documentation

Added 14 regression cases covering:

• Non-binary integers and fractional labels.
• Positive and negative infinity.
• Valid integer, float, and boolean labels.
• Invalid labels in a second human-rater column.

Before the fix, the new tests report 8 failures and 6 passes. After the fix, all 107 tests in test_human_labeled_dataset.py pass, including the 93 existing tests.

The targeted test module was run with --noconftest, bypassing database-wide fixtures. Ruff lint, formatting, and git diff checks pass. The full repository suite was not run.

Updated the CSV validation method's docstring. JupyText was not run; no notebooks were changed.

[Jacqueline122]

@microsoft-github-policy-service agree