Repository navigation
fix(bridge): honor 4D boolean mask polarity in causal loss - #1808
Conversation
|
|
||
| # HF-style 4D boolean attention masks keep positions marked True. Override | ||
| # for architectures whose forward path uses True to block attention. | ||
| bool_4d_mask_is_keep: bool = True |
There was a problem hiding this comment.
MPT's forward treats a boolean 4D mask as True = masked (see the MPT attention bridge), so this default flips its loss. Base scored MPT correctly, and head now gives a different loss than the equivalent 2D mask. Could MPT's adapter override this the way the native one does?
There was a problem hiding this comment.
Good catch. MPT uses True=masked for 4D boolean attention masks. I added bool_4d_mask_is_keep=False to MPTArchitectureAdapter in e5a31ab. The new MPT regression verifies that a blocked-polarity 4D mask reproduces both the full logits and loss of the equivalent 2D mask.
|
|
||
| @pytest.mark.parametrize("padding", [False, True]) | ||
| @pytest.mark.parametrize("return_type", ["loss", "both"]) | ||
| def test_gpt2_bool_4d_keep_mask_loss_matches_2d(padding: bool, return_type: str) -> None: |
There was a problem hiding this comment.
The new tests cover a single keep-polarity architecture, which is how the MPT issue slipped through. Could you parametrize this over a few tiny HF architectures and check that the 4D loss matches the 2D loss under whichever polarity reproduces the 2D logits?
There was a problem hiding this comment.
Done in e5a31ab. The regression is now parameterized over tiny, locally initialized GPT-2, Llama, and MPT models. Each case uses its independently specified forward-mask polarity and checks full 4D-vs-2D logit equality before comparing losses; padding, loss/both, and explicit labels are covered. The focused related suite passed (202 tests), and mypy is clean.
|
Great work! Approved |
Description
Fixes #1807.
Bridge causal loss previously treated every 4D boolean attention mask as
True=blocked, even when the model's forward path usesTrue=visible. Equivalent 2D and 4D masks could therefore yield identical logits but different losses, includingNaNwithout padding. The loss mask now follows an architecture-adapter polarity property for implicit and explicit-label causal loss. TL-native and MPT explicitly retainTrue=blocked; GPT-2 and Llama use the defaultTrue=visibleconvention.After review, I added the MPT override and parameterized the tiny HF regression over GPT-2, Llama, and MPT. For each architecture, the 4D mask using its actual forward convention reproduces the full logits and loss from the equivalent 2D mask. The cases cover padded and unpadded inputs,
return_type="loss"/"both", and explicit labels withloss_per_token=True. Existing native boolean and additive 4D-mask tests remain covered.Validation
202 passedacross mask loss, native Bridge, adapter capability, MPT adapter, and tiny MPT integration tests.uv run --no-cache --no-sync mypy .: no issues in 383 source files.git diff --checkpassed on the changed files. The original commit also passed tracked-file repository format checks.Type of change
Checklist