Skip to content

fix(bridge): honor 4D boolean mask polarity in causal loss - #1808

Merged
jlarson4 merged 2 commits into
TransformerLensOrg:devfrom
emerardd:fix/bridge-bool-mask-loss
Sep 25, 2026
Merged

jlarson4 merged 2 commits into
TransformerLensOrg:devfrom
emerardd:fix/bridge-bool-mask-loss

Conversation

@emerardd

@emerardd emerardd commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes #1807.

Bridge causal loss previously treated every 4D boolean attention mask as True=blocked, even when the model's forward path uses True=visible. Equivalent 2D and 4D masks could therefore yield identical logits but different losses, including NaN without padding. The loss mask now follows an architecture-adapter polarity property for implicit and explicit-label causal loss. TL-native and MPT explicitly retain True=blocked; GPT-2 and Llama use the default True=visible convention.

After review, I added the MPT override and parameterized the tiny HF regression over GPT-2, Llama, and MPT. For each architecture, the 4D mask using its actual forward convention reproduces the full logits and loss from the equivalent 2D mask. The cases cover padded and unpadded inputs, return_type="loss"/"both", and explicit labels with loss_per_token=True. Existing native boolean and additive 4D-mask tests remain covered.

Validation

  • Focused related suite: 202 passed across mask loss, native Bridge, adapter capability, MPT adapter, and tiny MPT integration tests.
  • uv run --no-cache --no-sync mypy .: no issues in 383 source files.
  • pycln, pinned isort, Black, and git diff --check passed on the changed files. The original commit also passed tracked-file repository format checks.
  • The complete unit-test suite was not run locally. CI will re-run after this update; the earlier PR head's checks were all successful before the review change.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes (focused tests passed; complete suite not run)
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4 jlarson4 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The diagnosis in #1807 is solid, and making polarity an adapter property is the right solution. One comment on MPT's convention + one on test coverage for the default.


# HF-style 4D boolean attention masks keep positions marked True. Override
# for architectures whose forward path uses True to block attention.
bool_4d_mask_is_keep: bool = True

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MPT's forward treats a boolean 4D mask as True = masked (see the MPT attention bridge), so this default flips its loss. Base scored MPT correctly, and head now gives a different loss than the equivalent 2D mask. Could MPT's adapter override this the way the native one does?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. MPT uses True=masked for 4D boolean attention masks. I added bool_4d_mask_is_keep=False to MPTArchitectureAdapter in e5a31ab. The new MPT regression verifies that a blocked-polarity 4D mask reproduces both the full logits and loss of the equivalent 2D mask.


@pytest.mark.parametrize("padding", [False, True])
@pytest.mark.parametrize("return_type", ["loss", "both"])
def test_gpt2_bool_4d_keep_mask_loss_matches_2d(padding: bool, return_type: str) -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests cover a single keep-polarity architecture, which is how the MPT issue slipped through. Could you parametrize this over a few tiny HF architectures and check that the 4D loss matches the 2D loss under whichever polarity reproduces the 2D logits?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e5a31ab. The regression is now parameterized over tiny, locally initialized GPT-2, Llama, and MPT models. Each case uses its independently specified forward-mask polarity and checks full 4D-vs-2D logit equality before comparing losses; padding, loss/both, and explicit labels are covered. The focused related suite passed (202 tests), and mypy is clean.

@jlarson4

Copy link
Copy Markdown
Collaborator

Great work! Approved

@jlarson4
jlarson4 merged commit 073c1dc into TransformerLensOrg:dev Sep 25, 2026
26 checks passed
@emerardd
emerardd deleted the fix/bridge-bool-mask-loss branch October 17, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants