Skip to content

fix(bridge): set rmsnorm_uses_offset on the Qwen3.5, Qwen3Next and Nemotron adapters - #1869

Merged
jlarson4 merged 3 commits into
TransformerLensOrg:devfrom
willgosnold:fix/qwen3_5-rmsnorm-offset
Oct 8, 2026
Merged

jlarson4 merged 3 commits into
TransformerLensOrg:devfrom
willgosnold:fix/qwen3_5-rmsnorm-offset

Conversation

@willgosnold

@willgosnold willgosnold commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes #1868.

HF's Qwen3.5, Qwen3.5-MoE and Qwen3Next RMSNorms scale by (1 + weight), the same convention as Gemma. Their adapters never set cfg.rmsnorm_uses_offset, so NormalizationBridge scales by weight alone whenever it computes a norm's output itself. That happens when a forward hook edits hook_scale or hook_normalized, when a backward hook is attached to either, and in the LN-rule backward pass. Unhooked forward passes and run_with_cache call HF's module and were already correct.

This PR sets the flag in each adapter's __init__, before super().__init__() builds the component mapping. It has to be set there, because setup_blocks_bridge deep-copies the config into every block. Setting it on a booted bridge only reaches ln_final.

  • qwen3_5.py covers Qwen3.5, and Qwen3.5-MoE through inheritance.
  • qwen3_5_multimodal.py covers the Qwen3.5 multimodal adapter, and the Qwen3.5-MoE multimodal adapter through inheritance.
  • qwen3_next.py covers Qwen3Next.

Nemotron, added at review: NemotronLayerNorm1P applies gamma as (weight + 1) too, and its adapter had the same gap. nemotron.py now sets the flag as well. Its existing integration tests gain the identity-edit test on blocks.0.ln1 and ln_final. Without the flag the edit changed logits by 0.28 to 0.5; with it they match. The backward-hook and LN-rule paths are fixed too, and DLA's identity-norm guard now correctly rejects Nemotron, which it previously accepted. NemotronH uses a plain weight * x norm, so it doesn't need the flag.

User-facing change: on these five adapters, editing or backward-hooking a norm's hook_scale or hook_normalized now gives the same output as HF's forward, and LN-rule gradients through their norms use (1 + weight). Unhooked and observe-only use is unchanged.

Tests (behaviour only, in the existing test files):

  • TestQwen3_5RMSNormOffset (unit) builds a tiny Qwen3_5ForCausalLM and moves its norm weights off their zero init. It checks that an identity hook_scale edit on blocks.0.ln1, blocks.3.attn.q_norm and ln_final leaves the logits unchanged. It checks that a backward hook on ln_final.hook_scale leaves the forward output unchanged. It checks that the LN-rule gradient through blocks.0.ln1 equals (1 + weight) / denom.
  • TestQwen3NextRMSNormOffset (unit) runs the same identity-edit test on a tiny Qwen3NextForCausalLM.
  • The two multimodal adapters get the identity-edit test on blocks.0.ln1 and ln_final, added to their existing integration tests, which load the tiny trl-internal-testing checkpoints.
  • helpers.py gets two small helpers for the unit tests.

The Qwen3.5-MoE text adapter inherits the fix from Qwen3.5, so it has no test of its own.

Validation:

  • The 12 new test cases all fail on dev without the fix, with relative logit errors from 6.1e-3 to 1.0. With the fix they all pass, with errors around 1e-7.
  • make unit-test: 7144 passed, 61 skipped, 3 xfailed, 0 failed.
  • The seven touched test files (unit and integration), re-run after the final edits to the tests: 151 passed.
  • make check-format and uv run mypy . are clean.
  • I checked the other code that reads the flag. The fold_ln paths in weight_processing.py, the batched-expert fold in transformer_bridge.py and the weight-processing benchmark skip these adapters, because they set supports_fold_ln = False. On tiny models, enable_compatibility_mode() gives identical logits with and without the flag, and log-probs match HF to 1e-6.
  • Not run: tests that download full Qwen3.5 or Qwen3Next checkpoints. My machine has no GPU and 14 GB of RAM.

Not in this PR: the question in the issue about whether these adapters should also get Gemma's ArithmeticTensorConversion(ADDITION, 1.0) for their norm weights.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (none needed)
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

Claude helped me track this down and write the fix; I've reviewed the change and run the tests myself.

HF's Qwen3.5, Qwen3.5-MoE and Qwen3Next RMSNorms scale by (1 + weight).
Without the flag, NormalizationBridge dropped the 1 + whenever it computed
a norm itself: on hook_scale/hook_normalized edits, with a backward hook
attached, and in the LN-rule backward pass.

Fixes TransformerLensOrg#1868.

Worked through with a hand from Claude.

@jlarson4 jlarson4 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for putting this fix together @willgosnold, it looks great! Just one comment below

# split [query|gate] per head at forward time; slicing the gate out changes outputs.
setattr(cfg, "gated_q_proj", True)
# HF's Qwen3NextRMSNorm scales by (1 + weight), not weight (cf. Gemma).
setattr(cfg, "rmsnorm_uses_offset", True)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nemotron's LayerNorm1P also scales by (weight + 1), and our Nemotron architecture adapter doesn't declare this flag. Would you be willing to add this same fix for Nemotron as part of your PR?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Done in a4f9e8d: the Nemotron adapter now sets rmsnorm_uses_offset, with an identity-edit test on the tiny Nemotron checkpoint. Without the flag the edit changed logits by 0.28 to 0.5; with it they match. It also fixes the backward-hook and LN-rule paths, and DLA's identity-norm guard now correctly rejects Nemotron, which it previously accepted. NemotronH uses a plain weight * x norm, so it doesn't need the flag.

NemotronLayerNorm1P applies gamma as (weight + 1). The adapter already
disabled fold_ln for this, but never set the flag, so the bridge's own
norm paths (hook_scale/hook_normalized edits, backward hooks, LN-rule)
scaled by weight alone.
@willgosnold willgosnold changed the title Set rmsnorm_uses_offset on the Qwen3.5 and Qwen3Next adapters Set rmsnorm_uses_offset on the Qwen3.5, Qwen3Next and Nemotron adapters Oct 8, 2026
Jais2 subclasses the Nemotron adapter but uses plain nn.LayerNorm and
re-enables fold_ln, so inheriting the offset made folding use 1 + w and
broke compat-mode parity.
@willgosnold willgosnold changed the title Set rmsnorm_uses_offset on the Qwen3.5, Qwen3Next and Nemotron adapters fix(bridge): set rmsnorm_uses_offset on the Qwen3.5, Qwen3Next and Nemotron adapters Oct 8, 2026
@jlarson4
jlarson4 merged commit 825fbbb into TransformerLensOrg:dev Oct 8, 2026
27 checks passed
@jlarson4

jlarson4 commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the quick updates! Approved and merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants