Skip to content

fix removed names from import - #1802

Merged
jlarson4 merged 2 commits into
TransformerLensOrg:devfrom
MdSadiqMd:sadiq/fix-removed-names-from-import
Sep 23, 2026
Merged

jlarson4 merged 2 commits into
TransformerLensOrg:devfrom
MdSadiqMd:sadiq/fix-removed-names-from-import

Conversation

@MdSadiqMd

Copy link
Copy Markdown
Contributor

Description

The _REMOVED_IN_4_0 shim (added in #1759) exists so removed top-level names fail with a migration pointer instead of a bare error. It does that for attribute access:

>>> import transformer_lens
>>> transformer_lens.HookedTransformer
AttributeError: 'HookedTransformer' was removed in TransformerLens 4.0. Use TransformerBridge.boot_transformers(name), then enable_compatibility_mode() ... See docs/source/content/migrating_to_v4.md.

But it did not reach the from transformer_lens import HookedTransformer spelling — which is the one downstream packages actually use (e.g. circuit_tracer does from transformer_lens import HookedTransformer, HookedTransformerConfig):

>>> from transformer_lens import HookedTransformer
ImportError: cannot import name 'HookedTransformer' from 'transformer_lens' (.../transformer_lens/__init__.py)

CPython's from-import machinery calls getattr on the module and, on AttributeError, discards it and raises its own ImportError. The original is not chained (__cause__ and __context__ are both None), so the migration pointer is not merely hidden — it is gone. The __getattr__ shim's comment anticipated the submodule case explicitly and read as though the from-import case was covered; it was the one in between that got missed, and nothing under any test path exercised it.

Fix: __getattr__ now raises ImportError (not AttributeError) for removed names, with name=name metadata. A non-AttributeError raised from getattr is propagated by the from-import machinery rather than swallowed, so the pointer now reaches both spellings:

>>> from transformer_lens import HookedTransformer
ImportError: 'HookedTransformer' was removed in TransformerLens 4.0. Use TransformerBridge...
>>> transformer_lens.HookedTransformer
ImportError: 'HookedTransformer' was removed in TransformerLens 4.0. Use TransformerBridge...

Trade-off (intended, and documented in the comment): hasattr(transformer_lens, "HookedTransformer") now raises instead of returning False, because hasattr only swallows AttributeError. Version-detection code written this way now surfaces the migration pointer rather than a silent False. An exception inheriting from both ImportError and AttributeError is not possible (ImportError's C-level layout conflicts), so the two forms cannot be satisfied by a single type — this fixes the primary case (from-import) that downstream porting hits.

Regression tests are parametrized over every name in _REMOVED_IN_4_0 and pin that both the attribute form and the from-import form reach the pointer. They accept either AttributeError or ImportError on purpose, so they pin the reach-the-pointer property rather than the resolution.

Not addressed here (separate concern, out of scope): transformer_lens.__version__ is absent on 4.0 and not in _REMOVED_IN_4_0, so it answers with the generic no-attribute message; importlib.metadata.version("transformer-lens") still works.

Fixes #1801

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Screenshots

N/A

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@koriyoshi2041 koriyoshi2041 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at 3d8f746b. The focused module passes locally (24 tests), including every removed top-level name through both attribute access and from ... import .... The message assertion also guards against falling back to CPython's generic unchained ImportError, while unknown names retain normal AttributeError behavior. The documented hasattr trade-off is bounded to the removed-name set.

@jlarson4 jlarson4 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this! Just a couple small notes on other related areas we should probably touch as a part of this PR

Comment thread transformer_lens/__init__.py Resolved
Comment thread transformer_lens/__init__.py Resolved
Comment thread transformer_lens/__init__.py Resolved

@koriyoshi2041 koriyoshi2041 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at daae5940. The three follow-ups are implemented: the migration guide now documents the ImportError and feature-detection trade-off, the hook_points.HookedRootModule re-export preserves its migration pointer for both access forms, and focused regressions cover both paths. The full hosted matrix is green.

@MdSadiqMd

Copy link
Copy Markdown
Contributor Author

Hey @jlarson4 review comments are solved, let me know if you need anything from my side

@jlarson4

Copy link
Copy Markdown
Collaborator

Thanks @MdSadiqMd, looks great!

@jlarson4
jlarson4 merged commit 0525482 into TransformerLensOrg:dev Sep 23, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_REMOVED_IN_4_0's migration pointer is lost for from transformer_lens import HookedTransformer, the form downstream packages use

3 participants