You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Access self-attention matrix of vision transformer #6032
Is your feature request related to a problem? Please describe.
I am training a vision transformer using monai and would like to carry out some interpretability analysis. However, the current model code does not save the self-attention matrix during training, and it is not straightforward to pass it from the self-attention block to the model output.
att_mat= (torch.einsum("blxd,blyd->blxy", q, k) *self.scale).softmax(dim=-1)
att_mat=self.drop_weights(att_mat)
x=torch.einsum("bhxy,bhyd->bhxd", att_mat, v)
x=self.out_rearrange(x)
x=self.out_proj(x)
x=self.drop_output(x)
returnx
Describe the solution you'd like
An option to output the attn_mat from the self-attention block in the model forward pass (before matrix multiplication with the input) or access it after training as a class attribute.
I was thinking, the easiest way to achieve this without changing the API could be by making att_mat before dropout as a class attribute so could be accessed by SABlock().attn_mat. also, a parameter like store_attn:bool in the __init__ to prevent memory overhead when the feature is not required.
@wyli I just realized that ViT backbone is used by by UNetr and ViTAutoEnc ideally they should have the option to allow access to the the attn_mat else it will always be set as the default value and no way it can be changed by the user of UNetr and ViTAutoEnc. What do you think?
Activity
a-parida12 commentedon Apr 2, 2023
@wyli I would like to help with this issue.
I was thinking, the easiest way to achieve this without changing the API could be by making
att_matbefore dropout as a class attribute so could be accessed bySABlock().attn_mat. also, a parameter likestore_attn:boolin the__init__to prevent memory overhead when the feature is not required.Let me know your thoughts.
wyli commentedon Apr 2, 2023
sounds good, please make sure it's backward compatible, e.g. previously saved checkpoints can still be loaded by default.
feat(SABlock): access to the attn matrix (#6271)
a-parida12 commentedon Apr 5, 2023
@wyli so #6271 is merged? So do i put a new merge request for a potential solution?
feat(SABlock): access atten matrix jit compliant (#6308)
a-parida12 commentedon Apr 17, 2023
@wyli I just realized that
ViTbackbone is used by byUNetrandViTAutoEncideally they should have the option to allow access to the theattn_matelse it will always be set as the default value and no way it can be changed by the user ofUNetrandViTAutoEnc. What do you think?wyli commentedon Apr 18, 2023
Sure, please help create another feature request to follow up..