[Bug] AttributeError: 'SwinTransformer' object has no attribute 'out_type'

[kksummer]

分支

main 分支 (mmpretrain 版本)

描述该错误

我尝试使用mmpretrain/tools/visualization/vis_cam.py时出现了一个问题
按照官方文档进行 可视化 Transformer 类型网络的类别激活图

python tools/visualization/vis_cam.py \
    data/flower/train/daisy/2627815904_919373e7f5.jpg  \
    configs/swin_transformer/swin-base_16xb64_flower.py \
    checkpoint/swin_base_224_b16x64_300e_imagenet_20210616_190742-93230b0d.pth \
    --vit-like

然后出现了报错

Traceback (most recent call last):
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 273, in <module>
    main()
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 243, in main
    target_layers = get_default_traget_layers(model, args)
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 202, in get_default_traget_layers
    out_type = getattr(model.backbone, 'out_type')
  File "/home/ian/.conda/envs/mmlab/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1207, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'SwinTransformer' object has no attribute 'out_type'

我查看了在vis_cam中的代码

# vis_cam 194 line
    if args.vit_like:
        # For ViT models, the final classification is done on the class token.
        # And the patch tokens and class tokens won't interact each other after
        # the final attention layer. Therefore, we need to choose the norm
        # layer before the last attention layer.
        num_extra_tokens = args.num_extra_tokens or getattr(
            model.backbone, 'num_extra_tokens', 1)

        out_type = getattr(model.backbone, 'out_type')
        if out_type == 'cls_token' or num_extra_tokens > 0:
            # Assume the backbone feature is class token.
            name, layer = norm_layers[-3]
            print('Automatically choose the last norm layer before the '
                  f'final attention block "{name}" as the target layer.')
            return [layer]

然后尝试输出了model.backbone的所有属性，并没有找到Swin_Transformer的out_type属性

随后尝试了指定目标层

python tools/visualization/vis_cam.py \
    data/flower/train/daisy/2627815904_919373e7f5.jpg  \
    configs/swin_transformer/swin-base_16xb64_flower.py \
    checkpoint/swin_base_224_b16x64_300e_imagenet_20210616_190742-93230b0d.pth \
    --vit-like
    --target-layers backbone.norm3

还是报错，提示我norm3这层不可用，请选择下面的层
我几乎把所有层都试了一次，但还是出现这个问题

Traceback (most recent call last):
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 273, in <module>
    main()
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 239, in main
    target_layers = [
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 240, in <listcomp>
    get_layer(layer, model) for layer in args.target_layers
  File "/home/ian/Project/mmpretrain/tools/visualization/vis_cam.py", line 168, in get_layer
    raise AttributeError(
AttributeError: Cannot get the layer "'head'". Please choose from: 

data_preprocessor
backbone
backbone.patch_embed
backbone.patch_embed.adaptive_padding
backbone.patch_embed.projection
backbone.patch_embed.norm
backbone.drop_after_pos
backbone.stages
backbone.stages.0
backbone.stages.0.blocks
backbone.stages.0.blocks.0
...
...
backbone.stages.3.blocks.0.ffn.layers.0.0
backbone.stages.3.blocks.0.ffn.layers.0.1
backbone.stages.3.blocks.0.ffn.layers.0.2
backbone.stages.3.blocks.0.ffn.layers.1
backbone.stages.3.blocks.0.ffn.layers.2
backbone.stages.3.blocks.0.ffn.dropout_layer
backbone.stages.3.blocks.0.ffn.gamma2
backbone.stages.3.blocks.1
backbone.stages.3.blocks.1.norm1
backbone.stages.3.blocks.1.attn
backbone.stages.3.blocks.1.attn.w_msa
backbone.stages.3.blocks.1.attn.w_msa.qkv
backbone.stages.3.blocks.1.attn.w_msa.attn_drop
backbone.stages.3.blocks.1.attn.w_msa.proj
backbone.stages.3.blocks.1.attn.w_msa.proj_drop
backbone.stages.3.blocks.1.attn.w_msa.softmax
backbone.stages.3.blocks.1.attn.drop
backbone.stages.3.blocks.1.norm2
backbone.stages.3.blocks.1.ffn
backbone.stages.3.blocks.1.ffn.layers
backbone.stages.3.blocks.1.ffn.layers.0
backbone.stages.3.blocks.1.ffn.layers.0.0
backbone.stages.3.blocks.1.ffn.layers.0.1
backbone.stages.3.blocks.1.ffn.layers.0.2
backbone.stages.3.blocks.1.ffn.layers.1
backbone.stages.3.blocks.1.ffn.layers.2
backbone.stages.3.blocks.1.ffn.dropout_layer
backbone.stages.3.blocks.1.ffn.gamma2
backbone.norm3
neck
neck.gap
head
head.loss_module
head.fc

环境信息

{'sys.platform': 'linux',
 'Python': '3.8.16 (default, Mar  2 2023, 03:21:46) [GCC 11.2.0]',
 'CUDA available': True,
 'numpy_random_seed': 2147483648,
 'GPU 0': 'Tesla M40 24GB',
 'CUDA_HOME': None,
 'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0',
 'PyTorch': '1.12.1',
 'TorchVision': '0.13.1',
 'OpenCV': '4.7.0',
 'MMEngine': '0.7.3',
 'MMCV': '2.0.0',
 'MMPreTrain': '1.0.0rc7+e80418a'}

其他信息

No response

[Ezra-Yu]

Yes, it is a bug, Swin is different from other vit-like backbone. To fix this bug, you can modify this line

mmpretrain/tools/visualization/vis_cam.py

Line 202 in e80418a

out_type = getattr(model.backbone, 'out_type')

to :

out_type = getattr(model.backbone, 'out_type', 'avg_feat')

Welcome to create a PR to fix it.

[mzr1996]

Fixed by #1586