mindformers.models.swin.SwinConfig

class mindformers.models.swin.SwinConfig(image_size: int = 224, patch_size: int = 4, num_channels: int = 3, embed_dim: int = 128, depths: list = (2, 2, 18, 2), num_heads: list = (4, 8, 16, 32), window_size: int = 7, shift_size: int = 0, mlp_ratio: float = 4.0, qkv_bias: bool = True, layer_norm_eps: float = 1e-05, hidden_dropout_prob: float = 0.0, attention_probs_dropout_prob: float = 0.0, drop_path_rate: float = 0.1, use_absolute_embeddings: bool = False, patch_norm: bool = True, hidden_act: str = 'gelu', weight_init: str = 'normal', num_labels: int = 1000, loss_type: str = 'SoftTargetCrossEntropy', param_init_type: <module 'mindspore.common.dtype' from '/home/docs/checkouts/readthedocs.org/user_builds/mindformerstest/envs/stable/lib/python3.7/site-packages/mindspore/common/dtype.py'> = mindspore.float32, moe_config: mindformers.modules.transformer.moe.MoEConfig = <mindformers.modules.transformer.moe.MoEConfig object>, parallel_config: mindformers.modules.transformer.transformer.TransformerOpParallelConfig = <mindformers.modules.transformer.transformer.TransformerOpParallelConfig object>, checkpoint_name_or_path: str = '', **kwargs)[源代码]

Swin config class which defines the model size

参数

  • image_size – The input image size, Default 224.

  • patch_size – patch size, Default 4.

  • num_channels – channels of input images, Default 3.

  • embed_dim – embedding dimension, Default 128.

  • depths – number of transformer blocks for each swin layer, Default (2, 2, 18, 2).

  • num_heads – number of attention heads for each swin layer, Default (4, 8, 16, 32).

  • window_size – window size for swin, Default 7.

  • shift_size – window shift size, Default 0.

  • mlp_ratio – ffn_hidden_size = mlp_ratio * embed_dim, Default 4.

  • qkv_bias – has transformer qkv bias or not, Default True.

  • hidden_dropout_prob – drop rate of MLP, Default 0.

  • attention_probs_dropout_prob – drop rate of Attention, Default 0.

  • drop_path_rate – drop path rate of transformer blocks, Default 0.1.

  • use_absolute_embeddings – if using absolute position embedding, Default False.

  • patch_norm – use norm in SwinPatchEmbeddings, Default True.

  • hidden_act – activation of MLP, Default “gelu”.

  • weight_init – weight initialize type, Default “normal”.

  • num_labels – number of labels in downstream tasks, Default 1000.

  • loss_type – loss type, Default “SoftTargetCrossEntropy”.

  • param_init_type – , Default mstype.float32.

  • moe_config – , Default default_moe_config.

  • parallel_config – , Default default_parallel_config.

  • checkpoint_name_or_path – , Default “swin_base_p4w7”.

  • **kwargs –

实际案例

>>> # init a config with a model name
>>> config_a = SwinConfig.from_pretrained('swin_base_p4w7')
>>> # init a config with a config path
>>> import os
>>> from mindformers.mindformer_book import MindFormerBook
>>> config_path = os.path.join(MindFormerBook.get_project_path(),
>>>                        'configs', 'swin', 'run_swin_base_p4w7_224_100ep.yaml')
>>> config_b = SwinConfig.from_pretrained(config_path)
>>> # init a config with args
>>> config_c = SwinConfig(
>>>     patch_size=4,
>>>     in_chans=3,
>>>     ...
>>>     )