mindformers.models.swin.SwinConfig¶

class mindformers.models.swin.SwinConfig(image_size: int = 224, patch_size: int = 4, num_channels: int = 3, embed_dim: int = 128, depths: list = (2, 2, 18, 2), num_heads: list = (4, 8, 16, 32), window_size: int = 7, shift_size: int = 0, mlp_ratio: float = 4.0, qkv_bias: bool = True, layer_norm_eps: float = 1e-05, hidden_dropout_prob: float = 0.0, attention_probs_dropout_prob: float = 0.0, drop_path_rate: float = 0.1, use_absolute_embeddings: bool = False, patch_norm: bool = True, hidden_act: str = 'gelu', weight_init: str = 'normal', num_labels: int = 1000, loss_type: str = 'SoftTargetCrossEntropy', param_init_type: <module 'mindspore.common.dtype' from '/home/docs/checkouts/readthedocs.org/user_builds/mindformerstest/envs/latest/lib/python3.7/site-packages/mindspore/common/dtype.py'> = mindspore.float32, moe_config: ~mindformers.modules.transformer.moe.MoEConfig = <mindformers.modules.transformer.moe.MoEConfig object>, parallel_config: ~mindformers.modules.transformer.transformer.TransformerOpParallelConfig = <mindformers.modules.transformer.transformer.TransformerOpParallelConfig object>, checkpoint_name_or_path: str = '', **kwargs)[源代码]¶

Swin config class which defines the model size

Args:

image_size: The input image size, Default 224. patch_size: patch size, Default 4. num_channels: channels of input images, Default 3. embed_dim: embedding dimension, Default 128. depths: number of transformer blocks for each swin layer, Default (2, 2, 18, 2). num_heads: number of attention heads for each swin layer, Default (4, 8, 16, 32). window_size: window size for swin, Default 7. shift_size: window shift size, Default 0. mlp_ratio: ffn_hidden_size = mlp_ratio * embed_dim, Default 4. qkv_bias: has transformer qkv bias or not, Default True. hidden_dropout_prob: drop rate of MLP, Default 0. attention_probs_dropout_prob: drop rate of Attention, Default 0. drop_path_rate: drop path rate of transformer blocks, Default 0.1. use_absolute_embeddings: if using absolute position embedding, Default False. patch_norm: use norm in SwinPatchEmbeddings, Default True. hidden_act: activation of MLP, Default “gelu”. weight_init: weight initialize type, Default “normal”. num_labels: number of labels in downstream tasks, Default 1000. loss_type: loss type, Default “SoftTargetCrossEntropy”. param_init_type:, Default mstype.float32. moe_config:, Default default_moe_config. parallel_config:, Default default_parallel_config. checkpoint_name_or_path:, Default “swin_base_p4w7”. **kwargs

Examples:

>>> # init a config with a model name
>>> config_a = SwinConfig.from_pretrained('swin_base_p4w7')
>>> # init a config with a config path
>>> import os
>>> from mindformers.mindformer_book import MindFormerBook
>>> config_path = os.path.join(MindFormerBook.get_project_path(),
>>>                        'configs', 'swin', 'run_swin_base_p4w7_224_100ep.yaml')
>>> config_b = SwinConfig.from_pretrained(config_path)
>>> # init a config with args
>>> config_c = SwinConfig(
>>>     patch_size=4,
>>>     in_chans=3,
>>>     ...
>>>     )