mindformers.models.mae.ViTMAEForPreTraining

class mindformers.models.mae.ViTMAEForPreTraining(config=None)

Pretrain MAE Module. The supported model name could be selected from ViTMAEConfig.show_support_list().

参数

config (ViTMAEConfig) – the config of Mae model.

实际案例

>>> # input model name
>>> model_a = ViTMAEForPreTraining.from_pretrained('mae_vit_base_p16')
>>> # input config
>>> from mindformers import AutoConfig
>>> config = AutoConfig.from_pretrained('mae_vit_base_p16')
>>> model_b = ViTMAEForPreTraining(config)

init_pos_emd()

init values of pos_embed

init_weights()

ViT weight initialization.