mindformers.models.mae.ViTMAEForPreTraining

class mindformers.models.mae.ViTMAEForPreTraining(config=None)

Pretrain MAE Module. The supported model name could be selected from ViTMAEConfig.show_support_list().

Args:

config (ViTMAEConfig): the config of Mae model.

Examples:

>>> # input model name
>>> model_a = ViTMAEForPreTraining.from_pretrained('mae_vit_base_p16')
>>> # input config
>>> from mindformers import AutoConfig
>>> config = AutoConfig.from_pretrained('mae_vit_base_p16')
>>> model_b = ViTMAEForPreTraining(config)

init_pos_emd()

init values of pos_embed

init_weights()

ViT weight initialization.