mindformers.models.BaseModel

class mindformers.models.BaseModel(config: BaseConfig, **kwargs)

The base model that contains the class method from_pretained and save_pretrained, any new model that should inherit the class.

Note:

GeneratorMixin provides the method generate that enable the generation for nlp models.

Args:

config(BaseConfig): The model configuration that inherits the BaseConfig.

classmethod from_pretrained(pretrained_model_name_or_dir: str, **kwargs)

Instantiates a model by the pretrained_model_name_or_dir. It download the model weights if the user pass a model name, or load the weight from the given directory if given the path. (only support standalone mode, and distribute mode waits for developing!)

Args:

pretrained_model_name_or_dir (str): It supports the following two input types.

If pretrained_model_name_or_dir is a supported model name, for example, vit_base_p16 and t5_small, it will download the necessary files from the cloud. User can pass one from the support list by call MindFormerBook.get_model_support_list(). If pretrained_model_name_or_dir is a path to the local directory where there should have model weights ended with .ckpt and configuration file ended with yaml.

pretrained_model_name_or_path (Optional[str]): Equal to “pretrained_model_name_or_dir”,

if “pretrained_model_name_or_path” is set, “pretrained_model_name_or_dir” is useless.

Examples:

>>> from mindformers import T5ForConditionalGeneration
>>> net = T5ForConditionalGeneration.from_pretrained('t5_small')

Returns:

A model, which inherited from BaseModel.

classmethod get_support_list()

get_support_list method

load_checkpoint(config)

load checkpoint for models.

Args:

config (ModelConfig): a model config instance, which could have attribute “checkpoint_name_or_path (str)”. set checkpoint_name_or_path to a supported model name or a path to checkpoint, to load model weights.

remove_type(config)

remove type caused by save’

save_pretrained(save_directory: Optional[str] = None, save_name: str = 'mindspore_model')

Save the model weight and configuration file. (only supports standalone mode, and distribute mode waits for developing)

Args:

save_directory(str): a directory to save the model weight and configuration.

If None, the directory will be ./checkpoint_save, which can be obtained by the MindFormerBook.get_default_checkpoint_save_folder(). If set, the directory will be what is set.

save_name(str): the name of saved files, including model weight and configuration file.

Default mindspore_model.

Examples:

>>> import os
>>> from mindformers import T5ForConditionalGeneration, MindFormerBook
>>> net = T5ForConditionalGeneration.from_pretrained('t5_small')
>>> net.save_pretrained()
>>> output_path = MindFormerBook.get_default_checkpoint_save_folder()
>>> print(os.listdir(output_path))
['mindspore_model.yaml', 'mindspore_model.ckpt']

classmethod show_support_list()

show_support_list method