mindformers.trainer¶
MindFormers Trainer API.
Context Config. |
|
Parallel Context Config. |
|
TrainingArguments is the subset of the arguments we use in our default config which is relate to the training in MindSpore. |
|
Trainer package to trainevaluatepredict class. |
|
Translation Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> from mindformers.trainer import TranslationTrainer >>> from mindformers import T5ForConditionalGeneration, TranslationTrainer >>> # follow the instruction in t5 section in the README.md and download wmt16 dataset. >>> # change the dataset_files path of configs/t5/wmt16_dataset.yaml >>> trans_trainer = TranslationTrainer(model_name='t5_small') >>> trans_trainer.train() >>> model = T5ForConditionalGeneration.from_pretrained('t5_small') >>> trans_trainer = TranslationTrainer(model_name="t5_small") >>> res = trans_trainer.predict(input_data="translate the English to Romanian: a good boy!", network=model) [{'translation_text': ['hello world']}] Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
CausalLanguageModelingTrainer Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> from mindformers import CausalLanguageModelingTrainer >>> gen_trainer = CausalLanguageModelingTrainer(model_name="gpt2") >>> gen_trainer.train() >>> res = gen_trainer.predict(input_data = "hello world [MASK]") Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
Contrastive Language Image Pretrain Trainer. |
|
ImageClassification Task For Trainer. |
|
MaskedImageModeling Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> import numpy as np >>> from mindspore.dataset import GeneratorDataset >>> from mindspore.nn import AdamWeightDecay, WarmUpLR, ... DynamicLossScaleUpdateCell, TrainOneStepWithLossScaleCell >>> from mindformers.trainer import GeneralTaskTrainer >>> from mindformers.tools.register import MindFormerConfig >>> from mindformers.models import ViTMAEForPreTraining, ViTMAEConfig >>> class MyDataLoader: ... def __init__(self): ... self.image = [np.zeros((3, 224, 224), np.float32) for _ in range(64)] ... self.mask = [np.zeros((196,), np.int32) for _ in range(64)] ... self.ids_restore = [np.zeros((196,), np.int32) for _ in range(64)] ... self.unmask_index = [np.zeros((49,), np.int32) for _ in range(64)] ... def __getitem__(self, index): ... return self.image[index], self.mask[index], self.ids_restore[index], self.unmask_index[index] ... def __len__(self): ... return len(self.image) >>> train_dataset = GeneratorDataset(source=MyDataLoader(), ... column_names=["image", "mask", "ids_restore", "unmask_index"]).batch(2) >>> #1) use config to train >>> mae_trainer = MaskedImageModelingTrainer(model_name='mae_vit_base_p16') >>> mae_trainer.train(dataset=train_dataset) >>> #2) use instance function to train >>> mae_config = ViTMAEConfig(batch_size=2) >>> network_with_loss = ViTMAEForPreTraining(mae_config) >>> lr_schedule = WarmUpLR(learning_rate=0.001, warmup_steps=100) >>> optimizer = AdamWeightDecay(beta1=0.009, beta2=0.999, ... learning_rate=lr_schedule, ... params=network_with_loss.trainable_params()) >>> loss_scale = DynamicLossScaleUpdateCell(loss_scale_value=2**12, scale_factor=2, scale_window=1000) >>> wrapper = TrainOneStepWithLossScaleCell(network_with_loss, optimizer, scale_sense=loss_scale) >>> mae_trainer.train(wrapper=wrapper, optimizer=optimizer, dataset=train_dataset) Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
MaskedLanguageModeling Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> from mindformers import MaskedLanguageModelingTrainer >>> def generator(): >>> data = np.random.randint(low=0, high=15, size=(128,)).astype(np.int32) >>> input_mask = np.ones_like(data) >>> token_type_id = np.zeros_like(data) >>> next_sentence_lables = np.array([1]).astype(np.int32) >>> masked_lm_positions = np.array([1, 2]).astype(np.int32) >>> masked_lm_ids = np.array([1, 2]).astype(np.int32) >>> masked_lm_weights = np.ones_like(masked_lm_ids) >>> train_data = (data, input_mask, token_type_id, next_sentence_lables, ... masked_lm_positions, masked_lm_ids, masked_lm_weights) >>> for _ in range(512): ... yield train_data >>> dataset = GeneratorDataset(generator, column_names=["input_ids", "input_mask", "segment_ids", ... "next_sentence_labels", "masked_lm_positions", ... "masked_lm_ids", "masked_lm_weights"]) >>> dataset = dataset.batch(batch_size=16) >>> mlm_trainer = MaskedLanguageModelingTrainer(model_name="bert_tiny_uncased") >>> mlm_trainer.train(dataset=dataset) >>> res = mlm_trainer.predict(input_data = "hello world [MASK]") Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
QuestionAnswering Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> import numpy as np >>> from mindspore.nn import AdamWeightDecay, TrainOneStepCell >>> from mindformers.core.lr import build_lr >>> from mindformers.trainer import GeneralTaskTrainer >>> from mindformers.tools.register import MindFormerConfig >>> from mindformers.models import BertForQuestionAnswering, BertConfig >>> config = MindFormerConfig("configs/qa/run_qa_bert_base_uncased.yaml") >>> #1) use config to train >>> cls_task = QuestionAnsweringTrainer(model_name='qa_bert_base_uncased') >>> cls_task.train(config=config) >>> #2) use instance function to train Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
TextClassification Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> import numpy as np >>> from mindspore.nn import AdamWeightDecay, TrainOneStepCell >>> from mindformers.core.lr import build_lr >>> from mindformers.trainer import TextClassificationTrainer >>> from mindformers.tools.register import MindFormerConfig >>> from mindformers.models import BertForMultipleChoice, BertConfig >>> config = MindFormerConfig("configs/txtcls/run_txtcls_bert_base_uncased.yaml") >>> #1) use default config to train >>> txtcls_task = TextClassificationTrainer(model_name='bert_for_multiple_choice') >>> txtcls_task.train(config=config) >>> txtcls_task.evaluate(config=config) >>> input_data = ["The new rights are nice enough-Everyone really likes the newest benefits ", ... "i don't know um do you do a lot of camping-I know exactly."] >>> res = txtcls_task.predict(input_data=input_data) >>> #2) use instance function to train Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
TokenClassification Task For Trainer. Args: model_name (str): The model name of Task-Trainer. Default: None Examples: >>> import numpy as np >>> from mindspore.dataset import GeneratorDataset >>> from mindspore.nn import AdamWeightDecay, TrainOneStepCell >>> from mindformers.core.lr import build_lr >>> from mindformers.trainer import TokenClassificationTrainer >>> from mindformers.tools.register import MindFormerConfig >>> from mindformers.models import BertForTokenClassification, BertConfig >>> config = MindFormerConfig("configs/tokcls/run_tokcls_bert_base_chinese.yaml") >>> #1) use config to train or evaluate or predict >>> tokcls_task = TokenClassificationTrainer(model_name='tokcls_bert_base_chinese') >>> tokcls_task.train(config=config) >>> tokcls_task.evaluate(config=config) >>> input_data = ["表身刻有代表日内瓦钟表匠freresoltramare的“fo”字样。", "的时间会去玩玩星际2。"] >>> res = tokcls_task.predict(input_data=input_data) >>> print(res) [[{'entity_group': 'address', 'word': '日内瓦', 'start': 6, 'end': 9}], [{'entity_group': 'game', 'word': '星际2', 'start': 7, 'end': 10}]] >>> #2) use instance function to train or evaluate or predict Raises: NotImplementedError: If train method or evaluate method or predict method not implemented. |
|
ZeroShotImageClassification Task For Trainer. |