mindformers.trainer.CausalLanguageModelingTrainer

class mindformers.trainer.CausalLanguageModelingTrainer(model_name: Optional[str] = None)

CausalLanguageModelingTrainer Task For Trainer. Args:

model_name (str): The model name of Task-Trainer. Default: None

Examples:

>>> from mindformers import CausalLanguageModelingTrainer
>>> gen_trainer = CausalLanguageModelingTrainer(model_name="gpt2")
>>> gen_trainer.train()
>>> res = gen_trainer.predict(input_data = "hello world [MASK]")

Raises:

NotImplementedError: If train method or evaluate method or predict method not implemented.

evaluate(config: Optional[Union[dict, MindFormerConfig, ConfigArguments, TrainingArguments]] = None, network: Optional[Union[Cell, BaseModel]] = None, dataset: Optional[Union[BaseDataset, GeneratorDataset]] = None, callbacks: Optional[Union[Callback, List[Callback]]] = None, compute_metrics: Optional[Union[dict, set]] = None, **kwargs)

Evaluate task for CausalLanguageModeling Trainer. This function is used to evaluate the network.

The trainer interface is used to quickly start training for general task. It also allows users to customize the network, dataset, callbacks, compute_metrics.

Args:

config (Optional[Union[dict, MindFormerConfig, ConfigArguments, TrainingArguments]]):

The task config which is used to configure the dataset, the hyper-parameter, optimizer, etc. It supports config dict or MindFormerConfig or TrainingArguments or ConfigArguments class. Default: None.

network (Optional[Union[Cell, BaseModel]]): The network for trainer.

It supports model name or BaseModel or MindSpore Cell class. Default: None.

dataset (Optional[Union[BaseDataset]]): The evaluate dataset.

It supports real dataset path or BaseDateset class or MindSpore Dataset class. Default: None.

callbacks (Optional[Union[Callback, List[Callback]]]): The eval callback function.

It supports CallBack or CallBack List of MindSpore. Default: None.

compute_metrics (Optional[Union[dict, set]]): The metric of evaluating.

It supports dict or set in MindSpore’s Metric class. Default: None.

generate_evaluate(config, network=None, dataset=None, compute_metrics=None, tokenizer=None, **kwargs)

Evaluate the text generate task. Return metrics with Rouge-1, Rouge-2, Rouge-l and BLEU.

predict(config: Optional[Union[dict, MindFormerConfig, ConfigArguments, TrainingArguments]] = None, input_data: Optional[Union[str, list, GeneratorDataset]] = None, network: Optional[Union[Cell, BaseModel]] = None, tokenizer: Optional[BaseTokenizer] = None, **kwargs)

Executes the predict of the trainer.

Args:

config (Optional[Union[dict, MindFormerConfig, ConfigArguments, TrainingArguments]]):

The task config which is used to configure the dataset, the hyper-parameter, optimizer, etc. It supports config dict or MindFormerConfig or TrainingArguments or ConfigArguments class. Default: None.

input_data (Optional[Union[Tensor, str, list]]): The predict data. It supports 1) a text string to be

translated, 1) a file name where each line is a text to be translated and 3) a generator dataset. Default: None.

network (Optional[Union[Cell, BaseModel]]): The network for trainer.

It supports model name or BaseModel or MindSpore Cell class. Default: None.

tokenizer (Optional[BaseTokenizer]): The tokenizer for tokenizing the input text.

Default: None.

Returns:

A list of prediction.

train(config: Optional[Union[dict, MindFormerConfig, ConfigArguments, TrainingArguments]] = None, network: Optional[Union[Cell, BaseModel]] = None, dataset: Optional[Union[BaseDataset, GeneratorDataset]] = None, wrapper: Optional[TrainOneStepCell] = None, optimizer: Optional[Optimizer] = None, callbacks: Optional[Union[Callback, List[Callback]]] = None, **kwargs)

Train task for CausalLanguageModeling Trainer. This function is used to train or fine-tune the network.

The trainer interface is used to quickly start training for general task. It also allows users to customize the network, optimizer, dataset, wrapper, callback.

Args:

config (Optional[Union[dict, MindFormerConfig, ConfigArguments, TrainingArguments]]):

The task config which is used to configure the dataset, the hyper-parameter, optimizer, etc. It supports config dict or MindFormerConfig or TrainingArguments or ConfigArguments class. Default: None.

network (Optional[Union[Cell, BaseModel]]): The network for trainer.

It supports model name or BaseModel or MindSpore Cell class. Default: None.

dataset (Optional[Union[BaseDataset, GeneratorDataset]]): The training dataset.

It support real dataset path or BaseDateset class or MindSpore Dataset class. Default: None.

optimizer (Optional[Optimizer]): The training network’s optimizer. It support Optimizer class of MindSpore.

Default: None.

wrapper (Optional[TrainOneStepCell]): Wraps the network with the optimizer.

It support TrainOneStepCell class of MindSpore. Default: None.

callbacks (Optional[Union[Callback, List[Callback]]]): The training callback function.

It support CallBack or CallBack List of MindSpore. Default: None.

Raises:

NotImplementedError: If wrapper not implemented.