mindformers.models.gpt2.GPT2Model

class mindformers.models.gpt2.GPT2Model(config)

The backbone of GPT network

Args:

config(GPT2Config): the config of network

Inputs:

input_ids: the tokenized inputs with datatype int32 input_mask: the mask indicating whether each position is a valid input

Returns:

output_state: Tensor, the output logit of backbone present_layer: Tensor, the current feature map embedding_table: Tensor, the embedding table for the vocabulary