mindformers.models.glm.GLMForPreTraining

class mindformers.models.glm.GLMForPreTraining(config: GLMConfig)

Provide glm training loss or logits through network.

Args:

config (GLMConfig): The config of GLMModel.

create_position_ids_np(input_ids)

Get position ids from input_ids with numpy

get_masks_np(input_ids)

get attention mask using numpy.

get_position_ids_np(input_ids, mask_positions, use_gmasks=None)

Get position ids from input_ids and mask_positions with numpy

prepare_inputs_for_generation(input_ids, **kwargs)

prepare inputs for generation.

slice_incremental_inputs(model_inputs: dict, current_index)

used for non-first iterations, slice the inputs to length 1.

update_model_kwargs_before_generate(input_ids, model_kwargs: dict)

update glm kwargs before generate.