mindformers.models.bloom.BloomConfig

class mindformers.models.bloom.BloomConfig(embedding_dropout_prob: float = 0.0, batch_size: int = None, seq_length: int = 1024, vocab_size: int = 250880, hidden_size: int = 64, num_layers: int = 2, num_heads: int = 8, expand_ratio: int = 4, hidden_dropout_rate: float = 0.1, attention_dropout_rate: float = 0.1, unk_token_id: int = 0, bos_token_id: int = 1, eos_token_id: int = 2, pad_token_id: int = 3, param_init_type: str = 'float32', embedding_init_type: str = 'float32', layernorm_compute_type: str = 'float32', softmax_compute_type: str = 'float32', compute_dtype: str = 'float16', hidden_act: str = 'gelu', parallel_config: mindformers.modules.transformer.transformer.TransformerOpParallelConfig = <mindformers.modules.transformer.transformer.TransformerOpParallelConfig object>, checkpoint_name_or_path: str = '', moe_config: mindformers.modules.transformer.moe.MoEConfig = <mindformers.modules.transformer.moe.MoEConfig object>, use_past: bool = False, use_seq_parallel: bool = False, use_select_recompute: bool = False, repetition_penalty: int = 1, max_decode_length: int = 1024, top_k: int = 5, top_p: int = 1, do_sample: bool = True, is_npu_acceleration: bool = False, **kwargs)

Bloom config class which defines the model size