mindformers.models.bert.BertForMultipleChoice¶

class mindformers.models.bert.BertForMultipleChoice(config={'assessment_method': '', 'attention_probs_dropout_prob': 0.1, 'batch_size': 16, 'checkpoint_name_or_path': '', 'compute_dtype': mindspore.float16, 'dropout_prob': 0.1, 'dtype': mindspore.float32, 'hidden_act': 'gelu', 'hidden_dropout_prob': 0.1, 'hidden_size': 768, 'initializer_range': 0.02, 'intermediate_size': 3072, 'is_training': True, 'layernorm_dtype': mindspore.float32, 'max_position_embeddings': 128, 'model_type': 'bert', 'moe_config': <mindformers.modules.transformer.moe.MoEConfig object>, 'num_attention_heads': 12, 'num_hidden_layers': 12, 'num_labels': 1, 'parallel_config': <mindformers.modules.transformer.transformer.TransformerOpParallelConfig object>, 'post_layernorm_residual': True, 'seq_length': 128, 'softmax_dtype': mindspore.float32, 'type_vocab_size': 2, 'use_one_hot_embeddings': False, 'use_past': False, 'use_relative_positions': False, 'vocab_size': 30522})[源代码]¶

Bert with dense layer for txt classification task.

参数: config (BertConfig) – The config of BertForMultipleChoice.
返回: Tensor, loss, logits.

实际案例

>>> from mindformers import BertForMultipleChoice, BertTokenizer
>>> model = BertForMultipleChoice.from_pretrained('txtcls_bert_base_uncased')
>>> tokenizer = BertTokenizer.from_pretrained('txtcls_bert_base_uncased')
>>> data = tokenizer(["The new rights are nice enough-Everyone really likes the newest benefits "],
...                  return_tensors='ms', max_length=128, padding='max_length')
>>> input_ids = data['input_ids']
>>> attention_mask = data['attention_mask']
>>> token_type_ids = data['token_type_ids']
>>> output = model(input_ids, attention_mask, token_type_ids)
>>> print(output)
[[0.05121157 0.3273056  0.31614417]]