mindformers.models.bert.BertForTokenClassification

class mindformers.models.bert.BertForTokenClassification(config={'assessment_method': '', 'attention_probs_dropout_prob': 0.1, 'batch_size': 16, 'checkpoint_name_or_path': '', 'compute_dtype': mindspore.float16, 'dropout_prob': 0.1, 'dtype': mindspore.float32, 'hidden_act': 'gelu', 'hidden_dropout_prob': 0.1, 'hidden_size': 768, 'initializer_range': 0.02, 'intermediate_size': 3072, 'is_training': True, 'layernorm_dtype': mindspore.float32, 'max_position_embeddings': 128, 'model_type': 'bert', 'moe_config': <mindformers.modules.transformer.moe.MoEConfig object>, 'num_attention_heads': 12, 'num_hidden_layers': 12, 'num_labels': 1, 'parallel_config': <mindformers.modules.transformer.transformer.TransformerOpParallelConfig object>, 'post_layernorm_residual': True, 'seq_length': 128, 'softmax_dtype': mindspore.float32, 'type_vocab_size': 2, 'use_one_hot_embeddings': False, 'use_past': False, 'use_relative_positions': False, 'vocab_size': 30522})

Bert with dense layer for name entity recoginition task.

Args:

config (BertConfig): The config of BertForTokenClassification.

Returns:

Tensor, loss, logits.

Examples:

>>> from mindformers import BertForTokenClassification, BertTokenizer
>>> model = BertForTokenClassification.from_pretrained('tokcls_bert_base_chinese')
>>> tokenizer = BertTokenizer.from_pretrained('tokcls_bert_base_chinese')
>>> data = tokenizer(["我在杭州华为工作。"], return_tensors='ms', max_length=128, padding='max_length')
>>> input_ids = data['input_ids']
>>> attention_mask = data['attention_mask']
>>> token_type_ids = data['token_type_ids']
>>> output = model(input_ids, attention_mask, token_type_ids)
>>> print(output)
[[[ 0.0886748  -0.23066735 -0.03969013 ...  0.07333283 -0.02968273
    0.06125224]
  [-0.03115104 -0.47599065  0.01928361 ...  0.22821501  0.42415133
    0.14563856]
  [ 0.05570185 -0.36168078  0.00884905 ... -0.15357454 -0.33119604
   -0.09889631]
  ...
  [ 0.01028904 -0.20403272 -0.1117363  ... -0.03601318  0.034153
    0.12492958]
  [-0.00420664 -0.1786235  -0.11332294 ... -0.07119732  0.02407441
    0.14775723]
  [-0.04674841 -0.1608555  -0.0781534  ... -0.06064402  0.0439388
    0.20419256]]]