[FIXED] HuggingFace-Transformers --- NER single sentence/sample prediction

Issue

I am trying to predict with the NER model, as in the tutorial from huggingface (it contains only the training+evaluation part).

I am following this exact tutorial here : https://github.com/huggingface/notebooks/blob/master/examples/token_classification.ipynb

The training works flawlessly, but the problems that I have begin when I try to predict on a simple sample.

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
loaded_model = AutoModel.from_pretrained('./my_model_own_custom_training.pth',
                                         from_tf=False)



input_sentence = "John Nash is a great mathematician, he lives in France"
tokenized_input_sentence = tokenizer([input_sentence],
                                     truncation=True, 
                                     is_split_into_words=False,
                                     return_tensors='pt')
predictions = loaded_model(tokenized_input_sentence["input_ids"])[0]

Predictions is of shape (1,13,768)

How can I arrive at the final result of the form [JOHN <-> ‘B-PER’, … France <-> “B-LOC”], where B-PER and B-LOC are two ground truth labels, representing the tag for a person and location respectively?

The result of the prediction is:

torch.Size([1, 13, 768])

If I write:

print(predictions.argmax(axis=2))
tensor([613, 705, 244, 620, 206, 206, 206, 620, 620, 620, 477, 693, 308])

I get the tensor above.

However I would have expected to get the tensor representing the ground truth [0…8] labels from the ground truth annotations.

Summary when loading the model :

loading configuration file ./my_model_own_custom_training.pth/config.json
Model config DistilBertConfig {
“name_or_path": “distilbert-base-uncased”,
“activation”: “gelu”,
“architectures”: [
“DistilBertForTokenClassification”
],
“attention_dropout”: 0.1,
“dim”: 768,
“dropout”: 0.1,
“hidden_dim”: 3072,
“id2label”: {
“0”: “LABEL_0”,
“1”: “LABEL_1”,
“2”: “LABEL_2”,
“3”: “LABEL_3”,
“4”: “LABEL_4”,
“5”: “LABEL_5”,
“6”: “LABEL_6”,
“7”: “LABEL_7”,
“8”: “LABEL_8”
},
“initializer_range”: 0.02,
“label2id”: {
“LABEL_0”: 0,
“LABEL_1”: 1,
“LABEL_2”: 2,
“LABEL_3”: 3,
“LABEL_4”: 4,
“LABEL_5”: 5,
“LABEL_6”: 6,
“LABEL_7”: 7,
“LABEL_8”: 8
},
“max_position_embeddings”: 512,
“model_type”: “distilbert”,
“n_heads”: 12,
“n_layers”: 6,
“pad_token_id”: 0,
“qa_dropout”: 0.1,
“seq_classif_dropout”: 0.2,
“sinusoidal_pos_embds”: false,
"tie_weights”: true,
“transformers_version”: “4.8.1”,
“vocab_size”: 30522
}

Solution

The answer is a bit trickier than expected[Huge credits to Niels Rogge].

Firstly, loading models in huggingface-transformers can be done in (at least) two ways:

1. AutoModel.from_pretrained('./my_model_own_custom_training.pth', from_tf=False)
2. AutoModelForTokenClassification.from_pretrained('./my_model_own_custom_training.pth', from_tf=False)

It seems that, according to the task at hand, different AutoModels subclasses need to be used. In this scenario I posted, it is the AutoModelForTokenClassification() that has to be used.

After that, a solution to obtain the predictions would be to do the following:

# forward pass
outputs = model(**encoding)
logits = outputs.logits

predictions = logits.argmax(-1)

Answered By - Timbus Calin