Leaderboard - UD Tagset

Rank Model name Model url Pretrained embeddings Dataset Metric Average Tokens Sentences Words UPOS XPOS UFeats AllTags Lemmas UAS LAS CLAS MLAS BLEX
1 combo3 LINK DCU-NLP/bert-base-irish-cased-v1 IDT AligndAcc 84.43 - - - 96.47 94.65 90.56 86.36 94.46 86.74 82.28 77.58 63.83 71.33
F1 87.59 99.65 97.58 99.65 96.13 94.32 90.25 86.07 94.14 86.44 81.99 77.45 63.72 71.21
2 stanza LINK fasttext IDT AligndAcc 83.78 - - - 96.28 94.74 88.84 86.13 94.66 87.12 81.59 75.88 62.17 70.35
F1 87.08 99.55 97.81 99.55 95.85 94.31 88.44 85.74 94.22 86.73 81.22 75.95 62.22 70.41
3 trankit LINK xlm-RoBERTa-base IDT AligndAcc 83.29 - - - 95.79 94.55 89.69 86.02 91.47 87.32 81.73 76.45 63.12 66.75
F1 86.62 99.54 97.35 99.54 95.36 94.12 89.28 85.63 91.05 86.93 81.36 76.31 63.01 66.63
4 combo LINK bert-base IDT AligndAcc 82.15 - - - 95.14 93.66 88.70 84.79 93.95 85.18 79.34 73.33 59.91 67.54
F1 85.87 99.65 97.58 99.65 94.81 93.33 88.39 84.49 93.62 84.89 79.06 73.35 59.92 67.55
5 combo LINK fasttext IDT AligndAcc 81.76 - - - 94.10 93.69 88.44 84.49 93.60 85.25 78.95 73.00 59.05 67.05
F1 85.57 99.65 97.58 99.65 93.77 93.36 88.13 84.20 93.28 84.96 78.68 73.03 59.08 67.08
6 udpipe LINK fasttext IDT AligndAcc 78.05 - - - 93.82 92.42 85.77 82.79 91.96 81.63 73.76 65.77 53.24 59.31
F1 82.81 99.65 98.23 99.65 93.49 92.10 85.48 82.50 91.65 81.34 73.50 66.01 53.43 59.53

Legend:

  • Prediction is done on plain text file.