A | B | C | D | E | F | G | H | I | |
---|---|---|---|---|---|---|---|---|---|
1 | |||||||||
2 | Task 1 | ||||||||
3 | Total teams: | 9 | |||||||
4 | Total submissions: | 26 | |||||||
5 | Team | File | F1 Score | Precision | Recall | ||||
6 | UA | pp_0.65_10_3.csv | 0.3715 | 0.4111 | 0.3389 | ||||
7 | UA | pp_0.7_9_2.csv | 0.3710 | 0.4967 | 0.2961 | ||||
8 | siat | siatrun1.txt | 0.3691 | 0.3005 | 0.4782 | ||||
9 | siat | siatrun3.txt | 0.3680 | 0.3026 | 0.4695 | ||||
10 | UA | pp_0.65_6.csv | 0.3559 | 0.3630 | 0.3492 | ||||
11 | siat | siatrun2.txt | 0.2964 | 0.2522 | 0.3595 | ||||
12 | LeiBi | run_bm25.txt | 0.2923 | 0.3000 | 0.2850 | ||||
13 | LeiBi | run_weighting.txt | 0.2917 | 0.2687 | 0.3191 | ||||
14 | JNLP | run3.txt | 0.2813 | 0.3211 | 0.2502 | ||||
15 | nigam | bm25P3M3.txt | 0.2809 | 0.2587 | 0.3072 | ||||
16 | JNLP | run2.txt | 0.2781 | 0.3144 | 0.2494 | ||||
17 | DSSR | DSSR_01.txt | 0.2657 | 0.2447 | 0.2906 | ||||
18 | JNLP | run1.txt | 0.2639 | 0.2446 | 0.2866 | ||||
19 | DSSR | DSSR_03.txt | 0.2461 | 0.2267 | 0.2692 | ||||
20 | TUWBR | TUWBR_LM_law | 0.2367 | 0.1895 | 0.3151 | ||||
21 | LeiBi | run_clustering.txt | 0.2306 | 0.2367 | 0.2249 | ||||
22 | TUWBR | TUWBR_LM | 0.2206 | 0.1683 | 0.3199 | ||||
23 | nigam | sbertP3M3RS.txt | 0.1542 | 0.1420 | 0.1686 | ||||
24 | nigam | s2vecP3M3RS.txt | 0.1484 | 0.1367 | 0.1623 | ||||
25 | DSSR | DSSR_02.txt | 0.1317 | 0.1213 | 0.1441 | ||||
26 | LLNTU | 2022.task1.LLNTUtfidCos | 0.0000 | 0.0000 | 0.0000 | ||||
27 | LLNTU | 2022.task1.LLNTUtanadaT | 0.0000 | 0.0000 | 0.0000 | ||||
28 | LLNTU | 2022.task1.LLNTU3q4clii | 0.0000 | 0.0000 | 0.0000 | ||||
29 | Uottawa | Task1Run3_UottawaLegalBert.txt | 0.0000 | 0.0000 | 0.0000 | ||||
30 | Uottawa | Task1Run1_UottawaMB25.txt | 0.0000 | 0.0000 | 0.0000 | ||||
31 | Uottawa | Task1Run2_UottawaSentTrans.txt | 0.0000 | 0.0000 | 0.0000 | ||||
32 | |||||||||
33 | |||||||||
34 | Task 2 | ||||||||
35 | Total teams: | 5 | |||||||
36 | Total submissions: | 14 | |||||||
37 | Team | File | F1 Score | Precision | Recall | ||||
38 | NM | monot5-ensemble.txt | 0.6783 | 0.6964 | 0.6610 | ||||
39 | NM | monot5-3b.txt | 0.6757 | 0.7212 | 0.6356 | ||||
40 | JNLP | run2_bert_amr_remove_reduntdant_filter.txt | 0.6694 | 0.6532 | 0.6864 | ||||
41 | JNLP | run3_bert_BM25.txt | 0.6612 | 0.6452 | 0.6780 | ||||
42 | jljy | run2_task2.txt | 0.6514 | 0.7100 | 0.6017 | ||||
43 | jljy | run3_task2.txt | 0.6514 | 0.7100 | 0.6017 | ||||
44 | JNLP | run1_bert_amr_remove_reduntdant.txt | 0.6452 | 0.6154 | 0.6780 | ||||
45 | jljy | run1_task2.txt | 0.6330 | 0.6900 | 0.5847 | ||||
46 | NM | monot5-base.txt | 0.6325 | 0.6379 | 0.6271 | ||||
47 | UA | res_score-0.95_max-1.txt | 0.5446 | 0.6105 | 0.4915 | ||||
48 | UA | res_score-0.5_max-1.txt | 0.5363 | 0.7869 | 0.4068 | ||||
49 | UA | res_score-0.95_max-5.txt | 0.4121 | 0.3049 | 0.6356 | ||||
50 | nigam | bm25EF.txt | 0.3204 | 0.1980 | 0.8390 | ||||
51 | nigam | bm25BC.txt | 0.2104 | 0.1300 | 0.5508 | ||||
52 | |||||||||
53 | |||||||||
54 | Task 3 | ||||||||
55 | Total teams: | 5 | |||||||
56 | Total submissions: | 15 | |||||||
57 | Team | sid | F2 | Prec | Recall | MAP | R_5 | R_10 | R_30 |
58 | HUKB | HUKB2 | 0.8204 | 0.8180 | 0.8405 | 0.8431 | 0.8473 | 0.8779 | 0.9466 |
59 | HUKB | HUKB1 | 0.7908 | 0.6532 | 0.8901 | 0.8725 | 0.8931 | 0.9084 | 0.9618 |
60 | OVGU | OVGU_run3 | 0.7790 | 0.7781 | 0.8054 | 0.8364 | 0.8321 | 0.8779 | 0.9389 |
61 | OVGU | OVGU_run1 | 0.7732 | 0.7745 | 0.7962 | 0.8167 | 0.8550 | 0.8855 | 0.9466 |
62 | OVGU | OVGU_run2 | 0.7718 | 0.7732 | 0.8008 | 0.8272 | 0.8244 | 0.8702 | 0.9466 |
63 | JNLP | JNLP.longformer | 0.7699 | 0.6865 | 0.8378 | 0.7934 | 0.8397 | 0.9160 | 0.9618 |
64 | JNLP | JNLP.electra_longformer | 0.7677 | 0.6386 | 0.8625 | 0.7629 | 0.8015 | 0.8015 | 0.8015 |
65 | JNLP | JNLP.electra_legal | 0.7670 | 0.6356 | 0.8607 | 0.8088 | 0.8397 | 0.9084 | 0.9618 |
66 | UA | UA_TFIDF2 | 0.7638 | 0.8073 | 0.7641 | 0.8287 | 0.8244 | 0.8550 | 0.9084 |
67 | UA | UA_TFIDF1 | 0.7607 | 0.8165 | 0.7549 | 0.8287 | 0.8244 | 0.8550 | 0.9084 |
68 | UA | UA_BM25 | 0.7566 | 0.8165 | 0.7503 | 0.8268 | 0.8092 | 0.8626 | 0.9237 |
69 | HUKB | HUKB3 | 0.7512 | 0.7890 | 0.7534 | 0.8303 | 0.8550 | 0.8931 | 0.9542 |
70 | LLNTU | LLNTU0066cc | 0.6416 | 0.6743 | 0.6391 | 0.7000 | 0.7252 | 0.7405 | 0.7634 |
71 | LLNTU | LLNTUffd700** | 0.5975 | 0.3823 | 0.7121 | 0.6432 | 0.7176 | 0.7481 | 0.7939 |
72 | LLNTU | LLNTUffcc00 | 0.5798 | 0.3716 | 0.6922 | 0.6295 | 0.7023 | 0.7328 | 0.7786 |
73 | |||||||||
74 | |||||||||
75 | Task 4 | ||||||||
76 | Total teams: | 6 | |||||||
77 | Total submissions: | 18 | |||||||
78 | Team | sid | Correct | Accuracy | Total questions: | 109 | |||
79 | - | BaseLine | 58 | 0.5321 | |||||
80 | LLNTU | LLNTUmojCraw | 107 | 0.9817 | Submission not accepted as it relies on external data sources | ||||
81 | KIS | KIS2 | 74 | 0.6789 | |||||
82 | HUKB | HUKB1 | 73 | 0.6697 | |||||
83 | KIS | KIS1 | 72 | 0.6606 | |||||
84 | KIS | KIS3 | 72 | 0.6606 | |||||
85 | HUKB | HUKB2 | 69 | 0.6330 | |||||
86 | HUKB | HUKB3 | 69 | 0.6330 | |||||
87 | LLNTU | LLNTUdeNgram | 66 | 0.6055 | |||||
88 | LLNTU | LLNTUdiffSim | 63 | 0.5780 | |||||
89 | OVGU | OVGU3 | 63 | 0.5780 | |||||
90 | UA | UA_e | 59 | 0.5413 | |||||
91 | UA | UA_r | 59 | 0.5413 | |||||
92 | UA | UA_structure | 59 | 0.5413 | |||||
93 | JNLP | JNLP1 | 58 | 0.5321 | |||||
94 | JNLP | JNLP2 | 58 | 0.5321 | |||||
95 | OVGU | OVGU2 | 58 | 0.5321 | |||||
96 | JNLP | JNLP3 | 56 | 0.5138 | |||||
97 | OVGU | OVGU1 | 52 | 0.4771 |