REsponsible & Accountable Learning (REAL)
@ University of California, Santa Cruz

Leaderboard

Rank Method Accuracy Method and Paper
1 ProMix 97.65±0.19 ProMix: Combating Label Noise via Maximizing Clean Sample Utility (Code)
2 ILL 96.40±0.03 Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations (Code)
3 PLS 96.09±0.09 PLS: Robustness to label noise with two stage detection. (Code)
4 SOP 95.61±0.13 Robust Training under Label Noise by Over-parameterization. (Code)
5 CORES* 95.25±0.09 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
6 Proto-semi 95.03±0.14 Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective (Code)
7 Divide-Mix 95.01±0.71 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
8 ELR+ 94.83±0.10 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
9 PES (semi) 94.66±0.18 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
10 BKD 93.65±0.10 Blind Knowledge Distillation for Robust Image Classification. (Code)
11 ELR 92.38±0.64 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
12 CAL 91.97±0.32 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
13 Negative-LS 91.97±0.46 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
14 F-div 91.64±0.34 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
15 Positive-LS 91.57±0.07 Does Label Smoothing Mitigate Label Noise?
16 JoCoR 91.44±0.05 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
17 CORES 91.23±0.11 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
18 Co-Teaching 91.20±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
19 Peer Loss 90.75±0.25 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
20 Co-Teaching+ 90.61±0.22 How does Disagreement Help Generalization against Label Corruption?  (Code)
21 VolMinNet 89.70±0.21 Provably end-to-end label-noise learning without anchor points.  (Code)
22 T-Revision 88.52±0.17 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
23 Forward-T 88.24±0.22 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
24 Backward-T 88.13±0.29 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
25 GCE 87.85±0.70 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
26 CE 87.77±0.38
Rank Method Accuracy Method and Paper
1 ProMix 97.39±0.16 ProMix: Combating Label Noise via Maximizing Clean Sample Utility (Code)
2 ILL 96.06±0.07 Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations (Code)
3 PLS 95.86±0.26 PLS: Robustness to label noise with two stage detection. (Code)
4 Proto-semi 95.48±0.17 Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective (Code)
5 SOP 95.28±0.13 Robust Training under Label Noise by Over-parameterization. (Code)
6 Divide-Mix 95.16±0.19 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
7 PES (semi) 95.06±0.15 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
8 CORES* 94.45±0.14 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
9 ELR+ 94.43±0.41 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
10 BKD 92.98±0.17 Blind Knowledge Distillation for Robust Image Classification.  (Code)
11 ELR 91.46±0.38 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
12 CAL 90.93±0.31 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
13 Co-Teaching 90.33±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
14 JoCoR 90.30±0.20 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
15 Negative-LS 90.29±0.32 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
16 Positive-LS 89.80±0.28 Does Label Smoothing Mitigate Label Noise?
17 Co-Teaching+ 89.70±0.27 How does Disagreement Help Generalization against Label Corruption?  (Code)
18 F-div 89.70±0.40 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
19 CORES 89.66±0.32 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
20 Peer Loss 89.06±0.11 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
21 NVRM 89.00±0.05 Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting. (Code)
22 T-Revision 88.33±0.32 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
23 VolMinNet 88.30±0.12 Provably end-to-end label-noise learning without anchor points.  (Code)
24 GCE 87.61±0.28 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
25 Backward-T 87.14±0.34 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
26 Forward-T 86.88±0.50 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
27 CE 85.02±0.65
Rank Method Accuracy Method and Paper
1 ProMix 97.55±0.12 ProMix: Combating Label Noise via Maximizing Clean Sample Utility (Code)
2 ILL 95.98±0.12 Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations (Code)
3 PLS 95.96±0.16 PLS: Robustness to label noise with two stage detection. (Code)
4 Proto-semi 95.48±0.33 Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective (Code)
5 SOP 95.31±0.10 Robust Training under Label Noise by Over-parameterization. (Code)
6 Divide-Mix 95.23±0.07 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
7 PES (semi) 95.19±0.23 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
8 CORES* 94.88±0.31 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
9 ELR+ 94.20±0.24 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
10 BKD 93.00±0.08 Blind Knowledge Distillation for Robust Image Classification.  (Code)
11 ELR 91.61±0.16 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
12 CAL 90.75±0.30 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
13 Negative-LS 90.37±0.12 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
14 Co-Teaching 90.30±0.17 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
15 JoCoR 90.21±0.19 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
16 CORES 89.91±0.45 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
17 F-div 89.79±0.12 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
18 Co-Teaching+ 89.47±0.18 How does Disagreement Help Generalization against Label Corruption?  (Code)
19 Positive-LS 89.35±0.33 Does Label Smoothing Mitigate Label Noise?
20 Peer Loss 88.76±0.19 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
21 VolMinNet 88.27±0.09 Provably end-to-end label-noise learning without anchor points.  (Code)
22 T-Revision 87.71±1.02 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
23 GCE 87.70±0.56 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
24 CE 86.46±1.79
25 Backward-T 86.28±0.80 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
26 Forward-T 86.14±0.24 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
Rank Method Accuracy Method and Paper
1 ProMix 97.52±0.09 ProMix: Combating Label Noise via Maximizing Clean Sample Utility (Code)
2 ILL 96.10±0.05 Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations (Code)
3 PLS 96.10±0.07 PLS: Robustness to label noise with two stage detection. (Code)
4 Proto-semi 95.67±0.09 Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective (Code)
5 SOP 95.39±0.11 Robust Training under Label Noise by Over-parameterization. (Code)
6 PES (semi) 95.22±0.13 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
7 Divide-Mix 95.21±0.14 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
8 CORES* 94.74±0.03 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
9 ELR+ 94.34±0.22 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
10 ELR+ 92.96±0.21 Blind Knowledge Distillation for Robust Image Classification.  (Code)
11 ELR 91.41±0.44 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
12 CAL 90.74±0.24 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
13 Co-Teaching 90.15±0.18 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
14 Negative-LS 90.13±0.19 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
15 JoCoR 90.11±0.21 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
16 Positive-LS 89.82±0.14 Does Label Smoothing Mitigate Label Noise?
17 CORES 89.79±0.50 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
18 F-div 89.55±0.49 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
19 Co-Teaching+ 89.54±0.22 How does Disagreement Help Generalization against Label Corruption?  (Code)
20 Peer Loss 88.57±0.09 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
21 VolMinNet 88.19±0.41 Provably end-to-end label-noise learning without anchor points.  (Code)
22 T-Revision 87.79±0.67 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
23 GCE 87.58±0.29 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
24 Forward-T 87.04±0.35 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
25 Backward-T 86.86±0.41 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
26 CE 85.16±0.61
Rank Method Accuracy Method and Paper
1 ProMix 96.34±0.23 ProMix: Combating Label Noise via Maximizing Clean Sample Utility (Code)
2 PLS 93.78±0.30 PLS: Robustness to label noise with two stage detection. (Code)
3 ILL 93.55±0.14 Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations (Code)
4 SOP 93.24±0.21 Robust Training under Label Noise by Over-parameterization. (Code)
5 Proto-semi 92.97±0.18 Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective (Code)
6 PES (semi) 92.68±0.22 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
7 Divide-Mix 92.56±0.42 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
8 CORES* 91.66±0.09 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
9 ELR+ 91.09±1.60 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
10 BKD 87.41±0.28 Blind Knowledge Distillation for Robust Image Classification.  (Code)
11 CAL 85.36±0.16 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
12 Co-Teaching 83.83±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
13 CORES 83.60±0.53 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
14 ELR 83.58±1.13 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
15 JoCoR 83.37±0.30 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
16 Co-Teaching+ 83.26±0.17 How does Disagreement Help Generalization against Label Corruption?  (Code)
17 Negative-LS 82.99±0.36 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
18 Positive-LS 82.76±0.53 Does Label Smoothing Mitigate Label Noise?
19 F-div 82.53±0.52 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
20 Peer Loss 82.53±0.52 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
21 NVRM 81.19±0.05 Artifical Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting. (Code)
22 GCE 80.66±0.35 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
23 VolMinNet 80.53±0.20 Provably end-to-end label-noise learning without anchor points.  (Code)
24 T-Revision 80.48±1.20 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
25 Forward-T 79.79±0.46 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
26 CE 77.69±1.55
27 Backward-T 77.61±1.05 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
Rank Method Accuracy Method and Paper
1 ProMix 73.79±0.28 ProMix: Combating Label Noise via Maximizing Clean Sample Utility (Code)
2 PLS 73.25±0.12 PLS: Robustness to label noise with two stage detection.  (Code)
3 Divide-Mix 71.13±0.48 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
4 PES (semi) 70.36±0.33 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
5 ILL 68.07±0.33 Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations (Code)
6 SOP 67.81±0.23 Robust Training under Label Noise by Over-parameterization. (Code)
7 Proto-semi 67.73±0.22 Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective (Code)
8 ELR+ 66.72±0.07 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
9 CAL 61.73±0.42 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
10 CORES 61.15±0.73 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
11 Co-Teaching 60.37±0.27 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
12 JoCoR 59.97±0.24 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
13 ELR 58.94±0.92 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
14 Negative-LS 58.59±0.98 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
15 Co-Teaching+ 57.88±0.24 How does Disagreement Help Generalization against Label Corruption?  (Code)
16 VolMinNet 57.80±0.31 Provably end-to-end label-noise learning without anchor points.  (Code)
17 Peer Loss 57.59±0.61 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
18 Backward-T 57.14±0.92 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
19 F-div 57.10±0.65 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
20 Forward-T 57.01±1.03 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
21 GCE 56.73±0.30 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
22 Positive-LS 55.84±0.48 Does Label Smoothing Mitigate Label Noise?
23 CORES* 55.72±0.42 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
24 CE 55.50±0.66
25 T-Revision 51.55±0.31 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)

Contact

Please contact us via {yangliu, jiahengwei, zwzhu, haocheng}@ucsc.edu， if you want to add your methods onto the leaderboard.
We are also maintaining a curated list of most recent papers & codes in learning with noisy labels here.