REsponsible & Accountable Learning (REAL)
@ University of California, Santa Cruz

Leaderboard

Rank Method Accuracy Method and Paper
1 SOP 95.61±0.13 Robust Training under Label Noise by Over-parameterization. (Code)
2 CORES* 95.25±0.09 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
3 Divide-Mix 95.01 ± 0.71 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
4 ELR+ 94.83±0.10 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
5 PES (semi) 94.66±0.18 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
6 ELR 92.38±0.64 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
7 CAL 91.97±0.32 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
8 Negative-LS 91.97±0.46 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
9 F-div 91.64±0.34 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
10 Positive-LS 91.57±0.07 Does Label Smoothing Mitigate Label Noise?
11 JoCoR 91.44±0.05 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
12 CORES 91.23±0.11 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
13 Co-Teaching 91.20±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
14 Peer Loss 90.75±0.25 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
15 Co-Teaching+ 90.61±0.22 How does Disagreement Help Generalization against Label Corruption?  (Code)
16 VolMinNet 89.70±0.21 Provably end-to-end label-noise learning without anchor points.  (Code)
17 T-Revision 88.52±0.17 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
18 Forward-T 88.24±0.22 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
19 Backward-T 88.13±0.29 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
20 GCE 87.85±0.70 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
21 CE 87.77±0.38
Rank Method Accuracy Method and Paper
1 SOP 95.28±0.13 Robust Training under Label Noise by Over-parameterization. (Code)
2 Divide-Mix 95.16±0.19 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
3 PES (semi) 95.06±0.15 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
4 CORES* 94.45±0.14 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
5 ELR+ 94.43±0.41 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 ELR 91.46±0.38 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
7 CAL 90.93±0.31 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
8 Co-Teaching 90.33±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
9 JoCoR 90.30±0.20 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
10 Negative-LS 90.29±0.32 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
11 Positive-LS 89.80±0.28 Does Label Smoothing Mitigate Label Noise?
12 Co-Teaching+ 89.70±0.27 How does Disagreement Help Generalization against Label Corruption?  (Code)
13 F-div 89.70±0.40 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
14 CORES 89.66±0.32 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
15 Peer Loss 89.06±0.11 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
16 NVRM 89.00±0.05 Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting. (Code)
17 T-Revision 88.33±0.32 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
18 VolMinNet 88.30±0.12 Provably end-to-end label-noise learning without anchor points.  (Code)
19 GCE 87.61±0.28 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
20 Backward-T 87.14±0.34 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
21 Forward-T 86.88±0.50 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
22 CE 85.02±0.65
Rank Method Accuracy Method and Paper
1 SOP 95.31±0.10 Robust Training under Label Noise by Over-parameterization. (Code)
2 Divide-Mix 95.23±0.07 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
3 PES (semi) 95.19±0.23 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
4 CORES* 94.88±0.31 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
5 ELR+ 94.20±0.24 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 ELR 91.61±0.16 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
7 CAL 90.75±0.30 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
8 Negative-LS 90.37±0.12 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
9 Co-Teaching 90.30±0.17 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
10 JoCoR 90.21±0.19 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
11 CORES 89.91±0.45 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
12 F-div 89.79±0.12 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
13 Co-Teaching+ 89.47±0.18 How does Disagreement Help Generalization against Label Corruption?  (Code)
14 Positive-LS 89.35±0.33 Does Label Smoothing Mitigate Label Noise?
15 Peer Loss 88.76±0.19 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
16 VolMinNet 88.27±0.09 Provably end-to-end label-noise learning without anchor points.  (Code)
17 T-Revision 87.71±1.02 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
18 GCE 87.70±0.56 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
19 CE 86.46±1.79
20 Backward-T 86.28±0.80 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
21 Forward-T 86.14±0.24 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
Rank Method Accuracy Method and Paper
1 SOP 95.39±0.11 Robust Training under Label Noise by Over-parameterization. (Code)
2 PES (semi) 95.22±0.13 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
3 Divide-Mix 95.21±0.14 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
4 CORES* 94.74±0.03 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
5 ELR+ 94.34±0.22 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 ELR 91.41±0.44 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
7 CAL 90.74±0.24 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
8 Co-Teaching 90.15±0.18 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
9 Negative-LS 90.13±0.19 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
10 JoCoR 90.11±0.21 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
11 Positive-LS 89.82±0.14 Does Label Smoothing Mitigate Label Noise?
12 CORES 89.79±0.50 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
13 F-div 89.55±0.49 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
14 Co-Teaching+ 89.54±0.22 How does Disagreement Help Generalization against Label Corruption?  (Code)
15 Peer Loss 88.57±0.09 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
16 VolMinNet 88.19±0.41 Provably end-to-end label-noise learning without anchor points.  (Code)
17 T-Revision 87.79±0.67 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
18 GCE 87.58±0.29 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
19 Forward-T 87.04±0.35 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
20 Backward-T 86.86±0.41 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
21 CE 85.16±0.61
Rank Method Accuracy Method and Paper
1 SOP 93.24±0.21 Robust Training under Label Noise by Over-parameterization. (Code)
2 PES (semi) 92.68±0.22 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
3 Divide-Mix 92.56±0.42 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
4 CORES* 91.66±0.09 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
5 ELR+ 91.09±1.60 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
6 CAL 85.36±0.16 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
7 Co-Teaching 83.83±0.13 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
8 CORES 83.60±0.53 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
9 ELR 83.58±1.13 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
10 JoCoR 83.37±0.30 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
11 Co-Teaching+ 83.26±0.17 How does Disagreement Help Generalization against Label Corruption?  (Code)
12 Negative-LS 82.99±0.36 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
13 Positive-LS 82.76±0.53 Does Label Smoothing Mitigate Label Noise?
14 F-div 82.53±0.52 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
15 Peer Loss 82.53±0.52 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
16 NVRM 81.19±0.05 Artifical Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting. (Code)
17 GCE 80.66±0.35 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
18 VolMinNet 80.53±0.20 Provably end-to-end label-noise learning without anchor points.  (Code)
19 T-Revision 80.48±1.20 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)
20 Forward-T 79.79±0.46 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
21 CE 77.69±1.55
22 Backward-T 77.61±1.05 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
Rank Method Accuracy Method and Paper
1 Divide-Mix 71.13±0.48 Dividemix: Learning with noisy labels as semi-supervised learning.  (Code)
2 PES (semi) 70.36±0.33 Understanding and Improving Early Stopping for Learning with Noisy Labels. (Code)
3 SOP 67.81±0.23 Robust Training under Label Noise by Over-parameterization. (Code)
4 ELR+ 66.72±0.07 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
5 CAL 61.73±0.42 A Second-Order Approach to Learning with Instance-Dependent Label Noise.  (Code)
6 CORES 61.15±0.73 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
7 Co-Teaching 60.37±0.27 Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels.  (Code)
8 JoCoR 59.97±0.24 Combating noisy labels by agreement: A joint training method with co-regularization.  (Code)
9 ELR 58.94±0.92 Early-Learning Regularization Prevents Memorization of Noisy Labels.  (Code)
10 Negative-LS 58.59±0.98 Understanding Generalized Label Smoothing when Learning with Noisy Labels.
11 Co-Teaching+ 57.88±0.24 How does Disagreement Help Generalization against Label Corruption?  (Code)
12 VolMinNet 57.80±0.31 Provably end-to-end label-noise learning without anchor points.  (Code)
13 Peer Loss 57.59±0.61 Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates.  (Code)
14 Backward-T 57.14±0.92 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
15 F-div 57.10±0.65 When Optimizing f-Divergence is Robust with Label Noise?  (Code)
16 Forward-T 57.01±1.03 Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach.  (Code)
17 GCE 56.73±0.30 Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.  (Code)
18 Positive-LS 55.84±0.48 Does Label Smoothing Mitigate Label Noise?
19 CORES* 55.72±0.42 Learning with Instance-Dependent Label Noise: A Sample Sieve Approach.  (Code)
20 CE 55.50±0.66
21 T-Revision 51.55±0.31 Are Anchor Points Really Indispensable in Label-Noise Learning?  (Code)

Contact

Please contact us via {yangliu, jiahengwei, zwzhu, haocheng}@ucsc.edu， if you want to add your methods onto the leaderboard.
We are also maintaining a curated list of most recent papers & codes in learning with noisy labels here.