[Live Machine Learning Research] Plain Self-Ensembles (I actually DISCOVER SOMETHING) - Part 1

I share my progress of implementing a research idea from scratch. I attempt to build an ensemble model out of students of label-free self-distillation without any additional data or augmentation. Turns out, it actually works, and interestingly, the more students I employ, the better the accuracy. This leads to the hypothesis that the ensemble effect is not a process of extracting more information from labels. OUTLINE: 0:00 - Introduction 2:10 - Research Idea 4:15 - Adjusting the Codebase 25:00 - Teacher and Student Models 52:30 - Shipping to the Server 1:03:40 - Results 1:14:50 - Conclusion Code: https://ift.tt/3f5JHgi References: My Video on SimCLRv2: https://youtu.be/2lkUNDZld-4 Born-Again Neural Networks: https://ift.tt/2k3fMKN Deep Ensembles: A Loss Landscape Perspective: https://ift.tt/2RwurQL Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C