TUNIT: Rethinking the Truly Unsupervised Image-to-Image Translation (Paper Explained)

Image-to-Image translation usually requires corresponding samples or at least domain labels of the dataset. This paper removes that restriction and allows for fully unsupervised image translation of a source image to the style of one or many reference images. This is achieved by jointly training a guiding network that provides style information and pseudo-labels. OUTLINE: 0:00 - Intro & Overview 1:20 - Unsupervised Image-to-Image Translation 7:05 - Architecture Overview 14:15 - Pseudo-Label Loss 19:30 - Encoder Style Contrastive Loss 25:30 - Adversarial Loss 31:20 - Generator Style Contrastive Loss 35:15 - Image Reconstruction Loss 36:55 - Architecture Recap 39:55 - Full Loss 42:05 - Experiments Paper: https://ift.tt/3e2yM6P Code: https://ift.tt/2Y7j8kR Abstract: Every recent image-to-image translation model uses either image-level (i.e. input-output pairs) or set-level (i.e. domain labels) supervision at minimum. However, even the set-level supervision can be a serious bottleneck for data collection in practice. In this paper, we tackle image-to-image translation in a fully unsupervised setting, i.e., neither paired images nor domain labels. To this end, we propose the truly unsupervised image-to-image translation method (TUNIT) that simultaneously learns to separate image domains via an information-theoretic approach and generate corresponding images using the estimated domain labels. Experimental results on various datasets show that the proposed method successfully separates domains and translates images across those domains. In addition, our model outperforms existing set-level supervised methods under a semi-supervised setting, where a subset of domain labels is provided. The source code is available at this https URL Authors: Kyungjune Baek, Yunjey Choi, Youngjung Uh, Jaejun Yoo, Hyunjung Shim Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB