Unsupervised content and style learning for multimodal cross-domain image translation

Abstract

Abstract Recently, impressive progress has been made in cross-domain image translation using image generation models pre-trained on massive amounts of data since these pre-trained generative models have strong generative capabilities.However, due to the fact that these pre-trained generative models are usually generated based on the principle of diffusion, there are often the following issues, such as color pattern distortion and the inability to maintain content structure during the translation process. Existing methods fail to meet the requirements of cross-domain image translation, where the shape (content structure) of the image should remain unchanged or undergo minimal change. To address these challenges, we propose an unsupervised content and style learning method for cross-domain image translation. Our approach utilizes self-structure attention loss to preserve content structure and color constraint loss to map the color space of the reference image to the translated image. This effectively separates the image’s content structure from its color style, enabling high-quality multi-modal cross-domain image translation. Experiments on multiple datasets demonstrate that our method outperforms state-of-the-art approaches in metrics such as LPIPS, NDB, JS, IS, and excels in maintaining content structure and color patterns.

Keywords

• Diversity
• Cross-domain image translation
• Multi-modal
• Nnsupervised