Customizable GAN: A Method for Image Synthesis of Human Controllable

Abstract

In the research of computer vision, artificial controllability of image synthesis is a significant and challenging task. At present, there are two available methods. One is to utilize a simple contour to determine the shape of the synthetic object. This method has a promising effect, but it can only control the shape information of the synthetic object, but not the specific content. The other is to employ the text description to synthesize the corresponding image, which effectively controls the specific content of the synthesis, but it cannot do anything for the synthesized shape. In this paper, we propose a highly flexible and human customizable image synthesis model based on simple contour and natural language description, in which the specific content of contour and text description can be determined artificially. The contour determines basic synthetic object shape, and the natural language describes specific object content. Based on these, highly authentic and customizable images can be synthesized. The experiments are executed in the Caltech-UCSD Birds (CUB) and Oxford-102 flower datasets, and the experimental results demonstrate the effectiveness and superiority of our method. The results not only maintain the contour but also conform to the natural language description. Simultaneously, the high-quality image synthesis results, based on artificial hand-drawing contour and text description, are displayed to illustrate the high flexibility and customizability of our model.

Keywords

• Artificial neural networks
• computer vision
• image generation
• image processing
• text analysis
• text processing