Research on Virtual Try-on via Style Transformation and Local Rendering

Abstract

At present, people mainly rely on traditional try-on to observe the effect of clothes, and the research on how to make clothe virtual try-on on the image of characters through the computer has gradually become a hot topic. The results of the existing methods of virtual fitting show that the details of characters are lost and the features of non-fitting areas are changed, which affect the appearance of clothes and the judgment of tryers on clothes. Aiming at the above problems, a try on method combining style transformation and local rendering was proposed. First of all, in order to ensure the accuracy of try-on, a strategy of pixel-level semantic segmentation of human image was proposed to find the location of clothing area, and at the same time provide the implementation conditions for local rendering; Second, in order to realize try-on between different clothing styles, a learnable style transformation module was constructed; Again, in order to make the clothing better fit the body shape of the tryer, a learnable clothing deformation module was proposed; Finally, in order to retain the original details of the characters and maintain the appearance of the fitting image, a local rendering strategy was proposed, which only renders the clothing area to ensure that the details of non-fitting area will not be lost. The experimental results show that the method has a significant effect in maintaining details such as characters’ arms, hands, and other aspects. In addition to fitting area, the feature information of non-fitting area is better preserved, and the image is more realistic and of fidelity after fitting. This method was applied to the try-on task for the first time, which solves the problem that try-on cannot be done correctly when the pose is complex, so that it can meet the requirements of multi-pose try-on in daily scenarios, and also provide new research ideas for virtual try-on field.

Keywords

• virtual try-on
• image fidelity
• semantic segmentation
• style transformation
• local rendering