Refining Line Art From Stroke Style Disentanglement With Diffusion Models

Abstract

A beginner who wants to create illustrations has difficulty improving his/her ability without expert advice. Especially in the initial steps, line drawings are critical but hard to evaluate because there are many assessment points, such as shape, variation in thickness, stroke fluency, and shadow expression. Moreover, there is no well-summarized line art dataset based on expert knowledge to support skill refinement. Furthermore, the evaluation criterion is always subjective. To solve this problem, we custom-build systematized line artworks formed by cataloged stroke styles and propose a machine learning method that can automatically give clues to refining the artworks. We request 10 professional-level artists to create line art in six patterns; the stroke styles of the images are systematically summarized. Using this specific dataset, we train an auxiliary classifier to identify and remove features of those patterns to refine all line artwork commonly. We also implement an enhancement step that uses diffusion models to add more informative details to the generated results. The proposed method can automatically identify where strokes are needed to change and generate high-quality refined versions. Our method performs better than the previous method regarding L2, lpips, and SSIM scores while giving specialized clues to different stroke styles.

Keywords

• Disentangled representation
• image generation
• line art refinement
• denoising diffusion probabilistic models