Comparing Neural Style Transfer and Gradient-Based Algorithms in Brushstroke Rendering Tasks

Abstract

Non-photorealistic rendering (NPR) with explicit brushstroke representation is essential for both high-grade imitating of artistic paintings and generating commands for artistically skilled robots. Some algorithms for this purpose have been recently developed based on simple heuristics, e.g., using an image gradient for driving brushstroke orientation. The notable drawback of such algorithms is the impossibility of automatic learning to reproduce an individual artist’s style. In contrast, popular neural style transfer (NST) algorithms are aimed at this goal by their design. The question arises: how good is the performance of neural style transfer methods in comparison with the heuristic approaches? To answer this question, we develop a novel method for experimentally quantifying brushstroke rendering algorithms. This method is based on correlation analysis applied to histograms of six brushstroke parameters: length, orientation, straightness, number of neighboring brushstrokes (NBS-NB), number of brushstrokes with similar orientations in the neighborhood (NBS-SO), and orientation standard deviation in the neighborhood (OSD-NB). This method numerically captures similarities and differences in the distributions of brushstroke parameters and allows comparison of two NPR algorithms. We perform an investigation of the brushstrokes generated by the heuristic algorithm and the NST algorithm. The results imply that while the neural style transfer and the heuristic algorithms give rather different parameter histograms, their capabilities of mimicking individual artistic manner are limited comparably. A direct comparison of NBS-NB histograms of brushstrokes generated by these algorithms and of brushstrokes extracted from a real painting confirms this finding.

Keywords

• non-photorealistic rendering
• brushstroke rendering
• neural style transfer
• oil paintings
• postimpressionism
• realism