Four‐branch Siamese network based on sketch‐specific data augmentation for sketch recognition

Abstract

Abstract The shortened abstract is as follows: Sketch recognition has become an important hotspot issue because of sketch's intuitiveness and visualization. The existing sketch‐recognition methods based on handcrafted features and deep features are insufficient in the recognition of the local information of sketches, and the recognition accuracy is not ideal. Accordingly, this paper proposes a four‐branch Siamese network based on sketch‐specific data augmentation to generate discriminative feature representations and improve the sketch‐recognition accuracy. A sketch is an ordered list of strokes, we adopt the semantic information of strokes as the decomposition criteria to divide a sketch into three disjoint local blocks, and then combine the local blocks in pairs to form three new sketches. In order to give full play to the positive effect of local blocks on category prediction and enhance the fine‐grained capability of the network, three newly generated sketches and the original sketch are combined to construct a four‐branch Siamese network. Each branch network adopts the Sketch‐A‐Net architecture with the fully connected layer removed as the basic network, and we improve it by adding shortcut connection layer and multi‐scale weighted bilinear coding (MWBC) modules. Compared with the state‐of‐the‐art methods, the experimental results on the TU‐Berlin dataset demonstrate the excellent performance of our model.

Keywords

• data augmentation
• deep convolutional neural networks
• sketch recognition