Deep handwritten diagram segmentation

Abstract

Abstract Handwriting is a natural way to communicate and exchange ideas, but converting handwritten diagrams to application‐specific digital formats requires skill and time. Automatic handwritten document conversion can save time, but diagrams and text require different recognition engines. Since accurate segmentation of handwritten diagrams can improve the accuracy of later diagram recognition steps, the authors propose to solve the problem of segmentation of text and non‐text elements of handwritten diagrams using deep semantic segmentation. The model, DeepDP is a flexible U‐net style architecture that can be tuned in complexity to a level appropriate for a particular dataset and diagram type. Experiments on a public hand‐drawn flowchart dataset and a business process diagram dataset show excellent performance, with a per pixel accuracy of 98.6% on the public flowchart datasets and improvement over the 99.3% text stroke accuracy and 96.6% non‐text stroke accuracy obtained by state of the art methods that use online stroke information. On the smaller offline business process diagram dataset, the method obtains a per‐pixel accuracy of 96.9%.

Keywords

• convolutional neural nets
• deep learning
• document image processing
• handwritten diagrams
• image segmentation
• U‐net