Balinese Script Handwriting Recognition Using Faster R-CNN

Abstract

In Balinese culture, the ability to read Balinese script is one of the challenges young generations face. Advances in machine learning have proposed handwriting detection systems using both traditional and deep learning models. However, the traditional approach is usually impractical and is prone to inaccurate identification results. Convolutional neural network (CNN)-based models integrate feature extraction and classification into an end-to-end pipeline to increase performance. This research proposes that recognizing characters through an object detection approach makes an end-to-end process of localizing and classifying several characters simultaneously using the Faster R-CNN. Four CNN models, including ResNet-50, ResNet-101, ResNet-152, and Inception ResNet V2, were tested to detect 28 Balinese characters in a single form that covers 18 consonants and 10 digits using Intersection over Union (IoU) thresholds: 0.5 and 0.75. ResNet-50 and Inception ResNet V2 achieve 0.991 mAP at IoU of 0.5, while Inception ResNet V2 excels at IoU of 0.75. Further analysis showed that the class ‘nol’ had the lowest Recall due to many undetected ground truths. Meanwhile, class ‘ba’ had the lowest Precision due to its similarity to classes “ga” and “nga”. This research contributes to the experiment with Faster R-CNN in detecting handwritten Balinese scripts.

Keywords

• balinese script
• faster r-cnn
• handwriting
• recognition