Copy‐paste with self‐adaptation: A self‐adaptive adjustment method based on copy‐paste augmentation

Abstract

Abstract Data augmentation diversifies the information in the dataset. For class imbalance, the copy‐paste augmentation generates new class information to alleviate the impact of this problem. However, these methods rely excessively on human intuition. Over‐fitting or under‐fitting can occur while adding the class information, which is inappropriate. The authors propose a self‐adaptive data augmentation: the copy‐paste with self‐adaptation (CPA) algorithm, which improves the phenomenon of over‐fitting and under‐fitting. For the CPA, the evaluation results of a model are taken as an important adjustment basis. The evaluation results are combined with the information of class imbalance to generate a set of class weights. Different number of class information will be replenished according to class weights. Finally, the generated images will be inserted into the training dataset and the model will start formal training. The experimental results show that CPA can alleviate class imbalance. For TT100 K dataset, YOLOv3 is trained with the optimised dataset and its AP is increased by 2% for VOC2007 dataset, the mAP of RetinaNet on optimised dataset is 78.46, which is 1.2% higher than original dataset. For COCO2017 dataset, SSD300 is trained with the optimised dataset and its AP is increased by 1.3%.

Keywords

• image enhancement
• object detection
• object recognition