Multiple auxiliary classifiers GAN for controllable image generation: Application to license plate recognition

Abstract

Abstract One of the main challenges in developing machine learning (ML) applications is the lack of labeled and balanced datasets. In the literature, different techniques tackle this problem via augmentation, rendering, and over‐sampling. Still, these methods produce datasets that appear less natural, exhibit poor balance, and have less variation. One potential solution is to leverage the Generative Adversarial Network (GAN) which achieves remarkable results in the generation of high‐fidelity natural images. However, expanding the ability of GANs' to control generated image attributes with supervisory information remains a challenge. This research aims to propose an efficient method to generate high‐fidelity natural images with total control of its main attributes. Therefore, this paper proposes a novel Multiple Auxiliary Classifiers GAN (MAC‐GAN) framework based on Auxiliary Classifier GAN (AC‐GAN), multi‐conditioning, Wasserstein distance, gradient penalty, and dynamic loss. It is therefore presented as an efficient solution for highly controllable image synthesis red that allows to enrich and re‐balance datasets beyond data augmentation. Furthermore, the effectiveness of MAC‐GAN images on a target ML application called Automatic License Plate Recognition (ALPR) under limited resource constraints is probed. The improvement achieved is over 5% accuracy, which is mainly due to the ability of the MAC‐GAN to create a balanced dataset with controllable synthesis and produce multiple (different) images with the same attributes, thus increasing the variation of the dataset in a more elaborate way than data augmentation techniques.