Semantic interdisciplinary evaluation of image captioning models

Abstract

In our day-to-day life, synchronizing vision and language aspects plays a crucial role in solving various real-time challenges. Image captioning is one of them, and it aims to recognise objects, activities, and their relationships in order to provide a syntactically and semantically correct visual description. There are existing works of image captioning in various directions, such as news, fashion, art, and medical domains. The core architectural idea of image captioning is based on merging CNN, RNN, and transformer models. In practice, there are many conceivable combinations, and brute forcing all of them would take a long time. As we know, there is no work on interpreting image captioning models across various usecases. In this research article, we examine and analyze different image captioning models used across various domains, and multiple insights are extracted to determine the best combinational architecture for a new application without ignoring contextual semantics. We examined numerous designs and determined that LSTM is best for image captioning across several domains.

Keywords

• image captioning
• news articles
• interdisciplinary
• fashion images
• visually impaired people
• medical images