Neural Topic Generation Utilizing Attention Mechanisms With Transformer‐Based Embeddings for Root‐Cause Analysis of Manufacturing Defects in Electronic Products

Abstract

ABSTRACT Root‐cause analysis (RCA) is a critical process for identifying and mitigating manufacturing defects, particularly in the electronics industry, where minor issues can lead to significant operational and financial consequences. Traditional approaches to RCA, relying heavily on manual inspections or rule‐based systems, often fail to scale with the growing complexity and volume of defect‐related data. This study introduces a neural topic generation model that leverages transformer‐based embeddings from BART and T5 to automatically identify latent topics representing defect patterns, providing actionable insights into their root causes. By integrating these models with attention mechanisms and a VAE, the model effectively handles unstructured textual data, generating interpretable and coherent topics. The performance of the proposed models is compared with traditional NTM models, such as NTM with Word2Vec and VAE‐based NTM. Experimental results show that NTM‐BART achieved the highest coherence score (0.468) and a low perplexity score (124), while NTM‐T5 achieved the lowest perplexity score (119) and a coherence score of 0.434. In contrast, traditional NTM models exhibited significantly lower coherence scores (0.296 for NTM with Word2Vec) and higher perplexity (207 for VAE‐based NTM), underscoring their limitations. These findings highlight the ability of BART and T5 to generate coherent and interpretable topics, making them highly effective tools for RCA in complex manufacturing environments. The study emphasizes the transformative potential of advanced NLP techniques in industrial applications, paving the way for smarter, more efficient manufacturing systems.

Keywords

• attention mechanisms
• BART
• manufacturing defects
• natural language processing
• neural topic modeling
• root‐cause analysis