Ultrasound-assisted directed energy deposition (UADED) is a promising technology for improving the properties of printed parts. However, process monitoring during UADED remains a challenge as ultrasound obscures the physical characteristics of DED. Here, the physical phenomena during UADED are captured using in-situ imaging and an unsupervised learning with auto-encoding is proposed to reconstruct images of the melt pool and analyse the features of spatter and plume to achieve the forming quality monitoring. This method enables effectively identifying the dynamic relationship in melt pool-spatter-plume, and the average recognition accuracy for reconstructed images reaches 94.52% during fully connected auto-encoders. Based on recognition results, the spatters during UADED are more intense, but the plume phenomenon is weakened compared to DED. The reason is that the flow mode was changed into reciprocal flow due to ultrasound. Subsequently, experiments indicated the method possessed high accuracy and robustness. The paper aims to provide a reference source for research studies on intelligent monitoring metal 3D printing.