In this paper, we describe the relevance of diagnosing the lining condition of steel ladles in metallurgical facilities. Accidents with steel ladles lead to losses and different types of damage in iron and steel works. We developed an algorithm for recognizing thermograms of steel ladles to identify burnout zones in the lining based on the technology and design of neural networks. A diagnostic system structure for automated evaluating of the technical conditions of steel ladles without taking them out of service has been developed and described.