Low-alloy 42CrMo4 steels were studied by 57Fe Mössbauer spectroscopy (MS), X-ray diffractometry (XRD), and Energy Dispersive X-ray Spectroscopy (EDS) measurements. The investigations were performed on metallographic samples, which were subjected to a series of successive grinding and polishing with a progressively finer grit. Conversion X-ray Mössbauer spectroscopy (CXMS) was used to determine the occurrence of austenite in steel samples. It is a unique method detecting the austenite content very sensitively. Six samples with different surface preparation were investigated, starting with 4.8% of austenite on an as-cut sample, and a large decrease in the retained austenite to 2.6% was observed after the first grinding of a hardened cut sample. Additionally, an unexpectedly large decrease in the austenite content to 2.3% was found due to the final polishing. A second time applied successive grinding and polishing of all samples resulted in identical austenite content determined by CXMS of approx. 5%, which proved the applicability of the CXMS method. Generally, the result calls attention to the importance of preparation of metallurgical samples by grinding and polishing where the results can vary significantly on the level of surface processing.