A novel non-magnetic Fe-21Cr-15Ni-5Mn-2Mo austenitic stainless steel with high strength and plasticity has been developed. The microstructure and liquid helium temperature (4.2 K) tensile properties of the top and bottom samples of large-size forged flat steel after solution treatment at 1090 °C were investigated. The results showed that the average grain size of the bottom sample (48.0 ± 6.7 μm) was smaller than that of the top sample (58.8 ± 15.3 μm), and the MX precipitates and Z phases were distributed in the matrix of the samples. The 4.2 K strengths of the samples at the top and bottom were high, and large amounts of annealing twin boundaries played a certain role in strengthening. After cryogenic tensile testing, large amounts of deformation twins, stacking faults, and dislocations were generated inside the austenite grains of both samples, which helped the material to obtain higher plasticity and strength. The top and bottom samples possessed excellent synergies of strength and plasticity at 4.2 K, and the 4.2 K tensile properties of the top sample were as follows: ultimate tensile strength (UTS) of 1850 MPa, yield strength (YS) of 1363 MPa, and elongation (EL) of 26%. The tested steel is thus believed to meet the requirements of combined excellent strength and plasticity within a deep cryogenic environment, and it would be a promising material candidate for manufacturing superconducting coil cases to serve in new generation fusion engineering.