CrSBr is a recently discovered two-dimensional anti-ferromagnet. It has attracted much attention due to its superior properties for potential optoelectronic and spintronic applications. However, its complex refractive index with layer dependence has not been systematically studied yet. In this work, we studied the room-temperature complex refractive indices of thin CrSBr flakes of different thicknesses in the visible light range. Using micro-reflectance spectroscopy, we measured the optical contrast of thin CrSBr flakes with respect to different substrates. The complex refractive index was extracted by modeling the optical contrast with the Fresnel equations. We extracted the band gap values of CrSBr in the few-layer limit. We determined the band gaps for monolayer, bilayer, and trilayer CrSBr to be 1.88 eV, 1.81 eV, and 1.77 eV, respectively. As a comparison, the band gap for multilayer CrSBr is outside our measured range, that is, below 1.55 eV. Our results suggest that the band gap of CrSBr decreases as thickness increases.