The Yb-based compound YbPtGe with the orthorhombic ε-TiNiSi-type structure was reported as a heavy-fermion ferromagnet with the phase transition temperature, TC = 5.4 K. To investigate the magnetically ordered states in YbPtGe, we measured the elastic moduli under a magnetic field along the b-axis in addition to those along the a-axis previously reported. The temperature and field dependences of the elastic moduli show clear anomalies, indicating the phase transitions. From these results, we have drawn the magnetic phase diagram under magnetic fields along the b-axis and proposed a new phase boundary in addition to the phase boundary reported. The new phase boundary may arise from modulation of the magnetic structure.