Spontaneous parametric down-conversion (SPDC) has become a key method for generating entangled photon pairs. Periodically poled thin-film lithium niobate (TFLN) waveguides induce strong SPDC but require complex fabrication processes. In this work, we experimentally demonstrate efficient SPDC and second harmonic generation using modal phase matching methods. This is achieved with inverse-designed optical mode converters and low-loss optical waveguides in a single nanofabrication process. Inverse design methods provide enhanced functionalities and compact footprints for the converter. Despite the extensive achievements in inverse-designed photonic integrated circuits, the potential of inverse-designed TFLN quantum photonic devices has been seldom explored. The device shows an on-chip conversion efficiency of 3.95% W−1 cm−2 in second harmonic generation measurements and a coincidence count rate up to 21.2 kHz in SPDC experiments. This work highlights the potential of the inverse-designed TFLN photonic devices and paves the way for their applications in on-chip nonlinear or quantum optics.