So Shiho Tang (SSHT) is a traditional herbal medicine commonly used in Asian countries. This study evaluated the anti-inflammatory effect of SSHT and the associated mechanism using lipopolysaccharide (LPS)-stimulated RAW 264.7 macrophages and murine dextran sodium sulfate (DSS)-induced ulcerative colitis models. Pre-treatment of RAW 264.7 macrophages with SSHT significantly reduced LPS-induced inflammation by decreasing nitrite production and regulating the mitogen-activated protein kinase pathway. Meanwhile, in mice, DSS-induced colitis symptoms, including colon shortening and body weight loss, were attenuated by SSHT. Moreover, representative compounds of SSHT, including glycyrrhizic acid, ginsenoside Rb1, baicalin, saikosaponin A, and saikosaponin B2, were quantified, and their effects on nitrite production were measured. A potential anti-inflammatory effect was detected in LPS-induced RAW 264.7 cells. Our findings suggest that SSHT is a promising anti-inflammatory agent. Its representative components, including saikosaponin B2, ginsenoside Rb1, and baicalin, may represent the key active compounds responsible for eliciting the anti-inflammatory effects and can, therefore, serve as quality control markers in SSHT preparations.