While hydrogen passivation has led to enhanced luminescence in many erbium doped materials, its effects on Er oxides/silicates compounds has rarely been demonstrated. Here we report effects of forming gas annealing on the luminescence properties in such Er compound materials. A broad band photoluminescence in the ultraviolet/visible range, likely arising from structural defects in the material, is significantly suppressed after forming gas annealing. Concurrently, the Er near-infrared luminescence intensity and its lifetime increase by about a factor of two and three, respectively. The samples are further characterized with Rutherford backscattering for composition information, optical absorption for optically excitable Er concentrations and extended x-ray absorption fine structures for Er local environments. We discuss the hydrogen passivation effects in the context of diffusion limited relaxation processes and suggest pathways to further improving near-infrared luminescence properties in Er compound materials.