Understanding the performance of polymer dielectrics at different temperatures is becoming increasingly important due to the rapid development of electric cars, electromagnetic devices, and new energy production solutions. Cyclic olefin copolymers (COCs) are an attractive material due to their low water absorption, good electrical insulation, long-term stability of surface treatments, and resistance to a wide range of acids and solvents. This work focused on the dielectric and electrical properties of cyclic olefin copolymer (COC)/Al2O3 composites over a wide range of temperature and frequency domains, from room temperature to cryogenic temperatures (around 125 K). Permittivity, electrical conductivity, and electrical modulus are given consideration. A composite of up to 50% Al2O3 mixed with COC was prepared via a conventional melt-blending method. The final samples were formed in sheets and processed using injection and extrusion moldings. It was found that formulations with Al2O3 concentrations ranging from 10 to 50% resulted in higher electrical conductivity while maintaining the viscosity of the composite at a level acceptable for polymer-processing machinery. Our data show that COC/alumina composites present substantial potential as materials for high-frequency applications, even at the regime of cryogenic temperatures.