Waste material classification is critical for efficient recycling and waste management. This study proposes a novel, low-cost material classification system based on a single, voltage-tunable Ge-on-Si photodetector operating across the visible and short-wave infrared (SWIR) spectral regions. Thanks to its tunability, the sensor is able to extract spectral information, and the system effectively distinguishes between seven different materials, including plastics, aluminum, glass, and paper. The system operates with a broadband illuminator, and material identification is obtained through the processing of the photocurrent signal at different bias voltages with classification algorithms. Here, we demonstrate the basic system functionality and near real-time classification of different waste materials.