In this paper, we studied the optimized conditions for adding inorganic quantum dots (QD) to the P3HT:PC70BM organic active layer to increase the sensitivity of the indirect X-ray detector. Commonly used QDs are composed of hazardous substances with environmental problems, so indium phosphide (InP) QDs were selected as the electron acceptor in this experiment. Among the three different sizes of InP QDs (4, 8, and 12 nm in diameter), the detector with 4 nm InP QDs showed the highest sensitivity, of 2.01 mA/Gy·cm2. To further improve the sensitivity, the QDs were fixed to 4 nm in diameter and then the amount of QDs added to the organic active layer was changed from 0 to 5 mg. The highest sensitivity, of 2.26 mA/Gy·cm2, was obtained from the detector with a P3HT:PC70BM:InP QDs (1 mg) active layer. In addition, the highest mobility, of 1.69 × 10−5 cm2/V·s, was obtained from the same detector. Compared to the detector with the pristine P3HT:PC70BM active layer, the detector with a P3HT:PC70BM:InP QDs (1 mg) active layer had sensitivity that was 61.87% higher. The cut-off frequency of the P3HT:PC70BM detector was 21.54 kHz, and that of the P3HT:PC70BM:InP QDs (1 mg) detector was 26.33 kHz, which was improved by 22.24%.