Device-to-Device (D2D) communication is a promising method for the emerging Internet of Things. Secure information exchange plays a key role in the application of D2D communication. Considering that the wireless devices are powered by batteries, in this paper, a lightweight secure D2D system is designed by using multiple sensors on mobile devices. Specifically, by leveraging an acceleration sensor equipped in two wireless devices, a lightweight and efficient key distribution scheme for secure D2D communication is proposed. Based on the distributed secure key, an efficient near-field authentication is developed with a speaker and a microphone to determine whether these two devices are physically close; and a secure information exchange scheme with high efficiency, which includes message encryption/decryption and message authentication, is presented over the audio channel and the RF channel. The Extensive experiments are provided to demonstrate that our system can achieve a secure information exchange between two wireless devices with low energy consumption and computing resources.