Design and Verification of Secure Mutual Authentication Protocols for Mobile Multihop Relay WiMAX Networks against Rogue Base/Relay Stations

Abstract

Mobile multihop relay (MMR) WiMAX networks have attracted lots of interest in the wireless communication industry recently because of its scalable coverage, improved data rates, and relatively low cost. However, security of MMR WiMAX networks is the main challenge to be addressed. In this paper, we first identify several possible attacks on MMR WiMAX networks in which a rogue base station (BS) or relay station (RS) can get authenticated and gain control over the connections and show that the current standard does not address this problem well. We then propose a set of new authentication protocols for protecting MMR WiMAX networks from rogue BS attack, rogue RS attack, and suppress-replay attack. Our protocols can provide centralized authentication by using a trusted authentication server to support mutual authentication between RS and BS, between RS and RS, and between mobile station (MS) and RS. Moreover, our protocols can also provide distributed authentication with a license issued by the trusted server. We use a formal tool called Scyther to analyze and verify the security properties of our protocols. The results show that our protocols can counter rogue BS and RS attacks and suppress-replay attack and are not susceptible to any known attacks.