Deep learning (DL) has attracted increasing attention on account of its significant processing power in tasks, such as speech, image, or text processing. In order to the exponential development and widespread availability of digital social media (SM), analyzing these data using traditional tools and technologies is tough or even intractable. DL is found as an appropriate solution to this problem. In this paper, we keenly discuss the practiced DL architectures by presenting a taxonomy-oriented summary, following the major efforts made toward the SM analytics (SMA). Nevertheless, instead of the technical description, this paper emphasis on describing the SMA-oriented problems with the DL-based solutions. To this end, we also highlight the DL research challenges (such as scalability, heterogeneity, and multimodality) and future trends.