Understanding emotions is one of the greatest capabilities of human beings, as it allows the understanding of facial expressions that facilitate the capture of important information about other individuals, which are used for the perception of mental or emotional states. Advances in Artificial Intelligence and Visual Computing, more specifically in Deep Learning with the advent of Artificial Neural Networks, have enhanced the ability of machines to infer human emotions through image analysis. This paper presents a Systematic Literature Review (SLR) with the purpose of researching, mapping and summarizing studies that address techniques or algorithms more efficiently. The convolutional neural network models analyzed in this review are based on deep learning with an emphasis on expression and microexpression recognition. The results suggest that database uses, with laboratory controlled images, combined with CNN’s such as VGG and ResNet, have excellent performances in their tests. For better understanding, we will detail and compare all the methods obtained in the review.