Nuclei classification is a mandatory process to obtain scoring information for whole slide images (WSIs). In immunohistochemistry (IHC) staining specifically for estrogen receptor (ER) biomarker, an Allred score based on the proportion and intensity of cancer nuclear staining is widely used in histopathology practice to predict response to hormonal treatment. This manually exhaustive process can be accelerated with the help of computational intelligence. In this article, we present a thorough analysis of 37 WSIs of breast cancer cases with over 2.8 million segmented nuclei. ER-stained nuclei were classified into negative, weak, moderate and strong intensities using DenseNet deep learning architecture, contributing to Allred scoring. Seven different models and configurations were exhaustively analysed in six tests to obtain the scoring reaching the best concordance of 56.8% and 81.1% with the pathologist’s manual score and suggested hormonal treatment. We also discussed in detail the causes that lead to the non-concordances. This study follows the pathologists’ workflow in obtaining the Allred score but is fully automated. It provides a basis for the development of more complex deep learning models, particularly for nuclei classification and achieving accurate scoring of ER-IHC stained WSIs.