Compared with conventional fluorescence biomarker labeling, the classification of cell types based on their stain-free morphological characteristics enables the discovery of a new biological insight and simplifies the traditional cell analysis workflow. Most artificial intelligence aided image-based cell analysis methods primarily use transmitted bright-field images or holographic images. Here, we present the first study of the convolutional neural network (CNN) analysis on three-dimensional (3D) side-scattering cell images out of a unique 3D imaging flow cytometer study. Human cancer cell lines and leukocyte classifications were performed to investigate the information carried by the spatial distribution of side-scattering imaging of single cells. We achieved a balanced accuracy of 98.8% for cancer cell line classification and 92.3% for leukocyte classification. The results demonstrate that the side-scattering signals can not only produce general information about cell granularity following the common belief but also carry rich information about the properties and functions of cells, which can be uncovered by the availability of a side-scattering imaging flow cytometer and the application of CNN. Thereby, we have opened up a new avenue for cell phenotype analysis in biomedical and clinical research.