Abstract Background Duodenal adenocarcinoma (DA) is a rare yet aggressive malignancy, with increasing incidence in the last decades. Its low frequency has hampered a thorough understanding of the pathogenesis of the disease and of its biology, limiting the identification of tailored therapeutic options. A large body of evidence has clearly shown the clinical relevance of immune cells in solid tumors, correlating immune features with post-surgical prognosis. The aim of this study was to analyze the immune contexture in a cohort of duodenal adenocarcinomas surgically resected at our Institution and define its correlation with clinical variables. Methods Tissue slides from paraffin-embedded tumor specimens of 15 consecutive DA and 3 adenomas that underwent a pancreaticoduodenectomy in our center between 2010 to 2018 were immunohistochemically stained. The density (percentage of immune reactive area, IRA%) of immune markers CD45RO, CD8, CD20, IL-17, PD-1, CD68 was quantified by computer-assisted image analysis. Demographic, clinical, histopathological data were collected. Results In our population, median IRA % (IQR) of immune subsets was respectively CD45RO-TILs 2.19 (2.14), CD8-TIL 0.42 (0.81), CD20-TILs 0.22 (0.51), CD20-TLT 2.84 (4.64), CD68-TAM 2.19 (1.56), IL17+ cells 0.39 (0.39), PD1-TILs 0.19 (0.41). The median follow-up was 47.5 (22.4–63.3) months. At statistical analysis, the density of CD8-TILs inversely correlated with lymph node ratio (p = 0.013), number of metastatic lymph nodes (p = 0.019), and was lower in N+ adenocarcinomas compared to N0 (1.07 vs 0.29; p = 0.093), albeit not significantly. Stratifying patients for the N status, the density of CD8-TILs decreased with the increasing of the N stage (p = 0.065) and was lower in patients who experienced recurrence and died for the disease (0.276 vs 0.641; p = 0.044). Notably, also CD68-TAM distribution was different in patients who had recurrence versus patients who did not (1.028 vs 2.276; p = 0.036). Conclusions Immune cells showed variable expression in correlation with common prognostic factors, suggesting T cell infiltration may play a protective role towards lymphatic spread of disease and nodal metastatization. Furthermore, T cell density and macrophage infiltration were associated to a lower risk of recurrence and disease related death. A multicentric approach may be indicated to allow analysis of larger cohorts of patients, potentially increasing the power of our observations.