Esophageal squamous cell carcinoma (ESCC) is the predominant type of esophageal cancer in the Asian region, including Japan. A previous study reported mutational landscape of Japanese ESCCs by using exome sequencing. However, somatic structural alterations were yet to be explored. To provide a comprehensive mutational landscape, we performed whole genome sequencing (WGS) analysis of biopsy specimens from 20 ESCC patients in a Japanese population. WGS analysis identified non-silent coding mutations of TP53, ZNF750 and FAT1 in ESCC. We detected six mutational signatures in ESCC, one of which showed significant association with smoking status. Recurrent structural variations, many of which were chromosomal deletions, affected genes such as LRP1B, TTC28, CSMD1, PDE4D, SDK1 and WWOX in 25%–30% of tumors. Somatic copy number amplifications at 11q13.3 (CCND1), 3q26.33 (TP63/SOX2), and 8p11.23 (FGFR1) and deletions at 9p21.3 (CDKN2A) were identified. Overall, these multi-dimensional view of genomic alterations improve the understanding of the ESCC development at molecular level and provides future prognosis and therapeutic implications for ESCC in Japan.