Greehey Children’s Cancer Research Institute, UT Health San Antonio, San Antonio, United States; Department of Population Health Sciences, UT Health San Antonio, San Antonio, United States
Zhenqing Ye
Greehey Children’s Cancer Research Institute, UT Health San Antonio, San Antonio, United States; Department of Population Health Sciences, UT Health San Antonio, San Antonio, United States
Yidong Chen
Greehey Children’s Cancer Research Institute, UT Health San Antonio, San Antonio, United States; Department of Population Health Sciences, UT Health San Antonio, San Antonio, United States
Xiaojing Wang
Greehey Children’s Cancer Research Institute, UT Health San Antonio, San Antonio, United States; Department of Population Health Sciences, UT Health San Antonio, San Antonio, United States
Greehey Children’s Cancer Research Institute, UT Health San Antonio, San Antonio, United States; Department of Population Health Sciences, UT Health San Antonio, San Antonio, United States
Quantifying the activity of gene expression signatures is common in analyses of single-cell RNA sequencing data. Methods originally developed for bulk samples are often used for this purpose without accounting for contextual differences between bulk and single-cell data. More broadly, few attempts have been made to benchmark these methods. Here, we benchmark five such methods, including single sample gene set enrichment analysis (ssGSEA), Gene Set Variation Analysis (GSVA), AUCell, Single Cell Signature Explorer (SCSE), and a new method we developed, Jointly Assessing Signature Mean and Inferring Enrichment (JASMINE). Using cancer as an example, we show cancer cells consistently express more genes than normal cells. This imbalance leads to bias in performance by bulk-sample-based ssGSEA in gold standard tests and down sampling experiments. In contrast, single-cell-based methods are less susceptible. Our results suggest caution should be exercised when using bulk-sample-based methods in single-cell data analyses, and cellular contexts should be taken into consideration when designing benchmarking strategies.