Methods to Integrate Natural Language Processing Into Qualitative Research

Abstract

Background: Qualitative methods analyze contextualized, unstructured data. These methods are time and cost intensive, often resulting in small sample sizes and yielding findings that are complicated to replicate. Integrating natural language processing (NLP) into a qualitative project can increase efficiency through time and cost savings; increase sample sizes; and allow for validation through replication. This study compared the findings, costs, and time spent between a traditional qualitative method (Investigator only) to a method pairing a qualitative investigator with an NLP function (Investigator +NLP). Methods: Using secondary data from a previously published study, the investigators designed an NLP process in Python to yield a corpus, keywords, keyword influence, and the primary topics. A qualitative researcher reviewed and interpreted the output. These findings were compared to the previous study results. Results: Using comparative review, our results closely matched the original findings. The NLP + Investigator method reduced the project time by a minimum of 120 hours and costs by $1,500. Discussion: Qualitative research can evolve by incorporating NLP methods. These methods can increase sample size, reduce project time, and significantly reduce costs. The results of an integrated NLP process create a corpus and code which can be reviewed and verified, thus allowing a replicable, qualitative study. New data can be added over time and analyzed using the same interpretation and identification. Off the shelf qualitative software may be easier to use, but it can be expensive and may not offer a tailored approach or easily interpretable outcomes which further benefits researchers.