74 Development and content validation of a tool to assess quality of primary care practice

Abstract

OBJECTIVES/GOALS: Primary care practices struggle to identify which combination of care structures and processes need to be implemented to improve practice performance and subsequently, patient outcomes. The goal of this study is to develop and validate a tool to assess care structures and processes that are associated with better quality and patient outcomes. METHODS/STUDY POPULATION: Data from a scoping review, Delphi study, and qualitative interviews with high-performing primary care practices contributed to the development and content validation of the Tool for Advancing Practice Performance (TAPP). From these sources we identified 314 items representing care structures (e.g., care team makeup, use of electronic health records) and processes (e.g., care coordination, panel management). We developed criteria for deleting and rescuing items and received input from our expert panel to refine the pool of items. We eliminated items that were redundant and lacked clarity/specificity. The tool was further modified based on feedback from cognitive interviewing and pilot testing with practice managers, quality improvement leaders, and physicians from primary care practices. RESULTS/ANTICIPATED RESULTS: The pool of 314 items was winnowed to 188 after applying criteria for deleting and rescuing items. During the expert review, 70 items were eliminated and 8 new items were added, resulting in a working tool of 126 items. We conducted eight cognitive interviews with the 126-item tool and received feedback on the content, item structure, and language, which led to the elimination of 13 items that were poorly or incorrectly understood by respondents. We also modified the language of 23 items for clarity. After cognitive interviewing, the resulting tool comprised 113 items. Fifteen practices piloted the tool and no additional items were eliminated. We modified the instructions for completing the tool and resolved technical issues related to online administration. DISCUSSION/SIGNIFICANCE: TAPP is a novel tool for assessing care structures and processes that are associated with better quality and patient outcomes in primary care settings. The tool can be used by researchers and primary care clinicians to identify areas for improvement in practice performance and patient outcomes related to chronic disease prevention and management.