Librarians’ Reported Systematic Review Completion Time Ranges Between 2 and 219 Total Hours with Most Variance due to Information Processing and Instruction

Abstract

A Review of: Bullers, K., Howard, A. M., Hanson, A., Kearns, W. D., Orriola, J. J., Polo, R. L., & Sakmar, K. A. (2018). It takes longer than you think: Librarian time spent on systematic review tasks. Journal of the Medical Library Association, 106(2), 198-207. https://doi.org/10.5195/jmla.2018.323 Abstract Objective – To investigate how long it takes for medical librarians to complete steps toward completion of a systematic review and to determine if the time differs based on factors including years of experience as a medical librarian and experience completing systematic reviews. Design – Survey research as a questionnaire disseminated via email distribution lists. Setting – At institutions that are members of the Association of Academic Health Sciences Libraries (AAHSL) and librarians at Association of American Medical Colleges (AAMC) or American Osteopathic Association (AOA) member institutions. Subjects – Librarians of member institutions who have worked on systematic reviews. Methods – On December 11, 2015, AAHSL library directors and librarian members of AAMC and AOA were sent the survey and the recommendation to forward the survey to librarians on staff who have worked on systematic reviews. Reminders were sent on December 17, 2015, and the survey closed for participation on January 7, 2016. Participants who had worked on a systematic review within the past five years were asked to indicate experience by the number of systematic reviews completed, years of experience as a medical librarian, and how much time was spent, in hours, on the following: initial consultations/meetings; developing and testing the initial search strategy; translating the strategy for other databases; documenting the process; delivering the search results; writing their part of the manuscript; other tasks they could identify; and any instruction (i.e., training they provided to team members necessary for completion of the systematic review). Participants also further broke down the amount of their time searching, by percentage of time, in various resources, including literature indexes/databases, included studies’ references, trial registers, grey literature, and hand searching. Participants were also given space to add additional comments. The researchers reported summary statistics for phase one and, for phase two, excluded outliers and performed exploratory factor analysis, beginning with principal components analysis (PCA), followed by a varimax rotation, to determine if there was a relationship between the time on tasks and experience. Main Results – Of the 185 completed responses, 105 were analyzed for phase one because 80 responses were excluded due to missing data or no recent experience with a systematic review. The average respondent had between 1 and 6 years of experience: 1-3 years in librarianship (49.5%) and 4-6 years (23.8%). The time reported for completion of all tasks ranged from 2 to 219 hours with a mean of 30.7 hours. Most of the variance (61.6%) was caused by “information processing” and “interpersonal instruction/training” components. Search strategy development and testing had the highest average time at 8.4 hours. Within that category, databases accounted for 78.7% of time searching, followed by other searching methods. For remaining systematic review tasks, their averages were as follows: translating research (5.4 hours), delivering results (4.3 hours), conducting preliminary consultations (3.9 hours), instruction (3.8 hours), documentation (3.0 hours), additional tasks that were written-in by respondents (2.2 hours), and writing the manuscript (1.8 hours). The most common written-in tasks were development of inclusion/exclusion criteria, critical appraisal, and deduplication. Other write-ins included retrieving full-text articles, developing protocols, and selecting a journal for publishing the systematic review. For the second phase of analysis, 12 responses were excluded as extreme outliers, and the remaining 93 responses were analyzed to detect a relationship between experience and time on task. Prior systematic review experience correlated with shorter times performing instruction, consultation, and translation of searches. However, librarian years of experience affected the percentage of time on task, where greater years of experience led to more time spent consulting and instructing than the percentage for librarians with fewer years of experience. Librarians with greater than 7 years of experience skewed trends toward shorter time on task, and, with their data excluded, years of experience showed weak positive correlation with instruction and consultation. Conclusion – Because the average librarian participating on systematic review teams has had few prior experiences and because the times can vary widely based on assigned roles, duties, years of experience, and complexity of research question, it is not advised to establish expectations for librarians’ time on task. This may be why library administrators have disparate expectations of librarians’ involvement in systematic reviews and find it difficult to allocate and anticipate staff time on systematic review projects. While it may not be possible to set specific overarching guidelines for librarians’ expected time on systematic review tasks, librarian supervisors and library directors planning for their staff to offer systematic review services should work to develop extensive understanding of the steps for conducting and assessing systematic reviews in order to better estimate time commitments.