When Evidence Doesn’t Work (Editorial)

Abstract

I was listening intently to a discussion on the radio recently between Newfoundland and Labrador’s Minister of Education and aprofessor from Memorial University’s Math Department. They were debating the efficacy of the math curriculum in the province’s school system. As a parent of a grade 3 student, I have my own thoughts on how the curriculum is affecting kids’ math skills (and their anxiety levels, but let’s not go there). The professor echoed the concern that parents, teachers and students have been expressing: quite simply, it’s not working. Far too many children are failing math and are struggling with the both the content and pace of the required modules. Why am I telling you this? One particular comment made by the Minister of Education struck me. She said that there was evidence to suggest that this curriculum should work. While I’m always delighted to see the evidence based practice model being used, particularly for the betterment of my kids’education, it is dismaying to see that it is not always applied well. In this particular case, evidence was collected from somewhere and a decision was made to implement a new math curriculum based on the gathered evidence. Assuming that this truly was good evidence upon which to base such a decision, then I would have to concede that the appropriate steps were taken up until that point. Unfortunately, it appears that the entire process stopped there. As we know, one of the most important components of a thorough ebp‐based implementation is an internal evaluation. What might work somewhere else is not guaranteed to work in another environment, and it is essential to determine why an implementation or intervention worked or didn’t work. It would seem, in this case, that formal evaluations of the effectiveness of the new math curriculum have not been performed and therefore, the powers that be rely solely on the fact that it worked somewhere else. This is not evidence based practice at its finest.So, what happens when evidence doesn’t work? We try to figure out why it didn’t work. Did we miss something in the critical appraisals? What is inherently different in the population or system at hand? Are there other confounders in your environment thatyou had not considered (i.e. time of year, available resources, courses being offered, etc.)? As pointed out in this issue’s commentary, a good idea is to plan your project with research and assessment in mind. Not only will you be able to track the various stages of implementation and reactions to it, it will save you the time that you may have taken weeks or months later to retrospectively evaluate. And, never to let an opportunity be wasted, I would welcome an article submission outlining an evidence based implementation that didn’t work. If it doesn’t work, it doesn’t mean that you have failed. It means that there was something you had not anticipated that had a negative effect on your intervention. We can all benefit from such information. Speaking of benefiting, I will take this opportunity to bid a fond farewell to two of EBLIP’s original Editorial Board members: Denise Koufogiannakis and Pam Ryan. Although Denise and Pam have made numerous contributions to evidence based librarianship, their work on this journal has arguably made the biggest footprint. Denise co‐founded this journal and has worked tirelessly to create an avenue for high quality publishing in this subject area. She has passionately maintained the Evidence Summaries for each issue – a task that has required a great deal of both time and expertise. She created an excellent team of writers with whom she works closely and she consistently provides feedback to ensure that first‐rate summaries are published in every issue. Pam courageously agreed to take on the task of Production Editor with the first issue. No one on the Editorial Board had experience with the journal publishing software, OJS, and Pam was able to calmly work out the bugs, respond to our calls of frustration and panic, and she is solely responsible for the final look of the publication every 3 months. She has kept Denise, Alison and me on schedule and has caught more last minute typos and formatting issues than I could possibly count. Both Denise and Pam have led the journal to where it is today as a result oftheir commitment, expertise, enthusiasm, and sincere belief in what the board is trying to accomplish through this journal. On behalf of the Editorial Board, the Evidence Summary writers, the peer reviewers, the copyeditors, and the readers, I thank them both and wish them the best success in their future endeavors. Thankfully, their future endeavors include continued involvement with this journal. Denise will continue her work with Classics Evidence Summaries, which will be a semi‐regular feature, and both Denise and Pam have joined the Editorial Advisory team. On that note, I would like to welcome two new editorial board members. Lorie Kloda has joined the board as the Associate Editor, Evidence Summaries. Lorie has been contributing to the journal as an EvidenceSummaries author. She hails from McGill University where she is currently pursuing her PhD. Katrine Mallan is assuming the role of Production Editor. Katrine currently works at the University of Calgary as an instruction librarian. Please join me in welcoming both Lorie and Katrine. They may have big shoes to fill, but they have the skills, enthusiasm and expertise to do so seamlessly.This is our last issue for the year, and it’s a big one. Aside from 7 Evidence Summaries, we are featuring 5 summaries of classic articles. Many of you will, no doubt, be familiar with these seminal papers and will be interested to read how they fare today and the impact that they have had on our profession. Also in this issue are 3 original research articles and one article outlining how to create effective questions for surveys. There is much information here to discuss on your coffee break. Well, that’s year two under our belts. Have a lovely holiday season and a very happy and safe new year. We’ll see you in 2008!