PEER-REVIEWED RESEARCH AND READING INSTRUCTION

Federal guidelines state that in an IEP for reading, special education services (instruction and interventions), should be based not just on just any old type of research; rather, on “peer-reviewed research.” This short article will explore the mystery of peer-reviewed research (PRR). 

BLIND PEER REVIEW

In order for research to be scientifically based research, it has to be peer-reviewed. According to the US Department of Education, peer-reviewed research (PRR) “generally refers to research that is reviewed by qualified and independent reviewers to ensure that the quality of the information meets the standards of the field before the research is published” (71 Fed Reg. 46664, Aug. 14, 2006).  

The Process of Peer Review

Peer-review (or blind peer-review) refers to the process used to evaluate the quality of research. This is how it works: Once a study has been conducted, researchers write an article describing what they did and what they found. This article is sent off to an academic journal for consideration for publication. The editor of the journal selects reviewers who are considered to have expertise in the field. These reviewers evaluate the study without knowing who conducted it (hence the term, blind peer-review).  Reviewers consider things such as the clarity of the research question, the theoretical context in which the research question was set, the adequacy of the methodology, the analysis of the data, the interpretation of the data, the validity of the conclusions, and the quality of writing. They then have four options: (a) recommend it for publication, (b) recommend it for publication with revisions, (c) suggest specific revisions be made and that it be resubmitted for consideration, or (d) recommend the article be rejected.  

Peer-review simply denotes a process. The quality or rigor of this process varies. Reviewers and editors of highly prestigious academic journals use a process that is rigorous and very selective. These journals have low acceptance rates and tend to have considerable influence on the field. Other journals have a less rigorous review process and higher acceptance rates. However, all are still considered peer-reviewed journals.

Once published, the research is examined by researchers and practitioners within the field as well as the general public. Here it is further analyzed, evaluated, critiqued, debated, and embraced and/or rejected.  Journal articles are then written supporting or denouncing the findings of the original research. Other research is carried out to replicate, reinforce, extend, or refute the findings. Tenure and promotion are earned, grants are awarded, fellowships given, books are written, and guest appearances made. This all leads to new questions and new research. Thus, the cycle of research life continues.

The Reviewers

But who reviews the reviewers? One of the concerning factors with peer-review is the possibility of a skewed composition in the selection of members (Zirkel & Rose, 2009). It is possible that the reviewers are over-represented by individuals (a) who have a singular research paradigm, (b) who’s strong adherence to a singular theoretical model renders them data-resistant, (c) who have a strong political or ideological bias, or (d) who might profit financially (directly or indirectly) from a certain type of research publication.  

A panel of Experts

Also included in a definition of peer-reviewed research by the US Department of Education would be studies that are reviewed by a “panel of experts”.  This is a loosely defined construct. For example, each of the groups in Figure 1 have at various times been considered to be a panel of experts related to education or reading instruction. However, based on their experience, expertise, and world view, it is very likely that the people in each group would have vastly differing views on both research methodology and effective reading instruction. You can see then that the views and the subsequent recommendations made by any given panel of experts varies greatly and is highly dependent on who is doing the selecting and who is selected.  

The National Reading Panel

To illustrate what can go wrong with a panel of experts, in 1997 Congress asked the National Institute of Children’s Health and Development (NICHD) to work with the U.S. Department of Education to establish a National Reading Panel (NRP). The task of this panel of experts was to evaluate existing research related to reading instruction in order to find the best ways of teaching children to read. Congress perhaps had never heard of two prominent organizations devoted to doing just this: The International Literacy Association (ILA) and the National Council of Teachers of English (NCTE). These organizations are comprised of thousands of teachers, researchers, scholars, and other educational professionals. At the time this panel was put together, these organizations had been studying reading instruction for over 40 and 80 years respectively. Instead of utilizing the expertise, experience, and accumulated knowledge of these organizations, the NICHD selected 14 people and gave them 18 months to review selected reading research (see Figure 2).

And how was this 14-member panel selected? The director of the NICHD (who was a physician), consulted with the Secretary of Education (who was a lawyer with a degree in accounting). They selected a physicist to chair this panel of experts. They put together a 14-member panel that included a physicist, a certified public accountant, a sociologist, a pediatrician, a school administrator, and six people from psychology or educational psychology.

To be fair, some members of this panel had made significant contributions to the field of reading instruction; however, clearly excluded from this panel were those who did not adhere to a simple view of reading represented by the phonological processing model. Also excluded were those who did not embrace controlled experimental research as the exclusive an epistemologically privileged way of determining causality and making claims as to what constitutes knowledge.  It is fairly obvious that this panel of experts was over-represented by those with a singular theoretical perspective and research paradigm. In short, the panel of experts was put together to find a predetermined answer. However, real science does not start with the answer.

In 2000 the panel issued their 500-page report (National Reading Panel, 2000). While this report contained some useful information, none of it was new or ground-breaking (Allington, 2002). However, because it started with a definition of reading as merely sounding out words, it should not be surprising that the conclusions described a rather narrow and parochial view of what effective reading instruction was. 

IMPLICATIONS

           So what? What might you take from this very short chapter? Four things: First, if you did not fully understand the peer-review process before, hopefully you now have a better understanding. It is not a perfect process, but it is a process and this process is important. 

Second, the process is not without bias or flaws. Peer review does not magically make research unbiased or pure. It is not possible for human beings to have a completely objective, unbiased view of anything. Peer-review is simply another filter to try to remove some of the impurities related to bias, methodology, theoretical context, applications, and conclusions. 

Third, you are the ultimate filter. You are the most important peer-reviewer. In this respect, you must always ask: Does the strategy or approach work with the students in front of you? Does it enhance their ability to create meaning with print? Does it move them forward, unimpeded, in their journey to achieve their full literacy potential? It does not matter if a strategy or approach demonstrates significant results with a large sample size if it does not work with your sample size.

And fourth, federal government has many significant roles to play in enhancing the betterment of our society and improving the lives of all people.  However, identifying effective reading instruction is not one of them.

REFERENCE

Allington, R.L. (Ed.). (2002). Big brother and the national reading curriculum: How ideology trumped evidence. Portsmouth, NH: Heinemann.

Bracey, G.W. (2004). Setting the record straight: Responses to misconceptions about public education in the U.S. (2nd ed). Portsmouth, NH: Heinemann.

Coles, G. (2003). Reading the naked truth: Literacy, legislation, and lies. Portsmouth, NH: Heinemann.

National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implication for reading instruction.  (https://www.nationalreadingpanel.org).

Straus, S. L. (2005). The linguistics, neurology, and politics of phonics. Mahway, NJ: Lawrence Erlbaum Associates, Publishers.

Yatvin, J. (2002). Babes in the woods: The wanders of the National Reading Panel. In R. Allington (Ed.), Big brother and the national reading curriculum: How ideology trumped evidence (pp. 125-136). Portsmouth, NH: Heinemann.

Zirkel, P., & Rose, T. (2009). Scientifically based research and peer-reviewed research under IDEA: The legal definitions, applications, and implications. Journal of Special Education Leadership, 22, 36-50.

PODCAST

https://rss.com/podcasts/drandy/44867