Understanding Science, Research, and Research Methods

* This is an excerpt from a book I’m working on called, Understanding the Science of Reading: Context Matters.? It will be published by Guildford and out sometime in 2025.

Podcast version of this article

The philosophy of science is beyond the scope of this book.? Suffice to say, that people have widely differing views here.? My goal in this chapter is to bring a little more clarity to three important and often misunderstood terms: (a) science, (b) research, and (c) research methods.? In doing so, I hope to move the needle a little bit in helping you become more critical consumers of educational research

Understanding Science and Scientists

Science is both a noun and a verb.? It’s a noun when it refers to a field or a system of knowledge within a particular area such as physics, chemistry, or zoology.? It is a verb when it references the processes used to develop that system of knowledge (research). ?

Who then is a scientist?? That term is sometimes used to refer to a person with great knowledge in a field of science.? It’s also used to denote somebody doing research in a field of science.? But sometimes the term ‘scientist’ is misused to project the illusion of precision, objectivity, trust, and certainty.

“Scientists tell us that …”

“Scientists have discovered that …”

“Scientists now believe that …”

“Scientists have determined that …”

“Scientists have shown that … “

Any statement that begins with the term “scientists”, followed by a declarative statement can be said to be technically true for two reasons: First, there is no standard definition of a scientist.? What exactly is a scientist?? Who is a scientist?? Do you need to have a license to be a scientist?? Do you need to have expertise in the area in which you are doing your science??

And second, all you need is two.? You could find two scientists who believe something outrageous and still be technically true in saying “Scientists tell us that …”?

“Scientists tell us that dinosaurs didn’t exist.”?

“Scientists have discovered that the moon landing was a hoax.”

“Scientists now believe that the FBI is sending killer death rays through our TV sets.”

“Scientists have determined that all children need extensive phonics instruction in a predetermined sequence of skills followed by practice using these skills with decodable texts.”

“Scientists have shown that LETRS professional development for teachers will result in enhanced reading achievement for children.”

“Scientists have proven that whole language is a hoax.”

“Scientists now tell us that the teaching of the three cueing systems is causing reading scores to plummet.”

“Scientists believe that the massive reading crisis is caused by balanced literacy and meaning-based reading instruction.”

Since you could find at least two people who call themselves scientists and who ascribe to any of the statements above, all these statements are technically true.? However, all they really tell us is that (a) two or more people said something and (b) these people identified themselves as being scientists.

The lesson is that anytime someone writes or says, “Scientists say that …”? or “Research shows that …” without citing a source or making their sources available in some form, that person should be deemed a clown.? A clown is one who purports to have great knowledge, when in fact they have little (Bozo, 2024).

Understanding Research

Science as a verb denotes a process used by scientists to understand our world.? This process is called research.? At its most basic level, research is the process of asking questions, collecting data, and using that data to answer those questions.? However, as described in Chapter 1 while collecting data is one part of research, it is not research by itself.? For research to be research, there must be certain conditions in place:

1. The question or area of inquiry is defined upfront.

2. The process used to collect data is recognized and approved by the larger scientific community.? This community-approved process is called a method or a research method (see below).?

3. The data-collecting process is systematic. ?That means there is a plan in place before the research project begins as to what data to collect, how, how often, and from whom.? It is not simply your impression of an experience.? (Note that descriptive research describes and analyzes a data set that currently exists, however, this type of research does not manipulate variables or attempt to define why something occurred.)

4. The data collected are analyzed and presented as results or findings.

5. Logical inferences, explanations, or conclusions are made based on the data collected and only on the data collected.?

6. The method is described in such a way that another person could repeat the process.?

7. The study is subjected to blind peer review.? ?

8. The researcher does not have a stake in the outcome.

A Non-Example of Research?

Now let’s pretend that I conducted a study where I asked people to tell me stories about reading instruction.? They’d tell me things and I’d write these things down.? This would be data.? But what if I only wrote down the things that would create a sensational story?? What if I only interviewed people who had a certain story to tell?? What if I only interviewed a certain type of people, or a small number of people, or only people who agreed with me?? What if I purposely excluded things that would disrupt the narrative I was trying to create?? Would be research??

No.? That would be a radio documentary.?

But what if I wrote down what a few people told me and said it was true of a lot of people?? What if I included decontextualized factoids to illustrate the stories that people told me?? What if I decided to call myself an expert in the things I wrote down because I wrote them down?? And what if I used the fame I got from writing things down to make a lot of money?? What would that be??

Irresponsible and self-serving.

Okay, what if I used a systematic process to collect data and that systematic process was approved by the scientific community?? And what if I described exactly how I collected my data and from whom so that somebody could replicate exactly what I did?? Would that be research?

No.? Research is not research unless or until it has been subjected to some form of blind, peer review.

Basic and Applied Research

There are two kinds of research: basic and applied.? Basic research is that which is done apart from the setting in which it is used.? In reading, basic research includes research focusing on brain imaging, eye movement, miscue analysis, and priming studies.? This kind of research is a critical component in adding to our understanding of things and building theories.? In the case of reading instruction, applied research has been essential in helping us understand the processes the brain uses to create meaning with print (read).

Applied research is that which is done in the type of setting in which it will ultimately be used.? In the case of reading instruction, it’s the research conducted in classrooms.? According to the SoR, this only kind of research that should be used to design programs or make policy decisions.? That means that a lot of very important research is ignored.? This results in the very narrow and distorted view of reading and reading instruction used to make SoR policies.

Applied research in any field is essential.? However, if this is the only type of research used to understand phenomena, the likely impact will be a misunderstanding of the phenomena being viewed.

Methods of Science

Research is the unbiased systematic observations or experiments used in science.? Methods denote how the data are collected in research.? The method used in science to collect and analyze data is known as the research method, the scientific method, or the method of science.? ?Contrary to common assumptions, there is no such thing as thee “method of science” or thee “scientific method”.? Rather, there are methods of science.? Meaning that a wide variety of scientific methods are used to study the unknown (Hodson, 1988; Stanovich, 1992), including reading instruction. ?These methods tend to be put into two broad categories: quantitative and qualitative research (cite). ?A very basic description of each is provided below.

Quantitative Research Methods?

In quantitative research, sometimes called experimental research, the researcher takes an active role in setting up an observation or experiment in order to isolate a variable. A variable is the quality or condition about which the researcher wants to draw conclusions. The goal of experimental research is to figure out the effect that one thing (independent variable or treatment) might have on another thing (the dependent variable or measure). In order to demonstrate a causal relationship, the researcher creates a controlled environment that isolates a particular variable by controlling all the other variables around it.? The research wants to be able to say with great confidence that one thing caused another thing to occur.

Quantitative research uses numbers to reach a state of knowing. The questions are stated up front and only data related to the research questions are observed and recorded. A cause-and-effect paradigm is then used to make sense of the world.

Important terms.? Some important terms in understanding controlled experiment research follow:

??????????? 1. The independent variable is the treatment or factor that the researcher manipulates to determine its impact. In educational research, it’s what’s done or not done to a group of students.

??????????? 2. The dependent variable is the particular result or the effect of the treatment. It’s what’s measured.? An easy way to remember the distinction between these two variables is to think of the dependent variable as depending on the treatment or independent variable.

??????????? 3. The treatment group or experimental group is the group of subjects, participants, or objects that are exposed to the treatment or independent variable.

??????????? 4. The control group is a group as similar as possible in all characteristics to the treatment group; however, this group is not exposed to the independent variable or treatment for the purposes of comparison.?

??????????? 5. The research question is that for which the researcher seeks to find an answer.

??????????? 6. The hypothesis is a tentative statement that can be supported if the outcome of the experimentation is as expected. ?If a number of different studies continue to support that hypothesis, it may be elevated to the status of theory.

Qualitative Research Methods?

In qualitative research, the researcher takes the world as it is.? It does not try to manipulate conditions in order to isolate variables.? Qualitative research instead uses some type of systematic observation in order to reach an understanding of phenomena. ?Here, the questions are more open-ended and less defined with plenty of room to collect a variety of data through collateral observations. ?

Think of Jane Goodall sitting in the middle of a bunch of gorillas taking notes.? This is qualitative research.? She’s studying gorillas in their natural setting.? Her research helps us understand how real gorillas react to and interact with phenomena in the real world.? Now picture Brian Cambourne sitting in a kindergarten class taking notes (cite).? He’s doing the same.? Both Jane and Brian are able to get to things that controlled experimental research simply can not.?? They’re able to see a bigger picture in terms of interacting and interdependent systems.?? Both Jane and Brian describe causal factors but with a much deeper understanding of the causes and effects.? Both help us understand that gorillas and children in jungles and classrooms are wonderfully unique and unstandardized entities.

Common qualitative research designs include ethnographies, case studies, grounded theory, phenomenology, action research, narrative research, and historical studies.? Qualitative research methods include interviews, focus groups, observations, and document analysis.

Final Word

Science uses a variety of unbiased, systematic observations and experimentation called research.? Research is a way of seeing reality, a procedure used to view and re-view the world in order to understand it.? The research method is the lens used by the researcher through which the world is viewed.? Different lenses provide different views of reality. ?Controlled, experimental research is able to look deeply at a small bit of controlled reality.? In contrast, qualitative research provides a much broader lens to observe a much more real world.? One type of research design is not better than the other.? Both are necessary to get a complete understanding of phenomena.

The research method used should always be determined by the question being asked.? In contrast, the SoR movement has determined that only a single type of research methodology can be used to ask and answer questions in the field of education.? They have ironically labeled this, scientifically based research (SBR).? The irony of this is captured by P. David Pearson in one of the greatest quotes ever regarding science, research, and reading:

“Regarding science, my fundamental claim is that reading research can never be truly rigorous, indeed truly scientific, until and unless it privileges all of the empirical and theoretical methodologies that characterize the scientific disciplines” (Pearson, 2004, p. 234).

This conversation continues in the next chapter.? Here I hope to provide an understanding of reading and the science of reading.

?

?