Misunderstanding No. 4 about Case Studies

Misunderstanding no. 4: The case study contains a bias toward verification, that is, a tendency to confirm the researcher’s preconceived notions.

Correction no. 4: The case study contains no greater bias toward verification of the researcher’s preconceived notions than other methods of inquiry. On the contrary, experience indicates that the case study contains a greater bias toward falsification of preconceived notions than toward verification.

See sources and read more here: bit.ly/1bHZ1Mx (you may have to copy and paste link).

The fourth of five misunderstandings about case-study research is that the method maintains a bias toward verification, understood as a tendency to confirm the researcher’s preconceived notions, so that the study therefore becomes of doubtful scientific value. Diamond (1996, 6), for example, holds this view. He observes that the case study suffers from what he calls a “crippling drawback,” because it does not apply “scientific methods,” by which Diamond understands methods useful for “curbing one’s tendencies to stamp one’s pre-existing interpretations on data as they accumulate.”

Francis Bacon (1853, xlvi) saw this bias toward verification, not simply as a phenomenon related to the case study in particular, but as a fundamental human characteristic. Bacon expressed it like this:

"The human understanding from its peculiar nature, easily supposes a greater degree of order and equality in things than it really finds. When any proposition has been laid down, the human understanding forces everything else to add fresh support and confirmation. It is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives than negatives."

Bacon touches upon a fundamental problem here, a problem, which all researchers must deal with in some way. Charles Darwin (1958, 123), in his autobiography, describes the method he developed in order to avoid the bias toward verification:

"I had . . . during many years followed a golden rule, namely, that whenever a published fact, a new observation or thought came across me, which was opposed to my general results, to make a memorandum of it without fail and at once; for I had found by experience that such facts and thoughts were far more apt to escape from the memory than favorable ones. Owing to this habit, very few objections were raised against my views, which I had not at least noticed and attempted to answer."

The bias toward verification is general, but the alleged deficiency of the case study and other qualitative methods is that they ostensibly allow more room for the researcher’s subjective and arbitrary judgment than other methods: they are often seen as less rigorous than are quantitative, hypothetico-deductive methods. Even if such criticism is useful, because it sensitizes us to an important issue, experienced case researchers cannot help but see the critique as demonstrating a lack of knowledge of what is involved in case-study research. Donald Campbell and others have shown that the critique is fallacious, because the case study has its own rigor, different to be sure, but no less strict than the rigor of quantitative methods. The advantage of the case study is that it can “close in” on real-life situations and test views directly in relation to phenomena as they unfold in practice.

According to Campbell, Ragin, Geertz, Wieviorka, Flyvbjerg, and others, researchers who have conducted intensive, in-depth case studies typically report that their preconceived views, assumptions, concepts, and hypotheses were wrong and that the case material has compelled them to revise their hypotheses on essential points. The case study forces upon the researcher the type of falsifications described above. Ragin (1992, 225) calls this a “special feature of small-N research,” and goes on to explain that criticizing single-case studies for being inferior to multiple case studies is misguided, since even single-case studies “are multiple in most research efforts because ideas and evidence may be linked in many different ways.”

Geertz (1995, 119) says about the fieldwork involved in most in-depth case studies that “The Field” itself is a “powerful disciplinary force: assertive, demanding, even coercive.” Like any such force, it can be underestimated, but it cannot be evaded. “It is too insistent for that,” says Geertz. That he is speaking of a general phenomenon can be seen by simply examining case studies, like Eckstein (1975), Campbell (1975), and Wieviorka (1992) have done. Campbell (1975, 181-2) discusses the causes of this phenomenon in the following passage:

"In a case study done by an alert social scientist who has thorough local acquaintance, the theory he uses to explain the focal difference also generates prediction or expectations on dozens of other aspects of the culture, and he does not retain the theory unless most of these are also confirmed . . . Experiences of social scientists confirm this. Even in a single qualitative case study, the conscientious social scientist often finds no explanation that seems satisfactory. Such an outcome would be impossible if the caricature of the single case study . . . were correct--there would instead be a surfeit of subjectively compelling explanations."

“The Field” itself is a “powerful disciplinary force: assertive, demanding, even coercive.” Like any such force, it can be underestimated, but it cannot be evaded. “It is too insistent for that,” says Geertz.

According to the experiences cited above, it is falsification and not verification, which characterizes the case study. Moreover, the question of subjectivism and bias toward verification applies to all methods, not just to the case study and other qualitative methods. For example, the element of arbitrary subjectivism will be significant in the choice of categories and variables for a quantitative or structural investigation, such as a structured questionnaire to be used across a large sample of cases. And the probability is high that (1) this subjectivism survives without being thoroughly corrected during the study and (2) that it may affect the results, quite simply because the quantitative/structural researcher does not get as close to those under study as does the case-study researcher and therefore is less likely to be corrected by the study objects “talking back.” George and Bennett (2005, 20) describe this all-important feature of case study research like this:

"When a case study researcher asks a participant "were you thinking X when you did Y," and gets the answer, "No, I was thinking Z," then if the researcher had not thought of Z as a causally relevant variable, she may have a new variable demanding to be heard."

Statistical methods may identify deviant cases that can lead to new hypotheses, but in isolation these methods lack any clear means of actually identifying new hypotheses. This is true of all studies that use existing databases or that collect survey data based on questionnaires with pre-defined standard questions. Unless statistical researchers do their own archival work, interviews, or face-to-face surveys with open-ended questions--like case study researchers--they have no means of identifying left-out variables (George and Bennett 2005, 21). According to Ragin (1992, 225; see also Ragin 1987, 164-71):

"[T]his feature explains why small-N qualitative research is most often at the forefront of theoretical development. When N’s are large, there are few opportunities for revising a casing [that is, the delimitation of a case]. At the start of the analysis, cases are decomposed into variables, and almost the entire dialogue of ideas and evidence occurs through variables. One implication of this discussion is that to the extent that large-N research can be sensitized to the diversity and potential heterogeneity of the cases included in an analysis, large-N research may play a more important part in the advancement of social science theory."

Here, too, this difference between large samples and single cases can be understood in terms of the phenomenology for human learning discussed under Misunderstanding No. 1. If one thus assumes that the goal of the researcher’s work is to understand and learn about the phenomena being studied, then research is simply a form of learning. If one assumes that research, like other learning processes, can be described by the phenomenology for human learning, it then becomes clear that the most advanced form of understanding is achieved when researchers place themselves within the context being studied. Only in this way can researchers understand the viewpoints and the behavior, which characterizes social actors. Relevant to this point, Giddens (1982, 15) states that valid descriptions of social activities presume that researchers possess those skills necessary to participate in the activities described:

"I have accepted that it is right to say that the condition of generating descriptions of social activity is being able in principle to participate in it. It involves “mutual knowledge,” shared by observer and participants whose action constitutes and reconstitutes the social world."

From this point of view, the proximity to reality, which the case study entails, and the learning process which it generates for the researcher will often constitute a prerequisite for advanced understanding. In this context, one begins to understand Beveridge’s conclusion that there are more discoveries stemming from intense observation of individual cases than from statistics applied to large groups. With the point of departure in the learning process, we understand why the researcher who conducts a case study often ends up by casting off preconceived notions and theories. Such activity is quite simply a central element in learning and in the achievement of new insight. More simple forms of understanding must yield to more complex ones as one moves from beginner to expert.

On this basis, the fourth misunderstanding--that the case study supposedly contains a bias toward verification, understood as a tendency to confirm the researcher’s preconceived ideas--is revised as follows:

The case study contains no greater bias toward verification of the researcher’s preconceived notions than other methods of inquiry. On the contrary, experience indicates that the case study contains a greater bias toward falsification of preconceived notions than toward verification.