Printable PDF VersionThe Nature of "Evidence" in Qualitative Research Methods
Steven Miller and Marcel Fredericks
Steven Miller, Ph D, Professor, School of Education, Loyola University, Chicago
Marcel Fredericks, PhD, Professor, Department of Sociology and Anthropology, Loyola University Chicago
Abstract: The article is an attempt to show that a continuing issue in qualitative research methods is describing and justifying how qualitative data become "evidence" for a claim. Several models from the field of Confirmation Theory are developed and described within the context of a qualitative research example. It is argued that for the qualitative research case, the meaning and application of what constitutes evidence is best viewed in terms of a primary logical distinction.
Keywords: naturalistic inquiry, evidence, logical conditions, inference
Citation information:
Miller, S., & Fredericks, M. (2003). The nature of "evidence" in qualitative research methods. International Journal of Qualitative Methods, 2(1). Article 4. Retrieved DATE from http://www.ualberta.ca/~ijqm
Although qualitative research continues its steady gains in terms of becoming a "mainstream" form of inquiry, there still remain certain basic or foundational issues that need, we believe, further explication. The need for additional analysis is as much rooted in the rapid development and acceptability of these forms of inquiry as it is in a traditional neglect arising from a lack of awareness or a type of subtle obfuscation. In any event, the issue we would like to give some additional attention to is that of "evidence". Admittedly, a notoriously ambiguous term (Miller, 1990), it is nevertheless one of the most crucial in providing a justification for qualitatively-derived research findings.
By this we mean the following: what is the justification we can (or do) provide for saying some "claim", "finding", or conclusion is warranted? 1 For the present purposes, the term justification simply means "what evidence do we have for saying (or concluding) that ‘X is an X’?" By way of clearing the conceptual landscape a bit, it can quickly be acknowledged that determining whether an X is an X raises both important ontological and epistemological concerns. On the ontological side, there are both general and specific concerns having to do with how a socially constructed construct (or "finding", for that matter) "exists", or in what sense its "existence" is expressed (Archer, Bhaskar, Collier, Lawon & Norrie, 1998; Loux, 2002; Moser, 1993). There are deep and crucial issues here but they are not the present focus. 2 Epistemologically, we are closer to what concerns how the concept of evidence relates to qualitative research findings. That is, do the methods employed in qualitative inquiry stand in an "epistemically credible" way to the "findings" they produce? When do findings, in other words, become evidence? Our findings must stand in some relationship to what we claim to be evidence if they are to serve (in some sense) as "truth markers" (Kirkham, 1997).
This distinction is one that is not made clear enough in interpretive inquiry. The general idea is to show how the data (no matter how generated) stand in relationship to the question, topic, or theme of interest. In other words, in what sense do we know that the data are (or are not) evidence? Data become evidence; they are not (alone) evidence; and this is an epistemological concern.
The difficult issue is how to best characterize the process by which data become evidence. There are many candidates here, but we will only mention a few to point out what is at stake. For example, from a clinical perspective, it has been suggested that some form of "abduction" would be appropriate (Upshur, 1997). But what exactly is abduction? Abduction is commonly believed to be a process of reasoning in which the logic of discovery is emphasized over the logic of justification. It is purported to be a search for how scientific hypotheses come to be, over how we try to assess them in terms of their truth or falsity (see Hanson, 1958). There has been a debate over whether such "logics" are possible — especially if they form a distinctive approach for reasoning that differs from traditional deductive and inductive logic (Audi, 1999).
A little more formally, abduction is sometimes traced to Aristotle’s Prior Analytics (Barnes, 1984), where given a known (certain) major premise, a weaker, minor, probable premise is conjoined producing a probable conclusion. The pragmatist C. S. Peirce is credited with naming the term, describing it as a process which includes: "(a) facts of type B have been observed; (b) a true statement of the form If A, then B can explain B. Therefore, probably A" (Mautner, 1999, p. 1; also, Hartshorne & Weiss, 1958). According to Shelley (1996, p. 279), Peirce’s view of scientific investigation included the following: "(1) observation of an anomaly, (2) abduction of hypotheses for the purpose of explaining the anomaly, inductive testing of the hypotheses in experiments, and (3) deductive confirmation that the selected hypothesis does predict the original anomaly (which is thereafter no longer an anomaly." Modern variations on abduction have resulted in what is now known as ‘Inference to the Best Explanation’ (IBE) (Lipton, 1993), although critics such as Rapport (1996) have argued that IBE is simply a version of Mill’s (1875/1952) system of induction (this will be explained in a related sense later on).
Is abduction a good model for qualitative research methods? On the face of it, it would seem to be, but there are problems, not so much of the model itself but how exactly it should be utilized for qualitative research purposes. 3 In other words, if we were to adopt abduction, how exactly should we proceed? The basic problem is the one mentioned above as (3): how do we test inductively the anomaly(ies) observed given our belief that the conditional statement, if A then B, does hold? This requires that the inductive methods be specified and applied in such a way that, indeed, qualitative data do become evidence for a claim. But this process is exactly what is in need of explanation, and it is not presumed by the possibility that abduction might apply. The question is: how does it apply? It is this issue that we are trying to address.
Before continuing, it may be worth mentioning that a somewhat earlier attempt to see if and how essentially qualitative data could become evidence was developed by Edelson (1984). In a somewhat ironic twist, since he was essentially trying to substantiate the validity of the psychoanalytic process by way of qualitative data, Edelson attempted to show the overall validity of traditional theories of verification and hypothesis testing for this enterprise. Thathe did not, in our opinion, is simply to say that, for qualitative inquiry, the way data do become evidence may lie in the direction of "confirmation theory" itself.
So, although the concept of evidence in some ways is more closely connected with epistemological perspectives, we will not be concerned with any specific theory of "truth" 4 (Morton, 1997). What the analysis will involve is an examination of how theories of evidence are formulated, and an attempt to draw some parallels with "evidence" in qualitative inquiry, arguing that at least one of these theories must in some more direct way be included in discussions of qualitative research findings. One may object to this project by saying there already exist sufficiently adequate frameworks for knowing when and how qualitative findings do constitute evidence — such things as "auditing", "member checking", "transferability", "structural collaboration", and so on (Eisner, 1991; Lincoln & Guba, 1985). But this is not necessarily the case. The problem is similar to what is known as the "use-mention" fallacy (Mautner, 1999, pp. 581-582), which can be stated as follows: "Does auditing increase the validity of qualitative findings?" versus "Does auditing increase the ‘validity’ of qualitative findings?" In the first statement validity is being "used" suggesting that its meaning is not problematic — its use has been established. In the second statement, however, the term "validity" is put in quotes because it suggests that there may be some unique way the term is or needs to be looked at, such as, is the term to be seen in comparison with quantitative research ways of using the term? It may also be seen that the term "audit" above could be examined in the same way.
The point of introducing this distinction is to show that in qualitative inquiry we often act as though such issues as "evidence" are not a problem — simply by the fact that their "mention" is sufficient to establish their use. We do not believe it is, and the result is avoiding an issue that in many ways is at the heart of doing qualitative analysis. In what follows, we will attempt to explicate the outlines of the issue and then comment on its possible relevance for qualitative analysis.
This is a seemingly simple question, but one surprisingly not asked in many research contexts. Part of the reason is that evidence is difficult to define. For example, Mautner (1999, p. 184), in a dictionary of philosophy, says evidence is "that which provides a ground for a belief or a theory." Audi (1999, p. 293), in another philosophical dictionary, says evidence is "information bearing on the truth or falsity of a proposition." By way of passing, but eventually relevant for a meaning of evidence, a proposition is a statement, usually in the form of a declarative sentence, which contains (explicitly or implicitly) a "that" phrase; thus, "we believe that most qualitative researchers will find it difficult to define ‘evidence’ clearly" would be at least one type of proposition. A proposition, then, asserts something is the case and we are trying to show that the "information" we possess can establish the truth or falsity of it; or, in another sense, establish its "probability", "likelihood", "warrantability", or something similar.
These last terms, or some form of them, are often used in qualitative analysis under the ambiguous heading of "induction" or "inductive". The term is ambiguous because it is often unclear if "induction" is being referred to as a system of (logical) inference or as a kind of methodological stance in some way(s) unique to qualitative inquiry. In other words, for qualitative inquiry generally, there are often confused notions about what we do, substantively. The confusion is not unique to qualitative forms of inquiry of course (e.g., law), but what constitutes a model for adequate evidence in qualitative inquiry is important because this is, ultimately, the justification for the legitimacy of the conclusions we say we are making.
There are two questions to begin with: (1) what is "qualitative" evidence, and (2) how does it exactly configure into qualitative analysis? The first point requires a distinction that, as previously indicated, is often overlooked; namely, data are not evidence. We may have data but they only become evidence when (and if) we have some sort of model for when and how this takes place. The finding of such "models", as we will call them, then become the answer to the second question. However, such models are varied and complex, and these factors make it difficult to often clearly demarcate what evidence is, not only for the qualitative case but for any case. The overall approach we will be using here relies heavily on Achinstein’s (2001) most recent work. Achinstein, often considered the "Dean" of theorists on evidence, presents a comprehensive and exhaustive account of such theories. Our intent here is to utilize at least some of his general perspectives on what makes evidence evidence. One overriding issue will, of course, be to examine what perspectives may apply to the qualitative case. We will begin with the Logical Constraints (our term) point of view.
To illustrate this approach, let us assume the following scenario. We are interested in studying how women from near-Eastern countries, who are recent immigrants, are handling issues of "cultural adaptation." 5 We have a defensible "purposeful" sample and have decided to conduct indepth, life-history type interviews. While our questions are open-ended, one particular overriding question is something like the following: Do the respondents indicate by similar phrases their experiencing "alienation" in their host country? Leaving aside other considerations such as what "alienation" means in this context, our major concern in the Logical Constraint view of evidence is to see if some logical notion of evidence can be adduced for the above scenario.
The major ideas here are those of necessity and sufficiency, namely if A is a necessary condition for B, then B is not "there," or does not receive a satisfactory explanation in the absence of A — no B without A — in other words. Likewise, A is a sufficient condition for B if, with the presence of A, B always occurs. Of course, a condition can be both necessary and sufficient, and neither necessary nor sufficient. Understandably, one may simply say, "so what?" At least a partial answer is that establishing a possible logical connection is a minimal condition for rationality that is also an intimate part of what it means to "come to know." We are after, in other words, a type of claim, as indicated in the condition phrase "if ... then" which gives us relevant information about assertions we are making. The "if ... then" format is the underpinning of much research thinking in the natural and social sciences, most notably in trying to establish criteria for prediction (Martin, 1997). If A, then B. From a strictly logical point of view, the antecedent ("if") sets the condition for sufficiency, while the consequent ("then") sets, in turn, the condition for necessity. The significance of all this in terms of evidence is that it gives us a clearer picture of how statements relate to one another. In other words, as with all research, we are eventually trying to establish some type of relationship between or among factors of interest. Thus, if something is to count as evidence for something else, the Logical Constraint model may produce some initial guidance.
Returning to our hypothetical research example, let’s assume that the women interviewed all express some indication of "alienation." If we leave it as is, we could at best say there is some "evidence" that is possibly sufficient for these women in their experience of alienation. Furthermore, if whatever other characteristics they may have (e.g., occupational differences) do not influence their perceptions of alienation, the case is strengthened that, for this group from these countries, the phenomenon has been documented. However, the Logical Constraint model for evidence does not exactly work like this because an antecedent condition ("if") has not been stipulated and, although we may still have an idea of evidence and how it "works", a clearer definition of necessary and sufficient conditions must be given.
What is missing is some idea of "variation." For example, we could state the findings in this way: "If middle-Eastern origin, then alienation." The antecedent here is sufficient for the consequent for this sample within this context. But is it a necessary condition,? Is there no alienation in the absence of specific national origin? Such a determination depends on a number of factors related to how we view the nature of evidence. If the cultural adaptation-alienation link (for women) is a more general phenomenon, then women from other regions would need to be considered. Assuming alienation still exists irrespective of national origin, sufficiency would still hold. If differences in national origin produce, on the other hand, differences in alienation, a type of necessary condition may be present in the sense that alienation is not consistently present in the "absence" of a certain type of nationality.
Another distinction may be of some use. For the qualitative research case, we can speak of evidence of a claim versus evidence for a claim. While general, it may still be a distinction with a difference. For many claims in qualitative research it seems to us that qualitative data as evidence are evidence of a claim in the sense of providing for a (possible) sufficient condition. That is, much of the evidence produced by qualitative research methods entails a type of inference known as enumerative induction (Barker, 1989), whereby as many "positive" instances as possible are collected. Evidence of is therefore a weaker claim than evidence for which would provide for a necessary condition. By a "weaker" claim we simply mean that it is easier to suggest alternative (or rival) interpretations, not that the proffered interpretation is false. This does not preclude cases, of course, where a claim for a necessary condition can be made. The value of the Logical Constraint model for qualitative inquiry lies in the fact that qualitative findings are scrutinized for their adequacy as evidence "of" (and, perhaps, "for") something. If we cannot even begin to probe how our findings can be evidence, logically, for an interpretation, then the enterprise does remain, in a negative sense, "subjective." While the logical characteristics of evidence provide an overall framework for evaluation, there is more to the story, some aspects of which are now considered.
In looking at "evidence" and qualitative inquiry, we are not saying that it is only within this field that issues of evidence predominate, but rather that the field has not "stepped back" sufficiently far enough to view its own presuppositions in this regard. And, importantly, it is these presuppositions that, ultimately, must provide an epistemological foundation for the field itself. 6 If we have no clear ideas of how qualitatively-derived evidence functions in the whole process of warrantability, then there is little justification for the entire enterprise itself. The following examples illustrating the complexity of the evidence concept are drawn from Achinstein (2001). After briefly explaining what they are, we will attempt to show parallels to the qualitative case. In doing this, we will use the conventional notation of discussions of theories of evidence where "e" refers to evidence and "h" refers, either broadly or specifically, to an hypothesis, claim, assertion, interpretation, or theory of some kind. Achinstein (2001, p. 13) states the issue clearly as follows:
When the claim is made that something is evidence that a hypothesis is true, what exactly is being claimed? Is there some unique concept of evidence by reference to which we can understand what is being said? I will argue that there is not one concept of evidence but several in use in the sciences.
The term "sciences," by the way, can refer to any field where considerations of evidence are relevant.
Achinstein refers to (initially) four ways of trying to understand the relationship e-to-h which he calls ES (the epistemic situation), Subjective evidence, Veridical evidence, and Potential evidence. One way of understanding these conditions, and their relevance for interpretive analysis is to view them in the context of a situation where one is doing research on some issue, formulates a claim (h) in relation to it, and then says the claim is credible by way of certain other statements (e). What Achinstein is trying to develop, then, are conditions or prerequisites for trying to understand what it means to say that one has evidence and that the evidence stands in some relationship to the claim.
The epistemic situation (ES) refers to the necessity of seeing our proposed e-h claim as being one of beliefs that are in need of some type of justification, especially our beliefs concerning our evidence (e). The point is that if we are to believe that our claim (h) is true, then we must be in a situation (ES) to say that our evidence is true, given our claim. As Achinstein points out (pp. 20-22), the key is to realize that the e-h context is one in that, for there to be a link between the two, there must be a justification for the beliefs (propositions, statements, etc.) about the evidence (e). The justification is in the acknowledgment (and eventual "demonstration") that "e" consists of true beliefs. Achinstein also adds that ES is an "objective" notion of evidence in the sense that whether there is evidence for h does not depend on anyone’s particular belief that it is so. In other words, while ES alerts us to the fact that a justification of our beliefs concerning "e" is a necessary condition for understanding the e-h relationship, whether "e" is justifiably true for h is an objective matter. In this regard, he also indicates that ES may produce a situation where "h" is false, even though the evidence is justified. That is, our beliefs about propositions may be independently true but they are directed toward an "h" that is false. While sounding contradictory, such a situation is not necessarily so: we may justifiably believe that the women in our example are experiencing alienation given what we know about their shifting to other cultural settings, but the hypothesis may nevertheless still be false.
Although it is a rather abstract notion, what ES-evidence helps us to do is to critically focus on the need to think more about "evidence" as a set of beliefs that, in turn, require some form of justification (see Kirkham, 1997, pp. 41-59). Thus, if it is our conclusion that these particular women are experiencing alienation, what evidence, by way of their particular narrative accounts, do we have that they are? We will have "beliefs" that they do, but how can we go about justifying such beliefs? The ES-condition also introduces us to the need to say, or at least consider, the terms "true" and "false" when thinking about the e-h connection. 7
Achinstein’s analysis also develops the ideas of subjective, veridical and potential evidence. Only the general framework of these will be mentioned. Then we will expand on the notion of "potential" evidence as related to the qualitative research case. Subjective evidence is not evidence that is somehow inferior or non-objective. It is simply a condition of evidence which has as its defining characteristic the fact that it is someone’s evidence, be it an individual or group. Subjective evidence is importantly considered in the context of time — it is X’s evidence at time T. According to Achinstein (p. 23), for evidence to be "subjective evidence" it must fulfill the following conditions:
In subjective evidence, the belief condition is strong in addition to requiring that X be in the situation (ES), even though the situation itself is required to say something about e and h. Likewise, subjective evidence does not require e to be true, only that there are (good) reasons for so believing. Veridical evidence, on the other hand, requires h to be true. Potential evidence does not require h to be true, but does require e to be so. It is not, like ES-evidence, relativized to any given epistemic situation, and unlike subjective evidence, potential evidence must not only have e true (or, if e were true, h would probably be so also), but it must provide "good reasons" for believing this to be the case. Believing for good reasons is an important concept that we will return to. What these examples show is that the concept of evidence is both complex and crucial in gaining a clearer understanding of what the conduct of qualitative research is actually about.
All of the considerations mentioned so far, although rather abstract, do have consequences for interpretive inquiry. Some of these consequences will be discussed shortly. However, to further illustrate how the language of evidence is used in the context of what is known as "confirmation theory," the following definitions by a famous early proponent of confirmation theory, Rudolf Carnap (1950) will be given, as cited and discussed by Achinstein (2001, pp. 44-48). They are:
In addition to these definitions, and related to them, is a well know variation called the "positive relevance" definition, given as e "is evidence that h if and only if p(h/e) > p(h)" (Achinstein, 2001). This version of positive relevance is often expanded to include relevant "background" information, (b), so that: e is evidence that h, given b, if and only if p(h/e and b) > p(h/b). Confirmation theory, as described by Achinstein throughout the rest of his book, becomes increasingly complex. Much of this complexity is based, however, on these basic ideas of confirmation. Therefore, we will use only these in discussing issues relevant to qualitative research.
As we see it, a central (perhaps the) question for qualitative analysis is the following: In what way(s) do qualitative research findings "confirm" a claim? The question is more difficult than it initially appears. Why so? Simply, there is no clear-cut or standard way of answering it; yet, much hinges on whether or not we can answer it. Let’s use our previous example to illustrate, changing it just slightly. We have some reasons ("good" reasons?) to assume that our sample of women are experiencing some type of "alienation" and this is produced by the kinds of stress they are undergoing in terms of cultural adaptation issues. The "alienation" is our "h" and the culturally-produced "stress" factors will be our evidence, "e". We, again, gather extensive indepth interview data and make a claim that they are experiencing alienation by way of culturally-based stress factors.
The question is: how can we support such a claim within the context of confirmation theory? ES-evidence here constitutes the possibility that "alienation" is the true interpretation, although it could be false, given that in that situation, she believes e to be true — and provides some justification for that belief. Another way of putting it would be: Given this study, in this setting, with these methods, would it be reasonable to say that e is true and, if true, would support h? We could argue that ES-evidence is at least present for this issue, although indicating how it is justified would take some further explication.
Whether such explication could be produced is exactly what is at issue for the qualitative case. Our ways of judging the interpretation of "stress" in the indepth interviews could, likewise, provide veridical evidence for h, if we assume h to be true and e provides good reasons for the truth of h. In our example, there is the possibility that h is true, but no way to make the case. Now the important point is that if we explicitly claim it to be so (and who doesn’t want to have true statements?), the requirements on e (i.e., to provide really good reasons) becomes more stringent; perhaps involving both necessary and sufficient conditions. More likely, but still begging the question of how explicitly, we could claim some allegiance to our example providing a case of potential evidence. Here we do not claim it to be true, but given how we argue for e, there’s a pretty good possibility h will in effect be true. Now, if we could make a case for potential evidence (a good case), it would also go beyond ES-evidence in the sense that the truth of e would be objective, not relativized to the context of our particular study. That is, the evidence for h is simply the case because the reasons for it are "good" irrespective of our particular epistemic situation. Of course, e is someone’s evidence for h, and so is subjective evidence and may or may not be potential, ES or veridical evidence for h (Achinstein elaborates these possibilities, specifically, p. 31).
For our example, then, if someone would ask us if our evidence for the existence of "stress" is "good" evidence for "alienation", we would need to think about, and then specify, why it is. Again, these rather abstract considerations about the nature of evidence are valuable, and especially for the qualitative case, in that they force us to more closely examine how we make our research claims, either explicitly or implicitly. At best, our example is a candidate for the possibility of potential evidence.
If we now switch the focus to the other ways of conceptualizing the e-h relationship, with applications for the qualitative research case, what do we have? Specifically, can our claim that our evidence for "stress" is potential evidence for "alienation" be given some further credibility as viewed from the perspective of Carnap’s ideas about "probabilistic" theories of evidence? The "probabilistic" theories of evidence that could relate to the qualitative research case include Carnap’s notion of "qualitative" and the "positive relevance" one. The term "probabilistic" is put in quotes since it more accurately refers to the concept "in relation to". Although, the "comparative" and positive relevance definitions do imply some idea of quantification, the intent of all these definitions is to suggest that the assessment of evidence is relational or "comparative" in a broad sense of the term.
For example, the two "qualitative" senses of evidence formulated by Carnap suggest that e is stronger evidence for h compared to h´ and/or e is stronger evidence for h compared to e´. These are interesting formulations since they judge the credibility of the e-h connection by shifting the focus from h to e. Both, importantly, also imply that h´ and e´ can be known. If they cannot, the "qualitative" formulation fails. The term "stronger" is also ambiguous but it does focus on the importance of the comparative aspect. However, both of these formulations fail when we look at our example of stress and alienation because both h´ and e´ are either unknown or could be anything. If we take h´ as simply the negative of h (alienation), the range is much too wide. The same considerations hold for e and e´. The issue is "stronger" compared to what?
If we move to the "quantitative" formulations suggested by Carnap we have, first, "the degree of support or confirmation that e confers upon h is r (where r is a number)" (Carnap, 1950). The positive relevance version is: "e is evidence that h, given b, if and only if p(h/e + b) > p(h/b)" (Achinstein, 2001). The most interesting parallels for the qualitative research case are those attempts to determine some indicator of and strength of an "inter-rater reliability" measure (Boyatzis, 1998). Such attempts, however, may not be desirable for two reasons: first, such measures may undermine the (non-quantitative) uniqueness of the enterprise itself, and second, such measures provide, at best, an indirect confirmation in the sense that one cannot directly assess e on h, but only by having "others" first look at e and then come to some type of agreement or consensus that e does apply to h. Likewise, such assessments leave open the question of how "strong" the agreement needs to be. The positive relevance condition, then, is somewhat better than some of the other evidence models but its application to qualitative research is problematic.
If some form of the "relevance" definition is rejected for qualitative research, what are we left with? How can we make the e-h case? There is the claim that since the entire field of Confirmation Theory is itself filled with examples and counterexamples (i.e., many models of what confirmation as a process really is), the lack of clear application to the qualitative research case is not important. But this conception can be quickly rejected simply because any inference process must consider under what conditions one is entitled to draw a certain conclusion; for example, one in which something (data) becomes evidence for concluding something. The issue is how the e-h relationship can best be characterized for qualitative research, not whether it ought to be.
But this still doesn’t tell us how. One other alternative that holds some intuitive appeal is to see if the e-h relationship could in some way be a deductive one, even though in qualitative research we often (mostly correctly) insist that our approaches are somehow uniquely "inductive," but at times based on the lack of a complete understanding of the nature of inductive logic. Now, even if the overall process of qualitative research is "inductive" in the sense that its conclusions must be "probable" (as all inductive arguments must be), the issue here is whether we could use another concept that is usually regarded as belonging to (loosely speaking) deductive logic. This concept, a very tricky one in its own right, is "explanation" (Pitt, 1988). One notion very widely held is that we do research to provide explanations; we want to "explain" h by way of e. However, to provide an explanation is to employ some ideas from what is usually considered to be deductive logic.
What we are suggesting is that if some version of what is commonly known as the hypothetico-deductive model (h-d) could be applied to the qualitative research case, perhaps this could give us a way of determining how qualitative data become evidence for h. Moreover, the h-d model could be valuable because it is basically a logically based one, and our opening argument was that perhaps the best (and only) case that can be made for the qualitative research enterprise is just such a "logic-based" one.
Using Achinstein’s (2001, pp. 147-149) definitions once more, we have: (1) "Basic hypothetico-deductive condition: If h together with b (i.e., background evidence) entails e, but b by itself does not, and if e is true, then e is evidence for h, given b." One of the key logical terms in the definition is "entails." This is a strong and basic logical feature which says that A entails B, if and only if, it is logically impossible for both A and not-B to be the case. Another way of expressing the relationship would be: B can be logically deduced or "follows" from A. For our specific example, we would want to know if we could say, "does Alienation (along with Background evidence such as setting or context) entail cross-culturally induced stress?" One problem with the h-d approach, as a way of evaluating evidence, is that there may be competing "hs" that can equally well be consistent with the other conditions. Thus, instead of "alienation," we could possibly substitute some thing like "differences in socio-economic status" that, with the same background conditions, could entail e and e could be true.
Achinstein (p. 148) modifies the h-d condition by what he calls the Basic Explanation Condition: "If h, if true, would correctly explain e, and if e is true, there is evidence that h." Here, the term "explain" is included, but as Achinstein states (p. 149), "Although H entails e it does not explain why e is true. Explanation is a richer, more demanding idea than entailment." So, even if we could "predict" with complete accuracy (i.e., show entailment) that "stress" follows from "alienation," we might not be able to explain why. Borrowing the usual example from quantitative data analysis, a perfect correlation (r = +1) between two variables does not necessarily "explain" the relationship.
All of these examples from Confirmation Theory suggest that the basic e-h model is more complex than it appears. Furthermore, for the qualitative research case, the one of interest here, the basic confirmation models do not seem to give us a clear cut way to establish or assess what it means to make the case for the how of qualitative analysis. What then is possible?
How can or does qualitative data become evidence for a claim? Or to use the language from confirmation theory, are there "good reasons" to believe that qualitative research findings are evidence? We conclude that when all is said there remains only one model that is capable of doing so, and one that combines the logical distinction of necessity and sufficiency (i.e., Logical Constraints) with what is known as Enumerative Induction. This is a non-deductive form of reasoning, where, given a similarity of instances, we attempt to conclude something. Sometimes called a "generalization," like all inductive arguments, the "conclusion" in enumerative induction goes beyond the instances that are cited. Or put a little differently, in drawing our conclusion we believe our data do provide some "good reasons" for such a conclusion.
The idea of "generalization" here is not the research-based idea of "generalizable to other settings," but rather the recognition that any form of inductive argument simply extends its conclusions based on but going beyond the "relevant" data. The conclusion is "probable" but if considered "strong" evidence then its "generalization" is more plausible.
Perhaps a better way of understanding this is to look at one of the classic "canons" of induction by John Stuart Mill (1875/1952) in his System of Logic. One of these is referred to as the Method of Agreement and basically it is about trying to find "common instances" that are (possibly) related to some claim, phenomenon, issue, and so on. The more "positive instances," the stronger potential evidence for a claim. A basic model for the Method of Agreement might look like the following (from Manicas & Kruger, 1976, p. 257):
E (y1, y2,...) E (y1, y2,...) E (y1, y2,...) 
A E The uppercase letters on the left refer to the conditions that either are "there" or could be, and the E is the claim we are making. The small subscripts refer to other factors that may be affecting the situation (directly or indirectly) but for which we cannot account. The conclusion is that some factor, A, is probably responsible for E.
There are two important ideas here for the qualitative research case, the logical and evidentiary. Returning to our example, let’s assume two things: first, that our in-depth interviews have given us reason to believe that these women are experiencing alienation; and second, from the transcripts we also adduce that some type of stress may be responsible. How we decide that both are present is, of course, one of the most critical issues in qualitative analysis and presents a complicating factor for the e-h analysis. But for this context, suffice it to say we have identified our two factors, alienation (E) and stress (A). In our data analysis, we consistently find A with E. Sometimes other factors seem to be present (e.g., B) but not in all cases whereas A is.
Here we can return to the necessary and sufficient condition distinction. The factor A may be a sufficient condition for E. We cannot completely rule out some other factor (e.g., B) as being a necessary condition, but for these data, in this context, sufficiency is more plausible. Showing such a logical connection is no easy matter, and doing so increases our belief in the interpretation. Thus, the Method of Agreement is one possible explicit justification for the qualitative research case. Moreover, the case is strengthened to the degree it is possible to add other similar instances. Of course, the finding of even one "negative" case undermines the force of the sufficiency argument. If such a relationship can be shown, the qualitative data then become evidence (in a logical sense) for "confirmation." It may be "objective" evidence also since "A" could be the determining factor irrespective of what one believes about it. However, gathering such evidence in a particular way, within a given context, does provide ES-evidence for the claim. Given that it is someone’s evidence (i.e., "subjective"), and that e provides "good reasons" for its truth, it could provide potential evidence for the claim as well. Furthermore, if one chooses to view e-h indirectly by way of some type of quantification (e.g., "reliability") measure, a positive relevance interpretation could be minimally supported.
Lastly, we would like to mention a situation which complicates the qualitative research case but also argues for its uniqueness. The complexity relates to the fact a two-step process is involved in making the evidence argument. The first step is often overlooked by qualitative researchers as something in need of explicit comment. It involves the realization that the process that generates categories such as "alienation" and "stress" is itself an "evidentiary" process in need of explanation and justification. For example, the question becomes: how, from the transcripts of the respondents, did the researcher determine those linguistic units that are (then) necessary and/or sufficient for the creation of the category? It is only after this process is justified can the other considerations about evidence be made. Again, some type of reliability measure can be employed at this stage, but it raises the same questions as when it is applied at a later stage.
These are some of the major epistemological issues that, we believe, still confront qualitative research methods. Saying that "confirmation" is the central issue and that the logical constructs of necessity and sufficiency are "foundational" for our enterprise does not seem to be saying much, after all. Perhaps so, but how can we even proceed if there is no explicit notion of how qualitative data become evidence? Our belief is that we cannot. However, recognizing what is involved and being willing to discuss it in the context of actual research studies constitutes an important first step in showing the credibility of the qualitative research paradigm.
1. Another way of making this point is to draw a distinction between description and explanation (Lipton, 1993). For qualitative data analysis, the term "description" can be used in a double sense: as what is being reported as data and how clearly we can "describe" the inferential processes being used in the analysis. The latter sense is the one being used here. Moreover, a given "explanation" may or may not follow from a particular description. Back to text
2. The ontological side of qualitative research methods is often not analyzed, in our view, sufficiently. For instance, even if a "constructivist" view is seen as compatible with qualitative analysis, it leaves undefined why this is so. Put a little differently, can realism be compatible with qualitative methods as well as constructivism? If constructivism is not, does this imply qualitative methods wholly "create" their own reality? (see Searle, 1995). Back to text
3. One of the problems of abduction is to fail to see that there is the possibility of the fallacy of affirming the consequent, even if the conclusion is put in probabilistic terms. Back to text
4. It should be noted that we are not saying formal epistemological theories are irrelevant to the evidence issue here, but only that it would detract from our purpose. In a broader sense, issues of evidence are relevant to all epistemological theories whether correspondist, verificationist, pragmatic or deflationist. Our own preference, in passing, is a form of reliabilism within the context of Goldman’s (1999) notion of "veritism." Thus, we are not either ignorant of epistemological theories or of their potential value concerning the issues at hand. Back to text
5. One of our doctoral students is in the process of formulating this topic. Back to text
6. By an "epistemological foundation" we, again, mean the way "coming to know" is conceived, but not some particular epistemological theory. Back to text
7. We are not saying ES situation will determine some particular notion of "truth" or falsity but only that it brings to the fore the non-avoidance of these terms when the issue of how qualitative data become evidence is raised. Again, various epistemological theories will have different takes on the nature of true and false beliefs. Back to text
Achinstein, P. (2001). The book of evidence. Oxford: Oxford University Press.
Archer, M., Bhaskar, R., Collier, A., Lawson, T. & Norrie, A. (Eds.). (1998). Critical realism: Essential readings. London: Routledge.
Audi, R. (Ed.). (1999). The Cambridge dictionary of philosophy (2nd ed.). Cambridge, MA: Cambridge University Press.
Barker, S. F. (1989). The elements of logic (5th ed.) New York: McGraw-Hill.
Barnes, J. (Ed.). (1984). The complete works of Aristotle: The revised Oxford translation (2. Vols.). Princeton, NJ: Princeton University. Press
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage.
Carnap, R. (1950). Logical foundations of probability (2nd ed.). Chicago, IL: University of Chicago.
Edelson, M. (1984). Hypothesis and evidence in psychoanalysis. Chicago, IL: University of Chicago Press.
Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. New York: Macmillan.
Goldman, A. I. (1999). Knowledge in a social world. Oxford: Oxford University Press.
Hanson, N. R. (1972). Patterns of discovery. Cambridge, UK: Cambridge University Press.
Hartshorne, C., & Weiss, P. (Eds.). (1998). Collected papers of Charles Sanders Peirce (Vols. 7-8). Cambridge, MA: Harvard University Press.
Kirkham, R. L. (1997). Theories of truth: A critical introduction. London: Blackwell.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.
Lipton, P. (1993). Inference to the best explanation. London: Routledge.
Loux, M. J. (2002). Metaphysics: A contemporary introduction. London: Routledge.
Manicas, P. T., & Kruger, A. N. (1976). Logic: The essentials. New York: McGraw-Hill.
Martin, R.M. (1997). Scientific thinking. Peterborough, Canada: Broadview Press.
Mautner, T. (Ed.). (1999) The Penguin dictionary of philosophy. London: Penguin Books.
Miller, S. I. (1990). Confirmation and qualitative evidence instances: Justifying the use of qualitative research methods. Quality and Quantity 24, 57-63.
Mill, J. S. (1952). A system of logic (8th ed.). London: Longmans-Green. (Original work published 1875).
Morton, A. (1997). A guide through the theory of knowledge. London: Blackwell.
Moser, P. K. (1993). Philosophy after objectivity: Making sense in perspective. New York: Oxford University Press.
Pitt, J. C. (Ed.). (1988). Theories of explanation. Oxford: Oxford University Press.
Rappaport, S. (1996). Inference to the best explanation: Is it really different from Mill’s methods? Philosophy of Science, 63(1), 65-80.
Searle, J. (1995). The construction of social reality. New York: Free Press.
Shelley, C. (1996). Visual abductive reasoning in archeology. Philosophy of Science, 63 (2), 278-301.
Upshur, R. (1997). Certainty, probability and abduction: Why we should look to C. S. Peirce, rather than Godel for a theory of clinical reasoning. Journal of Evaluation in Clinical Practice, 3 (3), 201-206.
IJQM Editor: Janice Morse