Although qualitative research methods have grown increasingly popular,confusion exists over how their quality can be assessed and the idea persists that qualitative research is of lesser value when compared to quantitative research. Quantitative and qualitative research have different historical roots and are based on very different concepts, yet the dominance of positivist ideas about what constitutes good quality, valid research in psychology has often led qualitative research to be evaluated according to criteria, that are designed to fit a very different paradigm. Inevitably, the diverse perspectives which use qualitative methods and their differing views on how people should be studied mean there is disagreement and controversy over how quality should be evaluated. Despite this, it is seen as important to develop common criteria which allow the quality of qualitative research to be evaluated on its own terms.
When I was first introduced to qualitative methods during my undergraduate course, I was excited by the possibility of being able to work with words, which I am fascinated by, in such an in-depth way. The ideas behind these methods also opened up the opportunity to ask different kinds of questions and explore aspects of psychology that I had sometimes felt were missing from quantitative research. However, the main qualitative project had been preceded by courses in cognitive and biological psychology and I found that getting to grips with these concepts entailed a shift in thinking about the ways that research can be carried out and the impact this has on the knowledge that is produced. In one sense the qualitative projects had much in common with the quantitative assignments. Familiar steps were followed of reviewing the existing literature, developing research questions, designing a study, recruiting participants, collecting and analysing data and interpreting the analysis in the light of research questions and existing research. In another sense it seemed to break the rules that had been so essential for the quantitative projects, such as the importance of statistically valid sample sizes, ensuring objectivity of the researcher, control and manipulation of variables and report writing in the third person. There were still ‘right’ ways of carrying out a good quality study but these were based on some very different ideas about how people should be studied and knowledge should be produced.
Should qualitative studies be evaluated by the standards of quantitative research?
According to writers such as Wendy Hollway (2007b) and Lucy Yardley(2011), the dominance of the quantitative approach in psychology since the 20th century has meant that whether or not research is judged as valid has often been erroneously based on the standards set down for quantitative research. A major problem with this is that these two approaches are based on very different conceptual frameworks. The criteria for assessing the validity of quantitative research are rooted historically in the desire for psychology to achieve scientific status and were modelled on principles derived from physics. Cognitions and behaviours were viewed as individual and universal, so experiments could be carried out which controlled and manipulated variables to reveal knowledge. As long as a statistically valid sample was used, the results from the experimental sample could be generalised to the population (Hollway, 2007a). Based on such concepts, it makes sense to consider factors such as sample size and objectivity when considering validity, but the underlying ontological and epistemological assumptions of qualitative research are different. Qualitative research has its origins in disciplines including sociology and anthropology, and emphasises factors such as the importance of social context and the impossibility of achieving an entirely neutral and objective standpoint. Hollway (2007b) argues that because qualitative research comes from a different paradigm its principles must be understood on an independent basis from those arising from quantitative research. An example of this is the concept of generalisability. Instead of statistical generalisation, qualitative researchers generally aim for theoretical generalisation, providing insights that may be useful in similar contexts; small samples may often be appropriate for this and in many cases a statistically valid sample would be impractical. Reicher and Taylor (2005) argue that to assume one approach is correct is to judge the methods of other perspectives whilst ignoring their positions. Instead, it is important to understand perspectives according to the terms of rigour and coherence for each position.
Judging qualitative research according to the principles of quantitative research not only leads it to be seen of lesser value if it fails to meet these criteria but is likely to overlook the value of characteristics which are of particular importance to this approach. Howitt (2010) points out that a qualitative study that fulfils the criteria used to evaluate quantitative research such as reliability, validity and replicablity, may exclude aspects of central importance to qualitative research. By these standards, a study using qualitative analysis which assumes the researcher is neutral and unbiased may be seen as valid because it meets the criteria of objectivity. Yet, this would be to ignore the importance most qualitative work places on the interactive role of the researcher in producing the data in combination with participants, losing the valuable insights that reflexive analysis can bring.
Can quality concepts in quantitative research be at all useful when evaluating qualitative research?
Unsurprisingly, there is disagreement over this and views depend largely on the researcher’s position on reality (Howitt, 2010). According to Howitt (2010), there are some ideas that may be relevant when looking at quality in all types of research, including persuasiveness and coherence of the researcher’s argument and whether the claims of the researcher are convincingly established. However, the diversity of backgrounds and philosophical bases of qualitative research methods mean several of these ideas are more applicable to some approaches than others. Where similar concepts are useful they cannot always be directly transferred, but need some adjustment to take account of the different underlying assumptions. For example, the conventional view of validity assumes there are real and fixed phenomena (such as personality traits) that can be revealed and measured. Many qualitative researchers would disagree with this stance. Instead, assessing validity of qualitative research may focus on validity of the analysis undertaken by examining its goodness of fit with the data. Potter (1998) has described this as ‘justification of analytic claims’ (as cited in Howitt, 2010).
It is not only the broader ontological and epistemological assumptions of qualitative and quantitative approaches that differ; different perspectives which use qualitative methods are based on different and sometimes conflicting underlying assumptions. From a social psychoanalytic perspective the person is understood as a conflicted psyche interacting with the social world, so analysis involves interpretations of unconscious, unspoken dilemmas. In contrast, from a discursive psychological point of view the person is socially constructed through talk in interaction; analysis focuses exclusively on talk and it would be incongruous to look for hidden meanings. Such diversity leads to different research questions, numbers of participants and methods of analysis being appropriate for different perspectives. This means that criteria for judging validity of qualitative research are necessarily less standardised and involve more selectivity than those for quantitative research. It is therefore, important for qualitative studies to demonstrate cohesiveness by asking questions, using methods and making interpretations that are appropriate to the theoretical approach taken and to be clear about the process so that it is open to evaluation (Yardley, 2011). For my own project, I took a discursive psychological perspective so asked questions about how the concept of fairness was constructed through discourse. I then analysed the data by focusing on the actions performed by the talk. The project would not have made sense as a coherent whole if I had then made interpretations about the unconscious dilemmas of my participants.
So, what makes a good quality qualitative study?
Despite this diversity of qualitative perspectives, common criteria have been developed to enhance and assess validity in qualitative research (e.g. Taylor, 2001, cited in Howitt, 2010 & Yardley, 2000). Yardley’s (2011) core principles for evaluating the validity of qualitative research are: sensitivity to context, commitment and rigour, coherence and transparency and impact and importance. Yardley emphasises that although it is important that value is demonstrated, validity criteria should be seen as highlighting quality issues rather than providing a rigid checklist that restricts the freedom and flexibility of researchers. Such criteria acknowledge the different conceptual framework underlying most qualitative research and the variety of ontological and epistemological viewpoints and methodologies.
Understanding how to ensure and recognise quality in qualitative research can be confusing, especially for those coming from a quantitative background. It requires an understanding of the conceptual framework underlying qualitative research generally as well as the specific perspective from which the research comes. Doing so allows the research to be judged according to its own standards and highlights that these approaches are of equal value when understood in their own terms. I personally found that gaining an understanding of these issues helped improve my critical thinking when considering all types of research.
References
Hollway, W. (2007a). Social psychology past and present. In W. Hollway, H. Lucey, & A. Phoenix (Eds.), Social psychology matters (pp. 1-29). Milton Keynes: The Open University.
Hollway, W. (2007b). Methods and knowledge in social psychology. In W. Hollway, H. Lucey, & A. Phoenix (Eds.), Social psychology matters (pp. 33-64). Milton Keynes: The Open University.
Howitt, D. (2010). Introduction to qualitative methods in psychology. Harlow: Pearson.
Reicher, S. & Taylor, S. (2005, September). Dialoguing across divisions: Similarities and differences between traditions, The Psychologist, 18(9), 547-549. Retrieved from http://www.thepsychologist.org.uk/archive/archive_home.cfm?volumeID=18&editionID=127&ArticleID=919
Yardley, L. (2000). Dilemmas in qualitative health research. Psychology and Health, 15, 215-228.
Yardley, L. (2011). Demonstrating validity in qualitative research. In J. A. Smith (Ed.), Qualitative psychology: A practical guide to research methods (pp. 234-251). London: SAGE.
