Qualitative Thematic Analysis of Transcripts in Social Change Research: Reflections on Common Misconceptions and Recommendations for Reporting Results

My impression of the extant literature is that most of the writing on QTA has focused on the fields of psychology and healthcare. To my knowledge, there is little explicitly addressing the field of communication, my own research area, and so this review attempts to address the usefulness of QTA across a variety of social science fields, including communication.

Within many social science fields, there is a tension between quantitative, more positivistic approaches to studying human behavior and qualitative and more constructivist or critical epistemologies (Goertz & Mahoney, 2012). This tension has manifested in confusing guidance for scholars. Within communication studies, even though many conduct qualitative analysis, the field is often associated more with positivist perspectives and traditional quantitative content analysis methods (Graneheim & Lundman, 2004; Vaismoradi et al., 2013). To be clear, this paper focuses on those scholars approaching research from a constructivist, interpretive, or critical perspective, asking questions that cannot be easily quantified. That said, content analysis exists along an epistemological continuum, and it is possible for multiple approaches to be combined (Braun & Clarke, 2021).

Thematic analysis and qualitative thematic analysis are terms that are often used in papers but can mean many things. It is clear there remains much confusion over what it is, and perhaps more importantly, what it is not. Thematic analysis is often used to mean content analysis, and can therefore be qualitative, quantitative, or a combination of the two (Braun & Clarke, 2021). Qualitative thematic analysis (QTA), in its most basic form, is content analysis or thematic analysis that is qualitative. While several authors have created specific guidelines for how to conduct QTAs, most notably Braun and Clarke (2006), it remains a “flexible” tool, which is part of why so many are drawn to it (Braun & Clarke, 2006; Vaismoradi et al., 2013; Braun and Clarke’s 2006 oft-cited article usefully lays out a step-by-step guide for conducting QTAs while still maintaining flexibility in approach, which I will not fully review here, but their recommendations include creating initial codes for the data and then sorting those codes into themes. As codes or topics (more descriptive) become themes (more interpretive), and themes are potentially combined to create larger themes, the analysis takes on increasingly greater levels of interpretation and abstraction.

Given how broad QTA is, it is perhaps more useful to clarify what QTA is not. It is not coding using a priori hypotheses or codebooks (Braun & Clarke, 2021; Vaismoradi et al., 2016). It is not (typically) analysis of institutional publications or media such as newspapers, television shows, or government documents. It does not require multiple coders and measures of intercoder reliability (Braun & Clarke, 2021; Morse, 1997; Vaismoradi et al., 2013).

The issue of intercoder reliability is contentious, and others would disagree. Some suggest that more coders are better and/or that intercoder reliability is a desirable feature of QTA (Castleberry & Nolen, 2018; Maguire & Delahun, 2017). Some conduct studies that combine deductive coding (with multiple coders, based on theoretical frameworks, hypotheses, and pre-defined codebooks) and inductive coding (developing themes based on the text without a priori ideas) and so conduct QTA in tandem with deductive approaches that may benefit from intercoder reliability (Fereday & Muir-Cochrane, 2006; Xu & Zammit, 2020). This may cause some confusion around whether QTA requires multiple coders, but to study messy, complex social phenomena, we must study real people and their real words. We need to take the data as they stand without trying to fit them into preconceived hypotheses or theories (though theories can be used in the analysis, as I will discuss further on). This means there is enough interpretation involved that expecting multiple individuals to read the data the same way is not usually possible.

QTA is not the same as critical discourse analysis (CDA), even though both techniques can be used with a critical lens and may address some of the same societal concerns. They are, in a sense, two sides of a coin, with QTA typically (though not always) analyzing the voices of those with lived experiences of social phenomena and CDA typically (though not always) analyzing media texts produced by powerful elites (e.g., newspapers or Hollywood films).

Part of the confusion around QTA stems from different epistemologies of researchers and different beliefs regarding what kinds of questions QTA can or should answer. This paper addresses researchers who use QTA to understand why particular social phenomena occur. In the social scientific paradigm, causal hypotheses for explaining human behavior are typically defined, tested, and refined. On the other hand, researchers with more interpretive or constructivist starting points ask broader research questions, without formal causal hypotheses, and with the understanding that interpretation of findings is subjective, and thus value laden. Researchers grounded in more interpretive or critical epistemologies are drawn to questions that produce messy, complex answers about why humans behave as they do – questions for which it is difficult or impossible to create quantitative variables upon which to conduct statistical analyses, and more difficult still to develop coding instructions that would pass intercoder reliability tests. Scholars working within interpretive or critical paradigms are likely to have conscious or unconscious hypotheses or assumptions going into research, but do as best as they can to keep those assumptions in check and to remain open to findings that do not fit within expectations. Acknowledging one’s epistemological starting point explicitly when reporting findings is thus helpful for establishing expectations regarding how the analysis should proceed and what conclusions can be drawn.

There is widespread acknowledgement among those who work with this method that it faces skepticism (Castleberry & Nolen, 2018; Graneheim & Lundman, 2004; Lawless & Chen, 2019; Nowell et al., 2017). In my view, this is even more true when the research claims to represent the voices of marginalized and underserved populations. This is the case for several reasons, namely: (1) underserved groups are infrequently heard from in the public sphere; (2) outsider academics are not seen as legitimate representatives of marginalized groups; (3) QTAs stemming from research on marginalized populations often lack measures of intercoder reliability; (4) findings from small communities, especially ones that feel most unfamiliar, are not considered to be generalizable; and (5) findings often appear cherry-picked.

There are many ways to define and categorize themes, sub-themes, topics, categories, and codes (among other terms used). Vaismoradi et al. (2016) and Braun and Clarke (2021) usefully outline distinctions between codes and topics, which tend to be explicit, descriptive, and aligned with manifest content (e.g., ‘comments about length of wait times at clinics’), and themes, which are more interpretive, latent, and get at “the essence of what the interviewees are saying” and their experiences (Vaismoradi et al., 2016, p. 102). These pieces are useful for trying to understand the variety of terms used to code and categorize data in QTA.

While approaches may differ, they hold in common the suggestion that scholars need to approach the data many times, re-reading, recoding, refining codes, and comparing codes until things fall into place and the researcher feels they can tell some kind of narrative about what is happening in the data. My own process is to (very) roughly code some large chunk of the text (usually between 50% and 100%) until I feel I am starting to see patterns emerge or reaching saturation with regard to at least one research question. I then like to start drawing – creating mind maps, often referred to as thematic maps, concept maps, hierarchies, or visualized axial coding. (See Vaismoradi et al. (2016) for a good discussion of different levels of abstraction.) This aids in clarification, but others have different strategies. I also like to draw mine on paper, although there are a variety of digital tools to assist with this process. Figure 1 represents the final concept map used during my research on COVID-19 vaccine hesitancy (though this was probably the 50th iteration of it). This process is conducted in tandem with coding new data, recoding old data multiple times, and/or refining codes, themes, and sub-themes until things seem to feel reasonably well settled. This process of constant comparison between new and old data (Glaser & Strauss, 1967) is time consuming, but necessary to ensure the researcher’s interpretations are consistent throughout the dataset and reflect what is occurring in the data.

Using data analysis software such as NVivo or Atlas.ti greatly facilitates the coding process, since in QTA a single sentence may fall under more than one code, category, or theme, making this difficult to track without such software. (Some scholars advocate mutually exclusive categories or codes, meaning that the same sentence cannot fall into two categories or codes, but the general consensus is that this is only the case if you intend to conduct statistical analyses (Insch et al., 1997).

We have already discussed above that intercoder reliability need not be a goal of QTA. An example from our COVID-19 project illustrates how the same data can result in two entirely different conclusions. During our research on COVID-19 vaccination hesitancy, a pattern I saw in the transcripts was that pop-up clinics that successfully persuaded individuals to vaccinate sometimes used messages that emphasized convenience, along the lines of: “You could do this right here and now and be done with it, so if you were thinking about it but there hasn’t been a good time, now is a good time.” Individuals could decide in the moment, without an appointment, at no cost, with no identification or insurance needed. This concept, vaguely formed, framed many of my codes, as one of the themes that emerged from the project was that many members of the underserved communities we were working with, many of whom had already had Covid (for a variety of structural reasons related to housing and to who were considered essential workers), faced so many other challenges in their daily lives that getting vaccinated was simply not a priority. But this was my own reading, and was admittedly influenced by my previous research.

Others might have interpreted the data differently. A comment within the transcripts like “people don’t want to leave their neighborhood to go to a pharmacy 10 minutes away where you have to make an appointment and have to show insurance,” which to me lent credence to the idea that inconvenience and time constraints were impediments to vaccination, could have been interpreted in almost the opposite way – that convenience (a pharmacy only 10 minutes away, for which you could easily make an appointment online ahead of time, and was free as long as you had insurance) seemed to not be enough to promote vaccination.¹ Indeed, some people I spoke with who disagreed with my findings stated this: the fact that some were loath to travel 10 minutes to a pharmacy suggested that inconvenience was not a barrier, and thus that interviewees likely had more political reasons for not getting vaccinated, or believed in conspiracy theories, and did not wish to tell me. If inconvenience was even suggested as a reason for vaccination delay, skeptics felt, it was probably an excuse made to appease the interviewer.

In other words, different researchers may have different takeaways from the same dataset. If intercoder reliability cannot serve as a proxy for rigorous research, the individual researcher must make the case that their personal, perhaps even biased, read of the data is legitimate. This is the topic of the next section.

Many papers say that the method of “qualitative thematic analysis” was employed, or that they “analyzed” transcripts of interviews or focus groups, without describing what they mean, or using only vague descriptions (Castleberry & Nolen, 2018; Vaismoradi et al., 2013). This immediately raises red flags, suggesting the analyst read the transcripts once or twice, followed their gut, and that was the end of it. Authors should include, at the very least, a name for the method they used to analyze the data (e.g., qualitative thematic analysis, reflexive thematic analysis), a description of the dataset analyzed (e.g., transcripts of 25 in-depth interviews), a brief description of how the method was carried out (e.g., whether themes were developed from initial descriptive coding of all the data, whether techniques such as constant comparison or axial coding were used), which software program (if any) they used to analyze the data, how many coders were used, and why they considered this a legitimate method for the research questions at hand. While scholars typically include a method section in their papers, the method often focuses on protocols followed for data collection rather than data analysis, suggesting an explanation of data analysis protocols is unnecessary.

The single qualitative thematic analyst need not shy away from using a single coder, but should plainly acknowledge and defend it. A reader who notices a lack of explanation for having only a single coder will likely have their antennae go up, especially if they are accustomed to deductive or quantitative methods. Make it clear that having a single coder is an acceptable, often necessary, way to conduct QTA.

As discussed above, cherry-picking is a real phenomenon, and can lead to outsiders dismissing findings. Saying that a phenomenon exists in the world and finding one quote to illustrate it does not read as rigorous research. You can say it emerged as a theme, as if it were lying there waiting for someone to unearth it, but readers are unlikely to believe it without a preponderance of evidence.

Unfortunately, I have found there is an extreme discomfort regarding quantifying frequencies in QTA. Some of the most-referenced articles on QTA specify that counting or quantifying themes should not be done. They argue that counting should not “stand as a proxy for significance” (Vaismoradi et al., 2013, p. 404), or that if the research aim is “to build conceptual understanding, then there is little need for quantifying qualitative data” (Thompson, 2022, p. 1416). I have received pushback from scholars when making the suggestion on paper drafts. For some scholars, there seems to be a sense that the research is not as pure or meaningful when it is reduced to mere numbers, that it is reductive, that participants are somehow discounted, or that scholars are pretending to be within the social scientific paradigm when they should not have to pretend.

I strongly disagree with this position and put forth that arguments made regarding themes are much more convincing when numbers are attached. The number of times a theme is repeated by unique individuals is suggestive of its importance (as is how strongly it is emphasized by a particular speaker or speakers). This is similar to what Owen (1984) refers to as recurrence (how often a theme arises), though Owen does not explicitly advocate for considering recurrence of a theme across unique individuals, nor for reporting the number of recurrences.

Recurrence is an indicator of the strength and salience of a theme. For example, if I state that a theme in my interviews regarding healthcare access is that patients of color experience racism during hospital visits, and I include a single quote of an interviewee who said this, this finding is almost certain to be dismissed by skeptics, who will likely believe I entered the research with an a priori agenda. But if I state that five of 20 interviewees experienced this, the finding instantly becomes harder to brush aside. This may seem like a simple point, but I have been surprised by not only how infrequently scholars report such numbers, but also at how resistant they are to reporting numbers when requested to do so. I do not mean to imply that a resistance to counting suggests fraud, but rather that there is a perceived and unhelpful epistemological clash between quantification and qualitative, inductive methods.

This recommendation also applies to media analyses, such as thematic analyses or framing analyses of press coverage. Too often I see papers that state a particular type of news event, for example, is framed in a certain way. One or two examples are included to ‘prove’ the point, but no information is given as to how common this framing was in the data set. Without this, I am quite skeptical as to whether this was a common trend.

Counting how many times a code appeared is not meant to ‘prove’ a point, nor to make the research ‘appear’ more scientific. The act of counting in itself does not make something positivistic or ‘truthier.’ All it does is provide an honest picture of what is happening within the data, so that readers can decide for themselves how much credence they give to the findings. It may be the case that a theme only appears once, but because of context the analyst still thinks it is important or provides nuance that is useful. Once again, the point is not to provide rules for which findings should count, but to be as honest as possible. While Owen advocates counting recurrences within a single interview, I recommend counting and reporting the number of unique individuals mentioning a theme, as one interviewee mentioning something a dozen times is quite different than a theme recurring in 12 of 20 interviews.

Provide examples of themes by quoting a variety of unique respondents verbatim. Likewise, for discourse analyses or framing analyses of media, use examples and provide references for the media sources that are being excerpted, if possible. This suggestion is not revolutionary; Braun and Clarke (2006, p. 95) note a lack of examples as one of the five major pitfalls of QTA. However, their warning remains often unheeded. I still see many QTAs and framing analyses in which a single quotation is given and the audience is expected to believe that the one quotation is symbolic of the lot. Skeptics are likely to view single quotations, or quotations coming from one source, as cherry-picked. In my own experience of others reading my papers, I have also often been told that this is the most interesting and engaging piece of the research! It is unsurprising that the real words of vulnerable populations are more interesting than the verbiage of an academic, but it is worth remembering, as it is unfortunate that so much of their own perspective is eschewed in favor of that of the verbose scholar.

Of course, word limits imposed by journals pose a challenge for including lengthy interview excerpts. Some scholars advocate for journals allowing greater word counts for QTAs (Braun & Clarke, 2021; Levitt et al., 2017) for this reason, especially now that most journals are read online. Another option would be to provide a file of more extensive excerpts on the journal’s or authors’ websites.

This is a newer idea, and I admit that I have not yet done this (but intend to for future projects). It only occurred to me during my last research project, and it was too late to try this as it would have violated the consent agreements. While it is increasingly common, and in some cases even required, to publish datasets online (see, for example, the European Commission’s open data policy (“European Commission,” nd)), most of this data is quantitative. Publishing qualitative research data online is not totally novel, but it remains rare. The Qualitative Data Repository was created in 2015 (housed at Syracuse University in the United States) to, in part, make “the process and products of qualitative research more transparent” (“About the Qualitative Data Repository,” para. 4). The UK Data Service (2023) provides a list of international qualitative data archives.

This is also akin to the concept of auditing to increase transparency (Guba & Lincoln, 1989; Nowell et al., 2017). Auditing requires providing a paper trail that discusses discussion-making processes and other crucial points in the analysis process. Similarly, publishing field notes or journaling thought processes have also been suggested as ways to promote transparency (Guba & Lincoln, 1989, p. 242; Nowell et al., 2017; Vaismoradi et al., 2013). However, I have my doubts as to whether more than a very small handful of people ever look at these – the very thought of trying to read someone else’s field notes gives me a headache. I am also skeptical that simply saying a log of decisions was audited would provide me comfort if I did not find the findings believable to begin with.

I would argue that concepts around auditing are outdated in the age of increasing digital access. Auditing got its start as a way to promote transparency in qualitative research before the internet was widespread. Guba and Lincoln (1989, p. 243) describe the audit as a process in which these notes are literally handed over to an auditor who “attests to the quality and appropriateness of the inquiry process.” At that time, of course, publishing transcripts whole hog on the internet, so that anyone could perform their own ‘audit’ of the material, was not a possibility.

Now, however, transcripts that would previously be ignored by most could be used by others interested in the topic to conduct their own original analysis of the data. This serves the double benefit of making findings more transparent (and therefore more rigorous) while also increasing access to the voices of the marginalized, which could increase research relevant to these groups and amplify their voices.

If feasible, verify themes with your interviewees or affected communities. They will be able to tell you whether you are hitting the mark (Castleberry & Nolen, 2018; Guba & Lincoln, 1989). Guba and Lincoln (1989, pp. 238–239) refer to this as “member checks.” They state, “If the evaluator wants to establish that the multiple realities he or she presents are those that stakeholders have provided, the most certain test is verifying those multiple constructions with those who provided them” (p. 239). This will likely not appease those critics of QTA who believe respondents lie, but it will help appease those who question the researcher’s subjective interpretation of findings.

Better yet, include members of the affected community in the analysis. It is increasingly recognized that affected communities have insights that are invaluable for building knowledge (Kogen, 2022; Liebenberg et al., 2020) and furthermore that projects that include affected communities in knowledge building have greater societal impact than those that do not (van den Besselaar et al., 2018, p. 57).

This recommendation addresses the perceived generalizability problem inherent in qualitative analysis, as discussed in the first part of this paper. While many scholars, even those who conduct QTAs, suggest they are not useful for building theory, my position and that of many others (e.g., Ayres et al., 2003; Braun & Clarke, 2021; Garvey & Jones, 2021; Simons, 2015), is that qualitative research helps us understand the complex, messy, essence of social phenomena, and that this is fundamental for building theory about human behavior or about societal structures. The very point of a theory is that it is at least somewhat generalizable under certain circumstances.

For example, in my research with IPV survivors, which focused on clients of one specific organization, one finding was that internal communication, such as through private journaling, helped survivors feel more empowered. It could, of course, be the case that journaling for survivors who were clients of a different organization might not work to the same degree. But when the researcher really digs in to understand why journaling and other forms of internal communication are helpful, it suddenly becomes much clearer how such an intervention could be integrated into different organizations operating in different locations or contexts. In that study (Kogen, 2022), I offered a theoretical model regarding the role of journaling and other forms of internal communication in healing, empowerment, and ultimately collective action.

In this case, as with grounded theory (Glaser & Strauss, 1967), themes garnered from QTA may be taken to increasingly greater levels of abstraction until the researcher feels they have gotten to the essence of the phenomenon, or at least a portion of that essence, and arrived at something resembling a theory, or an added nuance to existing theory. While it will not be the case that every dataset holds the potential to build theory, a researcher can and should be open to that possibility.

Braun and Clarke (2006) are admittedly skeptical of using QTA to build theory, and refer to others using QTA in this way as conducting “grounded theory ‘lite’” because they do not commit to the full grounded theory process as described by Glaser and Strauss (1967). Braun and Clarke do not explicitly state that QTA cannot be used to build theory, but assert that researchers using QTA “need not subscribe to… implicit theoretical commitments” (p. 81). I would argue that QTA can be used to build theory when constant comparison is used and there is sufficient data to build a case.