Keywords Methodological study - qualitative analysis - coding time - health services research
- study planning - qualitative interviews
Schlüsselwörter Methodologische Studie - Qualitative Analyse - Kodierzeit - Versorgungsforschung -
Studienplanung - Qualitative Interviews
Background
A reliable estimation of the time and resources needed to conduct a study is
essential for sound research. This is especially true for health services research,
where timely results are often needed to inform healthcare practice [1 ]. To meet these needs, different
suggestions have been made for the adaption of study designs and methods: In
qualitative research, these suggestions include so-called rapid designs [2 ]
[3 ], omitting or speeding up the process of transcribing data [4 ], and pragmatic approaches to data
analyzing, e. g. by using frameworks [5 ],
tables and spreadsheets [6 ] or mapping
techniques [7 ]. Data analysis in
particular is discussed as a labor-intensive and time-consuming task [8 ]
[9 ]
[10 ]. Data analysis is
described as often underestimated and challenging: “Even seasoned qualitative
researchers can find the process of coding their datum corpus to be arduous at
times. For novice researchers, the task can quickly become baffling and
overwhelming“ [11 ]. Applied researchers
in particular might find data analysis a ‘daunting task’ [6 ].
Within this context, the common approach of audio-recorded interviews and in-depth
analysis of verbatim transcription has been challenged. So far, the focus in this
debate has been on transcriptions, highlighting the potential of approaches directly
using notes and/or recordings for analysis [4 ]. At the same time, speech recognition software and the use of
artificial intelligence (AI) might significantly speed up transcription time [12 ]. Time needed for data analysis, on the
other hand, is more difficult to evaluate, as a broad range of approaches is used
in
qualitative research. Estimations of time and effort for data analysis are rare and
mainly stem from anecdotical experience. For example, in a recent textbook, data
analysis is projected to take up to half of the time within a research project [9 ]. For analyzing focus groups, it is
suggested that inexperienced researchers might need 30% more time than experienced
researchers [13 ].
There is only a limited number of studies investigating data analysis empirically,
mostly comparing traditional forms with rapid forms of data analysis: A comparative
study by Gale et al. [14 ] based on 30
semi-structured interviews showed that the in-depth analysis took 10 weeks longer
to
complete than rapid analysis. Tylor et al. [15 ] compared rapid and thematic framework analysis based on 21
semi-structured interviews, focus groups and documents and concluded that rapid
analysis delivered a modest time saving: Early analysis and review took about a
third of the time of thematic analysis, but rapid analysis interpretation and write
up took more than six times longer than thematic analysis. Nevedal et al. [16 ] compared framework-based deductive
traditional and rapid approaches in two similar data sets of semi-structured
interviews of approximately 50 hours audio total: In sum, data analysis took
5.5 h/interview within the traditional approach applied to interview transcripts
(n=57) and 3.9 h/interview within the rapid approach applied to notes and timestamps
on audio recordings (n=72). Further data interpretation required the same number of
hours in both approaches (100 h). Eaton et al. [17 ] compared thematic analysis from scribed interviews (documentation of
comprehensive notes) and verbatim transcription of the same six interviews. When
compared to verbatim transcription, processing data into text form and subsequent
analysis was associated with significant time saving. Neal et al. [16 ] developed a procedure for rapid
identification of themes from audio recordings (RITA), stating a coding time 13%
longer than the length of an interview.
Though findings are mixed, these studies generally indicate rapid approaches to be
faster. However, details on analysis time were unspecific in most studies (e. g.
regarding interview duration or number of interviews) and separation of analytic
steps (e. g. transcription, analysis and interpretation) somewhat indistinct, making
it difficult to draw reliable conclusions for planning and conducting qualitative
studies.
Aim
The aim of this study, therefore, was to provide an empirical account on
estimating the time required for qualitative analysis. We focused on in-depth
analysis of transcript audio recordings from interviews, as this approach is
still widespread. Hereby we wanted to provide a reference for a) further
methodological studies investigating the balance of rigour and speed and b) the
selection of traditional and/or rapid methods to meet the analytical needs in
health services research.
Methods
We carried out a methodological study [18 ]
documenting and comparing the time taken for data-coding within five interview-based
studies in health services research undertaken at the Department of General Practice
and Health Services Research, University Hospital Heidelberg, Germany. This
corresponds to all studies analyzing qualitative data at the time of this study at
the department.
Project context of included interview studies
All five included interview studies were embedded in larger mixed-method projects
investigating health care services (s. [Tab. 1 ]): CCC was a process evaluation study of a counseling
intervention to educate cancer patients on complementary and integrative health
care and to promote interprofessional health care (CCC-Integrative study) [19 ]. ExKoCare was an
observational study and explored coordination and uptake of recommended
cardiovascular care in ambulatory care [20 ]
[21 ], PRiVENT
was a process evaluation study investigating the implementation of interventions
by weaning experts in patients’ risk of long-term invasive ventilation in
intensive care units [22 ]
[23 ], RESILARE developed quality
indicators measuring crisis resilience in primary care practices [24 ] and UCC developed a disease
management concept for venous leg ulceration [25 ]. All studies were publicly funded
for three to five years.
Tab. 1 Overview of project contexts of included interview
studies
CCC
ExKoCare
PRiVENT
RESILARE
UCC
Full title
Process evaluation of a counselling intervention designed to
educate cancer patients on complementary and integrative
health care and promote interprofessional collaboration in
this area (CCC-integrative-study)
ExKoCare: Cooperation networks of ambulatory health care
providers: exploration of mechanisms that influence
coordination and uptake of recommended cardiovascular
care
PRiVENT: Prevention of invasive ventilation
RESILARE: building crisis resilience of primary care
practices by developing and evaluating quality
indicators
UCC: Development and evaluation of an evidence-based and
patient-oriented care concept for the primary care of
patients with venous leg ulceration
Term
2019–2023
2019–2023
2020–2025
2021–2024
2020–2024
Funding
Innovation Committee of the Federal Joint Committee, Germany
(G-BA): 01NVF18004
German Research Foundation: 416396249
Innovation Committee of the Federal Joint Committee, Germany
(G-BA): 01NVF19023)
Innovation Committee of the Federal Joint Committee, Germany
(G-BA): 01VSF20029
Innovation Committee of the Federal Joint Committee, Germany
(G-BA): 01VSF19043
Overall study design
Mixed-method process evaluation
Mixed-method observational study
Prospective interventional multi-center study
Three-part design: 1) Systematic literature research and
qualitative study, 2) modified RAND/UCLA approach, 3)
piloting and mixed-methods process evaluation
Observational cross-sectional mixed-methods process
evaluation
Data collection and data analysis
A spreadsheet was developed for documentation of coding comprising project name,
interview participants, interview number, interview length, date of coding,
coder names and coding duration (CU, online appendix 1). Documentation was done
independently by all researchers (CA, TF, SK, NK, RPD, CU), guided by written
instructions.
In order to be able to compare coding times, time (in minutes) required for
coding was divided by the length of the interview (in minutes). A
coding-time-per-interview-time-ratio (CIR) of 1 means that the coding time
corresponds to the interview time. A CIR>1 means coding took longer than the
interview duration, while a CIR<1 indicates coding was shorter than the
interview duration.
Data were analyzed descriptively by means of absolute frequencies. Besides
tables, a boxplot was used to visualize differences between coding researchers,
showing minimum value, first quartile (25th percentile), median,
third quartile (75th percentile), and maximum value. Values were
depicted as outliers when the observation was 1.5 times the interquartile range
below the first or above the third quartile. To illustrate trends over time,
data were compared in chronological order of coding, displaying trend lines to
indicate general directions.
We followed the PRISMA-ScR guideline for design and reporting of this study where
applicable (s. online appendix 2), as a reporting guide for methodological
studies is not yet available [26 ]
[27 ] .
Ethics approval
Ethics approvals for the original studies were granted by the respective bodies
[19 ]
[20 ]
[22 ]
[24 ]
[25 ]. Participants provided consent for
the respective studies and subsequent data use. For the purpose of this
methodological study, only documentation of coding time by the respective
researchers was used. All researchers provided consent.
Results
Sample description and overview
All included interview-studies examined health care practice in terms of
processes, quality and/or outcomes using an explorative approach informed by
previous findings (s. [Tab. 2 ]). All
were explicitly informed by conceptual and/or theoretical perspectives, e. g.
frameworks. For data collection, all used semi-structured interviews with
different stakeholders comprising open-ended questions on a limited number of
themes. Interviews were conducted in German via telephone, audio-recorded and
transcribed verbatim. The number of included interviews per study varied from 11
to 27 interviews. Except for one study, all interviews in the respective study
phase of each project conducted by the time of this study were included. Due to
personnel changes in the research project, only the first 11 interviews of 26 in
RESILARE were included in this study. Within UCC , CCC ,
and RESILARE , additional interviews were conducted and/or analyzed after
completion of this study. Within most studies, the interviews included were the
initial and/or only qualitative data analyzed. Within CCC , recordings of
consultation were analyzed beforehand; within RESILARE , one focus group
was analyzed before and one during interview analysis. Within all studies, data
analysis was applied to complete interviews.
Tab. 2 Overview of included interview
studies
CCC
ExKoCare
PRiVENT
RESILARE
UCC
Aim
Exploration of HCPs’ perspectives on the intervention
Exploration of GPs’ perception of coordination with
cardiologists and obtainment of cardiological knowledge
Evaluation of the implementation of weaning boards and
weaning councils in intensive care units
Exploration of stakeholder perceptions of potential crises,
mitigation strategies and awareness of climate change
adaptation
Process evaluation of the pilot implementation of a care
concept for venous leg ulcerations.
Conceptual background
Consolidated Framework for Implementation Research (CFIR)
[26 ]: Applied
in overall design of process evaluation, including
development of interview guides and conduction of data
analysis
Prior quantitative findings on cooperation27 ]
[28 ]
Consolidated Framework for Implementation Research (CFIR)
[26 ]: Guided
semi-structured interviews and used to sort codes within
data analysis
Theories on behaviour change [29 ]
[30 ]
Theoretical Domains Framework of Behaviour Change (TDF) [29 ]
Time of interview conduction
2021/08–2021/12
2021/06–2021/11
2022/04–2022/06
2021/07–2021/10
2021/07–2021/12
Number of interviews
16
27
14
11 (of 26)
26
Participant groups
2 (12 counselors and 4 counseling team leaders)
1 (GPs)
2 (7 clinicians, 7 other HCP)
1 (GPs)
4 (7 GPs, 6 other HCP, 8 patients, 5 stakeholder)
Main themes in focused coding
Enabling (n=4) and hindering (n=4) factors for the
implementation, examined on patient, provider, and system
level
Motives and orientation of GPs, occasion for coordination,
information behaviour, teamwork (n=4)
Determinants of the implementation of weaning boards and
weaning councils at the structural and personal level of
intensive care staff (n=4)
General, organisational resilience, climate change adaptation
and mitigation, potential quality indicators (n=3)
Program acceptance, perceived effects, contextual factors,
intervention fidelity and reach (n=5)
Number of analytical codes
45
27
23
20
23
Number of analytical codings
779
656
637
735
1570
Targeted publications
1–2 papers
1–2 papers
1 master’s thesis (subsequent paper)
1–3 papers
1–2 papers
CCC: Process evaluation of a counselling intervention designed to educate
cancer patients on complementary and integrative health care and promote
interprofessional collaboration in this area (CCC-integrative-study);
ExKoCare: Cooperation networks of ambulatory health care providers:
exploration of mechanisms that influence coordination and uptake of
recommended cardiovascular care; PRiVENT: Prevention of invasive
ventilation; RESILARE: building crisis resilience of primary care
practices by developing and evaluating quality indicators; UCC:
Development and evaluation of an evidence-based and patient-oriented
care concept for the primary care of patients with venous leg
ulceration
Across included studies, different types of qualitative data analysis were used,
e. g. reflexive thematic analysis [28 ] and framework analysis [29 ]. All of these share a combined inductive and deductive approach,
comprising four broad stages: a) familiarization, b) open coding, c) focused
coding and d) development of analytical themes/write up [9 ]. The first two steps as well as the
development of analytical themes and write-up often have substantial overlaps,
wherefore, within this study, we concentrated on examining the third stage of
focused coding. Focused coding is an umbrella term used to describe targeted
analysis based on initial themes identified during open coding. While open
coding involves exploring data into various directions to uncover initial
insights, focused coding narrows down the analysis to highlight those insights
most relevant to the research questions. Therefore, within focused coding, the
data is recoded according to specified themes and codes [30 ]. Based on the results of focused
coding, researchers refine their understanding and develop a clearer picture of
analytical themes within the data.
In focused coding, the qualitative data analysis software MAXQDA was used in all
projects. Number of main themes, analytical codes and codings differed, but were
largely on a comparable scale. For one researcher (SK), the analysis was part of
a master’s thesis in form of a complete manuscript draft. Within all other
cases, the analysis was done with the aim to publish research results within at
least one scientific paper.
Researchers’ characteristics and roles within the interview studies
At the time this study was conducted, all six involved researchers were based at
the Department of General Practice and Health Services Research, University
Hospital Heidelberg, Germany, having collaborated in research and/or teaching in
different compositions beforehand. Two researchers had a professional background
in nursing (CA, SB). Academic education varied, but all researchers had a
background in health services research. The level of experience in (qualitative)
research differed: Three were post-doctoral researchers with experience in the
conduction and supervision of several qualitative research projects for at least
five years (NK, RPD, CU), three were doctoral (CA, TF) or master’s students (SB)
with experience in one or two qualitative research projects at the time of the
respective study. One researcher was involved in two of the included interview
studies (RPD).
Within the studies, researchers either took the role of primary researcher within
the overall study, were responsible for the entire project at the time of this
study or were mainly involved in the qualitative interview study, often adopting
an advisory role within the overall study as well (s. [Tab. 3 ]). Regarding the qualitative
interview studies, primary researchers were always involved in interview
conduction. In most studies, transcription was done by supporting staff using
transcription software. Only one researcher transcribed the interviews herself
(SB). All researchers read and coded all interviews. Data analysis was discussed
among respective research teams.
Tab. 3 Researchers roles within the interview study (at
the time of this study)
CCC
ExKoCare
PRiVENT
Resilare
UCC
NK
CA
CU
SK
RPD
RPD
TF
Role within the primary study
Primary researcher/investigator
x
x
–
–
–
–
x
Focus on qualitative study
–
–
x
x
x
x
–
Advisor of overall study
–
–
x
–
x
x
–
Tasks (involved in) within the qualitative study
Interview guide development
x
x
x
x
–
x
x
Interview conduction
x
x
–
x
–
x
x
Transcription
–
–
–
x
–
–
–
Analysis of the data set
x
x
x
x
x
x
x
Write-up and publication
x
x
x
x
x
x
x
CCC: Process evaluation of a counselling intervention designed to educate
cancer patients on complementary and integrative health care and promote
interprofessional collaboration in this area (CCC-integrative-study);
ExKoCare: Cooperation networks of ambulatory health care providers:
exploration of mechanisms that influence coordination and uptake of
recommended cardiovascular care PRiVENT: Prevention of invasive
ventilation; RESILARE: building crisis resilience of primary care
practices by developing and evaluating quality indicators; UCC:
Development and evaluation of an evidence-based and patient-oriented
care concept for the primary care of patients with venous leg
ulceration; CA: Christine Arnold, CU: Charlotte Ullrich, NK: Nadja
Klafke, RPD: Regina Poß-Doering, SB: Sabrina Brinkmöller, TF: Thomas
Fleischhauer
Interview data and focused coding
Across studies, focused coding was assessed in 94 interviews with a total
interview duration of 52 hours and 44 minutes (see [Tab. 4 ]). Number of interviews per
study ranged from 11 to 27, with a mean duration of 36 minutes. Interview length
mean ranged from 21 minutes to 50 min. Total coding time amounted to 76 hours,
with a mean of 32 min per interview. Coding-time-per-interview-time-ratio (CIR)
ranged from 0.75 to 1.52 in mean interview time with a mean of 0.99 across
studies.
Tab. 4 Duration of focused coding
CCC
ExKoCare
PRiVENT
Resilare
UCC
Sum/mean
A. Interviews
Number
16
27
14
11
26
94
Duration: total
11:44:00
09:28:53
07:09:20
09:07:36
15:15:18
52:44:14
Duration: mean
00:44:00
00:21:04
00:30:40
00:49:47
00:35:12
00:36:09
Min
00:20:00
00:11:20
00:20:04
00:15:25
00:18:43
Max
00:59:00
00:36:39
00:48:31
01:16:09
01:18:38
SD
00:12:52
00:06:43
00:05:47
00:15:25
00:14:31
Time interval of coding
4 months
1 month
1 month
1 month
3 month (TF)
Order of coding
Chronological
Chronological
Chronological
Mostly chronological
Chronological
Theme development
Inductively + CFIR
Primarily inductively
Inductively + CFIR
Primarily inductively
Inductively + TDF
B. Focused Coding
Researcher
NK
CU
CA
SB
RPD
RPD
TF
Total Coding Time
08:42:00
09:07:00
07:04:00
10:15:00
07:01:00
13:16:00
20:54:00
76:19:00
Per Interview
Mean
00:32:37
00:20:16:
00:15:42
00:43:56
00:38:16
00:30:37
00:48:14
00:32:48
Min
00:15:00
00:10:00
00:07:00
00:25:00
00:20:00
00:15:00
00:15:00
00:15:20
Max
01:05:00
00:55:00
00:30:00
01:00:00
01:00:00
01:00:00
01:30:00
01:10:00
SD
00:14:09
00:09:23
00:05:47
00:10:25
00:13:47
00:10:05
00:21:45
00:14:14
CIR (coding-time-per interview-time-ratio)
Mean
0.76
0.96
0.75
1.52
0.84
0.93
1.39
0.99
Min
0.38
0.61
0.44
0.82
0.37
0.51
0.70
0.65
Max
1.33
1.65
1.09
2.99
1.33
1.55
2.22
1.34
SD
0.26
0.26
0.17
0.56
0.33
0.28
0.44
0.17
C. Consenting
Total
4h
4h
–
–
5h
Number of meetings
5
12
–
–
3
Duration
30–60 min
10–30 min
–
–
90–120 min
CCC: Process evaluation of a counselling intervention designed to educate
cancer patients on complementary and integrative health care and promote
interprofessional collaboration in this area (CCC-integrative-study);
PRiVENT: Prevention of invasive ventilation; RESILARE: building crisis
resilience of primary care practices by developing and evaluating
quality indicators; UCC: Development and evaluation of an evidence-based
and patient-oriented care concept for the primary care of patients with
venous leg ulceration; CA: Christine Arnold, CU: Charlotte Ullrich, NK:
Nadja Klafke, RPD: Regina Poß-Doering, SB: Sabrina Brinkmöller, TF:
Thomas Fleischhauer
Distribution and skewness of data differed between projects (s. [Fig. 1 ]). Distribution of data is
compact for most researchers, with two exceptions (PRiVENT and UCC: TF).
Distribution was rather symmetric in one coding (ExKoCare: CA) and somewhat
positively skewed in one project (UCC: RPD). Outliers can be found in CIR of two
researchers (ExKoCare: CU: 1.65, PRiVENT: 2.99). The median CIR lay between 0.71
(CCC) and 1.41 (UCC: TF) with a mean median of 0.98. Across projects, the
25th percentile was 0.82 and the 75th percentile was
1.02, with an interquartile range of 0.2.
Fig. 1 Coding-time-per-interview-time-ratio (CIR) per project and
researcher.
Development of coding duration
Focused coding was performed over a period of one to six months. Within all
studies, interviews were coded roughly in chronological order of the conduction
of interviews. Codes were refined within focused coding. Within three studies
(CCC, ExKoCare and UCC), codes and codings were regularly consented in 3 to 12
meetings respectively, amounting to approximately 4 to 5 hours.
All studies included at least 11 interviews, with a mean CIR of 1.34 (min: 0.43;
max: 2.99) of the first interview and a mean CIR of 1.00 (min: 0.71; max: 1.79)
of the eleventh interview (s. [Fig.
2 ]). Mean CIR of the respective last interview was 1.09 (min: 0.38;
max: 1.85). Overall, focused coding tended to get quicker over time. However,
variation among studies was high. Trendlines indicate the biggest increase in
speed within the PRiVENT study (first interview: 2.99; last interview: 1.42) and
a narrower but almost parallel increase in speed in CIR within UCC, ExKoCare and
CCC. RESILARE was an exception, as the trendline showed an increasing coding
duration (first interview: 0.43; last interview 1.03). Within all studies, there
was fluctuation in CIR over time.
Fig. 2 Coding-time-per-interview-time-ratio (CIR) per
interview.
Discussion
Principal findings
Results of this study provide a reference for estimating the time required for
qualitative analysis. Previous studies indicated that rapid approaches,
especially omitting transcription, were faster than traditional approaches to
qualitative data analysis [4 ]
[10 ]
[16 ]
[17 ]
[31 ]
[32 ]. Our results show that on average,
the time spent on focused coding corresponds to the duration of the interviews
(CIR: 0.99). This is comparable to findings on application of rapid
identification of themes from audio recording (RITA) with a coding time of
68 min per 60 min interview, translating to a CIR of 1.13 [16 ]. However, the application of a
framework-based deductive analysis approach led to a coding time of 275 h for 50
interview hours, translating to a CIR of 5.5 [33 ]. This difference might be due to
this study addressing the whole process of data analysis, including independent
coding by two analysts and regularly adjudicating differences, not only the step
of focused coding.
Our results indicate that CIR tended to get quicker over the course of an
analysis, with a mean decrease of 0.34 from the first (1.34) to the eleventh
(1.00) interview. This decrease was expected by the research team: in
qualitative analysis, the coding system becomes more precise over time and
researchers become more confident in the application of codes. Starting with a
higher factor, the time saved was greatest within a master’s thesis (SB). For
doctoral students, factors were lower and more stable in one project (ExKoCare:
CA, mean 0.75, SD: 0.17) and higher in another project (UCC: TF, mean 1.39, SD:
0.44). Potential influencing factors may be that in ExKoCare, only one
participant group (GPs) was included, the doctoral student (CA) always coded
second (after CU), and coding was carried out over a relatively condensed period
of one month with regular consent meetings (n=12). In UCC, by contrast, four
different participant groups were included (GPs, practice assistants, patients,
other stakeholders), and coding was done largely independently with fewer
consent meetings (n=3) of the two researchers involved within a coding period of
three months.
Limitations and reach
To facilitate comparability, only studies from one research department were
included, where all researchers had a background in health services research and
had collaborated in research and/or teaching in different compositions
beforehand. The Heidelberg master’s degree program in health services research
and implementation science connected all researchers participating in this
study: Some were teachers, of whom all taught research methods and some were
(former) students; some were both. Therefore, a common (although not uniform)
understanding and practice of qualitative analysis can be assumed. All interview
studies explored health care practice, used semi-structured interviews in German
and had a comparable approach to data analysis using the same QDA-Software. All
studies included interviews with one participant per interview, mostly health
care providers. While homogeneity in these regards improves comparability,
applicability of results to studies with other study populations, methods of
data analysis, order of interviews analyzed or use of concepts and theories has
to be examined.
Researchers differed in professional and disciplinary backgrounds, experience in
qualitative research methods and degree of involvement in the studies. The
interview studies varied regarding number and length of interviews, timeframe of
conducting and analyzing interviews and overall project context. While these
differences reflect a common reality in health services research, they limit
reach and further data analysis. Therefore, echoing previous studies [15 ], further investigations are
needed, e. g. using comparable data sets and research teams.
CIR over the course of interview analysis was accessed by descriptive analysis
and should be interpreted with caution. Larger data sets are needed to test
trends statistically.
Estimating timescales in qualitative analysis
Our study addressed focused coding. To estimate the time required for qualitative
analysis, further aspects have to be considered: In our experience, preceding
steps of familiarization, open coding and first analytical memos take at least
two to three times the interview duration. Subsequent consolidation of
analytical themes usually continues in write-up and could take a few weeks. In
all studies, a combined inductive and deductive approach was used that included
focused coding as a definable analytical step. However, repetition and
overlapping of analytical steps is common in qualitative research [9 ]
[30 ] and might be more prominent in
other approaches (e. g. Grounded Theory). In addition, within the included
studies, research objectives and therefore qualitative analysis was targeted to
meet the scope of one to two research papers. More time would be needed to meet
more comprehensive research objectives.
Our findings suggest that in estimating the required time for qualitative
analysis, researcher and data characteristics have to be considered: Qualitative
research experience, degree of involvement in the research project and
proficiency in concept and theories as well as richness of data and composition
of participant groups might influence time expenditure of coding. Previous
research highlighted research experience as a relevant factor [13 ]. As our findings suggest, project
characteristics such as scope of the rtesearch question, timeline and peer
feedback might be further influencing factors. While our results indicate that
coding becomes faster over time, there are limitations to increasing speed as
qualitative analysis is a highly concentrated activity that cannot be carried
out indefinitely at one time.
Our study was based on interview transcripts. While omitting transcription has
been described as a major factor in quickening data analysis [4 ], this factor might lose
significance with the increasing availability and precision of automatic speech
recognition software and the use of AI in transcription [12 ]. At the same time, some studies
found improvement of accuracy and richness of interpretation when coding
directly from recordings, especially when used by experienced researchers [10 ]
[31 ]
[32 ]
[33 ]. Another study pointed to the
benefits of integrating traditional and rapid qualitative analysis for
intervention development in a transnational study [34 ].
Artificial intelligence and the limits of calculability
The potential of AI for automatic coding in qualitative analysis is increasingly
discussed. Qualitative data analysis programs such as Atlas.ti and MAXQDA have
already implemented AI features promising substantial reduction of total data
analysis time. First explorations show that AI can make qualitative research
more time efficient, especially in deductive approaches [35 ]
[36 ]. AI can serve as a useful tool for
enhancing researchers’ capabilities when applied within a larger analytical
process, structured e. g. by explicit prompts [35 ]
[36 ]
[37 ]. Limitations lay in capturing more
subtle and interpretive themes [36 ],
the inherent bias of AI, influenced by hegemonial ideas prevalent in society
[38 ] and questions of data
protection. These factors should be considered in future research investigating
the efficacy of AI-supported qualitative data analysis.
In order to assess the proclaimed benefits of AI, references for estimating time
requirements and resources in qualitative research are required. However, from
our study, no firm guidance can be derived at on how long qualitative research
should take (at most) as research designs and research contexts differ: Research
questions may be more or less exploratory, research interests may be more
descriptive or more analytical, researchers may be more experienced or less
experienced, project contexts may allow more or less focus on certain work
packages. Our results could rather serve as a reference for project planning to
avoid underestimation [8 ]
[9 ]
[10 ]
[11 ] and to consider rigor and speed
according to the specific objective and context of a particular study.
Conclusion
Reliable references are required for estimation of time and resources needed
to conduct an empirical study. In any specific research project, however,
this effort must be balanced against the objective of the analysis,
including the desired level of accuracy, detail and depth. Qualitative data
analysis, like all research endeavours, demands specific training, skills,
endurance and adaptability. Effective allocation of resources is crucial,
yet methodological rigour cannot be arbitrarily economized, given the
multiple factors influencing empirical research. Our study emphasizes the
importance of considering factors such as composition of data, researchers’
experience and degree of involvement when planning research objectives and
designs within a specific timeframe. Further research is needed to specify
how particular parameters such as nature of the study population, method of
data analysis and use of concepts and theories affect coding in qualitative
analysis.
Availability of data and materials
Availability of data and materials
Data generated or analyzed during this study are included in this published article
and its supplementary information files.
Authors’ contributions
CU, CA, RPD and MW conceived the idea for this study. CU developed the project and
was responsible for concept und design. CA, SB, TF, NK, RPD and CU documented coding
times providing data for this study, context information and ideas. CU drafted and
prepared the manuscript, with critical input on data presentation and methods from
CA and MW. CA, NK, RPD and MW provided regular input throughout the project. All
authors reviewed and approved the final manuscript.
This article is part of the DNVF Special Issue “Health Care Research and
Implementation”.
