Debates about the software
Authors of this page: Celia Taylor¹, Ann Lewins²
and Graham R. Gibbs¹
Affiliation: ¹University of Huddersfield and
Date written: 12th Dec 2005
When software to support qualitative analysis was first written it was often by enthusiasts who were themselves undertaking qualitative analysis and who had some skills in programming (or knew someone who did). The diversity of software reflected their individual approaches. The software writers and many other qualitative researchers saw the benefits and potential of using software, but almost from the start, there were others who voiced cautious and even sceptical views. During the late 1980s and 1990s a large body of literature was published that was concerned with this debate about the positive and negative impacts of software use. However, the software developed quickly over the same period and developers addressed many of the concerns that sceptics had expressed and at the same time developed new functions and new approaches to analysis. The first section of this page summarises many of the themes that were debated during the 1980s and 90s where the focus was often on comparing software use with older, manual methods. The second section brings these issues up to data by examining some of the more recent developments in CAQDAS using different methodologies and the debates about them.
The early debates about CAQDAS
Size of dataset
The data management features of CAQDAS software enables researchers to work with larger datasets without losing track of both the details of the data and the developing analysis However, one danger of this, as Seidel (1991) pointed out, is that large datasets can lead to a focus on breadth rather than depth. He worried that in-depth, careful analysis may be lost to a shallower type of exploration on larger and larger datasets, just because it had become more possible. Shallow explorations on larger data sets are not necessarily bad, especially if that is what the sponsors of research want. But it does sacrifice one of the strengths of the qualitative approach – getting a detailed insight into the social situation being investigated. The software has made this possible, but as Fielding and Lee (1998) have argued, it is not the availability of CAQDAS packages that has led to large projects but rather researchers who are undertaking large projects seek out the software. The size of the project, they suggested, is more likely to be determined by methodological stance or by the sponsor. In fact, Fielding and Lee provide some evidence that on average, datasets and samples in qualitative projects have not been getting larger. At the other extreme, Seale (2000) has questioned the value of using CAQDAS software on a small data set.
Using CAQDAS software has a number of implications for the quality of research. It is easier to keep track of the material and manage the data (Tesch, 1990). Using software allows a trail of analysis to be preserved which makes it easier to replicate the research and assist towards transparency (Conrad and Reinharz, 1984). Retrievals and searches on data can be repeated in a consistent way. This enables analysts to carry out retrospective checks to ascertain whether analytic closure was well grounded or not (Fielding and Lee, 1998). But Miles and Huberman (1994) have argued that an audit trail of this kind is not always automatic and still requires systematic and structured record keeping which is costly in terms of time and effort. If the researcher puts in the extra work and makes use of the software assisted management of memos it can help to ensure codes are applied consistently. Then, Conrad and Reinharz (1984) have pointed out, using the software, it is easier to identify deviant cases and extract small pieces of significant data buried in a large mass of information. In a variety of ways, then, the software can support better quality of analysis and avoid the researcher missing key data and interpretations. However, this is at the cost of more effort and work in documenting procedures.
Creativity and Thinking
Tesch (1991) argued that though the researcher still does the thinking, judging and interpreting in all qualitative research, CAQDAS software can, at least, perform technical and clerical tasks quickly and efficiently. The software, therefore, allows you to play with the data and try new angles and analytical approaches which would not be possible manually (Tesch, 1990). Such additional flexibility when using software permits the researcher to experiment and test ideas, without committing to them if they turn out to have no value. For example, it is possible to re-code more easily when using software than when doing it by hand. Tesch (1989) suggested that this ability to play with the data fosters analytical insight. Pfaffenberger (1988), though, has stressed the need to be reflective about the analytic effects of technology and remain aware of any of its undesirable effects.
Efficiency in Data Management
Tesch (1990) has argued that using CAQDAS software for what were once manual procedures saves time and is more efficient. It is easier to keep track of material and find items of interest and significance. Fielding and Lee (1998) have made the same point. They suggested that use of the software increases access to the data, reduces paperwork and helps keep track of data from multiple sources. They argued that well-managed data can be reused and returned to with new analytical ideas and objectives. However, as they pointed out, organising the database still requires a researcher to be systematic. Software also makes it easier to reorganise someone’s work and have a record of how it was arranged before. For some commentators the main value of CAQDAS software is for data organisation and retrieval (Coffey and Atkinson, 1996:176).
Distance from data
Weitzman and Miles (1995) identified the importance to researchers of feeling close to the data they are analysing. There is disagreement about whether or not CAQDAS software makes this easier or harder. Many users of software in the 1980s felt that using computers (for all its other benefits) did tend to create a distance between them and the data set. However, Fielding and Lee (1998) have argued that using manual methods cannot guarantee closeness to data and in many cases it is the amount and nature of data can be overwhelming. They questioned what ‘being close to the data’ actually means. They suggested that the closeness provided by manual methods could refer to the actual physical handling of the data. Though what is meant here is the physical handling of paper transcripts rather than the actual speech or situations they refer record. Ragin and Becker (1989) have argued that, in fact, CAQDAS software can provide closeness to data and intensive interactive analytical style. Proponents of manual methods might argue that although it is easier to move around the database than shuffle paper, it is easier to flip a page than scroll down, quicker to read from a printed page than from the screen, easier to annotate in a number of different ways. The argument for the need to handle data could relate to the importance of being familiar with the content (Fielding and Lee, 1998), and, it has to be admitted, some of the early programs did not make this as easy as working with paper-based transcripts. Indeed, Weitzman and Miles (1995) have argued that distance from the data could be reduced with improvements in the interface design. More recent versions of programs have addressed this, first by using standard windowing features and also by including tools that make movement back to the original data easy and quick.
A related and common complaint about CAQDAS programs when they were first developed was that coding and retrieval with them tended to decontextualise the passages retrieved. For example, Coffey and Atkinson (1996) argued that the principal code and retrieve functions of CAQDAS meant that data becomes fragmented and the sequence of processes and interactions could be lost. Fielding and Lee (1998) responded that this could be addressed with proximity searches and hyper linking functions now found in many packages. They suggested that it is difficult to separate problems of coding per se and problems of coding within the software and the way it is designed to code. They point out it is important to be familiar with the data if working onscreen.
Richards and Richards (1991) expressed concerns that projects or data might be designed to fit the functionality of CAQDAS programs and Fischer (1994:100) was worried that software will lead to methods being poorly understood. More generally many were concerned that many programs supported thematic approaches to analysis (particularly grounded theory) and, as Drass (1989) pointed out, the software was not suitable for certain methodologies that did not use themes. This was supported by Seale (2000) who suggested CAQDAS programs were not suitable for conversation analysis and discourse analysis. However, Some researchers doing such analyses do use programs for preliminary, case selection stages and to identify words and phrases in a large body of text indicating areas for closer examination.
Speed and superficiality
Tesch (1990) argued CAQDAS software saved time and was more efficient than manual procedures. The danger, as Fielding and Lee (1998) cautioned, is that computers facilitate quick and dirty research with the possibility of premature closure. In an examination of many published papers that mentioned the use of CAQDAS Seale (2001) noted that many did very little more than a simple code and retrieve which then formed the basis of a thematic report. Many papers referred to grounded theory but even here some only undertook what has been referred to as ‘pattern analysis’ rather than use the full analytic procedures of grounded theory.
On the other hand the cost of using the software to the full is the time it takes. Most experienced users would suggest that some processes are expedited by using the software, but the researcher is able to investigate the data and perform so many actions with the software, that in actuality time savings are not made.
Conrad and Reinharz (1984) suggested that CAQDAS software made teamwork easier. However, as Russell and Gregory (1993) argue, the cost is that it requires everyone to have a copy of the software to open the files. Many early programs did not support the easy exchange of information from one researcher to another and could not even open more that one file. Teams, therefore, had to assign just one person to control the program and hence be responsible for the data and analysis recorded there. More recent programs can accommodate data being worked on by more that one analyst. As Fielding and Lee (1998) have noted, because it is awkward physically to store paper based data, one of the main benefits of using CAQDAS software is improved consistency as it forces a team to resolve their differences when using the software.
In the early phases of CAQDAS development Becker, Gordon and LeBailly (1984) pointed out new codes could be added at any time and the data could be coded in several different ways at once. Fielding and Lee (1998) suggest that with the support of software it is easier to assign codes and to be systematic. They suggest coding using paper has a tendency to be less refined because of the reduced flexibility of manual processes. CAQDAS software does not lock the analytic procedure into a rigid coding schema and iterative processes of coding and recoding are natural parts of CAQDAS use. In the same mould, Richards and Richards (1989) suggested that CAQDAS software allowed a more complex analysis which lead to a more complex coding structure. However, there is a downside. Coffey and Atkinson (1996) have argued that the dominance of the code and retrieve paradigm in most CAQDAS programs has lead to an over-reliance on the ‘coding’ method, where other methods, such as narrative analysis, might be more appropriate to the needs of particular approaches and types of data. Part of the remit of the Cardiff based ESRC QUALITI project which started in 2005, is to work towards the creation of practical guidelines and innovative approaches to the analysis of various types of qualitative data, and their work may consider software alternatives to coding.
Many qualitative researchers are wary if not downright hostile to the use of numbers and statistics in their analyses. However, Ragin and Becker (1989) argued that CAQDAS software would narrow the gap between quantitative and qualitative data. The danger, as Tesch (1989) pointed out, is that many quantitative researchers think CAQDAS software is just about the quantification of qualitative data (for better or for worse).
For those who need to make links between qualitative and quantitative data much software now supports it. Fielding and Lee (1998) have noted that CAQDAS enabled the export of e.g. code frequency counts to statistical packages and Richards & Richards (1991), whose software program NUD*IST was one of the first to allow imported quantitative data enabling complex interrogation of subsets, went as far as to suggest that CAQDAS software challenges the dichotomy of quantitative and qualitative methods.
Support and Awareness
Those using CAQDAS have not been well served by those providing support in their own institutions. Fielding and Lee (1998) pointed out that computer staff providing support for the programs often have a background in natural science and therefore found it difficult to relate to the CAQDAS software. They also recognised that CAQDAS software was not well known outside the qualitative analysis community and there was a lack of support from computer staff.
Recent developments and debates about CAQDAS software
We are still writing this section and it will be included on the website soon.
Becker, H.S., Gordon, A.C. and LeBailly, R.K. (1984) 'Fieldwork with the computer: criteria for assessing systems', Qualitative Sociology, 7 (1-2): 16-33.
Brent, E (1984) 'Qualitative computing: approaches and issues' Qualitative Sociology, 7 (1/2): 36-60.
Coffey, A. and Atkinson, P. (1996) Making sense of Qualitative Data, London: Sage.
Conrad, P and Reinharz, S (1984) 'CAQDAS software and qualitative data: editors’ introductory essay', Qualitative Sociology, 7 (1/2): 3-15.
Douglas, J. (1976) Investigative Social Research. London: Sage.
Drass, K. A. (1989) 'Text analysis and text-analysis software: a comparison of assumptions'. in Grant Blank, James L McCartney and Edward Brent (eds), New in Technology in Sociology: Practical Applications in Research and Work. New Brunswick: NJ: Transaction Publishers.
Fielding, N.G. and Lee, R.M. (1993) Using Computers in Qualitative Research, London: Sage
Fielding N.G. and Lee R.M. (1998) Computer Analysis and Qualitative Research, London SAGE.
Fischer, M. D. (1994) Applications in Computing for Social Anthropologists, London: Routledge.
Gerson, E. (1984) 'Qualitative research and the computer', Qualitative Sociology, 7 (1/2): 61-74.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: a sourcebook of new methods. Beverly Hills, CA: Sage.
Ragin, C.C. and Becker, H.S. (1989) 'How the microcomputer is changing our analytic habits', In: G. Blank et al. (eds), New Technology in Sociology: Practical Applications in Research and Work. New Brunswick, NJ: Transaction Publishers.
Pfaffenberger, Bryan (1988) Microcomputer Applications in Qualitative Research. Newbury Park, CA: Sage.
Richards L. and Richards T. (1989) ‘The impact of computer techniques for qualitative analysis’, Technical Report, no. 6/89, Dept of Computer Science, La Trobe University.
Richards L. and Richards T. (1991) 'The Transformation of Qualitative Method: Computational Paradigms and Research Processes' In Nigel G. Fielding and Raymond M. Lee (eds), Using Computers in Qualitative Research, London: Sage. pp. 38-53.
Russell C. K. and Gregory, D. M. (1993) ‘Issues for consideration when choosing a qualitative data management system’, Journal of Advanced Nursing, 18: 1806-16.
Seale, C (2000) Using CAQDAS software to Analyse Qualitative Data, In: David Silverman (ed) Doing Qualitative Research – A Practical Handbook. London: SAGE
Seale, C.F. (2001) 'Computer-Assisted Analysis of Qualitative Interview Data', in J. F. Gubrium and J. A. Holstein (eds), Handbook of Interview Research: Context and Method. Thousand Oaks, Ca: Sage. pp. 651-670.
Seidel (1991) 'Method and Madness in the Application of Computer Technology to Qualitative Data Analysis'. In Nigel G. Fielding and Raymond M. Lee (eds), Using Computers in Qualitative Research, London: Sage.
Tesch, R. (1989) 'The correspondence between different kinds of qualitative analysis and different kinds of software', Paper presented at the Symposium on Qualitative Knowledge and Computing, University of Surrey, 11-12 July.
Tesch, R. (1990) Qualitative Research: Analysis Types and Software Tools. London and Philadelphia: Falmer Press.
Tesch (1991) 'Software for qualitative researchers analysis needs and program capabilities', In Nigel G. Fielding and Raymond M. Lee (eds), Using Compluters in Qualitative Research, London: Sage.
Weitzman, E.A., & Miles, M.B. (1995). Computer Programs for Qualitative Data Analysis: A Software Source Book. Thousand Oaks: Sage