HEALTH POLICY AND PLANNING; 12(4): 372-380 © Oxford University Press 1997 How to do (or not to do) ... Comparing manual with software analysis in qualitative research: undressing Nud.ist G LEWANDO-HUNDT,' § BECKERLEG,' A EL ALEM? AND Y ABED? ‘London School of Hygiene and Tropical Medicine, UK, and *Gaza Health Services Research Centre. ‘There has been increasing interest in the use of com- puter software to analyze qualitative data. A variety of programmes are available for this purpose such as Textbase Alpha, Anthropac, Zylndex, Ethnograph 3.0, and Q.S.R, NUD.IST.' Some researchers are wary about the value of using computer software for analysis instead of the tried and tested ‘cut and paste" manual indexing methods, or are reluctant to learn how to use the software when uncertain of its benefits Aim and method We set out to review the differences and similarities between using manual and software analysis. We were interested in the relative ease of each method, fo What extent the analysis was facilitated by either method, and if either or both methods could be used as a means of testing the internal validity of qualitative analysis by two different people, We took the same data from 24 group interviews and analyzed them separately. One of the authors used manual analysis (SB) and the other (GLH) used Q.S.R. NUD.IST. ‘When we had both completed the analysis in terms of indexing and identifying themes, we compared our separately derived indexing. We chose a theme/ category which we both had independently identified, but called slightly different names, and then separ ately wrote up the data around this theme. We then compared the two sections in terms of content and discussed the process of analysis to identify dif- ferences, strengths and weaknesses, and to define more precisely how the software acts as an aid and support, The data are part of a five-year study (1995-2000)? to evaluate and improve maternal and child health (MCH) care in the Gaza Strip (Palestinian National Authority). Using a range of qualitative and quan- titative methods, the study has collected data on women’s and men's views and experiences of services, both as users and providers. The data used for this comparison of manual and software analysis are a set of 24 group interviews with 157 women carried out between April-September 1995. The Tesponses informed the design of structured and in-depth interviews, The analysis was undertaken in 1996 Study setting Gaza is an ancient port which was used for shipping, spices from Arabia to the Mediterranean. It was part of Palestine within the Ottoman Empire. Sub- sequently it came under the administration of the British mandate of Palestine (1917-1948) and then, except for a short period from 1956-7, the Gaza Strip was under the administration of Egypt from 1948 until 1967. In 1967 it came under Israeli occupation. An autonomous Palestinian National Authority was in- troduced in the Gaza Strip in the spring of 1994, ‘The population of approximately one million in Gaza are either long-term residents of the area (25%) or registered as refugees (75%) (UNRWA 1994). This affects health care because refugees and their descen dants may use clinics of the United Nations Relief and Works Agency (UNRWA), of which there areHow to do (or not to do) . .. 373 28 serving approximately 70-80% of the population These clinies provide free comprehensive primary care. In contrast, the residents cannot use these clinics but have access to Ministry of Health (MoH) clinics which provide free ante-natal care and immunization services. Curative care is provided free to children under three and to those with health insurance. In addition to these two main providers, there are a number of NGOs running clinics and a flourishing private sector of physicians and pharmacies (Schnitzer and Roy 1994) ‘The group interviews were carried out with women living in three areas of the Gaza Strip. The Rimal district of Gaza City has a mixed population of residents and refugees totalling about 60 000 people (Abed 1992). It is the most developed part of Gaza and the area is characterized by modern buildings with satisfactory water and sewage services. Curative health services are provided by governmental and non-governmental organizations. MCH facilities are provided by the MoH through the Rimal clinic Refugees have access to the primary health care centre of the UNRWA in Rimal Jabalia, once a village with an economy based on growing fruit and vegetables, is situated three Kilometres to the north of Gaza City. As farm land has been developed to provide housing for the rapidly ‘growing urban population, the agricultural sector has declined and people have been forced to seek employ- ment elsewhere. The vast majority of the population of 24 000 (Abed 1992) are residents of Gaza, and are not registered as refugees. Jabalia is served by an MoH clinic which provides curative and preven- tative care and is equipped with a small laboratory Jabalia Camp adjacent to Jabalia village, with a population of 79 000, is the largest and most densely populated refugee camp in Gaza, The violent clashes which took place there between Palestinian demonstrators and the Israeli army in December 1988 are generally seen as the spark which ignited the Intifada (UNRWA 1994). Health services in Jabalia Camp are provided by a UNRWA health centre and an MoH clinic. Collecting the data The project research team was trained in focus group methods by Beckerleg and Bl Alem. Beckerleg con- centrated on the development of focus group guides, facilitation of interviews and managing group dynamics, Team members were required, in turn, to participate in discussions, take notes, and facilitate focus groups covering a range of topics. The aim of these sessions was to train the team in basic focus group techniques as described by Krueger (1988). The techniques were designed for use with parti- cipants living in Europe and the USA, rather than diverse social settings, and required adaptation to Gaza, Other manuals, such as by Dawson et al (1993), take a more cross-cultural approach. El Alem worked with the team to develop group inter- viewing techniques for use in the specific milieu of Gaza. Firstly, the team decided that the tape record- ing of interviews might inhibit discussion and would be inappropriate since the recent history of occupa- tion and resistance has left a legacy of caution and suspicion. Therefore, the interviewers were trained in note taking. Secondly, Palestinian society is very hierarchical and in a group setting some individuals may be inhibited by status differences. Hence, in order to ensure that all participants were able to a their views, the team were trained to give every parti- cipant the opportunity to respond. While this method prevented the domination of the groups by indi viduals, it also slightly inhibited group interaction. on issues of health care delivery with each other and the interviewers in a lively open manner. The interview guide was piloted and the interviews were carried out by four female Palestinian inter- viewers, three of whom were qualified nurses work ing as research assistants and one of whom was the study coordinator with a training in Human Resource Management. Interview teams were comprised of a facilitator and note-taker. The interviews were con- ducted in Arabic and recorded in Arabic, and then translated into English. There were two weaknesses in the data, One derived from the method of inter- viewing which limited extended spontaneous explora- tion of a topic. The other was that, for some of the earlier interviews, some weeks elapsed between the time of interview and note taking, and the time of detailed writing up. Twelve interviews were conducted in the homes of mothers who had recently delivered babies. Names of mothers living in the three study areas were selected randomly from the birth registry held at the Gaza Health Services Research Centre; difficulties were encounterd in locating some of these women owing to incomplete addresses on birth certificates.374 Letters were sent to the women selected requesting their consent to participate in group interviews. They were also requested to invite kin and neighbours to participate in a discussion to be held in their homes ona specific day. The other 12 interviews were con- ducted in the main MCH clinics in the study areas Hence, four interviews were held in Jabalia camp UNRWA clinic, four in Jabalia village government clinic and four in Rimal government clinic Participants recruited in home or clinic settings con stituted different types of groups. In women's homes, the host knew all the guests, and most, if not all, of them would know each other. Some or all of them would mect together in other group settings. ‘These gatherings of women for group interviews are, therefore, ‘natural groups’ (Coreil 1995). The women recruited in clinics might be strangers or known to one another, In the close knit communities of Jabalia village and Jabalia Camp, where people have more connected networks, many participants in the group interviews would know each other or meet outside the clinic In Rimal, where many recent returnees from overseas or upwardly mobile families have settled, fewer women attending clinics would be known to one another. These women were more like the par ticipants in a focus group interview (Krueger 1988). Data analysis When analyzing qualitative interviews, the researcher sets out to identify themes and topics by a process of categorization and indexing, and develops theoretical constructs and logic from the data. hhas been described in a number of texts (Miles and Huberman 1994; Strauss and Corbin 1990; Silver- ‘man 1993) and was done here on the same data both manually (Beckerleg) and using software (Hundt), Beckerleg’s manual analysis Initially, I read through the interviews in chrono- logical order and discussed them with the research team. However, the main analysis of the results was conducted when I reread the results. First, I grouped the interviews into home or clinic setting and read each group as a batch. The research teams had reported to me that the interviews conducted in clinics elicited fuller responses than those carried out in homes, but I did not find this to be borne out by the actual records of interviews. The interviewers felt more comfortable in the clinie setting. As visitors in How to do (or not to do) women’s homes they accepted hospitality, but asked questions in a directive manner which went far beyond the role of guest. It was envisaged that inter- views away from the clinic in relaxed familiar set- tings would enable women to speak more freely of their experiences of the clinics. Responses were indeed openly critical of the services provided in a range of UNRWA and government-run clinies. But women interviewed in clinics were equally frank. Next I grouped and read interviews according to the three neighbourhood locations. Alll interviews with Rimal residents and clinic users were read together. followed the same procedure for Jabalia MoH clinic and Jabalia UNRWA clinic, I marked unusual or representative responses. Representative responses were those issues that were raised in most of the groups. Unusual responses were those that revealed different reasoning or factors not previously cited by others. I was able to detect some differences in users views about the different clinics. Finally, I read and analyzed the interviews question by question, so that all the responses to each ques- tion were read one after another. Unusual and typical quotes were noted. Responses did not always closely match the questions. Therefore views and specific Tesponses were grouped under one of five themes which emerged from the data. These were choice of services, quality of care, favouritism, drug supply and quality, and issues of facilities, equipment and service. When we met to compare the themes and indexing, we decided to focus on choice of services and to each write up this section of the data separately. We chose this theme from the five as it was central to understanding patterns of utilization and underpinned issues in the other themes. Hundt’s use of the software Like manual analysis, I started by reading the data in hard copy. Then I had to format the data to make it software-friendly. There is no shortcut around stage. With the help of a research student who was familiar with the software, I set up a project name for the data and entered the interviews as documents, The interview notes were in Word 6. In order to be able to index them, each interview had to be con- verted into an ASCII file format and then entered as an on-line document within the project which was called “gaza women’s groups’. I renamed them all so that I had a list of interviews which were iden- tifiable by setting and location. (This meant that we both had our data sets differently numbered,How to do (or not to do) 375 something which could have been avoided if we had forescen it.) ‘This stage seemed cumbersome to me. Rather than getting on with developing a matrix around themes, T was struggling with creating a project file and loading documents 1 already had. The preliminaries seemed endless but this was mostly because 1 was learning the system. Similar formatting procedures are necessary for all software programmes (Aljunid 1996) and should not deter their use. Indexing ~ creating a tree with nodes Once the documents were on-line in a NUD. IST for- mat, I could embark on indexing by identifying themes in each interview and by creating categories. In NUD.IST this is done by creating a tree with nodes and branches. The terminology is anthropomorphic since the sub-categories are cailed children of larger categories which are parents. In this case the branches ff the tree (parent nodes) emerged from the topics of the interview and were equivalent to themes. Node 1 sorted the interviews according to setting (clinic or home) and according to neighbourhood. This meant that later on I could request the software to sort out all the data relating to a topic in a particular neighbourhood if T wished. The other nodes were the themes that emerged from the data. These were motives for using the clinic (Node 2), choice of ser- vice (Node 3), aspects of the services which are admired (Node 4), aspects which are disliked (Node 5), issues about drug supplies (Node 6), barriers to care (Node 7), most important aspects of care (Node 8), ideas for improvements (Node 9), health work ‘of mothers (Node 10), and childbirth experiences (Node 11). The sub-categories (children nodes) were topics within these main themes. Thus for each interview, you read the interview and assign sentences to different headings. You can summarize the index- ing by using the tree diagram (Figures | and 2) or alternatively listing the codes of each node. The tree developed 11 branches and 87 nodes. Many sentences seemed to fit into several categories. One of the handicaps was that the programme does not enable you to print out a list of all your nodes or to display the whole tree. In order to do this, you need to print out each node separately and pin them up on a noticeboard ‘The ability to categorize and index each sentence of the interviews meant that I felt certain that all the data was logged under nodes but sometimes the detail blurred the conceptual understanding of the material Tealso took a long time: T found T could only do 1 or? interviews at a sitting. Each interview took 40-60 minutes, and since days and weeks elapsed between indexing, I would forget the nodes. If you index the data continuously, with no lengthy breaks, you can keep a good grasp of the nodes and the procedure can be quicker. I was concerned that I developed a rigidity - that I had created nodes and sub-nodes from early interviews and continued with them in later ones even if my understanding changed The programme enables you to redesign your tree (merge and rename nodes) and this is a key strength enabling you to recategorize and reconceptualize the data easily which would take more time if wor manually. Subsequently you are able to cross reference or “intersect” nodes and review the data which are sorted, For example, you can request the software to intersect all the interviews of one neighbourhood, Rimal (node 1, 2, 3), with choice of Private doctor (node 3, 4). The data are sorted and on screen for review in minutes. The programme enables you to discard or save these intersected reports and addiction to this activity needs to be guarded against! Comparison of indexing Beckerleg had five main themes: i) Choice of ser- vice, ii) Quality of care, iti) Favouritism, iv) Drug supply and quantity, and v) Facilities, equipment and service. Hundt had eleven main themes. Two of the themes, namely Choice of service and Drug supply and quantity coincided. Beckerleg’s theme of Favouritism was a sub-category of Issues that are disliked in the clinic within Nudist, and Quality of care and Facilities, equipment and services were listed within Nudist indexing under Suggestions for improvement and Aspects of the clinic that people valued or disliked. The section of the data chosen for writing up after consultation on the themes identified was Choice of services Manual analysis write-up Women were able to command considerable choice in clinic use. Women in all three study areas said that they used a number of health services besides the government or UNRWA clinic serving their district. The group discussions revealed that women are able to compare a range of clinics from both sectors Movement between government and UNRWA clinics by both refugees and residents is common.