Methodology | Open | Published:
Assessing household wealth in health studies in developing countries: a comparison of participatory wealth ranking and survey techniques from rural South Africa
Emerging Themes in Epidemiologyvolume 4, Article number: 4 (2007)
Accurate tools for assessing household wealth are essential for many health studies in developing countries. Household survey and participatory wealth ranking (PWR) are two approaches to generate data for this purpose.
A household survey and PWR were conducted among eight villages in rural South Africa. We developed three indicators of household wealth using the data. One indicator used PWR data only, one used principal components analysis to combine data from the survey, while the final indicator used survey data combined in a manner informed by the PWR. We assessed internal consistency of the indices and assessed their level of agreement in ranking household wealth.
Food security, asset ownership, housing quality and employment were important indicators of household wealth. PWR, consisting of three independent rankings of 9671 households, showed a high level of internal consistency (intraclass correlation coefficient 0.81, 95% CI 0.79–0.82). Data on 1429 households were available from all three techniques. There was moderate agreement in ranking households into wealth tertiles between the two indicators based on survey data (spearman rho = 0.69, kappa = 0.43), but only limited agreement between these techniques and the PWR data (spearman rho = 0.38 and 0.31, kappa = 0.20 and 0.17).
Both PWR and household survey can provide a rapid assessment of household wealth. Each technique had strengths and weaknesses. Reasons for differences might include data inaccuracies or limitations in the methods by which information was weighted. Alternatively, the techniques may measure different things. More research is needed to increase the validity of measures of socioeconomic position used in health studies in developing countries.
Research into the socioeconomic determinants of health requires accurate tools for assessing socioeconomic position. While in developed countries pre-existing data are often available, these resources rarely exist in developing countries and original data must be collected . Economists generally regard detailed data on household income and/or expenditure as the gold-standard measure of current socioeconomic position. However, health researchers rarely have the resources or expertise necessary to conduct such assessments. Furthermore, total wealth, reflecting the balance between income and expenditure over a longer period, may be a more appropriate marker of socioeconomic position when health outcomes are considered. Consequently, rapid techniques for assessing household wealth are needed.
A variety of proxy measures of socioeconomic position have been developed. These have included shortened income or expenditure questionnaires, and measures of housing quality, education or nutritional status  Recently, researchers have used statistical techniques to combine multiple socioeconomic variables, usually including at least data on housing and ownership of fixed assets, into a measure of household wealth. The aggregation of such data can be achieved through a simple count, weighting of variables based on local consultation, or through the application of statistical procedures such as principal components analysis (PCA) [2–7]. However, there is no consensus on what variables should be included in such analyses . Furthermore, there remains limited evidence on the association between asset indices and more established measures of wealth or socioeconomic position [9, 10].
An alternative technique is to use participatory wealth ranking (PWR), in which community members rank the wealth of households in their community. This approach is widely used in development practice , but rarely used in health studies. PWR can generate useful statistics and provide valid information on relative wealth [12–16].
We conducted a household survey and PWR in rural South Africa. We constructed three indicators of household wealth, using the data from each of the two techniques separately and also by combining them. We assessed internal validity where this was possible, assessed agreement between the results of the techniques in their ranking of household wealth, and investigated the reasons for any differences.
The study was conducted in eight rural villages of Limpopo Province, South Africa. The province is among the most deprived in the country, with nearly 50% of the population under 15 years old, unemployment in excess of 40%, and high levels of labour migration [17–20]. The data come from the baseline evaluations of a cluster randomised trial .
Participatory Wealth Ranking (PWR)
PWR was conducted in the local language by specialised facilitators from a local development NGO (Small Enterprise Foundation, Tzaneen). Data were recorded on pre-designed data collection forms [22, 23].
Community members residing in the same village section, most often women from poor households, drew a map of their residential area and listed the households on cards. Following this, groups of 4–6 residents were asked to characterise households that were "very poor", "poor, but a bit better off", and "doing OK". The proceedings of this discussion were captured by the facilitator in the form of "general statements". Households were then ranked from the poorest to the wealthiest according to these definitions and piles of households of comparable wealth generated. Participants were then asked to describe the characteristics of the households in each ranking pile ("pile statements"). Neither the number of wealth ranks nor the number of households in each rank was determined in advance, although at least four separate piles had to be generated during the process.
The ranking process was then repeated twice more with different groups of four to six community members, so that statements were collected and each household ranked on three separate occasions.
A random sample of approximately 200 dwellings from each village (total N = 1640) was visited at least three times where necessary to collect data. Interviews were conducted in the local language. Interviewers received extensive training and data entry was validated through data cleaning procedures. Questionnaires captured salient aspects of socioeconomic well-being including household members' education and employment status, details of the dwelling construction, ownership of a small number of assets, details of the most important household incomes and information on food security.
Generating indicators of household wealth
Three approaches were used to generate a measure of relative household wealth. The first used data only from the participatory wealth ranking; the second used data from the household survey, but with their selection and weighting informed by PWR; the third used only data from the survey, employing principal components analysis (PCA) to determine the weights.
Method 1: an index of household wealth from PWR
Details of the scoring method used are provided in detail elsewhere . Briefly, within each of the three ranking processes, piles of households were assigned a score such that the wealthiest pile received a score of 100 and the poorest pile a score of 0. Scores for the remaining piles were calculated as Score for pile n = 100*((N-n)/(N-1)), where n was the pile number and N was the total number of ranking piles.
Coded pile statements made in relation to the piles generated were assigned the numeric score allocated to the pile. An average pile statement score was calculated as the mean of the pile scores to which that statement was associated, covering the full PWR process in all eight villages (Table 1). A wealth index was calculated for each household as the mean of the pile statement scores of all the pile statements made in relation to the piles into which each household was ranked.
Method 2 : an index of household wealth from household survey data informed by PWR
Survey data were used to generate an indicator of household wealth, using PWR to inform which factors to use and how to weight the data. Where data were available on aspects of household wealth relating to each of the 10 commonest themes identified in PWR, this was used to inform the calculation of the index of household wealth (Table 2). Broadly, where relevant PWR pile statements identified "very poor" households, the most related survey item was given a score of -2, and where relevant statements identified households "doing OK" the associated survey item was scored 2. A sliding scale for intermediate situations was developed where this was possible. For school attendance, scoring was stratified on the basis of age. On the basis of this scoring system, each household could receive a maximum score of 9 (wealthiest) and a minimum score of -10 (poorest).
Method 3 : an index of household wealth from household survey data with weightings assigned by PCA
Fourteen variables capturing salient aspects of household wealth, decided upon a priori following literature review and piloting in the local area, were made available for entry into the PCA. Items included were not limited to durable assets  (Table 3). Asset values were derived from the survey data by multiplying the number of owned assets that were new (less than 2 years), relatively new (2–6 years), or old (>6 years) by estimations of the value of those assets, which came from a small sub-study. Other variables were drawn from the questionnaire. Non-continuous variables were coded even-spaced ordinally.
Two factors not associated in the expected direction with the value of selected non-livestock assets per person (screening variable) in a χ2-test (p < 0.05) were not included in the PCA (percentage of household members of working adult age and land tenure). The remaining factors were included. PCA transforms a set of correlated variables into a set of uncorrelated 'components'. When variables hold information about some underlying concept, PCA can produce the best single composite variable among all possible linear functions of the original variables . The component explaining the greatest proportion of the total variance is called the first principal component. This weights the data in proportion to how well each variable is correlated with the others and was used as the indicator of household wealth.
A number of analyses were run. Factors with component loadings less than 0.2 on the first principal component were excluded (household electricity supply, quality of water supply and the nature of the second most important ranked household income). Nine factors were included in the final analysis in which the first principal component explained 22.7% of the variance of the factors in the model. The greatest weight was given to the density of household living conditions (scoring coefficient = 0.44), with the value of non-livestock assets (0.42) and the food security indicator (0.39) also being important. The lowest weighting was given to the proportion of individuals receiving an income (0.23). A wealth index was calculated for households where data were available on all variables. This component was normally distributed and had a mean of 0 and a standard deviation of 1.
Statistical analysis of consistency and agreement
For the PWR method only, the intra-cluster correlation coefficient, a measure of internal consistency, was first calculated from a random-effects ANOVA to describe the level of agreement in rankings of wealth between each of the three rankings made for each household .
Secondly, the association of each index with the individual survey indicators was estimated. Households were divided into wealth-rank tertiles on the basis of each of the methods. The association between these tertiles of wealth and each specific indicator of wealth from the survey was assessed using a χ2-test.
Finally, the three techniques were compared in their ranking of household wealth. The agreement of each technique placing households into wealth tertiles was estimated with a kappa coefficient. Spearman rank correlation coefficients were also calculated. While correlation coefficients measure the level of predictability of one variable on the basis of another, they do not directly assess agreement; thus a correlation coefficient of 1 will be measured if all values of one variable are twice that of another, though these clearly do not agree.
The wealth ranking process identified 9824 dwellings in 79 village sections in the eight villages, and wealth ranking data were available for 9671 of these (98.4%). Some 3556 general statements were coded describing the general properties of households seen as "very poor" (1240), "poor, but a bit better off" (1097) or "doing OK" (1216). A further 8856 pile statements were coded, describing the characteristics of the households included in each of the piles assembled by the wealth ranking process. Some 47 statements were made more than 15 times in both stages of the process (Table 1), with all but one of the statements ("Got jobs/employed") being mentioned exclusively in relation to a single wealth category. Successful interviews were completed with 1482/1640 (90.4%) households.
Distribution and determinants of wealth
Households judged "very poor" by PWR participants were struggling to feed themselves and to clothe or educate their children, with little access to jobs or housing (Table 1). Households deemed "poor, but a bit better off" had access to low paid jobs and exhibited a basic ability to meet food and educational needs. Finally, households that were "doing OK" had access to good food, drove cars and had big or attractive housing. Some members were employed in high-return and/or high-stability activities.
Survey data (Table 3) suggested modern assets were widely distributed, though 28.2% of households reported owning none of the listed assets. Livestock assets were common. Dwellings were built of simple materials. Some 18.4% of households had no access to a toilet. Electricity supply was determined largely by village, with two villages remaining largely unelectrified. Water accessibility was generally low. Some 19.7% of households had no adults receiving a regular income, while many households were headed by an individual with no education (38.0%). Some 35.5% of households often had a meal consisting only of basic foodstuffs. Cars were owned by 19.0% of households. School attendance was high for young children but lower at later ages.
Internal consistency of PWR
The single-measure intra-class correlation coefficient from a random effects two-way ANOVA on the three rankings of household wealth, over 9671 households, was 0.81 (95%CI0.79–0.82) denoting a high level of agreement.
Association between wealth indices and different dimensions of wealth
Data on individual socioeconomic variables were significantly correlated (p < 0.01) with each of the wealth indices in most cases (Table 4). Land tenure was least strongly associated with the PCA measure (p = 0.026). Household electrification was not strongly associated with the measure of household wealth generated by either of the methods that used the survey data, although it was associated with the PWR index (p = 0.002). Water accessibility was least strongly associated with the PWR index (p = 0.028). Finally, the proportion of adults who were of productive age (14–60 years) was not strongly associated with household wealth as estimated by any of the techniques.
Agreement between the indices
The survey data methods were quite strongly correlated (Spearman rho = 0.69, p < 0.001, n = 1442), and there was a reasonable degree of agreement in their placing of households into wealth-rank tertiles (Kappa = 0.43).
The PWR wealth index was significantly, but weakly, correlated with both the index combining PWR and survey information (Spearman rho = 0.38, p < 0.001, n = 1443) and the PCA-based method (Spearman rho = 0.31, p < 0.001, n = 1451). The levels of agreement in placing households into wealth tertiles were low (kappa statistics of 0.20 and 0.17 respectively).
We constructed three indices of household wealth using data from a household survey and participatory wealth ranking. PWR and the survey identified similar dimensions of socioeconomic well-being as important. The two indices developed from survey data showed a reasonable level of agreement in ranking households into wealth tertiles. However, there was limited agreement between the survey-data based indices and the index based only on information from PWR. Methodological differences meant that it was not surprising that the methods differed in their results, though the magnitude of the differences noted was surprising.
The three approaches differed in at least two dimensions. The first dimension was whether information was provided by household members (as for both of the techniques using survey data), or by other community members (for the PWR only approach). The second dimension was whether community views were used to weight the importance of different aspects of wealth (as for the approaches that used PWR data), or whether external statistical rules were used (as in the PCA method). Nevertheless, there were striking similarities in the associations seen between the three wealth indices and each of the survey variables collected. The strongest associations between individual variables and the PWR wealth index were seen for variables associated at a significance level of p < 0.001 with both survey indices, while weaker associations also generally mapped across all three indices. The only exceptions to this were with the variables on household electrification and water supply.
Despite these similarities, the PWR index showed relatively low agreement with the survey-based measures, even when themes from the PWR were used to inform the selection and weighting of data. Two potential reasons for the lack of agreement are; firstly, each may have suffered from inaccurate data collection or weighting; secondly, the techniques may measure different things.
The survey attempted to maximise accurate reporting through collecting data on objective indicators, fieldworker training and stressing the importance of honesty to participants. Nevertheless, reporting biases may have occurred . PWR partially accounts for this, since information is acquired from neighbours and is triangulated. However, households may conceal information from their neighbours. PWR may therefore best measure conspicuous consumption. PWR participants might also misreport household wealth. However, the high level of internal consistency for the household wealth ranks obtained from three separate groups of PWR participants provided some evidence against this. This finding differs from a previous report of low reliability for group-informant food-security ratings . However, reasons for the low reliability reported by those authors were addressed in this study since trained facilitators worked with a homogenous group of PWR participants at all rankings and emphasised local definitions of poverty, participation and consensus. However, PWR may provide invalid results, but high levels of internal consistency, if participants, who were mostly poor women, ascribe a greater weight to certain dimensions of poverty (for example, being widowed) than would other groups in society.
Survey data included information on employment, educational status and asset-ownership of migrants, since temporary migrants are important contributors to the rural economy in South Africa [28–30]. However, no information was available on levels of income remittance. PWR participants may be poorly informed about the wealth of migrants or their levels of remittance. However, PWR participants may also have had a more nuanced understanding of the role of migrants in generating household wealth than it was possible to capture from the survey data.
Each method might have weighted the importance of different aspects of household wealth differently. PCA assigns weights to variables according to mathematical rules, while wealth ranking participants assess households in ways that are complex and non-transparent. Our approach to PCA incorporated different facets of wealth, as in previous applications,  and drew out the common underlying correlation between them. However, the first principal component explained only 22.7% of the total variance, suggesting that factors included were not well correlated. The index where PWR was used to inform the selection and weighting of survey data has intuitive appeal. However, it was not possible to directly map PWR statements to survey data, and the weighting system applied to the data was somewhat arbitrary. While combining data on multiple dimensions of socioeconomic well-being should provide a more stable marker than individual variables on their own, the selection of variables for inclusion in such analyses requires further study, as does the widespread practice of including ordered categorical and binary variables in PCA.
Finally, there was also room for differences in interpretation in PWR. Wealth ranking was conducted in Sepedi, applying a translation of the question, "What are the characteristics of a very poor household?" to start the ranking process. Many characteristics identified by PWR participants resonated with the survey data. Nevertheless, the way in which PWR participants judge household wealth was inevitably unclear. One possibility is that PWR participants may have ranked households more directly on their current level of welfare than the survey based methods.
In our comparison of three approaches to assessing household wealth, the method by which data were collected was more important than the method by which variables were selected or weighted in determining agreement between the rankings. None of the techniques was precise in defining what aspects of wealth they wished to measure, so ultimately the indices may have measured different things. Survey data on individual variables may be most appropriate when comparison is needed between different settings or time-periods. PCA is a useful tool for the reduction of multiple indicator data, yet in this application did not agree with household wealth ranking ascribed by community members. PWR allowed a measure of wealth to be generated for about 200 households in a given geographical area over a two-day period by a skilled practitioner. Although the use of this technique will require epidemiologists to attain new skills, PWR may represent a rapid, useful and internally valid tool for health researchers in situations where locally-grounded data on household wealth are required.
Morris SS, Carletto C, Hoddinott J, Christiaensen LJM: Validity of rapid estimates of household wealth and income for health surveys in rural Africa. J Epidemiol Community Health. 2000, 54: 381-387. 10.1136/jech.54.5.381
Booysen Fle R: Using demographic and health surveys to measure poverty- an application to South Africa. J Econ Economet. 2002, 26 (3): 53-70.
Durkin MS, Islam S, Hasan ZM, Zaman SS: Measures of socioeconomic status for child heath research: comparative results from Bangladesh and Pakistan. Soc Sci Med. 1994, 38 (9): 1289-1297. 10.1016/0277-9536(94)90192-9
Filmer D, Pritchett LH: Estimating wealth effects without expenditure data - or tears: an application to educational enrolments in states of India. Demography. 2001, 38 (1): 115-132.
Henry C, Sharma M, Lapenu C, Zeller M: Assessing the relative poverty of microfinance clients. A CGAP operational tool. Washington D.C. , International Food Policy Research Institute; 2000.
Sahn DE, Stifel D: Exploring alternative measures of welfare in the absence of expenditure data. Rev Inc Wealth. 2003, 49 (4): 463-489. 10.1111/j.0034-6586.2003.00100.x.
Schellenberg JA, Victora CG, Mushi A, de Savigny D, Schellenberg D, Mshinda H, Bryce J: Inequities among the very poor: health care for children in rural southern Tanzania. Lancet. 2003, 361: 561-566. 10.1016/S0140-6736(03)12515-9
Houweling TA, Kunst AE, Mackenbach JP: Measuring health inequality among children in developing countries: does the choice of the indicator of economic status matter?. Int J Equity Health. 2003, 2 (1): 8 10.1186/1475-9276-2-8
Lindelow M: Sometimes more equal than others: How the choice of welfare indicator can affect the measurement of health inequalities and the incidence of public spending. Oxford , Centre for Study of African Economies, University of Oxford; 2002.
Vyas S, Kumaranayake L: Constructing socio-economic status indices: how to use principal components analysis. Health Policy Plan. 2006, 21 (6): 459-468. 10.1093/heapol/czl029
McGee R: Analysis of participatory poverty assessment (PPA) and household survey findings on poverty trends in Uganda. Brighton , Institute for Development Studies; 2000.
Adams AM, Evans TG, Mohammed R, Farnsworth J: Socioeconomic stratification by wealth ranking: Is it valid?. World Dev. 1997, 25 (7): 1165-1172. 10.1016/S0305-750X(97)00024-7.
Barahona C, Levy S: How to generate statistics and influence policy using participatory methods in research. Reading , Statistical Services Centre; 2002.
Chambers R: Participation and numbers. Q-Squared Working Paper No 13 2005.
Scoones I: Investigating difference: Applications of wealth ranking and household survey approaches among farming households in Southern Zimbabwe. Dev Change. 1995, 26: 67-88.
Temu AE, Due JM: Participatory appraisal approaches versus sample survey data collection: a case of smallholder farmers well-being ranking in Njombo District, Tanzania. J Afr Econ 2000.
Kahn K, Collinson M, Tollman S, Wolff B, Garenne M, Clark SJ: Health consequences of migration: Evidence from South Africa's rural northeast: Johannesburg. ; 2993.
Lestrade-Jefferis J: The labour market. In The People of South Africa Population Census 1996 Edited by: Udjo EO. Pretoria , Statistics South Africa; 2000.
McIntyre D, Muirhead D, Gilson L: Geographic patterns of deprivation in South Africa: informing health equity analyses and public resource allocation strategies. Health Policy Plan. 2002, 17 (Suppl 1): 30-39. 10.1093/heapol/17.suppl_1.30.
Udjo EO, Lestrade-Jefferis J: Demographic Profile of South Africa. In The People of South Africa Population Census 1996 Edited by: Udjo EO. Pretoria , Statistics South Africa; 2000.
Pronyk PM, Hargreaves JR, Kim JC, Morison LA, Phetla G, Watts C, Busza J, Porter JDH: Effect of a structural intervention for the prevention of intimate partner violence and HIV in rural South Africa: results of a cluster randomized trial. Lancet. 2006, In Press:
Simanowitz A, Nkuna B: Participatory Wealth Ranking Operational Manual. Tzaneen , Small Enterprise Foundation; 1998.
Simanowitz A, Nkuna B, Kasim S: Overcoming the obstacles of identifying the poorest families. Unpublished Report ; 2000.
Hargreaves JR, Morison LA, Gear J, Porter JDH, Makhubele MB, Kim JC, Busza J, Watts C, Pronyk PM: "Hearing the voices of the poor": Assigning poverty lines on the basis of local perceptions of poverty; a quantitative analysis of qualitative data from participatory wealth ranking in rural South Africa. World Dev. 2007, 35 (2): 212-229. 10.1016/j.worlddev.2005.10.021. 10.1016/j.worlddev.2005.10.021
Muller R, Buttner P: A critical discussion of intraclass correlation coefficients. Stat Med. 1994, 13: 2465-2476. 10.1002/sim.4780132310
Falkingham J, Namazie C: Measuring health and poverty: a review of approaches to identifying the poor. London , DFID; 2002.
Bergeron G, Morris SS, Medina Banegas JM: How reliable are group informant ratings? A test of food security ratings in Honduras. World Dev. 1998, 26 (10): 1893-1902. 10.1016/S0305-750X(98)00087-4.
Hosegood V: An alternative approach to defining households in Southern Africa. In African Households Edited by: Van de Walle E. Illinois , Illinois Press; 2003.
Lurie M: Migration and AIDS in southern Africa: a review. S Afr J Sci. 2000, 96: 343-347.
May J, Carter M, Posel D: The composition and persistence of poverty in rural South Africa:An entitlements approach. In Policy Paper 15 Johannesburg , Land and Agriculture Policy Centre; 1995.
The study received financial support from AngloAmerican Chairman's Fund Educational Trust, AngloPlatinum, Department for International Development (UK), The Ford Foundation, The Henry J. Kaiser Family Foundation, HIVOS, South African Department of Health and Welfare, and the Swedish International Development Agency. Thanks also to Chris Martin, an intern with RADAR who coded many of the wealth ranking statements. James Hargreaves is supported by an ERC/MRC interdisciplinary fellowship.
The author(s) declare that they have no competing interests.