The three stage method had a low sensitivity of 48.6%, but cost 63% of the two stage survey. Sensitivities were slightly higher for the one-stage (57.1%) and two-stage (51.4%) methods. Specificities were marginally lower in single-stage (97.7%) compared to the three-stage method (100%). It would be important to improve the sensitivity of the questions in stage I in future work. However, estimates of prevalence obtained using the current three stage method can be adjusted to allow for the low sensitivity where this method is likely to be useful due to logistical and cost considerations e.g. in HDSS sites.
In this study, we examined a population-based three-stage system for detecting ACE (where a subject was classified as a case if they were positive in all the three stages). Other studies have validated only hospital-based survey methods [17, 20, 32] or determined the validity of a screening tool (within a three-stage method) in detecting epileptic seizures but not epilepsy . Unlike the validation of population-based studies, generalization of validation parameters of hospital-based studies is limited by selection bias (due to overrepresentation of cases with more severe forms of epilepsy). Additionally, people who lack knowledge of epilepsy may fail to seek treatment and therefore are not captured in hospital-based validations. Other limitations in hospital-based validation studies are inadequate sample sizes and they may not represent the field situation. Thus these studies therefore provide less accurate (often inflated) estimates of sensitivity than population-based validation . The validation of population-based methods is applied directly to a wider population, hence the findings are more generalizable, although this is usually costly. Furthermore, compared to validation of population-based studies, the validity of hospital-based studies may not be influenced by stigma-related concealment of seizures.
The precision of the validation depends upon the sample size, which for our study was statistically determined and was larger than for other validation studies [17, 32, 33]. In our study, the number of false positives was highest in SI; since the SI questions targeted all (including febrile) convulsions. Even a small proportion of false positives in a large population would have considerable logistical and cost implications. For instance, 2.3% of false positives among this study population of 233,880 individuals results to 5379 false positive individuals that would otherwise have to be screened in SIII of the prevalence survey. Inclusion of SII substantially reduces the false positives screened in SIII, which in our study declined from 2.3% to 0.1%.
We found that multi-stage methods (SI&SII and SI&SII&SIII) had poorer sensitivity than SI alone. This was observed in another validation study where (like in our study) if only epilepsy specific questions were used, sensitivity was substantially lower (and the specificity higher) than when questions on epilepsy and other seizures were used [33, 34]. These observations suggest that questions about seizures under any circumstances are important in avoiding false negatives.
In our study, stigma-related non-response could be the main cause of the low sensitivity of the three-stage method. This is suggested by the moderate to high perceived stigma scores for all the false negatives compared to the epilepsy population (those identified in the prevalence survey) in which only 33% felt stigmatized . Stage I (which contributed the largest proportion of false negatives i.e. 15/18) was conducted by HDSS field staff who are usually resident in the study community and who were also involved in the routine re-enumerations within the study area. The sensitivity of SI may also depend on the cultural setting (e.g. SI screening questions might be more frequently misinterpreted in communities with low literacy levels) and the skills of the field staff in administering sensitive questions (pertaining to a stigmatized condition). For example, the percentage of false negatives was much lower (23.4%) in an Australian when a screening questionnaire was administered to study known (physician diagnosed) epilepsy cases . In contrast, the clinical survey (our “gold standard”) was conducted by clinicians experienced in the diagnosis of epilepsy, of whom individuals with epilepsy or their guardians may have been more trusting and/or have expected benefits such as treatment or advise. A high sensitivity (79.3%) was estimated in another field validation in which rural doctors conducted an equivalent of SII of our prevalence survey . However, it is difficult to compare our validation with this study due to differences in case definitions and sources of cases [12, 17].
A lack of awareness of convulsions in family members by the household heads could also have led to the low sensitivity of SI. Household heads (or their spouses) were the primary respondents in the HDSS re-enumeration. Individuals identified as having convulsions by the household head in SI (or their caretakers if they were children or cognitively impaired) were interviewed in the subsequent survey stages and in the Clinical Survey.
Three cases of epilepsy were lost in the prevalence survey between SI and SII because field personnel who conducted SII were unable to distinguish between febrile and non-febrile convulsions. The sensitivity of the three-stage method could therefore be improved by better training the SII field staff to distinguish between different types of convulsions.
A two-stage epilepsy survey conducted by clinicians would be more expensive and of longer duration than a three-stage survey in which the initial stages are carried out by non-medical field personnel. This is because medically trained personnel require higher salaries and lack the field-work experience of lay field workers. However, for this additional expenditure, the improvement in sensitivity would be limited. Only false negatives in SII would be eliminated and so the sensitivity would at best be equivalent to the sensitivity of SI.
Limitations of the study
A major limitation of this study was the attrition of subjects over the two years of the Clinical Survey. These losses were observed despite efforts to locate individuals. There were, however, no differences in age, sex and SI status (either positive or negative in SI of the three-stage screening) between those lost and those that completed the study.
There could have been an increase in awareness of epilepsy in this population following the three-stage prevalence survey. This could have reduced stigma and increased knowledge of epilepsy, yielding more positive responses within the Clinical Survey, since the latter survey was conducted over a 2 year period after the prevalence survey. This hypothesis is supported by a study conducted on the same population after the prevalence survey which reported only moderate levels of perceived stigma .