### A Brief Review of LQAS and B-LQAS

#### LQAS

Classical LQAS is primarily a classification procedure [16]. In its simplest form, the goal is to classify the unknown prevalence of a binary indicator as greater than or equal to some critical threshold,*p**, or less than this threshold. To do so, the number of cases,*Y*, in a simple random sample of size*n* are compared to a predefined*decision rule, d*. If fewer than*d* cases are observed, then the prevalence is classified as*low (p < p**). Otherwise, it is classified as*high (p ≥ p**). The sample size and decision rule are chosen to achieve error probabilities of*α* and*β*, where the former is the maximum acceptable probability of a false negative (or a false*low*) over some range of prevalences and the latter is the maximum acceptable probability of a false positive (or false*high*) over some other range of prevalences.

To define these ranges, LQAS uses upper and lower thresholds,*p*_{
U
}and*p*_{
L
}. The sample size and decision rule are chosen so that the probability of a false negative when the true prevalence is greater than or equal to*p*_{
U
}is less than or equal to*α*. Likewise, the probability of a false positive when the true prevalence is less than or equal to*p*_{
L
}is less than or equal to*β*. In the industrial literature, these errors are referred to as the consumer and producer risks. In health applications, generally*p*_{
U
}is chosen to be equal to the critical threshold*p** and*p*_{
L
}is chosen to reflect the desired detectable deviation from that threshold [16]. However, some have suggested using*p** =*p*_{
L
}[17], which might be appropriate depending on the application. When deciding on how to implement the procedure, it is important that the investigator keep in mind what*p* represents, particularly if*p* is the prevalence of an undesirable outcome.

The Operating Characteristic (OC) Curve completely summarizes any LQAS design. For a given value of*p*, this is defined as

OC\left(p|d,n\right)=\mathrm{Pr}\left(Y<d|p\right),

(1)

where*Y* is assumed to be binomially distributed with parameters*n* and*p*. Plotting this quantity for the entire range of*p* yields the desired curve. A satisfactory LQAS design will have the following properties:

\begin{array}{ccc}OC\left({p}_{U}|d,n\right)& \le & \alpha \\ 1-OC\left({p}_{L}|d,n\right)& \le & \beta .\end{array}

For example, in Figure1, we see anOC curve with*n* = 200 and*d* = 14, which corresponds to the LQAS design used by Deitchler*et al*[2,3] to classify the prevalence of malnutrition with a critical threshold*p** = 0.10 and with*p*_{
U
}= 0.10 and*p*_{
L
}= 0.05. In that application,*α* was set to 0.10 and*β* to 0.20. We see that theOC curve is less than*α* = 0.10 at the upper threshold and greater than 1 -*β* = 0.80 at the lower threshold, and thus meets the design requirements.

#### B-LQAS

B-LQAS is similar to classical LQAS in that the final goal is decide whether to classify the prevalence as greater than or equal to some threshold*p** , or less than this threshold, so that appropriate action be taken. However, in the Bayesian context, we allow a prior distribution of the parameter,*p*, to be part of the analysis. As with classical LQAS, we choose the sample size,*n*, and the decision rule,*d*, to achieve certain criteria when performing the classification*before* observing the data. In contrast to classical LQAS, these criteria are based on posterior properties of our decision, or what we believe*after* observing the data. For example, it might be important to know what is the probability we have made the correct decision, or classification, and we have the choice of two probabilities, depending on which decision we make.

To get a better understanding of the intuition behind this approach, consider Figure2, where we have plotted a hypothetical prior distribution of the prevalence of malnutrition. This distribution has mean 8.5% with 77% of its mass between 5% and 10%. Overlaid on this plot are twoOC curves. The solid curve corresponds to a classicalOC curve with*d* = 14, or the decision rule that we would choose using classical considerations. When we consider the prior distribution, we might argue that it makes less sense to use this decision rule, which prioritizes error above 10% and below 5%, since we seldom expect to see a prevalence as high or as low. The dashed line corresponds to a classicalOC curve with*d* = 28. With the chosen prior, this appears to be a better design as it prioritizes the region of largest prior mass when choosing a decision rule. That is, the design prioritizes correct classification of prevalences which are*most likely* given our prior beliefs. Hence, prior beliefs about the parameter of interest should play a vital role in determining an appropriate design, and in explaining its properties.

For the sake of illustration, in this paper we assume the conjugate Beta prior on the prevalence,*p* ~*Beta*(*a*,*b*), to demonstrate the effect on theOC curves. That is, we let the prior distribution,*π*(*p*), take the structural form

\pi \left(p\right)=\frac{{p}^{a-1}{\left(1-p\right)}^{b-1}}{B\left(a,b\right)},

where*a*,*b* > 0 and*B*(*a*,*b*) is the beta function [18]. The Beta provides a rich family of distributions, allowing for a range of flexible prior shapes. Further, there is some precedence for its use [10]. The parameters*a* and*b* control the shape of the prior distribution. To aid interpretation, we might think about*a* and*b* as the prior number of successes and failures, respectively. Therefore, a large value of*b* relative to*a* yields a distribution skewed to the left.

For pedagogical reasons, we choose these parameters to reflect a variety of potential prior beliefs (see Figure3A). For example, when*a* = 1 and*b* = 1, the prior is completely flat, which might correspond to a lack of prior knowledge or possibly prior indifference. When*a* = 2 and*b* = 10, this corresponds to a prior density with most of its mass below 30%, which is a realistic assumption as malnutrition prevalence is rarely as high as 30%. For example, in the CE-DAT global database of over 1400 malnutrition surveys conducted in emergency situations, only 59 reported a prevalence as high as 30% in children 6-59 months [19]. However, we also look at the case where*a* = 4 and*b* = 2 and where*a* =*b* = 5, reflecting a prior belief that the prevalence is in fact quite high, even though this condition is probably quite unlikely in the present context. The properties of a B-LQAS design can once again be formalized in theOC curves, although we now focus our attention on the BayesOC curves. A key difference between classical LQAS and B-LQAS is the reliance on not one, but two curves to determine appropriate designs, since we need to condition on either a high or low classification. In this paper, we define the following BayesOC curves

BO{C}_{P}\left(x|n,d\right)=\mathrm{Pr}\left(p<x|Pass\right),

(2)

BO{C}_{F}\left(x|n,d\right)=\mathrm{Pr}\left(p<x|Fail\right),

(3)

where the event*Pass* = {*Y* ≥*d*} and*Fail* = {*Y* <*d*},where*Y* is the number of "successes" in a sample of size*n*. Plotting (2) and (3) as a function of*x* yields the desired curves. We can write

\begin{array}{c}1-BO{C}_{F}\left(pU|n,d\right)=\frac{{\int}_{{p}_{U}}^{1}{\displaystyle \sum {}_{y=1}^{d-1}f\left(y|p\right)\pi \left(p\right)dp}}{{\int}_{0}^{1}{\displaystyle \sum {}_{y=0}^{d-1}f\left(y|p\right)\pi \left(p\right)dp}}\\ BO{C}_{P}\left({p}_{P}|n,d\right)=\frac{{\int}_{0}^{{p}_{L}}{\displaystyle \sum {}_{y=d}^{n}f\left(y|p\right)\pi \left(p\right)dp}}{{\int}_{0}^{1}{\displaystyle \sum {}_{y=d}^{n}f\left(y|p\right)\pi \left(p\right)dp}},\end{array}

where*π*(*p*) is the prior distribution of*p* and*f* (*y*|*p*) is the sampling distribution of*Y* given the parameter,*p*. As a result, the BayesOC curves are less straightforward to calculate than the classicalOC curves, as some integration over the unknown parameter is required. In some cases, these integrals can be analytically intractable, in which case one would have to appeal to numerical methods to evaluate the expressions. In any single application we ultimately take only a single action, but we need to consider both BayesOC curves. Note that in the case of malnutrition, if*Y* ≥*d*, this indicates a high burden of malnutrition. Therefore, the use of the word*Pass* is not instinctual. However, we might think of*Pass* as "qualifying for humanitarian aid" to facilitate the interpretation. We continue with this notation to provide a unified framework.

The interpretation of each of these curves allows us to make probabilistic statements about the parameter of interest, given the results of our diagnostic procedure; such as statements like those made by Bilukha and Blanton [5], which in their context are incorrect. Intuitively, it would be desireable to control for the probability that the prevalence is low when we say it is high (or declare a*Pass*), for example, and vice-versa. Using the BayesOC curves, we can choose*n* and*d* so that the Bayes classification errors are controlled. For an analogue to classical LQAS, we can enforce the following:

1-BO{C}_{F}\left({P}_{U}|n,d\right)\le \alpha

(4)

BO{C}_{P}\left({p}_{L}|n,d\right)\le \beta ,

(5)

where*p*_{
U
}=*p** and*p*_{
L
}is some lower critical threshold. However, it is also possible to choose*p*_{
L
}=*p*_{
U
}=*p**, which might be more appealing to some practitioners. This latter case is discussed at length in the context of Phase II clinical trials by Wang*et al*[20]. Ultimately, the choice depend on the application and the priorities of the investigators. We discuss this issue further in the next section.

In Figure3B, we see the BayesOC curve plotted as a function of the prevalence threshold*(x* in equations (2) and (3)) with*n* = 200 and*d* = 14. When*α* = 0.10 and*β* = 0.20, we see that the constraints posed in (4) and (5) for*p*_{
U
}= 0.10 and*p*_{
L
}= 0.05 are met for all considered priors. That is,

\begin{array}{c}1-BO{C}_{F}\left({p}_{U}|n,d\right)=\mathrm{Pr}\left(p\ge {p}_{U}|Fail\right)\\ \le \alpha =0.10,\\ BO{C}_{F}\left({p}_{U}|n,d\right)=1-\mathrm{Pr}\left(p\ge {p}_{L}|Pass\right)\\ \le \beta =0.20.\end{array}

Hence, when we choose*d* = 14, which corresponds to the classical solution, we achieve reasonable Bayesian properties as well.

Note that when we let*p*_{
L
}=*p*_{
U
}= 0.10, the error at the upper and lower thresholds increases slightly as compared to the case when*p*_{
U
}= 0.10 and*p*_{
L
}= 0.05 for the considered priors. That is, 1 -*BOC*_{
P
} < (*p* = 0.10 |*n*,*d*) decreases from essentially one, to just over 0.80, in the case when*a* = 2 and*b* = 10, which is still within the design constraints. This is important, however, as it highlights the tradeoff between classification precision and accuracy in these applications. Namely, by classifying as above*p*_{
U
}=*p** or below*p*_{
L
}, we are asking for slightly imprecise results, but doing so with high accuracy. However, if we classify as above or below*p*_{
U
}=*p**, we are asking for highly precise results, but doing so with less accuracy. We revisit this notion in more depth below.

### Maximizing the Figure of Merit

In general, the above outlined approach is feasible yet time consuming, as it might require an investigator to look at a range of sample sizes and decision rules to arrive at a given design. A more automated design selection is achieved by using the following*Figure of Merit* (FOM),

\begin{array}{c}FOM\left(n,d\right)={\displaystyle \int {}_{0}^{{p}_{L}}\mathrm{Pr}\left(Y<d|p\right)\pi \left(p\right)dp}\\ +{\displaystyle \int {}_{{p}_{U}}^{1}\mathrm{Pr}\left(Y\ge d|p\right)\pi \left(p\right),}\end{array}

(6)

and for a given sample size, one might choose the decision rule to maximize this quantity. In the decision theoretic literature, this quantity is known as the Bayes Risk of a zero-one utility (negative loss) function [1]. When*p*_{
U
}=*p*_{
L
}, theFOM of a given design is the average probability of correct classification given a prior*π*(*p*). When*p*_{
L
}<*p*_{
U
}, dividing the above quantity by a factor of 1 -*Pr*(*p*_{
L
}<*p* <*p*_{
U
}) yields the average probability of correct classification of priority locales. This scaling of theFOM does not affect the maximization, but can help with interpreting the result. Interestingly, (6) can be rewritten as

\begin{array}{r}FOM\left(n,d\right)=\mathrm{Pr}\left(Y<d\right)BO{C}_{F}\left({p}_{L}|n,d\right)\\ +\mathrm{Pr}\left(Y\ge d\right)\left[1-BO{C}_{P}\left({p}_{U}|n,d\right)\right],\end{array}

which shows that an optimal design is one that maximizes a weighted average of the BayesOC curves, where the weights are the marginal probabilities of passing and failing the procedure. The marginal distribution of*Y* is also referred to as the*prior predictive* distribution. That is, the predictive distribution of*Y* given only our prior assumptions. Hence, we weight more heavily the BayesOC curve which has the greater prior predictive probability of occurring.

Continuing with our example, Figure4A shows the plot of theFOM as a function of*d* for the situations when*n* = 200,*p*_{
U
}= 0.10,*p*_{
L
}= 0.05 and 0.10, and four prior distributions. When*p*_{
L
}= 0.05, the optimal decision rule hovers around*d* = 14, or the same as the classical LQAS solution. Yet, it is important to note that both when*a* = 5,*b* = 5 and*a* = 4,*b* = 2, the optimal decision rules are less than 14. Further, even though the curves are very nearly flat in the displayed range of rules, these are true maxima due to the fact that the prior mass below*p*_{
L
}is non-zero. Scaling the maximumFOM appropriately reveals that the average probability of correct classification of priority locales is close to 100%, indicating the appropriateness of the design for detecting extremes (i.e. areas where*p* ≤*p*_{
L
}or*p* ≥*p*_{
U
}).

When*p*_{
L
}=*p*_{
U
}= 0.10, the maximumFOM decreases, albeit slightly, and the optimal decision rule increases to*d* = 17 or*d* = 20, depending on the prior. Therefore, if our prior belief is that the malnutrition prevalence is low, we require a*greater* number of malnourished children in our sample to be convinced otherwise. But if we believe that the malnutrition prevalence is high, we will need fewer malnourished children in our sample to be convinced that the prevalence is indeed low, and thus possibly triggering an earlier intervention. This is a consequence of incorporating prior information into our analysis. In either case, it is important to realize that the optimal design does a good job of classifying areas. That is, in both cases, the maximumFOM is greater than or equal to 90%. Therefore, with a sample of size*n* = 200, the probability that we correctly classify an area is greater than or equal to at least 0.90.

It is interesting to note that when*a* = 1 and*b* = 1, the in difference prior, the optimal decision rule is equal to 20, or*np**. Both Bilukha and Blanton [5] and Rhoda*et al*[17] have suggested using*d* ≈*np** for the decision rule in the classical setting. The use of a flat prior with*p*_{
L
}=*p*_{
U
}=*p** gives a Bayesian justification for such a choice, although the use of a flat prior does not ordinarily make sense for this application, as it is uncommon for the prevalence of acute malnutrition to reach as high as 30%, much less 80% or 90% [19]. We discuss this in more depth in the following sections.

### Balancing Accuracy and Precision

TheFOM measures the overall*accuracy* of the B-LQAS procedure, and it is attractive to constrain our procedure to achieve at least a minimumFOM. Namely, we constrain theFOM so that

\begin{array}{ccc}OM\left(n,d\right)& \ge & 1-\delta ,\end{array}

(7)

where the parameter*δ* controls the overall level of accuracy in the procedure. This might be considered a more appealing design metric than*α* and*β*. Of course, when*p*_{
L
} =*p*_{
U
}, constraints (4) and (5) imply

\begin{array}{ccc}FOM\left(n,d\right)& \ge & \left(1-\alpha \right)\mathrm{Pr}\left(Y\ge d\right)\\ & & +\left(1-\beta \right)\mathrm{Pr}\left(Y<d\right).\end{array}

Hence, in this special case, the optimal design which meets (7) might be chosen as that design for which a weighted average of the producer and consumer risks, weighted according to the prior belief of passing or failing, is greater than or equal to 1 -*δ*. If*α* =*β*, then we have that*α* =*β* =*δ*, further simplifying the parameterization.

The precision demanded of the procedure impacts the accuracy. That is, the choice of*p*_{
U
}and*p*_{
L
}affects the properties of the design. Formally, define the*precision* as 1-|*p*_{
U
}-*p*_{
L
}|.When*p*_{
U
}=*p*_{
L
}, the precision is equal to one. But as*p*_{
L
}deviates from*p*_{
U
}, the precision decreases. Indeed, when at their maximal difference, the precision is zero. In our example,*p*_{
U
}= 0.10 and*p*_{
L
}ranges from 0.05 to 0.10, so that the precision ranges from 0.95 to 1.00.

In Figure4B, we plot the maximum average probability of correct classification of priority locales (or the appropriately scaledFOM) as a function of the precision, fixing*p*_{
U
} = 0.10 and allowing*p*_{
L
}to vary from 0.05 to 0.10. Therefore, when the precision is equal to 0.95, this corresponds to*p*_{
L
}= 0.05 and*p*_{
U
}= 0.10. When the precision is equal to one, then*p*_{
L
}=*p*_{
U
}= 0.10. Assume that we want a design that achieves an overall accuracy of 0.95 (1 -*δ* = 0.95). We see that for three of the four considered priors, the maximumFOM is well above 0.95 for all considered precisions, and therefore we should on average correctly classify over 95% of locales with these procedures. However, for the situation when*a* = 2 and*b* = 10, which is likely the more realistic prior for this application, the maximumFOM drops below 0.95 as*p*_{
L
}approaches*p*_{
U
}, or the precision approaches one. Hence, it is not always possible to achieve the desired level of accuracy for all precisions, short of increasing the sample size; illustrating the trade of between the two.