Practitioners and participants of SQUEAC assessments will most likely have experienced some uncertainty when building a prior for conjugate analysis between Stages II and III of a SQUEAC assessment. The step represents a seed change from exhaustive collection and triangulated analysis of data. After forming and testing hypotheses about our understanding of programme coverage, Stage III now requires us to introduce probability and belief in preparation for a final coverage estimate.

Naturally, practitioners seek a deeper understanding of how this part of the process works, and a basic understanding of Bayes Theorem is a pre-requisite. But essentially, to reach a percentage estimate of programme coverage with good accuracy purely from a survey would require a significant sample size (compare SLEAC methodology where 40 cases are required simply to classify coverage). It is also burdened with the usual practical limitations in terms of sampling, bias and errors in the field. But as a group of participants with a rigorous understanding of the positive and negative influences on coverage, the team are also in a position to make an ‘educated guess’ at what coverage is with varying degrees of certainty. Unless there is zero certainty (which would also mean no mode value), this guess allows us to reduce the amount of survey data required for an accurate estimate, and provides a reflection on the results of the new data through Bayes conjugate analysis.

As long as the two estimates of coverage are not conflicting (i.e. the p value of the z-test is less than 0.05), the conjugate analysis provides an accurate estimate according to both sources. Since the methods and limitations of new data collection are well-documented, common concerns about this phase tend to be focused on methods and limitations of prior building, which are the focus below.

The prior is a curve, with a shape. The shape depends on two elements: the mode value (what is our best guess) and the highest and lowest probable values we would estimate programme coverage could be. These elements together govern the shape of the prior curve and the latter relate directly to uncertainty i.e. the range of probable values for coverage.

Despite curiosity about the reliability of a prior estimate, a review of 267 SQUEAC assessments presents strong evidence that supervisors can properly specify a prior with accurate mode and appropriate levels of uncertainty in order to avoid prior-likelihood conflicts (only one Stage III from 227 available for analysis resulted in conflict).

So, what is the best way to formulate a prior? How do we start? What methods can we use to find a mode value? What if we get it wrong? How do we know how strong the prior is?

On approaching the building of the prior, time can be well spent reviewing all the information gathered so far through studying, amending and completing any concept maps, mind maps and boosters and barriers records.

The tools used to develop an overall view of coverage (e.g. boosters and barriers, concept map, mind map) are all quantifiable by assigning each element as a positive or a negative influence. For example for concept maps this would be each relationship, or for boosters and barriers, this is already assigned. In this way, each tool can be used to reduce the range of probable values from 0% and 100% to other minimum and maximum probable values and a mid-point can be found.

 At the very least a supervisor will know at this stage if coverage is likely to be very high, something moderate, or whether the team have an unrealistic view of programme coverage. The latter usually manifests as ‘wishful thinking’ where the team over-estimate coverage regardless of evidence indicating low coverage. Examples of prior building for each of these scenarios can be found here.

Also at this stage, each individual from the team will be able to make a rational guess at what coverage might be, which together with this fresh overview of all information, can be well-utilised for drawing a histogram of the investigation team’s belief of what coverage is likely to be. This exercise also yields an estimate of each element required for manual calculation of the shape parameters of the prior (the mode, minimum and maximum values).

Generally, the most developed tool will be the boosters and barriers tool where it has become standard practice to use both a simple mode from a count of negative and positive factors (as above) and a weighted sum of negative and positive factors that incorporates emphasis on factors thought to have the greatest negative or positive influence on programme coverage.

When using the weighted boosters and barriers method attention should be given to the scoring systems and some basic rules apply:

• Only integer values should be used. Whether scoring each booster or barrier (1-3) or (1-5) in terms of their level of influence, the accuracy of scoring these by fractions is spurious. After all, this exercise is guiding a guess.
• No booster or barrier should be scored zero. To do so is to imply zero influence and therefore the factor is neither a booster nor a barrier.
• Scoring should be applied in a comparable and consistent manner. As long as this rule is practiced, results from different systems (i.e. 1-3 or 1-5) will be similar.

Another method of weighting boosters and barriers includes participatory weighting in the community. This is useful in ensuring the assessment reflects the priorities of the community (see more here). The community does not need to know the SQUEAC methodology to correctly express their opinion on which barriers and boosters they consider important/less important, however it is important that community members are well guided through the exercise to ensure the outputs are of a high quality. Similarly, the Link NCA methodology also engages the community in a ranking exercise for causes of malnutrition, information on this can be found here (on page 88).

Once at least three prior modes have been estimated, a mean average of these can be calculated to use in the BayesSQUEAC calculator. However, if there are significant differences among them, it is generally a very good indication that something is going wrong somewhere. This should not happen, and each method should be reviewed.

Guidance for quantifying uncertainty suggests between +/-20% (when more certain) and +/-25% (when less certain) for the prior curve. However, using the histogram for minimum and maximum probable values allows manual calculation of shape parameters for the BayesSQUEAC calculator, therefore reflecting more specifically the assessment at hand.

But here, a word of warning against overly strong priors (i.e. with little uncertainty). The stronger the prior, the lower the sample size for collection of new data. However, the narrower curve that results has less chance of overlap with the curve from the new data, i.e. more chance of conflict. So remember to recognise your uncertainty.

“It is not about getting very close to the truth… it is about getting nearer to the truth than a random guess” – Mark Myatt