With the results of the small study, small survey or small area survey which they carry out during Stage 2 data collection, surveyors need to validate or disprove the hypotheses they set. This page outlines how this is done and what actions to take with different outcomes.
Surveyors can validate or disprove hypotheses using data collected during small surveys and small-area surveys using a simplified LQAS (Lot Quality Assurance Sampling) technique.
SAM is a relatively rare phenomenon. This means that the sample size (i.e., the number of cases found) in small surveys and small-area surveys will usually be too small to estimate coverage with reasonable precision (i.e., as a percentage with a narrow 95% confidence interval). It is possible, however, to classify coverage accurately and reliably (i.e., as being above or below a standard or threshold) with small sample sizes with the LQAS technique.
This technical note explains in greater detail why, even with small samples, the LQAS technique can be used during a SQUEAC survey to validate or invalidate hypotheses.
Analysis of data using the simplified LQAS classification technique involves examining the number of cases found (n) and the number of covered cases found:
The threshold value (d) depends on the number of cases found (n) and the standard (p) against which coverage is being evaluated.
The Sphere minimum standard for coverage of OTPs in rural settings is 50%. The following rule-of-thumb formula may be used to calculate a value of d appropriate for classifying coverage as being above or below a standard of 50% for any sample size (n):
The ⌊ and ⌋ symbols mean that you should round down the number between the ⌊ and ⌋ symbols to the nearest whole number.
With a sample size (n) of 11, for example, an appropriate value for d would be:
For standards other than 50%, the following rule-of-thumb formula may be used to calculate a suitable threshold value (d) for any coverage proportion (p) and any sample size (n):
For example, with a sample size (n) of 11 and a coverage proportion (p) of 70% (i.e., the Sphere minimum standard for coverage of OTPs in urban and camp settings), an appropriate value for d would be:
The sample size (n) is seldom decided in advance of collecting data but is the number of current SAM cases (or current and recovering SAM cases) found by a survey. This is usually limited to the number of cases that can be found by a single survey team in a single day. The appropriate value for d is calculated after the survey data have been collected.
A template excel sheet is available in the Tools section which analyses the data gathered during small-area and small surveys and either validates or disproves hypotheses.
If the hypothesis being tested in a small study can be expressed quantitatively then the simplified LQAS classification technique may be used to analyse the study data.
For example, the study hypothesis is:
Less than 80% of cases that have been in the outpatient treatment program (OTP) for at least 4 weeks and have failed to gain weight have received counselling from clinic staff.
Examination of 102 beneficiary record cards found 13 children that had been in the program for at least 4 weeks and had failed to gain weight. Short interviews with the carers of these children revealed that 4 of them had received counselling from SFP staff. The decision threshold is:
Since 4 is not greater than 10, the hypothesis is confirmed.
Once the hypotheses results are confirmed, surveyors should decide what next steps to take. There are three possible scenarios:
|The hypothesis is proven||Surveyors have confirmation that the barriers identified are having an impact on coverage and can then be sure that the importance that they placed on the barrier being tested is correct.|
|The hypothesis is part proven (e.g. the hypothesis is proven, but the "null hypothesis" is not)||- It may be clear to the survey team why one hypothesis was not validated. Therefore they can continue to Stage 3 but incorporate the findings into the formulation of the prior.
- Or, if time allows, the survey team could select more villages / communities in which to test the hypothesis and carry out the survey again.
|The hypothesis is NOT proven||This will be because: a) the hypothesis was wrong; b) there was a sampling error.
The survey team should therefore formulate a new hypothesis and / or select new villages / communities where they can test the hypothesis and repeat the sampling and analysis.
Stage 2 results analysis template :
Quality assurance guidelines for Stage 2