Did you know that the Welfare Food Scheme, which operated in the UK between 1940 and 2006 was never formally evaluated? It is hard to imagine that such a long running, nationally funded programme was never investigated to find out if was effective, but in some ways this may reflect the ‘duty of care’ aspect of the scheme. The strong belief that vulnerable women and children should be supported nutritionally, regardless of whether an outcome could be measured, may well have carried the scheme through many years of its life. One of the criticisms of the COMA review of the Welfare Food Scheme that was undertaken in the early 2000s was that there was very little data on the programme for them to evaluate, and without data how do we know a policy is actually doing what it set-out to do? My research also found that any evaluation of the new Healthy Start was also considered a little late in the day. Most evaluation experts will tell you that the time to consider evaluation and monitoring of any programme is before the programme rolls out, that way evaluators can collect any baseline data and be clear on what information is needed and how it can be collected. A report from 2007, outlines options for evaluating Healthy Start (note: Healthy Start rolled out in 2006). Never the less, the Department of Health commissioned two evaluations of Healthy Start that reported in 2013. This post gives an overview (but for more information I strongly recommended reading both reports which are available here). This is a longer post than usual, but I can’t emphasise enough how important these evaluations are in terms of adding to the understanding of Healthy Start, so here goes...
Both evaluations published in 2013 broadly address how Healthy Start operates on the ground, however they have slightly different, yet corresponding remits. Specifically, Lucas et al. (2013) undertook a process evaluation of Healthy Start across 13 Primary Care Trusts (PCTs). The objective was to gather experiences of beneficiaries, frontline staff and staff employed at small independent retailers. Methods comprised of qualitative research to glean the views of beneficiaries (n=107), frontline staff (n=65) and staff employed in small independent retailers (n=20). Like Hills (2006), Lucas et al.’s (2013) evaluation does not engage with large numbers of participants, thus findings are not generalizable. Rather findings are considered as indicators of issues emerging in practice.
McFadden et al. (2013) undertook a larger multi-method evaluation to look at voucher and vitamin use in Healthy Start and consider the feasibility of economic impact evaluation. This evaluation includes:
· systematic review of qualitative and economic literature
· qualitative research with Healthy Start participants in London and Yorkshire and the Humber (n=113) and practitioners (n=49)
· Participant demographic questionnaire (n=109)
· national electronic consultation with health professionals (n=620)
· cross-sectoral workshops (n=56)
· consideration of both commercial and public data sets.
· consideration of the first purposeful sample in 2010 Infant Feeding Survey
The methodology draws on range of methods, however the number of beneficiaries interviewed remains relatively low. This reflects a similarity between this evaluation and the former evaluations and studies on Healthy Start.
It is unclear why the Department of Health commissioned two evaluations simultaneously. Neither evaluation makes reference to the other. Despite this, many of the recommendations and implications were similar or the same, thus the benefit of having two evaluations is that they reinforce some of each other’s findings. McFadden et al. provide almost 55 recommendations based on their research, whereas Lucas et al. (2013) provide 26 implications of research. Despite the issues that emerged across the evaluations, in both reports there is a general belief that Healthy Start is an important scheme that should remain in place. Key general recommendations that both evaluations make are:
· Universalising vitamins
· Support local areas to make better links with local services such as children’s centres
· Raise awareness of Healthy Start in general
· Train any professional who has contact with pregnant women about their role in Healthy Start.
Both evaluation teams experienced issues accessing certain datasets (Lucas et al. 2013; McFadden et al.). McFadden et al. (2013) considered the utility of the data collected by the companies commissioned by Department of Health to manage the Healthy Start phone line and the reimbursement unit – Homescan and Kantor. Neither company could provide data in a workable format that could support the evaluation. In addition, the evaluators identify data that exists, but was not accessed. For example, McFadden et al. (2013) tried to access commercially accessible data and found that although commercial information on how Healthy Start is used in Tesco (one of the most used supermarkets by Healthy Start beneficiaries) exists and is held by Dunhumby – a large market research company, it could not be accessed, presumably due to the financial cost.
It was valuable for McFadden et al. (2013) to include data from the first purposeful sample included in the 2010 Infant Feeding Survey. McFadden et al. (2013) state that the 2010 Infant Feeding Survey (McAndrew 2012) is ‘the single most promising dataset for analysing the demand of Healthy Start –supported products and for judging the impact of vouchers on this demand’ (p.126). For the first time, in 2010 the IFS included a purposive sample of women who would be eligible for Healthy Start and included questions regarding their infant feeding practices. The value of the inclusion is that prior, there had been no data on the infant feeding practices of families in receipt of Healthy Start. Key findings were that the sample population had considerably lower rates of breastfeeding then the general population and Healthy Start vouchers were primarily spent on infant formula.
The evaluation reports provide really valuable insights into the how Healthy Start operates on the ground and provides scope for lots of further research on specific aspects of Healthy Start in practice. The reports can be accessed on the resources page of this website.