Well-designed and well-delivered evaluation will be a core part of the evidence base for widespread adoption of measures proven to reduce gambling harms.
Good evaluation looks not only at the process, but also the impact on people and behaviour. In order to achieve the overall aim of the strategy, a greater understanding is needed of the impact that interventions and activities have on how people gamble, how they experience harm, and how they respond to prevention and support activities and interventions. This means that evaluation needs to be built in from the beginning of an intervention or project.
The key principles of good evaluation, as set out in the existing protocol are:
- Robustness and credibility: Appropriate evaluation generates robust evidence. This includes drawing on quantitative and qualitative data and incorporating the consumer and/or user voice.
- Proportionality: Evaluation should be proportionate to the risk and scale of the intervention, so scale should be considered and documented at the outset.
- Independence: Independent evaluation is perceived as more objective, robust and credible, but may not be proportionate for all interventions.
- Transparency: Evaluations should be as open as possible about the rationale and details of the intervention, the evaluation process, results generated, and conclusions. Transparency increases confidence and credibility and allows stakeholders to think about how lessons learned can be transferred.
Evaluation has started to take place under the Commission’s independent research programme, to help understand the effectiveness of activities designed to reduce the risk of gambling harms, and working with partners to do so.
Examples include evaluating the impact of reducing friction for customers to encourage setting limits in online gambling, trialling safer gambling messaging on gaming machines in bingo premises, and the work to evaluate the effectiveness of multi-operator self-exclusion schemes.
To support evaluation across all partners
Evaluation is not confined to what the gambling industry does. It is important to find out what works in wider prevention programmes, and in treatment and support options. The success of the strategy will be reliant on how all parties involved approach and use evaluation to help determine the right mix of interventions and options to prioritise activity and reduce gambling harms. Good evaluation will enable and empower commissioning bodies to base funding and prioritisation decisions on evidence of what does and doesn’t work.
Longer term evaluation needs
Over the longer term, those involved in delivering the strategy may benefit from a more coordinated approach to evaluation. Options to prioritise and coordinate evaluation, including ownership and implementation, will be explored during the life of the strategy.
We will work with evaluation experts, and a range of stakeholders delivering interventions to reduce harm, including gambling businesses, to review and revise the 2016 RGSB evaluation protocol. It may be appropriate to further supplement the protocol and existing guidance with additional or updated practical tools and advice, in order to raise standards in evaluation at the individual operator level and to begin to embed proportionate evaluation into both current practices, and at the beginning of new practices.
Crucially, we want to understand what does not work as much as what does, so that we can match our expectations to the most effective methods of reducing harms.
It is important that new measures or programmes are properly evaluated. For the Gambling Commission, this includes monitoring progress of significant new policy initiatives or regulatory changes and measuring progress against the strategy from the start. We will support and encourage other bodies to do the same.