Joanna Schwarzman, Adrian Bauman, Belinda J. Gabbe, Chris Rissel, Trevor Shilton, Ben J. Smith

Evaluation and Program Planning (2019) 74: 76-83

https://doi.org/10.1016/j.evalprogplan.2019.03.002

Highlights
• Measurement tools to assess the evaluation capacity or practice in health promotion organisations are lacking.
• The newly developed Evaluation Practice Analysis Survey (EPAS) is a reliable and valid instrument to assess elements of evaluation practice in health promotion organisations.
• The EPAS comprises 25 scales addressing individual, organisational and system level factors that influence health promotion evaluation.
• The instrument demonstrated good to excellent internal reliability of 23 scales, and predictive validity for a number or organisational and resource variables.
• The EPAS has potential application in the planning and evaluation of capacity building initiatives in health promotion organisations.

Abstract
The demand for improved quality of health promotion evaluation and greater capacity to undertake evaluation is growing, yet evidence of the challenges and facilitators to evaluation practice within the health promotion field is lacking. A limited number of evaluation capacity measurement instruments have been validated in government or non-government organisations (NGO), however there is no instrument designed for health promotion organisations. This study aimed to develop and validate an Evaluation Practice Analysis Survey (EPAS) to examine evaluation practices in health promotion organisations. Qualitative interviews, existing frameworks and instruments informed the survey development. Health promotion practitioners from government agencies and NGOs completed the survey (n = 169). Principal components analysis was used to determine scale structure and Cronbach’s α used to estimate internal reliability. Logistic regression was conducted to assess predictive validity of selected EPAS scale. The final survey instrument included 25 scales (125 items). The EPAS demonstrated good internal reliability (α > 0.7) for 23 scales. Dedicated resources and time for evaluation, leadership, organisational culture and internal support for evaluation showed promising predictive validity. The EPAS can be used to describe elements of evaluation capacity at the individual, organisational and system levels and to guide initiatives to improve evaluation practice in health promotion organisations.