New ways of doing and thinking – Lessons from the HSHAZ Evaluation
Historic England's Senior Evaluation Officer, Alex, reflects upon the methodologies used within the evaluation of the HSHAZ programme and why bringing in approaches from other industries has been so important.
By Alex Wardrop, Senior Evaluation Officer, Historic England.
Part of the Heritage Counts series. 3 minute read.
More than mere shop fronts.
More than mere dots on maps.
More than the meagrely forgotten.I've never been there
but I know it with sound and rhythm.
I know 67 sites
led there by resilient fox on a scooter
who opens currents of exchange, knowledge, pride.
This red haired crepuscular canine shows
new ways of doing,
new ways of being - being together -
coming into being.
This poem was created as part of the High Streets Heritage Action Zones cultural programming and in just a few poetic lines, it captures what the programme was and could achieve. This message echoes across the evaluation of this programme; showing that bringing different ways of doing and thinking together can create magic. For example, there can’t be many programmes and evaluations that include giant puppets, dancing disco balls, record-breaking tea parties, poetry, quasi-experimental methods and cutting-edge economic analyses.
At its heart, the High Streets Heritage Action Zones (HSHAZ) programme was testing what was possible when it came to place-based, heritage-led regeneration. Therefore, its evaluation needed to reflect this ambition, drawing insights from public health, transport, and the creative arts, among others. The evaluation is a model for the magic of being open to the guidance, standards and methods beyond your own sector or specialism. By approaching evaluation in this way, we have been better able to understand the programme’s impact.
First up – the Transport sector
Our evaluation consultants, AMION, recognised that any analysis of high streets must include an understanding of how people move through places. They used guidance from the Department for Transport as part of a Social Cost Benefit Analysis. By mobilising this guidance, the art works created in partnership with communities could be included within the economic analysis taking place.
AMION assessed 155 temporary art installations developed through the cultural programme as wayfinding aids for residents and visitors. By thinking about public art in this way it enabled AMION to broaden the scope of the economic assessment. This approach helped tackle the challenging task of how to measure artistic outputs.
In using guidance and evidence from the transport sector, this programme evaluation can contribute to the conversation about economic impact. Similar to the discussions surrounding Culture and Heritage Capital (CHC) programme.
Through the CHC Programme, we are testing methods to better understand the public value that culture and heritage can generate such as research on the public and church goers’ willingness to pay towards maintenance and restoration of these historic places of worship.
And importantly – Public health
When considering evaluating an intervention tackling the challenges facing high streets, we also approached them as a public health issue, as much as an economic one. There is substantial evidence and growing conversations about the role that buildings and heritage more generally as well as the urban realm has for our health and our well-being.
The evaluation methods reflect this close relationship between the aims of the programme and that of the public health sector. For example, we understand the impact of over 119,000 sqm of public realm improvements as the creation of spaces for people to people to gather and rest in (spaces to improve healthy living). AMION did more than this and they employed the RE-AIM framework as one way to interrogate the impact of the programme. In doing this, AMION show how mobilising evaluation techniques from other sectors buildings our understanding of what heritage means for people and places.
The RE-AIM framework systematically enables people to explore the Reach, Effectiveness, Adoption, Implementation and Maintenance of an intervention.
This framework is a standard and trusted model that public health specialists use to design and evaluate their programmes and impact.
Not forgetting - Creativity and Collaboration
When evaluating complex programmes, like HSHAZ, you need to bring different methods and data together and let several stories sparkle at once, like an evaluation disco ball.
In addition to the approaches already mentioned, AMOIN used a method called Process Tracing to help understand how the HSHAZ programme has contributed to observable changes. One of the strong conclusions identified through this method was how the programme increased collaboration and partnership.
Across the country, communities contributed creatively to the physical changes on their high streets - making murals, mosaics or public art and sculpture. On top of this, renovated buildings have become revitalised and revitalising spaces for creativity and collaboration.
The evaluation tells us a story about the power of bringing creative arts, community participation and heritage conservation together.
To bring the programme’s impact to life, the evaluation curates a constellation of ideas, methods, data, images, and stories. We tested data and approaches in novel ways, as my colleague Tom Kanchanatheera shows us as he takes us on an adventure in data science.
While not every evaluation can be this expansive, the spirit of this work - learning from others, embracing creativity, finding new ways of thinking about and doing evaluation - is something we all need a bit more of.
Also of interest
