Moving from the micro to macro level learning by Dr Susanne Burns
It is now almost two years since the Pandemic hit in March 2020. I was due to lead a one day workshop on evaluation for the Wales Wide Training Programme (WWTP)[1] in Abergavenny in mid-March so of course, the training was cancelled and our initial intention was to reschedule it for September 2020. How optimistic that now seems! I don’t think any of us had any idea what was ahead of us in March 2020. It was scary and totally unknown and as our buildings closed and many of us had to self-isolate at home, our lives changed irrevocably. As an independent consultant working across the UK for a number of different clients, it was a time of huge uncertainty. Gradually we realised that our delivery models had to change and new ways of working had to be found.
So as the Covid-19 pandemic took hold we decided that we should take the training online. My nervousness at doing so was probably shared by most of the arts sector as we sought to find ways to maintain our work and connect with people. Tracey Brown, who coordinates the WWTP from Rubicon, and I decided that we should pilot the online evaluation workshop with the staff team at Rubicon as part of their ongoing online CPD work. The workshop went really well, feedback was overwhelmingly positive and I felt confident that we could roll it out more widely. So, what had been a one day workshop for 12 people had suddenly become three online workshops reaching more than 24 people across Wales.
Participants from Susanne Burns sessions said:
"The evaluation forum with Sue was easily accessible and genuinely got us all excited about the greater impact and ideas we could unlock through evaluation. Sue took the time to speak to everyone in the group and showed us how we could adapt resources to our variety of needs and the different strands of work at Rubicon. Just a few weeks after taking part in the training, I have already been able to implement the training in real-life situations to help with funding applications. It has given the whole team a better understanding of how we can help each other in our variety of roles and work more collaboratively to achieve the best possible results."
"Having sat with this for a couple of days, I realise how immensely useful and focusing this session was. I immediately went on to create a Planning & Evaluation framework for three co-designed projects that I'm about to facilitate the planning process for. This session also reminded me of the planning/assessment cycle in teaching and how one always informs the other, and of my commitment to bring together contributing voices into the planning & evaluation process."
The success of this approach led to two important developments that have furthered my practice and approach significantly over the last two years.
Rubicon Director, Kathryn Williams, had been working on a new business plan for the organisation and having attended the pilot workshop felt that there was an opportunity to embed a deeper approach to evaluation and learning within and across the organisation. I started working with the staff team to generate an overarching organisation-wide framework that aligned with the business plan and we instigated what we called first-generation projects. A series of retrospective case studies were produced by the team that told stories and highlighted learning from previous projects; two reviews of previous work were to be edited by myself – WWTP and Rubicon’s work in the health sector; and each member of the team adopted the overarching framework and translated it into a mini project or service-based framework relevant to their work. This macro/ micro approach was designed to generate both detail in the evidence collected at the project or service delivery level as well as higher-level analysis and narratives. This was the first time I had worked with an organisation to develop an organisation-wide approach and I am now taking this approach with several other organisations. It was cost-effective because we delivered everything digitally. Drop-in sessions and open email communication provided support for individuals between whole team sessions and the whole staff team had training and ongoing support in embedding and internalising an approach to learning, reflection and evaluation that will hopefully impact the culture of the organisation as it moves forward.
Secondly, on the back of feedback received, WWTP asked for a follow-up programme of work that would be delivered digitally and which would adopt a more scaffolded approach to training in evaluation and reflection.
"I would love the opportunity to reconvene with the same group a few months down the line to share an evaluation process we may be engaged in, to open it up to discussion and to receive feedback (to feed-forward) - supported by Susanne. Reflective practice in action with peer support whilst simultaneously 'sharing the learning' across the sector in a small and safe way."
We designed a three-stage approach that included the introductory evaluation forum for new participants, a planning workshop with previous forum participants to work on developing frameworks and plans for project evaluation; a drop-in session for new participants for any troubleshooting; progress check-in for ongoing projects being implemented by previous participants and a Best Practice Forum to share learning outcomes of the projects through presentations from previous participants.
What had begun as a self-contained training day had suddenly expanded into a programme of work that was reaching a range of different individuals and organisations across Wales. Rubicon is an extraordinary organisation unafraid to take risks and willing to be challenged. Their reach is immense and their support for the dance ecology in Wales is evident in their leadership of the WWTP. What became evident to me in carrying out this work was that the benefits of working at a macro level – whether that was organisation-wide or sector-wide – were incredibly powerful in terms of shifting perception and culture around evaluation. I suspect the approach is also potentially powerful in generating systemic change but that will take more time to evidence.
There was a lot of learning. Worthy of note was when we paired commissioners with the dance artists they were commissioned in one of the sessions. This led to much deeper meaningful approaches to evaluation that went beyond quantitative outputs and impacted beyond the specific project to the wider commissioning context leading to a request for training for the commissioning organisation.
"It's been great and thanks so much for welcoming us into it. We were communicating on the chat throughout about ways we can evaluate the project I am doing with them, so already ideas are in the mix that has been sparked by what Susanne said. It was so beneficial to have them there as we've gone on the journey of re-framing what evaluation is for us together."
As a direct result of this work, I have subsequently provided independent support for four cultural organisations in Wales, mentored an individual artist and supported a commissioner of art in health project in developing their approaches to evaluation. I have also led training workshops as part of the Co-Creating Change Network on adopting this micro to macro approach to evidence and learning. What would have initially been one training session for 12 participants has extended exponentially in reach and potentially affected attitudinal changes towards evaluation as reflection, learning and investigation as well as evidence gathering. It has confirmed for me a belief that if we can find ways to develop macro-level evaluation support both at an organisation-wide level as well as a sectoral level, there is potential for systemic change. Rubicon took a brave step in seeking to develop their organisation-wide approach and we are continuing to learn a lot about what works and what works less well when trying to achieve this. The work that we continue to do together has an internal value but also provides some important learning for the wider sector.
Dr Susanne Burns
Independent Consultant
[1] WWTP aims to inspire and sustain community dance in Wales by providing training that responds to sector needs and is primarily a training programme for those working in dance across Wales. It is managed by Rubicon Dance and creates and connects a network of eighteen organisations and two hundred and eighty individuals working together to support training and professional development for the community dance sector across Wales. The programme is supported by the Arts Council of Wales. The WWTP was established in 2013 with seven community dance organisations as partners. At the beginning of 2021 there were eighteen partners that include not only community dance organisations, but venues, universities and national companies from across Wales including Powys, Carmarthen, Rhondda Cynon Taf, Caernarfon, Monmouthshire, Ceredigion, Cardiff, Llanelli and Flintshire. The WWTP began with four strands in 2013 and now has nineteen strands some of which have been established in response to need and developed throughout the pandemic.