Working on outcomes: Themes and learning from the Impact Measurement Masterclass

Between April and June 2020, Quality Matters was invited by IDEA to deliver an Impact Measurement Masterclass, for formal and non-formal education practitioners to explore outcome measurement with Development Education (DE) organisations. The masterclass was delivered as a series of virtual workshops and conversations attended by 19 individuals representing 15 IDEA member organisations and individuals from across the sector.  From the outset, participants all agreed that demonstrating the outcomes of their work was important, but that organisations need to take several factors into account for an outcome measurement process to be successful and meaningful for both participants and staff alike.

Outcome measurement can be defined as measuring the change that has occurred, because of a specific activity or project, for a range of stakeholders. It involves understanding both the positive and negative, and intended or unintended outcomes, created by this work. Analysing outcome data can support organisations with their future strategic planning, ongoing quality improvement and decision-making about a project or activity.

To look at how an outcome-focused approach might be used by DE organisations, we will explore a few issues addressed in the training.

Identifying the changes that matter most. For DE organisations, a strength is that Result-based Frameworks (RFs) and Result Chains are widely used to define the outcomes and outputs of projects and activities. For charities and non-profit organisations, a theory of change (or other Project Management tools such as RFs) is an integral step in identifying and defining outcomes to be generated by their activities or programmes. At the start of an outcome measurement process, such a framework can help an organisation to answer key research questions, such as:

  • What are the problems this activity is trying to address?
  • What are the outcomes that matter most to stakeholders?
  • How does your organisation's activity create these outcomes?
  • How do you know if people have achieved these outcomes?

By answering these questions, an organisation can establish the scope of its outcome measurement process. It's certainly a good starting place for engaging in a broader discussion with a staff team about what an organisation is trying to accomplish and setting a proportionate scope, including the number of stakeholders who will be involved. Establishing the scope also came up in terms of trying to decide on measuring outcomes for individual learners,or taking a broader approach to measuring changes for other stakeholder groups. It was recognised that outcomes related to policy and advocacy work often require different indicators to methods for measuring behavioural or attitudinal outcomes for participants.

Finding the right tools for the job. Training participants were interested in more examples of tools used for measuring the extent of change, or when an outcome has been achieved (or not). There are outcome measurement tools used within the Irish Development Education sector that participants could point to as examples. For instance, World Wise Global Schools developed a self-assessment tool to measure changes in both students’ and teachers’ skills, knowledge, attitudes and actions.

We also looked at examples of good practice, from other sectors in or outside of Ireland, to better understand how organisations can measure social and behavioural outcomes. For the Impact Measurement Masterclass, Quality Matters prepared a report with examples of various tools used in Development Education work. While DE organisations have the necessary skills and practical experience with undertaking research and evaluation, some may not be familiar with validated tools which are available and may be used for measuring outcomes.

Measuring intangibles can be difficult. Using outcome measurement as a way of measuring change relating to 'intangible' or 'soft' outcomes (such as changes in attitudes or beliefs) resonated with many participants. Most organisations use a combination of outputs, quantitative data and feedback from their beneficiaries to show the impact of their work, but some feel a more focused approach to demonstrating educational, social, attitudinal and behavioural outcomes would be useful. The first step is answering simple questions about which specific activity you are measuring the impact of, and which groups of individuals (or stakeholders) experience this change.

Some things are easier to measure than others. The truth is that, for many charities and non-profit organisations, there is no single method, survey or tool to measure outcomes for an intervention. As we know, different organisations require different approaches to effectively measure their work. But, if organisations don't have the time or resources to invest in developing appropriate surveys, questionnaires or measurement tools, they won't be able to work out how to measure their 'intangible' or 'soft' outcomes. Many participants recognised there are benefits to outcome measurement and the growing role for IDEA may be to help organisations with building this capacity and solving these practical challenges in years to come.

Opportunities to learn and improve. It may seem obvious, but one of the interesting discussion points from the Masterclass workshops was how organisations would use their outcome data as an opportunity to learn and improve, and to demonstrate to funders how programmes are effective. Many participants discussed how their organisations were already collecting data used for a variety of purposes, and how this data may already demonstrate both the effectiveness and outcomes generated by their work. In this context, the real challenge of starting an outcome-focussed approach is determining what resources are available, what data is already being collected and what will be measured by an organisation.

As for what might encourage more DE organisations to measure outcomes, or help to decide if outcome measurement is worthwhile, an outcome-focussed approach will support any organisations to make evidence-informed decisions about their interventions and to test if this work is making a difference for their stakeholder groups. Don't try shooting for the stars; think about outcome measurement being proportionate to your work (and staff resources) and finding simple means of undertaking outcome measurement with your stakeholders, so they find this experience rewarding, meaningful and positive.

Philip Isard is a Senior Project Specialist with Quality Matters

September 17, 2025
Date: Wednesday 26 November, 10.00am – 3.30pm Location: Richmond Barracks, Inchicore, Dublin 8, D08 YY05 IDEA is looking forward to welcoming all members of the Code of Good Practice for Development Education to our next Code network meeting on Wednesday, 26 November, in Richmond Barracks in Inchicore, Dublin. There are places for two representatives (staff, volunteers, etc.) from each Code member. One of the commitments in joining the Code is to contribute to the Community of Practice for this Code, including sharing successes and learning with other Code members and attending at least one of two Code network meetings annually. Register below!
September 12, 2025
Date: Tuesday 21 October, 12 noon – 12.45pm Location: Georgian Suite of Buswells Hotel, 23-27 Molesworth Street, Dublin 2 Over the past few months, IDEA and its members co-created a new strategic vision and roadmap for GCE in Ireland for the next five years. This is the successor to Vision 2025 and is called Vision 2030. We are delighted to announce that we will launch the publication of Vision 2030 on Tuesday 21 October 2025 in the Georgian Suite of Buswells Hotel, 23-27 Molesworth Street, Dublin 2, 12 noon – 12.45pm. Refreshments will be provided. Please save the date and register below for the event. Please note that there is limited space at this event and places will be allocated on a first come, first served basis. Once capacity has been reached, a waiting list will be opened.
September 12, 2025
We’re Hiring - Capacity Development Programme Manager The Irish Development Education Association (IDEA) is looking for a Capacity Development Programme Manager to join our team. This is a unique leadership opportunity to help shape the future of Global Citizenship Education (GCE) in Ireland and beyond. At IDEA, we work with over 100 members, from large NGOs like Trócaire and Concern to smaller organisations and individual educators, all committed to building a more just, equal, and sustainable world. The Capacity Development Programme Manager role is central to this mission. Key role responsibilities: Lead IDEA’s capacity development programme , ensuring our members and the wider sector have access to high-quality training, workshops, and networking opportunities. Design and deliver flagship events , including our annual conference, webinars, and seminars that strengthen skills, knowledge, and collaboration. Drive innovation and impact , ensuring our programmes respond to members’ needs, emerging challenges, and international best practice. Champion impact measurement and research , supporting our members to demonstrate the transformative power of Development and Global Citizenship Education. Collaborate across IDEA teams on policy, advocacy, and communications, ensuring capacity-building directly contributes to systemic change. Represent IDEA nationally and internationally , ensuring Ireland continues to be seen as a leader in the global conversation on education, equality, and sustainability. The role offers: A permanent contract (subject to funding) Hybrid working based in Dublin city centre Competitive salary, employer pension contribution and 25 days annual leave The opportunity to work at the heart of a dynamic national network with strong international links If you are an experienced leader with a passion for education, social justice, and building the capacity of civil society, this could be the role for you. Deadline for applications is Monday, 13 October, 5.00pm. Interviewees will be invited to interview on October 15th . Interviews will be held in the IDEA office on Monday October 20th. For the full role description and application process, please download the Job Specification here