Online Evaluation of Development Education: Not a new Universe
Compared with face-to-face work, the organisation and delivery of Development Education online is challenging, time consuming and requires new ways of supporting participants’ learning. Does that mean that online evaluation also requires a new way of thinking and new approaches?
As discussed at two IDEA training-workshops on online evaluation (14th and 20th April 2021), the short answer is ‘no’ and ‘yes’. ‘No’ because various online approaches give opportunities to use the same or similar approaches as offline. Also, ‘yes’, as with almost all online work, online evaluation requires a significant investment in planning. Knowing what you want to find out about and why, and then finding the appropriate online tools takes, certainly initially, more time than it would for offline work.
The first of the two training-workshops, with 25 participants, started from the point that participatory learning forms a key component of Development Education. For workshop participants this revolved around collective and individual learning that stimulates learners’ interaction and collaboration, their engagement and motivation, learning by doing, relevance of the learning to the learner, and shared reflective action. Evaluation of those aspects of a Development Education programme or project requires an insight into, for example:
- The level of enjoyment of learners in taking part and their engagement with the learning
- The extent to which learners have gained new knowledge, understanding and skills
- The effect of the process and the content on learners’ performance
- The impact on learners’ behaviour and actions
- The consequences of it all on moving the organisation’s aims forward
- The process of the project or programme.
Each project or programme will of course have its own specific evaluation questions to be addressed. In finding answers to those questions using a range of online tools can be helpful. Tools such as:
- Online quizzes and other game-type activities – to assess skills and knowledge
- Drop-and-drag activities – to make connections between two or more different phenomena, such as different development issues, causes, or effects
- Dialogue simulations – to present different perspectives and arguments
- Forum posts and blogs – giving learners an opportunity to interact, whilst giving the evaluator an opportunity to check levels of comprehension of an issue
- Open-ended essay style questions – to evaluate higher level learning and its application
- Online peer reviews – to give each participant an opportunity to reflect on their own learning and then communicate that to other participants for feedback and further discussion
- Observations – either live or from a recording of online events, for example to gain an insight into the level of interaction/participation of participants, of their understanding of an issue, or of their creativity in addressing an issue
- Online polls or surveys – to obtain feedback on the process, the level of interest in the learning or the relevance to learners of what has been learned.
The second of the two training-workshops added to these techniques by discussing eleven different online evaluation scenarios suggested by participants for further exploration. The discussions gave practical examples of how techniques such as those above can be integrated into real-life DE projects and gave additional suggestions for online evaluation, including:
- Evaluating the effectiveness of webinars – by referring to registrations, participants and retentions, the interest shown in relevant resources after the events (downloads), the use (and quality) of the chat box and the use of opportunities to ask questions during the webinar,
- Assessing creativity – in work groups/breakout rooms use Padlet, Googledocs or Jamboard to capture ideas that have been developed,
- Assessing learner enjoyment and engagement – during a multi-event online project using online journals (completed by learners, if need be anonymised) to provide summary recordings of their feelings and learning highlights and lowlights,
- Evaluating the development of self-confidence/self-esteem of individual learners – doing this online (through one-to-one interviews) will require a level of familiarity of the learners with the evaluator, i.e. s/he needs to be known by the learners as part of online events. Obtaining one-to-one engagement after the online event is problematic but may be possible, for example, in a break-out room during an ongoing project event/lesson
- Assessing the quality of the learning process – using a ‘360 degrees peer review (online)’.
To keep track of all online techniques and their results, the use of an online whiteboard (such as Miro or Jamboard) can be very helpful: showing the evaluation questions asked with links to the different online tools used and the information that using these tools has given you.
All this does not suggest, however, that there are no problems with online evaluation. In common with any form of online communication and online teaching and learning, issues of ‘Zoom fatigue’ or of a lack of participation and engagement with the screen are not necessarily resolved. Notwithstanding this, tools such as those highlighted do encourage and enable active engagement – particularly if they form part of a process of participatory learning.
Online evaluation does not require evaluators to step into a different universe: many of the existing offline evaluation approaches can be and have been adapted for online use. What is different though is that making full use of online evaluation, requires integrating evaluation techniques into the implementation of the project or programme – not something to be done ‘after the event’. More than with offline evaluations, obtaining online inputs from participants towards or after the end of their involvement in a project or programme is highly problematic and may not give the diversity of information required for a quality evaluation. For organisations that have tended to do their evaluative exercises towards the end this will require a new way of planning: clarifying from the start what needs to be evaluated and integrating evaluative activities as part of the online delivery process.
This blog is written by Harm-Jan Fricke , a Development Education/Global Citizenship Education project manager, evaluator and workshop facilitator
