March , 2020 — by Elizabeth McGee
In my role as a Consultant specializing in Evaluating Complex Systems Change Initiatives, I work closely with clients aiming to transform large-scale systems. This demands the skill to simplify complex initiatives for effective evaluations.
A Bit About Systems Thinking
In recent decades, both the public and private sectors have increasingly embraced ‘systems thinking’. Experts have proven to be a more effective approach for solving complex societal problems. Systems change work is inherently complex and can feel abstract. Thus evaluations for these initiatives need to be set up in a way that responds to this complexity by clearly identifying the changes we hope to see at different levels and how those changes build on one another to create the desired population-level impact.
At LEAP Consulting, we use framing adapted from ORS Impact & Spark Policy Institutes’ evaluation of evaluation of Collective Impact Initiatives. This framing helps our clients articulate the changes they want to see as a result of their initiatives’ work. Specifically, it helps to examine the contribution to large-scale impact, as well as progress towards short-, mid- and long-term goals on route to large-scale impact.
This framing classifies Evaluating Complex Systems Change Initiatives at four discrete levels:
1. systems change strategies
2. early changes in the landscape
3.longer-term systems changes
4. population change
It has helped our clients think more concretely about how systems change cascades over time. It has also helped us as evaluators to determine the most suitable evaluation methods for each level of change.
*Figure Adapted from Study Report “When Collective Impact Has an Impact” by ORS Impact & Spark Policy Institute
Hot Tip
When evaluating complex systems initiatives, work closely with the client at the beginning to develop a Theory of Change (ToC) using frameworks such as systems change strategies, early landscape changes, long-term systems change, and population change. See the ToC as a “living document” that can be revisited and revised on a regular basis to align with the strategic direction of the initiative if it evolves over time.
Lessons Learned
Maintaining a clear perspective of how all the initiative’s components at different levels work together to contribute to changes within the target population benefiting from your systems change becomes crucial. A Theory of Change (ToC) is vital in this process and serves as a helpful tool for framing and contextualizing evaluation findings and data points later in the initiative.
Originally posted as an American Evaluation Association 365 Blog Post.