AAM 2012 – Minneapolis, Day 3, Tuesday (Measuring Complex Behavioral Change)
Day three session highlight was on summative evaluation methodologies undertaken by the International Coalition of Sites of Conscience. The ICSC is a US-based group that is a worldwide network of “Sites of Conscience” – historic sites specifically dedicated to remembering past struggles for justice and addressing their contemporary legacies. A full description of their work can be found through the Resource Links, below.
It recognized that traditional quantitative metrics to measure social change is at best a blunt tool to express organizational impact efficacy. At worst, it’s misdirected or misleading – both to the creators of the evaluation and those in its review. The tool being implemented at a handful of member institutions is called Outcome Harvesting. It uses statement-based criteria to specifically indicate a realistic expression of impact in a dynamic and multivariable world. For example, how can the Liberation War Museum in Dhaka, Bangladesh measure its role in educating children in tolerance when the immeasurable outside influence exists in the broader local or world community? There is a tremendous need to measure outcomes, both for internal insight as well as gathering and maintaining organizational support.
I can say that many of my clients that strive to effect positive change within the public they serve are in the need of three things:
- A broadly applicable tool to develop mission-compatible programs and exhibits that have predictable and measurable outcomes prior to broad implementation
- A method by which to design and activate these programs and exhibits within their institutions that excel in all expected arenas of public engagement
- A means to effectively evaluate the programs and exhibits in a fashion that directly provides feedback to reinforce, modify or abandon the efforts of 1 and 2.
At its best this is a open-ended process that has no specific endpoint, but rather a chain of punctuated “check ins” that inform the next steps. Rather than finding oneself at the end of a road wondering how an organization wound up at a particular place, this approach commits fewer resources and less institutional effort in the development of change initiatives. And then supports the efforts through reinforcing and course correcting feedback.
In many ways, the evaluation tool being tested by the ISCS could be well served by the Community Based Social Marketing strategy crafted by Doug McKenzie-Mohr, Ph.D. at the University of Toronto. CBSM is a specific design process to developing approaches aimed at achieving specific mission-based outcomes. Its foundation is born out of the public healthcare world and is now coming into its own in the realm of environmental and resource conservation advocacy.
All that said, I find myself compelled to look for opportunities to enable these very important sites of conscience to develop site-based exhibits that fit within an effective, measurable and powerful visitor experience. There is a lack of design vocabulary, dialogue and toolsets that effectively reach known outcomes in change-based exhibitions, public places and experiences.