Evaluation has a bad reputation for a reason. Sometimes assessment tools are used towards unjust ends—hello educational tests and tracking! Sometimes, folks solicit feedback and use that feedback in wonky ways. (Have you ever seen constructive criticism result in the eradication of a program? I have.)
It’s no wonder that so many of us are burned out on surveys and focus groups, specifically, and skeptical of data, generally.
Evaluation has been complicit in deepening social inequalities, dehumanizing learners, and fortifying dominant power structures. Sometimes, evaluation is a process in which implicit bias manifests in structural outcomes. Organizations lose funding. Programmatic changes leave particular populations without services. And the very process of providing feedback can be oppressive. Questions utilize inaccessible language and often ignite a sense that our work or learning or growth doesn’t measure up.
As someone squarely situated in the world of social justice nonprofits, I’m critical of evaluation—how it works and how it’s used. I also rely on feedback and information. I use data.
Currently, I serve as the Director of Social Justice Education and Community Engagement at The Tzedek Social Justice Fellowship. We offer full-time paid professional and personal development opportunities for emerging social justice leaders. Over the course of 11 months, Fellows work 30 hours each week at a local nonprofit, social justice organization in Asheville, NC and spend 10 hours each week developing life and leadership skills with Tzedek staff and other local and national thought leaders and activists.
Through a unique combination of work, training, and community building, Fellows develop the skills and resiliency needed to become strong and effective activists, allies, and social justice leaders. It’s an experience designed to amplify Fellows’ impact in the world.
To create impact, we need to understand the experience of our Fellows and our partner organizations. And so we find ourselves building out pre- and post- measures, soliciting Fellows’ participation in focus groups, and surveying host organizations.
But as we build out assessments, we’re aware that data isn’t neutral. And everyone doesn’t experience evaluation in the same ways.
At Tzedek, we’ve been asking ourselves what a transformative approach to evaluation looks like. Transformative approaches emerge when we, first, ask how inequalities and violence are enacted through seemingly benign social structures and, then, develop alternative strategies that center human flourishing.
Currently, we’re working out what a transformative approach to evaluating the Fellowship program could look like. So far, we’re trying to carve out tools using the following questions:
*Does this evaluation tool empower learners to celebrate emerging skills?
*How does our process demonstrate value for multiple forms of knowledge and growth, understanding that success and impact look different for each of us?
*Does this tool measure strengths and resist punitive or stigmatizing language?
*Is this process aligned with our values?
*How will we use this data to create an evolving and responsive program that shifts in response to our Fellows and the times?
We don’t have all the answers, but our hope is that by asking these questions we resist deploying evaluation in ways that work against the very world we seek.
Dr. Heather Laine Talley is a writer and sociologist. Her articles on topics as diverse as feminism, romance, food, inequalities, and philanthropy have appeared in books, academic journals, and across the digital world. In 2014, her book Saving Face: Disfigurement and the Politics of Appearance was published by New York University Press.