December 17, 2019 | By Maria Romero and Sierra Noakes
As funders, government agencies, and service providers become increasingly focused on program evaluation results to make evidence-based decisions, evaluators and other researchers seek to answer this question.
In the last 20 years, there has been an increased focus on equitable evaluation, or evaluation that takes into account culture and context. Growing awareness has led to the development of multiple methodologies to guide equitable evaluation practices. These methods include: culturally responsive evaluation, participatory evaluation, collaborative evaluation, and empowerment evaluation.
Recognizing the nuances of a community is essential to collaborating with its members to improve programs aimed at promoting positive changes. Communities vary in their goals, aspirations, and social connections. They often experience challenges that can be best understood by intentionally including their perspectives in the evaluation process.
Traditionally, evaluation as a field has neglected to recognize the expertise of those being evaluated and rarely engages community members throughout the program evaluation. Many evaluators draw upon well established power dynamics in which researchers and funders hold the power and valorized knowledge while community members are studied from a distance.
Developing partnerships between evaluators and community members fosters equitable evaluation practices that are dedicated to continuous improvement. Through this partnership, evaluators can engage communities to inform desired outcomes, establish valuable research questions, identify appropriate methodologies, and contextualize findings. As we learned at the American Evaluation Association‘s 2019 conference, these partnerships often lead to unearthing innovative data collection strategies, stronger, co-designed research questions, and more meaningful evaluation outputs.
For educators, equitable evaluation matters because it offers the opportunity to better cultivate knowledge with their students and the larger school community. Whether educators are conducting a needs assessment or summative evaluation, including feedback and developing thought partnerships with fellow educators, students, and families allows for more fruitful lessons and paths forward.
Co-conducting a needs assessment along with participants creates projects that tackle the issues most important to the target community. This process encourages people taking on an evaluator role to build rapport and relationships with stakeholders while learning more about the context and culture. Additionally, this approach can help evaluators identify and better understand needs from the perspective of program participants, program designers, and the funders of the evaluation. For example, a developer conducting a needs assessment with teachers who use the product can acquire important insights as to what actually works and what additional supports are needed in the classroom.
When co-designing an evaluation project, we can learn from the unique perspective of stakeholders to collaboratively determine what methods and tools may work best. For example, using an online survey may not be the best method of data collection if community members have minimal access to the internet. Additionally, an evaluator may discover and build upon a form of data collection that the community already uses to collect and engage community voices. An after school program co-designing an evaluation with participants can learn what changes are desired from the perspective of those who are most impacted.
Seeking the perspectives of community members while analyzing results can lead to authentic, representative findings that might have otherwise been overlooked or misinterpreted. Evaluators who may not share the same background and experiences as those who are a part of the project may view results differently. One way to practice this approach is through shared data analysis. Working with participants to understand findings can lead to better recommendations and actionable takeaways. For example, a teacher co-analyzing data with students can learn from their perspectives about how they interpreted course objectives and can refine goals accordingly.
To learn more about Digital Promise’s equity work, check out our latest report “Designing a Process for Inclusive Innovation: A radical commitment to equity.” Subscribe to Digital Promise’s Learning Sciences Connections’ newsletter to read stories that highlight relevant and useful findings from the research community.
By Rastee Chaudhry and Barbara Pape
