Breakout Sessions 1

9:45 AM to 11:00 AM

GROWTH MINDSET & BEING THE CHANGE

Pursuing Equitable Evaluation: A Team Approach to Developing a Reflexive Practice

Eleanor Fuchs, Molly Illes, and Laura Potter

Beginning in the spring of 2022, the Evaluation Team at the Center for Applied Research and Educational Improvement (CAREI) at the University of Minnesota embarked on an initiative to develop "Guiding Principles in Pursuit of Equitable Evaluation Practices," including prompting reflexive questions aligned to each principle. With the help of valuable resources (e.g., Okun, 2021; Kirkhart, 2010; Baptiste, et. al, 2017), the team leaned into the tension inherent in the creation of a collaborative critical reflection tool. 

Through ongoing review of existing resources as well as synchronous and asynchronous discussions, the team embraced patience and vulnerability to outline Guiding Principles that reflect the values and needs of CAREI's Evaluation Team. Anything but a checklist, this resource serves as a prompt for ongoing reflection, capacity-building, and iteration through a series of questions associated with interconnected principles. 

Attendees of this session will participate in a fishbowl discussion that includes a short presentation about the Guiding Principles, its development, and use. The CAREI team will share their experiences and will then invite the audience to step into the circle to discuss their own reflexive practice through a series of prompts. While intended to be an internal document, attendees will receive a copy of the Guiding Principles as a reference for the presentation.

CO-CREATION & POWER-SHARING

Working Together for a Purpose: Youth and Adults Share Insights from a Youth-Centered Evaluation Project

Jenna Sethi, Essence Blakemore, Neese Parker, Molly Dubois, and the 3M Youth Evaluation Team

Youth are often not seen as experts in any of the spaces they inhabit, yet their insights are essential for driving change. In many organizations and programs, youth are not invited to share their perspectives nor are they seen by adults as  “competent citizens with a right and responsibility to engage in their communities” (Richards-Shuster, 2012, p. 88; Noguera, 2003). Young people have much to tell us if we create spaces together that honor their voices and stories. As experts of their own experiences, their insights are crucial for informing improvement across a range of environments and programs. Youth-centered evaluation provides a unique opportunity to share power and co-create meaning, but it takes intentionality and strong planning to do it well. 

In this interactive, conversational workshop, the presenters (youth and adults) will share how they engaged in an eight-month evaluation process together to conduct research and share findings that would impact 3M’s STEM-focused philanthropic strategy. They will share concrete examples of key relationship-building and skill-building activities that participants can apply to their own work in the future. This project was funded by 3M and completed as a partnership between Informed Change and Youthprise.

EMBODIMENT & HEALING

When respondent comments hurt: toward harm reduction strategies for evaluators

Renae Youngs and Mariyam Naadha

Ensuring that evaluation subjects can answer questions honestly and candidly is one important aspect of our practice. Unfortunately, that candor can also cause harm to the people tasked with collecting or analyzing the resulting data. What do we do when evaluation subjects express views that have the unintentional impact of leaving an evaluator feeling attacked or retraumatized?

Since evaluators should not silence the voices of respondents, how do we mitigate or reduce the potential harms of comments that denigrate our identities? In this session, two evaluators who recently had this experience will create space for peer-to-peer sharing. We have more questions than answers and intend for this session to generate strategies, and perhaps a sense of solidarity, among attendees.

Topics for discussion in this session might include:

• Validation that these experiences are real and can cause harm
• Strategies to limit harm upfront 
• Strategies to limit harm mid-project
• Strategies to repair harm post-project
• Strategies to raise, interpret, and help clients grapple with results that cause harm 
• Strategies for allies or accomplices to reduce or deflect harm from colleagues

This session is for evaluators or analysts whose work has ever involved handling data in which evaluation subjects have denigrated our own identities, or those of someone on our project team.

Breakout Sessions 2

12:30 PM to 1:45 PM

EMBODIMENT & HEALING

Sanar Creando: Using Art Therapy as a form of expression for healing in a Latino community charter school 

Patricia Mudoy, Hana Bibliowicz, Rodolfo Gutiérrez, and Ericka Lara

Murals help communities tell their stories and engage them in dialogs. The active participation in the creation of a mural seeks to help students delve into their self-identity, express their cultural heritage, and channel their feelings where the words are not enough.

Sanar Creando was a multidisciplinary community-based project that used culturally inclusive expressive arts activities as a tool to assist students K-8 to express their identity and process the impact that the COVID-19 pandemic and the uprising following George Floyd's murder had in their lives and portray the expression of those feelings through a series of mural paintings. 

This session aims to give participants a small taste of the creative process that students went through to gain insight and reflect their feelings onto a mural. Using similar art directives, participants will reflect on different themes such as cultural identity, belonging and community and design their own mural. We will later contrast and compare the results of the original project with the public's and discuss our experience evaluating the implementation of this project.

GROWTH MINDSET & BEING THE CHANGE

Project Pilots: Opportunities to Test our Wings

Dr. Kamarrie Coleman and Rachel Engh

As learning partners and evaluators, we guide organizations through quality improvement processes, often using the plan, do, study, act (PDSA) model. We do this work to support organizations in understanding how they can deliver on their goals more efficiently and effectively. In our efforts to ground ourselves in research best practices, we use this frame without always taking to heart its implications for our own actions and planning processes as evaluators. 

In this session, ACET, Inc. will share what we learned through a pilot project with a large county agency to understand who they serve and how this learning informed a large-scale survey effort at over 20 service delivery sites. Piloting the processes allowed us to test them at select sites and adapt protocols for site specific needs for increased efficiency and data reliability when we scaled up to more sites across the county system. 

What if more of our evaluative work included pilot phases when we, as evaluators, could test, reflect, refine, and try again? How might the organizations we support also benefit from collaborating with us in this way? In this session, we will lead participants through an exercise to reflect on an evaluation project that could have benefitted from a pilot phase, identifying specific inflection points and approaches to test our wings.

CO-CREATION & POWER-SHARING

We're from the Government! Co-Creating Evaluation with Community from within a Giant Bureaucracy 

Dana DeMaster, Bekele Adamu, and Jessica Jerney

Join Ramsey County Health and Wellness Evaluation and Research for a discussion and shared learning of how we've been transforming evaluation from within a large government organization. Internal evaluators have many unique opportunities and challenges and coming from the government has its own complexities. In this session we will share our experience moving toward power-sharing, co-creation, community-centered evaluation within the specific context of government human services and then open the discussion to learn together.

On our journey we have developed and implemented co-learning plans with impacted communities and service providers; partnered with vets, homeless youth, and people with mental illness to design instruments; incorporated community engagement in defining program and service goals; and changed management and supervision practices to support and encourage this work.

Topics will include:

• Risk taking and pushing innovation from within a risk averse organization,
• Communicating a changing philosophy and expectation to business partners,
• Personal and professional growth necessary to walk this path,
• Practical ideas and how tos, and
• Management, supervision, and leadership to push change.

We have by no means figured this all out! We would love to discuss your learnings and experiences, as well as ideas and ways to share data analyses and developing recommendations with community and service recipients.

The format will include a panel discussion followed by a group discussion and shared learning. Depending on the size of the group, smaller discussion groups centered on personal and professional growth and change; pushing risk and innovation in a large organization; and management and supervision through change.

Breakout Sessions 3

2:00 PM to 3:15 PM

CO-CREATION & POWER-SHARING

High School: The Board Game

Kelsey Duffy and Kylie Van Dyke

Cradle 2 Career in Rochester, MN proposed the idea of creating a board game to a group of college students at University of Minnesota, Rochester. The students interviewed high schoolers and collected local-level data to create a game that was localized to our community. The result is a card game in which you select a student character with a set of specific strengths, work towards academic and career goals, and respond to high school milestones as they occur. This game serves as an avenue to explore issues of systemic inequity and personal preference. Come play with us!

GROWTH MINDSET & BEING THE CHANGE

Shifting power within non-profit ecosystems: How can we center communities we serve

Sabrina Roowala and Sarah Lawrence 

Non-profit organizations strive to be attentive to the diverse and sometimes overwhelming needs of the clients they serve. At the same time, they must work within the constraints of tight budgets, low staff capacity, and fixed donor reporting requirements.

The Center for Victims of Torture (CVT) works with torture and war trauma survivors from around the world, providing psychotherapy and social work services. Similar to other organizations, evaluators at CVT develop approaches and systems that adhere to the complex internal and external demands of the non-profit ecosystem. This activity will be rooted in the realities of serving survivors and will explore how our evaluation team and clinical staff navigate competing priorities that might otherwise negatively impact communities we serve. 

Together, we explore how evaluators can influence evaluation standards to better fit the reality of our client's lived experiences. Scenarios will include balancing the demands of funders that may conflict with a trauma-informed evaluation approach and navigating pressures towards standardization that do not meet the needs of a diverse client population. We have an opportunity to shift the methodological and content expectations of evaluation stakeholders and influence the field to adapt and recognize differences in communities we serve.

Embodiment & Healing

Acts of Accountability: Destabilizing Personal Power & Privilege with Evaluator Reflexivity

Elizabeth Taylor-Schiro

We all come to our evaluation work with multiple identities, many of them steeped in contexts dominated by colonization and white supremacy. These identities influence the way we connect with and respond to the communities we work with and the issues they are grappling with. How we approach our work and relationships with these communities. 

The problem with this goes deeper than the impacts. The problem is rooted in the lack of the evaluator's willingness and ability to acknowledge the power, privilege, and biases that comes with the role of an evaluator. in order for change at the the level of systems and institutions, disruption needs to begin within the individual through critical and active self-reflection - consistently and constantly interrogating our own experiences, values, and biases, but also looking at how those impact our decisions and actions; and then how can we mitigate those in our evaluation work. 

However, it can be difficult to know where to start. This session hopes to provide a starting place to practice reflexivity through conversations around the question “How do we hold ourselves accountable to both the self, as well as to the communities we serve as evaluators?”