Making Strides in Measuring the Impact of Youth-Serving Arts Organizations

Arts nonprofits working with youth are often asked, “What impact does your program have on the youth you serve?” They respond that youth keep coming. They bring friends. They’re proud of what they create. Some even make careers in the arts. But how often can youth-serving arts organizations really make the case that it was their program that made a difference in life outcomes of their youth? Can a case be made on more than attendance figures and anecdotes? Since 2008, the Boston Youth Arts Evaluation Project (BYAEP) has been working on these very questions.

While serving on a 2003 panel for the National Endowment for the Arts, I was troubled when several of the exemplary youth art groups I fund in Boston did not seem competitive on paper. Despite the high quality of youth work and arts instruction I had witnessed on the ground, many arts groups did not convincingly outline their outcomes and impact. I spoke about this challenge with Josiah Spaulding, CEO of the CitiCenter (formerly the Wang Center), and Charles McDermott, deputy director of the Massachusetts Cultural Council. These conversations evolved into a 2005 conference with a focus on arts evaluation called “Measuring Up.” More than four hundred people attended, fueling conversations about evaluation and outcomes between and among funders and nonprofits statewide.

One of the “Measuring Up” panelists was Kathe Swaback, program evaluator for Raw Art Works (RAW). Based in Lynn, Massachusetts, RAW’s staff of arts therapists takes art into the streets, public housing, youth incarceration facilities, clinics, soup kitchens, schools, and homes. The organization also offers programming in its studio in downtown Lynn. That day in 2005, Kathe approached me with an idea to test the premise that the National Research Council and Institute of Medicine had put forward in 2002, namely, that quality youth programs provide opportunities, a positive climate, and connections to create change in the lives of youth. Kathe suggested that if a core group of Massachusetts youth-serving arts organizations committed to working together over a number of years, that group could build a robust framework for linking program quality to outcomes for youth.

Three years and many conversations later, I received a proposal from Raw Art Works for BYAEP. For a modest, three-year grant of $90,000, Kathe and the staff proposed to work with a core group of five arts organizations: RAW, Medicine Wheel Productions of South Boston, Hyde Square Task Force of Jamaica Plain (Boston), the Theater Offensive of Cambridge, and Zumix of East Boston. Representing multiple disciplines (dance, public art, visual art, theater, and music), and with ten to twenty-two years of experience, these groups serve youth ages fourteen to twenty-two, more than 60 percent of whom are youth of color, with Hyde Square Task Force and Zumix involving primarily Latino youth, and the Theater Offensive the LBGT community.

The Barr Foundation ¹ awarded the BYAEP grant in September 2008, and Kathe was soon off and running. That October, she hosted a Youth Arts Forum for thirty nonprofit groups to share best practices in evaluation and to introduce BYAEP’s core collaborators. After the forum, BYAEP core collaborators have consistently applied the five tools of the BYAEP Framework: self-evaluations, program evaluations, artistic responses, teacher evaluations, and alumni evaluations. Since 2008 they have spent hundreds of hours looking at artwork by young people; creating, disseminating, and analyzing over two hundred youth evaluations; and sharing knowledge. In the process, they have developed a compelling overlapping frame — I Am, I Create, We Connect — for characterizing the kinds of impacts they are seeing. That framework explores many realms of affective development and community connection. It also explores youth identity formation, particularly the contributions of arts learning and creativity to resilience, engagement, and critical thinking (see youth statements at the end of this report).

Reviewing and discussing data across five programs over three years, one theme has been increasingly daunting: the devastating impact of violence in young people’s personal lives. Even within high-quality, highly supportive programs, violence diminishes drive and suffocates optimism about the future. One young participant said that after losing four friends to violence he had lost count, and he now “feels numb.” Others claim that the art programs are their only antidote to street life.

In BYAEP, the insights of practitioners combined with the voices of youth themselves, and with the academic expertise of the evaluator and several esteemed researchers. The project is ongoing, the process iterative, and the evaluation framework still improving. Yet many Boston-area youth arts organizations and arts initiatives are already benefiting from the preliminary findings. The next step is fund-raising to develop a common data platform and no fewer than three conference sessions to present and defend the methodology.

For more information, visit www.byaep.com or join us Monday, October 18, 2010, at the Chicago GIA Conference for the session titled “Measuring and Holding Change: Assessing Youth Art Programs.”

In the words of the youth:

I CREATE:
To see art being made, and staff who work to make RAW the best place it can be so that the possibilities are endless for RAW artists. — W2B

I AM:
I have learned that I have a powerful, important voice and to not second-guess myself. — RAW Chief

WE CONNECT:
When I walk into RAW, I count on being heard and being in an environment where I am safe and welcome anytime. — R2R Beginning

Notes

1. The Barr Foundation is a private family foundation in Boston, Massachusetts. Its mission is to enhance the quality of life for all of Boston’s residents. Barr’s primary grantmaking areas are public education and climate change. While arts and culture are a secondary focus, Barr is among the largest funders of the arts in Greater Boston. Historically, Barr’s arts grantmaking has emphasized work that advances the foundation’s public education and environmental goals; strengthens Boston’s major and midsize arts institutions; improves understanding of diverse ethnicities and traditions; and increases civic engagement.