Improving the Assessment of Student Learning in the Arts

January 2012, 161 pages. National Endowment for the Arts, 1100 Pennsylvania Ave, NW, Suite 618, Washington, DC 20506, (202) 682-5400, www.arts.gov

Download:

   Improving the Assessment of Student Learning in the Arts (2.1Mb)

Given the increased focus on assessment and accountability since the 1990s, the National Endowment for the Arts (NEA) identified the need to capture the current status of arts assessment. In 2005, the NEA began requiring a narrative statement of assessment practices to apply for arts education funds. Project applicants needed to explain their assessments methods and types of tools used to measure student knowledge and skills. Through several grant cycles, it became clear to NEA staff that applicants did not necessarily differentiate between program evaluation and assessment of student learning. As such, the NEA commissioned WestEd to examine current trends, promising techniques, and successful practices being used to assess student learning in the arts throughout the country, as well as identify potential areas in which arts assessment could be improved. Although the original intent of the study was to identify strong models of assessment practices that could serve as examples for possible replication, the study found that such models were not available and are in fact a need of the field. Thus, this report provides a description of the current state of arts assessment, including a review of the high-quality literature available, common practices being used to assess student learning, and needs of the field to improve arts assessment.

Through this study – the first of its kind – the NEA and WestEd sought to collect, analyze, and report on information about current practices and the needs of the field related to the assessment of K-12 student learning in the arts. Understanding the assessment experiences and practices of arts education stakeholders, including their needs, is one step toward helping improve student assessment in the arts. The goals of the current study included identifying:

• Available resources, tools, and documentation related to the assessment of student learning in the arts
• Current experiences and practices in assessing student knowledge in the arts
• Current experiences and practices in assessing student skills in the arts
• Trends in locating and using assessment tools
• Needs of the field to improve the assessment of student learning in the arts

Findings

There is a lack of publicly available high-quality assessment tools, informational documents, how-to resources, and technical reports related to K-12 student learning in the arts.

• Of 727 individual items reviewed, only 148 (20.4%) were both relevant to the study and of high quality.
• The majority of high-quality assessment tools focused on visual arts and/or music.

Few research and technical reports are publicly available.

• Although evaluation reports for arts projects are required by many funders, they are generally not being released publicly.
• Much of the available research literature focuses on learning through the arts rather than learning in the arts.

A lack of clarity exists regarding the difference between arts knowledge and arts skills.

• Survey respondents reported measuring student knowledge with methods more appropriate for measuring skills and vice versa (e.g., use of a paper/pencil test to measure student skills). This calls into question the validity of assessments designed and used by those who may not fully understand the difference.

Survey respondents use a variety of assessment tools to collect data for multiple purposes.

• All groups of respondents reported using many different types of skills assessment tools, including rubrics, observation protocols, portfolio reviews, and performance-based assessments.
• The majority of survey respondents reported that the tool they found most useful was created by a teacher or teaching artist.
• Reasons for collecting data included formative feedback, program evaluation, and district/school accountability. School staff most often reported using data for student grades, while arts and cultural organizations and state/county arts council staff were significantly more likely to collect data as a funding requirement.

The majority of high-quality, publicly available assessment tools are created by large-scale testing agencies and state education agencies.

• In general, assessments created by these larger agencies scored higher for quality than did assessments created by individuals or smaller organizations.

A need exists for a single, comprehensive clearinghouse for tools, information, and resources focused on assessing student knowledge and skills in the arts.

• Existing documents identified during the literature review process were scattered across many websites, journal articles, books, and other documents. High-quality materials were often mixed in with low-quality materials, making them even more difficult to locate and identify.
• Overall, the quality of assessment tools found on the web was low – yet more than three-quarters of survey respondents reported they use Internet search engines to look for assessment tools, often with little success.
• Survey respondents identified the need for exemplar tools (e.g., specific assessment tools, examples, and item banks) and models of successful assessment practices to learn from and replicate.
• More than three-quarters of survey respondents also reported they would create a new assessment tool if they needed one; however, locally developed tools tended to receive lower ratings for quality (potentially stemming from a lack of clarity between knowledge and skills and a lack of understanding about what is and is not a rubric).

There is a need for professional development related to arts assessment.

• Overall, the arts education field is eager to assess student learning. However, the field needs guidance and assistance to implement high-quality assessment practices.
• Findings from the literature review and survey responses indicated a need for training on the difference between assessing student knowledge and skills in the arts.
• Findings from the literature review and survey responses indicated a need for training regarding rubrics, particularly what constitutes a rubric, how it is properly used, and what components are necessary to develop or select a high-quality rubric.
• More than half of survey respondents reported receiving training on arts assessment via professional development workshops or conferences, whereas fewer than half of all respondents, including fewer than one-quarter for some respondent groups, reported receiving undergraduate- or graduate-level training on assessing student learning.
• Respondents reported needing additional training on topics such as: locating and identifying valid assessment tools, using rubrics and other assessment methods, and using assessment to demonstrate the importance of the arts.

Survey respondents reported needs of the field around four main categories – guidance, trained professionals, making the case, and additional needs.

• Guidance: a clear framework aligning standards, curriculum, and instruction; access to exemplar tools; models of assessment practice; resources; and professional learning communities (PLCs) to share knowledge and ask questions.
• Trained professionals: professional development, university training, and certification programs in all art forms to improve instruction and assessment in the arts.
• Making the case: demonstrating the value of the arts, including having research to show the impact of arts education; garnering support from school and district leaders; and implementing statewide or high-stakes testing as a method to increase the perceived importance of the arts among policymakers and other stakeholders.
• Additional needs: funding, time, technology, meeting the needs of diverse students, overcoming anti-assessment sentiment, and addressing the "subjective myth" that the arts are subjective and cannot be assessed objectively.

Overall, the arts education field is eager to assess student learning – survey respondents reported using a variety of assessment tools to collect data for multiple purposes. However, the field needs further guidance and assistance to implement high-quality assessment practices. A clearinghouse is needed to address the lack of publicly available high-quality assessment tools, informational documents, how-to resources, and technical reports related to student learning in the arts. There is also a strong need for professional development, both to address misconceptions and to improve the assessment of student learning in the arts. In particular, professional development is needed to clarify the distinction between knowledge and skills in the arts; to clearly define rubrics, how to locate/develop them, and how they are used to assess learning; and to dispel the myth that the arts are subjective and thus not able to be objectively assessed. The following recommendations to improve the assessment of student learning in the arts were derived based on the literature review and findings from the nationwide survey.

Recommendations

Assemble a national advisory committee to bridge assets and come to consensus on how to improve arts assessment.

• Include members from all stakeholder groups (e.g., teachers, teaching artists, district staff, policymakers, arts/cultural organization staff, researchers).
• Tasks for the advisory committee should include conceptualizing an arts-assessment clearinghouse, setting a national research agenda, and prioritizing professional development topics for the field.

Create an online clearinghouse for high-quality arts assessment materials.

• Develop a one-stop shop where teachers, teaching artists, practitioners, and researchers can access reliable and valid assessment tools, helpful how-to resources, high-quality research and evaluation reports, and relevant informational documents pertaining to arts assessment.
• Ensure high-quality, vetted measures and resources are available, leading to improved validity in arts assessment. Agencies and organizations looking to develop assessment tools must be willing to commit the time, money, and resources necessary to design high-quality measures.
• Features of a high-quality clearinghouse should include:
  • High-quality informational documents on myriad topics, including the distinction between knowledge and skills in the arts and glossaries of important evaluation- and assessment-related terms.
  • High-quality how-to resources on myriad topics, including how to locate/identify/develop valid and reliable assessment tools, especially rubrics.
  • High-quality exemplar assessment tools, including measures that are appropriate for varying art forms and grade levels, address the needs of appropriate audiences (e.g., classroom teachers, teaching artists), offer time-sensitive options, are easily adaptable with instruction on how the tool can be modified, and provide data that can contribute to larger research efforts.
• While the clearinghouse could take years to fully establish, in the short-term create a website with informational materials and/or professional development programs on high-priority topics like the difference between knowledge and skills in the arts and how to identify, develop, and use a quality rubric.
• First steps should be taken on the NEA website to maximize the impact of pre-existing traffic flow since survey respondents indicated they already visit the site in search of assessment-related tools and resources.

Establish online professional learning communities (PLCs).

• PLCs could be part of the clearinghouse or a separate entity.
• Maintain special communities for different stakeholder groups (e.g., policymakers, teachers, teaching artists, arts/cultural organization staff, researchers).
• Establish communities across content or topic areas, such as a location where participants can upload their assessment tool and receive constructive feedback, or share ideas on meeting the needs of diverse students.

Increase professional development offered in the area of arts assessment.

• Provide current information aimed at developing common understandings, sharing successful practices, and building the knowledge and skills needed to implement assessment in the arts.
• Tailor professional development to arts education audiences, such as separate tracks for teachers, teaching artists, policymakers/administrators, researchers, and arts/cultural organization staff. Professional development providers should be selected based on the specific needs of each group.
• Establish criteria for high-quality professional development that encompass webinars and regional trainings/conferences.
• Address specific need for professional development related to rubrics, including how to accurately define and identify them, how to select a high-quality rubric, and how to modify or develop a rubric to meet assessment needs.
• Address specific need for professional development related to recognizing high-quality assessment materials – particularly tools – since more than three-quarters of survey respondents reported using the Internet to search for measures.
• Partnerships among offices of education, arts councils, universities, and researchers can be used to bridge assets and provide comprehensive professional development programs.

Develop a national arts assessment research agenda and prioritize dissemination of tools and reports.

• Additional research is needed on a variety of topics (e.g., identifying models of successful practice in various settings, demonstrating how learning in the arts is beneficial to students).
• Priorities should be defined by the advisory committee and made public so researchers can respond.
• Establish a website for vetted, high-quality research offering both brief summaries and full reports. Encourage researchers to publish findings on this site and in other locations accessible to the arts education community.
• Assist funders and others to make evaluation tools and reports public. Address the issues of negative results and participant confidentiality.

Stop re-creating the wheel.

• Take action to reduce duplication and maximize efficacy.
• Establish a clearinghouse for high-quality arts assessment tools and resources, allowing practitioners to easily locate high-quality, vetted materials that meet their needs.
• Funders with constrained resources should work together to determine criteria and methods for grantee reporting, streamlining the process for recipients of multiple funding sources and possibly allowing for some comparative data.
• Push to make evaluation reports publicly available, allowing arts organizations and researchers to learn what methodologies have been used in the past and with what results, identify and build upon best practices, and avoid pitfalls through lessons learned.

Much is needed to improve the assessment of student learning in the arts. Establishing a national clearinghouse for high-quality assessment tools, informational documents, how-to resources, and research and evaluation reports is a key priority and would allow teachers, teaching artists, policymakers, practitioners, researchers, and other interested parties to easily access vetted materials. Professional learning communities should be established to support arts educators in their assessment efforts and allow for sharing of both questions and best practices. In addition, extensive professional development needs to be offered to targeted audiences on a wide variety of topics in order to increase the quality and validity of arts assessment. Three specific areas of need are:

1. developing valid and reliable rubrics;
2. recognizing high-quality assessment materials; and
3. understanding the difference between arts knowledge and skills, particularly with regard to appropriate assessment methods.

A representative advisory committee should be tasked with setting a national research agenda, prioritizing professional development topics, and conceptualizing the clearinghouse. While not exhaustive, these recommendations set the stage for significant improvements in the quality, efficiency, cohesion, and usefulness of student assessment in the arts, in order to ultimately improve student learning.