Next generation assessments for measuring complex learning in science

Type Report
Title Next generation assessments for measuring complex learning in science
Author(s)
Publication (Day/Month/Year) 2011
URL http://www.ocde.us/CommonCoreCA/Documents/Road_ahead_for State​Assessments_May_2011_Renniecenterforeducationresearchpolicy.pdf#page=31
Abstract
The adoption of the Common Core State Standards presents states across the nation with an unprecedented opportunity
to enhance the educational opportunities they provide students. States that have adopted the Common Core State
Standards are now in the early stages of revising curriculum frameworks, adopting new instructional materials, developing
new systems of assessment, and providing professional development for teachers to prepare them to deliver instruction
aligned to the new standards. This process has the potential to fundamentally transform public education for the majority
of U.S. students. It is therefore essential that policymakers and education leaders take full account of the issues and challenges
that lie ahead as early as possible in the implementation process.
New assessments can be a key driver of the successful implementation of the Common Core State Standards, providing
support for deeper learning and holding educators accountable for their students’ progress toward true college and
career readiness. When they are well-designed and well-used, assessments can motivate students and teachers, and focus
their attention on the knowledge and skills that really matter for student success. Conversely, though, assessments that are
poorly designed and implemented can narrow the curriculum, impoverish instruction, and undermine students’ enthusiasm
for learning.
Recognizing the transformative importance of assessments in realizing the promise of the Common Core, the U.S.
Department of Education has funded two consortia of states that will work together to develop new assessments aligned to
the new standards in English-language arts and mathematics. The Partnership for the Assessment of Readiness for College
and Careers (PARCC) comprises 25 states including Massachusetts and California, while the SMARTER Balanced
Assessment Consortium (SBAC) comprises 29. (Some states participate in both consortia.)
The two consortia are beginning to address the challenges of next generation assessments. But even as they strive to
expand the frontiers of current knowledge and practice, the constrained financial resources of most states and the short
timeline for implementation (new assessments are to be in use by 2014) makes the prospect of radical changes daunting.
The essential end game is to develop rigorous assessments that effectively and efficiently serve the twin purposes of
accountability and supporting better instruction.
As the two consortia begin their work, our hope is that they aim to build a system that responds to the immediate challenge
of measuring student performance against the Common Core State Standards, and that establishes a firm foundation
Table of Contents
Introduction ............................................................................................................................................. i
Computerized Adaptive Assessment (CAA): The Way Forward
Mark D. Reckase, Michigan State University .................................................................................................. 1
Strengthening Assessment for English Learner Success: How Can the Promise
of the Common Core Standards and Innovative Assessment Systems Be Realized?
Robert Linquanti, WestEd.......................................................................................................................... 13
Next Generation Assessments for Measuring Complex Learning in Science
Jody Clarke-Midura and Chris Dede, Harvard University
Jill Norton, Rennie Center for Education Research & Policy ............................................................................ 27
Closing Remarks.................................................................................................................................... 41
Glossary of Key Assessment Terms........................................................................................................ 41ii
The Road Ahead for State Assessments
for assessments that can continue to evolve toward ever-greater precision and utility over time. As the consortia move to
develop a system of assessment that measures student performance in core academic subjects against national standards,
the goal is to ensure that the systems are sufficiently robust and adaptive that they can provide the information needed to
assess diverse student populations and more complex and challenging subject matters.
This report includes three papers that address critical “next generation” issues in assessment policy that can help to
guide the choices made about system design: computer adaptive assessments, assessment of English learners and assessing
science. None of these topics has received the attention that it deserves in the current debate on assessment policy. These
three papers cannot provide a definitive or comprehensive plan for the next generation of assessments, but they do describe
some of the critical attributes of a better system. The common conclusion in all three papers is that assessment policy will
have to take full advantage of new technologies to provide useful and timely information to students and teachers about
the quality and effectiveness of teaching and learning. The authors’ provide a vision of new assessments that goes beyond
the horizon of current practice.
The first of these three issues is computer-adaptivity. Both PARCC and SBAC are committed to developing computerbased
assessment systems, but SBAC plans to develop a computer adaptive system, while PARCC does not. This is the
most significant difference in the strategies of the two consortia, and could have lasting implications for the next generation
of state assessments. The practical challenges that must be overcome to develop a state-wide computer adaptive system
are substantial, but such a system may establish the platform that will enable states to solve some fundamental assessment
problems, including the assessment of career readiness and the assessment of English learners. Mark Reckase of Michigan
State University discusses the state-of-the-art in computer adaptive assessment, and identifies the costs and consequences
of an immediate move toward the implementation of a computer adaptive system.
The second issue is the assessment of English learners. This is an urgent issue in California, where more than half
of school-aged children come from homes where English is not the primary language, but it is also an issue of growing
importance in states across the country. The fundamental question is how to design assessments that accurately measure
students’ mastery of academic content and not simply their mastery of Standard English. On the one hand, this requires
the development of better assessments for evaluating students’ English skills, as acknowledged in a separate federal grant
program for the development of English-language proficiency assessment systems. On the other hand, though, it will also
require the development of instruments that are simultaneously robust and flexible enough to assess the performance of
English learners in all subject matters, and not just in English. Robert Linquanti of WestEd surveys the critical issues in
the assessment of English learners, and points the way toward a system that ensures fairer and more accurate assessment
for all students, including English learners.
The third issue is assessment in science, where the inadequacies of traditional assessments have been particularly troubling.
Multiple-choice tests generally do a poor job of assessing students’ knowledge and skills in science. They are especially
ineffective at determining how well students are developing sophisticated inquiry skills—a key capability for science,
technology, engineering, and mathematics (STEM) careers. Chris Dede and Jody Clarke-Midura of the Harvard Graduate
School of Education describe the potential for next generation state assessments in science that utilize computer technology
to deliver and score assessments, and to report the results to teachers. The authors argue for new types of assessments
that deepen students’ understanding of core science concepts over time. They illustrate their argument with a case study
of virtual performance assessments (VPAs) in science that are currently in development at Harvard.
The work that PARCC and SBAC are doing marks a big stride forward in assessment policy, but this is only the beginning
of a long journey. Our hope is that the assessments that both consortia are developing will not only help to address
the challenges posed by the implementation of Common Core State Standards, but also put us on a path toward assessments
that more accurately and effectively measure and support students’ learning, and their progress toward readiness for
college and careers far into the future.

Related studies

»