"The Test" - Must Read, With Caveat.

March 11, 2015 Jonathan E. Martin

The Test, by NPR’s Anya Kamenetz, is a must-read for anyone who is reading this post. Almost by definition, if you are here on this page, you are someone interested in both the current state and the future of testing. Subtitled, Why Our Schools are Obsessed with Standardized Testing, and Why You Don’t Have to Be, she’s right to identify the limitations of testing, and she’s great at providing a breezy overview of the state of testing and what’s coming soon.

That said, this must-read leaves one equally delighted and disappointed. The Test is broken into four sections, the latter two of which are quite valuable, sharing important innovations in testing methodology, showing the ways tests can be far more valuably integrated into teaching, and providing (mostly) spot-on advice to parents on how to counsel their kids about testing.

But the first two sections fall far short. She writes about testing today with a hyperbolic petulance (“We subject our offspring to endless measurement… damaging our children in a testing arms race”), supporting her polemic with anecdotes from popular media (“children are locking themselves in bathrooms”).

The Test Cover

She opens with a top ten list of things to be concerned about regarding testing. Many of the items on this list are indeed very worrisome - they demand attention and correction. However, Kamenetz frames them not as items we should commit to improving, but as “arguments against testing.” In doing so, she muddies this section, and because it is the lead section, she confuses the narrative of the book as a whole.

Consider the very first item on her list of arguments against testing: “We’re testing the wrong things.” To illustrate her point, she uses a telescope metaphor: “the lens we are using is sharp enough, but we are focusing on just a few stars at the expense of a universe of knowledge.” Do other readers see the gap here? That we would use telescopes to focus on only a few stars is certainly a major opportunity loss, but it is no “argument against” telescopes!

She compounds the problem by citing Harvard testing expert Daniel Koretz, to the effect that some important educational goals “can be tested, but are not considered a high enough priority to invest the time and resources required.” But this citation is anything but an argument “against testing” - when you think about it, it is an argument for more testing. And this is just the first of her ten items.

Look, we can all agree there are many flaws in the current testing regime, but why don’t we pursue a line of inquiry about how we can fix them, rather than jump on the popular alarmist bandwagon and argue, poorly, against testing as a whole.

Anya KamenetzAnya Kamenetz

This isn’t just a problem of semantics; labeling her ten items as “arguments against” instead of “problems with” is emblematic of the overall rhetorical stance she takes when aligning herself with anti-testing populism. Rather than seeing the testing opposition fever as a prompt for a prescription, she instead spends the first part of the book raising the temperature.

She moves on to review the history of testing, and doing so is important to illuminate how we got to our current mess. However, she cherry picks, highlighting what are indeed the missteps or egregious actions in this history while doing little to recognize the many ways in which testing has leveled playing fields for students from non-elite schools or has helped us to recognize and confront the deep equity gaps in our educational services, except in a brief and somewhat dismissive aside (“lonely defenders”).

The chapters entitled “The Four Teams” and “Measuring What Matters” are, however, interesting, stimulating, even inspiring. Yes we can is the message here: yes we can use testing, measurement, and assessment productively and powerfully to enhance our childrens’ education.

She is especially enthusiastic about broadening testing for social and creative intelligence and noncognitive skills, and in this part of her book, she becomes aligned entirely with the work of the SSATB Think Tank. Here she shifts from opposition to testing to advocacy for it: “We desperately need to measure in ways we can trust and in ways that themselves promote these skills.” To organize new approaches to measuring these capacities, Kamenetz constructs four categories, grouped in a colorful but silly way as teams: Robot, Monkey, Butterfly, and Unicorn.

Patrick KyllonenPatrick Kyllonen

To be reductionist, the character assessment work currently underway at SSATB places us on Team Monkey, the one “using traditional psychological assessments to measure new [sic] skills and attitudes: social-emotional learning, motivation, well being.” From Montgomery County, Maryland to the international OECD PISA assessment, educators and assessors are increasingly using surveys like the Gallup student survey to collect this information and use it to improve schooling.

Team Monkey’s captain? In Kamenetz’s eyes, none other than our own Patrick Kyllonen, a research scientist at ETS. Why do I say “our own?” Because he is leading the team engaged by and working with SSATB and the “G32” group of schools in the development of our new Character Skills Assessment. As Kamenetz explains, he’s a leader in the work to make these kinds of tests “harder to game than simple agree/disagree questions.”

Teams Robot, Butterfly, and Unicorn also intrigue. Team Robot uses “learning analytics” to track student learning in online environments and generates data on their progress, as exemplified in the Khan Academy and Knewton platforms. Team Butterfly celebrates “performance assessments” - portfolios of authentic student work such as essays, experiments, videos, and more evaluated with comprehensive rubrics by trained assessors.

Team Unicorn represents “an attempt to have it all,” measuring a breadth of student capacities as they complete performance tasks in data-collecting online environments. Gamification of assessment is perhaps the crown jewel of this approach, and researchers are working hard to make it happen. Among the leading experts cited in this regard are Norma and Vivienne Ming, from the research lab SOCOS, friends of SSATB with whom the Think Tank met last spring and who are profiled in the 2014 Think Tank Special Report.

Though it’s true testing needs fixing, it also offers extraordinary promise for improving our childrens’ education. Read The Test to learn more about that promise; not to be reminded (yet again) that there is an anxiety in the world about the flaws in testing today.

About the Author

Jonathan E. Martin

Principal Jonathan E. Martin Educational Services (AZ) Jonathan Martin has 15 years experience as an independent school head, most recently as Head of St. Gregory College Preparatory School (AZ). He holds degrees from Harvard University (BA, Government, cum laude); Starr King School for the Ministry (M.Div., Unitarian ministry preparation); and the University of San Francisco School of Education (MA, Private School Administration). In 2008, he was a Visiting Fellow at the Klingenstein Center at Teachers College, Columbia University. He previously headed Saklan Valley School (CA) and Maybeck High School (CA). In the first stage of his educational career, he taught History, Social Studies, and English at Maybeck, and served in a role equivalent to Dean of Students. From 2010-12 he was a member of the board, and Program & Professional Development Chair, of the Independent School Association of the Southwest (ISAS). He was a contributor to the new National Association of Independent Schools publication A Strategic Imperative: A Guide to Becoming a School of the Future.

More Content by Jonathan E. Martin
Previous Article
Building an Admission Office for the 21st Century
Building an Admission Office for the 21st Century

As the demands for complex marketing and recruitment strategies grow each year, fueled by the simultaneous...

Next Article
The Annual Report: Archive Data for Data-Driven Decisions
The Annual Report: Archive Data for Data-Driven Decisions

While reviewing many excellent blogs, articles, and presentations on the topic of admission data and repor...

Turn up the dial on your digital marketing. Check out our new webinar series!

Register Now