Measuring Non-Cognitives: Views from the NPEA Conference

Jonathan E. Martin
May 21, 2013

Measuring Non-Cognitives: Views from the NPEA Conference

Jonathan E. Martin
May 21, 2013

Measuring Non-Cognitives: Views from the NPEA Conference

Jonathan E. Martin
May 21, 2013

Measuring Non-Cognitives: Views from the NPEA Conference

Jonathan E. Martin
May 21, 2013

“The distinction I am making about perseverance and intelligence in high achievement is not merely an abstract one, some intellectual sleight of hand. It has everything to do with how an adolescent views himself and faces future challenges….It is to say talent is not destiny; character is destiny.” William McMullen, Head, Taft School (CT)

How Harvard Selects Its Students

Admission decisions at Harvard are not easy to make, but Fitzsimmons told us they try to stay focused on two key questions:

  • What kind of difference will this student make at Harvard, and how will he/she contribute to the learning of his/her peers in the college?
  • What will this student do as a graduate of Harvard; what kind of impact will she or he potentially have in the world over the next 75[!] years?

Roughly 75% of Harvard admits are admitted exclusively or especially for their cognitive qualities, while about 25% are admitted for the “bump” their applications get from demonstrated, compelling non-cog attributes. A comparison of these two different admit populations reveals that both have extremely high graduation rates (about 98%).

Fitzsimmons looks for what he describes as “persistent follow-through.” In his words, have applicants put in their “Gladwellian/metaphorical” “10,000 hours” in some particular pursuit—in music, athletics, service, art or other activity? Additionally, personal qualities and character traits, he told us, are “BIG” in their process. Among the things they rate on their 1-6 scale – from reviewing essays, applications, recommendations, and interview reports – are love of learning, intellectual curiosity, openness to new ideas and people, and intellectual originality.

Fitzsimmons warmly and enthusiastically encouraged the work of our Think Tank in identifying, elevating, and assessing for some or several important non-cognitive qualities. “It would be great to identify clearly key traits, traits you really think are great for kids to develop, and the observable correlated actions, and then declare how important they will be to the admission process. By deciding what’s most important in addition to the cognitive, and promulgating that we are selecting for them in assessment, we will have a really positive impact on these traits becoming widely taught, reinforced, and learned, even if, as we do, we find a diminishing of our ability to discriminate among applicants with these traits. That will be a tradeoff worth having.”

Fear the Non-Cog Mosh Pit

"The non-cog mosh pit?” Angela Duckworth explained the problem to SSATB’s Think Tank after her keynote address at this year’s National Partnership for Educational Access’ (NPEA) annual conference. In her fast-talking, faster-thinking, wide-ranging, chock-full-of-nuggets conversation, she told us that there is a fast-growing recognition of the importance of a wide array of key non-cog qualities—including mindset, grit, resilience, goal orientation, social belonging, and intrinsic motivation. But there is too much in the mix right now—it’s getting a little crazy.

"The mosh pit is becoming not useful because it is too wide, too broad, too much for kids to manage and focus on.” Instead, she urged the attendees at NPEA and the members of our Think Tank: “Choose something singular and specific, and experiment with it: testing carefully all along the way if it is working. We need to research, design interventions, experiment, and study results.”

After we explained the purpose of our Think Tank research, Dr. Duckworth responded with great affirmation. “Of course you are doing this work; the project of spreading the word and developing new tools for assessing the non-cog domain is on everyone’s mind right now. The culture is shifting in such a significant way to recognize, appreciate, and form a common understanding of these concepts and their importance. I can totally see why SSATB and independent schools would be taking a lead on this: independent schools have so much more room to innovate and experiment. Go for it.”

Duckworth has climbed out of the mosh pit by choosing the trait she believes deserves the most attention – grit. When asked about evaluating grit in admission, she quickly pointed out that we already see evidence of grit in what we collect. GPA, she argued, is partly a result of grit, not just intelligence, and teacher recommendations do give us good evidence on this. About using her self-assessment tool, she told us, “I think caution is merited.” However, she did suggest the possibility of using self-assessments in combination. The trick, she said, is to “triangulate” – i.e. assess via multiple measures such as self-assessment, teacher assessment, perhaps parental assessment, and student resumes. “Average them, and the problems begin to cancel out.”

She says that for each leg on the stool of grit evaluation, a comprehensive rubric must be prepared (which she told us she is doing in a large-scale project with Grant Wiggins). For each criterion, spell out in concrete and vivid detail what each category looks like in practice, with associated observable actions. Train people how to use these rubrics when interviewing or evaluating essays, and train them carefully.

EMA Members can view the full report in our Member Community.

Become a member to gain access to our full magazine
and more professional development tools.

Subscribe to learn more about EMA and our services.

Jonathan E. Martin
May 21, 2013

Jonathan Martin has 15 years experience as an independent school head, most recently as Head of St. Gregory College Preparatory School (AZ). He holds degrees from Harvard University (BA, Government, cum laude); Starr King School for the Ministry (M.Div., Unitarian ministry preparation); and the University of San Francisco School of Education (MA, Private School Administration). In 2008, he was a Visiting Fellow at the Klingenstein Center at Teachers College, Columbia University. He previously headed Saklan Valley School (CA) and Maybeck High School (CA). In the first stage of his educational career, he taught History, Social Studies, and English at Maybeck, and served in a role equivalent to Dean of Students. From 2010-12 he was a member of the board, and Program & Professional Development Chair, of the Independent School Association of the Southwest (ISAS). He was a contributor to the new National Association of Independent Schools publication A Strategic Imperative: A Guide to Becoming a School of the Future.

Ready to make EMA part of your enrollment toolkit?

Subscribe to learn more about EMA and our services.

Measuring Non-Cognitives: Views from the NPEA Conference

Jonathan E. Martin
May 21, 2013

“The distinction I am making about perseverance and intelligence in high achievement is not merely an abstract one, some intellectual sleight of hand. It has everything to do with how an adolescent views himself and faces future challenges….It is to say talent is not destiny; character is destiny.” William McMullen, Head, Taft School (CT)

How Harvard Selects Its Students

Admission decisions at Harvard are not easy to make, but Fitzsimmons told us they try to stay focused on two key questions:

  • What kind of difference will this student make at Harvard, and how will he/she contribute to the learning of his/her peers in the college?
  • What will this student do as a graduate of Harvard; what kind of impact will she or he potentially have in the world over the next 75[!] years?

Roughly 75% of Harvard admits are admitted exclusively or especially for their cognitive qualities, while about 25% are admitted for the “bump” their applications get from demonstrated, compelling non-cog attributes. A comparison of these two different admit populations reveals that both have extremely high graduation rates (about 98%).

Fitzsimmons looks for what he describes as “persistent follow-through.” In his words, have applicants put in their “Gladwellian/metaphorical” “10,000 hours” in some particular pursuit—in music, athletics, service, art or other activity? Additionally, personal qualities and character traits, he told us, are “BIG” in their process. Among the things they rate on their 1-6 scale – from reviewing essays, applications, recommendations, and interview reports – are love of learning, intellectual curiosity, openness to new ideas and people, and intellectual originality.

Fitzsimmons warmly and enthusiastically encouraged the work of our Think Tank in identifying, elevating, and assessing for some or several important non-cognitive qualities. “It would be great to identify clearly key traits, traits you really think are great for kids to develop, and the observable correlated actions, and then declare how important they will be to the admission process. By deciding what’s most important in addition to the cognitive, and promulgating that we are selecting for them in assessment, we will have a really positive impact on these traits becoming widely taught, reinforced, and learned, even if, as we do, we find a diminishing of our ability to discriminate among applicants with these traits. That will be a tradeoff worth having.”

Fear the Non-Cog Mosh Pit

"The non-cog mosh pit?” Angela Duckworth explained the problem to SSATB’s Think Tank after her keynote address at this year’s National Partnership for Educational Access’ (NPEA) annual conference. In her fast-talking, faster-thinking, wide-ranging, chock-full-of-nuggets conversation, she told us that there is a fast-growing recognition of the importance of a wide array of key non-cog qualities—including mindset, grit, resilience, goal orientation, social belonging, and intrinsic motivation. But there is too much in the mix right now—it’s getting a little crazy.

"The mosh pit is becoming not useful because it is too wide, too broad, too much for kids to manage and focus on.” Instead, she urged the attendees at NPEA and the members of our Think Tank: “Choose something singular and specific, and experiment with it: testing carefully all along the way if it is working. We need to research, design interventions, experiment, and study results.”

After we explained the purpose of our Think Tank research, Dr. Duckworth responded with great affirmation. “Of course you are doing this work; the project of spreading the word and developing new tools for assessing the non-cog domain is on everyone’s mind right now. The culture is shifting in such a significant way to recognize, appreciate, and form a common understanding of these concepts and their importance. I can totally see why SSATB and independent schools would be taking a lead on this: independent schools have so much more room to innovate and experiment. Go for it.”

Duckworth has climbed out of the mosh pit by choosing the trait she believes deserves the most attention – grit. When asked about evaluating grit in admission, she quickly pointed out that we already see evidence of grit in what we collect. GPA, she argued, is partly a result of grit, not just intelligence, and teacher recommendations do give us good evidence on this. About using her self-assessment tool, she told us, “I think caution is merited.” However, she did suggest the possibility of using self-assessments in combination. The trick, she said, is to “triangulate” – i.e. assess via multiple measures such as self-assessment, teacher assessment, perhaps parental assessment, and student resumes. “Average them, and the problems begin to cancel out.”

She says that for each leg on the stool of grit evaluation, a comprehensive rubric must be prepared (which she told us she is doing in a large-scale project with Grant Wiggins). For each criterion, spell out in concrete and vivid detail what each category looks like in practice, with associated observable actions. Train people how to use these rubrics when interviewing or evaluating essays, and train them carefully.

Jonathan E. Martin
May 21, 2013