From The Yield, Winter 2013
1. All tests are not created equal!
While standardized testing has garnered more than its fair share of media attention in the last few years, it’s important to remember that the SSAT is an admission test. Given under “standard conditions” (meaning it has uniform conditions and administration at all test centers as well as uniform scoring procedures), it is not to be treated as other intelligence or aptitude tests, classroom tests, English proficiency, or placement tests. In the case of the SSAT, the test is designed to predict first-year academic success in an independent school.
2. The SSAT is a combination of aptitude and achievement.
The SSAT is designed to determine if a student has the baseline skills necessary to succeed (achievement) in a given environment, and to examine student’s ability to reach beyond and process new data (aptitude). In this way, the test provides reliable information about the current status of a student’s skills in tested academic domains, as well as indicating the potential to extend those skills(1).
3. Questions on the SSAT are written by independent school teachers.
Teachers in our schools, who have been trained by The Enrollment Management Association in the science of “item writing,” are the source of test questions. To develop the actual test forms used, The Enrollment Management Association convenes review committees composed of both content experts and independent school teachers. The committees reach consensus regarding the appropriateness of the questions. Questions accepted by the committee are then pretested (i.e. administered to SSAT test-takers in the unscored experimental section of the test) and analyzed. Questions that are statistically sound are selected and assembled into test forms.
4. A well-written admission test question is only answered correctly, on average, 50% of the time.
For a classroom teacher looking to measure the content and skills students have learned, this scoring percentage would be considered an utter failure! Admission test questions are designed to allow for differentiation in the applicant pool. This differentiation is necessary; if all students performed equally well on the SSAT, it would fail to be useful when making selection decisions between and among applicants.
5. The SSAT is a norm-referenced test.
A norm-referenced test interprets an individual test-taker’s score relative to the distribution of scores in a comparison group, referred to as the norm group. The SSAT norm group consists of all the test-takers (same grade/same gender), who have taken the test for the first time on one of the Standard Saturday or Sunday SSAT administrations in the USA or Canada over the last three years. It’s important to note that students taking the test are applying to college preparatory, independent schools, so the SSAT has a highly-competitive norm group. A percentile score of 60% means something much different on the SSAT than it does on a classroom test!
6. Test-taking strategy matters.
The SSAT is uses formula scoring. Students receive one point for every correct answer, zero points for omitted questions, and minus one-quarter point for each incorrect answer. A student’s strategy for guessing and omitting questions is a valuable element in interpreting the student’s score. For example, if a student does not omit questions, it could be that the student didn’t understand how to take the test or didn’t understand the directions.
7. Equating adjusts for differences across test forms.
Different SSAT forms are built and administered to students each year. Test developers follow test specifications when they assemble new forms so that different forms can be parallel in the degree difficulty as much as possible. In reality, it is inevitable that there are variations in form difficulty. A statistical procedure referred to as score equating is used to adjust for minor form difficulty differences, so that scores reported to students on different forms are comparable.
8. The scaled score is the most consistent for comparing one student’s abilities to another.
Unlike the percentile scores, which are variable depending on the pool of students taking the SSAT in any given three year period (i.e. the norm group), the scaled score indicates actual performance on the test as derived from the raw score of rights, wrongs, and omitted questions during the score equating process. SSAT scores are reported for each subsection on a scale of 440-710 for the Middle Level test and on a scale of 500-800 for the Upper Level test.
9. The SSAT is highly “reliable.”
As the questions on a particular test represent a mere sample of the many questions that could possibly have been included, one must consider how closely the test results agree with the results that would have been produced by a different set of similar questions. In testing, this concept is referred to as “reliability.” For scaled scores, a reliability coefficient of 1.00 indicates perfect reliability; a coefficient of .00 indicates no reliability at all. The Middle Level SSAT tests have reliability coefficients ranging between .82 and .93. The Upper Level SSAT tests have reliability coefficients ranging between .82 and .94.
10. Validity is best assessed on a school-by-school basis.
Validity indicates how well a test measures what it purports to measure. Typical measures of validity are the correlations between test score (predictor variable) and performance (e.g., grade point average) in school (this relationship is the criterion variable). Ultimately, the validity of the SSAT depends on how it is used by the individual school. Therefore, the user school should conduct its own validity study whenever possible. The Enrollment Management Association provides a validity study service to members.
The Enrollment Management Association offers free score interpretation webinars for admission professionals and admission committee members. These online sessions allow for Q&A with a member of our Outreach Team. Visit your MAP to see if there is an upcoming webinar, or to view a recording of a past score interpretation webinar. For an individualized session for your office or admission committe, contact your Outreach Director.
(1) McClellan, C. (2013). Admission Testing in Context. The Enrollment Management Association, Special Report: Think Tank on the Future of Assessment, pages 7-8, 23.