The SAT's Experimental Section

The concept of an experimental section on the SAT isn’t new. There has been an experimental section of one kind or another throughout the test’s development, but what form it took, and who took it, has varied, with major consequences.

For the last two SAT administrations - the March and May tests - we have received reports from our team that suggest all students may see an experimental section going forward. This is a noteworthy change - from March 2016 until recently, only a subset of students received the experimental section on the SAT. To understand the potential reasons for the apparent shift toward universal administration, here is some (recent) history on the SAT experimental section.

Until 2016, every student taking the test had an equal chance of being given the experimental section. With the debut of the revised SAT in 2016, the CollegeBoard changed how they administered the experimental section. Only students who opted out of the essay would receive experimental questions, in order to keep the test from going too long for the essay-writers (the length of the test was one of the major criticisms that the revised SAT was designed to address).

This new approach kept the time down, but it had the unforeseen consequence of creating an experimental sample group that did not represent the broader testing population. As our founder, Jed Applerouth, points out: “At that point, the essay was required by many of the most selective schools in the nation, and thus the non-essay cohort of students were relatively weaker test takers than the cohort of students who took the essay.” What’s more, it was a small group: despite the fact that less than thirty schools in the nation require the SAT essay for admissions, 70% of students who took the SAT in 2017 took the essay. In Jed’s own words, “the College Board spent several years pre-testing questions on a sample of students that was not representative of the broader population of test takers.”

The result? Easier test questions. Since the experimental questions were tested on students who - by and large - were not applying to top-tier schools, the data suggested that the test questions needed to be easier to make the SAT a fair test. That might sound like a good thing, if you’re a high school student gearing up to take the SAT, but it really isn’t. Easy tests have extremely harsh curves, so missing a single question could drop a student from an 800 to a 770. There’s no margin for error. What’s more, the practice materials released by the College Board don’t match the newer tests that use these poorly-calibrated questions. Many students who consistently scored well on practice tests scored much lower on the real exams, leading to general outcry and online petitions to rescore the test.

Under these circumstances, we weren’t exactly surprised to hear from students who took the March test that they all received the experimental section, regardless of whether they took the essay or not. We couldn’t find any information from the College Board about a change in administration, but we received further reports from the May test.

Students taking the test in May reported the same experience that the March testers did. They received an experimental section, regardless of whether they took the test with the essay or not.

What really struck them was the proctors’ script during this section. The proctors explained that some of the questions in the experimental section actually counted for points, while some of the questions in the standard four test sections were experimental. The proctors didn’t clarify which questions counted and which were experimental.

It’s difficult to predict how the College Board might score a test with experimental questions in the midst of the test and real test questions in the experimental section, but we are hoping that the May Question-and-Answer Service will shed some light.

Time will tell if administering the experimental section to a broader group of students will help the SAT normalize its score curve for future tests. Until that happens, we may see more months like June of 2018, where the test is easy and the scoring unpredictable.