Applerouth
  • Test Prep
    • College Admissions
    • ACT Prep
    • SAT Prep
    • AP Exams Prep
    • PSAT Prep
    • Grad Admissions
    • GRE Prep
    • HS Admissions
    • SHSAT Prep
    • SSAT/ISEE Prep
  • Academic Tutoring
    • College Subjects
    • AP/IB Subjects & Exams
    • High School Subjects
    • Middle School Subjects
    • Elementary School Subjects
    • Executive Functioning
  • Self-Study
  • Free Resources
    • Free Practice + Events
    • Free Practice Tests
    • Free Events
    • Experts’ Corner
    • Applerouth’s Blog
    • Virtual Bookshelf
    • Resources for Educational Consultants
    • SAT & ACT Free Resources
    • 2020 ACT Changes
    • About the PSAT
    • Admissions Testing Policy Updates in response to COVID-19
    • About the SAT
    • About the ACT
    • What is a good SAT score?
    • What is a good ACT score?
    • SAT Test Dates and Registration
    • ACT Test Dates and Registration
    • SAT and ACT Test Day Checklist
    • SAT vs ACT
  • About Us
    • Our Difference
    • Methodology
    • Process
    • Results
    • Our Team
    • FAQ
    • General
    • Private Tutoring
    • Group Classes
    • Mock Tests
    • Testing
    • Online Tutoring
  • Contact
  • Login

The Trouble With the Curve

by Jed Applerouth, PhD on December 17, 2014

Something strange happened on the November SAT.  The morning of score release one of our tutors emailed us, confounded by his 750 on the math section.  He missed a single item on the math test, which typically would have yielded a score of 770-790.  In November, his single wrong answer dropped his score an unprecedented 50 points!  Moreover, other students who missed 2 questions attained scores of 720, and those who missed two questions and omitted two questions came up with a 680!  There is something funky with that curve!

Our tutors who had taken the test commented immediately afterwards how easy the math felt.  It appears there were simply too few difficult items on November’s SAT.  By creating a test lacking an adequate number of challenging items, the College Board forced too many students to the top of the scoring distribution.  Test-writers aim for a normal distribution of scores, with enough challenging items to differentiate between a student who scored a 740, a 760, and a 780.  With too few challenging items, the November test pushed too many students towards perfect and near perfect raw scores.  To spread the raw scores out over the scaled-score distribution, it had to push students who missed one item to a 750, and two items to a 720.

To put this in context, we examined the relationship between raw and scaled scores for students who attained a composite score of 50 on the math section, examining the College Board data all the way back to 2005.  There is a meaningful range of score conversions, from a high of 740 in 2011 to a low of 700 in 2013. It appears a student from the November test with a composite score of 50 would have attained a score of 680-690, a small drop below the 2013 score conversion.

But the big drop, the most remarkable difference is at the very top: the 53 raw, 750 scaled conversion.  That got everyone’s attention.

We’ve received multiple inquiries from our counselor and educational consultant colleagues: is this a trend?  Is November harder?  Are smarter kids taking November and should we counsel our students away from November?  In a nutshell: No.  November is a fine test.  By all accounts, this scoring conversion for the 2014 November Math test was an anomaly.  The test writers wrote an easy math test, and the score distribution reflects that.  Is this is a sign of things to come?  Not likely.  The December and January tests will most likely bring things back in line with typical score conversions, and it’s unlikely we’ll see this same skewed distribution next November.

2012 data2013 data2014 data

We pulled student data from the past 5 years for each testing period, showing in the charts above the November tests from 2012, 2013, and 2014, and found some points worth mentioning. First, evaluating the past three tests, there is no observable trend in the student outcomes for that month. The 2012 test saw fewer high-scoring students; 2013 had an even spread; and 2014, by comparison, saw a gap between low-700 scores and 800. Some November tests saw higher and lower scores, but the month as a whole did not suggest higher or lower outcomes for students. When we pulled the data for the other months, we saw a similar variance for the other test dates. This data supports the College Board’s assertion that there is no predictably easy or hard test date.

Second, when we look at November 2014, we see an unusually high number of perfect 800s and low-700s, but not much in the middle. With an easier math section, the College Board likely saw many more students score a perfect 800. In order to keep the upper end of the distribution curve accurate, it pushed the otherwise mid-700 students further down the scale.

Looking at our data from the past 5 years and beyond, we feel that they strongly indicate that the College Board does a fairly good job of equating the tests and keeping one test date from differentiating itself from the rest. Easier and more difficult tests will rear their heads, as we saw in November, but their occurrence will not happen in a predictable way. The best advice we could give our families is to plan for several test sittings. Should one test prove more difficult, it is advantageous to have a second test to mitigate the unpredictability of any one SAT.


Applerouth is a trusted test prep and tutoring resource. We combine the science of learning with a thoughtful, student-focused approach to help our clients succeed. Call or email us today at 202-558-5644 or info@applerouth.com.


  • Bill Dingledine

    What’s not mentioned by Jed (he was not focused in this way) is that anything above 700 is in the top 3% on a standard normalized test such as the SAT; so a 700-740-760 really is not that significant.

  • Jon Weininger

    Mr. Dingledine, are you saying the differences between a 700, a 740, and a 760 are not significant? There are many merit scholarships at many schools that suggest otherwise.

    If you are saying this steep curve only applies to 3% of people, then I would ask about the many more students who hope to break into the 700s in math and plan to put in the work to get there.

    Also, whenever the curve is unusual on a test, opinions differ on who is impacted. However, few argue that the unusual curve is not worth discussing.

    (Disclosure: I am a tutor at Applerouth posting my personal opinions at my own discretion.)

  • Bill Dingledine

    Jon — you make some good points but the fact of a normalized test is that it has to follow the Bell curve or it is not considered an accurate test. Yes, there are those (3%) who want to get higher, but we forget how many are not at that level (yes, it sometimes seems that everyone wants 700+, but in reality it is only a small percent). As for the “curve”, even though it might seem that many are not scoring higher, it is a miniscule difference in reality. So few people understand that. Even very selective colleges, and readers in their admission offices, don’t always understand how the tests are designed and therefore think a 760 is “better” than a 700. Jed can help you understand this further.
    Disclosure on my part: I am a college counselor of 35 years and a former franchise owner and teacher for The Princeton Review.

  • Jon Weininger

    Mr. Dingledine, I don’t think anyone is saying the test is not normalized, and I agree that a normalized test follows a bell curve by definition. Maybe you could say there was historically unprecedented behavior at the right tail of the bell curve because the Nov SAT had fewer level 5 (highest difficulty problems). One question omitted or missed costs 50 points –one question. Hard to believe.

    I think I understand what you’re getting at, and your point is something that I strongly agree with. While everyone knows that a 760 is a higher number than a 700, the difference in the skill of the student who scores 760 on one test versus the student who scores 700 on one test may not be captured by those scores because there may be no difference. What I mean is 700 student may have a higher GPA, a more compelling essay, or vice versa, but you could not tell which student achieved which score by talking to them, observing their academic performance, interviewing their teachers, etc.

    I completely agree that one student with a 720 and another student with a 750 register in my mind as having basically identical ability on that section. That is my personal perspective after tutoring hundreds of students over the years. Should the difference between one question away from perfect really cost 50 points? It seems like everyone would say no. Make the highest level questions harder to make the curve less severe.

    I also agree that people not immersed in the test world may assign extra meaning to the difference between a 700/720/740/and so on. However, until a clear alternative presents itself, at least comparing scores is a quantitative, apples-to-apples effort.

    One final thought, while the 700+ score may represent 3% of test takers, I believe the 600+

  • David S

    Why don’t they just give percentile scores? Then everyone knows that top 1% versus top 3% are very close.

  • Randy German

    While a few outlier tutors might agree that there’s a relatively small difference in skill level between 700 vs. 760, the only people whose opinions matter are those very people in those admissions offices and in the US News Rankings office. Beauty is the eye of the beholder, and the admissions committee and rankings guides are the ones doing the beholding.

    Let’s look at the Nov. 2014 in the context of Vanderbilt. Vandy’s 75th percentile on math is a perfect 800 (minus zero on Nov. 2014). Vandy’s 25th percentile is 750 (-1 on Nov. 2014). And if someone were unlucky enough to miss 3 multiple choice questions and no grid ins on Nov. 2014, thus missing 4 raw points because of the wrong answer penalty…the score on Nov. 2014 was a 680….eons and eons below the bottom 25% for Vandy to the point that the person’s odds likely drops into the mid-single digit % chance with 680. What a difference 3 questions makes.

    College chemistry classes always aim to make the test intellectually harder and then curve the test quite nicely so as to tease out the skill levels more precisely. The SAT would be wise to do so in the new incarnation of the SAT.

  • Geek Master Jeff

    Thanks for sharing this article! Interesting phenomenon. Collegeboard is always keeping us on our toes!

Applerouth

jobs educators testimonials test materials calendar locations blog policies contact us

© 2022 Applerouth. All rights reserved.