When SAT and ACT Scores are Called into Question

The SAT is back in the national spotlight, this time for putting a hold on a test score that it deemed questionable. This happens with some regularity, but rarely does it make the national press. In this case the student, Kamilah Campbell, a high school senior from south Florida has hired the services of a civil rights attorney who has taken this issue to the national media to pressure the College Board to reinstate her scores in time for Kamilah to secure admission to a selective program at FSU.

Why did the College Board flag her scores? In this case the scores were flagged due to a similar answer/error pattern between Kamilah’s responses and those of other students at the same testing location. The College Board used this language in a letter to Kamilah:

"Our preliminary concerns are based on substantial agreement between your answers on one or more scored sections of the test and those of other test takers. The anomalies noted above raise concerns about the validity of your scores."

Kamilah and her attorney have claimed that her scores were flagged because of the sizeable score increase she attained, raising her baseline 900 to a 1230, picking up 330 points. A spokesman for the College Board, Zach Goldberg, responded that scores are never flagged solely on the basis of score gains. According to our student data, this claim seems to be accurate. In a given year we have 25 students who increase 330 SAT points or more on the redesigned SAT or 9 points or more on the ACT, while we typically have at most a single student whose score gets flagged. It seems, at least in our experience, that sizeable score increases alone do not always result in a score being flagged.

When the College Board or ACT, Inc. place a hold on a score, there is often little a student can do but wait for a verdict to be rendered by the testing agencies. In some circumstances, a student will be allowed to retest, and if a score is achieved that is close enough to the score in question (i.e., within 100 points), the initial score will be validated. Inside Higher Ed recently reported on a story where a student who had to retest to reinstate his ACT scores is now filing a lawsuit against ACT, Inc. The suit challenges the manner in which the testing company locks students into a binding arbitration agreement before allowing their case to proceed, a practice which may be unfair and possibly illegal.

In many cases students have no idea why their scores are being held, and they remain in a state of limbo. One of our colleagues, an independent college consultant in California reached out in November after one of his students who took the October SAT received a notice that her scores remained in a “pending” status. The scores were placed under administrative hold until December. The only information the student received was that there was an issue with the test center and the student was left to wonder whether she had achieved the score increases she needed and whether her scores would be validated in time for applications to go out in January. When the College Board finally released her scores, they gave no explanation to the student.

We know that the testing entities are regularly looking for evidence of cheating when they calculate scores. They run pattern analysis on tests submitted to see if students sitting close to each other have similar wrong answer patterns, which creates a system flag. When they collect tests in a given room, they do so in sequence, and they can analyze clusters of tests. If there is a problem with the tests from a room, they can dive deeper and examine the paper test booklets to attempt to glean more information. Several years ago there was a massive inquiry by the College Board into the SATs administered at one private school in Atlanta. Many students from that private school had received similar coaching for the SAT essay, leading to the submission of dozens of similar SAT essays. The College Board noticed the extreme similarities and cancelled the tests of dozens of students.

In other cases there is some other irregularity that is reported which leads to an administrative hold and investigation. Many things can go wrong with the administration of a test: timing errors, proctoring mistakes, defective materials, students not following directions, and more. Many of these issues (outlined in page 6-12 of this SAT proctoring document and page 33-38 of this ACT proctoring document) require the submission of an irregularity form, which may lead to an investigation.

When students’ tests are placed in an administrative hold, there’s typically very little to do but wait. And most students who find themselves in this state of limbo will never get an answer regarding what triggered the inquiry. The College Board’s terms and conditions make it very clear where the power lies:

When testing irregularities occur, we may cancel an administration or individual registrations, decline to score all or part of the test, or cancel the test score. We may do this whether or not the affected students caused the testing irregularities, benefited from them, or engaged in misconduct. We are solely responsible for determining whether testing irregularities have occurred, and our decisions are final. When appropriate, we give affected test takers the opportunity to take the test again within a reasonable timeframe, without charge. This is the sole remedy available to test takers as a result of testing irregularities. Students and parents may not review scores from the affected administration before choosing the option of taking a makeup test.

The lawsuits filed against the College Board and ACT, Inc. aim to shift some of the power to the consumer.

As of this afternoon, 1/15/2019, students can now sue the ACT, rather than be locked into binding arbitration when they are accused of cheating. The New Jersey judge in the Clare vs. ACT case found that the binding arbitration requirement was “unconscionable” and “void as against public policy." The ability to file suit against testing agencies like the College Board and ACT, Inc. will allow for a more comprehensive and expanded period of discovery. This will help take decisions that have real-life consequences out of the "black box" of the testing agencies and greatly increase transparency. We were notified of this new development by Bob Schaeffer of FairTest.org