10 Questions College Counselors Ask about Test Prep
In June I had the privilege of participating in the summer conference of the Association for College Counselors of Independent Schools (ACCIS). College counselors from the top schools in the country including Harvard-Westlake, Middlesex, Deerfield, Sidwell Friends, Hotchkiss, Trinity, and many others were in attendance. The theme of the conference was testing, and I was invited to participate in a 3-member panel focusing on the ins and outs of the college assessments and test prep. What do the top college counselors in the country want to know about testing? I took some time and wrote up my responses.
1) What role does confidence play in test taking?
Confidence is one of the most fundamental components of successful test taking. Students who believe in their ability to perform well ultimately achieve better outcomes. Students who lack confidence may experience heightened anxiety, which impairs working memory, focus and performance. Lack of confidence frequently becomes a self-fulfilling prophecy. The most direct way to increase student confidence is through the provision of mastery experiences via practice tests. With each successful practice test, a student’s sense of self-efficacy and confidence will grow, ultimately increasing the likelihood of a stronger performance on the actual assessments.
2) What defines a good test prep company?
- Consistency. A good company has consistent and effective quality control. It invests more in the training and oversight of its tutors.
- Retention of staff. The ability to retain qualified, experienced tutors is the hallmark of a good test prep company. Many companies pay poorly and/or mistreat their tutors, which leads to a turnover rate of 50% or higher each year (standard for this industry). This high turnover wipes out institutional learning. A good test prep company retains its best tutors who continue to develop and innovate.
- Integrity. Good test prep companies don’t make false promises. They are transparent. They keep their families and students in the loop, thereby eliminating most undesired surprises.
- Value. Good companies charge a fair price for their services: they do not encourage more prep than is necessary.
3) SAT versus ACT: how does your approach differ?
The ACT generally involves a shorter regimen of studying. When I prep students for the ACT, I spend more time on science, which is frequently the most foreign and challenging section for students. I sit there with a stop watch, and we do timing drill after timing drill, building speed and accuracy. I’m a sprinting coach when I’m teaching the ACT.
When I teach the SAT, my job is to help deconstruct and decode the test, to teach my students the language of the SAT and the tricks and techniques that work for each section. Critical reading frequently takes up the lion’s share of my time and energy on the SAT. No matter which test they are prepping for, students need to do practice tests for both the SAT and ACT.
4) What is the darker side of test prep?
There are certainly companies operating with little to no integrity in the marketplace. I’ve come across companies who conduct numerous shady practices:
- Sending out tutors who have high scores but are ill-prepared to teach. Some shops grossly under-train their tutors. They conduct brief interviews with high scoring candidates, send them a training manual and then ship them out to work with students. The trouble is, high scores alone do not a good tutor make. Often the highest scoring individuals, who have an intuitive grasp of math or reading, may be challenged to articulate their processes or transfer their skills to others. Without proper training, these brilliant test-takers may be ineffective tutors.
- Some test prep companies propose much more tutoring than is necessary. Many parents unfamiliar with this process do not know how to calibrate the right “dosage” of prep. We worked with a student who came to us with a 1460 CR+M baseline. A competing shop proposed 70 hours of prep; we sold her eight hours of tutoring and she achieved the gains she needed.
- Some companies have guarantees that are meaningless or even deceptive. Whenever you see anyone guaranteeing 300-400 points: run for the hills. That’s a racket. Every company has students who pick up 400-500+ points. However no company can guarantee that level of increase. Read the fine print. Perhaps the company will let your student repeat the 40-hour course that was ineffective the first time. How many students will be willing to start back at the beginning and invest another 40 hours? And what are the requirements to get the guarantee? Find out before you sign.
- Some companies create an artificial sense of exclusivity. In the end, we are all working with the same official materials provided by the test writers. Some tutors force students to sign non-disclosure contracts and charge obscene fees to justify their “trade secrets.” Again, this is a racket.
- Some companies manipulate their baseline scores to inflate perceived gains. It’s an old trick. Give students exceptionally hard baseline tests to deflate their intro scores. After the course is done and the students take a test of average difficulty, they will achieve significant gains from their bogus baseline. This is all artifice. Companies that do not use official tests for their baselines are walking a slippery slope.
5) Does test prep advantage those who need it least?
Not necessarily. I’ve worked with students who came from privileged backgrounds who had incredibly low scores, many suffering from significant learning disabilities. These kids absolutely needed help to be on par with their peers. Without a doubt, there are subsets of the population that are being underserved. The question of access and fairness is a real one. There are certainly haves and have-nots in our society, and those who have resources will use them to give their children every advantage: from private school to test prep to private universities. The question of fairness is a larger societal question and is certainly valid.
6) Is there anything a school can learn about itself, about its pedagogy, about its role in preparing students for college by looking at a class’s standardized test results?
I believe so. Certain schools have asked us to run analysis on their student body to identify gaps in their students’ learning. Sometimes students are missing a disproportionate number of abstract algebra problems, or a whole grade is weaker on certain grammar rules. We can see trends from school to school.
There is a bigger question regarding who is ultimately responsible for getting the students ready for the college assessments. The same questions arise regarding preparation for the AP tests. Is it the school’s job to teach to the tests? Most schools focus on the content but not the process of taking standardized tests. Schools can certainly teach students the process side of test-taking: how to identify wrong answers, how to deconstruct questions, how to prioritize time, how to work backwards and optimize all the information provided by the tests. By teaching students Algebra, a school is not necessarily preparing students for the SAT. By specifically drilling and deconstructing SAT Algebra questions, a school is preparing its students for the SAT. Schools must decide at a philosophical level if they are willing to play in this space.
7) a. Is there such a thing as beginning test prep too early?
There is definitely such a thing as prepping too early. The summer before junior year is the earliest I ever want to touch any SAT prep with a student. There are obviously exceptions to every rule—once I prepared a student for the sophomore PSAT to help him gain placement into a higher track English class. But that was an unusual case. Students starting SAT prep in middle school are on the wrong path. Until you’ve covered the actual content in your high school classes, it is wildly inefficient to prep. We want to review Algebra 2 in our sessions, not teach it for the first time.
b. What is the ideal timing for prep?
In my book the ideal timing to begin prep is early junior year. I love having a student knock out the December and January tests while he has momentum, returning to take the June test if it is needed. If we can get everything done by the end of junior year, the student is free to focus on other important tasks as a senior.
c. How do you know when enough is enough?
I do not recommend taking the test more than 3 or 4 times. There is a real risk of burnout. After the third or fourth iteration, most students begin to flat-line and hit a point of diminishing returns.
8 ) Are there broad claims – generalizations – that you can make about different types of prep for standardized testing, such as one-on-one is the most effective, or only the highly organized and motivated student can successfully prep on his/her own?
One-on-one prep is my preference, and according to the research I’ve come across in my graduate work, one-on-one instruction always yields the strongest educational outcomes. It’s the most efficient means of teaching, but it is obviously also the most resource intensive: one student, one tutor. This is going to cost more. That’s where groups come into play.
Groups can be very effective for some students, but they are not for everyone. Keep the really high scoring kids or the kids with incredible spikes/deficits in certain subjects out of the groups. Keep students with LD, motivational deficits, or severe test anxiety out of the groups: those students will do much better in private prep. Groups are excellent for students in the middle to upper middle end of the score range who need a balanced review.
The highly organized, self-regulated students rarely seek out our services. Students who are great at self-regulation will frequently manage their own prep and can often get by simply with the aid of good materials. Students who hit perfect 2400s are rarely the ones who do test prep.
9) Are there certain types of students, a certain profile of students who are most likely to benefit from test prep?
The students who are going to have the biggest leaps are generally those most willing to put in the time and make the requisite sacrifices. Time on task is a major predictor of performance. That’s why the Asian Nationals are having such phenomenal outcomes. One of my former tutors has set up shop in Shanghai and he keeps me abreast of the incredible studying regimens of the Asian students. In many cases, they triple the “time on task” of their American counterparts, reviewing 30-40 official tests, putting in 200 hours of their own time (not 200 hours of tutoring, but 200 hours of self-study). That kind of time commitment is going to make a difference. Self-regulated learners who are driven to succeed have the best shot at tremendous score gains.
10) Are there certain types of students who will be less likely to benefit from test prep?
- Students who are extremely stubborn and unwilling to relinquish their old methods will derive less benefit from prep.
- Students who don’t have any desire to score higher on the SAT or ACT, but whose parents are insisting on prep, will rarely have strong outcomes. Some students we’ve seen have already been accepted to a college and are quite content to matriculate there, but their parents are holding out for higher scores and more selective colleges. This is not the best recipe for success.
- Students with real motivational deficits will struggle. We can teach the skill, but it’s hard to provide the will to our students.
- Students who have overwhelming anxiety or deeply entrenched negative self-beliefs will struggle.
- Some students with severe LD will struggle if they do not have the appropriate accommodations.