Can we all just hold hands and shout “DUHHH!!!!” together? NPR reports on a new study this morning:
Today, some 800 of the roughly 3,000 four-year colleges and universities in America make SAT or ACT submissions optional. But before a new study released Tuesday, no one had taken a hard, broad look at just how students who take advantage of “test-optional” policies are doing: how, for example, their grades and graduation rates stack up next to their counterparts who submitted their test results to admissions offices.
. . . . . .
[Former Bates College Deanof Admissions William] Hiss’ study, “Defining Promise: Optional Standardized Testing Policies in American College and University Admissions,” examined data from nearly three dozen “test-optional” U.S. schools, ranging from small liberal arts schools to large public universities, over several years.
Hiss found that there was virtually no difference in grades and graduation rates between test “submitters” and “non-submitters.” Just 0.05 percent of a GPA point separated the students who submitted their scores to admissions offices and those who did not. And college graduation rates for “non-submitters” were just 0.6 percent lower than those students who submitted their test scores.
How now? It turns out that “high school grades matter–a lot:”
For both those students who submitted their test results to their colleges and those who did not, high school grades were the best predictor of a student’s success in college. And kids who had low or modest test scores, but good high school grades, did better in college than those with good scores but modest grades.
Hiss says it’s probably not so surprising that a pattern of hard work, discipline and curiosity in high school shows up “as highly predictive, in contrast to what they do in three or four hours on a particular Saturday morning in a testing room.”
I can’t find an actual link to the study–it looks like the website that NPR links to needs to be updated. However, those of us who question the stripping and de-funding of teachers’ salaries in favor of spending money on high-stakes testing at the K-12 level should consider the ways in which the existence of the SAT and the ACT helped pave the way for high-stakes testing at lower grade levels. Should it surprise anyone who works in education that past good grades–or three or four years of good grades, as William Hiss says–are more predictive of future good grades than any other measure?
I wasn’t an ACT or SAT resistor. I took the PSAT and the SAT in high school, and the GREs in college. I was even a National Merit Scholar finalist, and won thousands of dollars for college as a result. I know my way around a standardized test. But this study vindicates the work that teachers and students do over time to prepare for college. Should we really be surprised to learn that butts in the seats and working for four years yields more useful predictive data than prep classes, cramming software, and one or two Saturday mornings in a student’s junior year of high school?
Maybe my embrace of this report is because I was a good-grades getter who never believed in the “unheralded genius” who “tested well” but his grades were mediocre to terrible. Not that I didn’t believe he existed–and he is almost always a he, isn’t he? I just didn’t believe that he was all that smart, just undisciplined and a f^(k-up. I’m not saying that absolutely everything about school is entirely necessary for future college success, but K-12 education is essentially a thirteen year-long marshmallow test. Not everyone’s high school background is the same, but those who prioritize good grades will likely figure out a way to do so in college.
My father is a retired HR executive, and he spent much of his career hiring and firing people. Not a philosophical man, he was interested in real-world results. When I was going to college, I asked him if he ever considered a job applicant’s college or high school GPAs, and he responded “absolutely. Grades don’t necessarily tell you everything about what someone has learned, but they certainly tell you how hard that person is willing to work.” That’s my experience in nearly twenty years as a teacher. Students who get good grades make getting good grades a priority and they work harder at them.
What if they gave a test on Saturday morning and no one showed up?
Historiann 
The worry is that just using GPA leads to K-12 grade inflation. You have to know something about the schools in order to completely get rid of a standardized measure.
Presumably AP exams etc. help, though not all students have access to AP courses.
LikeLike
The NPR story link now has a live link to the study. I think this study shows that HS grade inflation is not an issue. There were 123,000 students in this study over most of a decade (2003-2010).
What the SAT and ACT really test are a student’s parent’s/parents’ income/s and their ability and/or willingness to pay for prep courses & software. This study finds a high degree of correlation between HS and college grades. The larger point of the study appears to be an argument in favor of ditching the board scores b/c they serve to limit applications from students who don’t have the aforementioned advantages.
LikeLike
I still have most of my “grade” school report cards (where, oddly enough, they don’t grade you a whole lot, but you learn about the “plays well with others” or “needs to improve penmanship” stuff). Regrettably, the cards for the execrable grades I got in junior high school, and the amazingly better ones I got in 9-12 are lost in the ether. The latter included a lot of detailed comments that teachers must have sweated over on deadline five or six times a year. The only things from my preparatory years that I think “predicted” anything about college were basic curiosity, which I think is/was a native trait; a certain fear of parental wrath that the Buckley Amendment would protect me from if I was in college now; and having really learned how to write. I wrote my way past so many national merit scholars the first year of college (some of whom were actually and shockingly to us “counseled out”) that I thought they were driving backwards and wondered if I was going the wrong way. After that it was mostly coast and flurry, like my hero of those days, Muhammad Ali. I thought of it as a perverse kind of “time management.”
LikeLike
Are exams a way to evaluate students or are they a mechanism to control and supervise students?
A decade ago, I eliminated exams in my classes. Instead, student get assignments and small projects. Works for me; student like it and do well.
LikeLike
Bravo to your dad!! My selective SLAC went test-optional in admissions about 7 years ago, and I have noticed no difference in quality of students. Overall they’ve gotten better, but I don’t know if there’s a demonstrable cause-and-effect with eliminating required SATs.
What I have seen a lot of over 18 years of teaching here are students (yes, almost all guys) who apparently tested well enough to get admitted, but once they arrive they can’t write and *especially* they can’t manage their time to save their lives. Organization and willingness to work hard take a student much further than “native genius,” whatever that is.
As a student myself I realized that I preferred writing papers to taking exams any day of the week. An exam is one shot, and if you have a bad day too bad, but a paper allows you to show what you can really do.
LikeLike
Hiss says it’s probably not so surprising that a pattern of hard work, discipline and curiosity in high school shows up
This is elitist BS. I’ve seen great volumes of (anonymized) high school grades and the (unpublished) results of longitudinal analyses that looked for predictors of success in those data. By predictors I mean: is there an optimal sequencing for classes, does a high grade in one particular class correlate with success more broadly, and so on.
Here’s what I learned: if a student doesn’t start high school in the right gateway classes, ze will end up with a poorer record than a student who did. This is because progress is rigidly sequential: if you don’t start in the right track you will never jump into it and never end up in the right upper division classes, with the right grades. Indeed, you will likely fall farther and farther behind the high scoring kids. The students who started in the “right classes,” nearly all came from the same few middle schools. I only know about those schools anecdotally but my impression was that they were relatively wealthy, with relatively involved parents and curricula that emphasized advancement.
Work habits are important but the fetishization of these is, imho, off the mark. The attributes we associate with academic success are cultivated in certain students and are part of a collection of factors that guide those students onward and upward. My take on this is that elite parents have the resources to cultivate elite students.
ACT/SAT is only one of the guiding factors. That it turns out to not be the most reliable is interesting. I guess it means that not all of the well-cultivated students perform well on these despite the extra help and that bright students who did not have that cultivation went on (as Indyanna describes) to bloom at university. I’m sure the testing-industrial complex is drafting its reply to this deeply flawed study even as I type.
LikeLike
Generally, this strikes me as a good idea, though I did get the impression from the NPR report (to which I listened medium-closely) that the institutions that had tried it seemed to be mostly small to medium size (though with student populations fairly diverse in income, ethnicity, citizenship status, family history of college education, etc.). I do wonder if they already have somewhat more thorough, labor-intensive admissions review processes that allow them to place GPAs in context more effectively than larger institutions could.
I do worry that privilege is still going to play a role. If nothing else, families with at least one parent who has time to supervise/nag the child(ren) while they plan and complete assignments, etc., and/or to plead for leniency when the they miss deadlines for a whole variety of reasons, from valid to not, are going to send a higher proportion of students whose high school grades aren’t really good “marshmallow test” measures on to college than families that may well be equally caring and supportive in many ways, but just don’t have the time to spend compensating for their offsprings’ still-developing executive function (and/or making sure that all disabilities receive accommodations, that said accommodations are indeed afforded, etc., etc.).
My view may be somewhat skewed by interacting a good deal with parents in some very expensive nearby suburbs with highly-educated, ambitious parents (not, for the most part, where my students come from, but where I grew up, and still go to church). Still, I think some privileged kids who aren’t quite ready for colleges as selective/challenging as their parents would like them to attend are going to get in, and some at least equally-able less-privileged kids are going to end up in a less selective school, or working for a while straight out of school, or joining the military.
Of course, those options may not be bad ones (assuming that the potential student makes it through military service unscathed, a big assumption in the last decade or so), and might, in fact, be better ones for the privileged kids as well (and yes, in most cases I know of, the young people who fit in this category are male). As parents become more involved at the college level, the chances of a privileged student who doesn’t entirely have his act together making it to the B.A./B.S. and then falling apart at the point of trying to enter the work world become higher.
But none of the above is an argument in favor of the SAT, just an observation that, whatever the admissions criteria are, privileged parents will target considerable time and resources toward making their offspring measure up as well as possible to those criteria (and have considerable incentive to do so, especially when we’re talking about getting into a public-Ivy flagship state u, or a private institution with an endowment large enough to offer generous financial aid even to comfortably middle-class families).
For whatever it’s worth, I’ve always done well on standardized tests, and have even written reading-comprehension questions for one high-stakes grad admissions test. I felt those questions measured real and relevant skills (e.g. distinguishing supporting points from main arguments, or an argument paraphrased in order to refute it from the author’s actual point), but I’m not sure about the value of other parts of those tests, and worry, especially at the college-admissions level, about cultural biases (at some point, being familiar and comfortable with mainstream culture may become an acceptable measure of readiness for to take on various roles, but it seems to me that college is the first real opportunity for many people from less-privileged backgrounds to begin to acquire that familiarity, and I’d hate to see someone hard-working and able lose that opportunity on the basis of, say, a more narrow vocabulary). I also wonder whether limiting time on the tests really serves any purpose (well, other than the convenience of the test administrators), especially since longer time is a common disability accommodation, and getting a disability certified is time-consuming and expensive (i.e. a likely marker of privilege).
LikeLike
Well,the University of California has oodles of research to back this up; and it’s one reason that the top 9% in every high school in CA is guaranteed admission to UC. Consistently, high school achievement correlated with success in college. And going by high school controls for any potential grade inflation, and does not privilege the elite. I’ll try to find a link to the report that was done on this, but the short version is that this is mot just true for SLACs.
LikeLike
I’m not saying what is, I’m saying what would be. If you don’t think that wealthy parents are going to buy their children higher grades once any and all objective measures to compare schools are gone, then you’ve obviously never seen the grades at Harvard. It’s only bourgeois and lower SES unis that have the luxury of failing (or even giving Cs) to students. (The GRE doesn’t serve the same purpose as the SAT because most kids don’t take it, and it tops out pretty low for students who had to do well on the SAT to get into college.)
Obviously there are kids who don’t test well and the tests are known to be culturally biased. Those points should be taken into account (though it is harder in the absence of affirmative action). There’s also going to be selection of high quality kids who don’t test well into the schools that don’t require tests– if testing were eliminated, there wouldn’t be that selection anymore. Even if something is internally valid for a small number of schools, it won’t necessarily be externally valid in the general equilibrium.
And we’ve admitted students to our grad program who got straight As in undergrad that we figured were just bad at standardized tests. Sometimes that’s true, but sometimes it really isn’t. The undergrad just had a lot of grade inflation and the GRE was measuring something.
re: top X% programs, wealthy parents game those as well. Kalena Cortes has a paper on students moving to take advantage of the top 10% plan. There are pros and cons to that happening. Top X% programs also hurt minority students compared to the affirmative action programs that they replaced, though they are better than nothing. (There’s a large literature on this topic both in TX and CA.)
re: Truffula’s comment, there’s a recent literature on that in economics as well– having a good teacher has big impacts, having the right classes not in the morning (math) has big impacts, having class not start too early has big impacts, having a professor in an early math class who gives you a lower grade can help you in a later class if the reason for the lower grade was they weren’t teaching to the test etc. etc. etc.
LikeLike
The author of this study isn’t proposing a cure to elitism in college admissions. He’s just looking at the data to see what’s predictive of success in college.
I agree that GPAs aren’t necessarily created equal, but for people who meet the basic requirement for college admission (i.e. a high school education), there is a way to sort out that crowd. As Susan says, it works in the UC system. I believe that students in Texas who graduate in the top whatever% of their high school classes are admitted to UT as well.
Relying on GPAs instead of board scores would appear not only to work, but if you read the study, it encourages more students from underprivileged backgrounds to apply to college precisely because they don’t have to report the results of one high-stakes test. No, it’s not a cure for elitism or a guarantor of complete equality of opportunity. But we’re talking college admissions, which is already a fairly elite subset of the population.
LikeLike
I will say, the kids I teach at a private K-12 tend to underperform on SAT/ACT because we don’t do a lot of multiple choice in most of our classes. Our kids do well on AP, but we tend to recommend only those kids who have a good shot at succeeding at them to take AP classes. Most of our kids do very well in college because a) they have awesome time management skills (we spend a lot of time on this) b) they have good reading and writing skills and have done some college level work already (even in non-AP classes) c) they’ve been exposed to college level databases like JSTOR and can do more than just a google search. They have been well-trained to ask a librarian for help and they are comfortable asking adults for help in an office hours setting. They have been trained in how to ask questions productively (not how do I get an A on this? what do you want? but identifying key points of the assignment and asking for clarification). For point of comparison, as a freshman in college, I had no idea what office hours were for. A history prof gave us a historiography assignment and I had no idea how to do it and could not make heads or tails of the assignment since I didn’t know what historiography was. I did miserably, of course. Another frosh did well and I asked how she knew what to do. “I asked the prof” she said. “You can do that?” I thought. “Wow!” Cue academic success.
BTW I was one of those kids who out-tested my grades but I was a) a late maturer physically and mentally and b) had an undiagnosed vision problem that affected my school work until the middle of tenth grade. I didn’t really do well in college until my last two years when I finally stopped growing.
LikeLike
Here is the study that is the subject of your blog post:
Defining Promise:
Click to access DefiningPromise.pdf
schoolsofthoughtny.wordpress.com
LikeLike
Thanks, Anna–that’s it!
LikeLike