February 28, 2011 by Bill

The debate over standardized test scores in college admissions is heating up in our own backyard. A few weeks ago, DePaul University announced it will no longer require undergraduate applicants to submit standardized test scores. Why not? Because the admissions office found that the scores added little to their ability to predict applicants’ academic success in college. And DePaul isn’t alone.

What does help predict academic performance, not surprisingly, are the applicants’ high school grades. The past is prologue, as Shakespeare put it. So DePaul has joined a growing group of colleges and universities, now in the hundreds, that have recognized the limits of standardized tests in the admissions process (see Fairtest.org for a list of schools that have made them optional or no longer consider them at all).

But, as others have argued, high school grades are not standardized: they differ from teacher to teacher and school to school, and they may be influenced by all kinds of cultural and environmental factors other than the student’s raw potential. So they’re not an ideal basis for comparison, especially when it comes to large applicant pools from different geographies and diverse circumstances.

Isn’t it better to rely on “standardized” tests, which are the same everywhere and for everyone? Aren’t SAT scores more, not less necessary to differentiate applicants in an era when grade inflation is making high school transcripts look increasingly alike? We live in a world that loves the single score, index, or metric that boils it all down for us. Standardized test scores lend some objectivity to the otherwise messy, subjective process of college admissions.

The trouble is, it’s not clear what tests like the SATs actually measure. Intelligence? Most psychologists would say they don’t measure innate intelligence or aptitude, or even academic potential; they measure developed abilities, and only certain abilities at that. Of course, the development of those abilities in any given individual is influenced by a host of environmental factors, including the hugely influential factors of race and class. ...

Those cultural effects have been disturbingly well documented in the literature, in books like “The Shape of the River: Long-Term Consequences of Considering Race in College and University Admissions,” by William G. Bowen and Derek Bok, and “The Black-White Test Score Gap,” edited by Christopher Jencks and Meredith Phillips. One example from that literature: In 1995, only 70 African-American high schoolers scored over 700 on the verbal portion of the SAT. Yet from “The Shape of the River” we know that many African-American students with lower SAT scores succeeded at highly selective schools and went on to become scholars, doctors, lawyers, business executives, etc.

I’ve been thinking back on those two books (both of which were published in 1998) and others like them, and wondering if I should be more cautious about interpreting standardized tests scores in the course of our work. Our research covers the entire student-life cycle, from prospective students and admits to current students and alumni. We sometimes include respondents’ SAT scores in our analysis of survey data, for example to see (retrospectively) how students’ entering SAT scores relate to later outcomes — for example, whether they get involved in campus activities, feel satisfied with their undergraduate experience, have a job offer when they graduate, and become engaged as alumni. (In some of those areas, there’s a strong correlation; in others, there isn’t.)

DePaul’s decision reminds me to be skeptical about our assumptions. What if, instead of telling us something valid and important about a student’s potential, the SATs, ACTs, and their ilk are actually telling us about the student’s circumstances — and society’s biases? What would that imply about the racial and socioeconomic diversity goals that many schools set, and how they balance those goals with the desire to admit applicants who score high on standardized tests?

I realized that, like many people in higher education, I’m working with SAT scores without a full understanding of their relationship to intelligence, academic performance, and potential to succeed more generally, both in college and after it.

And, it’s that last notion of “potential” that worries me most. Colleges and universities make admissions decisions in a matter of weeks that can affect people’s entire lives. We should always be thinking critically about the tools we use to make those decisions, and discarding those tools if and when they’re no longer helpful.

For the moment, my view is that standardized tests do have value when they’re one of many inputs into the admission decision. But we can easily take them too far or make them too central. I cringe to think of the lost potential in a world that over-emphasizes standardized tests and misses the broader picture of each person’s potential. DePaul has helpfully reminded us of what’s at stake.

1 Comment »