January 19, 2011 by Rachelle

This is my first blog post as a research fellow at Slover Linett. Bill invited me to comment on Academically Adrift: Limited Learning on College Campuses, a book that was published this week and has quickly sparked debate. For those of you who don't know me, I work in assessment at Northwestern University and am the principal investigator of the Teagle Assessment Project and other national studies of student outcomes. But enough about me...

My first-grader took an online standardized math assessment for the first time last month. “It was so easy, Mom,” he told me, “Anne and I raced to see who would get done first.” “I can believe that,” said his teacher when she looked at his score during our parent-teacher conference. He had missed some easy questions in his haste, so the test didn’t even offer him the subsequent hard questions, which the teacher knew he was capable of answering correctly. In the world of high-stakes testing, there were no stakes as far as my son was concerned. And the testing instrument didn’t seem to do a good job of measuring what he knew.

This week, a new book based on undergraduates’ scores on an online standardized assessment is receiving a lot of attention in the higher ed world (see The Chronicle of Higher Education, Inside Higher Ed, and the NY Times). The sound-byte is that students aren’t learning very much in college. And everyone is to blame: faculty who don’t take teaching seriously enough, presidents and boards and a tenure system that doesn’t reward teaching, colleges that don’t provide challenging curricula, students who prefer partying over studying. The list goes on.

But I have to stop and ask, how good is our ability to measure what students are learning in college? And how motivated are students to show us what they’ve learned when we ask them to do so using a standardized test? ...

Designing a test to measure learning in college is incredibly challenging, because part of what we want to assess are thinking and reasoning abilities, many of which are developed through in-depth disciplinary training within majors. How well can a one-size-fits-all assessment tap into that learning across all those disciplines and departments? How can we be sure that when we do measure critical thinking we’re assessing that which is developed in college and not the abilities students had coming into college?

Perhaps these problems are surmountable with enough time and effort on the development of a really good tool. But the biggest problem I see is that college students (just like first-graders) don’t always do what you want them to do. Just because we take the tests seriously and make policy and hurl sound-bytes based on scores on online assessments doesn’t mean that students are inclined to do their best to show us what they’re learning. In higher ed testing, the stakes are often high for everyone except the test-takers, who don’t need to “pass” the test to get their degree.

I’m not advocating for more high-stakes tests in higher education. I just think that before we throw in the towel and declare U.S. higher ed a failure, we need to look hard at the tools and processes we’re using to measure college learning. Let’s expand the set of instruments and approaches we use to examine learning in young adults. Let’s get multiple measures of learning, compare the results, and then sound the alarms if we don’t like what we find.
 

1 Comment »