The rap on research for the arts, museums, and education

June 24, 2011

Can we be ‘academically adrift’ while ‘racing to nowhere’?

The recent, gloomy book about our higher education system, Academically Adrift: Limited Learning on College Campuses, has sparked much debate. So has a documentary about high school students desperate to get into good colleges, Race to Nowhere. Put the two together and it starts to look like young people are stressing out and over-achieving in high school to get into the right universities, then not learning much when they get there. Is that true, and if so, how did we get here?

At first, what struck me about the book and the movie was how differently they see the situation. Academically Adrift, a rigorous sociological study written by Richard Arum of NYU and Josipa Roksa of the University of Virginia, assesses the “value added” by undergraduate education in America — in other words, learning outcomes. The story is not a happy one. Based on results from the Collegiate Learning Assessment (CLA), a standardized national instrument designed to measure critical thinking, the authors find that many of our college students have experienced little in the way of measurable learning during their undergrad years. To be specific, almost half of students (45%) showed no significant improvement on the CLA. Not exactly a passing grade for American higher education.

(Whether the CLA actually measures this accurately is a question unto itself. For the moment, let’s take it at face value and assume it does a decent job of assessing critical thinking, analytical reasoning, problem solving, writing, and other skills we want our colleges to teach.)

Then there’s the Race to Nowhere phenomenon. The film, co-directed by Vicki Abeles and Jessica Congdon, has garnered attention through grassroots distribution: screenings at schools followed by discussions among parents and teachers. That’s how I saw it recently at the school one of my children attends. The film tells the stories of high-schoolers pushed to the brink, working extremely hard, anxious about getting into the best colleges, depressed, and eventually burned out.

These kids are many things, but they are not “academically adrift.” They’re learning plenty. But they’re also paying the price for those achievements in areas like health, happiness, and quality of life. They are driven simultaneously to academic success and emotional collapse. 

Now, these are highly motivated, high-achieving students; obviously they don’t represent the full range of high schoolers in America. But with more prospects applying to a greater number of highly selective colleges each year, those colleges are becoming even more selective, and the race is only heating up. Race to Nowhere is a sobering look at an important and growing part of the higher ed ecosystem.

So the book and the film tell very different stories, and the two seem to be in tension with each other. Take the issue of homework (as they call it in high school) and time spent studying (in college). The authors of Academically Adrift noted in a New York Times op-ed that the average college student spends only 12 to 13 hours per week studying, about half of what she spent in 1960. In contrast, the question in high school circles is how to stem the rising tide of homework and give students their lives back a little. Another recent Times article cited Race to Nowhere as one of the catalysts for new policies by school districts to limit the time that students spend on homework. In the film, and in the discussions sparked by it among parents, it’s clear that many grade school and high school students are spending hours every night on homework.

So is the problem too little studying or too much? How can students be spending more time studying in high school than they do in college? And what’s the relationship between the two? ...

Full Post »
Categories: Accountability, Assessment, College admissions & marketing, Higher ed, Learning, Student research
Comments (1)  ::  Share This


March 21, 2011

Nastygram from the NY Times on visitor research

Maybe the Times arts critics have it in for the Brooklyn Museum. Or maybe they just don’t believe museum curators should get to know the audiences they’re creating exhibitions for. Then again, some museums don’t believe that either, which is why “front end” evaluation is often a botched job.

So I tried not to get defensive when I read this paragraph in art critic Ken Johnson’s review of Brooklyn’s new show on Plains Indian tipis.

Beyond some basic historical context, the exhibition offers no revelatory perspective on its contents. That might be partly because, as the organizers, Nancy B. Rosoff and Susan Kennedy Zeller (both Brooklyn Museum curators) point out in their catalog preface, part of the planning process involved focus groups and visitor surveys “to determine the level of visitor interest in and knowledge of the tepee and Plains culture.” They also invited a team of American Indian scholars, artists and tribal members to vet their plans. The result is an exhibition that speaks down to its audience, assuming a low level of sophistication, and that does as little as possible to offend or stir controversy.

On one level, this is the familiar highbrow take on visitor studies: If you ask the public what they want from an arts or culture experience, you’re doomed from the get-go. Focus groups yield lowest-common-denominator thinking, which should have no place in planning encounters with the great or challenging or profound. The museum should exercise its cultural authority and decide what visitors need to see and learn, without getting sidetracked by what they want.

But when you gather museum-goers in a focus group or ask them questions on a survey, do they really tell you, “I want this exhibition to talk down to me. I want the interpretation of objects to be bland and inoffensive”?

Of course not. The real issue here is what kinds of questions the museum asks and how it understands — and makes use of — the answers. I hasten to add that I haven’t seen the exhibition yet, and I may not agree with Johnson’s that it is condescending or bland. (From what I’ve been able to see online, it looks promising.) ...

Full Post »
Categories: Accountability, History museums, Museums, Research issues, Visitor experience, Visual art
Comments (3)  ::  Share This


January 19, 2011

In the race to test and assess, let's take the time to get it right

This is my first blog post as a research fellow at Slover Linett. Bill invited me to comment on Academically Adrift: Limited Learning on College Campuses, a book that was published this week and has quickly sparked debate. For those of you who don't know me, I work in assessment at Northwestern University and am the principal investigator of the Teagle Assessment Project and other national studies of student outcomes. But enough about me...

My first-grader took an online standardized math assessment for the first time last month. “It was so easy, Mom,” he told me, “Anne and I raced to see who would get done first.” “I can believe that,” said his teacher when she looked at his score during our parent-teacher conference. He had missed some easy questions in his haste, so the test didn’t even offer him the subsequent hard questions, which the teacher knew he was capable of answering correctly. In the world of high-stakes testing, there were no stakes as far as my son was concerned. And the testing instrument didn’t seem to do a good job of measuring what he knew.

This week, a new book based on undergraduates’ scores on an online standardized assessment is receiving a lot of attention in the higher ed world (see The Chronicle of Higher EducationInside Higher Ed, and the NY Times). The sound-byte is that students aren’t learning very much in college. And everyone is to blame: faculty who don’t take teaching seriously enough, presidents and boards and a tenure system that doesn’t reward teaching, colleges that don’t provide challenging curricula, students who prefer partying over studying. The list goes on.

But I have to stop and ask, how good is our ability to measure what students are learning in college? And how motivated are students to show us what they’ve learned when we ask them to do so using a standardized test? ...

Full Post »
Categories: Accountability, Assessment, Higher ed, Learning, Student research
Comments (1)  ::  Share This


November 08, 2010

A stolen painting shines a flashlight on the “public” domain of art

Did you hear about the museum in Sweden that didn’t realize three of its paintings were missing…until they were found by police raiding an apartment? Thank goodness for those “Malmo Art Museum” labels on the back of the canvasses. The story should remind us not to get too self-righteous about museums selling art to private collections, where (heaven forbid) they might not be available for public viewing.

Apparently the Malmo Art Museum in Sweden was unaware that a $1.5 million work by Edvard Munch and two other paintings in its collection had been stolen, until police discovered them in a raid. It’s a cringe-laugh-cry tale for anyone who cares about art and museums, with the laughter permissible only because it has a happy ending.

(By the way, the stolen Munch was "Two Friends," which I couldn't find an image of, not "The Scream." But the latter seems apt.)

For me, the story also shines light on the current debates about deaccessioning. (For those of you outside the visual art orbit, that’s the process by which museums sell, trade, or otherwise get rid of objects in their collections.) The furor over museums and universities like the National Academy Museum, Brandeis's Rose Art Museum, and Fisk University (pictured) trying to sell artworks from their collections to put themselves on a firmer financial footing — and the hawkish response from professional and accrediting bodies like AAM and AAMD — is understandable. But it hinges on several arguments about the harm that such sales would do to those institutions’ credibility and to the “public trust.” And the story of Malmo’s Munch suggests that one of those arguments doesn’t (pardon the pun) hold up.

Full Post »
Categories: Accountability, Engagement, Museums
Comments (4)  ::  Share This


July 12, 2010

Survey “coaching,” accountability, and dollars: a lesson from healthcare

“It only takes a second to fill out,” the x-ray technician told me cheerfully after an MRI I had yesterday. He was explaining that I would soon receive a survey in the mail asking about the service he provided, and he mimed checking off the boxes: “You just go down the list, five, five, five, five…”

Five, as you may have guessed, is the top satisfaction score.

Now, this was a community hospital affiliated with the University of Chicago Medical Center (which is a client of ours). But it’s an example of how all kinds of educational and cultural nonprofits could be thinking about the relationship between customer feedback, staff performance, and the bottom line.

At first it rubbed me the wrong way. My colleagues and I pride ourselves on being rigorous researchers, and we’ve criticized (here and here) survey processes that are less than scientific and objective. The whole point of social and market research is to get a true picture of how people think, feel, and act. You’re not allowed to coach them to give you high marks; you’re not supposed to influence them in any way.

But there was something else going on here, and it made me look more deeply at the role this kind of satisfaction research plays.

My tech’s name was Leo, which he wrote on the card he gave me so I would be sure to put it on the survey. Unprompted by any questions from me, he explained that the survey was a big part of the culture at the hospital. “We strive for five” is a staff mantra. At weekly meetings in each department, workers who received good survey ratings or comments are recognized. This presumably factors into their promotion and salary trajectories.

He even told me that the insurance companies link their reimbursement amounts to those patient satisfaction scores. I don’t know whether this is true or how much of the hospital’s revenue might be at stake in the formula. But what’s important is that Leo and his colleagues see the financial performance of the institution as dependent on the quality of the experiences it provides to individuals like me. ...

Full Post »
Categories: Accountability, Customer satisfaction, Higher ed, Museums, Performing arts, Survey research
Comments (5)  ::  Share This


About Us

We’re a Chicago firm that helps museums, arts organizations, and universities take a fresh look at their audiences and discover new ways to deepen the connection and broaden participation. More »

About this Blog

Asking Audiences explores the fast-changing landscape in which cultural and educational organizations meet their publics. What does relevance look like today? More »



Subscribe via RSS
Subscribe via Email

Blogs we love

Higher Ed
Blog U (Inside Higher Ed)
Chronicle of Higher Education
The College Puzzle (Michael Kirst)
Educated Nation
mStoner (Michael Stoner et al.)

Arts & Culture
Artful Manager (Andrew Taylor)
Createquity (Ian David Moss)
CultureGrrl (Lee Rosenbaum)
Jumper (Diane Ragsdale)
Life’s A Pitch (Amanda Ameer)
NAMP Radio (monthly podcasts)
Real Clear Arts (Judith Dobrzynski)

Museums
Future of Museums (Elizabeth Merritt)
ExhibitFiles
ExhibitTricks (Paul Orselli)
Expose Your Museum (Kathleen Tinworth)
Museum 2.0 (Nina Simon)
Museum 3.0
The Uncataloged Museum (Linda Norris)

Performing Arts
About Last Night (Terry Teachout)
Sandow (Greg Sandow)
Theater Loop