The rap on research for the arts, museums, and education

December 22, 2011

Diversity: it's not just an admissions issue anymore

The Obama administration’s new call for universities to increase diversity on campus is probably a welcome one for many schools. After years of court battles over state university admissions policies, centered on the University of Michigan, colleges and universities now have greater clarity about which levers they’re allowed to pull to attract a more ethnically diverse pool of applicants. But what happens when those more diverse classes get to campus?

Much research has been done on the benefits of being in school with a more diverse group of peers. For example, a recent study discussed in the book How College Affects Students notes that exposure to fellow students of diverse backgrounds is one of the key factors influencing whether freshmen return for their sophomore year and whether their experience improves their critical thinking skills. Think about that: more diverse classmates and dorm-mates leads to a more positive, successful undergrad experience.

And our own research for highly selective universities and grad schools has shown that prospects value diversity and take it into consideration when deciding which schools to apply to. When it comes to attracting underrepresented minorities, it certainly helps when they see people who look like them on campus, ideally both students and professors.

That research is echoed in the new guidelines, which were issued jointly by attorney general Eric Holder and education secretary Arne Duncan. "Diverse learning environments," says Holder in the accompanying press release, “promote development of analytical skills, dismantle stereotypes, and prepare students to succeed in an increasingly interconnected world.”

All true. I’m as strong a believer as anyone that greater diversity along all kinds of dimensions — racial, gender, socio-economic, geographic, attitudinal — in higher education is a good thing. What schools need to realize, though, is that with that greater diversity comes a greater need for support for those new members of the student body. Some of our research has shown that students from lower socioeconomic backgrounds, as well as those who are the first generation in their family to attend college, struggle with the demands and format of postsecondary education more than other students do. They’re more likely to have jobs, work more hours, and be less involved in co-curricular activities. In one study, we found that they needed more help from their advisors that other students did — but were less likely to seek out that help. These diverse newcomers can benefit from greater support, whether academic, co-curricular or social, to help them navigate the unfamiliar landscape.

Yet many schools don’t invest in such support programs, in part because they are not aware of those needs and in part because they don’t want to stereotype their new arrivals or treat them differently on the basis of ethnicity. The ideal is a melting pot, whether or not it actually exists.

Of course, those support programs are also an additional expense for the schools. So that well-intentioned reluctance to engage in anything like profiling may also mask a reluctance to spend more on student-life and academic counseling.

Whatever its causes, that reluctance is a shame. If a school embraces increased diversity as a strategic goal, it ought to carry that strategy through to the  students it affects, by acknowledging and meeting their unique needs. It’s also a good long-term decision: supporting underrepresented minorities and first-generation students would likely contribute to higher retention and graduation rates, and that means stronger rankings and better applicant pools. It may be an additional expense, but it makes both ethical and academic sense.

Does your institution support diversity just at the admissions stage, or throughout the student experience?
 


Categories: College admissions & marketing, Higher ed, Student research
Comment  ::  Share This


June 24, 2011

Can we be ‘academically adrift’ while ‘racing to nowhere’?

The recent, gloomy book about our higher education system, Academically Adrift: Limited Learning on College Campuses, has sparked much debate. So has a documentary about high school students desperate to get into good colleges, Race to Nowhere. Put the two together and it starts to look like young people are stressing out and over-achieving in high school to get into the right universities, then not learning much when they get there. Is that true, and if so, how did we get here?

At first, what struck me about the book and the movie was how differently they see the situation. Academically Adrift, a rigorous sociological study written by Richard Arum of NYU and Josipa Roksa of the University of Virginia, assesses the “value added” by undergraduate education in America — in other words, learning outcomes. The story is not a happy one. Based on results from the Collegiate Learning Assessment (CLA), a standardized national instrument designed to measure critical thinking, the authors find that many of our college students have experienced little in the way of measurable learning during their undergrad years. To be specific, almost half of students (45%) showed no significant improvement on the CLA. Not exactly a passing grade for American higher education.

(Whether the CLA actually measures this accurately is a question unto itself. For the moment, let’s take it at face value and assume it does a decent job of assessing critical thinking, analytical reasoning, problem solving, writing, and other skills we want our colleges to teach.)

Then there’s the Race to Nowhere phenomenon. The film, co-directed by Vicki Abeles and Jessica Congdon, has garnered attention through grassroots distribution: screenings at schools followed by discussions among parents and teachers. That’s how I saw it recently at the school one of my children attends. The film tells the stories of high-schoolers pushed to the brink, working extremely hard, anxious about getting into the best colleges, depressed, and eventually burned out.

These kids are many things, but they are not “academically adrift.” They’re learning plenty. But they’re also paying the price for those achievements in areas like health, happiness, and quality of life. They are driven simultaneously to academic success and emotional collapse. 

Now, these are highly motivated, high-achieving students; obviously they don’t represent the full range of high schoolers in America. But with more prospects applying to a greater number of highly selective colleges each year, those colleges are becoming even more selective, and the race is only heating up. Race to Nowhere is a sobering look at an important and growing part of the higher ed ecosystem.

So the book and the film tell very different stories, and the two seem to be in tension with each other. Take the issue of homework (as they call it in high school) and time spent studying (in college). The authors of Academically Adrift noted in a New York Times op-ed that the average college student spends only 12 to 13 hours per week studying, about half of what she spent in 1960. In contrast, the question in high school circles is how to stem the rising tide of homework and give students their lives back a little. Another recent Times article cited Race to Nowhere as one of the catalysts for new policies by school districts to limit the time that students spend on homework. In the film, and in the discussions sparked by it among parents, it’s clear that many grade school and high school students are spending hours every night on homework.

So is the problem too little studying or too much? How can students be spending more time studying in high school than they do in college? And what’s the relationship between the two? ...

Full Post »
Categories: Accountability, Assessment, College admissions & marketing, Higher ed, Learning, Student research
Comments (1)  ::  Share This


February 28, 2011

An admissions admission: We’re not sure what SAT scores measure

The debate over standardized test scores in college admissions is heating up in our own backyard. A few weeks ago, DePaul University announced it will no longer require undergraduate applicants to submit standardized test scores. Why not? Because the admissions office found that the scores added little to their ability to predict applicants’ academic success in college. And DePaul isn’t alone.

What does help predict academic performance, not surprisingly, are the applicants’ high school grades. The past is prologue, as Shakespeare put it. So DePaul has joined a growing group of colleges and universities, now in the hundreds, that have recognized the limits of standardized tests in the admissions process (see Fairtest.org for a list of schools that have made them optional or no longer consider them at all).

But, as others have argued, high school grades are not standardized: they differ from teacher to teacher and school to school, and they may be influenced by all kinds of cultural and environmental factors other than the student’s raw potential. So they’re not an ideal basis for comparison, especially when it comes to large applicant pools from different geographies and diverse circumstances.

Isn’t it better to rely on “standardized” tests, which are the same everywhere and for everyone? Aren’t SAT scores more, not less necessary to differentiate applicants in an era when grade inflation is making high school transcripts look increasingly alike? We live in a world that loves the single score, index, or metric that boils it all down for us. Standardized test scores lend some objectivity to the otherwise messy, subjective process of college admissions.

The trouble is, it’s not clear what tests like the SATs actually measure. Intelligence? Most psychologists would say they don’t measure innate intelligence or aptitude, or even academic potential; they measure developed abilities, and only certain abilities at that. Of course, the development of those abilities in any given individual is influenced by a host of environmental factors, including the hugely influential factors of race and class. ...

Full Post »
Categories: College admissions & marketing, Diversity, Higher ed
Comments (1)  ::  Share This


January 19, 2011

In the race to test and assess, let's take the time to get it right

This is my first blog post as a research fellow at Slover Linett. Bill invited me to comment on Academically Adrift: Limited Learning on College Campuses, a book that was published this week and has quickly sparked debate. For those of you who don't know me, I work in assessment at Northwestern University and am the principal investigator of the Teagle Assessment Project and other national studies of student outcomes. But enough about me...

My first-grader took an online standardized math assessment for the first time last month. “It was so easy, Mom,” he told me, “Anne and I raced to see who would get done first.” “I can believe that,” said his teacher when she looked at his score during our parent-teacher conference. He had missed some easy questions in his haste, so the test didn’t even offer him the subsequent hard questions, which the teacher knew he was capable of answering correctly. In the world of high-stakes testing, there were no stakes as far as my son was concerned. And the testing instrument didn’t seem to do a good job of measuring what he knew.

This week, a new book based on undergraduates’ scores on an online standardized assessment is receiving a lot of attention in the higher ed world (see The Chronicle of Higher EducationInside Higher Ed, and the NY Times). The sound-byte is that students aren’t learning very much in college. And everyone is to blame: faculty who don’t take teaching seriously enough, presidents and boards and a tenure system that doesn’t reward teaching, colleges that don’t provide challenging curricula, students who prefer partying over studying. The list goes on.

But I have to stop and ask, how good is our ability to measure what students are learning in college? And how motivated are students to show us what they’ve learned when we ask them to do so using a standardized test? ...

Full Post »
Categories: Accountability, Assessment, Higher ed, Learning, Student research
Comments (1)  ::  Share This


January 06, 2011

The happiness curve and alumni engagement

In the airport with my family over the holidays, I ran into a newsstand to pick up some things for our flight. There on the rack, the cover of The Economist caught my eye: “The joy of growing old (or why life begins at 46).” Being 46 at the moment, of course I bought it.

I thought the article might feature tips relevant to us mid-lifers, like how to prevent your knees from hurting when you hike, or how to remember what you walked into the kitchen for. What I found was something a little bigger-picture, something surprisingly related to my work with colleges and universities.

The not-particularly-happy news about happiness, according to the studies cited in the article, is that, in cultures around the world, people tend to start out happy and get increasingly less so, until they hit a collective rock bottom at age 46. Ouch.

The good news is that, as people continue to age, they report feeling happier and happier until their mid-80s, when (on average) they’re happier than they’ve ever been.

As I stared at the U-shaped chart, pondering all the ways I might feel happier in the future, I realized that I’ve seen this same graph before in our research with alumni.

Alumni tend to stay involved with their school for the first few years after graduation. But then engagement drops, bottoming out with alums in their 40s. It then begins to rise, with greater engagement and positive feelings as they age into retirement.

Common sense and our own research suggest that alums in their 40s are the most time-starved cohort, with jobs, kids, houses, and 401Ks taking up time and energy. So they have less time to devote to life’s optional commitments, like alumni events and class reunions. The Economist article made me wonder how happiness factors into the equation. If you don’t feel particularly good about your life, are you less likely to want to connect with your school and fellow alums? ...

Full Post »
Categories: Demographics, Engagement, Fundraising, Higher ed, Research findings
Comments (5)  ::  Share This


September 22, 2010

School is back in session: Our fall semester at “SLU”

On Fridays at noon, our office suspends the hustle and bustle of meetings, report writing, and phone calls and becomes a seminar room. We gather around our conference table to eat lunch and turn our attention to a topic, trend, or technique important to our work. At some point these lunches were dubbed (somewhat ironically) Slover Linett University, which was inevitably shortened to SLU.

These seminars run the gamut from skill-building workshops about, say, linear regression or ethnographic observation, to broader knowledge-building discussions about newly-released studies or articles, our own recent research findings, or a conference one of us just attended.

I learned about SLU during the interview process before I began working here, and I remember thinking how great it is for a workplace to provide a structure for employees to stay up-to-date on issues relevant to their field. I’ve only been here a few months, but I’ve already learned so much from just a handful of SLU sessions.

With eleven people here in the office, we’re still small enough that communication is pretty easy. But with our three senior associates specializing in different sectors — higher ed, performing arts, and museums — it’s harder than it looks to keep everyone up-to-date about what we’re learning. The weekly SLU lunches keep us all in the loop across all of our projects, so that important findings, ideas, and lessons learned stay fresh in our collective mind.
 

SLU was on vacation over the summer, as vacations and work-related travel took many of us out of the office. But now that it’s Fall, we’re beginning the new “academic year.” Here are a few of the topics that will be on the table (along with our lunches and notebooks): ... 
 

Full Post »
Categories: General, Higher ed, Museums, Performing arts, Slover Linett events
Comment  ::  Share This


August 06, 2010

Are your Twitter followers also your Facebook fans? Maybe not

NPR has been no slouch when it comes to social networking and now has more than a million Facebook fans. It recently surveyed that population, and the findings raise a question that has me and my colleagues wondering: how segmented is the social media audience by platform?

Social networking is on our minds quite a bit these days at Slover Linett, and not just in my own domain, higher ed. As many of you know, our collaborative study with CASE and mStoner looked at how colleges and universities are using social media for fundraising and alumni relations. Cheryl presented initial findings in New York and the research was written up in the Huffington Post, Inside Higher Ed, and elsewhere. And we’re conducting several surveys to help performing arts organizations understand how their audiences are using interactive technologies to connect with the institution and engage with the art form.

So the new NPR study caught my eye, and based on the 9 slides that their in-house research group has shared publicly, it looks like wonderful work. Some of the findings were a little surprising to me (which I always enjoy — surprises in research reports are like plot twists in a mystery novel). Perhaps the most interesting surprise was the apparent lack of overlap between NPR’s Facebook fans and its Twitter followers.

Only about 8% of the Facebook fans who participated in the survey say they also follow NPR on Twitter. Yet NPR has well over two million Twitter followers (the numbers are complicated because different departments and on-air personalities have their own Twitter accounts; NPR Politics has 1.8 million followers, and Weekend Edition host Scott Simon has 1.3 million).

So clearly there are plenty of NPR Facebook fans who aren’t following the organization on Twitter, and vice versa. NPR researchers Andy Carvin and Noel Cody, who posted the results I just linked to in NPR's research blog, were also surprised by this and note that the two communities appear to be more mutually exclusive than they thought. ...

Full Post »
Categories: College admissions & marketing, Higher ed, Museums, Other nonprofits, Performing arts, Research findings, Social media, Survey research
Comment  ::  Share This


July 23, 2010

Sorting out the social media buzz in higher ed

Our recent study on social media usage by colleges and universities has been generating conversation, predictably enough given the hot topic. But what’s the real revolution that’s occurring? It has to do with who’s communicating with whom.

Our study for the Council on Advancement and Support of Education (CASE), which we undertook in partnership with mStoner, was designed to give university development, alumni relations, and marketing professionals insights into what their peers are doing with social media, and why. We found that the biggest reason schools are investing time and money in social media is because their alumni (and other constituents) want them to. It’s expected, and it’s how an increasing proportion of alumni want to communicate with their alma mater.

No surprise there. And another recent piece of research amplifies this point. The U.S. News commissioned the Engagement Strategies Group to survey alumni around the country and learned that 47% of young alumni don’t feel their colleges do enough to connect with them, other than soliciting donations. So alumni want their schools to take a more active role in forging those relationships. That doesn’t mean more communication, of course. It means better communication.

Which brings us right back to social media, because Facebook, Twitter, and other network-driven communication platforms aren’t just new channels to carry the university’s message to alumni (or prospective or current students). They’re not even merely two-way streets: they’re whole neighborhoods of “streets,” new communities of information and exchange. 

A slide from our presentation at the CASE Summit for Advancement Leaders, 2010. ...  

Full Post »
Categories: Higher ed, Social media
Comment  ::  Share This


July 12, 2010

Survey “coaching,” accountability, and dollars: a lesson from healthcare

“It only takes a second to fill out,” the x-ray technician told me cheerfully after an MRI I had yesterday. He was explaining that I would soon receive a survey in the mail asking about the service he provided, and he mimed checking off the boxes: “You just go down the list, five, five, five, five…”

Five, as you may have guessed, is the top satisfaction score.

Now, this was a community hospital affiliated with the University of Chicago Medical Center (which is a client of ours). But it’s an example of how all kinds of educational and cultural nonprofits could be thinking about the relationship between customer feedback, staff performance, and the bottom line.

At first it rubbed me the wrong way. My colleagues and I pride ourselves on being rigorous researchers, and we’ve criticized (here and here) survey processes that are less than scientific and objective. The whole point of social and market research is to get a true picture of how people think, feel, and act. You’re not allowed to coach them to give you high marks; you’re not supposed to influence them in any way.

But there was something else going on here, and it made me look more deeply at the role this kind of satisfaction research plays.

My tech’s name was Leo, which he wrote on the card he gave me so I would be sure to put it on the survey. Unprompted by any questions from me, he explained that the survey was a big part of the culture at the hospital. “We strive for five” is a staff mantra. At weekly meetings in each department, workers who received good survey ratings or comments are recognized. This presumably factors into their promotion and salary trajectories.

He even told me that the insurance companies link their reimbursement amounts to those patient satisfaction scores. I don’t know whether this is true or how much of the hospital’s revenue might be at stake in the formula. But what’s important is that Leo and his colleagues see the financial performance of the institution as dependent on the quality of the experiences it provides to individuals like me. ...

Full Post »
Categories: Accountability, Customer satisfaction, Higher ed, Museums, Performing arts, Survey research
Comments (5)  ::  Share This


April 09, 2010

“Majority minority” and what it doesn’t tell us about the future of cultural attendance

Much is being made of the fact that, at some point 30 or 40 years from now, “non-Hispanic whites” will become America’s largest minority. But what will that mean for arts participation and museumgoing? In one sense, nothing at all.

A book review in this week’s New Yorker by Kelefa Sanneh, the magazine’s pop music critic, calls our attention to “Stuff White People Like,” that good-natured piece of social self-criticism in blog and book form by Christian Lander. The list of “stuff” reads like my firm’s client roster: film festivals (#3), non-profit organizations (#12), plays (#43), arts degrees (#47), graduate school (#81), public radio (#44), and of course classical music — or rather, “Appearing to Enjoy Classical Music” (#108). Jazz is also here, I think, under the heading, “Black Music that Black People Don’t Listen to Anymore” (#116).

Ouch.

Combine Lander’s jokey-but-perceptive point with the demographic shifts that will soon mark the end of white hegemony in the United States, and it may look like all of us — you arts and education professionals, and we consultants who help you — are in the wrong business. White, urban, liberal culture and the values associated with it have seen their heyday and are on the way out.
 
But Sanneh’s essay goes on to complicate that picture, if not undermine it altogether, by pointing out that the category of American whiteness is itself a moving target. Over the decades it has come to include “many previous identities that had once been considered marginal: Irish, Italian, Polish, Jewish.”

At one time, those ethnic minorities were visibly, audibly, even behaviorally other. Yet today, if you wanted to know whether someone is of Irish or Italian heritage, or is Jewish, you’d have to ask.

What changed over that period, the minority or the culture at large? Both. What it meant to be “Italian” or “Jewish” changed, and simultaneously what it meant to be “American” changed. And of course the two processes influenced each other.

Full Post »
Categories: Arts participation, Culture sector, Demographics, Higher ed, Metrics, Museums, Performing arts, Survey research
Comments (4)  ::  Share This


About Us

We’re a Chicago firm that helps museums, arts organizations, and universities take a fresh look at their audiences and discover new ways to deepen the connection and broaden participation. More »

About this Blog

Asking Audiences explores the fast-changing landscape in which cultural and educational organizations meet their publics. What does relevance look like today? More »



Subscribe via RSS
Subscribe via Email

Blogs we love

Higher Ed
Blog U (Inside Higher Ed)
Chronicle of Higher Education
The College Puzzle (Michael Kirst)
Educated Nation
mStoner (Michael Stoner et al.)

Arts & Culture
Artful Manager (Andrew Taylor)
Createquity (Ian David Moss)
CultureGrrl (Lee Rosenbaum)
Jumper (Diane Ragsdale)
Life’s A Pitch (Amanda Ameer)
NAMP Radio (monthly podcasts)
Real Clear Arts (Judith Dobrzynski)

Museums
Future of Museums (Elizabeth Merritt)
ExhibitFiles
ExhibitTricks (Paul Orselli)
Expose Your Museum (Kathleen Tinworth)
Museum 2.0 (Nina Simon)
Museum 3.0
The Uncataloged Museum (Linda Norris)

Performing Arts
About Last Night (Terry Teachout)
Sandow (Greg Sandow)
Theater Loop