The 'U.S. News' Rankings' (Faux?) Embrace of Social Mobility

For years, critics of the college rankings of U.S. News & World Report have said that they reward prestige and wealth. The institutions that are always on the top of the rankings -- places like Harvard, Princeton and Stanford Universities -- enroll students who are destined to succeed, the critics say. It should be no surprise (and not worthy of praise) that the students then do well.

What about the institutions that enroll students not destined to do well -- those who grew up poor, who in many cases went to poorly resourced high schools and who lacked family members with the social capital to help find a good college or launch a career? It is the colleges that succeed with such students that really deserve praise, these critics say. And as a result, a number of efforts have started to look at colleges that promote "social mobility" -- in other words, that help boost those from disadvantaged families into the middle class.

The 2019 U.S. News rankings are out today, and the rankings powerhouse is boasting that it has changed its methodology to take social mobility into account. And indeed -- if you leave the top of the heap -- one can see colleges going up in the rankings, some jumping more than is the norm in any single year. But if you think this is the year that U.S. News will finally credit the California State University or City University of New York campuses for perhaps working harder and doing more than the elites do to help less wealthy, well, don't hold your breath.

Who is No. 1 in the new social mobility-influenced U.S. News rankings? Princeton University. Who is No. 2? Harvard University. That would be the same top two as last year, before the changes. And you'll find Ivies and similarly prestigious institutions continue to do quite well -- despite having a fraction of the disadvantaged students enrolled elsewhere.

To understand why it may be odd to see U.S. News boast about being influenced by the value of social mobility while ranking Princeton on top, consider the following statistics. Princeton has exceptionally generous policies for the low-income students it enrolls, and 15 percent of its students are eligible for Pell Grants, a common proxy for being low income.

But if you examine the data on Princeton that Raj Chetty of Harvard University prepared for The New York Times, you'll find that 72 percent of Princeton students come from the top 20 percent of family incomes in the United States. That includes 17 percent from the top 1 percent and 3.1 percent from the top 0.1 percent. Only 2.2 percent come from the bottom 20 percent of family income.

So how is it that the addition of social mobility factors hasn't dislodged the institutions on top of the rankings? The answer is in what U.S. News changed and what it didn't change.

The magazine took its "outcomes" measure and increased its weight from 30 to 35 percent of its formula. Part of that is graduation rates, and part of the formula credits colleges that "outperform" expected graduation rates based on their student demographics.

New this year in the outcomes section are two social mobility factors that together make up 5 percent of the total ranking. One looks at the graduation rates of Pell Grant recipients, and the other compares Pell-recipient graduation rates to those of all students. Both of those figures are then adjusted for the share of all students who are Pell recipients. So if two colleges have the same Pell graduation rates, but one has a larger share of Pell recipients, the second college would earn more points in the formula. U.S. News counts the graduation rate formula as also indicating social mobility and so says that 13 percent of its formula is now based on social mobility.

The magazine killed one part of its methodology -- the acceptance rate -- that has long been seen as rewarding colleges for the number of applicants they reject. But this was worth only 1.25 percent of the formula. U.S. News reduced the weight on, but kept in, standardized test scores at 7.75 percent, down from 8.125 percent. This factor has long been criticized by those who have noted that students from wealthier families earn, on average, higher scores on the SAT and ACT than do those who aren't wealthy.

U.S. News also decreased modestly (from 22.5 to 20 percent) "expert opinion," which is based on surveys of college administrators and high school counselors -- and has long been derided for rewarding colleges for having good reputations over time, largely a result of prestige and history.

Some factors that favor wealthy colleges over others are unchanged: 20 percent of the formula is based on "faculty resources" and 10 percent on spending on students.

For some institutions, the changes in the formula have resulted in large gains. The University of California, Riverside, is up 39 spots, to a three-way tie for 85th. Riverside is known for having high graduation rates (73 percent, well above the national median of 42 percent), while also having a student body that is diverse in all kinds of ways -- Riverside is a "majority minority" campus where 12 percent of students are white. Fifty-six percent of its students receive Pell Grants (five times the share at Harvard University).

Kim A. Wilcox, chancellor at Riverside, said he was pleased to see that U.S. News and others "are beginning to recognize diversity, social mobility and student success as hallmarks of what make a great university." Wilcox noted that Riverside graduates more Pell recipients each year than any other research university. He said he was proud of that distinction "regardless of the rankings."

But Wilcox added that he would like to see more shifts in the way colleges are evaluated. "It will take time to reverse decades of deference to traditional assumptions of institutional quality," he said.

Other college leaders say that the measures U.S. News is using -- even with low-income students -- distort more than they illuminate.

Consider Trinity Washington University, which U.S. News considers to be somewhere below 142nd among regional Northern universities, and so not an institution that receives an ordinal rank. More than 80 percent of students at Trinity are Pell eligible, and most are first-generation students.

Patricia McGuire, the president, says colleges should be judged by the share of Pell-eligible students they enroll, but that using traditional graduation rates makes no sense for any number of reasons. First, graduation rates are a lagging indicator, she noted. Institutions that are not among the elites with 90-plus percent rates may go up and down, and the most recent rate may have little to do with the experience prospective students might have.

More important, she said, institutions with large shares of disadvantaged students know that many students don't graduate within six years (the federal rate, and the one used in the rankings). A measure that works for the Ivies may not reflect the experience of those at Trinity or elsewhere, she said. At Trinity, the federal rate has fluctuated in recent years from 35 to 47 percent. But 60 percent of students are completing, she said, just on a longer time frame. Colleges like Trinity enroll more students with significant financial challenges, she said, and more students who still need help to prepare for college. That means some students will drop out, and others will come back.

Measures that may work to look at elite colleges for small shares of Pell students don't work at other institutions, she said. And the vast majority of low-income students will not enroll at elite colleges, she noted.

"U.S. News just does not understand the population of low-income students at all," McGuire said.

Even before this year's rankings were released, the chancellor of Winston-Salem State University published a letter saying that his institution no longer participates in the survey of presidents, and that it will not seek to boast about any improvements. Chancellor Elwood L. Robinson noted that he has no problem with metrics, and noted that Winston-Salem State does well on measures of improving social mobility of students. Further, he said that his university has the highest share of graduates in the University of North Carolina system who end up working in the state.

But he said that Winston-Salem State and other historically black institutions are punished by U.S. News for their mission. Access and affordability are top priorities, he said. But the U.S. News rankings "require focus in areas that are antithetical to our historical mission," he wrote.

Some of those who have pushed the idea of measures of social mobility include Harvard's Chetty, whose research has found that the University of Texas System, the State University of New York at Stony Brook and California State University at Los Angeles are particularly likely to propel their students into success. He has also noted the success of several CUNY colleges, all of which have much higher shares of Pell-eligible students than do those on the top of U.S. News rankings. The Washington Monthly also includes measures on the success of Pell students (although its top national universities have a lot of overlap with those of U.S. News).

CollegeNET has a different approach, publishing social mobility data that look at family income of students, tuition rates and early-career salaries, while ignoring factors such as institution reputation. CollegeNET also rejects using Pell eligibility as a factor, since a minority of families who are decidedly not poor qualify for them.

Jim Wolfston, president of CollegeNET, said via email that U.S. News was trying to use its methodology changes as "a fig leaf" to hide that its rankings still favor wealthy institutions and wealthy students. He noted the parts of the methodology that favor the elite institutions.

"The U.S. News rankings are not rankings of higher education," Wolfston said. "They are rankings of the perpetuation of economic privilege. We all know the research. The tightest direct correlation to higher SAT scores is family income. Schools with the highest incoming SAT scores are therefore 'better,' not because they deliver a powerful education delta to their students, but because they enroll more scions of the rich?"

He said U.S. News appears to have noted the interest of the public in social mobility while not making meaningful changes in the rankings to truly promote social mobility. He said it was like a soda company "announcing a slight reduction in sugar content" while still selling unhealthy beverages.

Robert Morse, who leads the college rankings program at U.S. News, said that doing more on social mobility has long been a goal of U.S. News. "We’ve wanted to measure whether schools are successful at serving all of their students for a long time, and this was the first year that it was possible to incorporate this information into our methodology because of the federal government," he said. "They mandated that schools report the Pell and non-Pell graduation rate data, which meant that the data can now be used to accurately compare schools."

Asked about how the traditional top-ranked institutions kept their positions (or very similar ones) even with the new emphasis on social mobility, Morse said that a number of universities are in fact doing better in the rankings this year because of the additional measures on social mobility. He cited the University of California, Los Angeles, tied for No. 19, as one such institution. It was tied for 21st last year.