Top schools don't always get top marks

 

University rankings are not the most reliable indicator of an institution's educational clout

 
 
 
 
Students should look to the  institution that best fits their needs.
 

Students should look to the institution that best fits their needs.

Photograph by: Candace Elliott, The Journal, File, Freelance

The Journal's editorial on university rankings was an excellent commentary, pointing out the significant variability from one ranking to the next and the dangers of relying on them to assess a university's quality.

The editorial invited the University of Alberta to rank the rankers and we could not resist.

University rankings, like other rankings of cities, companies and even countries, are here to stay.

They provide the public with information purported to assess quality -- information that on the surface seems to help people cut through the "information overload" that characterizes our times.

Indeed, rankings can be a useful source of information as higher education becomes globalized and students around the world seek a high-quality education. The rankings compare universities on a variety of indicators, attempting to hold them accountable to global publics.

This is a worthy exercise, timely in an era of accountability, when considerable expenditure of public and private dollars is involved. However, one must remember that these third-party organizations are profit-driven; their rankings are designed to sell publications or online advertising.

The overall aim of these rankings is to identify the highest quality universities in the world.

For many of us, a ranking system is not required to create a reliable short list of the top 20 universities.

Students from all over the world are beating a path to their doors, where they receive an education that transforms them into enlightened citizens, honing their talents, releasing their potential. Faculty at the top universities are making groundbreaking discoveries that benefit society, and employers are lining up to hire their graduates.

The U of A has set itself a lofty goal of being among the top 20 public universities in the world.

But our effort is based on benchmarking against top public universities on a series of measures, not on rankings. We seek to be thought of on that short list. Details of our approach are on the U of A's website under "Dare to Discover -- Report Card."

So what is the role and quality of third-party rankings? They may have value because they are independent and at arm's length. One should question whether a single indicator can accurately measure the quality of a complex institution such as a university.

However in the spirit of ranking the rankings, we proceed. There are currently three major international rankings that receive widespread commentary: The Academic World Ranking of Universities, the QS World University Rankings and the Times Higher Education Rankings.

These all identify the top 20 universities reasonably well. These rankings gather data from universities for a variety of measures and turn them into aggregate averages to arrive at an overall score for each institution.

- The Academic Ranking of World Universities is the most objective of the three, relying solely on research indicators. The ranking is heavily weighted toward institutions whose faculty or alumni have won Nobel Prizes or other top awards, and relies on counting how often a faculty member's published research is referred to by others in prestigious journal articles (citations). This ranking does not measure the quality of teaching or the quality of humanities scholarship.

- The QS World University Rankings, in contrast, mix subjective and objective measures. In addition to measuring faculty publication citations, QS conducts a survey of reputation among academics and employers. Student/faculty ratio is used as a proxy for teaching quality and internationalization is measured by faculty and student origin. A criticism of these rankings is that the use of student/faculty ratio as the sole measure for teaching quality and the use of opinion surveys to measure the quality of research and graduates are, at best, blunt instruments.

- This year, Times Higher Education partnered with Thomson Reuters to compile a new ranking, which was released a few days ago. This ranking is also based on objective and subjective measures. A school's final score is based on an aggregate of teaching, research reputation/income, citations/ research influence, industry income and international mix. Academics are surveyed to rate teaching and research quality, a precarious exercise since teaching is difficult, if not impossible, for academics to assess at institutions other than their own. At first blush, this ranking appears to capture a broader range of indicators and does a good job of identifying the top 20. We applaud the University of Toronto for being named among the top 20; this is very good for Canada and Canadian universities.

However, when one looks under the hood, beyond the top 20 universities, the new ranking has some very peculiar outcomes that do not pass the "reasonableness test." Alexandria University in Egypt received one of the highest scores for citations while also receiving the lowest score for research reputation. By comparison, No. 1 Harvard received near perfect scores on both measures. Logic would suggest that the aggregate score for research reputation should correlate with the score for citations, which measure research influence.

However, in spite of the vast discrepancy between its scores, Alexandria, which has never been on any other international ranking, came in ahead of many top universities. Furthermore, many universities considered by peers as among the best do not appear in the top 200.

Something is awry.

So how do we rank the rankings? We assign an A for effort and C to D for all of them for overall effectiveness in measuring the quality of higher education institutions.

Universities are among the most complex and important organizations in the knowledge age, so this is no easy task, and no single indicator can measure overall quality.

To students, who are trying to choose a university, our advice is to determine the institution that best fits your needs -- it might be a small, high-quality undergraduate institution or an internationally recognized teaching-research university such as the University of Alberta.

To the public: Take rankings with a truckload of salt. To the rankers: Keep on improving the rankings!

Indira Samarasekera is president and vice-chancellor of the University of Alberta. Carl Amrhein is the university's provost and vice-president academic.

- - -

RANKING INSTITUTIONS

- The Academic Ranking of World Universities was established and first published by two institutes in the Graduate School of Education at Jiao Tong University, Shanghai, China. It is now conducted by Shanghai Ranking Consultancy in Shanghai. The 2010 edition can be found at www.arwu.org/ARWU2010.jsp

- The QS World University Rankings are prepared by QS, an London-based organization that describes itself as the world's leading network for top careers and education. Its rankings are at www.topuniversities.com

- The Times Higher Education rankings are commissioned by Times Higher Education magazine. The data are collected and analyzed by research data firm Thomson Reuters. Its rankings are at www.timeshighereducation.co.uk/world-university-rankings

- The University of Alberta's "Dare to Discover -Report Card" can be found at www.president.ualberta.ca//pdfs/Dare%20to%20Discover%20 Report%2008-09%20FINAL.pdf

 
 
 
 
 
 
 

Story Tools

 
 
Font:
 
Image:
 
 
 
 
 
Students should look to the  institution that best fits their needs.
 

Students should look to the institution that best fits their needs.

Photograph by: Candace Elliott, The Journal, File, Freelance

 
 
 
 
 
 
 

More Photo Galleries

Smoky Lake Pumpkin Fair

Gallery: Smoky Lake pumpkin fair...

Photos from the 22nd annual Great White North Pumpkin...

 
NAIT's 36th Annual Bridge Building Competition

Gallery: NAIT bridge building

Photos from NAIT's 36th Annual Bridge Building Competition...

 
Polar Impact exhibit

Gallery: Polar Impact

Take a virtual tour of the new "Polar Impact" exhibit...

 
 
 
 
 
 
 

Most Popular News

 
 
 
 
 

Breaking News Alert

 
Sign up to receive e-mail alerts on breaking news from The Edmonton Journal.