Why we don’t need university rankings

12+ Submit an article

Received: 15.03.2025 Published: 15.03.2025

UDC: 378.4 EDN: ENKZEG

Abstract

The proliferation of global university rankings as a tool for research assessment demands critical examination, given the growing volume of methodological criticism and their potentially detrimental impact on academic systems. The aim of this study is to conduct a comprehensive analysis of the inherent problems of rankings and to evaluate current alternatives within the context of shaping science and technology policy. Based on a systematic review of English- and Russian- language academic and grey sources, persistent methodological shortcomings were identified: insufficient transparency of calculation algorithms, limited applicability of bibliometric data for assessing social sciences and humanities (SSH), subjectivity of weighting coefficients in composite indices, and an inability to adequately reflect the quality of the educational processes and managerial efficiency. It was established that rankings relying on a composite indices are fundamentally unsuitable for evaluating organizations due to inherent methodological limitations; rankings without such a composite indicator, based on transparent and reproducible methodology with open access to source data, can be used analytically provided there is strict consideration of the limitations inherent in quantitative metrics. The expediency of transitioning to mission-driven assessment models was substantiated. The significance of the study lies in developing a scientifically grounded position relevant to reforming research assessment systems in Russia and BRICS countries, and in providing arguments for integrating science of science approaches into the practice of science and technology forecasting and governance.

Keywords

Agencies

Исследование подготовлено в рамках темы 1024033100002-8 «Развитие методологических подходов к построению системы мониторинга и оценки результативности научно-технологической, образовательной и инновационной деятельности в Российской Федерации».

How to cite

Для цитирования:

Кочетков Д.М. Почему нам не нужны рейтинги университетов // Экономика науки. 2025. Т. 11, № 3. С. 97–109.

For citation:

Kochetkov D.M. Why we don’t need university rankings // Economics of Science. 2025. Vol. 11, no. 3. P. 97–109.

Коды статьи

  • УДК 378.4
  • EDN ENKZEG
  • Тип статьи (РИНЦ) Научная статья (Научная статья)

How to Cite

Kochetkov D.M. Why we don’t need university rankings // Economics of Science. 2025. Vol. 11, no. 3. P. 97–109.

References

  1. Арефьев, А.Л. (2014). Глобальные университетские рейтинги как новое явление в российской высшей школе. Социологическая наука и социальная практика, 3(07), 5–24.
  2. Арефьев, А.Л. (2015). Об участии российских вузов в международных рейтингах. Россия реформирующаяся, 13, 213–231. EDN: TOCAEP
  3. Большеротов, А. Л. (2013). Мировой рейтинг университетов: Догнать и перегнать. А нужно ли? Часть I. Рейтинги университетов мира. Жилищное Строительство, 4, 17–23. EDN: QIABWH
  4. Бюллетень Счетной палаты. (2021). 2(279). https://ach.gov.ru/statements/byulleten-schetnoy-palaty-2-279-2021-g (дата обращения: 31.07.2025).
  5. Гузикова, Л.А., & Плотникова, Е.В. (2014). Позиции и перспективы участников Проекта 5-100-2020 в международных рейтингах университетов. Вопросы методики преподавания в вузе: ежегодный сборник, 3(17), 15–27. EDN: TBHBSX
  6. Кочетков, Д. (2024, ноябрь 12). Нужен ли нам суверенный рейтинг университетов БРИКС. Независимая Газета. http://www.ng.ru/nauka/2024-11-12/15_9133_raiting.html (дата обращения: 31.07.2025).
  7. Кочетков, Д.М. (2025). Об истории, предмете и задачах российского науковедения. Социология науки и технологий, 16(1), 91–122. EDN: SSZEDP, https://doi.org/10.24412/2079-0910-2025-1-91-122
  8. Кушнева, О.А., Рудская, И.А., & Ферсман, Н.Г. (2014). Мировой рейтинг университетов и программа «5100-2020» Министерства образования и науки РФ как путь повышения конкурентоспособности российских университетов. Общество. Среда. Развитие, 2(31), 17–26. EDN: SGTXKN
  9. Лазар, М.Г. (2019). Последствия увлечения количественными показателями в науке и высшем образовании. Ученые Записки Российского Государственного Гидрометеорологического Университета, 54, 134–144. EDN: KQSSSJ, https://doi.org/10.33933/2074-2762-2019-54-134-144
  10. Леонова, Т.Н., Маланичева, Н.В., & Маланичева, А.С. (2017). Международные рейтинги как инструмент оценки конкурентоспособности университетов. Вестник университета, 10, 125–130. EDN: YLQMAQ
  11. Пузатых, А.Н. (2019). Участие в рейтингах университетов мира как определяющий фактор развития на образовательную политику и развитие вузов. Образовательная психология в поликультурном пространстве, 4(48), 105–113. EDN: QKSQMJ, https://doi.org/10.24888/2073-8439-2019-48-4-105-113
  12. Пятенко, С.В. (2019). Самостоятельная жизнь рейтингов и результатов образования. Оригинальные исследования, 9(4), 18–28. EDN: ZEKPAT
  13. Удалова, Г.П. (2011). Доминирование – Лидерство у приматов (эволюционный аспект). Теория и методология в архаике. Лидерство в архаике: условия и формы правления, с. 25–49. EDN: SNDHYT
  14. Эскиндаров, М.А. (2022). В России нужно развивать свой внутренний университетский рейтинг. Ректор вуза, 4, 42–47. EDN: HIAESA
  15. Abramo, G. (2017). Bibliometric evaluation of research performance: Where do we stand? Voprosy Obrazovaniya / Educational Studies Moscow, (1), 112–127. EDN: YHZCSX, https://doi.org/10.17323/1814-9545-2017-1-112-127
  16. Bellantuono, L., Monaco, A., Amoroso, N., Aquaro, V., Bardoscia, M., Loiotile, A.D., Lombardi, A., Tangaro, S., & Bellotti, R. (2022). Territorial bias in university rankings: A complex network approach. Scientific Reports, 12(1), 1–16. EDN: DYITQA, https://doi.org/10.1038/s41598-022-08859-w
  17. Billaut, J.-C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shanghai ranking? Scientometrics, 84(1), 237–263. EDN: POHVSJ, https://doi.org/10.1007/s11192-009-0115-x
  18. Brent, T. (2023, October 12). Utrecht University withdraws from global ranking as debate on quantitative metrics grows. Science|Business. Retrieved July 31, 2025 from https://sciencebusiness.net/news/universities/utrecht-university-withdraws-global-ranking-debate-quantitative-metrics-grows
  19. Castillo, E. (2023, June 8). These Colleges Dropped Out of U. S. News Rankings | BestColleges. BestColleges. Retrieved July 31, 2025 from https://www.bestcolleges.com/news/these-schools-dropped-out-of-us-news-rankings/
  20. Dimzov, S., Matošić, M., & Urem, I. (2021). University rankings and institutional affiliations: Role of academic librarians. Journal of Academic Librarianship, 47(5). EDN: SKMMPH, https://doi.org/10.1016/j.acal-ib.2021.102387
  21. Docampo, D. (2011). On using the Shanghai ranking to assess the research performance of university systems. Scientometrics, 86(1), 77–92. EDN: EFEQST, https://doi.org/10.1007/s11192-010-0280-y
  22. Docampo, D., & Cram, L. (2014). On the internal dynamics of the Shanghai ranking. Scientometrics, 98(2), 1347–1366. EDN: XYPUBP, https://doi.org/10.1007/s11192-013-1143-0
  23. Espeland, W.N., & Sauder, M. (2016). Engines of Anxiety: Academic Rankings, Reputation, and Accountability. Russel Sage Foundation.
  24. Fauzi, M.A., Tan, C.N., Mukhtar, M., & Awalludin, N. (2020). University rankings: A review of methodological flaws implications of university rankings. Issues in Educational Research, 30(1), 79–96. EDN: NDFLCG
  25. Gao, X. (Andy), & Zheng, Y. (2020). ‘Heavy mountains’ for Chinese humanities and social science academics in the quest for world-class universities. Compare: A Journal of Comparative and International Education, 50(4), 554–572. https://doi.org/10.1080/03057925.2018.1538770
  26. Hazelkorn, E., Loukkola, T., & Zhang, T. (2014). Rankings in Institutional Strategies and Processes: Impact or Illusion?
  27. Holmes, R. (2024, April 27). ‘THE’ rankings: What happens to universities that leave? University World News. Retrieved July 31, 2025 from https://www.universityworldnews.com/post.php?story=20240423081048420
  28. Huang, C.K., Neylon, C., Brookes-Kenworthy, C., Hosking, R., Montgomery, L., Wilson, K., & Ozaygen, A. (2020). Comparison of bibliographic data sources: Implications for the robustness of university rankings. Quantitative Science Studies, 1(2), 445–478. https://doi.org/10.1162/qss_a_00031
  29. Huang, M.H. (2012). Opening the black box of QS world university rankings. Research Evaluation, 21(1), 71–78. https://doi.org/10.1093/reseval/rvr003
  30. Iesbik Valmorbida, S.M., Rolim Ensslin, S.P., Ensslin, L., & Ripoll-Feliu, V.M. (2016). Rankings universitários mundiais. Que dizem os estudos internacionais? REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 14(2), 5–29. https://doi.org/10.15366/reice2016.14.2.001
  31. INORMS Research Evaluation Working Group. (2022). Fair and responsible university assessment: Application to the global university rankings and beyond.
  32. Ioannidis, J.P.A., Patsopoulos, N.A., Kavvoura, F.K., Tatsioni, A., Evangelou, E., Kouri, I., Contopoulos-Ioannidis, D.G., & Liberopoulos, G. (2007). International ranking systems for universities and institutions: A critical appraisal. BMC Medicine, 5, 1–9. EDN: TBAGHW, https://doi.org/10.1186/1741-7015-5-30
  33. Klumpp, M. (2019). Sisyphus revisited: Efficiency developments in european universities 2011-2016 according to ranking and budget data. Review of Higher Education, 43(1), 169–219. https://doi.org/10.1353/rhe.2019.0094
  34. Kochetkov, D. (2023). Review of the Russian-language academic literature on university rankings and a global perspective. Humanities and Social Sciences Communications, 10(1), 576. EDN: VVEUKZ, https://doi.org/10.1057/s41599-023-02082-2
  35. Kochetkov, D. (2024). University rankings in the context of research evaluation: A state-of-the-art review. Quantitative Science Studies, 5(3), 533–555. EDN: CFFFED, https://doi.org/10.1162/qss_a_00317
  36. Krauskopf, E. (2021). The Shanghai global ranking of academic subjects: Room for improvement. Profesional de la Informacion, 30(4), 1–13. EDN: LHHULR, https://doi.org/10.3145/EPI.2021.JUL.08
  37. Li, J. (2016). The Global Ranking Regime and the Reconfiguration of Higher Education: Comparative Case Studies on Research Assessment Exercises in China, Hong Kong, and Japan. Higher Education Policy, 29(4), 473–493. EDN: OXDUNN, https://doi.org/10.1057/s41307-016-0015-7
  38. Loukkola, T., Peterbauer, H., & Gover, A. (2020). Exploring Higher Education Indicators (May).
  39. Merton, R.K. (1968). The Matthew Effect in Science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. https://doi.org/10.1126/science.159.3810.56
  40. Pakkan, S., Sudhakar, C., Tripathi, S., & Rao, M. (2021). Quest for ranking excellence: Impact study of research metrics. DESIDOC Journal of Library and Information Technology, 41(1), 61–69. EDN: NREHBW, https://doi.org/10.14429/djlit.41.1.16328
  41. Pandiella-Dominique, A., Moreno-Lorente, L., García-Zorita, C., & Sanz-Casado, E. (2018). Model for estimating Academic Ranking of World Universities (Shanghai Ranking) scores. Revista Espanola de Documentacion Cientifica, 41(2), 1–14. https://doi.org/10.3989/redc.2018.2.1462
  42. Safón, V. (2013). What do global university rankings really measure? The search for the X factor and the X entity. Scientometrics, 97(2), 223–244. EDN: KQTOQM, https://doi.org/10.1007/s11192-013-0986-8
  43. Safón, V., & Docampo, D. (2020). Analyzing the impact of reputational bias on global university rankings based on objective research performance data: The case of the Shanghai Ranking (ARWU). Scientometrics, 125(3), 2199–2227. EDN: CCCEFV, https://doi.org/10.1007/s11192-020-03722-z
  44. Sharma, Y. (2022, may 11). Three major universities quit international rankings. University World News. Retrieved July 31, 2025 from https://www.universityworldnews.com/post.php?story=20220511170923665
  45. Shreeve, R.L. (2020). Globalisation or westernisation? The influence of global university rankings in the context of the Republic of China (Taiwan). Compare: A Journal of Comparative and International Education, 50(6), 922–927. EDN: SNHYPJ, https://doi.org/10.1080/03057925.2020.1736403
  46. Soldani, J. (2019). Grey Literature: A Safe Bridge Between Academy and Industry? ACM SIGSOFT Software Engineering Notes, 44(3), 11–12. https://doi.org/10.1145/3356773.3356776
  47. Tuesta, E.F., Bolaños-Pizarro, M., Neves, D.P., Fernández, G., & Axel-Berg, J. (2020). Complex networks for benchmarking in global universities rankings. Scientometrics, 125(1), 405–425. EDN: RZWNPT, https://doi.org/10.1007/s11192-020-03637-9
  48. van Raan, A.F.J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143. EDN: YCSIYP, https://doi.org/10.1007/s11192-005-0008-6
  49. van Raan, A.F.J., van Leeuwen, T.N., & Visser, M.S. (2011). Severe language effect in university rankings: Particularly Germany and France are wronged in citation-based rankings. Scientometrics, 88(2), 495–498. EDN: PCAVSO, https://doi.org/10.1007/s11192-011-0382-1
  50. Vernon, M.M., Andrew Balas, E., & Momani, S. (2018). Are university rankings useful to improve research? A systematic review. PLoS ONE, 13(3), 1–15. https://doi.org/10.1371/journal.pone.0193762
  51. Waltman, L.R., Wouters, P., & van Eck, N.J. (2017, may). Ten principles for the responsible use of university rankings. CWTS Blog. Retrieved July 31, 2025 from https://www.cwts.nl/blog?article=n-r2q274

Articles

All articles