2007 Performance Ranking of Scientific Papers for World Universities


Researcher/ Higher Education Evaluation and Accreditation Council of Taiwan

Mu-Hsuan Huang Professor/ National Taiwan University

Background

  This project is to evaluate and rank the scientific paper performance for the top 500 universities worldwide. The performance measures are composed of nine indicators to assess a university’s overall scientific paper performance along three criteria: research productivity (accounts for 20%), research impact (30%), and research excellence (50%). Different from the QS World University Rankings by Times Higher Education Supplement which focuses on university ranking, and the Academic Ranking of World Universities by Shanghai Jiao Tong University which focuses on academic ranking, this project focuses on scientific papers performance ranking. The emphasis on current research performance makes the indicators a fairer one than some traditional indicators such as a university’s reputation or the number of Nobel Prize winners affiliated with that university, which tend to favor universities with longer histories or universities in developed countries.

  This project employs quantitative data drawn from Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) to evaluate the scientific paper performances of world universities. Today, publishing in international academic journals is the predominant mode of scientific research output. Statistics on the articles published in international academic journals provide an objective representation of each university’s research performance. Furthermore, since the Institute for Scientific Information (ISI) began to publish SCI and SSCI in 1961, the two databases have grown over the years to include a good number of academic journals that are both international in scope and comprehensive in subject representation. However, results of this project may favor those universities’ performances in sciences and social sciences and under represent their performances in humanities research. Although ISI also publishes Arts & Humanities Citation Index (A&HCI), this project excludes this database because it may fail to objectively and accurately represent the research performance of humanities researchers worldwide – A&HCI mainly indexes journals in English language, while humanities researchers often publish in their native languages and in various forms of publications (e.g., books). Analyses of the data obtained from SCI and SSCI allow fairer comparisons across universities globally. Generally speaking, the indicators used in this project have at least the following three features:

  1.It emphasizes the quality of research; the indicators assessing research quality (research impact and research excellence) account for 80% of the performance score.

  Research impact and research excellence evaluate the quality of a university’s research output. The calculation of each university’s score is based on the number of citations of its published articles, h-index of the last two years, number of Highly Cited Papers, number of articles published in high-impact journals(Hi-Impact journal articles), and the number of subject fields in which the university has outstanding performance(fields of excellence)(these indicators will be explained further in the Indicators section).

  2.It balances biases due to a university’s size or faculty number.

  Traditionally the size of a university affects its ranking when the number of articles is used as a sole indicator for research output. Because the number of articles is closely tied to the number of faculty members, rankings employing numbers of articles often favor larger universities. This project corrects that flaw by incorporating the average number of citations, the number of subject fields where the university demonstrates excellence, and h-index in the calculation of universities’ performance scores (explained below). The inclusion of the three indicators accounting for 40% of the total score balances the assessments of quality and quantity of research and provides a fairer representation of a university’s performance regardless of its size.

  3.It takes into account a university’s short-term research performance (constituting 50% of the score), thus ensuring a fairer comparison between universities of various lengths of history.

  The indicators used in this project seek to represent both the long-term and short-term research performances of a university. The inclusion of indicators evaluating short-term performances corrects the flaws resulted from undifferentiating indicators that favor universities of longer histories. These short-term performance indicators include: the number of articles in the current year, the number of citations in the last two years, h-index of the last two years, and the number of articles in high-impact journals in the current year (explained further in the Indicators section).

  Through the use of the proposed indicators, this project attempts to objectively portray the current research development in Taiwan’s universities in comparison to universities worldwide. The relative strengths and weaknesses revealed in the ranking provide insights into the higher education administration and resources allocation. This project also intends to update the ranking annually and provide ranking information by subject fields in order to encourage the development of specific subject strengths in Taiwan’s universities.