Exploring the Reliability of Research Indicators Reported by Romanian Universities in 2019

• Main Authors: Vlad Ștefan Barbu, Florentina Furtunescu, Bogdan Murgescu, Carmen Pintilescu
• Format: Article
• Language: English
• Published: University Babes-Bolyai 2021-07-01
• Series: Journal of Research in Higher Education
• Subjects: research indicators; reliability; fields of science; h-index; cnatdcu standards
• Online Access: http://jrehe.reviste.ubbcluj.ro/index.php/2021-2/2021-issue-no-1-vol-v/vlad-stefan-barbu-florentina-furtunescu-bogdan-murgescu-carmen-pintilescu-2/v-s-barbu-f-furtunescu-b-murgescu-c-pintilescu_art/

The paper focuses on the aggregate data published by the National Council for Higher Education Financing regarding the research section of the quality indicators reported by Romanian public universities in 2019 and used for the allocation of the performance-based additional funding. The research section includes 4 indicators, which amount to 46% of the additional funding, i.e. to an average of about 10% of the total institutional funding from the public budget. The data are reported by the Romanian universities each odd year since 2015, and for the first time in 2020 national rankings for most indicators and some national averages have been made public. The data is extremely valuable for a diagnosis of the Romanian higher education system and for highlighting the performance of various universities. Therefore, the accuracy of these data is crucial. The authors of this paper acknowledge the efforts of the National Council for Higher Education Financing and of its staff provided by the Executive Unit for Financing Higher Education, Research, Development and Innovation to check the data reported by the universities, but also the limits of the current verification process. The paper uses statistical methods to identify outliers and investigates the rankings for a selection of fields of science, by using three among the four research indicators. Thus, it discusses some of the more blatant reporting anomalies which contradict the pre-existing conceptions regarding the comparative performance of universities. While the statistical findings do not support the suspicions of systematic attempts of over-reporting the research outcomes at university-level, they still outline several persisting errors. These errors highlight the need to strengthen the verification process, which will require additional resources; the authors suggest that enhanced transparency and organised cross-verification among universities can significantly improve the outcome, and help providing a significant set of reliable public data on the research performance of the Romanian universities.