EUA Council for Doctoral Education
Making research assessment reform work for the next generation of researchers
As research assessment practices are changing in many countries, we need to be aware of unintended effects. This article pinpoints three crucial areas where universities can support doctoral candidates and early-career researchers as they navigate the new frameworks.
Recently, an ambitious movement to reform research assessment processes has emerged. Notably, this aims to acknowledge a broader set of competencies and qualifications associated with a career in higher education and research. In short, it should not only be the number of journal articles, the journal impact factor, and the H-index that make or break an academic career. Moreover, research assessment is performed in all kinds of settings and needs to be tailored to the specific assessment situation, be it a researcher’s application for funding or a promotion, or indeed the evaluation of entire research units.
In this context, many colleges and universities throughout Europe are implementing research assessment frameworks, while others are in the process of revising their existing frameworks. In general, there is much agreement that to measure quality and excellence, the assessment process needs to balance qualitative and quantitative dimensions. Recognising several competencies as the foundation for merit is also part of this process. Yet, given the importance of hiring decisions in academia, both for applicants and the institution, a key question is how research assessment reforms impact the hiring of academic staff.
But are universities actually so one-dimensional in their selection processes? A prominent argument for initiating research assessment reform has been the irresponsible ‘quantification’ of indicators. And the rise of easily accessible indicators has indeed made an impact in assessment practice. However, the more interesting question is whether revised assessment frameworks are a real improvement from the perspective of doctoral candidates and early career researchers who see their future in higher education. Here, there are three crucial things to keep an eye on.
1: A broader scope, not additional demands
New frameworks are broadening the sets of qualifications and skills considered in the assessment process. These include teaching qualifications, leadership skills, and experiences in turning knowledge into use for practical, societal purposes. However, there is a risk that young scholars may perceive this expanded list as new demands being added to the requirements that already exist.
Research qualifications will remain the key dimension for many comprehensive research-intensive universities, and the danger is that creating a broader set of qualifications will raise the bar for the next generation of researchers. Here, universities need to take responsibility for making sure that hiring and promotion practices reflect the diversity sought by new assessment frameworks. It is up to universities to make sure that their departments and institutes are staffed in ways that acknowledge the many responsibilities of, and expectations directed at universities.
2: Making qualitative dimensions transparent and easy
Balancing quantitative and qualitative dimensions in the assessment process is an objective with considerable support. Yet, there is a challenge related to how the qualitative dimensions are designed and evaluated. Quantitative metrics and indicators are popular due to the fact that they, at least at surface level, provide more objective standards for comparison. If qualitative dimensions are to play a stronger role, we need to work collectively to develop transparent and easy ways to make such qualifications visible.
Not least, it will be important to train, assist and researchers in understanding how they can reflect upon their qualifications and document them in convincing ways. We need to develop guidelines, templates and descriptions that are helpful for those being exposed to the new ways of assessing research. Practices should avoid cumbersome rules, as well as standards and guidelines that do not match the diversity of qualifications research groups and institutions look for.
3: The international angle
Research assessment is international in its nature. Evaluators are often academics coming from universities abroad, funding bodies are international, and early career researchers often seek employment in countries other than their own. This reality must be reflected in the whole apparatus surrounding research assessment. Moreover, universities have an important responsibility in providing information about new assessment frameworks.
Academic cultures are diverse, not only between countries but also between universities. Therefore, universities need to work together to promote and follow through on new research assessment standards. Exchanging practices, learning from each other, and not least critically evaluating the effects and impacts of new assessment frameworks will be crucial in the years to come. What are the system-level effects of the new practices? What will the consequences be for gender, diversity, and academic quality? As universities, we have a special responsibility for more evidence driven approaches in this area. While new research assessment frameworks could imply real improvements, we should be open for possible unintended effects along the way.
“The Doctoral Debate” is an online platform featuring original articles with commentary and analysis on doctoral education in Europe. Articles focus on trending topics in doctoral education and state-of-the-art policies and practices. The Debate showcases voices and views from EUA-CDE members and partners.
All views expressed in these articles are those of the authors and do not necessarily reflect those of EUA Council for Doctoral Education. If you would like to respond to this article by writing your own piece, please see The Doctoral Debate style guidelines and contact the CDE team to pitch your idea.