Modernising Research Assessment

Key messages: 
 

1 The current system used to evaluate and assess research careers for hiring, promotion and funding, is inadequate and does not serve its intended purpose effectively. 
 

• Researchers have expressed serious concerns about the current research assessment practices, noting that the assessments do not acknowledge their capabilities and contributions. 
   
• The current assessment approaches do not incentivise diversity within the research workforce, hampering progress and inclusivity. 
   
• Research culture and teams are being eroded as the focus is primarily on ‘publish or perish.’ 
   
• There is a lack of recognition for innovative and multidisciplinary research, resulting in a standardised approach that overlooks the unique nature of different research fields. 
   

• Significant barriers exist for researchers in terms of career opportunities, collaboration, and mobility between sectors, hindering growth and stifling progress in research fields. 

   

2.  Assessment practices heavily prioritise publication numbers, citations, and journal prestige, which perpetuates the status quo and hinders the success of underrepresented groups. 

3. The system has created a problematic relationship between universities, publishers, funders, and global ranking agencies, due to the pursuit of higher rankings and prioritising quantity over quality. 

4. Narrow research metrics stifle innovation and multidisciplinary research, and do not translate across different sectors. 

5. The current practices fail to recognise the value of experience outside of the research sector and hinder mobility between academia, industry, and government. 

It is crucial to find new ways to assess research careers to promote a diverse and effective research workforce, interdisciplinary collaboration, and career mobility.  

ACOLA has identified six pillars of modern research assessment to improve the practices in Australia moving forward: 

The findings from this report contribute to broader efforts by the Office of the Chief Scientist, the National Science and Technology Council (NSTC), and other government initiatives to support the knowledge economy and address barriers to retention and diversity in STEM fields.

Australia seeks to be a leading global hub for innovation and knowledge, and to have an exceptional research workforce that is highly skilled, motivated and talented. To achieve this objective, understanding systems of research assessment and how they impact on the practice of science and research is critical. Research assessment can improve research management, output and attractiveness to industry partnerships. However, the reliance on particular indicators and metrics can have unintended consequences and can impact researchers’ career paths and research focus. Understanding the positive and negative consequences will help inform the future of research assessment in Australia.

This report explores the nature and influence (positive and negative) of research assessment (broadly defined) on individuals, including career progression and cross-sector mobility. The report covers all organisations that employ researchers (public or private), all stages of the research endeavour and career, and all disciplines.

The report provides evidence on how research institutions nationally evaluate researchers. It delves into how research activities can not only lead to various outputs, but can also be an indicator for other critical organisational skills such as leadership, mentorship and participation in industrial or governmental bodies. The report is inclusive and considers researchers from diverse career stages, backgrounds and disciplines, as well as considering various types of research outputs and impacts.
The findings from the report show that research assessment practices differ across organisations and disciplines, reflecting their respective strategic and operational objectives. These differences, along with unintended consequences, must be recognised and accommodated when evaluating researchers and the quality of research.

The findings highlight how assessment shapes the research workforce (for example, career progression, diversity, and mobility between sectors) and how assessment influences not only the decisions of individual researchers (for example, publishing approaches, research focus, collaboration, and research methods) but also the decisions of research organisations (for example, research culture, appointments, and strategic investments).

A significant number of researchers expressed dissatisfaction with current research assessment processes. They considered that their research and capabilities are not being accurately, fairly and transparently recognised, which in turn is affecting their recruitment, promotion and research pursuits. Transitioning between sectors also posed challenges as employers struggle to understand the transferability of skills, which is hindering the recruitment of researchers with the necessary capabilities.

There are multiple and varied forms of research assessment across disciplines, and assessment processes need to acknowledge these differences. A modern, outward-facing research assessment should not only acknowledge but also celebrate the different ways in which research quality, excellence and impact are understood and measured. This includes the time it takes for research to have an impact and that not all high quality, valuable research will result in commercialisation or a policy change in the short- to medium-term.

The report underscores the importance of modernising research assessment practices, rewarding and incentivising innovation, promoting groundbreaking and risky research, and encouraging interdisciplinary approaches and collaboration. 2 ACOLA