European Science Organisations Reach Agreement on Reforming Research Assessment

Following a six-month collaborative process involving more than 350 European organisations from over 40 countries, an Agreement on Reforming Research Assessment has been reached and made public today. ALLEA has contributed to the Agreement as part of a core group of 20 research organisations that supported the drafting team throughout the process.

The Agreement summarises a shared vision on how to reform assessment practices for researchers, research projects and research performing organisations, with overarching principles founded on quality, impact, diversity, inclusiveness, and collaboration.

The envisioned reforms are centred around the following four core commitments:

  1. Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research
  2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators
  3. Abandon inappropriate uses in research assessment of journal- and publication-based metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index
  4. Avoid the use of rankings of research organisations in research assessment

Several additional supporting commitments aim to enable the move towards new research assessment criteria, tools and processes, and to facilitate mutual learning, communicate progress and ensure that new approaches are evidence informed. The Agreement further includes anticipated timeframes for implementing the reforms and evaluating progress and describes the operational structures for a coalition of organisations devoted to working together to implement the changes.

Next Steps

The collection of signatures to join the Coalition supporting the Agreement will be launched on 28 September 2022 at the EU Research and Innovation Days. A General Assembly of Coalition members will further decide on the governance of the Coalition, the strategy guiding the operations and activities of the Coalition as a whole, its annual work-plan and budget. The first General Assembly is expected to take place towards the end of this year.

As a European umbrella organisation of academies of sciences and humanities, ALLEA was able to provide an interdisciplinary perspective based on shared European academic values. ALLEA contributed to the Agreement through its Permanent Working Group on Science & Ethics and the Working Group European Research Area. Previously, ALLEA had also worked with the Global Young Academy on recommendations on the topic.

In a parallel endeavour, the Council of the European Union has recently adopted its Conclusions on Research Assessment and Implementation of Open Science. ALLEA welcomes the principles set out in the Conclusions for designing novel approaches to research assessment and emphasises that there is no “one-size-fits-all” format: any reforms should be driven by researchers taking responsibility for improving research assessment in their communities, following the core concept of self-regulation set out in the European Code of Conduct for Research Integrity. (Read ALLEA’s full response here)

More information

  • The complete Agreement for Reforming Research Assessment can be found here, as well as an overview of frequently asked questions regarding the Agreement and the Coalition.
  • Further information on the drafting process and the actors involved can be found here.

ALLEA Welcomes Council Conclusions on Research Assessment and Open Science

ALLEA welcomes the adoption of the Conclusions on Research Assessment and Implementation of Open Science by the Council of the European Union on 10 June. See ALLEA’s full response here.

The Conclusions are in agreement with points that ALLEA has made over the years, in particular on the necessity of appropriately implementing and rewarding open science practices and the development of research assessment criteria that follow principles of excellence, research integrity and trustworthy science.

At the same time, ALLEA continues to stress that it matters how we open knowledge, as the push for Open Access publishing has also paved the way for various unethical publishing practices. The inappropriate use of journal- and publication-based metrics in funding, hiring and promotion decisions has been one of the obstacles in the transition to a more open science, and furthermore fails to recognize and reward the diverse set of competencies, activities, and outputs needed for our research ecosystem to flourish.

ALLEA therefore welcomes the principles set out in the Conclusion for designing novel approaches to research assessment, with particular weight on recognizing (1) the critical role for peer review in research assessment and (2) the importance of integrity and ethics in developing criteria focused on quality and impact. 

ALLEA underscores that the described reforms are urgently needed and require concerted efforts from the international academic community, supported by infrastructures for exchanging best practices as well as the necessary financial resources to implement these. 

Read ALLEA’s full response

ALLEA Open Science Task Force to Be Represented in Upcoming Events

ALLEA’s Open Science Task Force has been invited to participate in several upcoming events to present their latest work and vision on Open Science practises. The task force will be represented in these events by its chair and ALLEA Vice President, Professor Luke Drury, from the Royal Irish Academy. 

The upcoming events include:

 

Meeting of the US National Institute of Health Biomedical Informatics Coordinating Committee (BMIC)

20 April 2022

Following his participation at the US National Academies of Sciences, Engineering, and Medicine (NASEM) Roundtable on Aligning Incentives for Open Science, Prof Luke Drury has been invited to join the NIH Biomedical Informatics Coordinating Committee (BMIC) internal meeting on ‘Open Science, Integrity and Innovation’ as a guest speaker on 20 April to introduce ALLEA’s latest work on Open Science.  

The BMIC was established in 2007 to improve communication and coordination of issues related to clinical- and bioinformatics at US National Institute of Health (NIH). It is a forum where cross-cutting issues related to biomedical informatics, data science, and open science are communicated, discussed, and coordinated. 

 

AESIS seminar on Open Science & Societal Impact

20 April 2022

The international network for Advancing and Evaluating the Societal Impact of Science (AESIS) will be hosting an online seminar on ‘Open Science & Societal Impact’ on 20 April in partnership with the Federation of Finnish Learned Societies. Prof Luke Drury has been invited to chair the plenary opening panel titled ‘Open Science for Societal Good’ (9:40 – 11:50 EEST / 8:40- 10:50 CEST). 

Main topics to be discussed in the event include incentivisation strategies and policies to stimulate Open Science; safe spaces to facilitate open scientific discourse in academia; examining geopolitical implications of global policies for access to research data; Open Science policies and practices to foster public trust and understanding in science, among others. More information and registration are available here. 

 

Virtual Panel Discussion: Building Structural Equity and Inclusion in Open Scholarship

6 May 2022

The United Nations Dag Hammarskjöld Library (main organizer of the annual UN Open Science Conference) will host the virtual panel discussion ‘Building Structural Equity and Inclusion in Open Scholarship’ on 6 May (7:30 EST / 13:30 CEST) as part of the 2-day global forum 7th Multi-stakeholder Forum on Science, Technology and Innovation for the Sustainable Development Goals. 

Prof Luke Drury will join as a guest panellist to discuss, among others, recommendations from ALLEA’s recent statement on ‘Equity in Open Access’ as well as ALLEA’s contributions to the UNESCO Recommendation on Open Science. More information and registration are available here. 

  

Learn more about ALLEA’s Open Science Task Force

 

Patent System Needs Adjustment to Harmonize with Open Science Objectives, European Academies say

A new ALLEA statement examines the current patent system in the context of the ideals and objectives of open science and recommends, among others, the introduction of grace periods in patent applications to make knowledge open as early as possible.

In a new statement published today, ALLEA, the European Federation of Academies of Sciences and Humanities, advocates for the harmonisation of the patent system with open science. The academies urge policymakers to introduce a grace period of at least one year to ensure rapid open publication of research findings.

In addition, the authors conclude that patent income must not be seen as a substitute for public funding and patent activity should be used with great caution as an evaluation metric in assessing the performance of research institutions, projects, and individuals.

The statement, prepared jointly by ALLEA’s Open Science Task Force (OSTF) and the Permanent Working Group Intellectual Property Rights (PWGIPR), analyses the current debate on the possible synergies and perceived tensions between open science and patent protections.

The publication explores these two apparently contradictory views on research policy. On the one hand, a utilitarian view underlines the value of research as a key pillar of innovation in modern societies, wherein patents are considered important tools to valorise research findings. At the same time, an increasingly vocal open science movement advocates for knowledge generated through research to be considered as a global common good to be shared as openly and as rapidly as possible.

The authors consider that “there is no fundamental opposition between open science and protection of IPR; ideas can be freely shared even if their commercial use is subject to restrictions, and indeed this is only possible because of patent law. However, there are clearly operational problems with the way the patent system is currently structured.”

With the right adaptations to existing patent law, knowledge valorisation does not need to prevent early sharing of research findings. On the contrary, “a reformed patent system is essential to the widespread adoption of open science, and could even incentivise it”, states Luke Drury, Chair of the ALLEA Open Science Task Force.

In its conclusions, the statement recommends:

  1. The introduction of a carefully formulated grace period of at least one year in patent applications to allow open publication prior to obtaining protection.
  2. The existing research and experimentation exceptions should be strengthened and broadly interpreted to underpin the free non-commercial use by researchers of knowledge disclosed in patents.

In addition, it notes that:

  1. While patent income and license fees may play a useful role in supplementing the budgets of public research bodies and the salaries of some individuals, this must not be seen as a substitute for public funding.
  2. Patent activity should be used with great caution as an evaluation metric in assessing the performance of research systems, bodies, and individuals. Incentivising the accumulation of non-performing patents is counterproductive and a waste of resources.
  3. The value of curiosity-driven open research in publicly funded research and education bodies needs to be better acknowledged as the bedrock on which innovation and entrepreneurial activity is built, even if it is hard to quantify and valorise.
  4. Related to the last point, the role of distributed communities and teams of researchers needs to be better recognised. The emphasis in patent law on individual inventors is unhelpful in this regard and does not properly reflect how science operates.

Read the full statement

ALLEA Joins the European Commission Coalition on Research Assessment Reform

ALLEA has joined the European Commission’s core group working on reforming research assessment. The group will support the drafting of an agreement led by the European University Association, Science Europe and the European Commission on key issues and timelines for implementing changes.

The coalition is composed by funding organisations, research performing organisations, national/regional assessment authorities or agencies, associations of research funders, of research performers, of researchers, as well as learned societies and other relevant organisations.

ALLEA is represented by Deborah Oughton, member of the ALLEA Permanent Group Science and Ethics and representative of the Norwegian Academy of Science and Letters. She is a Professor at the Environmental Sciences and Natural Resource Management Faculty of the Norwegian University of Life Sciences.

Towards a Research Assessment Reform

In 2021, the European Commission published the scoping report ‘Towards a reform of the research assessment system’. The publication presents the findings from a consultation with European research stakeholders and identifies the goals that should be pursued with a reform of research assessment. The report proposes a coordinated approach based on principles and actions that could be agreed upon by a coalition of research funding and research performing organisations committed to implement changes.

Research assessment reform is one of the topics ALLEA has worked jointly with its Member Academies and partners in recent years. In July 2021, ALLEA and the Global Young Academy (GYA) published a report covering the key takeaways of their webinar ‘Research Assessments that Promote Scholarly Progress and Reinforce the Contract with Society’. The event brought together science and policy stakeholders to rethink current research assessment models.

The key areas for research assessment identified by the stakeholders were how to strike a balance between funding of research to advance scientific progress and public accountability, how to assess the societal relevance of research and who defines the criteria, and how research assessment should be done.

In 2020, ALLEA, the Global Young Academy and STM (International Association of Scientific, Technical and Medical Publishers) organised a series of workshops about the future of peer review in scholarly communications. A short summary report is available here.

UNESCO General Conference Adopts Recommendation on Open Science

The UNESCO Recommendation on Open Science has been adopted at the 41st session of the UNESCO General Conference on 23 November 2021, making it the first international framework on open science. This follows a resolution from the 40th session of UNESCO’s General Conference in 2019, where 193 Member States tasked UNESCO with the development of an international standard-setting instrument on Open Science.

In developing the Recommendation on Open Science, UNESCO gathered contributions through Multistakeholder Consultations. A global online consultation on Open Science was conducted between February and July 2020 in the form of an online survey, which was open to all stakeholders and was available in English, French, and Spanish.

ALLEA participated in the design of this survey, which was coordinated by the International Science Council. As part of the UNESCO Open Science Partnership, the ALLEA Open Science Task Force also responded to the UNESCO Multistakeholder Consultations on Open Science with a statement submitted on 15 December 2020, which you can find here.

The UNESCO Recommendation on Open Science complements the 2017 Recommendation on Science and Scientific Research. It also builds upon the UNESCO Strategy on Open Access to Scientific Information and Research and the new UNESCO Recommendation on Open Educational Resources.

 

Aim of the UNESCO Recommendation on Open Science

The aim of the UNESCO Recommendation is to provide an international framework for open science policy and practice that recognises disciplinary and regional differences in open science perspectives, takes into account academic freedom, gender-transformative approaches and the specific challenges of scientists and other open science actors in different countries and in particular in developing countries, and contributes to reducing the digital, technological and knowledge divides existing between and within countries.

The Recommendation outlines a common definition, shared values, principles and standards for open science at the international level and proposes a set of actions conducive to a fair and equitable operationalisation of open science for all at the individual, institutional, national, regional and international levels.

To achieve its aim, the key objectives and areas of action of the UNESCO Recommendation are as follows:

i. promoting a common understanding of open science, associated benefits and challenges, as well as diverse paths to open science;
ii. developing an enabling policy environment for open science;
iii. investing in open science infrastructures and services;
iv. investing in human resources, training, education, digital literacy and capacity building for open science;
v. fostering a culture of open science and aligning incentives for open science;
vi. promoting innovative approaches for open science at different stages of the scientific process;
vii. promoting international and multi-stakeholder cooperation in the context of open science and with view to reducing digital, technological and knowledge gaps.

 

Read UNESCO Press Release

Read the UNESCO Recommendation on Open Science

Read the Report on UNESCO’s Global Online Consultation on Open Science

Read ALLEA’s Recent Statement on Equity in Open Access

Learn more about ALLEA’s Open Science Task Force

 

It Matters How We Open Knowledge – ALLEA Statement on Equity in Open Access

Today, the European Federation of Academies of Sciences and Humanities (ALLEA) published the statement “Equity in Open Access” that addresses how “gold” open access publishing routes and large read-and-write deals contribute to establishing inequitable structures within academic research.

“If we make scholarship free to read, but very expensive to write, we end up reinforcing inequitable structures of privilege and power within the academic system; this is not a price we should be prepared to pay,” says Prof Luke Drury, Chair of the ALLEA Open Science Task Force.

The statement builds on this year’s theme of the International Open Access Week (25-30 October), ‘It matters how we open knowledge: building structural equity’, which was in turn inspired by one of the four core values of Open Science, as defined in the recently released UNESCO Recommendation on Open Science:

“Open Science should play a significant role in ensuring equity among researchers from developed and developing countries, enabling fair and reciprocal sharing of scientific inputs and outputs and equal access to scientific knowledge to both producers and consumers of knowledge regardless of location, nationality, race, age, gender, income, socio-economic circumstances, career stage, discipline, language, religion, disability, ethnicity or migratory status or any other grounds.” (UNESCO Recommendation on Open Science, Page 7)

It also reflects the first of the eight key principles for scientific publishing recently adopted by the General Assembly of the International Science Council:

“There should be universal open access to the record of science, both for authors and readers, with no barriers to participation, in particular those based on ability to pay, institutional privilege, language or geography.”

 

Plan S and “Gold” Open Access 

In 2018, a consortium of major research funding and performing organisations started demanding long-due reforms in the academic publishing industry, an initiative widely known as Plan S (see also ALLEA’s previous Response to Plan S).

Widespread support for Plan S has triggered first steps in gradually disassembling the paywalls that continue to shield scientific literature from its readers. The so-called “gold” open access route (which makes articles freely available online for anyone to read) is considered an important tool towards Open Science but scientists that wish to publish via this route are often charged with substantial “article processing charges”.

“While for obvious reasons this route is promoted by commercial publishers, it effectively replaces a barrier to access with a barrier to participation.”, the authors state. As part of these reforms, large “read and write deals” are being negotiated between library consortia and commercial publishers, a notable example of this being the German “Projekt DEAL”.

 

Reinforcement of Inequitable Structures

Although collective deals can be beneficial to individual researchers that are affiliated with organisations covered by such agreements, ALLEA highlights several important inequities resulting from these developments:

  1. “[These deals] effectively incentivise such researchers to publish in the journals covered by the deal, which are often expensive journals that trade on their high ‘impact factor’ – a metric noted as problematic by Open Science initiatives.”
  2. “This tacit incentivisation risks further increasing the market dominance of the big commercial publishers and clearly disadvantages smaller specialist and learned society publishers.”
  3. “It takes no account of the fact that, at least in the humanities, there are still a significant number of researchers not affiliated with institutions covered by the deals, nor in some cases with any institution.”
  4. “It privileges established over early career researchers. It ignores the needs of researchers based in the Global South, in smaller institutions, or in industry. It favours well-funded areas of research over equally important, but less well-resourced areas.”

The authors argue that “It is a false framing of the discourse to say that either the reader or the writer has to pay; in most cases it is actually a third party (the library consortia in the case of the big deals) and ultimately it is the taxpayer for most publicly funded research.”

The statement describes several alternative open access publishing models, but the authors emphasize that a global solution to open access across all disciplines will only be available once adequate resources and infrastructure are made available.

 

About the statement

The statement was prepared by ALLEA’s Open Science Task Force, which aims to contribute to the development, coordination and implementation of Open Science policies and initiatives with a particular emphasis on issues relevant to the greater European area. The task force draws on the expertise of ALLEA’s academy members in promoting science across all disciplines as a global public good that is as open as possible and as closed as necessary.

Download the Statement

Shaping the Future of Peer Review

ALLEA, the Global Young Academy (GYA), and STM (International Association of Scientific, Technical and Medical Publishers) published today the summary of a series of cross-sectoral workshops on the future of peer review in an open, digital world.

Experts from across the world, representing different cultural and disciplinary traditions of peer review, convened virtually in November 2020 to discuss the future of peer review in an Open Science environment. Participants explored which models can best serve and reward the research community in both an enhanced and sustainable way.

Peer review is an essential element of scholarly communication and documentation processes and contributes to ensuring the quality and trustworthiness of modern research. The traditional models of peer review are, however, challenged by new digital modes of publication, and the wider range of research outputs envisaged as part of the move towards Open Science.

The workshops comprised a broad array of experts and actors including researchers, research funders, universities, publishers, libraries, the Open Science community and trade bodies. Main themes and areas for further consideration that emerged during the discussions included:

  1. Clarifying peer review and the roles of different actors in the system
  2. Building capacity for peer review: training, mentoring, inclusion and diversity
  3. Leveraging technology to deliver enhanced peer review
  4. Changes should be motivated by a strong evidence-base, collected through research, pilots and experimentation

Read the full summary.

ALLEA launches Open Science Task Force

ALLEA has launched a task force dedicated to open science and chaired by Luke Drury (Royal Irish Academy). The ALLEA Open Science Task Force will contribute to the development, coordination and implementation of Open Science policies and initiatives with an emphasis on issues relevant to the European Research Area.  

The group will draw on the expertise of ALLEA’s national academy members in promoting science as a global public good that is as open as possible and as closed as necessary and paying close attention to specific considerations of the social sciences and humanities.  

The task force will:  

  • work together with the Global Young Academy in assisting the creation and implementation of the European Commission Open Access publishing platform;  
  • contribute to the expert consultations on the UNESCO Recommendation on Open Science; 
  • liaise with other pertinent stakeholder organisations on Open Science. 
  • amplify the voice of the European Academies in this policy area

Past work on Open Science 

ALLEA has actively contributed to the open science debate since the early 2000s through various initiatives and working groups. Recently, it published the ALLEA Response to Plan S , the open access proposal initiated by European funders, as well as the policy paper Towards Implementing the European Open Science Cloud 

For more information on the rationale behind the task force, check out Luke Drury’s op-ed on the ALLEA Digital Salon.  

Why Open Science Is Here to Stay

Openness is one of the defining characteristics of modern science and scholarship. The idea that there should be some secret esoteric knowledge reserved for initiates has long been banished from serious research and survives only in some non-academic fringe groups. Even research in industrial R&D facilities is now routinely published, if only in the form of patent applications. The one major and sad exception is of course some military and security research, and there are also a small number of cases where fully open science is not appropriate, for example, in environmental research to protect endangered species or in medical research to protect patient confidentiality.

The fundamental concept is noble and powerful. Ideas, theories, and their supporting intellectual frameworks should constitute a common good of all humanity, freely shared for our mutual enjoyment and benefit. This concept is anchored in the Universal Declaration of Human Rights which states, in article 27.1 that “Everyone has the right freely to participate in the cultural life of the community, to enjoy the arts and to share in scientific advancement and its benefits.” This framing of science as a cultural activity from which nobody is excluded and to which everyone can contribute, and from which everyone can benefit without in any way reducing the benefit available to others, defines it as what economists term a pure public good.

 

“Ideas, theories, and their supporting intellectual frameworks should constitute a common good of all humanity, freely shared for our mutual enjoyment and benefit.”

 

The reality however is different. Large parts of scholarly publication have been captured by commercial bodies whose primary interest is shareholder value and not the common good. The sharing of data is partial, inconsistent, and inadequately resourced. Science is too often confused with innovation and valued only for its immediate utility. Openness is paid lip service, but is often not properly rewarded in research evaluations, funding decisions and career progression. If we want open science to realise its full potential, there is an urgent need to reform processes and attitudes as well as to invest in sustainable infrastructures and organisations to support it. The necessity of such change has been dramatically brought home by the COVID-19 pandemic, where the traditional structures of science have been exposed as too slow and sclerotic to deal with a rapidly changing scientific and policy landscape.

 

“The necessity of such a change (towards open science) has been dramatically brought home by the COVID-19 pandemic, where the traditional structures of science have been exposed as too slow and sclerotic to deal with a rapidly changing scientific and policy landscape.”

 

At the same time, however, we have to recognise that many features of the traditional system are there for good reasons, and that moving to a more open and agile system is not without risk. Managing change in a complicated and interconnected system is challenging and raises many issues, some legal, some ethical, some practical as well as more philosophical ones concerning the purpose, nature and conduct of scholarship itself. It is vital that the academic community actively participates in discussing these issues using the full range of analytic tools developed in our various disciplines as well as our lived experience as researchers.

Europe is not unique in this regard, and science being universal, this discussion has to be cognisant of the global context. However, it is also the case that some issues have a special salience within the European context. For all these reasons, ALLEA feels that it is appropriate to establish a special task force on Open Science to address these challenges, to allow ALLEA to respond in a coherent and timely manner to developments, and to amplify the voice of the European academies within this global debate.

 

Luke Drury, Chair of the ALLEA Open Science Task Force and ALLEA Board Member.