Evaluation enhanced and standardised

The SNSF invests around a billion francs in Swiss research every year. Its core task is to select and fund the best projects and researchers from across Switzerland. "We apply the same high standards to our work as we do to the research we fund," Thomas Werder Schläpfer, member of the Executive Management, emphasises. "An evaluation process that recognises excellence should itself be excellent."

Until now, the SNSF has used different evaluation procedures, depending on the funding scheme. Now it has enhanced and merged them into a uniform, modular process. This applies to all schemes, with a few exceptions.

The most important innovations

• Six main steps, adaptable as needed: The process comprises six main steps, starting with the administrative verification of applications and culminating in the communication of results. The three key steps of the evaluation proper are the external review, the internal review and the discussion by the relevant evaluation panel. The panel is composed of researchers who work for the SNSF on an honorary basis. For some schemes, further steps are added. "By adopting this flexible format we can achieve the balancing act between standardisation and any necessary customisation," says Thomas Werder Schläpfer.
• Two independent recommendations: After the external review, each application is independently evaluated by at least two members of the evaluation panel. Their recommendations provide the basis for the subsequent discussion.
• Rating scale of 1 to 9: The reviewers and panel members rate the applications on a uniform scale of 1 to 9. A grade of 1 means that an application has few or no merits and many serious weaknesses. Projects graded with a 9, on the other hand, are outstanding in all respects. The scale creates clarity and comparability.
• Individual voting: Each application is rated by each member of the panel. The individual grades are only visible after everyone has voted. This ensures an independent vote.
• Creating the ranking list: The combination of numerical rating scale and individual voting makes it possible to establish a ranked list of proposals using a statistical procedure. Here, the SNSF uses a Bayesian ranking method that takes account of random fluctuations and other uncertainties.
• Preselection for rejection possible: If the external and internal reviews clearly identify a number of weaknesses in a proposal, the panel may preselect it for rejection. In such cases the proposal is not discussed in detail.
• Separate evaluation and funding decision: The SNSF now makes a clear distinction between scientific evaluation and funding decisions. Based on the ranking of proposals and the available funding, the National Research Council determines the “funding line”. Proposals with a score above the line will receive funding.
• Lottery option: If the funding line runs through a group of proposals of the same quality, the Research Council may make a random selection by drawing lots. This option has been in place since 2021.

Big step in the right direction

"By adopting these innovations, we are creating a uniform process and once again increasing quality, efficiency and transparency," says Thomas Werder Schläpfer. "While this is a big step in the right direction for our research funding, it is by no means the end. As an organisation that wants to go on learning, we will carefully monitor and further refine the evaluation as needed, always based on the scientific state of the art."

The SNSF is applying the standardised procedure with immediate effect. Details of the individual steps as well as the criteria and other basic principles can be found on the website.

• Evaluation Procedure