Excellent SEP-evaluation score for all institutes
Recommendations mainly about national role, gender balance and attention for PhDs
In autumn 2023, all NWO Institutes were visited by an international assessment committee in the context of the SEP evaluation (Strategic Evaluation Protocol). At the end of May, the committees’ reports were published. Without any exceptions, the institutes scored ‘excellent’ in terms of scientific content and their position. Across the spectrum, the SEP committees’ recommendations mainly concerned the institutes’ national role, gender balance and attention for PhDs. Peter Spijker (NWO-I office), Martin van Breukelen and David van Walderveen (DIFFER), and Jessica Dempsey (ASTRON) look back at the SEP evaluations with satisfaction.
Critical friends
NWO-I holds these SEP evaluations every six years to help institutes gain insight into the best course for them, both in retrospect and while looking ahead. The evaluation is carried out by a committee of six people, mainly researchers, but at times representatives from industry or people with a societal position as well. Peter Spijker, head of Strategic Support at the NWO-I office, refers to these committee members as ‘critical friends who understand what an institute does and at the same time are able to provide critical reflection’.
Themes for evaluation
Each committee was asked to reflect on three subjects: research quality, social relevance and viability. Further, the NWO-I board added four other themes: Open Science, taking care of PhDs, academic culture and personnel policy (please see text box for further explanation).
The subjects of the SEP evaluation:1) Research quality: scientific relevance of the research, academic reputation and leadership within the discipline, position in the (inter)national research landscape. |
Coherence
In May, the NWO-I Foundation board discussed the outcomes of the evaluations with individual institutes and examined the common factors. Peter summarises the common thread: “Some points of attention were found at all institutes. These do not concern shocking things but rather confirm what the institutes already knew. The scientific quality was certainly considered but, in most cases, the committee had already been convinced of this on the basis of the self-evaluation reports. The committees’ experts mainly examined the coherence of everything: is there a coherent strategy, do the institutes clearly stand out in the national context, or does it fulfil a niche role in the Netherlands? Does it have a clear external profile? Sometimes, the committees contained people with more distance to the academic world who are able to see where opportunities for an institute lie.”
All the institutes scored really well but there is no room to slack
Points of attention found at all institutes concerned employees and culture-related subjects, such as diversity and inclusion, gender balance and social safety. All committees said that they were pleased with the efforts made on the basis of these themes but, at the same time, concluded that the institutes ‘are not there yet and there is no room to slack’. Peter: “One example of an institutes-overarching recommendation is to improve the mentoring programmes for PhDs and postdocs. Not only to support them in their career within the institute but, in particular, to prepare them for their future careers. Or what the policy is for employees who are about to become a group leader or professor. Does the institute want them to become a professor or not? We need to formulate that more clearly. These valuable recommendations offer much scope for the future. All evaluation reports conclude that, on balance, all institutes are in a very good position.”
Site visit
Each institute received a two-day visit from the committee, during which knowledge exchange, interviews with researchers and a look around the institute took centre stage. This site visit from the committee formed the highlight of the evaluation.
At ASTRON, the SEP committee even added an extra day to the site visit: two days was not enough time for the members because they visited all of the telescopes with the technical crew and spent longer than planned in the meetings with PhDs and postdocs. The committee at ASTRON was made up of five women and one man, leaders in astronomy or related disciplines, and included several junior members. “This is because ASTRON wanted to emphasise its efforts for young employees”, says Jessica Dempsey, director of NWO-Institute ASTRON.
You are fantastic
For Jessica, who began at ASTRON in 2022, the SEP evaluations were, first of all, a brilliant crash course about the recent history of the institute: “With all experts together, and the collective historical knowledge, it became very clear what all ASTRON has achieved in the past six years. That gave me the opportunity to tell all employees just how fantastic they are.”
“The confirmation from these external experts that we have chosen the right course for the next six years gives the institute a moral boost. Our national role is crystal clear: on behalf of the astronomy community in the Netherlands, we exploit the large national infrastructures and we must hold on to this leading role. We also perform well for the theme ‘Open Science’: ASTRON has an ‘Open Skies’ policy, which means that everybody can request observation time. There is still room for improvement in this field since the data are very complex and not everybody is expert enough to interpret these. Therefore, part of our mission is to make radio astronomy more accessible.”
Younger generation’s perspective
According to Jessica, the SEP committee’s recommendations for ASTRON very clearly came from the perspective of the younger generation. “The committee appreciated the important new efforts made by ASTRON to achieve greater inclusion and diversity. Inclusion is an extra pillar of our mission, together with sustainability. Their comment was: ‘You have begun with the best of intentions’. An important recommendation was that we must organise more interaction between PhDs and postdocs because – the committee concluded – about 50% of this group do not know each other, but they do have a need for institute-wide activities.”
Valuable advice for the future DIFFER strategy
Martin van Breukelen is the institute manager at DIFFER, and David van Walderveen is the team leader Strategic Support. Together, they organised the SEP evaluation at DIFFER. Both of them are relatively new at DIFFER and they found the SEP evaluation to be an ideal way to get to know the institute. Martin says that he was particularly surprised by the changes that have taken place at the institute in recent years.
Lunches with technicians and PhDs
On day one, the site visit programme covered the scientific side. David: “All group leaders gave a presentation and the committee visited the facilities.” The second day started with a – obviously not programmed - conversation about the results of the Dutch general election the previous evening. David: “The committee wanted to know what this would mean for research in the Netherlands and, consequently, for DIFFER as well. Then, we moved on to the operational side, including academic culture, organisation and funding. The two lunches were with technicians and PhDs, a broad mix without management.” Martin: “The committee felt that the lunches were the highlights of their visit. From the conversations, they gained a good impression of the atmosphere in the labs and, for example, whether colleagues experience work pressure. The technicians were nervous beforehand but everything proceeded in a relaxed manner.”
Room for improvement
Martin: “The committee quickly indicated that ‘the research quality of DIFFER is world-leading, no comment’. Next came the points that leave room for improvement, such as the gender imbalance and the composition of research groups. We were advised to strive for a different allocation of personnel in terms of career development, tasks and responsibilities as well as gender. This discussion has been taking place internally for some time. We are trying our best but it is not possible to change the composition at short notice.”
David adds: “Another recommendation concerns the contact with PhDs: ‘Strengthen this and better communicate what you have to offer’. Between the evaluation report and site visit, we had just launched an onboarding programme and therefore we had already begun to tick the box for this subject.”
“What I personally take from this the most were the questions about the positioning of DIFFER and how this can be linked to other institutes and roadmaps”, continues David. “How do you determine the priorities in research, contacts with industry and your national role?” Martin: “The committee also took a critical look at our investments in infrastructure. In effect, the SEP is not so much an evaluation as a valuable set of recommendations for your future strategy.”
Follow-up to the evaluation
For the institutes, the SEP evaluations are an important calibration point for the strategy and efforts over the next six years. The NWO and NWO-I boards regularly ask the institutes about actions they have taken in response to the recommendations, and actively provide support in realising these. The next SEP evaluation is planned for late 2029.
More information about the SEP evaluation
On 27 May, the full news item with evaluation reports was also published on the NWO website.
Text: Anita van Stel
Newsletter Inside NWO-I, June 2024
You can find the archive of the newsletter Inside NWO-I on the NWO-I website.