CSTI Expert Panel: Implementing a survey to hear from researchers — aims to ease feelings of exhaustion due to evaluations and applications.

Many researchers are frustrated because of the amount of time they have to spend preparing evaluations and applications for competitive funds when the results are not reflected in their next project, and it can make them feel like they are wasting their time as they have no idea of what purpose these documents have been used for. Additionally, a lot of time is taken up for various types of assessments and personnel appraisals. The Expert Panel of the Council for Science, Technology and Innovation (CSTI) will conduct a survey targeting researchers, University Research Administrators (URAs) and technical and administrative staff concerning fatigue relating to evaluations and applications at universities. The results will be integrated into CSTI's final report and will be leveraged to improve the research system by securing time to be devoted to research. The Cabinet Office hopes to conduct the survey as soon as possible.

One of the factors responsible for the relative decline in the position of Japan's research quality is the decrease in the amount of time researchers spend on research. Since March last year, the Expert Panel has been discussing issues that should be addressed to ensure that researchers have time to dedicate to their research. The Expert Panel's final report identifies eight challenges to be addressed by the government and university management: managing and utilizing DX (digital transformation) research data, encouraging the shared use of research facilities and equipment, improving conditions for engineers and other professional personnel, ensuring the quality and quantity of URAs, reviewing role sharing between teaching and research faculty, easing the burden of preparing university entrance exam test questions, reducing the number of internal university meetings and formulating strategies to cope with the fatigue mentioned above. Some of these were presented as guidelines in February for improving the quality and quantity of research time and were included in the Package for Comprehensive Promotion of Research Universities with a Regional Core and Distinctive Characteristics.

However, the level of discussion for one of these items, evaluation/application fatigue, is not yet sufficiently developed because discussions are at various stages, requiring a review of the government‐wide evaluation system on the one hand and a review of individual research funding systems on the other. Therefore, the government will first undertake a survey by questionnaire and discuss the results, which will then lead to institutional reform through a liaison meeting of the relevant ministries and agencies.

The secretariat's proposal is to ask respondents to include only the external funds obtained and the number of past applications and successful cases in their profile information, and to ensure that the information remains anonymous.

As evaluatees, the respondents will be asked to describe the evaluation requirements and the burden of preparing applications or reports, while identifying specific project titles, such as competitive funds, projects or grant programs from outside the government. The same will be done for internal evaluations, such as institutional evaluations and personnel appraisals. As evaluators, the respondents will also be asked to answer questions about the burdens of screening and assessing faculty funds, conducting peer reviews of papers and evaluating graduate theses, master's theses and doctoral dissertations. In addition, they will be invited to offer their opinions on the research support system in general.

Will the many issues raised make it difficult to implement by April?

It may be difficult to implement the secretariat's present proposal by the end of April because of the many issues raised. President Teruo Fujii of the University of Tokyo, said, "We should clarify further the kinds of evaluation items that take a lot of time and effort. For example, in education‐related projects, we must conduct the widest assessment possible of graduates to determine how they are performing. In this case, effort is expended in consolidating all the information. By contrast, reviewing theses and dissertations is essential work, so it is not considered a burden."

President Takaaki Kajita of Science Council of Japan, commented, "We need to work out a way to embrace the notion of application fatigue. A 10% approval rate for competitive research funding means that 90% of applications are not successful, and after repeating such applications year after year, the original ideas are no longer worth pursuing due to scientific progress. There is such a sense of wasted effort. We need to be able to hear from voices in the field."

Executive Advisor Hiromichi Shinohara, of Nippon Telegraph and Telephone Corporation (NTT), said, "The information that evaluators need and the points that evaluatees want to emphasize are different. We should clarify what items are unnecessary in current evaluation items, and we hope that the questionnaire will be used for this purpose. It would be better to determine the final design of the questionnaire after we conduct a trial run."

Professor Mutsuko Hatano of Tokyo Institute of Technology said, "Internationally, there is a shift from quantitative to qualitative evaluations. Wouldn't a questionnaire be able to draw that out?"

Professor Hiroaki Suga of the University of Tokyo said, "The National Institutes of Health (NIH) of the United States of America has three grant application deadlines a year, making it possible for reviewers to adjust their schedules. This could be possible for funded research grants."

In response, President Kazuhito Hashimoto of Japan Science and Technology Agency (JST), said, "We must not forget that compared to the United States of America, we have two to three times the number of researchers but research costs that are 20 times higher. We are looking at how we can do things in Japan. The important point is to show that the government recognizes these problems and works to improve them."