A research team led by Assistant Professor Kan Hatakeyama and Professor Teruaki Hayakawa of the School of Materials and Chemical Technology at the Department of Materials Science and Engineering at the Tokyo Institute of Technology (Tokyo Tech) announced that they used the large-scale language model GPT-4 developed by OpenAI to model tasks in chemical research to validate its potential capabilities and challenges. The validation results on benchmark tasks in four domains—chemical event recognition, analysis, prediction, and planning—showed that GPT-4 can effectively provide useful knowledge and insights for various tasks in chemical research. However, issues such as a lack of advanced knowledge and errors in information recognition were also identified. The results were published in the October 9, 2023 issue of the international journal Science and Technology of Advanced Materials: Methods.
The large-scale language model GPT-4 developed by OpenAI was made public in March of this year, and interest in artificial intelligence (AI) has increased dramatically since then. This model has been shown to have capabilities equivalent to or superior to those of humans, including a broad range of knowledge and the ability to perform a variety of tasks.
Further improvements in performance are expected based on the scaling laws (an empirical rule indicating that the performance of large-scale language models consistently improves with increases in model size, training data, and computational resources) and Moore's law (an empirical rule stating that the number of transistors on integrated circuits doubles approximately every two years). These advancements are expected to have applications in various fields.
In this study, the research team focused on the organic materials field in particular to verify its capabilities. Consequently, GPT-4 had knowledge data on the physical properties and characteristics of various compounds, allowing it to answer graduate-level questions. However, there were numerous incorrect answers when it came to recognizing complex molecular structures and advanced-level chemical reactions.
In terms of chemical event analysis and prediction, GPT-4 was able to explain why specific molecules exhibited higher physical properties (e.g., redox potential) compared to their counterparts based on the presence or absence of functional groups. It could also provide reasoned explanations for the properties of unknown compounds based on their chemical knowledge. This demonstrates the ability to infer based on prior chemical knowledge.
For chemical event planning, GPT-4 was also able to propose the number of compounds to be prepared and the reaction time required to obtain the desired yield in a model reaction system. Based on instructions in natural language, GPT-4 could generate the control program for a robotic arm that performs the experimental manipulation. It was also possible to make suggestions for optimization-specific experimental procedures and reaction conditions, resulting in increased efficiency and success probability for the experiment. However, there were some issues, such as the fact that they have little advanced knowledge at the level of academic papers, and some tasks showed errors in the prediction of physical properties.
Solutions require the development of language models specific to specialized knowledge as well as integration with existing informatics methods. The potential for new insights into unsolved issues and phenomena that can be gained by streamlining operations and utilizing AI's vast knowledge and reasoning capabilities has raised expectations.
Hatakeyama said, "Through our research, it has become clear that large-scale language models such as GPT-4 can support a wide range of chemical research. We are currently conducting research to train language models to learn cutting-edge scientific discoveries and to use a robotic arm to automate chemistry experiments."
Journal Information
Publication: Science and Technology of Advanced Materials: Methods
Title: Prompt engineering of GPT-4 for chemical research: what can/cannot be done?
DOI: 10.1080/27660400.2023.2260300
This article has been translated by JST with permission from The Science News Ltd. (https://sci-news.co.jp/). Unauthorized reproduction of the article and photographs is prohibited.