NICT Releases Large-Scale Web Analysis Engine WISDOM X on a Trial Basis
Capable of Finding Answers to Diverse Questions

The National Institute of Information and Communications Technology (NICT) released a deep learning version of WISDOM X on a trial basis on their website on March 31. The large-scale web analysis engine was developed by the Data-driven Intelligent System Research Center (DIRECT), Universal Communication Research Institute, NICT.

For the past 10 years, NICT has been carrying out research and development of technology to find answers to questions using searchable text on the Internet. Most existing search engines provide results of only about 10 or so documents at once that include the search keyword, and users need to read a large number of those documents to find a comprehensive answer to their question. In comparison, question response systems such as WISDOM X offer a list of straightforward answers to questions posed to them, enabling users to easily and quickly gain a comprehensive overview of the related information, and even to make valuable and unexpected discoveries.

This type of overview or unexpected discovery is a necessary resource for tackling work where uncertainties may emerge such as innovation or risk management, which are becoming increasingly important in recent years. NICT believes that WISDOM X, originally released on a trial basis in 2015, is able to make such understanding easier, and capable of providing valuable hints for brainstorming.

However, the original system was developed before deep learning became common, and used machine learning. Thus began the development and now the trial release of the improved, deep learning version.

The upgraded version is developed using BERT, a neural network system currently garnering attention, and a proprietary modification of BERT combined with adversarial deep learning, to result in a system capable of flexibly answering a diverse range of questions.

BERT is an advanced technology released by Google in 2018. It is advanced technology that updated the existing optimum performance capabilities towards various tasks in the natural language processing (NLP) domain, such as answering questions, and had a significantly large impact on deep learning research for NLP.

The new version of WISDOM X is a high performance system that learned using approximately 350 GB of text from the Internet, using the massive BERT neural network, along with a massive, high quality learning data base developed by NICT, and the improved version combined with their own uniquely developed technology, enabling it to answer a wider range of questions with greater accuracy. The system uses 6 billion pages of text extracted from the Internet as the information source for answering questions.

For example, the original version was unable to answer a question such as, "Are there any problems related to the aging population that may be solvable by AI?" Whereas the new version can. The new version can also answer "how-to" questions that were unanswerable with the original system, such as "How can AI solve the problem of the aging population?" or "How can AI contribute to CO2 capture technology?"

The new system can also answer questions phrased using casual every day speech, such as "I've got cheese and spring onions, what snack can I make with them?" Furthermore, the system can also answer questions on a diverse range of topics, such as "What are some 'b-grade gourmet' restaurants in Izu?" or "What should I see as a tourist in Nara?" by accessing its massive supply of information from the Internet.

NICT plans to expand this technology for a variety of AI systems, such as dialog systems geared towards solving important issues faced by society, including nursing for the elderly and disaster prevention.