About Laboratory

As the complexity of information processing tasks increases, the criteria for evaluation become more varied. For basic arithmetic operations, evaluation is based solely on whether the answer is correct or incorrect. However, when it comes to tasks involving future predictions, while performance can be evaluated probabilistically – such as by accuracy rates – based on past data, it remains unverifiable whether that accuracy rate can be sustained with future inputs until the future arrives. Therefore, intelligent systems may sometimes be difficult to evaluate.

In the field of AI, researchers are working on enhancing the explainability and interpretability of results. Generative AI tasks, which have gained significant attention in recent years, are evaluated based on validity rather than on correct answer rates because there is no single correct answer. When developing new intelligent systems, evaluation criteria should be established concurrently.