Serious AI Alignment Issues in Mathematics - Terence Tao et al.

A statement was issued by multiple Fields Medalists, including Terence Tao, pointing out a serious misalignment of goals between AI companies and the mathematics community. They expressed concern that while improved LLM capabilities have made it possible to solve unsolved problems, AI companies' 'problem-solving' focused benchmarking could be harmful to mathematical science and the community.

Misalignment of Goals Between AI and the Mathematics Community

The essence of mathematical research is not merely deriving correct answers, but the process of building new insights and methodologies. However, 'true/false' propositions mass-produced by AI are being rapidly published without proper documentation, isolation of new methodologies, or citation of prior research, causing serious attribution and plagiarism issues. This risks destroying the soil of the existing research ecosystem rather than nurturing new ideas.

Threat to Education and Transmission Processes

The mathematics community views students and ideas as core resources, emphasizing growth and the transmission of wisdom through human interaction. AI's ability to directly generate results from existing work conflicts with traditional educational goals of asking questions based on understanding and mastering concepts. It was pointed out that the spread of AI solutions could sever the human chain of transmission among mathematicians.

Future Direction

Over 20 Fields Medalists, including Terence Tao, emphasized that while AI can potentially accelerate mathematical research, whether the results will be constructive or destructive depends on human decisions as the controllers of technology. They urged the mathematical community, AI companies, and society at large to urgently address these threats to intellectual labor and alignment issues.