What kinds of mathematics are LLMs good at?

OpenAI announced remarkable achievements in mathematics and theoretical computer science, including the first construction of a non-sofic group and the resolution of a challenge in Ramsey theory.

Despite these achievements, it is difficult to assert that LLMs surpass humans in all aspects of mathematics. Looking at the major problems LLMs have solved, there is a tendency for them to show more prominent ability in finding counterexamples rather than providing direct proofs.

To support the hypothesis that LLMs are particularly competent at finding counterexamples, the following two things are needed:

• A clear definition of when solving a problem is considered 'finding a counterexample'
• A potential explanation for why LLMs are particularly suitable for solving that type of problem

Simply negating a universal quantification statement cannot be defined as the process of finding a counterexample. Therefore, a clear classification of why LLMs are suitable for specific types of problems, especially finding counterexamples, and a theoretical explanation of the reasons are required.