If LLMs Have Human-Like Attributes, So Does Age of Empires II

This points out a methodological flaw that frequently occurs in anthropomorphic evaluation in LLM research. Many studies assume that human-like attributes such as morality or empathy exist in LLMs and attempt to measure them, but this premise carries a significant risk of distorting experimental design and result interpretation.

The research emphasizes the importance of the substrate. Implementing a neural network within a Turing-complete environment—such as Age of Empires II—can produce an entity similar to an LLM. This means that the human-like behavior exhibited by LLMs may not be an inherent capability of the model, but rather a non-unique attribute determined by the environment in which the model runs or its output method.

In conclusion, the approach of assuming the existence of anthropomorphized attributes in advance and then experimenting carries a significant risk of falling into circular reasoning. Therefore, for more rigorous research, it is necessary to adopt a null hypothesis assuming 'LLM non-uniqueness' that does not take LLMs' anthropomorphized attributes for granted, and to establish explicit measurement criteria.