LG AI Research Publishes Mechanistic Interpretability Study on Graph Transformer Internal Mechanisms
Key point
The study elucidates the principle of local structure restoration via attention heads in basic graph tasks such as degree calculation.
Details
LG AI Research released research results analyzing the internal workings of the graph transformer model TokenGT using Mechanistic Interpretability techniques. Unlike previous mechanistic interpretability studies that primarily focused on LLMs, this research is an early case elucidating the internal mechanisms of graph data processing models widely used in chemistry and biology.
The research team analyzed models trained on three graph tasks: degree calculation, cycle existence determination, and shortest path distance calculation. The results showed that although graph operation functions were not explicitly built into the model architecture, the attention heads in the first layer restore the connectivity relationships between nodes and edges. In particular, it was confirmed that the ID matching mechanism, where node tokens strongly assign attention values to connected edge tokens, is the core process recording degree information in the residual stream.
It was revealed that the local structural information formed in the initial layers is reused not only for simple degree calculations but also for more complex tasks such as ring inclusion and shortest path. Activation patching experiments demonstrated that manipulating the initial degree representation affects the performance of subsequent tasks, revealing that the model performs reasoning based on a common understanding of graph structure rather than learning independent knowledge for each task.
This study is a case that mechanistically explains how transformers process graphs without graph-specific architectures. LG AI Research plans to apply this technique to improve the interpretability of molecular graph models and verify chemical rules learned by AI for solving real scientific problems such as molecular property prediction in the future.