Analysis of Implicit Value Theories in AI Alignment Research

The paper 'Toward a Theory of Value in AI Alignment', authored by researchers from Google Research, DeepMind, and other institutions, analyzed research practices in the field of AI Value Alignment. The researchers examined 94 highly cited papers to trace the value theories implicitly adopted by the field.

Key Findings:

• 79%: Did not define or describe what 'value' is.
• 82%: Used 'preference' as a proxy for value.
• 87%: Adopted 'utility maximization' as the technical framework for alignment.

Core Analysis:

• Technical Reductionism: Modern alignment techniques such as RLHF and DPO assume that human values can be reduced to a scalar reward, a process that tends to omit definitions of the nature of value.
• Absence of a Sociotechnical Perspective: By focusing primarily on mathematical formalization, alignment research overlooks the fact that AI systems are social and cultural products, as well as the system safety issues arising from design decisions.
• Regulatory and Policy Risks: The paper warns against situations where the undefined concept of 'alignment' is used as a basis for legal compliance or government procurement.

The paper emphasizes that alignment research requires critical reflection not only on technical improvements but also on the underlying view of humanity used to operationalize value.