AI BriefingPatrick Collison argues AI's positive future scenario is opaque and AI moral worth poses high-stakes utilitarian risks
·2026.11.09 03:22
Key point
Collison notes that while AI risks are well-defined, the specific equilibrium of a positive future remains unclear, and granting AIs moral worth could dominate utilitarian calculations due to their vast projected numbers.
Details
- The positive scenario for AI is difficult to visualize; common benefits like curing diseases are merely ingredients, leaving key questions about human work, resource allocation, and control unanswered.
- Granting moral worth to AIs creates a high-stakes utilitarian problem: because AIs are expected to be vastly more numerous (potentially quadrillions) and intelligent, even a small amount of moral worth would make their aggregate value outweigh that of humans.
- Resolving the tension of AI moral worth requires rejecting one of three propositions: that AIs can have moral worth, the validity of utilitarianism, or the likelihood that AIs will far outnumber humans.
- Current public debate focuses on immediate issues like datacenters, but popular support for AI may shift as people engage with these medium-term questions about societal structure and ethical status.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Gemini uses personal account chat history for model training by defaultAnthropic Hosts Private Meetings with Religious Scholars on AI Consciousness and Moral Formation
Microsoft AI CEO Criticizes Anthropic: 'Model Welfare' Training Is Dangerous