AI Briefing[AAMAS 2024] Multi-Agent Reinforcement Learning for Real-World Applications (Part 2)
Key point
To overcome differences in observation timing among agents in asynchronous environments, an AOCC structure was proposed to improve MARL learning efficiency.
Details
In real-world problems, Asynchronous MARL situations frequently occur where each agent has different action durations or starting points. Existing studies have utilized Mac-CERTs, independent histories per agent, and Mac-JERTs, the combined history of all agents, to process asynchronously collected data.
However, the existing Mac-JERTs approach had a limitation in that it failed to effectively represent history by using the last observation as a duplicate input when processing agent records at points where there was no latest observation.
To solve this, LG AI Research proposed the Agent-Oriented Centralized Critic (AOCC) structure. This structure consists of the following key components.
- Agent-oriented History Encoder: Encodes the per-agent history composed of Mac-CERTs.
- Aggregation Module: Aggregates the history encoding of each agent, adding Positional Encoding to prevent confusion between time points.
This structure avoids duplicated observation inputs and enables more effective history composition, improving the performance of value function and action policy learning.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
DeepSeek Releases DSec: Elastic Sandbox Infrastructure for Large-Scale Agentic Training
Trajectory Introduces Density-Aware Training to Optimize AI Cost-Per-Task