Investigating cooperative behaviour in multi-agent reinforcement learning through task-oriented scenarios

Abstract

Collaboration is a fundamental aspect of both natural and artificial environments, allowing individuals to achieve objectives that would otherwise be difficult or impossible to accomplish alone. In multi-agent systems, effective collaboration enables agents to combine their strengths and co-ordinate their actions to solve complex problems, often resulting in more efficient solutions compared to single-agent approaches. In the context of Multi-Agent Reinforcement Learning (MARL), collaboration is particularly challenging due to the large number of environmental factors that must be learned. Agents must not only learn how they can interact with the environment around them, but also how the environment changes as other agents interact with it. These challenges are further amplified as the number of agents and the complexity of the objectives increase, leading to exponentially larger action spaces and more precise co-ordination requirements. This work investigates the dynamics of collaboration in MARL, focusing on cooperative tasks within a simulated environment. A custom 2D Unity environment was built in which agents must work together to accomplish the shared goal of moving boxes to designated target locations. The environment features two types of boxes, each with a different method of interaction, with certain boxes requiring multiple agents to move them simultaneously. This research aims to understand the necessary conditions for agent cooperation in tasks involving both physical manipulation and co-ordination. Furthermore, by altering the different environmental conditions, we can observe the factors that affect the agents’ ability to learn to co-operate, thereby enabling future research to more effectively design their systems and achieve improved results.