Is there a plan to support Many-to-Many (N-to-N) interactions

[jia06dhadkfbnuas]

Hi there,

I am planning to extend the current project to support Many-to-Many (N-v-N) air combat scenarios.

My Core Question: Regarding the Reward Function: If I switch from 1v1 to a multi-agent setting, is a complete redesign of the reward structure necessary?

I am concerned about issues like:

Credit Assignment: How to balance individual rewards vs. team rewards.

Cooperation: Encouraging covering teammates rather than just selfish survival.

Does the current codebase have any built-in mechanisms (like shared rewards or communication channels) to support MARL (Multi-Agent RL)? Or do you have any recommendations on how to adjust the reward function for this specific environment?

Thanks for your help!

[xcwoid]

Hello,
"My Core Question: Regarding the Reward Function: If I switch from 1v1 to a multi-agent setting, is a complete redesign of the reward structure necessary?"
No, you can just keep the reward at the end that represents the whole team.

"Credit Assignment: How to balance individual rewards vs. team rewards."
One way is to see how often your agent ended up in the enemy's weapon engagement zone for a negative reward and vice versa for a positive reward.

I plan to rewrite the repo to make it more modular, to have fight dynamics models on the lowest level, fdm objects on the higher level, and the agent highest up. It should be much easier to work with.

I will keep in mind the N-vs-N scenario

[jia06dhadkfbnuas]

"Credit Assignment: How to balance individual rewards vs. team rewards." One way is to see how often your agent ended up in the enemy's weapon engagement zone for a negative reward and vice versa for a positive reward.
Thank you for the suggestion. Analyzing the frequency of positive and negative rewards is indeed helpful for evaluating agent behavior in the engagement zone. Following this line of thought, I would like to clarify: in a multi-v-multi confrontation scenario, is it feasible to achieve joint optimization of offense and defense? My understanding is that we do not need to train attack and defense policies separately; rather, the agents can learn to balance both through a unified reward function during interaction. Is this understanding correct?
I plan to rewrite the repo to make it more modular, to have fight dynamics models on the lowest level, fdm objects on the higher level, and the agent highest up. It should be much easier to work with.
Thank you for the update! That sounds like a fantastic refactoring plan. Decoupling the fight dynamics from the agents will definitely make the codebase much more modular and extensible. I really appreciate your effort on this and look forward to the new version!
I will keep in mind the N-vs-N scenario
I'm really looking forward to the N-vs-N scenario.