IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARL

IMP-MARL offers a platform for benchmarking the scalability of cooperative MARL methods in real-world engineering applications.

In IMP-MARL, you can:

• Implement your own infrastructure management planning (IMP) environment or execute an available IMP environment.
• Train IMP policies through state-of-the-art MARL methods. The environments can be integrated with typical ecosystems via wrappers.
• Compute expert-based heuristic policies

Additionally, you will be able to:

• Retrieve the results of a benchmark campaign, where MARL methods are assessed in terms of scalability.
• Reproduce our experiments.
• Add your results to ours through the plot scripts.

This repository has been developed and is maintained by Pascal Leroy & Pablo G. Morato.

Please consider opening an issue or a pull request to help us improve this repository.

Future developments are described in the roadmap.

Requirements

To work with our environments, one only needs to install Numpy.

However, to reproduce our results, more packages are required and installation instructions are provided here.

Tutorials

• Create your own IMP environment scenario
• IMP’s interface explained
• Train agents like in the paper and/or reproduce the results
• Retrieve the results of the paper and execute the plot scripts

Sets of environments available

• (Correlated and uncorrelated) k-out-of-n system with components subject to fatigue deterioration.
• Offshore wind structural system with components subject to fatigue deterioration.

Note: A campaign cost can be activated in any environment.

Available wrappers and examples

All wrappers are available in imp_wrappers.

• Ready: PyMarl: Multi and single agent wrappers.
• Ready: EPyMarl: Multi agent wrapper.
• Ready: Gym: Single-agent wrapper.
• Ready: Gymnasium: Single-agent wrapper.
• Ready: PettingZoo : Multi-agent wrapper.
• Ready: Rllib: Single-agent training with RLLib and Gymnasium wrapper.
• Ready: MARLlib: Examples include random agents and how to train with MARLlib.
• WIP: TorchRL example: WIP

Run an IMP environment

env = Struct({'n_comp': 3,
               'discount_reward': 0.95,
               'k_comp': 2,
               'env_correlation': False,
               'campaign_cost': False})

obs, done = env.reset(), False
while not done:
    actions = {f"agent_{i}": random.randint(0,2) for i in range(3)}
    obs, rewards, done, insp_outcomes = env.step(actions) 

Citation

If you use IMP-MARL in your work, please consider citing our paper:

IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARL

@inproceedings{
leroy2023impmarl,
title={{IMP}-{MARL}: a Suite of Environments for Large-scale Infrastructure Management Planning via {MARL}},
author={Pascal Leroy and Pablo G. Morato and Jonathan Pisane and Athanasios Kolios and Damien Ernst},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2023},
url={https://openreview.net/forum?id=q3FJk2Nvkk}
}