Abstract
In this paper, we propose a max-min entropy framework for reinforcement learning (RL) to overcome the limitation of the soft actor-critic (SAC) algorithm implementing the maximum entropy RL in model-free sample-based learning. Whereas the maximum entropy RL guides learning for policies to reach states with high entropy in the future, the proposed max-min entropy framework aims to learn to visit states with low entropy and maximize the entropy of these low-entropy states to promote better exploration. For general Markov decision processes (MDPs), an efficient algorithm is constructed under the proposed max-min entropy framework based on disentanglement of exploration and exploitation. Numerical results show that the proposed algorithm yields drastic performance improvement over the current state-of-the-art RL algorithms.
![Seungyul Han and Youngchul Sung, "A max-min entropy framework for reinforcement learning," accepted to Conference on Neural Information Processing Systems (NeurIPS) 2021 1 성영철1](/wp-content/uploads/drupal/성영철1.png)
![Seungyul Han and Youngchul Sung, "A max-min entropy framework for reinforcement learning," accepted to Conference on Neural Information Processing Systems (NeurIPS) 2021 2 성영철2](/wp-content/uploads/drupal/성영철2.png)