Optimizing Federated Reinforcement Learning Algorithm for Data Management of Distributed Energy Storage Network

The development of energy storage networks has facilitated the rapid expansion of new energy-based power systems. However, the emergence of large-scale energy storage devices has also led to a significant increase in energy data volumes. Federated learning provides a solution by allowing energy data owners to train AI models without sharing local energy data, which is particularly advantageous for handling heterogeneous data.

This paper explores the application of federated learning in managing energy data within distributed energy storage networks. Specifically, we leverage deep reinforcement learning algorithms to optimize the selection of device subsets, aiming to mitigate data bias caused by non-identically and independently distributed (non-IID) data while enhancing convergence rates.

To achieve our objectives, we employ deep reinforcement learning to dynamically select the optimal subset of devices in the federated learning process. Additionally, we introduce a reputation replay array mechanism to address the issue of free-rider users and ensure fair modeling without payment penalties. We analyze energy data characteristics within distributed energy storage networks and simulate unstructured short data fragments using datasets such as 20 Newsgroups and AG News.

Our experiments show that our proposed model outperforms FedAvg and TiFL on the 20 Newsgroups and AG News datasets, especially under non-iid conditions. Our model significantly reduces communication rounds by up to 47% and 39%, respectively. It also maintains high accuracy and resilience against dishonest nodes, ensuring the quality of the training model.

Our research concludes that combining federated learning with deep reinforcement learning not only solves the problems of data management and privacy protection in distributed energy storage networks, but also promotes the sustainable development of new energy systems.