Skip to content
2000
  1. Home
  2. A-Z Publications
  3. Recent Patents on Electrical & Electronic Engineering (Formerly Recent Patents on Electrical Engineering)
  4. Fast Track Articles
  5. Fast Track Article
image of Optimizing Federated Reinforcement Learning Algorithm for Data Management of Distributed Energy Storage Network

Abstract

Background

The development of energy storage networks has facilitated the rapid expansion of new energy-based power systems. However, the emergence of large-scale energy storage devices has also led to a significant increase in energy data volumes. Federated learning provides a solution by allowing energy data owners to train AI models without sharing local energy data, which is particularly advantageous for handling heterogeneous data.

Objective

This paper explores the application of federated learning in managing energy data within distributed energy storage networks. Specifically, we leverage deep reinforcement learning algorithms to optimize the selection of device subsets, aiming to mitigate data bias caused by non-identically and independently distributed (non-IID) data while enhancing convergence rates.

Method

To achieve our objectives, we employ deep reinforcement learning to dynamically select the optimal subset of devices in the federated learning process. Additionally, we introduce a reputation replay array mechanism to address the issue of free-rider users and ensure fair modeling without payment penalties. We analyze energy data characteristics within distributed energy storage networks and simulate unstructured short data fragments using datasets such as 20 Newsgroups and AG News.

Results

Our experiments show that our proposed model outperforms FedAvg and TiFL on the 20 Newsgroups and AG News datasets, especially under non-iid conditions. Our model significantly reduces communication rounds by up to 47% and 39%, respectively. It also maintains high accuracy and resilience against dishonest nodes, ensuring the quality of the training model.

Conclusion

Our research concludes that combining federated learning with deep reinforcement learning not only solves the problems of data management and privacy protection in distributed energy storage networks, but also promotes the sustainable development of new energy systems.

© 2024 Bentham Science Publishers
Loading

Article metrics loading...

/content/journals/eeng/10.2174/0123520965325440240925051735
2024-10-03
2024-11-23

  • From This Site

    /content/journals/eeng/10.2174/0123520965325440240925051735
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Tan K.M. Babu T.S. Ramachandaramurthy V.K. Kasinathan P. Solanki S.G. Raveendran S.K. Empowering smart grid: A comprehensive review of energy storage technology and application with renewable energy integration. J. Energy Storage 2021 39 102591 10.1016/j.est.2021.102591
    [Google Scholar]
  2. Brickell J. Shmatikov V. The cost of privacy: Destruction of data-mining utility in anonymized data publishing. KDD 2008 70 78 10.1145/1401890.1401904
    [Google Scholar]
  3. Fung B.C.M. Wang K. Yu P.S. Anonymizing classification data for privacy preservation. IEEE Trans. Knowl. Data Eng. 2007 19 5 711 725 10.1109/TKDE.2007.1015
    [Google Scholar]
  4. Chen M. Shlezinger N. Poor H.V. Eldar Y.C. Cui S. Communication-efficient federated learning. Proc. Natl. Acad. Sci. USA 2021 118 17 e2024789118 10.1073/pnas.2024789118 33888586
    [Google Scholar]
  5. Bokkisam H.R. Singh S. Acharya R.M. Selvan M.P. Blockchain-based peer-to-peer transactive energy system for community microgrid with demand response management. CSEE J. Power Energy Syst. 2022 8 1 198 211
    [Google Scholar]
  6. McMahan B. Moore E. Ramage D. Hampson S. Arcas B.A.Y. Communication-efficient learning of deep networks from decentralized data. Proc. 20th Int. Conf. Artif. Intell. Stat. 2017 54 1273 1282
    [Google Scholar]
  7. Smith V. Chiang C-K. Sanjabi M. Talwalkar A. Federated multi-task learning. Proc. Adv. Neural Inf. Process. Syst. 2017 4424 4434
    [Google Scholar]
  8. Bonawitz K. Eichner H. Grieskamp W. Huba D. Ingerman A. Ivanov V. Kiddon C. Konecny J. Towards federated learning at scale: System design. arXiv: 1902.01046, 2019
    [Google Scholar]
  9. Qiu Z. Charging-rate-based battery energy storage system in wind farm and battery storage cooperation bidding problem. CSEE J. Power Energy Syst. 2022 8 3 659 668
    [Google Scholar]
  10. Zhao Z. System optimization of federated learning networks with a constrained latency. IEEE Trans. Vehicular Technol. 2021 99 1 5
    [Google Scholar]
  11. McMahan H.B. Moore E. Ramage D. Arcas B.A.Y. Federated learning of deep networks using model averaging. arxiv 2016
    [Google Scholar]
  12. Lo S.K. Lu Q. Wang C. Paik H. Zhu L. A systematic literature review on federated machine learning: From a software engineering perspective. arxiv 2020
    [Google Scholar]
  13. Kairouz P. Advances and open problems in federated learning arxiv 2019
    [Google Scholar]
  14. Pan Q. Wu J. Bashir A.K. Li J. Yang W. Al-Otaibi Y.D. Joint protection of energy security and information privacy for energy harvesting: An incentive federated learning approach. IEEE Trans. Industr. Inform. 2022 18 5 3473 3483 10.1109/TII.2021.3105492
    [Google Scholar]
  15. Liu Q. Yang B. Wang Z. Zhu D. Wang X. Ma K. Guan X. Asynchronous decentralized federated learning for collaborative fault diagnosis of PV stations. IEEE Trans. Netw. Sci. Eng. 2022 9 3 1680 1696 [J] 10.1109/TNSE.2022.3150182
    [Google Scholar]
  16. Yu H. Liu Z. Liu Y. Chen T. Cong M. Weng X. Niyato D. Yang Q. A Sustainable incentive scheme for federated learning. IEEE Intell. Syst. 2020 35 4 58 69 10.1109/MIS.2020.2987774
    [Google Scholar]
  17. Lei L. Tan Y. Dahlenburg G. Xiang W. Zheng K. Dynamic energy dispatch based on deep reinforcement learning in IoT-driven smart isolated microgrids. IEEE Internet Things J. 2021 8 10 7938 7953 10.1109/JIOT.2020.3042007
    [Google Scholar]
  18. Huang Q. Huang R. Hao W. Tan J. Fan R. Huang Z. Adaptive power system emergency control using deep reinforcement learning. IEEE Trans. Smart Grid 2020 11 2 1171 1182 10.1109/TSG.2019.2933191
    [Google Scholar]
  19. Huang B. Wang J. Deep-reinforcement-learning-based capacity scheduling for PV-battery storage system. IEEE Trans. Smart Grid 2021 12 3 2272 2283 10.1109/TSG.2020.3047890
    [Google Scholar]
  20. Wang H. Kaplan Z. Niu D. "Optimizing federated learning on non-IID data with reinforcement learning", IEEE INFOCOM 2020 - IEEE Conference on Computer Communications, Toronto, ON, Canada, 06-09 July 2020, pp. 1698-1707.
    [Google Scholar]
  21. Zhan Y. Li P. Guo S. "Experience-driven computational resource allocation of federated learning by deep reinforcement learning", 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), New Orleans, LA, USA, 18-22 May 2020, pp. 234-243.
    [Google Scholar]
  22. Zeng M. Wang X. Pan W. Zhou P. Heterogeneous training intensity for federated learning: A deep reinforcement learning approach. IEEE Trans. Netw. Sci. Eng. 2023 10 2 990 1002 [J] 10.1109/TNSE.2022.3225444
    [Google Scholar]
  23. Wei W. Liu F. Mei S. Hou Y. Robust energy and reserve dispatch under variable renewable generation. IEEE Trans. Smart Grid 2015 6 1 369 380 10.1109/TSG.2014.2317744
    [Google Scholar]
  24. Fortenbacher P. Ulbig A. Koch S. Andersson G. “Grid-constrained optimal predictive power dispatch in large multi-level power systems with renewable energy sources and storage devices,” in IEEE PES Innov. Smart Grid Technol. Europe 2014 Oct 1 6
    [Google Scholar]
  25. Zhu L. Liu Z. Han S. "Deep leakage from gradients", Proceedings of the 33rd International Conference on Neural Information Processing Systems Red Hook, NY, USA, 08 December 2019, pp 14774 - 14784.
    [Google Scholar]
  26. Lu Y. Huang X. Dai Y. Maharjan S. Zhang Y. Blockchain and federated learning for privacy-preserved data sharing in industrial IoT. IEEE Trans. Industr. Inform. 2020 16 6 4177 4186 [J] 10.1109/TII.2019.2942190
    [Google Scholar]
  27. Konečný J. McMahan H.B. Yu F.X. Federated learning: Strategies for improving communication efficiency. Preprint arXiv: 1610.05492. 2016
    [Google Scholar]
  28. Chai Z. Ali A. Zawad S. "Tifl: A tier-based federated learning system", Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing Stockholm, Sweden, 23 June 2020, pp.125–136.
    [Google Scholar]
/content/journals/eeng/10.2174/0123520965325440240925051735
Loading
Optimizing Federated Reinforcement Learning Algorithm for Data Management of Distributed Energy Storage Network
Bentham Science Publishers ; https://doi.org/10.2174/0123520965325440240925051735
/content/journals/eeng/10.2174/0123520965325440240925051735
/content/journals/eeng/10.2174/0123520965325440240925051735
Loading

Data & Media loading...


  • Article Type:     Research Article
Keywords: reinforcement learning ; energy storage ; Data management ; federated learning
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test