Intelligent Energy and Carbon Management in 6G: A Hybrid DRL-PPO Optimization Framework
Keywords:
6G networks, Deep Reinforcement Learning (DRL), green AI, intelligent sensing, Carbon-Aware NetworkingAbstract
6th generation (6G) networks are anticipated to deliver highly-reliable, low-latency communications (URLLC); therefore, the increased use of Terahertz (THz) transceivers and Holographic MIMO surfaces will result in significant sustainability challenges as a result of the excessive amount of energy these devices will require for their operations. As part of this research, we present a framework using deep reinforcement learning (DRL) techniques known as Proximal Policy Optimization (PPO) that can effectively address two goals (conflicting) - minimizing energy consumption through intelligent channel sensing techniques and minimizing carbon footprints by using grid-aware
resource allocation techniques. We demonstrate how a Carbon-Aware PPO agent can decouple network performance from carbon emissions by collecting operational energy usage records from live 5G networks and dynamically scaling them based on 6G power restrictions. Overall, we found that this approach could reduce carbon emissions approximately 84.5% versus traditional maximum performance baselines without causing any quality of service (QoS) violations; thus, effectively circumventing all limitations of static sleep-scheduling heuristic approaches.
Downloads
References
R. Kumar, S. K. Gupta, H.-C. Wang, C. S. Kumari, and S. S. V. P. Korlam, “From Efficiency to Sustainability: Exploring the Potential of 6G for a Greener Future,” Sustainability, vol. 15, no. 23, p. 16387, 2023.
DOI: https://doi.org/10.3390/su152316387
X. Fernando and G. Lazaroiu, “Energy-Efficient Industrial Internet of Things in Green 6G Networks,” Applied Sciences, vol. 14, no. 18, p. 8558, 2024. DOI: https://doi.org/10.3390/app14188558
L. Ismail and R. Buyya, “Artificial Intelligence Applications and Self- Learning 6G Networks for Smart Cities Digital Ecosystems: Taxonomy,Challenges, and Future Directions,” Sensors, vol. 22, no. 15, p. 5750,
DOI: https://doi.org/10.3390/s22155750
T. Yigitcanlar, R. Mehmood, and J. M. Corchado, “Green Artificial Intelligence: Towards an Efficient, Sustainable and Equitable Technology for Smart Cities and Futures,” Sustainability, vol. 13, no. 16, p. 8952, 2021. DOI: https://doi.org/10.3390/su13168952
M. Murroni, M. Anedda, M. Fadda, P. Ruiu, V. Popescu, C. Zaharia, and D. Giusto, “6G-Enabling the New Smart City: A Survey,” Sensors,vol. 23, no. 17, p. 7528, 2023. DOI: https://doi.org/10.3390/s23177528
W. Konh¨auser, “Digitalization in Buildings and Smart Cities on the Way to 6G,” Wireless Personal Communications, vol. 121, pp. 1289–1302, 2021. DOI: https://doi.org/10.1007/s11277-021-09069-9
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, “Proximal Policy Optimization Algorithms,” arXiv preprint arXiv:1707.06347, 2017. DOI: https://doi.org/10.48550/arXiv.
06347
Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd ed., Cambridge, MA, USA: MIT Press, 2018.
Matti Latva-aho and Kari Lepp¨anen, “Key Drivers and Research Challenges for 6G Ubiquitous Wireless Intelligence,” University of Oulu, Oulu, Finland, Rep. 2019.
Chih-Lin I, Corbett Rowell, Shuangfeng Han, Zhikun Xu, Gang Li, and Zhengang Pan, “Toward Green and Soft: A 5G Perspective,” IEEE Communications Magazine, vol. 52, no. 2, pp. 66–73, 2014. DOI: https: //doi.org/10.1109/MCOM.2014.6736745
J. C. Silva, L. F. M. Vieira, and M. A. M. Vieira, “Deep Reinforcement Learning for Energy Optimization in Next-Generation Networks,” IEEE Latin America Transactions, vol. 20, no. 3, pp. 450–458, 2022. DOI:
https://doi.org/10.1109/TLA.2022.9667145
M. A. Rodriguez, P. H. J. Nardelli, and C. B. Both, “Smart Grid Integration with 6G Architecture for Sustainable Communications,” IEEE Latin America Transactions, vol. 21, no. 1, pp. 112–120, 2023.
DOI: https://doi.org/10.1109/TLA.2023.10015243
L. F. Gomez, A. M. C. Correia, and J. L. S. Ferreira, “Resource Allocation in THz Communications using Machine Learning,” IEEE Latin America Transactions, vol. 19, no. 8, pp. 1305–1312, 2021. DOI:
https://doi.org/10.1109/TLA.2021.9475612
W. Saad, M. Bennis, and M. Chen, “A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems,”IEEE Network, vol. 34, no. 3, pp. 134–142, 2019. DOI: https://doi.org/
1109/MNET.001.1900287
F. Tariq, M. R. A. Khandaker, K. -K. Wong, M. A. Imran, M. Bennis,and M. Debbah, “A Speculative Study on 6G,” IEEE Wireless Communications,vol. 27, no. 4, pp. 118–125, 2020. DOI: https://doi.org/10.
/MWC.001.1900488
M. Elsayed and M. Erol-Kantarci, “AI-Enabled Future Wireless Networks:Challenges, Opportunities, and Open Issues,” IEEE VehicularTechnology Magazine, vol. 14, no. 3, pp. 70–77, 2019. DOI: https://doi.org/10.1109/MVT.2019.2919236
Y. Sun, M. Peng, and S. Mao, “Green AI for 6G Networks: A Comprehensive Survey,” IEEE Communications Surveys & Tutorials, vol. 25,no. 2, pp. 1234–1267, 2023. DOI: https://doi.org/10.1109/COMST.2023.3241567
N. C. Luong et al., “Applications of Deep Reinforcement Learning in Communications and Networking: A Survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019. DOI: https:
//doi.org/10.1109/COMST.2019.2916583
X. Wang, Y. Zhang, and Z. Han, “Carbon-Aware Routing for 6G Satellite-Terrestrial Integrated Networks,” IEEE Transactions on Green Communications and Networking, vol. 7, no. 1, pp. 156–168, 2023. DOI:
https://doi.org/10.1109/TGCN.2022.3218541
J. Zhang et al., “Energy-Efficient THz Communications for 6G: A Deep Learning Approach,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 215–228, 2022. DOI: https://doi.org/10.1109/
JSAC.2021.3126085
