基于深度强化学习的RIS辅助通感融合网络：挑战与机遇
基于深度强化学习的RIS辅助通感融合网络：挑战与机遇
随着深度强化学习（DRL）技术的广泛应用，基于DRL的可重构智能表面（RIS）辅助的通信感知一体化（ISAC）展现出巨大的潜力。然而，由于数据卸载和模型训练的高成本，基于现有ISAC框架实现网络智能仍面临着巨大的挑战。本文深入分析了DRL技术在ISAC领域的应用，探讨了RIS辅助的ISAC建模及其解决方案，并讨论了未来发展趋势。
摘要
随着深度强化学习(DRL)技术的广泛应用，基于DRL的可重构智能表面(RIS)辅助的通信感知一体化(ISAC)展现出巨大的潜力。然而，由于数据卸载和模型训练的高成本，基于现有ISAC框架实现网络智能仍面临着巨大的挑战。为了克服该问题，本文深入分析了DRL技术在ISAC领域的应用，探讨了RIS辅助的ISAC建模及其解决方案，该技术能够解决覆盖区域受限、算法复杂度高以及高频传输等问题。为了推动这些技术的实施，本文进一步讨论了RIS辅助ISAC网络中DRL技术的未来发展趋势，包括潜在的应用和需要解决的问题。
RIS辅助ISAC系统架构与设计
图 1 RIS辅助ISAC系统的应用演示场景
RIS辅助ISAC系统的DDPG设计
图 2 RIS辅助ISAC系统的DDPG设计
RIS辅助ISAC系统的算力网络
图 3 RIS辅助ISAC系统的算力网络
损失函数值与迭代次数的关系
图 4 损失函数值与迭代次数的关系
不同ISAC方案雷达探测率与迭代次数的关系
图 5 不同ISAC方案雷达探测率与迭代次数的关系
基于深度学习的RIS辅助通信的最新进展
优化指标 | RIS指标 | DRL指标 | 场景 | 技术 | 结果 |
|---|---|---|---|---|---|
最大化保密率[9] | 相移控制波束赋形设计 | 参数设计 | BS到RIS多用户 | DQNMDP | 提高保密率 |
最大化加权和速率[11] | 相移控制波束赋形设计 | 参数设计 | BS到RIS多用户 | 比较检索法DQNN | 提高加权和速率 |
提高频谱效率[14] | 相移控制 | 互信息优化 | BS到用户 | DDPGTD3 | 提高频谱效率 |
最大化和速率[15] | 相移控制波束赋形设计 | 环境学习 | RIS到用户多用户 | NOMA协议DRSAC | 提高和速率 |
最大化总保密率[16] | 相移控制波束赋形设计 | 环境学习参数设计 | 多用户多窃听者 | DDPGMDP | 提高总保密率 |
参数设置
参数名称 | 参数值 |
|---|---|
RIS个数 | 10 |
训练大小 | 1000 |
训练学习率 | 0.001 |
迭代次数 | 20 |
测试集样本个数 | 500 |
参考文献
[1] LIU An, HUANG Zhe, LI Min,et al. A survey on fundamental limits of integrated sensing and communication[J].IEEE Communications Surveys & Tutorials, 2022, 24(2): 994–1034. doi:10.1109/COMST.2022.3149272.
[2] CHEN Zhen, TANG Jie, HUANG Lei,et al. Robust target positioning for reconfigurable intelligent surface assisted MIMO radar systems[J].IEEE Transactions on Vehicular Technology, 2023, 72(11): 15098–15102. doi:10.1109/TVT.2023.3284454.
[3] MEALEY R M. A method for calculating error probabilities in a radar communication system[J].IEEE Transactions on Space Electronics and Telemetry, 1963, 9(2): 37–42. doi:10.1109/TSET.1963.4337601.
[4] ZHANG J A, RAHMAN M L, WU Kai,et al. Enabling joint communication and radar sensing in mobile networks—a survey[J].IEEE Communications Surveys & Tutorials, 2022, 24(1): 306–345. doi:10.1109/COMST.2021.3122519.
[5] TONEX. Introduction to 6G | IMT-2030[EB/OL]. https://www.tonex.com/training-courses/introduction-to-6g-imt-2030/, 2020.
[6] CHEN Zhen, HUANG Lei, XIA Shuqiang,et al. Parallel channel estimation for RIS-assisted internet of things[J].IEEE Transactions on Intelligent Transportation Systems, 2024, 25(8): 9762–9773. doi:10.1109/TITS.2024.3364248.
[7] Communication Network., ZTE and China Unicom Achieve World's First 5G Mid-Band Network Verification of Reconfigurable Intelligent Surface in External Networks[EB/OL] https://www.c114.com.cn/news/127/a1167167.html, 2021.Communication Network.,ZTE and China Unicom Achieve World's First 5G Mid-Band Network Verification of Reconfigurable Intelligent Surface in External Networks[EB/OL] https://www.c114.com.cn/news/127/a1167167.html, 2021.
[8] CHEN Zhen, TANG Jie, ZHANG Xiuyin,et al. Hybrid evolutionary-based sparse channel estimation for IRS-assisted mmWave MIMO systems[J].IEEE Transactions on Wireless Communications, 2022, 21(3): 1586–1601. doi:10.1109/TWC.2021.3105405.
[9] HUANG Chongwen, MO Ronghong, and YUEN C. Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning[J].IEEE Journal on Selected Areas in Communications, 2020, 38(8): 1839–1850. doi:10.1109/JSAC.2020.3000835.
[10] XU Wangyang, AN Jiancheng, XU Yongjun,et al. Time-varying channel prediction for RIS-assisted MU-MISO networks via deep learning[J].IEEE Transactions on Cognitive Communications and Networking, 2022, 8(4): 1802–1815. doi:10.1109/TCCN.2022.3188153.
[11] YANG Helin, XIONG Zehui, ZHAO Jun,et al. Deep reinforcement learning-based intelligent reflecting surface for secure wireless communications[J].IEEE Transactions on Wireless Communications, 2021, 20(1): 375–388. doi:10.1109/TWC.2020.3024860.
[12] XU Wangyang, GAN Lu, and HUANG Chongwen. A robust deep learning-based beamforming design for RIS-assisted multiuser MISO communications with practical constraints[J].IEEE transactions on Cognitive Communications and Networking, 2022, 8(2): 694–706. doi:10.1109/TCCN.2021.3128605.
[13] DEMIR Ö T and BJÖRNSON E. Is channel estimation necessary to select phase-shifts for RIS-assisted massive MIMO?[J].IEEE Transactions on Wireless Communications, 2022, 21(11): 9537–9552. doi:10.1109/TWC.2022.3177700.
[14] SAIKIA P, SINGH K, TAGHIZADEH O,et al. DRL algorithms for efficient spectrum sharing in RIS-aided MIMO radar and cellular systems[C]. MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM), Rockville, USA, 2022: 55–60. doi:10.1109/MILCOM55135.2022.10017985.
[15] ZHAO Jingjing, YU Lanchenhui, CAI Kaiquan,et al. RIS-aided ground-aerial NOMA communications: A distributionally robust DRL approach[J].IEEE Journal on Selected Areas in Communications, 2022, 40(4): 1287–1301. doi:10.1109/JSAC.2022.3143230.
[16] PENG Zhangjie, ZHANG Zhibo, KONG Lei,et al. Deep reinforcement learning for RIS-aided multiuser full-duplex secure communications with hardware impairments[J].IEEE Internet of Things Journal, 2022, 9(21): 21121–21135. doi:10.1109/JIOT.2022.3177705.
[17] 张在琛, 江浩. 智能超表面使能无人机高能效通信信道建模与传输机理分析[J]. 电子学报, 2023, 51(10): 2623–2634. doi:10.12263/DZXB.20221352.ZHANG Zaichen and JIANG Hao. Channel modeling and characteristics analysis for high energy-efficient RIS-assisted UAV communications[J].Acta Electronica Sinica, 2023, 51(10): 2623–2634. doi:10.12263/DZXB.20221352.
[18] ZHOU Hao, EROL-KANTARCI M, LIU Yuanwei,et al. Heuristic algorithms for RIS-assisted wireless networks: Exploring heuristic-aided machine learning[J].IEEE Wireless Communications, 2024, 31(4): 106–114. doi:10.1109/MWC.010.2300321.
本文原文来自《电子与信息工程学报》