Fast Learning Cognitive Radios in Underlay Dynamic Spectrum Access: Integration of Transfer Learning into Deep Reinforcement Learning

Cognitive radio (CR) as a leading technology in realizing dynamic spectrum access (DSA) offers a large efficiency and flexibility in the use of radio spectrum. The cognition feature embedded in CR dictates the need to learn from the effects of past actions on the environment to take more intelligent decisions in the future until finding the best policy to perform at a given environment condition. However, finding the best policy usually leads to long learning times. This paper studies how to accelerate the learning process in underlay CR when the nodes are equipped with a cognitive engine implementing a deep reinforcement learning algorithm for distributed resource allocation. Operating under underlay DSA, the CRs will adapt transmission parameters to maximize the average quality of experience (QoE) while satisfying both the underlay DSA interference constraint and the end-to-end transmitted traffic delay requirement. The study shows that the number of iterations for the algorithm to converge is reduced, without a sacrifice in average QoE, by transferring to a new node joining the network the knowledge learnt by the CRs already operating in the network. This paper further identifies the best practices to transfer knowledge between CRs so as to reduce communication overhead.