Deep Learning-Based Joint Optimization of Modulation and Power for Nonlinearity-Constrained System

For wireless communication systems with a long distance or severe interference, the insufficient transmit power limits the system performance. In this case, the maximum transmit power depends on the nonlinearity and the saturation region of the power amplifier (PA), which is referred to as a nonlinearity-constrained problem in this paper. To increase the transmit power as high as possible in a nonlinearity-constrained system, this paper proposes an autoencoder-based system to jointly optimize the modulation scheme and transmit power. The optimal solution can achieve a tradeoff between increasing the transmit power and reducing the nonlinear distortion. Meanwhile, the optimized signal constellation and the neural network-based receiver can effectively improve the capacity against nonlinear distortion. The simulation results indicate that the proposed method outperforms conventional methods in terms of symbol error rate (SER) and transmit power, and the SER of the proposed method is close to the SER lower bound of the nonlinear PA.