Resource Allocation in Cellular Optical Wireless Systems Using Reinforcement Learning

In modern communication networks, indoor users and their data rate demands are massively increasing. Radio-based systems struggle to meet these demands due to their limited available spectrum. Researchers proposed optical wireless communication (OWC) systems in indoor environments to meet these demands due to their high data rates, reliability and energy efficiency. As the number of users increases, efficient allocation of system resources becomes essential.
The thesis introduces, for the first time a Q-Learning (QL)-based resource allocation algorithm tailored for indoor OWC systems. The proposed method was evaluated on wavelength division multiple access (WDMA)-based visible light communication (VLC) and steerable laser-based OWC systems. The QL approach delivers resource allocation solutions comparable to the optimal solutions achieved by mixed integer linear programming (MILP), while operating without prior environmental knowledge. Nevertheless, the method’s reliance on discrete Q-tables limits its applicability to small-scale environments with fewer users and access points.
To address more complex scenarios, the thesis proposes a deep reinforcement learning (DRL) framework for resource allocation in indoor OWC systems. The DRL approach uses neural networks to replace the Q-table, enabling operation in larger, more dynamic settings. Evaluations show DRL achieves performance close to MILP benchmarks and outperforms simpler heuristics, such as distance-based allocation (DBA). However, its effectiveness depends on careful hyperparameter tuning and model design.
Finally, the thesis integrates artificial neural network (ANN)-based user positioning with DRL to manage resource allocation under user mobility. Using a
V
random waypoint (RWP) mobility model, the hybrid ANN-DRL system significantly improves learning performance in dynamic conditions. Yet, the system assumes a specific room configuration; environmental changes, such as moving furniture or introducing obstacles, may reduce positioning accuracy and require periodic retraining to sustain performance.