INTELLIGENT MEDIUM ACCESS CONTROL PROTOCOL FOR UNDERWATER PIPELINE MONITORING NETWORKS

This thesis studies the applications of Reinforcement Learning (RL) in designing an intelligent MAC protocols for linear chain Underwater Acoustic Sensor Networks (UASNs) suitable for marine pipeline monitoring. The key objective is to explore and devise simple strategies that re-imagine RL based algorithms with reduced inefficiencies due to overheads to improve channel utilisation and adaptability. Inspired by the successful implementation of RL on ALOHA in the recently proposed terrestrial ALOHA-Q, we explored the feasibility of applying similar approach in UASNs. The evaluation of ALOHA-Q in UASN, has shown the potential benefits to employing RL for adaptable underwater MAC design, however, new strategies on slot structure and method of feedback are needed for good utilisation. Based on the relationship between packet duration and propagation delay, this thesis proposed two efficient slot structures. The viability of these slot structures are pictorially analysed and empirically evaluated for incorporation in MAC protocol implementation. The thesis presents novel RL based algorithms without any explicit feedback signal. Rather, it exploits packet flow in a two stage mechanism to simultaneously drive a slot selection Q-learning algorithm and a stochastic averaging function that heuristically measured the network wide optimal flow harmony, thereby, effectively creating a simple, powerfully adaptive intelligent scheduling with huge performance improvement.