INTELLIGENT MEDIUM ACCESS CONTROL PROTOCOL FOR UNDERWATER PIPELINE MONITORING NETWORKS

This thesis studies the applications of Reinforcement Learning (RL) in designing an intelligent
MAC protocols for linear chain Underwater Acoustic Sensor Networks (UASNs) suitable
for marine pipeline monitoring. The key objective is to explore and devise simple strategies that
re-imagine RL based algorithms with reduced inefficiencies due to overheads to improve channel
utilisation and adaptability. Inspired by the successful implementation of RL on ALOHA in
the recently proposed terrestrial ALOHA-Q, we explored the feasibility of applying similar approach
in UASNs. The evaluation of ALOHA-Q in UASN, has shown the potential benefits to
employing RL for adaptable underwater MAC design, however, new strategies on slot structure
and method of feedback are needed for good utilisation.
Based on the relationship between packet duration and propagation delay, this thesis proposed
two efficient slot structures. The viability of these slot structures are pictorially analysed
and empirically evaluated for incorporation in MAC protocol implementation. The thesis
presents novel RL based algorithms without any explicit feedback signal. Rather, it exploits
packet flow in a two stage mechanism to simultaneously drive a slot selection Q-learning algorithm
and a stochastic averaging function that heuristically measured the network wide optimal
flow harmony, thereby, effectively creating a simple, powerfully adaptive intelligent scheduling
with huge performance improvement.