Reinforcement learning and swarm intelligence for cooperative aerial navigation and payload transportation

This thesis presents comprehensive studies on the development of an autonomous, scalable and adaptive control system for cooperative transport tasks including navigation to the grasping points, manipulation and transport with multiple unmanned aerial vehicles (UAVs). It proposes deep reinforcement learning (RL) and swarm intelligence methods to solve the complex challenges in the dynamic multi-robot system. The solution of swarm intelligence is proposed by developing a new hierarchical behavioural control framework that synthesises different swarm behaviours for different functions of cooperative transportation. The theoretical contribution includes the derivation of a static-equilibrium transport formation for large-scale UAVs on a payload, which can be achieved by the decentralised velocity control from the hierarchical controller. Compared with other formations, the simulation results demonstrate that this approach improves the stability of cable-suspended payload during transportation by quadrotors.

The learning-based controller is hard to deploy in the industry as it suffers from convergence and nonstationarity issues. To address the problems, multi-agent RL policies are developed to navigate multiple quadrotors to the grasping points of a payload. The demonstration learning and curriculum learning algorithms are incorporated into the control framework to guide policy training and facilitate knowledge transfer in different scenarios. The experiments have been validated by various payloads in simulation and then transferred into the real world with Crazyflie quadrotors. The collision-free multi-robot navigation is achieved with up to 90% success rate and less than 0.2-metre deviation.

The capability to effectively respond to variations caused by disturbances such as wind gusts is critical for the safe group transportation of a cable-suspended payload by UAVs. To deal with variations, the control system is designed by an adaptive trajectory tracking controller based on meta-model-based reinforcement learning with online adaptation and a novel correction policy, and a path planner that can sample collision-free goal states of the system for the controller based on the meta-collision predictor. The proposed controller reduces the average payload tracking error to less than 0.1 metres in tasks without obstacles. Furthermore, the trajectory tracking controller can effectively track the payload while ensuring collision-free safety of the dual-UAV-payload system during navigation among obstacles with more than 80% success rate.