RL-FN: A META-REINFORCEMENT FAST-LEARNING NETWORK FOR ENERGY-EFFICIENT AND ADAPTIVE QOSAWARE ROUTING IN FOG-ENABLED IOT NETWORKS

Abstract

While fog computing moves computation to the frontier of the Internet of Things (IoT) devices, routing tasks across a variety of fog nodes is challenging given the conflicting nature of latency, energy, and reliability requirements and the dynamic nature of network conditions. However, statically trained machine-learning routers are unable to adapt once deployed and deep reinforcement learning is too slow to train on fog gateways. In this paper we extend our work on Meta-Reinforcement Fast-Learning Network (RL-FN) that combines a fast neural network using Extreme Learning Machine (ELM) with model-free Q-learning. Fog gateway agents are approximated by a closedform, regularised ELM, and their Q-policy is refined by a normalised multi-objective reward which punishes for delay, energy and response time. The extension features a formal delay, queueing and energy model, an annealed exploration schedule, a deep Q-network (DQN) baseline, convergence analysis, computational-cost analysis, reward ablations and hidden-layers ablations up to 100 fog nodes. With approximately 18,000 task records, RL-FN achieves 1.3ms end-toend delay, 99% packet delivery ratio and 98% throughput at 1500 iterations, reducing the delay by 67.5% and 33.3% compared to AdaBoost and DQN, respectively, while maintaining 39.1% more residual energy than XGBoost, and training 160 times faster than DQN