A Reinforcement Learning–Driven Scheduler for Minimizing Uplink Delay in 5G Networks

Abstract

Uplink scheduling is a core challenge in 5G New Radio (NR), where diverse services—enhanced Mobile Broadband (eMBB), ultra-Reliable Low-Latency Communications (URLLC), and massive Machine-Type Communications (mMTC)—compete for shared spectrum under stringent delay and reliability constraints. Traditional policies (Round Robin, Best-CQI, Proportional Fairness) are simple and effective in limited regimes, but they expose well-known drawbacks in the uplink: RR preserves opportunity fairness yet struggles to suppress backlog under load; Best-CQI maximizes instantaneous rate but can starve cell-edge users; and PF lacks the agility to re-prioritize when traffic mixes or deadlines shift, leading to elevated tail latencies and delay-budget violations. This paper proposes an adaptive reinforcement-learning (RL) scheduler based on on-policy SARSA to minimize uplink delay while maintaining efficient spectrum use. The state encodes per-UE buffer status reports (BSR) and achievable rates (quantized for tractability), actions select a UE per resource-block group (RBG) at each slot, and a delay-aware reward (negative sum of BSRs) directly penalizes aggregate backlog. We implement the scheduler in a slot-driven 5G NR simulator with asynchronous HARQ and compare against RR, Best-CQI, and backpressure. Beyond average BSR, we evaluate end-to-end (E2E) delay, 95th/99th-percentile latency, URLLC delay-violation ratio (DVR), eMBB throughput, and mMTC delivery ratio. To address scalability and realism, we extend experiments from 4 UEs to 16–64 UEs and include mixed eMBB/URLLC/mMTC traffic. Results show that SARSA nearly matches RR on mean and tail delay while substantially reducing URLLC DVR relative to Best-CQI; under mixed traffic it preserves URLLC reliability close to RR yet improves eMBB throughput via opportunistic allocations. Stability analyses under high offered load and fast fading indicate bounded queues and improved robustness compared with Best-CQI and backpressure. These findings demonstrate a practical path to learning-enhanced, delay-conscious uplink scheduling within standards-conformant 5G stacks.