Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis
QFlow: A Reinforcement Learning Approach to High QoE Video Streaming over Wireless Networks
R. Bhattacharyya, A. Bura, D. Rengarajan, M. Rumuly, S. Shakkottai, D. Kalathil, R. Mok, and A. Dhamdhere, "QFlow: A Reinforcement Learning Approach to High QoE Video Streaming over Wireless Networks", in ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), Jul 2019.
|   View full paper:    PDF    DOI    |  Citation:    BibTeX   |

QFlow: A Reinforcement Learning Approach to High QoE Video Streaming over Wireless Networks

Rajarshi Bhattacharyya2
Archana Bura2
Desik Rengarajan2
Mason Rumuly2
Srinivas Shakkottai2
Dileep Kalathil2
Ricky K. P. Mok1
Amogh Dhamdhere1
1

CAIDA, San Diego Supercomputer Center, University of California San Diego

2

Texas A&M University, College Station

Wireless Internet access has brought legions of heterogeneous applications all sharing the same resources. However, current wireless edge networks that cater to worst or average case performance lack the agility to best serve these diverse sessions. Simultaneously, software reconfigurable infrastructure has become increasingly mainstream to the point that dynamic per packet and per flow decisions are possible at multiple layers of the communications stack. Exploiting such reconfigurability requires the design of a system that can enable a configuration, measure the impact on the application performance (Quality of Experience), and adaptively select a new configuration. Effectively, this feedback loop is a Markov Decision Process whose parameters are unknown. The goal of this work is to design, develop and demonstrate QFlow that instantiates this feedback loop as an application of reinforcement learning (RL). Our context is that of reconfigurable (priority) queueing, and we use the popular application of video streaming as our use case. We develop both model-free and model-based RL approaches that are tailored to the problem of determining which clients should be assigned to which queue at each decision period. Through experimental validation, we show how the RL-based control policies on QFlow are able to schedule the right clients for prioritization in a high-load scenario to outperform the status quo, as well as the best known solutions with over 25% improvement in QoE, and a perfect QoE score of 5 over 85% of the time.

Keywords: measurement methodology, mobile, QoE
  Last Modified: Fri Nov-15-2019 11:39:30 PST
  Page URL: http://www.caida.org/publications/papers/2019/qflow/index.xml