Project Summary

Measuring the Quality of Experience (QoE) in a real-world environment is challenging. Even though a number of platforms have been deployed to gauge network path performance from the edge of the Internet, one cannot easily infer QoE from that data because of the subjective nature of QoE. On the other hand, crowdsourcing-based QoE assessment, namely QoE crowdtesting, is increasingly popular in conducting subjective assessments for various services, including video streaming, VoIP, and IPTV. Workers on crowdsouring platforms can access and participate in assessment tasks remotely through the Internet. The experimenter can also select a pool of potential workers according to their geolocation or their historical accuracy. Existing QoE crowdtesting mainly evaluates emulated scenarios, instead of studying the impact of Internet events. This is because the launching of QoE crowdtesting is usually not based on network measurement results. Although we can measure network path quality from the workers, it will be difficult to correlate the assessment results with Internet events because of differences in the assessment time and network path being measured.

In this project, we propose a novel framework to launch QoE crowdtesting in a timely manner when adverse network events are detected. We will use existing network measurement infrastructures to detect network events, such as link congestion. Based on information of the events, the framework would initiate QoE crowdtesting to recruit workers who are potentially affected, who then will provide feedback on their perceived QoE. The main advantage of this reactive approach is that it will improve the effectiveness of launching QoE crowdtesting tasks thus helping to evaluate the impact of network events as they occur.

Research Plan

Expected Outcomes

• A platform which can automatically launch QoE crowdtesting according to network events
• A mechanism for creating suitable QoE crowdtesting and recruiting appropriate set of workers from the crowd
• A set of data obtained from the platform and the models derived from them