On March 26-27, 2014, CAIDA hosted the 6th workshop on Active Internet Measurements (AIMS-6) supporting science and policy.
AIMS 2014 was by invitation only. Registration is closed.
Place: Auditorium B210E/B211E Meeting Room,
San Diego Supercomputer Center, UCSD Campus, La Jolla, CA
This year, we want to explore existing capabilities and potential opportunities for network measurement in the wireless domain, and research infrastructure to support it. One motivation is the increasing attention to measuring the character of broadband access, including the U.S. FCC Measuring Broadband America effort (for both wireline and wireless connectivity). The more complex structure of cellular access, with its signalling protocols and potential to shape different sorts of application traffic, makes it harder to decide what to measure in order to characterize or compare wireless service.
Our goals for this two day workshop are: (1) to understand the wireless research infrastructure landscape(s), and measurement capabilities that support (or should support) it; (2) to determine what are the important questions that measurement can help answer who will be the customer for the measurements; and (3) to propose a vision/roadmap for wireless measurement research infrastructure and activities for the next decade. Key topics to be discussed include:
- research infrastructure to support mobile/wireless measurement
- review existing wireless research infrastructures
- what specific questions are being investigated with current wireless measurement research activities?
- what large-scale data respositories of wireless measurements exist and how are they used?
- who is funding work in this space?
- what does the research community need? who can help?
- mobile/wireless measurement capabilities/tools
- articulate what we can measure about wired and wireless networks from today's mobile devices, e.g., signal strength, coverage, performance, including latency (bufferbloat) and throughput (and relationship to QoE measurements), spectrum utilization, network topology, control signaling overhead, (device) energy usage, cross-layer measurements
- compare priorities and capabilities across different wireless media and protocol layers
- measurement from the edge
- how much measurement is occuring internal to commercial networks, for operational purposes or as part of developing new technology?
- which measurement questions are important to different parts of the ecosystem: carrier, consumer, regulator?
- incentives to deploy tools
- incentives to get enough people or the right people to deploy tools
- identification and mediation of privacy issues in mobile/wireless measurements
- incentives to share data
- monitoring spectrum usage
- what technologies, tools, or activities exist to support monitoring usage of emerging shared spectrum, e.g., whitespaces.
- how do operators of databases maintaining spectrum utilization information validate their data?
- how is the data being used or how could it be used?
- discuss opportunities for validation of mobile measurements of various types
- how much validation is happening/possible today?
Unlike a traditional workshop where attendees present talks on their work, we will have a series of sessions on the pre-selected topics with presenters preparing short talks and subsequent discussion. Not everyone necessarily gives a talk, but everyone would be expected to participate in the discussions. This format lets us dig deeper into the selected topics so that we can try to develop some consensus on a wireless measurement research agenda that may require concerted infrastructure investment.
All talks are 10 minutes long (unless indicated otherwise), plus 5 minutes for questions after each talk. Additionally, the following ground rules apply for all talks:
- include one slide on data sharing
- include one slide on visualization of data (extra credit)
- last slide: what you want to get from the audience / workshop
To generate discussion and to orient other participants to your talk, please send a URL or a PDF to firstname.lastname@example.org of something you'd like the audience to have read before your talk. This can be any of:
- a related URL that inspires your research
- a related URL detailing your research
- a URL related to your talk that you consider worth other participants' time to look over
- a recent blog entry or article so people can get an idea of who you are
- the actual PDF slideset which you'll be presenting
March 26 (Wednesday)
- 08:00 - 08:30 breakfast
- 08:30 - 09:00 Round of introductions: "What I want to get out of the workshop"
- 09:00 - 11:15 Observed Mobile Broadband Performance Around the World
Session Chair: kc claffy (CAIDA/UCSD)
- Jim Warner (UCSC), Wireless carrier claims vs. customer performance
- Walter Johnston (FCC), Measuring Broadband America
- Ahmed Elmokashfi (Simula Research Laboratory), Mobile Broadband reliability: a user perspective
- 09:45 Discussion
- Can we tease apart issues in the air protocols from the provisioning of the access backhaul and other aspects of the access system?
- What can we say about evolution of performance, and how to best quantify performance?
10:30 - 11:00 break
- 11:15 - 12:30 Non-Mobile Wireless Measurement
Session Chair: Aaron Schulman (Stanford University)
- Craig Partridge (Raytheon BBN Technologies), Network Measurement in a World of Software Defined Radios (15 min)
- Aaron Schulman (Stanford University), PlanX: Enabling Innovative Measurements of Wireless Networks (15 min)
- Nick Feamster (Georgia Tech), WTF? Locating Performance Problems in Home Networks (using SamKnows measurement infrastructure)
- 12:00 Discussion
- What instrumentation is needed for network monitoring of smart-grids and internet of things?
- What are the most useful measurement capabilities for spectrum agile devices?
12:30 - 13:30 lunch
- 13:30 - 16:30 Collecting Measurements on User Smartphones
Session Chair: Ethan Katz-Bassett (University of Southern California)
- Ethan Katz-Bassett (University of Southern California) and David Choffnes (Northeastern University), Measuring the Mobile Internet
- Valerio Luconi (University of Pisa), Portolan: Smartphone-based Crowdsourcing for Network Sensing
- Narseo Vallina Rodriguez (ICSI, UC Berkeley), Netalyzr for Android: challenges and opportunities
- Mike Wittie (Montana State University), MITATE: Mobile Internet Testbed for Application Traffic Experimentation
- Sharad Agarwal (Microsoft Research), mobile measurements: the mobile app / OS perspective
- 14:30 Discussion
- How can interactive applications prosper in mobile networks with aggressive traffic shaping policies?
- How network performance translates to mobile app and mobile web performance?
- What are the state-of-the-art mobile measurement tools, analysis, techniques, what is progress blocked on?
- What are the challenges of mobile performance measurement?
- How to design an effective measurement tool that can help
- measure Internet performance on a wide scale, and
- identify anomalies
- diagnose the reason behindobserved anomalies and performance degradations
- incentives to attract users/measurements over space and time
- incentives to get researchers to contribute measurements into a common repository
15:30 - 16:00 break
- 16:30 - 17:45 Spectrum Utilization
Session Chair: David Reed (University of Colorado Boulder)
- Ranveer Chandra (Microsoft Research), Microsoft Spectrum Observatory
- Walter Johnston (FCC), Increasing Mobile Spectrum Availability
- Hiroyuki Ishii (DOCOMO Innovations, Inc.), 5G and Wireless Measurements
- 17:10 Discussion
- What databases exist for spectrum utilization? Who maintains them? How are they used?
- 18:00 - 19:00 Student Poster Session
- Sanae Rosen (University of Michigan), Measuring performance impacts of RRC state transitions in cellular networks with user devices
- Xing Xu (University of Southern California), Investigating Performance Enhancing Proxies in Cellular Networks
- Abbas Razaghpanah (Stony Brook University), Identifying Traffic Differentiation on Cellular Networks
- Ashkan Nikravesh (University of Michigan), Toward Meaningful Mobile Network Performance Measurements
- Hongyi Yao (RobustNet Group, Univerisity of Michigan), A Network Measurement Library for Android Platform
18:30 - 20:00 Reception on-site
March 27 (Thursday)
- 08:00 - 08:30 breakfast
- 08:30 - 09:00 Roundtable: "What I learned from the workshop yesterday"
- 09:00 - 10:30 Federal Agencies Collaboration and Support for Innovation and Measurements
Session Chair: kc claffy (CAIDA/UCSD)
- Vijayarangam Subramanian (NTIA), Wireless Usage Measurement: Research Needs and Support for National Policy and Rule Making
- Ann Cox (DHS), What is role of federal government in mobile measurement space?
- Michael Piatek (Google), Measurement @ Google
- 10:00 Discussion
- What measurements and instrumentation are needed to support large-scale network performance research?
- How can funding agencies help with the incentive (to deploy tools) problem?
- 10:00 - 10:30 break
- 11:00 - 12:30 Available Mobile and Wireless Testbed Research Infrastructures
Session Chair: James Martin (Clemson University)
- James Martin (Clemson University), SciWiNet: Available Model Mobile and Wireless Testbed Research Infrastructure
- Justin Cappos (New York University), Programmatically Deploying Code on End User Devices: Seattle and Sensibility Testbed
- Jacobus Van der Merwe (University of Utah), PhantomNet: An end-to-end mobile network testbed
- 12:00 Discussion
- Multidisciplinary economic/policy challenges of deploying/managing/using (for market and policy decision making) a distributed mobile/wireless measurement platform
- 12:30 - 14:00 lunch
- 14:00 - 16:30 Economic and Policy Challenges
Session Chair: William Lehr (MIT)
- William Lehr (MIT), Mobile Broadband and Wireless Measurement Infrastructure: Policy Challenges and Opportunities
- David Reed (University of Colorado Boulder), Wi-Fi as a Commercial Service: New Technology and Policy Implications
- Bendert Zevenbergen (Oxford Internet Institute), Ethical privacy framework for network researchers collecting, measuring, analysing, sharing mobile connectivity data
- 14:45 Discussion
- What responsibilities should exist in terms of privacy protection and informed consumer consent in the collection and retention of mobile broadband data?
- What privacy-preserving data-sharing among/with researchers currently occurs?
- How should implementable policy be best developed in this area?
- How can we motivate wireless carrier cooperation?
- How can we make data easy to digest for users and policymakers?
- Balancing utility and risk in data sharing - Erin Kenneally's Less is More: A Disclosure Control Approach to Data Sharing
15:30 - 16:00 break
- 16:30 - 17:00 Adjourn, wrap-up, fill out survey
For this workshop, attendees are expected to make their own hotel reservations and transportation arrangements from their hotels to the workshop. For CAIDA's list of local hotels including shuttle availability, see the updated Local Hotels list (PDF). Contact the hotel directly for hotel shuttle schedules (if available) to the San Diego Supercomputer Center (SDSC).
This workshop is being held in the SDSC East Auditorium (Room B210E/B211E) that faces Hopkins Drive.
(For those GPS-enabled attendees, the GPS coordinates near the SDSC Auditorium is WGS84: 32°53'03.77"N, 117°14'20.31"W)
General driving directions to SDSC are located on the CAIDA Contact and Visitor Info page.
- Parking on campus
The most convenient parking is in the Hopkins parking structure at Hopkins Dr and Voigt Dr, just south of SDSC.
Parking Permits: Parking permits are required to park on UCSD Campus. On arrival to campus on the morning of Day 1, check in with a CAIDA staff member at the small turn-in driveway in front of the stairs of the SDSC building on Hopkins Drive. We will give you a parking permit for the day, and then point you to the Hopkins Parking Structure for parking.
Parking permits for subsequent days will be provided at the end of Day 1, just prior to the Reception.
After picking up your parking permit, it is recommended you go to the Hopkins Parking Structure next to SDSC and park on the lower levels. Walk back the way you came out of the bottom of the parking garage (level 2), and back to the stairs of the SDSC building. The auditorium is on the left just before the stairs, labeled Auditorium or B210E/B211E Meeting Room.
The AIMS 2014 workshop will be held in the SDSC Auditorium.
For transportation concerns, general questions and help, contact Cindy Wong at <cindy at caida.org> or (858) 534-5109.
Funding for this event is provided by the National Science Foundation and the U.S. Department of Homeland Security Science and Technology Directorate.