NSF Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021)

From January 11-12, 2021, we hosted the NSF-funded Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021) online via a closed-session teleconference. The session will not be recorded for public rebroadcast.

This is the first in a series of WOMBIR workshops. The second workshop, WOMBIR-2, is to be held in April.

Date: Jan 11 (Mon) - 12 (Tue) 2021, 11am EST (8am PST) - 3:30pm EST (12:30pm PST)
Place: Closed-session Video Teleconference via Zoom


A large part of computer science is the discovery and transition to practice of new concepts, advances in performance, and derivation of foundational principles. However, another essential part of the computer science agenda is to understand the behavior of large systems that arise from our earlier innovation. The Internet is the best example of a computer-science artifact so complex that its behavior cannot be derived from the specification of its components. One has to measure to see how it is behaving. For this reason, measurement needs to be an essential component of the portfolio of computer science research.

We propose to hold a virtual workshop focused on the identification of critical questions about the Internet that justify research, exploration of barriers to successful execution of that research, and collective activities that might facilitate that research. Long-term data collection and persistent infrastructure foster reproducibility and repeatability, robustness, and extensibility of research results. However, as in many fields, measurement and data sharing infrastructure is expensive to create, deploy, and maintain. We must find ways to nurture and support longitudinal data collection, but also weigh the cost of maintaining such infrastructure against the potential benefits of the generated data.

Another consideration for the workshop is whether there are critical questions that are relevant to the future of the Internet but not amenable to third-party measurement. If so, the research community may need new policies and institutions to support data collection and sharing, similar to other disciplines. Given this reality, what is the role of traditional NSF-funded academic research in advancing scientific study of the Internet?

Workshop Overview

The goals of this workshop are to identify critical research questions that warrant a call for network measurement (broadly defined), identify barriers and facilitators of that research, and discuss how research results can have impact beyond the research community.

As we think about how NSF can foster a robust community of network researchers, we see a range of issues that form an arc from question through technique to impact: We have structured this workshop around a set of issues that start with the identification of critical questions that justify research, and then explores the various possible barriers to the successful execution of that research, and collective activities that might facilitate that research.

For the critical questions that we identify, the workshop will explore:

  • What data is needed?
  • What infrastructure is needed to collect such data?
  • Given that access to data is the essential methodological component of modern data-driven analytic techniques (ML/AI), should the community collectively tackle the issues of equitable and sustainable data sharing and curation?
  • How can we develop best practices to facilitate cooperation or collaboration with commercial service providers as we collect data? How can the community deal with controlled sharing of proprietary data? Are practices such as secure enclaves, anonymization and acceptable usage agreements effective? What are the roles of various actors in the research ecosystem (e.g., PIs, institutions, funders) in fostering data sharing for Internet science?
  • The ultimate goal of network measurement is often to enable new action--how do we make knowledge useful? What steps make the data we collect useful at addressing real-world problems?

    Workshop Chairs

  • David Clark (MIT)
  • John Heidemann (USC)
  • kc claffy (CAIDA/UC San Diego)

    Steering Committee

  • Mattijs Jonker (U Twente)
  • Fabian Bustamante (Northwestern University)
  • David Clark (MIT)
  • John Heidemann (USC)
  • kc claffy (CAIDA/UC San Diego)

Structure of the Workshop

Potential attendees must submit a one-page white paper. Every white paper should identify one or more critical questions that can justify a call to arms. We will organize these questions into clusters, and the workshop will begin with a set of sessions, each centered on one of the clusters. In each cluster discussion, we will explore the issues listed above, with the goal of understanding, for this cluster of critical questions, a realistic path from research to impact.

We will then end the workshop with a plenary session where we compare the conclusions of the various cluster discussions, identify and explore common issues as well as issues that might have been missed, and attempt to draw overall conclusions.

We do not want to constrain the thinking behind the white papers by listing what the clusters will be. Typical topics for network measurement include performance, security, privacy, stability and resilience, growth, censorship, economics, and neutrality. But we want to hear what participants consider most important looking to the future. Whether the focus is new technology such as broadband wireless, cloud and data centers, mobile, new edge devices or underserved regions, and whether it is at layers of infrastructure, applications or user experience, the goal of each white paper should be a vision of how the field should approach challenges of the future.


The workshop will be January 11 and 12, Monday and Tuesday, starting at 11am EST (8am PST) and running for about 4 hours each day. For presenters, the session format will be 1 hour sessions, with 3 or 4 5-minute paper presentations, each followed by 5 minutes of Q&A, then the remainder of the time general discussion.

Bolded entries indicate a talk.

January 11 (Monday)

January 12 (Tuesday)

Participating in the Workshop

We solicit one-page white papers for researchers who wish to participate in the workshop. Each submission must provide the following information:

  • Identify one or more critical research questions.
  • For each question, identify which points in the above list of issues are of most concern.
  • How NSF could support one or more aspects of the research arc (new infrastructure, new analytic techiques, data sharing, long-term collection, and making knowledge useful).

Note: It is not a goal of this workshop for participants to present their latest research!

White papers that help shape one or more sessions will help us structure the workshop to delve more deeply into the material.

The workshop will be a virtual format using Zoom. We expect sessions to mix lightning talks with considerable discussion time, with 4 hours on each of the two days.

Participants must commit to attend one or more of the cluster discussions and the plenary session. Participants are welcome to attend all the cluster discussions, but we envision serial breakout sessions, a format we believe will better fit a virtual workshop.

    Key Dates

  • CFP out: Monday 30 Nov 2020
  • Registration deadline: Monday 21 Dec 2020 11:59:58pm PST
  • White papers due: Friday 18 Dec 2020 Wednesday 23 Dec 2020 11:59:59pm PST
  • Talk acceptance notification: Monday 31 Dec 2020
  • Workshop dates: Monday 11 Jan 2021 - 12 Jan 2021
  • Submissions are accepted at https://ant.isi.edu/wombir2021/
## Acknowledgments
National Science Foundation (NSF)
Funding for the WOMBIR workshop is provided by the National Science Foundation (NSF) grant CNS-2111828 Workshop on Overcoming Measurement Barriers to Internet Research.
Last Modified