Talk Abstract: As access network speeds increase, the access link is less and less often the performance bottleneck for home users. In this context, performance tests a la "speed test" are rapidly becoming inadequate. First, they measure capacity to a dedicated server, not to real application servers. Second, they are becoming too disruptive as they must send enough probes to fill up the link. Instead of focusing on active measurements of access performance, our goal is to develop a mostly passive measurement system to monitor the performance of user applications. Our research is investigating the capabilities and limitations of passive monitoring to measure application performance and explore how targeted active measurements can help locate the source of application performance bottlenecks.

Interested in Discussing: Measurements that can address future public policy needs

Talk Abstract: Monitoring mobile network performance is critical for optimizing the QoE of mobile apps. Until now, few studies touch upon the actual network performance that mobile apps may experience in a per-app or per-server granularity. In this paper, we analyze an around two-year-long dataset collected by a crowdsourcing per-app measurement tool, so as to gain new insights of mobile network behavior and application performance. We observe that 802.11ac devices do not actually work in their optimal mode for most of the time, and the root cause could be related to the signal quality. For cellular networks, the DNS settings on users' smartphones can have a significant impact on the mobile apps' network performance. Moreover, we notice that IM and VoIP services nowadays are not as good as Web services, because the traffic using XMPP experience longer latency than HTTPS. In order to prevent missing meaningful events through manual data processing, we propose an automatic performance degradation detection and localization method, for finding possible network problems in the huge, imbalanced and sparse dataset. Our evaluation and case studies show that our method is effective and the running time is controllable.

Talk Abstract: At Security Scorecard we scan the internet for known vulnerabilities, malware infections, and poorly configured infrastructure. We then attribute these findings to over 1 million companies and rate their performance compared to similar companies. Our platform is used by companies all over the world to monitor their own cyber risk as well as the risk that their vendors introduce.

Talk Abstract: A number of flow characteristics are used to identify, surveil, and potentially disrupt Internet sessions. Reasons for monitoring individual sessions range from legitimate network management to censorship and denial of service. We are actively developing multiple techniques for disrupting traffic analysis. This includes format transformation encryption to translate one protocol into another, application of botnet technologies, BGP injection to randomize IP traffic ranges used during sessions, traffic timing massaging, and use of multi-TCP to multiplex sessions. We will discuss the advantages and disadvantages of each. Privacy limitations and performance implications are discussed.

Interested in Discussing: Traffic analysis, BGP injection, Privacy, Security, Covert Communications

Talk Abstract: We try to characterize how paths are chosen in practice; anycast is particularly vulnerable to bad decisions because a distant replica can be selected, and understanding how these, sometimes longer AS path, decisions are made is not easy.

Interested in Discussing: Good probing infrastructure.

Talk Abstract: As access network speeds increase, the access link is less and less often the performance bottleneck for home users. In this context, performance tests a la "speed test" are rapidly becoming inadequate. First, they measure capacity to a dedicated server, not to real application servers. Second, they are becoming too disruptive as they must send enough probes to fill up the link. Instead of focusing on active measurements of access performance, our goal is to develop a mostly passive measurement system to monitor the performance of user applications. This talk will discuss our current system which measures video streaming quality completely passively. We will also discuss the lessons learned from deploying this system in 50 homes in the US and 10 homes in France.

Interested in Discussing: Measurements to support regulation of Internet access

Talk Abstract: As in a prior year, I would like to take a few minutes to get the group to talk about the larger (policy) implications of what they are doing.

Talk Abstract: Joint Talk/Work with Zakir Durumeric

Interested in Discussing: Security

Talk Abstract: We tackle the problem of identifying the most influential transit providers in each country that may have the potential to observe, manipulate or disrupt Internet traffic flowing towards that country. We develop two new Internet cartography metrics to overcome several challenges with making such inferences using BGP data. The transit influence (TI) metric estimates the share of addresses of an origin AS served by the transit AS. The Aggregate Transit Influence (ATI), captures the aggregate of all fractions of each country's origin ASes' addresses that the transit AS serves. We apply these two metrics to identify the most influential ASes in each country, and the origin ASes in those countries that heavily depend on transit ASes. We include extended case studies of the transit ecosystems of countries in Latin America, Africa and Europe, and we also investigate the role of state-owned ASes in the Internet ecosystem of their home country and in foreign countries. We believe these metrics advance our ability to characterize structural weaknesses in the global Internet topology.

Talk Abstract: Over the past few years, our research group has released several tools, datasets, and services to bring Internet-wide scanning into the reach of security researchers. These include the ZMap scanners, Scans.IO Data Repository, and Censys Search Engine. What originally started as a small open source project in 2012 continued to grow and much to our shock, in 2018, Censys was serving over 1.5 million user queries per day from 90K registered users as well as sustaining upwards of 10 Gbps of raw data downloads while accumulating 1-2 TB of new structured data per day----all based on active Internet measurements.

In retrospect, many of the problems we found most challenging were not the ones we expected. In this talk, I'll discuss our journey, including the challenges we encountered, strategies that worked (and did not), and what I would do differently. I'll touch on technologies that served us well, failed to scale or proved difficult to maintain, and the technologies that I'm excited about moving forward. I'll conclude with a short discussion of where I see Censys and Internet-wide scanning going in the future.

Talk Abstract: One of the biggest challenge in Internet cartography is to obtain from an IP level topology a router level topology. The step of grouping different interfaces into routers is called alias resolution.

Because active measurements reveal IP addresses but not the identities of routers, researchers have invented several techniques to discover common signatures shared among IP interfaces that provide evidence that these interfaces might be aliases. In this work, we propose a new technique, based on ICMP rate limiting, a feature that is present on all modern routers, and mandatory in IPv6.

More often considered as a constraint than an asset in Internet topology discovery, this feature has never been exploited to perform alias resolution. Therefore, in this work, we present: (1) A survey on the types of rate limiting behaviour that we have found. (2) A new technique for alias resolution, which both works in IPv4 and IPv6. (3) An evaluation of this technique on Internet2 ground truth.

Interested in Discussing: alias resolution, internet topology

Talk Abstract: Current status about crowdsourced QoE experiment with CAIDA. (Maybe it would be something like a cooperative talk with Ricky Mok)

Talk Abstract: The Route Views dataset has been a critical resource in the analysis of BGP dynamics and anomaly detection for more than two decades. Unfortunately, its usefulness has begun to atrophy as researchers look for new and innovative ways to access, process, and analyze data.

With renewed interest from engineers supporting the project, Route Views is ready to take the steps necessary to modernize the platform and meet the needs of today's researcher. To do so, Route Views needs to evolve. The collector infrastructure needs to support real-time data delivery mechanisms. The dataset needs to include new facets of the global routing infrastructure such as RPKI. The data acquisition mechanisms need to simplify the workflows of the researcher. Finally, Route Views needs proper governance to be able to adapt to the future needs of the research community.

CAIDA's AIMS workshop provides an opportunity for Route Views to re-engage with the community, with the intention of addressing unresolved questions about the project, and to expand on future improvements as outlined above.

Talk Abstract: * RIPE Atlas infrastructure: An overview about where we are at now, interesting bits and pieces about the existing setup (backend, frontend, scalable infrastructure components), data access and processing, what has changed / is changing nowadays / will come soon? Virtual anchors, software probes and the like.

* RIPE Atlas security aspects:An overview of what works, including risks and mitigation procedures. Enforcing usage quotas, multi-tenant aspects and ethical considerations.

Interested in Discussing: Anything measurement infrastructure ("as a service") related

Talk Abstract: We present net.tagger, a crowd-sourced approach to mapping physical communication infrastructure at the "microscopic" level. This is in contrast, and complementary to, existing macro-level infrastructure mapping projects (such as Internet Atlas and Topology Zoo). Currently, data on city, metro, and "street-level" deployment of infrastructure is scarce, to the point that even providers must often drive around to find their own equipment when an outage occurs. From an end user perspective it is nearly impossible to know if/where multiple providers share fate by utilizing common fiber conduits, towers, etc.

Our intent is to fill in these gaps in understanding by providing a mechanism for crowdsourcing relevant information via a simple-to-use smartphone app. The net.tagger app is currently functional as a proof-of-concept for both the Android and IOS platforms. The user interacts with the system by "tagging" infrastructure, i.e. taking pictures of infrastructure (manholes, handholes, orange dig markings, cell towers, conduit, etc.) and entering a small set of metadata as available, such as the provider name. The geo-coordinates and phone orientation are also captured by the app. This data is uploaded to our servers, stored in an OpenSteetMaps-compatible database and available for analysis. On the back end, analysis techniques include validation (through image recognition, and redundant tags, and external data sources), and connecting the dots between tags through pattern inference. The majority of infrastructure discovered at this microscopic level does not have a 1-to-1 corresponding IP-level element, however where that does exist we also want to identify those relationships.

The success of this project, as with any crowdsourcing effort, is dependent on adoption, which in turn is depending on correctly assessing users' motivation. We are encouraged by the success of efforts such as OpenStreetMaps (>1M contributors) and Google Local Guides (>50M contributors), both of which rely primarily on gamification and altruistic motivations. At this point we are seeking to achieve a critical mass of users, to gather an initial data set, and hope to incentivize this through a points-based system. Much of the back-end analysis goals cannot be effectively analysed without a reasonable density of tags in a particular region. With that in mind our goals for this talk are to increase exposure and solicit feedback on our approach.

See related: net.tagger

Interested in Discussing: Identifying relevant features of physical network infrastructure Incentivising user participation, without incentivising junk data submissions

Talk Abstract: The Route Views dataset has been a critical resource in the analysis of BGP dynamics and anomaly detection for more than two decades. Unfortunately, its usefulness has begun to atrophy as researchers look for new and innovative ways to access, process, and analyze data.

With renewed interest from engineers supporting the project, Route Views is ready to take the steps necessary to modernize the platform and meet the needs of today's researcher. To do so, Route Views needs to evolve. The collector infrastructure needs to support real-time data delivery mechanisms. The dataset needs to include new facets of the global routing infrastructure such as RPKI. The data acquisition mechanisms need to simplify the workflows of the researcher. Finally, Route Views needs proper governance to be able to adapt to the future needs of the research community.

CAIDA's AIMS workshop provides an opportunity for Route Views to re-engage with the community, with the intention of addressing unresolved questions about the project, and to expand on future improvements as outlined above.

Talk Abstract: Minimizing the time for a Domain Name System (DNS) lookup is a key contributor to more generally increasing performance and efficiency, both for the DNS and for the Internet communications that rely on it. In this presentation we focus on the delay between DNS recursive resolvers and authoritative DNS servers, measuring the round-trip time (RTT) between servers outside our control by employing novel techniques using standard DNS queries. We apply our techniques to a major wireless service provider, popular public DNS resolver services, and high-demand authoritative servers. We also analyze the back-end server behaviors for the public DNS services in our study.

Talk Abstract: I will cover highlights in the past few years of the spoofer project.

Talk Abstract: The everyday dependency on the Internet has major political, economical and security implications. This makes understanding the underlying physical paths connecting various end-points of great importance. This talk focuses on characterising commonly used IP geolocation methods and databases in mapping IP addresses of Internet Infrastructure to countries. To this end, we perform a set of measurements between 417 and 132 unique pairs over IPv4 and IPv6, respectively. Our results highlight differences between geolocation sources and methods. RIR delegation files and dedicated geolocation datasets like MaxMind and IP2Location have a high coverage of our collected datasets, whereas active-based geolocation datasets like IPmap and HLOC manage to cover at best half of the IPs. The three databases disagree in geolocating about 10% of the measured IPs. Most of these disagreement can be traced to mergers and acquisitions between ISPs. Furthermore, IPs belong to networks that are geographically spread are more likely to be geo-located differently in different databases. While the geo-hints encoded in the DNS names could help alleviating these limitations, we find many cases where the geo-hints communicate misleading information.

Talk Abstract: Many ISP networks employ virtual private networking (VPN) to provide services to their customers. A common mechanism to deploy VPNs uses virtual routing and forwarding (VRF) to create routing separation between VPNs on the same physical router. A VRF is a virtual routing table that corresponds to a VPN and participates in BGP route announcements, but is separate from any other routing table on its router. Surprisingly, no prior work has investigated the impact of VRFs on traceroute inference. Such work is vital to correctly interpreting traceroute responses, since VRFs violate general traceroute semantics by placing the forwarding interface address in the source field of the reply packet. Failure to account for VRFs in traceroute can lead to mistakes in Internet topology analysis, such as inferring router ownership and identifying interdomain links.

This talk provides background on how and why ISPs deploy VRFs. I introduce preliminary work on detecting VRF forwarding addresses in traceroute, using active and passive approaches. Finally, I show that these techniques perform accurately compared to ground truth from two research and education networks.

Talk Abstract: DHS Science and Technology is now a customer focused organization. A brief overview will be provided of current projects that have fy19 funding and prospective areas for future funding.

Interested in Discussing: Geolocation, artificial intelligence

Talk Abstract: The relative success or failure in practice of a range of censorship-circumvention, DDoS defense, and congestion discovery systems depend on BGP poisoning's feasibility or impracticality. Using a purpose-built Internet-scale measurement systems spanning 5 Border Gateway Protocol (BGP) routers, 8 previously unused IP prefixes, 5,000 distinct vantage points, and 3 countries, we capture and analyze 1,460 instances of BGP poisoning across 3% of Autonomous Systems (ASes) on the Internet. We measure the Internet's response to the underlying routing behaviors necessary for widespread success of BGP poisoning, and we then re-evaluate several existing systems using our findings. We highlight how these findings impact both existing and future security systems and techniques, and additionally build models that indicate how well certain ASes on the Internet can execute BGP poisoning effectively. Fundamentally, we address the growing division between security literature that assume active re-routing of traffic with BGP poisoning is possible versus work that assumes the opposite, providing insight for future Internet security research as well as validating prior work's reproducibility.

Interested in Discussing: Routing and Security (these are already on the agenda).

Talk Abstract: The MANIC project - Measurement and Analysis of Interdomain Congestion (https://manic.caida.org) - has developed a prototype system to monitor interdomain links and their congestion state, in order to provide empirical grounding to debates related to interdomain congestion. For enabling public access to the Time-Sequence Latency Probing (TSLP) data collected from Ark vantage points since early 2016, it also built the MANIC API (https://api.manic.caida.org/v1/). In this talk, we will present the said API, shedding light on its architecture, some of its end-points and their corresponding outputs structure. We will then highlight its possible use-cases before finishing by a demo of its functioning.

Interested in Discussing: IP links, Interdomain Congestion, API, TSLP

Talk Abstract: Enriching traffic data with additional business, routing, application, and security context can drastically increase the ability to reason about traffic, performance, and attack patterns. This talk will present a brief overview of a scalable architecture for enabling real-time syncing with metadata, DNS, and routing systems, for performing streaming joins to enrich traffic data, and sending the serialized output to multiple systems for storage, querying, and anomaly detection.

Interested in Discussing: Interested in cooperative ways of doing Internet-wise performance measurement (passive and synthetic).

Talk Abstract: Moving Atlas (Traceroute) data into Bigquery in order to access data in a faster, flexible and easier way.

Talk Abstract: The RIPE NCC operates an internal, private, Hadoop cluster for analysis of accumulated RIPE Atlas measurement results and some other metadata. This approach is functional, but query execution can be slow, maintenance is expensive, and it relies on multiple specialists to keep running. Recently we have been experimenting with using commodity cloud platforms, such as the Google Cloud Platform, to side-step these issues and simultaneously improve query performance, improve service availability, and reduce our operating costs. This work is currently being prototyped by the RIPE NCC's GII team.

Talk Abstract: Narrowband IoT (NB-IoT) is a wide area cellular technology for connecting low power devices. It has been engineered to ensure a reliable data transmission while keeping power consumption at a negligible level, because many of envisioned use cases target battery life of 10+ years. To this end, NB-IoT comes with more configuration parameters and power management techniques than traditional cellular technologies. However, the extensive focus on battery life complicates the task of performance measurements. For example, round trip times, packet loss rates or throughput are no longer sufficient to describe network performance. In fact, they always need to be assessed in light of the associated power consumption. In this talk, we give a brief overview of NB-IoT, present results from two commercial deployments and two different NB-IoT devices and explain how performance measurements should be taken and assessed.

Talk Abstract: Censorship evasion is a cat-and-mouse game between nation-states and researchers. Today's evade/detect cycle is largely manual: researchers first actively measure censors' networks to learn about how they operate, and then develop strategies to exploit shortcomings in the censors' designs and implementations. This unfortunately gives censors an asymmetric advantage: the time to manually measure and learn about censors often exceeds the time it takes a censor to patch its network.

In this talk, we propose a drastic departure from the manual evade/detect cycle: using artificial intelligence to adaptively probe censoring regimes and automatically discover strategies for circumventing them. We will present the design of our genetic algorithm-based architecture, and the results thus far from applying it to in-lab and real network censors, including the Great Firewall of China. Our architecture was able to independently derive virtually all prior work on client-side evasion of on-path censors, and has discovered wholly new strategies, giving us novel insight into the functionality of nation-state censors. We will also present ongoing and future applications of this approach. (kc said it was ok for us to submit this late. Sorry for any inconvenience!)

Interested in Discussing: Measuring adversarial networks

Talk Abstract: Censorship evasion is a cat-and-mouse game between nation-states and researchers. Today's evade/detect cycle is largely manual: researchers first actively measure censors' networks to learn about how they operate, and then develop strategies to exploit shortcomings in the censors' designs and implementations. This unfortunately gives censors an asymmetric advantage: the time to manually measure and learn about censors often exceeds the time it takes a censor to patch its network.

In this talk, we propose a drastic departure from the manual evade/detect cycle: using artificial intelligence to adaptively probe censoring regimes and automatically discover strategies for circumventing them. We will present the design of our genetic algorithm-based architecture, and the results thus far from applying it to in-lab and real network censors, including the Great Firewall of China. Our architecture was able to independently derive virtually all prior work on client-side evasion of on-path censors, and has discovered wholly new strategies, giving us novel insight into the functionality of nation-state censors. We will also present ongoing and future applications of this approach.

Talk Abstract: Large content and cloud providers have built traffic engineering systems to prevent congestion at interconnections at the edge of their networks. These systems take as input utilization and capacity information from devices at the edge, determine the current state, and then shift traffic as needed to prevent or mitigate congestion.

However, congestion can occur anywhere along a path to end-users. Ideally, these traffic engineering systems could monitor the end-to-end path for congestion and shift traffic to an alternate (uncongested) path when possible. Yet the inputs they rely upon are unavailable to providers for links beyond the edge of their networks. In addition, conditions beyond the edge of a provider's network are subject to the current state and dynamic traffic engineering decisions of other networks, and therefore cannot be modeled / predicted.

Given the lack of ground truth (utilization and capacity), traffic engineering systems must rely on other signals to determine if a path is congested. Prior work has discussed using active probing to monitor interconnections for congestion. However, detecting congestion anywhere along the path would require actively monitoring all links between the provider's edge and end-users, introducing significant scalability challenges. Further, these systems used active probing to facilitate third-party detection of congestion; a provider should be able to take advantage of their existing user traffic to detect congestion.

In this vein, we explore using performance signals to detect congestion. We discuss challenges in using signals believed to be correlated with congestion, such as latency and retransmission rate. For instance, it is difficult to distinguish whether a change in latency is due to a change in client population and/or a path change, or a buffer building at a congested link. Likewise, TCP's retransmission rate can vary due to latency, client access link speeds, and file size distributions; we have found that setting a threshold on retransmission rate to define congestion leads to erroneous conclusions.

Motivated by these challenges and the lack of insights around how congestion manifests in production networks, we first explore the dynamics of Internet congestion through both emulations and production data. To do so, we build a testbed that can emulate a wide variety of real-life traffic conditions. We show the impact of path latency, buffer size, congestion control algorithms, file size distributions, and connection reuse on congestion. In addition, we show that congestion comprises two stages -- latency-increasing and loss-inducing, and study how it transitions between them. In particular, we show that a pathology between TCP and deep buffers, combined with CDN connection behavior, can yield conditions in which demand at a link can exceed the link's capacity for an extended period of time yet results in zero or only minimal packet loss.

Second, we use our insights to design and implement a stateless technique to detect congestion in the loss-inducing stage. Our technique is simple, low-overhead, and deployed at Facebook. Third, we discuss why distinguishing congestion in the latency-increasing state from path changes may be impossible even with supplemental measurements or historical data.

Talk Abstract: The FCC has been collecting data since 2013 on the performance of mobile broadband Internet access service through its Measuring Broadband America program. Recently, this performance data was made publicly available. It is the first large-scale publicly available dataset on mobile broadband performance in the United States. An understanding of the performance of mobile broadband service is critical for many research efforts. Design of mobile applications often benefits greatly from knowledge about the distributions of QoS parameters, including download throughput, upload throughput, end-to-end delay, and packet loss. Design of next generation mobile technologies and protocols benefits from knowledge about QoS parameters and from identification of performance bottlenecks. Researchers benefit from broadband performance data that can be used to parameterize models used to test research ideas. There are few publicly available studies of mobile broadband performance based on large-scale datasets. Vendors of some propriety mobile speed test apps occasionally release reports based on their data. However, such reports do not disclose the distribution of any performance metrics, any information on packet loss, or any information on the variation of mobile broadband performance over days and hours. We will present a preliminary analysis of mobile broadband performance based on the FCC Measuring Broadband America dataset. We will present results on the distribution of download throughput, the distribution of upload throughput, the distribution of round-trip time across the mobile broadband provider's network and through an interconnection point, and packet loss. We will examine the variation of these performance metrics over 3 years. We will examine the variation of these performance metrics over days and hours, with an eye toward identifying the peak usage period. We will examine these performance metrics in urban and selected rural areas, with an eye toward formulation of useful disclosures.