Since mid-2000 we have been observing DNS request/response data, using a NeTraMet meter at UCSD. We measure request rate, RTT and response time for root/gTLD servers, and present it via 'strip charts' on our root/gTLD performance web page. The strip charts show RTT variations due to local network changes, distant network changes, and changes in server behaviour.

We plan to extend this work by collecting data from more meters at widely-distributed locations, observing the behaviour of ccTLD servers as well as roots and gTLDs, and using correlation analysis between the meters to provide more topology-related information.

This paper will summarise the project to date, and present preliminary results of data from ccTLD servers.

We analyze properties of two-way end-to-end delays present in today's Internet. We find that distributions of round trip times (RTTs) returned by ICMP and UDP traceroute probes sent from backbone monitors to a representative and well-mixed sample of IP addresses are to large extent independent of year, monitor location, sample size, time of the day and traceroute type. We demonstrate the following quantitative properties of RTT:

1. RTT density maxima correspond to continent pairs, although it is not a one-to-one corespondence since in certain locations positions of continental maximums can overlap.
2. Continuous spectrum of global RTT (the range in which density of log RTT changes smoothly) stretches from 10 ms to 100 sec.
3. The distribution of RTT viewed on a log scale has modes spaced within a factor of 3. Since the continuous spectrum is about 8 times wider, this distribution can be viewed as unimodal. The RTT spectrum is close to symmetric and polynomially decreasing on both sides. In particular, the global RTT median is close to its geometric mean, and "two sigma" interval contains over 93% of all RTT logarithms.
4. The probability of observing round-trip delay over t ms is close to 500 t^{-1.5} for t>200 ms.
5. The chance of observing an RTT that is more than twice the minimum RTT for the same destinantion is less than 10%.
6. The dependence between two RTTs for one destination has a specific cone-like shape when comparing different days or different traceroute types, so that that predicted RTT is bounded by the square and square root of available RTT.

We analyze the evolution of the global Internet interdomain routing system on AS, prefix and IP address level granularities, using snapshots of RouteViews BGP tables from 1997 to 2001. We introduce the notion of semiglobally routed prefixes, those present in the majority of backbone tables, and classify them into

• standalone -- those which have no subsets, no supersets;
• root -- have subsets, but no supersets; and
• subset, or more specific, which are subsets of other blocks.

Using these distinctions we find that from 1999 to 2001 many measures of routing system complexity demonstrated stability in the form of slow growth, dynamic equilibrium, and occasional contraction.

We find that many net change measures reflect contributions of opposite sign, and that true measure of variation, or churn, should take into account absolute magnitudes rather than their difference. Appearance and disappearance of prefixes, ASes and RouteViews peers, as well as status changes (an AS changing from transit to non-transit, or a prefix shifting from a standalone prefix to a root prefix) are instances of routing system churn. We find that the rates of long-term appearance and disappearance for IP addresses, prefixes and ASes have comparable magnitudes and what is perceived as growth is sometimes just a "tip of an iceberg" with two rates almost cancelling each other. One advantage of using our notion of semiglobal prefixes is that they exhibit less churn than global prefixes (those prefixes common to all backbone tables) and as such allow for derivation of more robust macroscopic statistics about the routing system.

We study route prefix instability at a medium time granularity for late 2001 using 2-hour snapshots of BGP tables, and find that half of all prefix reannouncements (flips) are contributed by 1% of all ASes, with government networks, telecoms in developing countries and major backbone ISPs at the top of the list of instability contributors. Small ASes (those who originate only a few prefixes into the global routing system) do not contribute more than their fair share of either route entries or churn. We conclude that during 1999-2001 many Internet metrics were stable, despite rapid change in the mebership of measured sets, and that the routing system's growth and instability are mostly caused by large and medium-sized ISPs.

This page describes CAIDA and related activities in Macroscopic DNS Measurements project started in 2001. We conduct both passive and active measurements in order to study the DNS root servers behavior, their connectivity and performance. We also analyze data collected at the servers themselves.

CAIDA research of the DNS root servers currently focuses on the following problems:

1. Continuous monitoring of the DNS root servers performance.
2. Investigation and modeling of BIND algorithm behavior.
3. Analysis of bogus queries and broken resolver configurations.
4. Evaluation and optimization of root servers' placement

BGP updates are inherently difficult to interpret due to lack of understanding of BGP dynamics. We introduce the concept of BGP Beacons to help calibrate update dynamics. A BGP Beacon is an unused prefix announced and withdrawn at well-known times. Two beacons have been lit up in the past two months. We describe our interesting findings by analyzing the public data from sources like RIPE and Route Views. Among the findings, we confirm that a single announcement or withdrawal message can suppress the route using the current setting of route flap damping. We also describe our analysis of BGP convergence dyanmics.

We investigate the problem of computing the types of the relationships between Internet Autonomous Systems. We refer to the model introduced in [Gao 2001, Subramanian 2002] that bases the discovery of such relationships on the analysis of the AS paths extracted from the BGP routing tables. We characterize the time complexity of the above problem, showing both NP-completeness results and efficient algorithms for solving specific cases. Motivated by the hardness of the general problem, we propose heuristics based on a novel paradigm and show their effectiveness against publicly available data sets. The experiments put in evidence that our heuristics performs significantly better than state of the art heuristics.

Common network engineering wisdom says that performance is degraded duing times of BGP churn, particularly performance to and from the prefixes churning.

With passive monitoring of BGP and active UDP and TCP measurement, we examine whether there is a correlation between degraded UDP and TCP performance, and hightened levels of announcements for prefixes.

In this study we look at 3 months of measurement pairs, where Akamai has a BGP feed proximal to either the measurement's querier or responder, and where a BGP prefix is churning that covers the other end of the active measurement.

Additionally (and unrelated), we will try to present some preliminary data on one or two of the following topics:

• How many prefixes generate HTTP requests on the Internet and how many hosts/prefix make outbound HTTP requests
• Of 750+ ASs studied, how many honor /24s and how many honor reserved address space (not distinguishing packet filtering from route filtering for now)
• Of 750+ ASs studied, how many accept and can generate loose source packets
• Of 250+ ASs studied, data about traceroute hop count vs. AS path length (distinguishing bidirectionally)
• A case for examining the NOTIFY behavior of BGP and considering whether NOTIFY should only optionally tear down sessions

In this talk, we show several observations on Internet traffic flow patterns and derive routing policies that give rise to the traffic flow patterns. First, our results show that an AS can reach a prefix via a peer link even though there is a path via a customer link. Second, we analyze the cause of the prevalence of these traffic patterns. Our analysis shows that an AS typically does not receive all potential routes from its customers or peers. This has several implications on the current practices on routing policies and traffic engineering techniques.

A number of recent studies are based on data collected from routing tables of inter-domain routers utilizing Border Gateway Protocol (BGP) and tools, such as traceroute, to probe end-to-end paths. The goal is to infer Internet topological properties. However, as more data is collected, it becomes obvious that data intended to represent the same properties, if gathered at different points within the network, can depict significantly different characteristics. While systematic data collection from a number of network vantage points can reduce certain ambiguities, thus far, no methods have been reported for fully resolving these issues. The goal of our study was to quantify the effect these anomalies have on key Internet structural attributes. We report on our analysis of over 290,000 measurements from globally distributed sites. We contrast results obtained from router-level measurements with those obtained from BGP routing tables, and offer insights as to why certain inferred properties differ. We demonstrate that the effect on some attributes, such as the average path length and the AS degree distribution can be minimized through careful data collection techniques. We also illustrate how using this same data to model other attributes, such as the actual forwarding path between a pair of nodes, or the level of AS path asymmetry, can produce substantially misleading results.

As part of an effort to develop BGRP, a scalable interdomain resource reservation protocol, we looked at communication relationships to build sink trees, i.e., the tree formed by all paths leading to a particular traffic destination. Using BGP and traffic data, we provide bounds on the number of resource reservations likely to be found in the IP backbone if we aggregate resource reservations.

Availability is a very well known notion in circuit switched networks such as PSTN. It is unappropriately used in packet networks such as the Internet where the PSTN definition does not apply.

We discuss availability issues in an IP backbone network. We illustrate the discussion with data coming from OC-48 links on the Sprint backbone. We identify components that should be included in the definition of network availability for packet networks and show how to improve the availability.

• Update of the Ripe NCC measurement infrastructure.
• RIS BGP analysis update.
• Flaps, holes, unallocated but announced IP space, and AS-distributions.
• Something on bandwidth measurements, a lot of progress has been made but not sure if it will be presentable by October.

Our current efforts involve trying to understand how to infer propagated behavior (e.g. session reset) in BGP trace data. Our work was originally motivated by a need to explain the advertisement waves that accompanied Code Red v2 and nimda attacks, but has applications also in assessing instability in BGP routing.

• The focus is on dark address space, or the prefixes accessible from one provider, but unreachable via some other providers. We have shown by analysis that more than 6% of the internet space lacks global connectivity. Together with analysis of murky address spaces, we examine the degree of partitioning within the Internet. Finally, we speculate on possible reasons for these partitions. The reasons ranges from peering disputes to minor misconfiguration errors.
• Analysis of BGP misconfigurations which lead to partitions.
• Topology inference by identifying redundancy among service providers.

Recent measurements indicate sustained high rates of duplicates ( > 30%, at smaller exchanges, such as CIXP, VIX, etc. ). We look at two relatively large BGP update collection points - Ripe NCC and Oregon, 12 and 25 full-feed peers respectively - to understand the periodicity and redundancy of announcements. We analyze Ripe NCC and Oregon data to determine if such occurences are localized or more global in nature.

Designing better systems and protocols to mitigate end-to-end path failures requires an understanding of the nature of these failures. Prior work has studied potential causes for path failures {Paxson97}. As a followup to this work, we further explore the nature of path failures, with an emphasis on locating path failures and describing failure characteristics (e.g., duration, BGP visibility, etc.) based on location. We present a method to map a failure to a particular location in an Autonomous System (AS) using a combination of alias resolution techniques and traceroutes, targeted for both failure inference and topology discovery. Our approach leverages work in topology mapping but is geared specifically to find AS edges employing a limited number of traceroutes. Using these techniques and active measurements from a globally distributed and topologically diverse testbed, we observe that most visible failures occur inside an AS close to the last hop; in the case of failures on the edge of a network, transit failures are typically more visible, but failures on peering links tend to last longer. Additionally, we observe correlations of failures with BGP events on six testbed sites and characterize the types of failures that are more likely to result in visible routing instability.

This paper describes a method of inferring logical relationships between network prefixes within an Autonomous System (AS) using only passive monitoring of BGP messages. By clustering these prefixes based upon similarities between their update times, we create a hierarchy linking the prefixes within the larger AS. We are frequently able to identify groups of prefixes routed to the same ISP Point of Presence (POP), despite the lack of identifying information in the BGP messages. Similarly, we observe disparate prefixes under common organizational control, or with long shared network paths. In addition to interesting network discoveries, this method allows passive mapping methods to make reasonable inferences about the topology within an AS, and provides additional information that could reduce the number of active probes required in traditional traceroute-based Internet mapping mechanisms.

Overprovisioning is a very common approach to providing quality of service in IP backbone networks. By ensuring the presence of enough capacity in the network so that demands are met, even at peak times and under failure conditions, significant queue buildup can be prevented. This assures that the three key IP QoS requirements, low delay, low jitter and low packet loss, are satisfactorily met.

In today's economic climate efficiency has become a key aspect of building networks, and operators are looking for the minimum amount of overprovisioning to meet QoS requirements. Rules of thumb, such as a maximum link load of 50%, might not be an acceptable approach any more.

In this presentation, we analyze some backbone traffic traces at different timescales. We show that aggregated traffic is well behaved, and present a methodology and simple empirical rule for capacity allocation on backbone links.

The Internet has become part of the critical communication infrastructure in many countries. Reliability is an important aspect for ISPs if they want to sell "Service Level Agreements" - but whenever traffic is exchanged with other ISPs, the performance that customers experience relies on all ISPs on the path to the destination.

Therefore understanding the dynamics of routing between ISPs is crucial for network operators. But basic characteristics of today's only deployed inter-domain routing protocol, BGP, are still poorly understood. For example our gaps in understanding include: the reasons behind routing instability; the influence of policy changes on BGP convergence properties; the interaction of EBGP, IBGP and other intra-domain routing protocols; the limits of BGP scalability. These are just a few of the questions, which are currently addressed by the IETF, researchers, and the panels and presentations at the networkers forums. To be able to solve problems and/or answer the questions above - we first need a methodology to analyze BGP tables and updates and how their variability relates to the structure of the Internet.

We start this talk by presenting some of our methods for characterizing BGP traffic combined with some results of our studies, e.g., heuristics on convergence properties. Next we discuss how network operators and researchers could benefit from our research tools in debugging, tracking global instabilities, and identifying problematic routing conditions. And finally we propose a test bed setup on how to explore and study the sensibilities and limitations of systems.

Today a router in the Internet has to face on each peering session an incessant stream of updates between 50 to 200 updates per minute, which temporarily rises to thousands or even to hundreds of thousands of updates per minute (e.g., during session reestablishment). To identify a structure in this, we take into account that routing table updates rarely happen in isolation at random points in time. Rather they are often caused by events such as failures, misconfigurations, flaps, policies changes, session resets, or even protocol divergence. Our goal is to associate updates with instability events and analyze their impact on routing.

First we try to capture the BGP convergence process. Based to the fact, that a single update may result in lots of updates observed at a distant place (due to various configurations, states and the interconnectivity), we group updates for each prefix into "update bursts". This is done in the same way as one group packets into flows. If a peer sends two updates for the same prefix within a short time window, defined via a timeout, they are considered to be part of the same update burst. We will show plots of various convergence properties, as well as the effects of route flap damping at several stages, and that the durations of the BGP convergence process can be surprisingly large - we found prefixes that are constantly experiencing updates, like reported in the Internet Draft on Persistent Route Oscillation Conditions. For identifying such diverging prefixes we use a simple heuristic based on the findings of Tim Griffin, who has shown that protocol divergence implies a dispute wheel: we divide the number of updates reusing a set of attributes by the number of total updates in a burst. Basically we find cycles in the convergence process using this method.

Beside the characteristics and properties of a single updated prefix, we analyze the impact of an AS. Routing instabilities generally not only affect one prefix, but due to session reset/teardown/establishment, filtering policies change, and router hardware/link failure/repair, a sizable set of prefixes are being updated. These effects are visible even if the responsible router/peering session is many AS-hops away. To capture those effects we keep track of how many updates have been processed by the router within a sliding time window on a per AS basis. If the percentage of updated vs. associated prefixes for an remote AS exceeds some predefined threshold, this update is marked as part of a possible session reset (because the mentioned effects look like a session reset to the observer). Ongoing work correlate updates from different peering points in the network, to trace the origin of the instability, to analyze update propagation, and their embanking due to policies.

Furthermore there is some concern about Internet vulnerability - like misconfigurations or attacks that lead to a melt down of the Internet. Several risks are already being addressed, e.g., in Barry Raveendran Greene's paper on "BGPv4 Security Risk Assessment". We try to use our tools to single out dangerous events and relate them to the actual impact on routers. For example from time to time we've observed some announcements with long AS path sets (over 100 ASes), which are distributed by some routers and crashes others. Speculation ranges from bugs over misconfigurations to attacks, our analysis found only a very slightly increased instability. We speculate that some common practices might have a much larger effect on routing, e.g. "smart routing technologies". Note that our studies currently don't include the impact on the traffic flow.

In summary we try to work towards a better understand of the convergence properties and dynamics of BGP, as well as tying to find limitations. Ongoing work is enhancing our research tools in a way that they are useful for operators: e.g., speedup debugging by identify dangerous or diverging prefixes and blame the "guilty" AS. Our work on BGP traffic is part of a larger research effort of bringing the variability of the Internet into test labs. The goal of the project is to study the impact of variability in a controlled environment. Regarding inter-domain routing we have a prototype tool, RTG, which generates BGP traffic based on a workload model derived from our characterization. This adds routing to existing tool-set of workload generators and traffic shaping tools. A test bed with all these components will enable us to experiment with, evaluate, and judge most Internet components and explore protocol scalability and limitations.

For further information about our characterization of BGP dynamics please look at our SIGCOMM'02 paper on "Realistic BGP Traffic for Test Labs" and/or download our tools at http://www.net.uni-sb.de/~olafm.