Goals of the project
The high-level objective of this research is to create a scientific basis for modeling Internet interdomain interconnection and dynamics. Specifically, we aim to understand the structure and dynamics of the Internet ecosystem from an economic perspective, capturing relevant interactions between network business relations, internetwork topology, routing policies, and resulting interdomain traffic flow. Despite much recent interest in the economic aspects of the Internet, such as network interconnection (peering), pricing, performance, and the profitability of various network types, two historical developments contribute to a persistent disconnect between economic models and actual operational practices on the Internet. First, the Internet became too complex -- in traffic dynamics, topology, and economics -- for currently available analytical tools to allow realistic modeling. Second, the data needed to parameterize more realistic models -- interdomain traffic characteristics, routing and peering policies and pricing/cost structures -- has simply not been available. The problem is fundamental, and familiar: simple models are not valid, and complex models cannot be validated.
We propose a computational approach that promises progress in in both dimensions: creating more powerful, empirically parameterized computational tools, and enabling broader validation than previously possible. We will use measurements of interdomain traffic, topology dynamics, routing policies and peering practices as input to our detailed model of AS interconnection, and compute the equilibrium -- a state where no network has the incentive to change its connectivity. To validate our model, we will verify that it can reproduce known macroscopic properties of the Internet AS topology as well as known trends in Internet evolution, based on publically available financial and topological data. We will then use our model to study various interconnection practices, the stability and dynamics of interdomain links, and economic properties of the resulting equilibrium.
Specifically, we aim to measure the following properties of the interdomain Internet.
- Interdomain traffic characteristics directly measured from different vantage points on the Internet.
- Structural characteristics of the Internet's interdomain topology, its evolution over time, and the the economic implications of these properties.
- Interdomain routing policies used by networks and the economic incentives behind those policies.
- Peering policies used by different network types as inferred from publicly available information such as peeringDB and Internet Exchange Points (IXPs).
The data by itself has the potential to yield important insights, but we will use this data to parameterize and validate our model of Internet interconnection evolution and dynamics. Each iteration of the model executes a provider and peer selection strategy for each network in turn, until it reaches a equilibrium state, in which no network has the incentive to unilaterally change its connectivity. Why is it important to study equilibria of an Internet ecosystem that is always in flux? We believe that given a certain set of external conditions (interdomain traffic patterns, pricing/cost structures, routing/peering policies), studying the resulting equilibrium can give us insights into the "best" that networks can do, as well as the properties of the global Internet with respect to topology, traffic flow and economics. The equilibrium we compute will be specific to the set of these external conditions. By studying equilibria that result from different parameterizations, we can answer a variety of what-if questions about the evolution of the Internet, including:
- The effect of changing traffic patterns on the economics, traffic flow, and topology of the resulting internetwork. For example, we can study the effect of the rise of large content providers that generate a significant portion of Internet traffic, or the rise of a popular high-volume P2P application.
- The effect of changing price/cost structures, such as cheap peering and transit.
- The effect of the increasing popularity of Internet Exchange Points (IXPs), which facilitate easy and inexpensive peering.
- The effect of the increased use of paid-peering, where routing decisions are similar to settlement-free peering, but one network pays the other.
Data collection
We will begin with raw data from the following data sources to capture economically-relevant information about interdomain topology, traffic, routing and peering policies.
Data type | Description | Source |
interdomain traffic | We will extract the source and destination ASes from netflow records collected from various vantage points, to measure interdomain traffic patterns. | Georgia Tech, UCSD, Internet2 |
Interdomain topology | BGP data from Routeviews and RIPE can be used to reconstruct interdomain topology snapshots | |
Interdomain topology | Traceroute data from Ark can be used to infer AS-level connectivity. | CAIDA's Ark infrastructure |
AS relationships | The business type of interdomain links inferred using AS-relationship inference algorithms. | CAIDA's AS-relationship project |
Routing policies | Routing policies used by ISPs as inferred using AS topology and AS relationship data | AS topology, AS relationship data |
Financial data | Information about company revenues, profits, and stock prices. | SEC filings, public financial data |
Additional traffic data needed
Interdomain traffic characteristics are an important input to our model, as transit payments, peering costs and operational costs all depend on aggregate traffic volume. Unfortunately, there is little knowledge of the global Internet interdomain traffic matrix, even a snapshot much less its dynamics. Interdomain traffic statistics collected from a transit provider network will allow us to infer at least qualitative properties of the interdomain traffic matrix, such as the distribution of traffic sent (received) by a network to (from) every other network. Is it Uniform, Zipf, or Pareto? How does it differ by network, and change over time? We can also use measured traffic characteristics to evaluate "what-if" scenarios that arise from changes to interdomain traffic characteristics. Traffic estimates may even help us to improve our AS-relationship algorithms, based on observed correlations between inter-AS peering likelihood and inter-AS traffic volume ratios. If you are in a position to share data under appropriate privacy and acceptable use agreements, please email info@caida.org.Project Deliverables
- We will release periodic snapshots of all data we collect, subject to private data-sharing agreements, to enable the study of the evolution of interdomain topology, traffic patterns, routing policies, peering policies, and the financial performance of ISPs.
- We will release the software implementation of our model to enable other researchers to investigate different parameterizations and what-if scenarios.
- We will disseminate --- through publications at conferences, journals and network operator venues such as NANOG --- insights obtained from the measurements and by applying our model to various what-if scenarios.
Publications resulting from this project
- Towards a Stastical Characterization of the Interdomain Traffic Matrix, Jakub Mikians, Amogh Dhamdhere, Constantine Dovrolis, Pere Barlet-Ros, Josep Sole-Pareta, in the proceedings of IFIP Networking, Prague, Czech Republic, May 2012.
- Measuring the Evolution of Internet Peering Agreements, Amogh Dhamdhere, Himalatha Cherukuru, Constantine Dovrolis, and K Claffy, in the proceedings of IFIP Networking, Prague, Czech Republic, May 2012.
- GENESIS: An Agent-based Model of Interdomain Network Formation, Traffic Flow and Economics, Aemen Lodhi, Amogh Dhamdhere and Constantine Dovrolis, in the proceedings of IEEE Infocom, Miami FL, March 2012 (to appear)..
- Towards a Cost Model for Network Traffic, Murtaza Motiwala, Amogh Dhamdhere, Nick Feamster, and Anukool Lakhina, in ACM Sigcomm Computer Communications Review.
- Twelve Years in the Evolution of the Internet Ecosystem, Amogh Dhamdhere and Constantine Dovrolis, IEEE/ACM Transactions on Networking, October 2011.
- The Internet is Flat: Modeling the Transition from a Transit Hierarchy to a Peering Mesh, Amogh Dhamdhere and Constantine Dovrolis, in the proceedings of the International Conference on emerging Networking EXperiments and Technologies (CoNEXT), Philadelphia PA, December 2010.
Prior Related Work by the Investigators
- A Value-based Framework for Internet Peering Agreements, Amogh Dhamdhere, Pierre Francois and Constantine Dovrolis, in the proceedings of International Teletraffic Congress (ITC), September 2010.
- Toward Topology Dualism: Improving the Accuracy of AS Annotations for Routers , Bradley Huffaker, Amogh Dhamdhere, Marina Fomenkov and kc Claffy, in the proceedings of Passive and Active Measurement (PAM) Conference, April 2010.
- Workshop on Internet Economics (WIE2009) Report, kc Claffy, in ACM SIGCOMM Computer Communication Review (CCR), April 2010.
- Evolution of the Internet AS-Level Ecosystem , Srinivas Shakkotai, Marina Fomenkov, Ryan Koga, Dmitri Krioukov and kc Claffy, Presented at The First International Conference on Complex Sciences: Theory and Applications (Complex 2009), and published in the European Physical Journal B, vol. 74, no. 2, March 2010.
- Dialing privacy and utility: a proposed data-sharing framework to advance Internet research, Erin Kenneally and kc Claffy, IEEE Security and Privacy special issue, 2010.
- Internet Mapping: from Art to Science, kc Claffy, Young Hyun, Ken Keys, Marina Fomenkov and Dmitri Krioukov, IEEE DHS Cybersecurity Applications and Technologies Conference for Homeland Security (CATCH), March 2009.
- Ten Years in the Evolution of the Internet Ecosystem, Amogh Dhamdhere and Constantine Dovrolis, in the proceedings of ACM/USENIX Internet Measurement Conference (IMC), October 2008.
- Can ISPs be Profitable Without Violating Network Neutrality?, Amogh Dhamdhere and Constantine Dovrolis, in the proceedings of ACM SIGCOMM Workshop on the Economics of Networks, Systems and Computation (NetEcon), August 2008.
Funding support
Support for the Economics of Internet Interconnection project is provided by the National Science Foundation (NSF) grant CNS-1017064 The economics of transit and peering interconnections in the Internet. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of NSF.