2008 was an exciting year for the Internet and no less exciting for CAIDA. As network-capable personal/computing devices became ever more affordable and ubiquitous, and developers continued the flow of [open] applications/protocols that make it easier to create, capture, edit, publish and share information at the increasing speeds allowed by optical fiber, cable, and wifi services, we continue to make vast empirically untested assumptions about how the Internet is financed, operated, and used. What's going on under the hood of the engine of our new digitized economy?

Over the last two decades, the Internet operational and research communities have gathered overwhelming evidence that underneath the exciting developments at the application level, the Internet's architecture faces overwhelmingly and relatively near-term challenges with arguably intractable technological, political, social, and economic dimensions. We have previously taxonomized these problems into four categories of concerns for the Internet as emerging critical infrastructure: safety, scalability, sustainability, and stewardship.

CAIDA's 2008 Annual Report describes our recent efforts to illuminate these aspects of the Internet, providing highlights from our research, infrastructure, and outreach activities. Our current research projects, primarily funded by the U.S. National Science Foundation (NSF), include several measurement-based studies of the Internet's core infrastructure, focused on the health and integrity of the global Internet topology, routing, addressing, and naming systems.

We made fundamental advances in several of our research projects this year, supported by increased coverage by our measurement infrastructure, and increased collaborations with colleagues around the world. Highlights from the annual report include:

• The first full calendar year of the most comprehensive annotated view of IPv4 topology thus far. We also began to deploy IPv6 topology measurement instrumentation.
• Some of our topology research focused on how different routing approaches in nature are maximally efficient on certain types of peculiarly structured topologies, conveniently, those structured like the Internet AS graph. Further, we found that self-similarity of clustering in real complex networks provides strong empirical evidence that some hidden metric spaces underlie these networks. In trying to model self-similar (scale-free) networks embedded into such a hidden space, we discover that a certain approach to routing -- greedy routing -- is phenomenally successful and efficient in such a model. We are still exploring the ramifications of this intense discovery, and the even more intriguing breakthrough that this hidden space seems to be hyperbolic.
• Our research into network growth dynamics also yielded two papers with surprising results about different regimes of network growth: (1)  that there may be a vast pre-asymptotic regime of complex network growth that gives rise to power-law like effects in degree distribution; (2)  a simple customer-provider-based modification of the preferential attachment model can account for Internet topology evolution, including the ISP consolidation toward monopoly.
• increased active and passive measurement infrastructure as well as continued maintainance of a catalog of Internet measurement data sets.
• coordination and analysis of another DITL's (Day in the Life of the Internet worth of data.
• updated our real-time traffic report generator, geographical visualizations of DNS workload to a given set of servers, updates to our IPv4 and IPv6 AScore posters, and visual maps of IPv4 address space consumption.
• a set of blog entries that became a short Internet research tutorial for policy folks.

For all the exciting details, we encourage you read the full report and post comments/questions, which we can integrate into next year's update of our strategic program plan.

Internet infrastructure economics research", and how to do reasonable examples of it, has come up a lot lately, so i'm posting a brief description of an academic+icann community workshop i've been recommending for a few years, which has yet to happen, and (I still believe) is long past due, and specifically more important than passing policies, especially emergency ones to allow IP address markets with no supporting research on the impact on security and stability of the Internet, and even at the risk of killing IPv6 altogether.]

Goal: a more structured conversation according to established discipline of scenario planning.

Objective: help understand what we don't know. different way of seeing, thinking, 're-perceiving' link system structure and behavior -- "model what you don't know"

---

Phase 1: SAST: strategic assumption surfacing and testing (SAST). Start with specific decision (in our case, IPv4 address markets/transfer), build out toward environment/context: (1) what are driving forces /trends in macro environment (2) what is uncertain, inevitable? rank forces by importance (3) what do decisions makers want to know? (4) what will they see as success or failure? (5) what considerations will shape these outcomes?

Phase 2: Interview key players

Phase 3: Create proposed scenarios (~4; no probability assignment, since this is not about predicting the future, but understanding and preparing for the future). Effective scenarios are:
(a) plausible and surprising
(b) have the power to break old stereotypes
(c) decision-makers assume ownership of the scenario
(d) participatory (help thoroughly flesh out scenario)
(e) few in number, the differences among which matter to decision-makers.

So we would need scenarios to cover routing table explosion, nationalization of the addressing allocation function (and thus likely other aspects of Internet infrastrtucture), and market cartelization), as well as for a takeoff of IPv6 growth.

Phase 4: Create scenarios as a group (workshop #1, 2 days)
(a) understand present, past, demographic and technology changes
(b) describe variety of possible futures
(c) delineate how scenarios above evolve
(c) identify indicators to track what may trigger scenarios
(d) link to specific decisions
(e) link to analysis process
(f) link to organizational procedures
(g) involve decision makers

(So (c) above is where you would make sure someone writes up a neutral analysis of the "NAT tax", that allegedly kills growth by strangling new applications and paving nonneutral networks. no easy trick, but the RIRs should make sure there is evidence of an earnest attempt.)

Workshop day 1: 1 hour defining issue; 3-4 hours key factors, environmental forces, setting on scenario matrix; 3-4 hours socialize, informally , compare impressions

Workshop day 2: 2nd thoughts on skeletal scenario logic; 1-2 hours: fleshing out one scenario together: beginning, middle, end. afternoon: break up into smaller groups to flesh out other scenarios, including preliminary and strategic impacts of each

Phase 5: follow up after workshop: 4-6 weeks of interim research while writing final scenarios
and exploring implications. circulate drafts, more interviews.

Phase 6: (possibly another workshop to) develop a framework for how to monitor indicators and reevaluate scenarios in light of empirical data.

Participants:
-- at least 1-2 represenatative from each RIR
-- 1-2 represenatatives from ICANN and advisory councils
-- 4 economists/media policy folks
-- 2-4 Internet routing operational experts
-- 1-2 from U.S. DOD (who have elephantine amounts of legacy IPv4 space)
-- researchers from related disciplines, with accepted abstract submission

(need representation/support/participation from: top management, key decision makers and implementers, broad range of functions and divisions represented imaginative, open minds, at least 2 people who can write up results in unbiased way)

---

---

References

Learning from the Future: Competitive Foresight Scenarios

The Sixth Sense: Accelerating Organisational Learning with Scenarios

Inevitable Surprises: Thinking Ahead in a Time of Turbulence

Creating Futures: Scenario Planning As a Strategic Management Tool

A handbook for scenario planning: practicing futurists Bill Ralston and Ian Wilson offer practical guidelines for using scenarios in business settings

The Changing Foundation of the Internet: Address Transfers and Markets

Reform Establishing the Rule of Law (pdf)

According to the Best Available Data: internet telemetry, v6

disclosure: ARIN has sponsored CAIDA research efforts in gauging IPv6 penetration and obstacles, some results presented at ARIN meetings (October 2005, April 2008, and October 2008), others on the research pages of CAIDA's website. ARIN has also told me it is planning to launch a more formal research program, which could be used to inform current and future policy debates.]

Stefan pointed me at a paper titled "Designing and Conducting Phishing Experiment" (in IEEE Technology and Society Special Issue on Usability and Security, 2007) that makes an amazing claim: it might be more ethical to not debrief the subjects of your phishing experiments after the experiments are over, in particular you might 'do less harm' if you do not reveal that some of the sites you had them browse were phishing sites.

This brings us to the question: Does a phishing experiment that deceives a subject and exposes the subject to a fake phishing attack adversely affect the subject's rights or welfare? As noted above, as long as the researcher can ensure the security of any personal information of any information released by the subject (the procedures of which are outlined below), neither a laboratory phishing study nor a naturalistic phishing study should adversely affect the welfare of the subject. However, we question whether the use of debriefing in naturalistic phishing studies might, in fact, adversely affect the welfare of the subject and propose that this, in part, is justification for not debriefing subjects in these types of phishing studies. In regards to adversely affecting the rights of subjects, the use of deception or waiving consent is not seen as a violation of a personal right, see 45 CFR 46 [5], 116 and [7]. Although laudable, the right to know the truth is not a recognized absolute right. However, the federal regulations and ethicists recognize that it is advisable to address this issue and use debriefing to provide the pertinent information relevant to the truth, when appropriate, see 45 CFR 46 [5], 116(d)4, and [7]. The question we raise is whether using debriefing in a naturalistic phishing study is appropriate.

"Designing and Conducting Phishing Experiment", Peter Finn and Markus Jakobsson, http://www.indiana.edu/~phishing/papers/finn-conducting.pdf

This is an interesting, but questionable position: "If people know what's happening, then they will be upset. But what they will be upset by is learning they were deceived, therefore we must completely deceive them." That's an argument that makes a case against itself in one sentence.

There are other problems with the approach, including the assumption of implicit rationality in the users; it does not address the prevalence or degree of anxiety and even fear of being observed in the digital media. The researchers present the problem as dichotomous, choosing not to explore methods that could establish the degree of difference between behavior during informed consent and non-consent. At what sample size and study interval do informed consent procedures change behavior? (If you told someone you were studying their behavior on Internet for the next hour, they'd probably change. But over the next year?) Also, what's wrong with knowing only conservative values of phishing vulnerability? If it's such a big problem, wouldn't even those estimates be influential in designing anti-phishing sites and informing policymakers and law enforcement?

There is a lot of research which is compromised -- or completely impossible -- with informed consent. But in cases where those compromises can be studied, and estimates of uncertainty established, perhaps researchers (especially psychology researchers?) should not be exempt from that process.

However, I've also heard from commercial security consultants that the "tricking users into getting phished without telling them" approach is exactly how many corporations measure the extent their own employees are getting phished on corporate networks. Of course, commercial entities don't need their internal research projects to pass IRB approval, or peer review, much less public review. The paper's most important contribution may be its acknowledgement of the lack of current guidelines for how to conduct ethical Internet research. DHS S&T's upcoming workshop on Ethical Issues in Network Research (26-27 May, by invitation) is happening not a moment too soon. More on this workshop later.

ICANN hired JAS to write an independent evaluation of ICANN's Security and Stability Advisory Committee, which I've served on since 2003. JAS published a first draft on 16 February 2009, which I commented on on a few days later. The same week I also spent a couple hours on the phone with the report authors Jeff Schmidt and William Yang, who intend to release a final draft of their SSAC review next week, which will incorporate the feedback received on the first draft. It's a tough job to evaluate a complex system like SSAC, but it's good to see ICANN proactively pursuing independent objective evaluations. I'll post a link to the final report here.

Last month (23 March) I was on an NTIA panel at the Department of Commerce, to recommend conditions on this broadband stimulus money, aka arm wrestling between companies. Gigi covers it in her blog; today was the deadline to finish my recommendations to DOC and NTIA:

1. Prepare for the inevitable realization of the underlying empirical fact: you can't make Wall-Street-approved margins moving bits around over long distances. You certainly can't have bit-moving margins and get science and innovation and critical infrastructure protection done. Other countries now ahead of us in broadband penetration have already figured out effective policies for stimulating broadband growth. As a general rule the DOC and NTIA should use this money to put the U.S. Congress in a better position to improve our current failing policies.
2. Architect some economic transparency into the rest of the industry. The erosion of voice revenue which has funded much of the Internet's evolution -- with the help of regulation allowing that subsidy -- means the business model for Internet transport is fundamentally unsustainable, which the carriers know better than anyone. But the entire Internet provisioning ecosystem is characterized by pervasive non-disclosure agreements, undocumented cross-industry subsidies and a common practice of treating even mundane operational practices and costs as trade secrets, which leaves regulators operating in a fog around the Internet infrastructure, not ideal conditions to write regulation. Recall, the last reported accounting error in our industry was bigger than this entire broadband stimulus package. Which is all admittedly noise compared to AIG, but it was the Biggest Bankruptcy Ever at the time. It's obvious why we're repeating history: we don't have any data.
3. Use this transparency to pursue a quantitiative macroscopic picture, starting with the economics. Use a chunk of the $350M to create a visual map of how the funding flows through the ecosystem, as well as acquire and publish quantitative data on pricing, penetration, performance, and peering of Internet transport providers. Spending some of these mapping funds on data-sharing and objective research and analysis will help establish a rigorous field of Internet economics, the lack of which engineers now admit limits our ability to fix technical problems of the Internet. Yet the two biggest conversations about pricing (network neutrality and metered billing) are happening with no cost and pricing data, much less data on traffic levels or patterns.
4. On a simillar wavelength, use some of the $350M allocated for mapping to fund a spectrum inventory with R&D support.
5. Establish a technical advisory team to peer review and steer progress on mapping and other data analysis projects, including privacy-respecting, legal, and ethical data handling practices. Document where limitations prevent sharing of data otherwise considered relevant to broadband stimulus goals.
6. Require wholesale non-exclusive access to any infrastructure owned by the company that takes stimulus funds. Support experiments with structural separation, like the UK's Openreach which has led to falling prices, or Australia's recently announced experiment with public infrastructure. (Yes, this pretends we didn't make the mistake of tossing common carriage concepts for the Internet. )
7. Leverage other funding communities, e.g., allow NSF or DHS to cost-share with broadband stimulus projects for R&D that provides particular leverage in building infrastructure, or for public-private partnerships that advance broadband and mapping capabilities. Example projects include a backbone infrastructure to connect community and municipal networks in exchange for research support or public safety. Require that funding recipients send at least one representative to academic research and scenario planning workshops. Provide funding and policy support for data collection to validate tools funded by NSF to measure traffic, bandwidth estimation, mapping, hygiene (filtering, config), penetration, usage patterns. (Cost-share with the $250M for innovative programs to encourage sustainable broadband adoption).
8. Leverage other infrastructure, like electricity grids and roads. Encourage partnerships. Don't allow recipients to lobby against competition. Use existing successful infrastructure projects as role models.
9. Disclose all traffic mangement behavior. Avoid deep packet inspection (DPI) techniques except for security reasons, until the privacy conversation matures.
10. Require that providers maintain responsive abuse contact, and accurate registry information for all DNS names and IP addresses hosted by the provider.

[this thread on transfers is too painful to watch. here's my take.]

Even if turning IP addresses into private property is the best policy decision of those available (which is far from demonstrated, since so little rigorous research of this question has actually occurred), executing such a policy by Board fiat while ARIN itself has no leadership is guaranteed to generate severe dissonance with ARIN's organizational mission which includes forging public legitimacy entirely from its transparent, open processes.

I'll be the first to admit that the recent dramas around transfers in ripe and apnic regions inspire a sense of urgency, although my concern is that established economic theory suggests these decisions will accelerate not only the end of IPv4 as we know (and designed) it, but also the end of any hope for pervasive IPv6 connectivity. Regardless, noone has acknowledged that's why we're down to DEFCON-2, or explained how it justifes Board maneuvers to declare arbitrary emergencies resulting in billion-dollar capital shifts with no regulatory oversight based on opaque reasoning and (still) no conflict-of-interest disclosures from Board members. With due respect to ARIN's legal counsel, this is not the behavior of an organization trying to Avoid Lawsuits. If there is some other pressure that Board members are perceiving that is not public, making it public immediately may risk our Grand Experiment going down in a puff of transparency, but is it worse than it being destroyed by non-disprovable accusations of corruption and conflicts of interest?

It's already a murky picture, and the Board should invest in making the policy debate less murky, not more so. There is a field of quantitatively grounded Internet policy research to establish here. The RIR's are in the best position to establish it. It's a higher priority than setting fire to IPv4, and has been for years. But I repeat myself.

If as Bill Woodcock suggests the ARIN Board members are now interested participating in the policy process, they should start by posting their views on IPv4 address privatization right below their COI disclosure paragraphs and bios. The Board has just made their independence part of the problem rather than part of the solution.

Update: In May 2015, ownership of Spoofer transferred from MIT to CAIDA

We are studying an empirical Internet question central to its security, stability, and sustainability: how many networks allow packets with spoofed (fake) IP addresses to leave their network destined for the global Internet? In collaboration with MIT, we have designed an experiment that enables the most rigorous analysis of the prevalence of IP spoofing thus far, and we need your help running a measurement to support this study.

This week Rob Beverly finally announced to nanog an update to spoofer he's been working on for a few months. Spoofer is one of the coolest Internet measurement tool we've seen in a long time --- especially now that he is using Ark nodes as receivers (of spoofed and non-spoofed packets), giving him 20X more path coverage than he could get with a single receiver at MIT.

IP source spoofing is still a common attack vector. Rob's ANA spoofer project first began quantifying the extent of source-address-based filtering in 2005, although with a limited set of tests and only a single destination receiving probes. In addition to adding several different tests that will improve the accuracy of inferences, spoofer now hooks into CAIDA's Ark infrastructure, which provides 20-30 additional destinations for test probe measurements. Spoofer's output is also presented in a snazzy visual form; Internet measurement doesn't get much sexier than this! Help us measure the extent of source-based address filtering on the Internet -- take a few minutes to download and run the tester -- the more diverse network locations you run spoofer from, the more accurate our results will be.

Related links: Spoofer FAQ; Ark project page.

[Ark project funded by DHS S&T's Cybersecurity Program N66001-08-C-2029 and NSF CISE's Computing Research Infrastructure award CRI-0551542.]

[drafted this entry a few months ago but have been reluctant to post because it's incomplete. but after reading about the ARIN Board's emergency proposal last week to create IPv4 address markets, variations of which have already been approved in European (RIPE) and Asia-Pacific (APNIC) IP address policy communities, i decided it's complete enough. -k.]

A few policy questions on which the RIR-community should funnel address-registration tax dollars into peer-reviewed research:

1. How is IP address allocation usefully compared to other communication media, e.g, spectrum allocation? What can we learn from models used in different countries? Establish metrics for evaluation of efficiency and consumer surplus generated for different models of allocation. What enforcement mechanisms are used in different regimes? Propose and evaluate cost and performance of such mechanisms.
2. Unlike spectrum, IPv4 addresses are a non-substitutable input for everyone participating in the Internet routing system. How is IP address management usefully compared to that of telephone numbers? Or Currency? The economic history of multiple currencies is sufficiently consistent as to merit its own economic law ("bad money drives out good.") -- how might it apply to address markets? Does the recent introspection about monetary policy provide any relevant insight?
3. In considering the privatization of IP addresses, what can we learn from examining other industries privatized in different (especially G8) countries: electricity, natural gas, trucking, airlines, telecom? What about comparisons to other fundamental market reforms in the last 200 years, e.g., Russia, China, India, Latin America. What would historians of these transformations have to say about the IPv4 runout dilemma?
4. What can we learn about IP integer markets from examining the political economy of other aspects of the Internet ecosystem we have privatized, e.g., infrastructure, naming (DNS) registration services? What about the trajectory of those industries do we want to emulate or avoid?
5. For completeness, list the arguments and stakeholders for and against reclamation of IPv4 space per RFC2050.
6. Update the PIARA study given what we have learned in the last decade.
7. Bill Herrin added an even better object of comparison to IP address management: freshwater management. I suspect studying water management in general would be enlightening. If it starts to sound like we're studying a public utility, we're on to something.

Ideally this background research and associated interdiscplinary conversations happen before -- or at the latest during -- the more urgently needed roadmapping and scenario planning exercises, which the RIRs should undertake or openly outsource, then summarize results to all address holders and seekers.

disclosure: ARIN has sponsored CAIDA research efforts in gauging IPv6 penetration and obstacles, some results presented at ARIN meetings (October 2005, April 2008, and October 2008), others on the research pages of CAIDA's website. ARIN has also told me it is planning to launch a more formal research program, which could be used to inform current and future policy debates. There's a lot of work to do, but the RIR's are in a fine position to do it.

I get this question as often as I get any question about the Internet. finally, a visiting intern Mia Zhang from Beijing Jiaotung University has done a thorough literature roundup, extracting the best available data pertinent to this question that she could find in the public domain.

Although we co-authored two papers [P-5] [P-6] on P2P traffic classification, we cannot give a single numeric answer to this question for Internet traffic today, for a variety of reasons. First, traffic is highly variable, across time, space, and users. Second, as the papers above suggest, there is no single standard method for determining that traffic is "peer-to-peer file sharing". Finally, it's near impossible to get representative traffic samples to a scientific forum where peer-review can occur. As a result, there are a lot of claims of peer-to-peer traffic levels and percentages, but none that could be used to justify changes in policies, regulations, or pricing models.

We do consider it scientifically relevant that the range of peer-to-peer file sharing traffic fractions observed is from 1.2% to 93% across the eighteen (of the 64) papers that provide such numbers, and that the average fractions reported in studies has increased considerably from 2002 to 2006 (Table 1). Table 2 and Table 3 show that results also vary considerably according to link location, and time (though according to the few studies reporting numbers, peer-to-peer traffic is heavier at night). One study [34] suggests that users use peer-to-peer applications more often at home than in the office. Finally, a study in Europe found considerably higher fraction of P2P traffic on a university link in Europe than some Canadian academics [34] found on their campus.

Many of these numbers are based on P2P detection via statistical or behavioral classification, not the most reliable method of detecting an application. More accurate methods involve examination of traffic contents, which is fraught with legal and privacy issues. Thus there is far too little data available to allow conclusive claims beyond "There is a wide range of P2P traffic on Internet links; see your specific link of interest and classification technique you trust for more details."

[Note: Ipoque, which sells proprietary products that help classify Internet traffic, just released their third annual Internet traffic study, with the first study (in 2006) focused only on P2P traffic. In this latest 2008/2009 study, Ipoque seems to see a much narrower range of P2P traffic on their customers' networks than indicated in publicly available Internet research papers. One cause of this discrepancy could be that Ipoque systems are more accurately identifying P2P traffic because they are examining traffic contents, which academic studies are typically unable to do. Since none of the data sets or classification algorithms used in the Ipoque study are available, we cannot reproduce their analyses or verify their claims.]

I was delighted to see Sid Faber and Tim Shimeall co-teaching a "Network situational awareness" course at Carnegie-Mellon University last semester, using DatCat and DITL data, they even put the class projects online. Not only did some of the students use DITL data (contributed by Japanese academics), as well as Internet2's netflow data, but they used DatCat to find both data sets. To quote Sid,

"About three weeks into the class, we finally got across one of the key features to the students: we were looking at how things really work on the internet, not just a theoretical discussion of RFCs. The data sets were invaluable, but we had challenges dealing with anonymization, sampling, and the overall volume of the data sets -- kind of understandable for the first offering of the course."

Sid also repeated something I've heard too many times from the research community -- that the anonymization scheme for the Internet2 flow data, which masks 11 bits of the IP address, makes the data nearly useless for the research they were trying to do. In particular, Muley & Allen's project on implementing traffic management could only identify blocks of traffic from networks, not individual computers. Nor could they do any network profiling, e.g., count web servers or identify client gateways or find un-NAT'd clients. The data also indicated substantial asymmetric routing -- where the data shows only one direction of a conversation on the given link -- which may be an artifact of the monitor setup, or the 1/100 packet sampling, or actual traffic patterns. Apparently other students working with data sets anonymized with prefix-preserving anonymized schemes had much more success.

On the other hand, a prefix-preserving anonymization scheme would make it impossible to investigate other questions, such as estimates of traffic flow between organizations attached to Internet2. Internet2 is in a difficult position, trying to accommodate as many researcher needs as possible, but with no guiding set of priorities and concrete questions from the network research community to influence data collection and curation architectures. One could argue the Internet2 should be investing resources in gathering this list of questions, but one could also argue such a list should come from the network research community, or network operators, or funding agencies, or lately, law enforcement.

We did brainstorm a set of questions researchers would like to learn about the Internet at a January 2008 workshop in support of DITL2008, but we mostly learned from that we were not going to be able to collect data to answer most questions on the list. Emile also set up a Google moderator page to invite others to vote on what empirical questions about the Internet they think are important, but we have not announced it widely yet. The reality is that Internet data is so scarce, and the pressure on researchers to publish papers so strong, that researchers often just look around for data sets that might lead to a publication, rather than investing effort in coming up with the most important questions, ascertaining what data is needed to study them, and then trying to solve all the legal, financial, and logistical obstacles to getting the data. So we muddle through, studying a system whose opaqueness gives the subprime mortgage industry a run for its money. Only our system is already obviously riddled with corruption and worse. Our time will come.

[Disclosure: I've spent the last year on Internet2's Research Advisory Council trying to advise Internet2 management that they must prioritize dealing with data privacy issues, so that they can reasonably support their network science mission. It's slow-going at Internet2 though, in about 14 months we've gotten as far as starting another committee, which Internet2 promises to announce on their web site soon.]