We have collected and reviewed papers published betweeen 1994 and 2009 (please email info@caida.org if you know one that should be added), starting with papers from peer-reviewed academic research conferences, and then including many papers cited from this intial seeding set of papers, as well as follow-up papers written by the same authors. We provide a flexible interactive table that supports selection of relevant attributes from these papers, including data sets and methods used, goals, and basic empirical findings. We use five paper categories: survey, analysis, methodology, tools, and other. Analysis papers typically attempt to derive trustworthy numbers on actual traffic cross-section, while methodology papers focus on methods of classifications. Click on a checkbox below to show that attribute for each paper in a separate column. Below this table is a similar table of attributes of the data sets analyzed in these set of papers.

ID	Title	Year	Publication	Authors	Paper Type	Classfication Goals	Classfication Characteristics	Method	Empirical Findings	% of traffic P2P	PDF	DataID
1	Toward the Accurate Identification of Network Applications	2005	PAM	A. Moore, K. Papagiannaki	Methodology	Coarse-grained Classification	Application Payload	Exact Matching	Port-based: 64.54%BULK;27.30%WWW; Content-based: 45.00%BULK;20.40%WWW;	1.5%		[D-48]
2	Flow Clustering Using Machine Learning Techniques	2004	PAM	A. McGregor, M. Hall, P. Lorier, J. Brunskill	Methodology	Coarse-grained Classification	Flow Characteristics	Machine Learning/Stat (EM)
3	Internet Traffic Classification Using Bayesian Analysis Techniques	2005	SIGMETRICS	A. Moore, D. Zuev	Methodology	Coarse-grained Classification	Flow Characteristics	Machine Learning/Stat (Bayesian)	65% accuracy on per-flow classification and better than 95% with refinements			[D-48]
4	Class-of-service Mapping for QoS	2004	IMC	M. Roughan, S. Sen, O. Spatscheck, N. Duffield	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (NN,LDA)				[D-59][D-68]
5	Is P2P Dying or just Hiding?	2004	GLOBECOM	T. Karagiannis, A. Broido, N. Brownlee, K. Claffy, M. Faloutsos	Methodology	Fine-grained Classification	Application Payload	Exact Matching	2003:HTTP(72%,47.7%);SMTP(1.3%,1.2%);P2P(8%,10.7%) 2004:HTTP(56%,52.1%);SMTP(3.2%,9.7%);P2P(14%,9.9%)	8%,10.7% in 2003;14%,9.9% in 2004		[D-69]
6	Transport Layer Identification of P2P Traffic	2004	SIGCOMM	T. Karagiannis, A. Broido, M. Faloutsos, K. Claffy	Methodology	Coarse-grained Classification (I)	Flow Characteristics	Heuristics	P2P traffic continues to grow unabatedly	15%-20%		[D-70]
7	An Analysis of Internet Chat Systems	2003	SIGCOMM	C. Dewes, A. Wichmann, A. Feldmann	Profiling/Analysis				miss less than 8.3% of all existing chat connections and to correctly classify at least 93.1%			[D-67]
8	Profiling Internet Backbone Traffic: Behavior Models and Applications	2005	SIGCOMM	K. Xu, Z. Zhang, S. Bhattacharyya	Profiling/Analysis							[D-49]
9	Accurate Scalable In-Network Identification of P2P Traffic	2004	WWW	S. Sen, O. Spatscheck, D. Wang	Methodology	Coarse-grained Classification (I)	Flow Characteristics	Heuristics	less than 5% false positive and false negative ratios			[D-71][D-72]
10	BLINC Multilevel Traffic Classification in the Dark	2005	SIGCOMM	T. Karagiannis, K. Papagiannaki, M. Faloutsos	Methodology	Coarse-grained Classification	Flow Characteristics	Heuristics	web:14%,37.5%,33.5%;data(ftp):67.4%,7.6%,5.4%;	1.2%,31.9%,31.3%		[D-48]
11	CoralReef Software Suite as a Tool for System and Network Administrators	2001	LISA	D. Moore, K. Keys, R. Koga, E. Lagache, K. Claffy	Tools
12	Packet-level Traffic Measurements from the Sprint IP Backbone	2003	IEEE Network	C. Fraleigh, S. Moon, B. Lyles, C. Cotton, M. Khan, D. Moll, R. Rockell, T. Seely	Profiling/Analysis				over 90% flows have packet sizes of 1495 bytes or greater	0.1%-80%(p2p+unknown)		[D-77]
13	Snort Lightweight Intrusion Detection for Networks	1999	LISA	M. Roesch	Tools							[D-66]
14	Internet Traffic Characterization	1994		K. Claffy	Profiling/Analysis
15	Discriminators for Use in Flow-based Classification	2004		A. Moore, D. Zuev, M. Crogan	Other							[D-48]
16	Identifying the TCP Behavior of Web Servers	2001		J. Padhye, S. Floyd	Profiling/Analysis							[D-58]
17	Inherent Behaviors for On-line Detection of Peer-to-Peer File Sharing	2007	IEEE Global Intenet	G. Bartlett, J. Heidemann, C. Papadopoulos	Methodology	Fine-grained Classification (I)	Flow Characteristics	Heuristics	achieve up tp an 83% true positive rate with only a 2% false positive rate			[D-6][D-7]
18	Graption: Automated Detection of P2P Applications Using Traffic Dispersion Graphs	2008	Technical Report	M. Iliofotou, P. Pappu, M. Faloutsos	Methodology	Coarse-grained Classification (I)	Flow Characteristics	Heuristics	more than 90% precision and recall for P2P detection	9.19%		[D-8][D-11]
19	Traffic Classification through Simple Statistical Fingerprinting	2007	SIGCOMM CCR	M. Crotti, M. Dusi, F. Gringoli, L. Salgarelli	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (Normalized thresholds)				[D-12]
20	Traffic Classification Using Clustering Algorithms	2006	SIGCOMM	J. Erman, M. Arlitt, A. Mahanti	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (K-Means, DBSCAN)	47.3%(Bytes,HTTP);35.1%(Bytes,P2P);6.0%(Bytes,SMTP)	35.1%(Bytes)		[D-36][D-39]
21	Unexpected Means of Protocol Inference	2006	IMC	J. Ma, K. Levchenko, C. Kreibich, S. Savage, G. Voelker	Methodology	Fine-grained Classification	Application Payload	Machine Learning/Stat (Product Distribution; Markov Processes; CSG)				[D-40][D-41][D-42]
22	On Inferring Application Protocol Behaviors in Encrypted Network Traffic	2006	Journal of Machine Learning Research	C. Wright, F. Monrose, G. Masson	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (HMM)	achieve greater than 90% for serveral protocols in aggregate traffic			[D-43]
23	Traffic Classification on the fly	2006	SIGCOMM CCR	L. Bernaille, R. Teixeira, I. Akodjenou, A. Soule, K. Salamatian	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (K-Means)				[D-44]
24	Blind Application Recognition Through Behavioral Classification	2005		L. Bernaille, A. Soule, M. Jeannin, K. Salamatian	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (HMM)
25	Early Application Identification	2006	CONEXT	L. Bernaille, R. Teixeira, K. Salamatian	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (K-Means, GMM, Spectral Clustering)	classify known applications with an accuracy over 90%; identify new applications as unknown with a probability of 60%			[D-15][D-16][D-17][D-18][D-19][D-20]
26	Early Recognition of Encrypted Application	2007	PAM	L. Bernaille, R. Teixeira	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat	more than 85% accuracy in recognizing the application in an SSL connection			[D-13][D-14][D-20]
27	Traffic Classification using a Statistical Approach	2005	PAM	D. Zuev, A. Moore	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (Bayes)	achieve better than 83% accuracy on both a per-byte and a per-packet basis			[D-48]
28	Appmon: An Application for Accurate per Application Network Traffic Characterization	2006	BroadBand Europe	D. Antoniades, M. Polychronakis, S. Antonatos, E. Markatos, S. Ubik	Tools
29	Profiling the End Host	2007	PAM	T. Karagiannis, K. Papagiannaki, N. Taft, M. Faloutso	Profiling/Analysis							[D-21][D-22]
30	Revealing Skype Traffic: when randomness plays with you	2007	SIGCOMM	D. Bonfiglio, M. Mellia, M. Meo, D. Rossi, P. Tofanelli	Profiling/Analysis							[D-24][D-25]
31	A Traffic Characterization of Popular on-line Games	2005	IEEE/ACM Transactions on Networking	W. Chang Feng, F. Chang, W. chi Feng, J. Walpole	Profiling/Analysis							[D-52]
32	Hit-list worm detection and bot identification in large networks using protocol graphs	2007	RAID	M. Collins, M. Reiter	Profiling/Analysis							[D-23]
33	Identifying Known and Unknown Peer-to-Peer Traffic	2006	IEEE NCA	F. Constantinou, P. Mavrommatiss
34	Offline/realtime Traffic Classification Using Semi-supervised Learning	2007	Perform. Eval	J. Erman, A. Manhanti, M. Arlitt, I. Cohen, C. Williamson	Methodology	Coarse-grained Classification		Machine Learning/Stat (K-Means)	37.4%(Bytes,P2P,Campus);80%(Bytes,P2P,Residential);61.5%(Bytes,P2P,WLAN)	37.4%(Bytes,Campus);80%(Bytes,Residential);61.5%(Bytes,WLAN)		[D-26]
35	Identifying and Discrimination between Web and Peer-to-Peer Traffic in the Network Core	2007	WWW	J. Erman, A. Manhanti, M. Arlitt, C. Williamson	Methodology	Coarse-grained Classification	Flow Characteristics	Machine Learning/Stat (K-Means)		38.3%		[D-27]
36	Acas: Automated Construction of Application Signatures	2005	SIGCOMM	P. Haffner, S. Sen, O. Spatscheck, D. Wang	Methodology	Fine-grained Classification	Application Payload (64 bytes)	Machine Learning/Stat (Naive Bayes, AdaBoost, Maximum Entropy)				[D-53]
37	Comparison of Internet Traffic Classification Tools	2007	IMRG WACI	H. Kim, K. Claffy, M. Fomenkova, N. Browlee, D. Barman, M. Faloutsos	Survey/Compare							[D-28][D-29][D-30][D-31][D-32][D-33][D-34]
38	A Survey of Techniques for Internet Traffic Classification Using Machine Learning	2008	IEEE Communications Surveys and Tutorials	T. Naguyen, G. Armitage	Survey/Compare
39	Towards Automated Application Signature Generation	2008	NOMS	B. Park, Y. Won, M. Kim, J.Hong	Methodology	Fine-grained Classification	Application Payload	Heuristics				[D-1]
40	Analyzing Peer-to-Peer Traffic across Large networks	2004	IEEE/ACM Transactions on Networking	S. Sen, J. Wang	Profiling/Analysis							[D-73]
41	Role Classification of Hosts within Enterprise Networks based on Connection Patterns	2003	USENIX	G. Tan, M. Poletto, J. Guttag, F. Kaashoek	Methodology		Flow Characteristics	Heuristics				[D-64][D-65]
42	Self-learning IP Traffic Classification based on Statistical Flow Characteristics	2005	PAM	S. Zander, T. Nguyen, G. Armitage	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (EM)				[D-36]
43	Tunnel Hunter: Detecting Application-Layer Tunnels with Statistical Fingerprinting	2008	Computer Networks	M. Dusi, M. Crotti, F. Gringoli, L. Salgarelli	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat				[D-2]
44	A Preliminary Look at the Privacy of SSH Tunnels	2008	ICC	M. Dusi, F. Gringoli, L. Salgarelli	Methodology	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (GMM)				[D-3]
45	A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detecion	2003	SIAM	A. Lazarevic, L. Ertoz, V. Kumar, A. Ozgur, J. Srivastava	Survey/Compare							[D-63]
46	Automatically Inferring Patterns of Resources Consumption in Network Traffic	2003	SIGCOMM	C. Estan, S. Savage, G. Vargheses	Methodology	Coarse-grained Classification	Flow Characteristics	Heuristics				[D-60][D-61][D-62]
47	Mining Anomalies Using Traffic Feature Distributions	2005	SIGCOMM	A. Lakhina, M. Crovella, C. Diot	Methodology	Coarse-grained Classification		Heuristics (K-Means, Hierarchical Agglomerative Algorithm)				[D-54][D-55]
48	Network Traffic Analysis using Traffic Dispersion Graphs (TDGs): Techniques and Hardware Implementation	2007	Technical Report	M. Iliofotou, P. Pappu, M. Faloutsos, M. Mitzenmacher, S. Singh, G. Varghese	Tools							[D-35][D-36][D-37][D-38]
49	Heuristics to Classify Internet Backbone Traffic based on Connection Patterns	2008	ICOIN	W. John, S. Tafvelin	Methodology	Coarse-grained Classification	Flow Characteristics	Heuristic	leave only 0.2% of the data unclassified; can identify 95% of P2P flows	42%(average) in connections, 79%(average) in traffic		[D-4]
50	Trends and Differences in Connections Behavior within Classes of Internet Backbone Traffic	2008	PAM	W. John, S. Tafvelin, T. Olovsson	Profiling/Analysis				P2P and HTTP traffic exhibit different peak times.P2P traffic was found to be clearly dominating with 90% of the transfer volums, especially during evening and night times. In contrast, HTTP traffic has its main activities(9% of the data volumes) during office hours.	93%(evening);91%(night);86%(office hours)		[D-4][D-5]
51	Flow Analysis of Internet Traffic: World Wide Web versus Peer-to-Peer	2005	System and Computers in Janpan	M. Perenyi, T. Dang, A. Gefferth, S. Monlnar	Profiling/Analysis				57.52% for WWW, 21.53% for P2P, 20.95% for other	21.53%		[D-56]
52	Identification and Analysis of Peer-to-Peer Traffic	2006	Journal of Communications	M. Perenyi, T. Dang, A. Gefferth, S. Monlnar	Methodology	Fine-grained Classification	Flow Characteristics	Heuristics		60%-80%		[D-45]
53	Analysis of Peer-to-Peer Traffic on ADSL	2005	PAM	L. Plissonneau, J. Costeux, P. Brown	Profiling/Analysis				40% of connections are only connection reattempts, and it concerns about 30% of peers	60% in 2004, 65% in 2003(lies on P2P ports)		[D-57]
54	Analysis of Internet Backbone Traffic and Header Anomalies Observed	2007	IMC	W. John, S. Tafvelin	Profiling/Analysis							[D-4]
55	Flow Classification by Histograms or How to Go on Safari in the Internet	2004	SIGMETRICS	A. Soule, K. Salamatian, N. Taft, R. Emilion, K. Papagiannaki	Methodology	Coarse-grained Classification	Flow Characteristics	Machine Learning/Stat (EM)				[D-74]
56	Streaming Video Traffic: Characterization and Network Impact	2002	WCW	J. Merwe, S. Sen, C. Kalmanek	Profiling/Analysis							[D-59]
57	The architecture of CoralReef: an Internet traffic monitoring software suite	2001	PAM	K. Keys, D. Moore, R. Koga, E. Lagache, M. Tesch, K. Claffy	Tools
58	A Preliminary Performance Comparison of Five Machine Learning Algorithms for Practical IP Traffic Flow Classification	2006	SIGCOMM	N. Williams, S. Zander, G. Armitage	Survey/Compare							[D-36][D-46][D-47]
59	Internet Traffic Identification using Machine Learning	2006	GLOBECOM	J. Erman, A. Mahanti, M. Arlitt	Methodogy	Fine-grained Classification	Flow Characteristics	Machine Learning/Stat (EM, Bayes)	81.2%http;3.1%smtp;2.0%dns			[D-36]
60	Estimating P2P Traffic Volume at USC	2007		G. Bartlett, J. Heidemann, C. Papadopoulos, J. Pepin	Profiling/Analysis		Port Number and Flow Characteristics	Exact Matching and Heuristics	3%-13% of active hosts on campus participate in P2P	21%-33%(less than, Byte)		[D-75]
61	A Longitudinal Study of P2P Traffic Classification	2006	MASCOTS	A. Madhukar, C. Williamson	Survey/Compare				30%-70% of the campus Internet traffic for 2003-2005 was P2P	30%-70%		[D-76]
62	On the Validation of Traffic Classification Algorithms	2008	PAM	G. Szabo, D. Orincasy, S. Malomsoky and I. Szabo	Other	Fine-grained Classification, for validate classifcation methods			P2P:70%;Web:26%;VoIP:2%;Streaming:1%;Secure Channel:1%;	70%		[D-78]
63	Accurate Traffic Classification	2007	WoWMoM	G. Szabo, I. Szabo and D. Orincasyo	Methodology	Fine-grained Classification						[D-79]
64	Observing Slow Crustal Movement in Residential User Traffic	2008	ACM CoNEXT	Kenjiro Cho, Kensuke Fukuda, Hiroshi Esaki and Akira Kato	Analysis		Port Number		Between May 2005 and May 2008, the average annual growth rate of (A1) RBB customers: 26% for inbound; 28% for outbound; 27% for the combined volume;	0.92%(gnutella,2005);0.25%(bittorrent,2005);0.12%(edonkey,2005);0.94%(gnutella,2008);0.22%(bittorrent,2008);0.13%(edonkey,2008);		[D-80][D-81]
65	Classification of Network Traffic via Packet-Level Hidden Markov Models	2008	GLOBECOM	Alberto Dainotti, Walter de Donato, Antonio Pescape and Pierluigi Salvo Rossi	Methodology		Flow Characteristics	Machine Learning (HMM)				[D-82][D-83]
66	TIE: a Community-oriented Traffic Classification Platform	2009	IMA	Alberto Dainotti, Walter de Donato, Antonio Pescape and Giorgio Ventre	Tools
67	Challenging Statistical Classification for Operational Usage: the ADSL Case	2009	IMC	Marcin Pietrzyk, Jean-Laurent, Guillaume Urvoy-Keller and Taoufik	Analysis		TCP Flow Characteristics	Machine Learning/Stat	Most bytes and flows are due to eDonkey; The vast majority of traffic in the HTTP Streaming class is due to Dailymotion and Youtube;	5%-40%(flows);10%-50%(bytes)		[D-84]
68	On Dominant Characteristics of Residential Broadband Internet Traffic	2009	IMC	Gregor Maier, Anjia Feldmann, Vern Paxson and Mark Allman	Analysis		Port Number and Application Payload	Exact Matching	HTTP carries more than 50% traffic; Flash Video contributes 25% of all HTTP traffic;	14%(bytes)		[D-85][D-86]

Datasets

Discussion

P2P traffic is one of the most challenging traffic types to classify, partly due to substantial legal interest in identifying it and even more substantial negative repercussions to the user if P2P traffic is accurately identified. The misaligned incentives between those who want to use and those who want to identify P2P applications, together with the tremendous legal and privacy constraints against traffic research, renders scientific study of this question near impossible, and even if possible, wide variation across links would prevent a simple numeric answer to the question of how much P2P traffic there is on the Internet. But our taxonomy does reveal insights: the fraction of peer-to-peer file sharing traffic observed ranges from 1.2% to 93% across the 18 papers that provide such numbers. We also know that the average fractions reported have increased considerably from 2002 to 2006 (Table 1). Tables 2 and 3 show that results also vary widely by link and geographic location. Table 3 suggests that P2P is more popular in Europe, probably due to stricter policies (MPAA and RIAA) in North America. Note that the Asian results are from Japanese data sets, in which 1.34% and 1.29% are based on port numbers and therefore likely to significantly underestimate the fraction of P2P traffic. Furthermore, the amount of P2P traffic also varies by time of day, with higher fractions at night (Table 4).

One study [34] suggests that peer-to-peer applications are used more often at home than in the office. Finally, a study [50] in Europe found a higher fraction of P2P traffic on a university link in Europe than some Canadian academics [34] found on their campus.

Some numbers are based on statistical or host-behavioral classification. The remaining numbers are based on P2P detection via payload signature matching, the most reliable method of detecting an application (if unecrypted), which however is fraught with legal and privacy issues.

UDP Traffic Analysis

It's still an accepted assumption that Internet traffic is dominated by TCP, which is also the basic of most current traffic classification works; however, the rise of new streaming applications (e.g. IPTV such as PPStream, PPLive) and new P2P protocols (e.g. uTP) trying to avoid traffic shaping techniques is expected to increase the usage of UDP as transport protocol.

In this analysis section, we collect some UDP analysis from existing works, and then compare the usage of UDP and TCP on several traffic traces colleced in different network and geographical locations, as well as in different time periods.

Table 5 shows that UDP/TCP ratio ranges from 0.01 to 0.20 based on the existing works (There is a high value in residential trace). For better evaluating the amount of UDP and TCP traffic on real-traces (in terms of flows, packets and bytes), we analyze several available traces collected in the period 2002-2009 on serveral backbone links located in the US and Sweden. Table 6 shows that the use of UDP as transport protocol has rapidly increased from 2002 to 2009, although TCP sessions are still responsible for most of packets and bytes. However, in terms of flows UDP turns out to be the dominant transport protocol.

Conclusion

This overview page presented a rough taxonomy of traffic classification approaches, based on features, methods, goals and data sets.

Our survey review also reveals shortcomings with current traffic classification efforts. The variety of data sets used does not allow systematic comparison of methods. Few research groups (can) share their datasets. Already true ten years ago, the field of traffic classification research still needs publicly available, modern data sets as reference data for validating approaches. The poor comparability of results is further amplified by the lack of standardized measures and classification goals. For example, there exists no clear definition for traffic classes such as P2P or file-sharing.

However, the taxonomy above allows meta-analyses of relevant open questions, such as trends and development of traffic classes or features, yielding new insights into Internet traffic. We showed this by shedding some insight on questions such as: "how much of modern Internet traffic is P2P?" Though we found some trends and indications, we have far too little data available to make conclusive claims beyond "there is a wide range of P2P traffic on Internet links; see your specific link of interest and classification technique you trust for more details."

Acknowledgments

This work was made possible thanks to funding from DHS-PREDICT, the National Science Foundation, Beijing Jiaotong University, and the China Scholarship Council.