Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis
www.caida.org > publications : papers : 2011 : data_management_challenges
Internet measurement data management challenges
M. Fomenkov and k. claffy, "Internet measurement data management challenges", in Workshop on Research Data Lifecycle Management, Jul 2011.
|   View full paper:    PDF    |  Citation:    BibTeX   |

Internet measurement data management challenges

Marina Fomenkov
kc claffy

CAIDA, San Diego Supercomputer Center, University of California San Diego

Effective Internet measurement raises daunting issues for the research community and funding agencies. Improved understanding of the structure and dynamics of Internet topology, routing, workload, performance, and vulnerabilities remains disturbingly elusive, in part for lack of realistic and representative datasets available to scientific researchers. The dearth is understandable; measurement of operational Internet infrastructure involves managing more complex and interconnected dimensions than measurement in most scientific disciplines: logistical, financial, methodological, technical, legal, and ethical. CAIDA has been navigating these challenges with modest success fifteen years, collecting, coordinating, curating, and sharing data sets for the Internet research and operational community in support of Internet science. Our three current biggest challenges which we hope to explore at the workshop are: sustainable collection, curation, and storage of large volumes of data; privacy-respecting sharing; and long-term archiving for reproducibility.

Keywords: data, measurement methodology
  Last Modified: Wed Oct-11-2017 17:03:58 PDT
  Page URL: http://www.caida.org/publications/papers/2011/data_management_challenges/index.xml