Macroscopic Internet visualization and measurement

k claffy
Brad Huffaker
CAIDA

(current version of software)

Visualization of the Internet beyond the purview of a single provider has become a challenging task that has received little attention from the research community. We have developed a tool for visualizing the infrastructure of multiple backbone providers simultaneously, and for updating and correcting information that may be invalid or out of date. The tool's accuracy unimitigatably relies on the cooperation of providers in keeping the information accurate. The tool also includes a module for determining performance or routing information about components of the infrastructure amenable to measurement.

Goal I: macroscope

Visualization of the Internet beyond the purview of a single provider has become challenging and received little attention from the research community. Commercial Internet service providers grow increasingly hesitant to voluntarily publish topology or peering information about their network, and those that do so have no standard format to use, leaving the maps that do exist fairly incomparable with eachother.

We have developed a tool for visualizing the infrastructure of multiple U.S. backbone providers simultaneously. We divided each backbone infrastructure into a group of nodes (POPs) and pipes between these nodes, drawing them based on their geographical location on a map of the United States.

At the same time, the evolution of cooperative research networks, currently framed by the NSF new connections to the vBNS and cooperating U.S. federal agencies, has given rise the the desire for being able to share performance information across infrastructures, perhaps even commerical ones. We are also developing a module that will allow one to request performance or routing information across arbitrary points of the infrastructure amenable to measurement.

a) creating the database

Our first step was to create not only a database of backbone topologies, but also the latitude and longitude information for each one. Even though we limited our scope to a small number of NSPs (the largest ones), completely representing all of even this subset at once leaves an image too congested to be meaningful.

We approached this problem using a Java applet as an interactive front end, through which the user can click on a backbone to direct the applet to add or remove it to the map in real-time. Since the user likely only wants to see a specific subset of the data at any given time (perhaps for use in determining peering possibilities), this flexibility in presentation offers clearer understanding of the data subsets of interest, and allows the user to control the complexity.

For the commerical backbones we used backbone topology data from the http://www.boardwatch.com/ web site, which many of our backbone contacts indicated was significantly inaccurate, but was all we had to start with. The research backbone topology was gathered from a number of sources. Do to the great difference in representation across these sources many assumptions where made, while this is regretable it was also unaviodable. Indeed, infrastructure is now so rapid that any database would have been obsolete before we had even finished the interface, so an important component of our tool would be a mechanism for accepting updates from backbone infrastructure representatives.

Figure 1: Simultaneously viewing several backbones, as indicated in the lower left frame, and also identified there by color and point of contact. The link over which the mouse sits reflects its city pair and associated ISP in the right frame.

We also allow the user to remove the background map (Figure 2), in case it is occludes a desired view.

Figure 2: View without the background map.

---

b) support for updating the database

A separate applet allows users to update information regarding infrastructures for which they may be responsible. The updates are sent to a human for off-line authentication and approval before the stored database is modified. Figure 3 shows the tool display after an authorized CRL representative has added 2 links from Atlanta, GA to Chicago, IL.

Figure 3: Support for updating backabone information that may be inaccurate or about to change due to upgrades/expansion. The map displays the modified backbone topology after an authorized CRL representative has added 2 links from Atlanta, GA to Chicago, IL.

---

Goal II: use as management platform

We are also developing a server-client application that allows the user to not only draw network diagrams such as those in the above platform, but also use the resulting diagram as an interface to their own or related network's statistics. Each node in the diagram represents a host or router, on which the network operator can set monitors to report real time status on the host.

Client-server implementation

As the multi-backbone visualization, the web user interface to the management platform is also a Java applet, which makes it easy for the user to draw and manipulate the tool interactively. However, it is not possible to implement the statistics monitoring functionality in an applet, because applets do not support communication with third party hosts (i.e., one besides the server and client hosts). for security reasons.

Therefore, we use a server written in C to implement the monitoring queries. The server then uses the {\em scotty) tool \cite{scotty} to query the monitored hosts and return the information to the Java client.

Client

Using information received in response to queries from the server, the client is responsible for displaying information, network topologies, and accepting input from the user. The client establishes a standard network socket to the server, although one can use a client for topology modification, just not network statistics queries, without a connection to a server. Client side disks may not be used unless the applet is running locally, as a standalone. Specific features of the Client include:

1. allows user to draw nodes and the network connections between them.
2. allows user to assign host names and IP address to each node.
3. add/move/delete/clear network nodes from the map.

Server

The server is a daemon that runs in the background, responding on a specific port to client queries regarding system statistics, e.g., network loads, disk usage, snmp, traceroute information. The first time the server receives a query regarding a particular host, the server passes the query on to the host in question, since the Java applet is itself unable to access any host but the one on which it runs. The server may then cache the information for some configurable interview so as not to overwhelm the network with repeated requests for identical or similar information.

The server has no graphical user interface. It is basically a daemon that disseminates and collects queries and responses, effectively a proxy. It also maintains a log of the services it performs. Server functions include:

1. start a thread, set up a socket and listen.
2. when a client tries to connect, spawn off another thread that will go to step 1.
3. respond to client requests with the requested information.
4. log the source of the request, the information requested, and cache the response for a configured timeout duration.
5. when all requests are serviced the thread will die.

Related Work

The second part of this tool is similar to the graphical display functionality of Tkined \cite{tkined}. Indeed, we were motiviated to implement our tool because of some of the limitations Tkined poses when trying to view its results, or reconfigure the administered topology, via the web. We have taken a client/server approach, where a client allows the user to specify a topology and make statistics queries to the server, who then dispatches the requests and response to the client with the response when it is available.

A key difference of our tool as a mangement mechanism is its ability to display to anyone with a web browser; one does not need to download the whole Tkined package to monitor a network on they're not necessarily an operator, but a serious user who understands enough to benefit from fairly low-level measurement information, as well as able to cooperatively debug problems involving his own administrative domain.

This tool does not attempt to replace propietary NOC-based tools such as SunNet Manager or HP openview. We rather intended it as an interface to publically (or limited publically, say to a research community) available statistics. Current statistics interfaces for research networks (and even operational ones) tend to involve deep branching of web-based stats or snmp MIB trees. Another related tool is ASExplorer [3], developed by Merit's IPMA project team. ASExplorer displays a representation of real-time routing topology and instability among Internet autonomous systems (ASes). A graphical interface map depicts ASes, which typically map to network service providers (NSPs) or Internet service providers (ISPs), connected by links reflecting their level of routing information exchange activity. Our tool currently focuses less on the exchange of routing data than on geographical topology and peering information, and then the measurement of a particular network of interest. We consider it complementary to ASExplorer, and perhaps holds future potential for integration with ASExplorer functionality.

Data

(to be included)

Figure 3: Zooming in on a single backbone, where each node supports workload or performance queries by an external scotty module. (Note: this is the Tkined version; we are still porting this to Java so interactive like the first part of the tool, rather than this static (oversized, sorry (: ) .gif image. But we don't have it ready yet...)

Conclusion

We have developed a tool that can bring to the public (or an authorized subset of it) information typically reserved for a backbone's private Network Operations Center (NOC). The first component of our tool provides the ability to visualize multiple backbone infrastructures in real-time, including topology as well as peering data. Because the tool will be considerably less useful to users if it reflects inaccurate data, we have devoted considerable attention to the usability of a mechanism for authorized updates of topology and peering information updates, with human checkpointing before changes are committed to the database.

The latter part of our tool also aims at opening up network monitoring and measurement capability to a class of users somewhere between `non-technical' and `full-time network operator'. Development of this module of the tool was inspired by our need for network researchers to be able to assess workload profiles on their research infrastructures, as well as evaluate and evolve measurement and and data visualization techniques for use by the larger Internet industry. We hope that the tool will benefit those outside our immediate domain, both intranet and Internet service providers, and indeed that requests for extensions to extend its utility will be forthcoming.

Bibliography

1. Scotty - Tcl Extensions for Network Management Applications, Jurgen Schonwalder, Simple Group, University of Twente, http://wwwsnmp.cs.utwente.nl/~schoenw/scotty/
2. Tkined - A Tcl/Tk-based network editor, Jurgen Schonwalder, Simple Group, University of Twente, http://wwwsnmp.cs.utwente.nl/~schoenw/scotty/#TKINED
3. ASExplorer tool: visualizes dynamic Internet routing behavior, including maps of Internet exchange point stability levels. Merit IPMA http://www.merit.edu/ipma/asexplorer/

---

Acknowledgments

This material is (partially) based on work sponsored by the National Science Foundation under NSF Cooperative Agreement No. NCR-9415666.

---

last updated 11 jul 97, questions, feedback to kc@caida.org