Evi Nemeth (firstname.lastname@example.org)
University of Colorado and
Cooperative Association for Internet Data Analysis (CAIDA)
Theresa Ott (email@example.com)
Kevin Thompson (firstname.lastname@example.org)
k claffy (email@example.com)
The Internet Engineering Curriculum Repository (IEC) project includes an archive of teaching and laboratory materials focused on networking that are appropriate for a wide variety of classes. This dynamic collection of materials makes it easier for faculty to keep their courses up to date and complete. It also captures some one-time events, such as colloquium talks, plenary addresses, and conference presentations that can enrich a student's understanding of a topic. We also discuss laboratory components of classes that can be built with industry/academia cooperation to improve the educational experience.
In early 1998, CAIDA, the Cooperative Association for Internet Data Analysis began efforts to create a web-based repository of Internet Engineering Curriculum materials. At that time many faculty were beginning to use the web for teaching materials, but the completeness and quality of the materials varied widely. The project was motivated by a serious need for trained network engineers in the United States and around the world due to the exponential growth of the Internet and the need for at most linear growth of students with networking background in their curriculum vita. At a NPACI (National Partnership for Advanced Computing Infrastructure) meeting of the top research universities in the U.S. in the Spring of 1998, only one fourth had undergraduate networking courses at their academic institutions.
The first time that a faculty member teaches a course, the preparation workload is many times the load of subsequent offerings of the same course. And if the course topic is outside their research area, they are often not aware of the classic literature on the topic, the recent seminal papers, and new results in the area. Networking technologies are changing quite rapidly, especially when compared to other disciplines or sub-disciplines in computer science, and so even keeping a course current requires additional preparation each semester. And while networking is clearly an important part of today's computer science curriculum, it was not as prominent when many current faculty were trained. The paucity of networking courses is not due to lack of interest on the students' part but rather to the lack of qualified faculty to teach them. The IEC repository of teaching materials was created to help faculty develop new networking courses and keep their courses current.
This paper describes: the building of the repository and its current contents; the Internet Teaching Laboratories (ITLs) associated with it; a CD to support student projects in Internet measurement and traffic analysis; and an experiment at using the web and the MBONE (Multicast Backbone) to deliver a networking class to students at two locations.
The IEC Repository (http://iec.caida.org) contains teaching materials contributed by their authors on a wide variety of network related topics. We have focused on networking classes from universities at the undergraduate and graduate level, but have also included vendor training materials, workshops, tutorials, seminars, and reference materials contributed by individuals from around the world.
We are beginning to experiment with augmenting the repository with audio/video materials. We have the 1999 Sigcomm Conference presentations and tutorials in RTP audio/video files available online. Ultimately we envision a situation where an instructor can assign a paper and expect the student not only to read it, but also to see and hear the author's original presentation of the material.
The primary value of the repository is not the collection of materials, but rather the fact that pointers to these materials are all in one spot. The materials are indexed, categorized, and reviewed for quality and appropriateness; their completeness is indicated.
Our initial motivation to build the IEC repository stemmed from the fact that CAIDA's ISP contacts, backbone engineers in the trenches keeping the Internet healthy, were often complaining that they could not hire qualified staff. College graduates who they hired had little or no networking background, even computer science or computer engineering majors. Telecommunications majors had telephone networking experience and coursework but no data networking training. Typically, it took a year to train new graduates to the point that they were useful as Internet engineers.
When we looked further, not only were the University classes non-existent or out-of-date, but they contained no laboratory component.
The IEC repository lets networking instructors get ideas for classes and assignments, access other instructors' lecture materials and exams, and see what other similar institutions do in their networking classes. It provides an indirect communication vehicle in the teaching world; direct communication is common in the research arena, but not in teaching.
The repository contains several types of resources. Courses are typically maintained at their home site and not archived at caida.org. The exception is for faculty who change and re-build their course web site every semester; for those instructors, if they request it, we archive the previous semesters' course and replace it with the current one at the end of the semester.
We categorize the materials in the repository as follows:
Our University courses are broken down into 4 categories: undergraduate, graduate, joint ugrad/grad, and non-technical networking courses. We provide a cover page for each course that summarizes the essentials: university, textbook, level, duration, pre-requisites, date offered, course URL, course description, and catalog description.
The easiest way to get a feel for the IEC repository is to see the cover
page for a sample course. This is a networking course taught by Jim
Kurose from the University of Massachusetts.
The navigational frame at the left can be used to go directly to the course syllabus, lectures, or homeworks. As a web site grows over the course of a semester, it sometimes becomes harder to navigate, especially for someone not familiar with the site. We have added this navigational frame to all courses. Clicking on the arrows with a + on them expands the topic; the example below is for Jim Kurose's course again, looking in detail at a lecture on the transport layer and congestion control.
The University Courses sections of the IEC repository have evolved more than the tutorials and A/V resources. We have found it difficult to get permission to include vendor training materials and sometimes find that tutorials given at conferences carry copyrights preventing their use even with the author's permission.
Our non-English section is just beginning with a set of courses in Japanese and one in French. We need to find collaborators in those countries who speak English and are willing to do basic course level indexing so non-English speaking users can see the course availability and details.
We use tools to scan the courses in the repository about once a semester and find dead links. We send the link information to the author to fix; most do fix them and thank us for pointing out the missing or moved links. Each course has a completeness icon varying from 1 (not at all complete) to 4 (very complete) checks. A 4-check course has lectures, assignments, syllabus, etc. and very few bad links. A single check indicates a course with only a syllabus or description file.
On our cover page for each course we add a navigational frame that allows users to quickly see the structure of the course site and to shortcut to an area via a small table-of-contents driven by a cgi-bin perl script that indexes lectures, assignments, syllabus, projects, reading, etc. It reads a data file that must be built by hand as each course is added to the repository. The data file defines the number of levels of the table of contents represented in the navigational frame and the text to display.
There are several indexes in the repository that we keep up to date when new courses are added:
The textbook index lists each course using a certain text together, so that a faculty member who has already chosen a text can see what other universities use that text, for what courses, and what materials are targeted to that textbook.
The assigned papers index lists the research papers assigned as reading in any course. Beside each paper is a count of the number of courses that assigned it. This allows faculty trained in another discipline of computer science to learn about the literature in the networking field. A paper with a count of 1 is probably in a very specialized area, whereas a paper with a count of 20 is probably a classic in the field.
The intellectual property rights to the course materials remain with their authors or universities. Some faculty own their course materials; others are owned by the university. In some cases, the university owns the courses but the faculty do not know it; if you are considering contributing your courses, please check with your university on ownership rights. We had one instructor rescind permission to list his course because his university objected.
We always ask each author's permission to include their materials in the repository. Often this initiates a dialog that keeps us informed if the materials move to a new location or server, or new courses are added. The iec.caida.org web site has a form for authors to fill out to submit their course for inclusion (iec.caida.org/contribute.html).
In order to make it easier for faculty from outside the networking area to use the IEC repository and design networking courses, we offer summer workshops of about a week duration. The classes are focused, going deeply into a topic rather than broadly covering several topics. The topics are taught by academic or industry experts. The target audience for these workshops is a University professor who will teach networking for the first time and the graduate student who will be the Teaching Assistant for the course.
The workshop classes are intensive, with a duration of 1-5 days, 1-2 days per topic in a lecture/lab format. The first was three days total with classes covering:
The August 1999 workshop was attended by about 40 faculty from throughout the United States.
The next workshop will be held June 26-30, 2000 with similar topics to the 1999 workshop. We intend to expand the routing class into a 2-day lab class and to have tutorials on multicast and quality of service, specifically covering the recent developments in these fields in the IETF.
Growing concerns about the number of network-savvy engineers entering the workforce has led to a novel collaboration between industry and academia to develop hands-on teaching laboratories to supplement existing university networking curricula.
Few of the courses in the repository have laboratory components, primarily due to the lack of equipment and trained staff to operate laboratories at most universities. This situation results in network engineers with no hands-on exposure to modern networking equipment in their university training. It is common in the computer industry in the U.S. for a vendor to offer an upgrade path that includes trading in the old equipment, which the vendor often does not really want to refurbish and resell. Some vendors let their sales force donate used (surplus) equipment to local universities, some crush it for scrap metal.
We were fortunate to be the recipient of such an upgrade program as InternetMCI, now Cable and Wireless, upgraded their backbone routers to the latest models, leaving Cisco with a surplus of last generation equipment. Generous donations of equipment from Cisco Systems and Cable and Wireless, engineering expertise from MCI Worldcom, and financial support from the National Science Foundation, is allowing CAIDA to distribute this surplus equipment and set-up Internet Teaching Labs (ITLs) at U.S. universities. CAIDA is creating ITL facilities at approximately 25 U.S. universities.
An IEC Advisory Board consisting of Internet leaders from ATT Research, Cisco Systems, Harvard, MCI Worldcom, University of College London, University of Kentucky, University of Massachusetts, and Verio assisted in selecting the schools to receive the lab equipment based on competitive proposals that were submitted. Technical merit and educational impact were the key criteria governing the selection. Included in those broad criteria were faculty involvement, curriculum development and implementation, campus support for the lab, and cooperation between Electrical Engineering, Computer Science, and Telecommunications Departments.
With this initial effort we hope to establish ITL facilities at many Universities -- small teaching schools as well as the large research Universities. The IEC repository will also collect and taxonimize the various laboratory exercises used in the labs. We expect other vendors and ISP's to view participation in this program as a great way to re-use trade-in equipment and strengthen our network engineering educational process without impacting sales. None of the ITL equipment can be used in the operational infrastructure; it is strictly for teaching labs. Continued donations can keep the initial laboratories up to date, diversify the equipment in them, and create new ITLs at other universities. Our ultimate goal is to enhance the number and quality of university graduates trained in Internet technologies.
We view the IEC repository and the ITL initiative as prototypes for potentially much larger global paradigms where academia, major Internet providers, and network equipment manufacturers work together to regularly augment educational materials in university teaching labs and collaborate to nurture the next generation of Internet engineers.
The Internet Teaching Lab at the University of Virginia demonstrates the effectiveness of the ITL model. Created in December 1998 through equipment donations and engineering expertise from individuals in the networking community, the lab is outfitted with several Cisco 7000s, 2 Lightstream 2020 ATM switches, 2 Cisco 2500 routers, 4 T-1 DSU/CSU's, 2 FDDI concentrators, and 6 student workstations. The larger routers are fully populated with a diverse set of network interface cards.
Students work in teams of two, in a laboratory class that accompanies the regular Internet Engineering class. Professor Jorg Liebeherr has designed the lab exercises and produced a lab manual; they are available at http://www.cs.virginia.edu/vintlab/. Student enrollment has doubled since the Lab section was introduced.
Another CAIDA activity, CoralReef, http://www.caida.org/tools/measurement/coralreef/, has produced software to read OCxmon or tcpdump trace files and extract header information. We have packaged this software system, sample trace files, and suggested exercises, to produce a CD, http://traffic.caida.org/, that instructors can use for student projects in networking courses. Locally collected data could be used for analysis, or if collecting data on campus networks were impossible (politically) the trace files on the CD suffice.
The CD is being distributed to several hundred instructors around the world in the hopes that more students will become interested in traffic measurement and analysis and produce new research results in the area.
During the fall semester of 1998, we experimented with delivering a networking course to two sets of students -- one at the University of California at San Diego, and the other at the University of Colorado. Both schools are connected via the vBNS, a lightly loaded, high speed research network connecting U.S. universities and research sites.
There were initial problems with the logistics: Colorado classes are 2.5 hours per week for 16 weeks, UCSD classes are 4 hours per week for 10 weeks. For part of the Colorado semester, UCSD was not even in session. The start of the UCSD quarter was after the last day that Colorado students were allowed to drop the class. Fortunately, the UCSD quarter fit entirely inside the Colorado semester and with help from administrators on pushing deadlines like the drop date we were able to find a format that fit each campus' requirements.
The class sessions were multicast over the Internet, half the time from the UCSD campus to remote students in Colorado and the other half of the time from Colorado to the students in California. The technology included two-way audio, two-way video, and a shared white board using the UNIX tools vat, vic, and wb. Teaching materials (slides) were displayed using the shared white board. The video ended up being rarely used; relevant content was mostly in the audio and white board. We lost one class due to routing problems on the vBNS; we repeated its transmission the next day. The quality of the transmission was generally excellent.
One difficulty, at first, involved questions from the remote site. There was no easy way to see a remote student's hand raised indicating that he wanted to ask a question. We solved this problem by having a student at the remote site sit at the workstation receiving the broadcast and when there was a question the student would type several question marks on the shared white board. Local students, seeing the whiteboard (displayed continuously on a large screen) alert the instructor that there was a remote question. We also had initial difficulty with signal feedback in the audio, set to full duplex mode to allow remote students to interrupt the lecture with questions. An operator at the broadcast site could have solved this, but one was not available.
In general, the class was quite successful. There were about 80 students and several teaching assistants and graders at each site, and the instructor was in San Diego for the first 5 weeks and in Colorado for the final 5 weeks. Instuctor office hours were available for the remote students over the same audio/video/white-board link. The course evaluations, on a scale of A-F, were a B. The set of students that was remote asked about half as many questions as the local students, both in the first half of the quarter and in the second half. An easy way for a remote student to signal a question from his seat might mitigate this imbalance.
The biggest challenges facing the IEC project are the maintenance and
automation of the website (iec.caida.org).
Additionally, class webpages are often incomplete
until the end of the semester and then suddenly wiped clean to start
the next semesters' class. We need to have the ability
to archive any courses that will be re-built by the instructor
each time the course is offered. Courses are currently added
by hand after appropriate review and indexing.
Automating the changes necessary to the indexes as a new course is
added would speed up the augmentation of the site. Expanding our
reviewing staff, to include individuals with an eye to automating the addition of courses,
would also help. On the other hand, our goal has never been to have a
huge collection of courses, but rather to have a good representative
sample of high quality courses and pointers to other resources to help
university professors. We view the IEC repository
and the ITL initiative as a prototype where academia,
major Internet providers, and network equipment
manufacturers work together to regularly augment
and create educational materials and university teaching
labs, with the intention to nurture
the next generation of qualified Internet engineers.
Support for the IEC is provided by the National Science Foundation (NSF), under NCR-9706181, the Association for Computing Machinery (ACM), and members of the Cooperative Association for Internet Data Analysis (CAIDA). CAIDA is based at the University of California's San Diego Supercomputer Center.