Overview

Each Network Infrastructure Data Science (NIDS) educational assignment consists of two GitHub repositories under the CAIDA organization:

• Public repository — student-facing content: the introduction, dataset documentation, tool guides, task instructions, and the notebooks the student completes
• Private repository — instructor-facing content: the fully worked answer key, plus everything needed to prepare the data resources the class uses

Both repositories are managed through the Team Network Infrastructure Data Science GitHub team.

This page covers what an assignment contains and the four steps to create one. The NIDS Assignment Style Guide specifies exactly how each file is written — filenames, heading order, question numbering, notebook cell structure — and should be followed while building the repositories.

Note: older repositories, tooling, and links may still say module — it means the same thing. This page, the repositories, and the Assignments listing all say assignment.

What the Assignment Introduces

Every assignment exists to introduce something the student has not seen before. Before writing anything, name what that is. It falls into four categories, and most assignments introduce something in more than one:

Be specific. “Internet topology” is not an answer; “router-level links inferred from traceroute, and when their inferred geolocation can be trusted” is.

Background

Provide the context a student needs before the new material makes sense: slides, webpages, papers, or links to other NIDS assignments that introduce prerequisite concepts. Frame it around the material being introduced rather than as general reading.

What the Assignment Assumes

Anything not covered by a prerequisite assignment is new material, and this assignment has to teach it. That single rule decides most of what the repository contains.

List the prerequisite assignments explicitly, and say what each one supplies — not just that it exists. The practical consequence for tools: if a prerequisite already taught the tool, cite the prerequisite. If the tool is new here, the assignment ships a guide for it. If a prerequisite covers it only partly, ship a guide covering just the difference.

Resources: Known and New

Full list of all the resources that will be used.

• Resources Known
  List the datasets, tools, and libraries the student is expected to understand before beginning this assignment. This list should be organized around the prerequisite assignments which introduce this information.

• Resources New
  List the datasets, tools, and libraries that are introduced for the first time in this assignment. For each new resource:

  • Provide enough detail for students to understand what the resource is and why it is used.

  • Link, to an existing, or create a new, catalog.caida.org recipe for scripts and deeper documentation. (optional)

    Catalog Recipes are technical methodologies used to derive a particular conclusion or a particular dataset, which can be used for further synthesis and analysis. These will include example code and more details than present in the assignment.

How the Tasks Force Engagement

Naming the new material is not enough — the tasks have to make the student actually use it. The test: a student should not be able to finish a task by falling back on what they already knew.

Two published assignments show what this looks like:

• nids-iyp’s three tasks deliberately re-do analyses the student already completed in earlier assignments. Because the answers are already familiar, the only thing left to learn is expressing them as a single graph traversal instead of a hand-written join across separate datasets.
• nids-ucsdnt-expanse makes the student port an interactive prototype into a Slurm batch array — not as an exercise for its own sake, but because that is the only way to cover the time period the analysis needs.

Analysis

Sequence the tasks so each piece of new material is exercised on its own before it is combined with anything else:

1. For each new resource, begin with a self-contained introduction — a small task that uses only that resource, so a mistake is easy to localize.
2. Then combine the known and new resources in a task that cannot be completed with either alone.

What the Student Produces

Name the deliverables explicitly, and mark each one with ⬅ in the README’s directory tree so there is no ambiguity about what gets submitted.

Most assignments produce a single notebook named for the repository — nids-itdk.ipynb in nids-itdk. Some need more: nids-telescope-traffic ships two notebooks, and nids-ucsdnt-expanse ships two notebooks plus a batch script and a Slurm submission file. Ship more than one only when the work genuinely divides — a prototype and a full-period analysis are different deliverables; splitting one analysis across two files is not.

Example: nids-iyp

Drawn from nids-iyp, the assignment that most clearly introduces both a dataset and a tool.

Where Each Part Lives

The sections above are a way of thinking about the assignment, not a list of files. They map onto the repository like this:

The NIDS Assignment Style Guide specifies the format of each of these files, along with the breadcrumb navigation, banner, and notebook cell conventions they all share.

Prerequisites

• A GitHub account
• Membership in Team Network Infrastructure Data Science — contact a CAIDA team member to request access
• A dataset and at least one corresponding catalog.caida.org recipe. If the recipe doesn’t exist, you will have to make it.

Where the repositories live while you build

Confirm which of these applies to you before creating anything, since it determines where you work:

• Directly in the CAIDA organization — you are added to the NIDS GitHub team, or a CAIDA GitHub account is created for you, and you create both repositories under the CAIDA organization from the start.
• In your own account, then transferred — you create both repositories under your personal or organization account, build the assignment there, and transfer ownership to CAIDA once it is ready (repository Settings → General → Danger Zone → Transfer ownership). Team access is configured after the transfer completes.

Either way the assignment ends up as a pair of CAIDA-owned repositories; only the starting point differs.

Step 1: Create the Public Repository

1. Create the student-facing repository. Use a descriptive name reflecting the dataset or topic (e.g., nids-bgp-hijacks).
2. Set visibility to Public.
3. Add the following files, written to the style guide:
   • README.md — banner, title, authors, learning objectives, the ordered overview of steps, directory structure, and glossary
   • Introduction.md — reading list, prerequisite assignments, and the conceptual background
   • Datasets.md — what each dataset is, how to access it, and its schema
   • A tool guide for each tool a prerequisite assignment doesn’t already cover, named for the tool (SQL.md, Cypher.md, Spark.md, Dpkt.md)
   • Tasks.md — the task checklist, with every question numbered sequentially
   • Task-<N>-<slug>.md — one guidance page per task, giving technique without giving the answer
   • The deliverables — normally a single nids-<topic>.ipynb, or several notebooks and scripts if the work genuinely divides; mark each with ⬅ in the directory tree
   • data/ — only if the dataset must be downloaded rather than read live; document how to populate it and never commit large data files
   • <x>_credentials.env.example — if the dataset needs credentials; the filled-in .env stays git-ignored
4. Target NRP’s hosted JupyterHub, so students need no local setup. Add local install instructions and dependency files only if the assignment ships its own persistent database, such as the Postgres behind nids-itdk or the Neo4j behind nids-iyp.

Step 2: Create the Private Repository

1. Create a second repository named with a -key suffix (e.g., nids-bgp-hijacks-key).
2. Set visibility to Private.
3. Add the answer key: a copy of every notebook in the public repository with all code written and all questions answered, run top to bottom with no errors and its outputs saved. A key notebook that does not run clean is not finished.
4. Add everything needed to prepare the assignment’s data resources — the loader scripts and notebooks, container and Kubernetes manifests, schema exporters, credential templates, and cleanup scripts an instructor must run before students can start — plus a README.md that walks an instructor through them in order. nids-itdk-key is the model: it goes from an empty NRP namespace through deploying Postgres, loading the data, creating the read-only student role, and handing credentials to students. If the assignment reads its data live and needs no preparation, say so in the README.md.
5. Optionally add grading rubrics, facilitation notes, or an environment-check notebook.

Step 3: Configure Team Access

1. If you built the repositories outside the CAIDA organization, transfer both to CAIDA first (Settings → General → Danger Zone → Transfer ownership).
2. Go to the Team Network Infrastructure Data Science repositories page.
3. Add both repositories (public and private) to the team.
4. Confirm the public repo has at least Read access for all team members; the private repo should be Write for instructors.

Step 4: Publish the Assignment

1. If the assignment uses a dataset not already represented in catalog.caida.org, find or create a catalog.caida.org recipe for it, and link the recipe from Datasets.md. Skip this if the dataset is already covered.

2. List the assignment on the NIDS Assignments page by adding an entry to content/projects/nids/assignments/assignments.json in the www-caida-web repository. This is what makes the assignment appear on the website:

   {
     "code": "DNS",
     "name": "DNS Ecosystem",
     "url": "https://github.com/CAIDA/nids-dns-ecosystem/blob/main/Introduction.md",
     "description": "Use OpenINTEL active DNS measurements and the LACeS Anycast Census to study DNS hosting concentration, name server redundancy and anycast deployment, and adoption of DNS security mechanisms across the Internet.",
     "dependencies": [],
     "tags": ["DNS", "DNSSEC", "Anycast", "OpenINTEL"]
   }
   

   code is a short uppercase identifier, url points at the public repository’s Introduction.md, and dependencies lists the codes of prerequisite assignments. Website changes go on their own branch and through a merge request — see the repository’s contributor guide.

3. Notify the NIDS team that the assignment is ready.