Detection and Deflection of DoS Attacks Against the Multicast Source Discovery Protocol

Prashant Rajvaidya, Krishna Ramachandran and  Kevin C. Almeroth
Department of Computer Science
University of California
Santa Barbara, CA 93106-5110
{prash, krishna, almeroth}@cs.ucsb.edu

Studying and solving security problems in the 
multicast infrastructure has been largely overlooked.  Efforts have
been more focused on developing, then deploying, and then debugging
the basic protocols.  Although a number of security
vulnerabilities exist, the most troubling are a set of
Denial-of-Service (DoS) attacks.  The main
reason for this concern is that the one-to-many nature of multicast
can significantly magnify the effects of these attacks.  Among the
possible multicast DoS attacks, those that target the vulnerabilities
of the Multicast Source Discovery Protocol (MSDP) can be most
damaging.  MSDP vulnerabilities are unusually easy to exploit and can
lead to infrastructure-wide damage.  In this paper, we study the
vulnerability of MSDP to DoS attacks. In reality, attacks on MSDP have
already occurred. Using these attacks as a basis, we develop and describe
several new strains of even more virulent MSDP-based DoS attacks.  We also
propose a family of solutions to detect and ``deflect'' the effects of each
strain of attack.  Our techniques are evaluated by simulating their
effectiveness against several attack workloads.