A real time packet filtering module

for network intrusion detection system

by
Guang Yang
A thesis submitted to the graduate faculty
in partial fulfillment of the requirements for the degree of
MASTER OF S!E"E
Ma#or$ omputer Science
Ma#or %rofessor$ R& & Se'ar
!o(a State )ni*ersity
Ames+ !o(a
,--.
Graduate ollege
!o(a State )ni*ersity
This is to certify that the Master/s thesis of
Guang Yang
has met the thesis requirements of !o(a State )ni*ersity
00000000000000000000000000000000
Ma#or %rofessor
00000000000000000000000000000000
For the Ma#or %rogram
00000000000000000000000000000000
For the Graduate ollege
ii
TABLE OF CONTENTS
ABSTRACT!
C"A#TER $ %NTRO&'CT%ON$
,&, "et(or' Security and %otential Threats&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,
,&1 !ntrusion 2etection&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1
,&3 4ey ontributions&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&3
,&5 Thesis Organi6ation&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&3
C"A#TER ( O)ER)%E* OF TC#+%# BASE& NET*OR, %NTR'S%ON-
1&, T%7!% 8asics&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9
1&,&, %rotocol :ierarchy&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9
1&,&1 !%&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9
1&,&3 )2%&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&;
1&,&5 T%&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&;
1&1 ommon <ulnerabilities&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&-
1&1&, !% Source Address Spoofing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&-
1&1&1 T% Sequence "umber %rediction&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&-
1&1&3 %ort Scanning&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,=
1&3 "et(or' !ntrusions&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,,
1&3&, 2enial of Ser*ice&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,,
1&3&,&, :ARGE" and E:O&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,1
1&3&,&1 SY" Flooding&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,1
1&3&,&3 Other 2enial of Ser*ice !ntrusions&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,9
1&3&1 Spoofing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,>
1&3&1&, lient?Side Spoofing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,>
1&3&1&1 Ser*er?Side Spoofing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,.
1&3&3 Ser*ice Specific !ntrusions&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,-
1&3&3&, Finger 2aemon Attac'&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&,-
1&3&3&1 Routing !nfrastructure !ntrusions&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1=
1&3&3&3 2"S Misuse&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1=
1&3&3&5 "FS&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&11
1&3&3&9 @?Aindo(s&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&13
C"A#TER . %NTR'S%ON &ETECT%ON AN& #AC,ET F%LTER%N/(0
3&, urrent Techniques in "et(or' Security&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&15
3&,&, Audit Trails&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&15
3&,&1 Fire(all&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&19
3&,&1&, Screening Router&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&19
3&,&1&1 Application Gate(ay&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1>
3&1 %ac'et Filtering&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1;
iii
3&1&, General !ssues&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1.
3&1&1 EBisting %ac'et?Filtering Systems&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1.
3&1&1&, CinuB SO40%A4ET&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1.
3&1&1&1 2ata Cin' %ro*ider !nterface&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1-
3&1&1&3 8S2 %ac'et Filter&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&1-
3&1&3 %ac'et apture Cibrary&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&3,
3&3 8ro$ An !ntrusion 2etection System 8ased on %ac'et Filtering&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&31
3&3&, 8ro Architecture&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&31
3&3&1 8ro Canguage&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&33
3&5 %ac'et Filtering for "et(or' !ntrusion 2etection&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&39
C"A#TER 0 %NTR'S%ON #ATTERN S#EC%F%CAT%ON LAN/'A/E.1
5&, ASC SyntaB&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&3;
5&1 %ac'et Structure 2escription&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&3;
5&3 onstraint hec'ing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&3-
5&5 Sample %atterns&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&5,
C"A#TER - S2STE3 &ES%/N AN& %3#LE3ENTAT%ON0.
9&, System Architecture&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&53
9&1 %ac'et Offset alculation&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&55
9&3 Filter Model for Single Rule&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&59
9&5 Filter !ntegration&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&5;
9&9 Rule %reprocessing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9,
9&9&, Rule 2ecomposition&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9,
9&9&1 onstraint Stac' onstruction&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&91
9&> Automaton onstruction&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&95
9&>&, Offset Selection&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&95
9&>&1 Sub?Automaton Sharing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9>
9&; ode Generation&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9;
9&. 2ata Generation&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&9-
C"A#TER 4 E5#ER%3ENTAL RES'LTS AN& CONCL'S%ON4(
>&, !ntrusion 2etection )sing ASC&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&>1
>&1 %reliminary %erformance Testing&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&>3
>&1&, SR<STAT$ Ser*ice Statistics&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&>3
>&1&1 %erformance omparison$ ASC *s& 8%F&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&>5
>&3 onclusion&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&>9
A##EN&%5 A #AC,ET &ATA STR'CT'RES FOR ASL41
A##EN&%5 B %NTR'S%ON #ATTERN SA3#LES1(
REFERENCES1-
i*
AC,NO*LE&/E3ENTS11
ABSTRACT
omputer net(or's bring us not only the benefits+ such as more computing po(er
and better performance for a gi*en price+ but also some challenges and ris's+ especially in the
field of system security& 2uring the past t(o decades+ significant effort has been put into
net(or' security research and se*eral techniques ha*e been de*eloped for building secure
net(or's& %ac'et filtering plays an important role in many security?related techniques+ such
as intrusion detection+ access control and fire(all& A pac'et?filtering system constitutes the
first line of defense in a computer net(or' en*ironment& The 'ey issues in the pac'et?
filtering technique are efficiency and fleBibility& The efficiency refers to the ability of a filter
to quic'ly capture net(or' pac'ets of interest+ (hile the fleBibility means the filter can be
customi6ed easily for different pac'et patterns&
!n this thesis+ (e present a real?time pac'et?filtering module+ (hich can be integrated
into a large?scale net(or' intrusion detection system& The core of this pac'et?filtering
module is a rule?based specification language ASC DAuditing Specification CanguageE+ (hich
is used in describing the pac'et patterns and reactions for a net(or' intrusion detection
system& The important features of ASC that are not pro*ided by other pac'et?filtering
systems are protocol independence and type safety& ASC pro*ides a number of ne( features
that distinguish it from other languages used for intrusion detection and pac'et filtering+ such
as pac'et structure description and protocol constraint chec'ing&
Ae de*elop the algorithms and heuristics for constructing fast pac'et filter from ASC
specifications& Our algorithms impro*e upon eBisting techniques in that the performance of
the generated filters is insensiti*e to the number of rules& Ae discuss implementation of
these algorithms and present eBperimental results&
*
C"A#TER $ %NTRO&'CT%ON
omputation models ha*e eBperienced a significant change since the emergence of
computer net(or's+ (hich allo( heterogeneous computers to communicate (ith each other&
2uring the past t(o decades+ most centrali6ed systems ha*e been replaced by a number of
interconnected computers& This factor has led to more computing po(er+ increased fleBibility
and better performance7price ratio&
:o(e*er+ at the same time+ (e also face many ne( challenges and ris's (ith
net(or'ed computing+ such as lac' of communication reliability+ coordination+ resource
management+ and so on& As more and more computer net(or's are brought into electronic
commence+ transaction management+ and e*en national defense+ people begin to pay
increasing attention to system security&
$$ Network Security and #otential T6reats
There are a number of security issues for a computer net(or' en*ironment F,G$
Availability: The system must be functional and correctly pro*ide ser*ices&
Confidentiality: The data transmitted from one system to the other must be
accessible only for the authori6ed parties&
Authentication: The identity associated (ith the data must be correct& The
identity can apply to a user+ host or soft(are component&
Integrity: The data being processed or transmitted can be modified only by the
authori6ed parties&
Non-repudiation: "either the sender nor the recei*er of data is able to deny the
fact of data transmission&
,
A system that meets the abo*e criteria can be considered as a secure computer
net(or' system& A hac'er (ho (ants to attac' a net(or'+ thus thin's of (ays to compromise
the abo*e criteria F,G$
Interruption: 2estroy a system or ma'e it una*ailable or unusable&
Interception: Obtain unauthori6ed access to data&
Modification: ompromise data integrity+ e&g& modify messages sent from one
system to another&
$( %ntrusion &etection
As defined by :eady et al& F1G+ an intrusion is
any set of actions that attempt to comprise the integrity, confidentiality or availability
of a resource.
!ntrusion leads to *iolations of the security policies of a computer system+ such as
unauthori6ed access to pri*ate information+ malicious brea'?in into a computer system+ or
rendering a system unreliable or unusable&
A full?blo(n net(or' security system should include the follo(ing subsystems$
Intrusion Detection ubsystem: 2istinguishes a potential intrusion from a *alid
net(or' operation&
!rotection ubsystem: %rotects the net(or' and security system itself from being
compromised by the net(or' intrusions&
"eaction ubsystem: This part either traces do(n the origin of an intrusion or
fights bac' the hac'ers&
The focus of this thesis is on the intrusion detection subsystem+ (hich constitutes the
first line of defense for a computer net(or' system& There are a number of approaches in
this field& Most of them fall into three primary categories$ anomaly detection+ misuse
detection and hybrid schemes&
The anomaly detection approach is based on a model of normal acti*ities in the
system& This model can either be predefined or established through techniques such as
machine learning& Once there is a significant de*iation from this model+ an anomaly (ill be
1
reported& 8y contrast+ a misuse detection approach defines specific user actions that
constitute a misuse and uses rules for encoding and detecting 'no(n intrusions F3G& The
hybrid detection approach uses a combination of anomaly and misuse detection techniques&
$. ,ey Contri7utions
%ac'et filtering is a critical technique in net(or' management+ fire(all strategy and
intrusion detection& :o(e*er+ the eBisting pac'et filtering systems ha*e a number of
limitations in system efficiency+ fleBibility and scalability& For instance+ a pac'et filter for
one protocol suite can not be easily changed to fit for another protocol suite& !n addition+
most pac'et filters suffer from significant performance degradation as the number of pac'et
patterns increases&
!n this thesis+ (e present a no*el approach for constructing a real?time pac'et?
filtering module that can be used for net(or' intrusion detection purpose& One of the main
contributions in our approach is a specification language designed for describing intrusion
patterns and reactions& This language pro*ides a number of features that distinguish it from
other specification languages used for intrusion detection or pac'et filtering+ such as protocol
independence and type safety& Another important focus of our (or' is the de*elopment of
fast pattern?matching algorithms Dfor pac'et filterE that are insensiti*e to the number of
patterns&
$0 T6esis Organi8ation
!n chapter 1+ (e gi*e a brief re*ie( of T%7!% DTransmission ontrol
%rotocol7!nternet %rotocolE protocol suite and se*eral security holes in the design and
implementation of T%7!%& hapter 3 sur*eys some eBisting techniques in building a secure
computer net(or' system& Ae also discuss some general issues on pac'et filtering that is one
of the main techniques in net(or' intrusion detection& !n chapter 5+ (e gi*e a detailed
description of our specification language and its application to intrusion detection& hapter 9
discusses the issues in the design and implementation of our pac'et?filtering module& The
primary concern is to reduce the processing time of a pac'et filter& !n the last chapter+ (e
3
pro*ide some eBperimental results from performance testing of our pac'et filter and
summari6e our (or'&
5
C"A#TER ( O)ER)%E* OF TC#+%# BASE& NET*OR, %NTR'S%ON
T%7!% is the common language used in the (orld of computer net(or's&
"e*ertheless+ there eBist se*eral security fla(s in the protocol design or implementation of
T%7!%& As a result+ net(or' hac'ers+ (ho intend to compromise the target net(or' systems
by eBploiting these security holes+ ha*e in*ented *arious intrusion methods&
($ TC#+%# Basics
2e*eloped under the sponsorship from 2AR%A D2efense Ad*anced Research
%ro#ects AgencyE+ T%7!% is the most (idely used communication protocol suite today& !t is
the de facto standard employed to interconnect computing facilities in modern net(or'
en*ironments&
2.1.1 Protocol Hierarchy
T%7!% is designed through a layered approach+ (ith each layer responsible for a
different facet of communication F5G& This hierarchical architecture ma'es each protocol
layer possible to e*ol*e independently (ithout affecting the ad#acent layers& !n addition+ data
encapsulation is achie*ed through *arious headers among different transportation layers li'e
!% header+ T% header or other application headers as sho(n in Figure 1&,& These headers
are important in 'eeping the state information for each net(or' connection and facilitating
multipleBing and de?multipleBing of transmission messages&
2.1.2 IP
!% is the (or'horse protocol of the T%7!% protocol suite& !t pro*ides an unreliable+
connectionless datagram deli*ery ser*ice& All the T%+ )2% D)ser 2atagram %rotocolE+
9
Figure ($ TC#+%# #rotocol "ierarc6y
!M% D!nternet ontrol Message %rotocolE+ and !GM% D!nternet Group Management
%rotocolE data are transmitted as !% datagrams F5G&
An !% header has the information li'e source !% address and destination !% address+
(hich plays an important role in routing a pac'et around the net(or's& A detailed
description of !% header can be found in F5G& Figure 1&1 sho(s the structure of a normal !%
header&
Figure (( %# "eader
user data
Appl 6eader user data
%# 6eader TC# 6eader application data
TC# 6eader application data
Et6er 6eader %# 6eader TC# 6eader application data Et6er trailer
!ersion lengt6 type of ser!ice total lengt6
identification flags fragment offset
time to li!e protocol 6eader c6ecksum
source %# address
destination %# address
options 9if any:
data
>
2eli*ering a pac'et to the correct destination is non?tri*ial+ especially in a large?scale
net(or' system& Each intermediate routing de*ice ma'es best effort to deli*er the !% pac'et+
but there is no guarantee that it (ill reach the destination finally& So+ a pac'et can be lost+
duplicated+ or deli*ered out of order F5G& !t is the tas' of higher layer protocols to correct
such errors&
2.1.3 UDP
)2% is a transport layer protocol+ but it does not offer much functionality o*er and
abo*e that of !%& The port numbers in )2% header identify the sending process and the
recei*ing process F5G+ (hile the chec'sum pro*ides a limited ability for error detection
DFigure 1&3E&
Figure (. '&# "eader
:o(e*er+ due to its simplicity and lo( o*erhead compared to connection?oriented
protocols+ )2% is suitable for the design of simple request7reply application protocol+ such as
2"S D2omain "ame SystemE+ S"M% DSimple "et(or' Management %rotocolE+ and so on&
2.1.4 TCP
T% is built on top of the !% layer+ (hich is unreliable and connectionless& 8ut T%
pro*ides the higher layer application a reliable connection?oriented ser*ice& As the tradeoff+
each T% connection requires an establishment procedure and a termination step bet(een
communication peers& T% also pro*ides sequencing and flo( control&
Aithout any option+ a T% header occupies 1= bytes as sho(n in Figure 1&5& The
sequence number is essential in 'eeping the sending and recei*ing datagram in proper order&
source port num7er destination port num7er
'&# lengt6 '&# c6ecksum
data
;
Figure (0 TC# "eader
There are siB flag bits (ithin a T% header+ namely )RG+ A4+ %S:+ RST+ SY" and F!"+
each of (hich has a special meaning in connection establishment+ connection termination or
other control phases& Aindo( si6e+ (hich specifies ho( many bytes of data can be accepted
each time by the recei*ing side+ is ad*ertised bet(een the t(o communication peers for the
purpose of flo( control&
T% establishes a connection in three steps+ commonly 'no(n as a three?(ay
handsha'e& Figure 1&9 sho(s a typical three?(ay handsha'e procedure bet(een a source
host S and a destination host 2&
S2N%SNs
S2N%SNd; AC,%SNs<$
AC,%SNd<$
Figure (- T6ree=*ay "ands6ake
source port num7er destination port num7er
se>uence num7er
acknowledge num7er
6eader reser!ed urg;ack;ps6;rst;syn;fin window si8e
TC# c6ecksum urgent pointer
options 9if any:
data 9if any:
.
First+ S sends a SY" pac'et to 2 in order to establish a connection& Mean(hile+ S
sets its o(n !S" D!nitial Sequence "umberE in sequence number field of the pac'et& )pon
recei*ing the request pac'et+ 2 sends bac' a SY"0A4 pac'et as the ac'no(ledgement
including its o(n !S" and the incremented !S" from S& As the ac'no(ledgement pac'et
reaches the source host S+ S immediately transmits an A4 pac'et bac' to 2& !n the last
A4 pac'et+ S needs to include the incremented !S" of 2 as the confirmation of the
connection& At this point+ the connection has been setup& There is one eBtra point that needs
to be mentioned$ suppose that host S does not send any SY" pac'et but recei*ed a
SY"0A4 pac'et from host 2+ it (ill then send bac' a RST pac'et to reset the connection&
(( Common )ulnera7ilities
2uring the past t(o decades+ many security problems of T%7!% protocol suite ha*e
been disco*ered& Mean(hile+ the net(or' hac'ers created a large number of intrusion
methods to eBploit those *ulnerabilities& Most of the eBamples in this section are ta'en from
8ello*in/s eBcellent paper on T%7!% security F9G&
2.2.1 IP Source Address Spoofin
As (e ha*e seen from the pre*ious section+ the !% address Deither source address or
destination addressE contained in an !% header is the only information needed by an
intermediate routing de*ice to ma'e a decision on ho( to route the !% pac'et& So+ anyone
(ho has access to the !% layer can easily modify the source address in the !% header of a
pac'et+ spoofing itself as from another host or e*en from a non?eBisting host&
2.2.2 TCP Se!uence "u#$er Prediction
From the three?(ay handsha'e+ (e 'no( that to establish a T% connection bet(een
t(o communication peers+ the source host must obtain the !S" of the destination host from
its ac'no(ledgement pac'et& )sually+ an !S" is more or less a random number F9G&
!f a hac'er can predict the !S"+ he7she can impersonate host S by sending a request
pac'et (ith the !% source address changed to S& Although the hac'er (ill not get the
-
SY"0A4 pac'et sent by 2+ he7she can still finish the establishment process by sending
bac' an A4 pac'et to host 2 (ith predicted !S"& As sho(n in Figure 1&>+ !S"g represents
the guessed !S" of host 2 by the hac'er&
!n 8er'eley systems+ the initial sequence number is incremented by a constant
number D,1. in 5&18S2 and ,19+=== in 5&38S2E once per second and by half that number
each time a connection is initiated F>G& Thus+ (hat a hac'er needs to do is #ust to initiate a
normal connection and remember the !S" recei*ed from the destination host& After that+ the
hac'er could calculate the !S" for the neBt connection attempt+ based on the round?trip delay
and the number of connections after the first connection& This approach has a high
probability of succeeding&
Figure (4 TC# Se>uence Num7er #rediction
2.2.3 Port Scannin
Strictly spea'ing+ port scanning is not a technique used directly to perform an
intrusion& !nstead+ its goal is to disco*er an eBploitable communication channel and then
launch the real attac'& The reason for doing port scanning is that some *ulnerable ser*ices
may not use a fiBed port number& As in the S)" "FS system+ some application ser*ers run
at an arbitrary port and register the port number to a specific ser*er+ (hich is called directory
ser*er& For the client programs of a particular application ser*er+ they need to first chec'
(ith the directory ser*er to obtain the port number for that application ser*er& )sually+ the
directory ser*er is (ell protected& So+ a hac'er needs another (ay to locate his *ictim&
There are se*eral methods that can be used to detect a potential communication
channel& For a listening T% ser*er+ the most elementary approach is to ma'e a real
connection& The )"!@ system?call connect can be used to open a connection (ith e*ery
port that the hac'er intends to eBamine& !f there is a listening ser*er+ the connect call (ill
% D& S'"(IS")*+ S,C - S
D S& S'"(IS"d*+ AC.(IS")*
% D& AC.(IS"*+ S,C - S
,=
succeed& Other(ise the port is unused& Another method is through SY" scanning+ in (hich
a SY" pac'et is sent to the *ictim as if it is going to create a real connection& As mentioned
in T% three?(ay handsha'e+ a returned SY"0A4 or RST pac'et indicates the presence or
absence of an acti*e ser*er on the port& Another *ariant of this approach is T% F!"
scanning& !nstead of sending SY" probes li'e in SY" scanning+ this method sends F!"
pac'et and (aits for a RST pac'et from a closed port& !n case of an acti*e listener+ it (ill
discard the F!" pac'et silently (ithout sending anything bac'&
)nli'e T%+ )2% is a connectionless protocol+ (hose simplicity ma'es port scanning
more difficult& Since )2% does not require a three?(ay handsha'e to establish a connection+
a )2% ser*er does not need to ac'no(ledge any probe pac'ets& Also+ no error messages are
returned for closed ports& :o(e*er+ most hosts send !M% Hport unreachableI message for a
pac'et intended for an unused )2% port& This gi*es hac'ers some clue& Since neither )2%
pac'ets nor the !M% messages are guaranteed to be deli*ered due to the unreliable nature of
the protocol itself+ a port scanner needs to ha*e some retransmission policy to ensure that lost
pac'ets do not lead to erroneous results&
(. Network %ntrusions
A number of net(or' intrusions ha*e been found till no(+ each of (hich utili6es one
or more security *ulnerabilities in T%7!% protocol specifications or implementations& These
intrusions include !% source address spoofing+ T% sequence number prediction as mentioned
earlier+ and other intrusions li'e SY" flooding+ 2"S misuse+ %ing of 2eath+ or some Ja*a?
related attac's& :o(e*er+ based on the intrusion patterns and impacts to the *ictim systems+
(e can di*ide the intrusions into t(o main categories$ denial of ser*ice and spoofing&
2.3.1 Denial of Ser/ice
The lifeblood of today/s (orld is information F.G& The denial?of?ser*ice intrusions
attempt to pre*ent or delay access to the information or the information processing systems&
The basic idea behind this type of intrusion is to tie up a ser*ice pro*ider (ith bogus requests
in order to render it unreliable or unusable&
,,
1&3&,&, :ARGE" and E:O
:ARGE" is a simple ser*ice pro*ided by almost all T%7!% implementation under
)"!@& !t runs on both )2% and T% port ,-& For e*ery incoming )2% pac'et+ the ser*er
sends bac' a pac'et (ith = to 9,1 randomly selected characters& Another (ell?'no(n ser*ice
is E:O+ (hich runs on )2% and T% port ;& The ser*er #ust responds to the client program
(ith (hate*er it recei*es&
These t(o ser*ices are normally used for the diagnostic purpose& :o(e*er+ they can
be employed by a malicious denial?of?ser*ice type intrusion& Assuming a HchainI has been
established bet(een a :ARGE" ser*ice and an E:O ser*ice+ (hat (ill happen neBtK
Each of them (ill produce output continuously+ leading to a huge number of pac'ets among
the net(or' and thus a denial of ser*ice on the machines (here the ser*ices are pro*ided&
Caunching such an intrusion is surprisingly easy& A simple )2% pac'et could set the
(hole net(or' into trouble& Suppose there are t(o hosts A and 8 and a hac'er on machine
@& Aith the help of !% source address spoofing+ a hac'er can send out a )2% pac'et to A
(ith 8/s !% address as the source address and ; as the source port+ (hile setting the
destination !% address as A/s !% address and ,- as the destination port& Ahen this pac'et is
recei*ed by A+ A (ill falsely thin' that 8 is requiring the :ARGE" ser*ice+ and sends bac'
a pac'et to 8/s E:O port& At this point+ a HchainI has been established successfully&
Subsequently+ large amount of traffic (ill be generated (ithin the net(or' (here hosts A and
8 reside& As a result+ net(or' users (ill feel an abrupt drop in the performance of their
net(or' applications&
Generally spea'ing+ :ARGE" and E:O type of intrusion is a 'ind of blind
attac'& There is no particular ob#ecti*e from a hac'er/s point of *ie(& The goal is to slo(
do(n the speed of the (hole net(or'&
1&3&,&1 SY" Flooding
)nli'e the simple :ARGE" and E:O intrusion+ SY" flooding is a specially
designed attac' that employs a flood of SY" pac'ets to consume the limited resource on the
,1
targeted host& !t results in delays to legitimate net(or' connection requests and e*entually
halts the ser*ice pro*ider&
As in the T%7!% implementations for )"!@+ a number of memory structures need to
be allocated for each T% connection request& Ta'e 8S2 system as an eBample$ a soc'et
structure is used to hold the communication elements De&g& protocol being usedE+ address
information+ request queues+ buffers and flags F5G& Moreo*er+ there are t(o eBtra memory
structures (ith special meanings to a T% connection+ namely !% control bloc' DinpcbE and
T% control bloc' DtcpcbE+ (hich 'eep the T% state information+ port numbers+ sequence
numbers and se*eral connection?related timers& Typically+ these structures (ill use a fe(
hundred bytes of memory F;G&
A normal scenario of a T% connection process starts (ith a system in C!STE" state
recei*ing a SY" pac'et+ (hich is to be eBamined for chec'sum immediately& !f the
chec'sum is incorrect+ the pac'et (ill be discarded silently+ (ith the eBpectation that the
remote site (ill retransmit a ne( pac'et& Other(ise+ the T% control bloc' associated (ith
this connection is searched for& !f no such item is found+ it means no ser*er process is
(aiting for this pac'et+ and then the pac'et (ill be remo*ed and an RST pac'et is returned to
inform the remote client& 8y contrast+ if a ser*er process is located+ se*eral memory
structures (ill then be allocated for this connection and a SY"0A4 pac'et (ill be sent
bac' as an ac'no(ledgement to the sender to continue the three?(ay handsha'e& Mean(hile+
the system enters into the SY"0RE<2 state and starts up a connection establishment timer&
The connection of this stage is al(ays called a half?open connection& Most T%7!%
implementations set the timer to eBpire after ;9 seconds& !f the final A4 pac'et arri*es
before the timer eBpires+ the request (ill lea*e 'ernel space and go to application space&
Other(ise+ the three?(ay handsha'e fails& )nder both cases+ the corresponding memory
structures (ill be released from 'ernel space&
From the description abo*e+ (e 'no( that the process of T% connection
establishment requires significant amount of (or' and resources at the ser*er side& So+ in
most systems+ there is a limit on the total number of half?open connections& A hac'er
eBploits this limitation and initiates a SY" flooding attac' by issuing a large number of
,3
connection requests (ith a spoofed source !% address to the target host+ (hich cannot tell a
malicious request from a legal request& After recei*ing the SY" pac'et+ the target host (ill
respond (ith SY"0A4 pac'et as usual& )nfortunately+ this time the final A4 pac'et (ill
ne*er come bac'+ for the request SY" pac'et has a spoofed source address and that address
is HunreachableI to the target host DFigure 1&;E& There are se*eral reasons for an !% address to
be HunreachableI& For instance+ the machine (ith that !% address is turned do(n+ or there
may be e*en no host (ith that !% address at all& Actually+ there may be some error messages
li'e !M% Hhost unreachableI or Hnet(or' unreachableI generated by a router+ coming bac'
during the time (hen the target host (aits for the final A4 pac'et& 8ut current
implementations of T%7!% typically ignore such error messages& 8efore the timer used for
T% connection establishment eBpires+ the memory allocated for a connection request (ill
stay in the 'ernel& As large numbers of bogus connection requests come to the target host+ it
(ill run out of 'ernel memory quic'ly& As a result+ if there is no more memory structures can
be allocated for the follo(ing connection requests+ they (ill be discarded silently&
S &
Spoofed S2N L%STEN
S2N?AC, S2N?REC)&
Figure (1 S2N Flooding
The 'ey issue in this type of intrusion is ho( to choose an HunreachableI source !%
address for an attac'ing pac'et& There are se*eral patterns follo(ed by the hac'ers&
ingle address$ all the attac'ing pac'et using same !% address
hort list: there is a small pool of addresses for e*ery outgoing pac'et to choose
No list: the source address is generated randomly
,5
2ifferent addressing method poses different challenges for an intrusion detection system&
The basis for this type of attac' is that T%7!% protocol suite does not pro*ide strong
authentication on its control pac'et F-G& The endpoint of a connection has no (ay to
authenticate its communication peer& As a result+ it is eBtremely difficult to trace the original
source of the spoofed !% pac'et& Therefore+ a hac'er can feel free to perform this 'ind of
intrusion (ithout (orrying about being trac'ed do(n&
1&3&,&3 Other 2enial of Ser*ice !ntrusions
Other forms of denial?of?ser*ice type intrusions also eBist+ li'e %ing of 2eath& %ing
of 2eath eBplores a bug in some T%7!% implementations that cannot handle the fragmented
!% pac'et correctly& !n this case+ a hac'er first brea's a normal pac'et into a series of
fragments+ then modifies the last one and ma'es the total length of all fragments eBceed the
maBimum pac'et length specified in T%7!% protocol& Ahen the recei*ing host assembles
those fragments+ it (ill o*erflo( its buffer in the T%7!% stac' due to the abnormal si6e of
the arri*ed pac'et& As a result+ system on that host (ill crash&
There are some intrusions+ (hich utili6e the broadcast property of transmission
media+ are limited to a CA" DCocal Area "et(or'E en*ironment+ especially Ethernet&
:o(e*er+ (e cannot o*erloo' those intrusions& !t is possible that some hosts are less secure
than other hosts on a CA"& A hac'er can perform a multi?step intrusion by first brea'ing into
a less secure host and then compromise the (hole net(or'& One of the threats to a CA" is
called SY"0RST generator+ (hich can bloc' most of the T% connections& Suppose that a
host A (ants to ma'e a T% connection (ith host 8+ it (ill first send a SY" pac'et to 8& !f
host @ also hears this message+ because of the broadcast communication media+ before 8
responds (ith a SY"0A4 pac'et+ @ can quic'ly send out a RST pac'et to A+ shutting do(n
the intended connection& Another intrusion eBample is one in (hich the flo( control
mechanism of T% communication is attac'ed& !n order to pre*ent a fast sender from
o*errunning the buffer of a slo( recei*er+ each T% pac'et has a (indo( si6e for its
communication peer& 2uring the communication process+ a third party host can impersonate
,9
the destination host+ sending a pac'et (ith 6ero (indo( si6e& Then+ both communication
parties can be halted due to the lac' of buffer ad*ertised by the communication peer&
Aith the increasing use of Ja*a in the (eb computing+ intrusions by malicious Ja*a
applets are another source of concern& Most of the applet intrusions fall into denial of
ser*ice+ in (hich a Ja*a applet consumes a lot of %) and memory resources of the client
machine&
2.3.2 Spoofin
Spoofing is another important hac'ing technique in the net(or' intrusions& 2ue to
the distributed nature of computer net(or's+ the primary method used to eBchange data
among different hosts is message passing& Therefore+ strong authentication is not easy to
achie*e compared to that in a traditional centrali6ed system+ especially among arbitrary
communication peers& A net(or' hac'er eBploits this (ea'ness and creates many intrusion
methods+ either spoofing himself as a legitimate client or ser*er&
1&3&1&, lient?Side Spoofing
!n client?side spoofing+ a hac'er impersonates himself as an authori6ed client and in
turn gains ser*ices from a ser*er& An eBample is pro*ided by the Hr?utilitiesI on most )"!@
systems&
HR?utilitiesI+ li'e rlogin+ rsh and rcp+ is a set of commands for remote operations
among different )"!@ systems& The security hole underlying Hr?utilitiesI is the
authentication scheme used by this set of commands&
Ta'e HrloginI as an eBample+ (hich uses T% as its transportation layer protocol and
is a simple client7ser*er application& Aith t(o hosts A and 8+ each of (hich HtrustsI the
other one+ (e can configure the file H7etc7hosts&equi*I or H&rhostsI on each host to let a user
(ith accounts on both hosts to login from one host to another (ithout being prompted for a
pass(ord& !n effect+ the user is authenticated *ia the host name of the machine he7she is
currently logged on&
!n ,--9+ ERTDTME oordination enter issued a security ad*isory addressed a 'ind
of intrusion called H!% SpoofingI+ in (hich the hac'ers created pac'ets (ith spoofed source
,>
!% address+ then eBploited applications that use authentication based on !% address+ li'e Hr?
utilitiesI F;G& !% spoofing consists of se*eral steps and uses both address spoofing and T%
sequence number prediction& Follo(ing are t(o scenarios that can happen+ one is a normal
HrlgoinI session+ (hile the other is a spoofing intrusion DFigure 1&.E&
)sually+ !% Spoofing ta'es the follo(ing steps$
Figure (@ %# Spoofing
First+ a *ictim host is selected and a pattern of trust is disco*ered+ e&g& (hich hosts
the *ictim host trusts& !n the eBample sho(n in Figure 1&.+ the *ictim host is S+
(hile it trusts host &
Then+ is Hshut do(nI+ either by SY" flooding that machine or by intercepting
the entire net(or' traffic to it& Alternati*ely+ the attac' may be initiated (hen is
do(n due to other reasons+ such as maintenance&
"eBt+ a normal T% connection request pac'et is sent to the *ictim host S to get
bac' a *alid sequence number& 8ased on the round?trip delay and the T%
"ormal Remote Cogin Session$
C S& S'"(IS"c*
S C& S'"(IS"s*+ AC.(IS"c*
C S& AC.(IS"s*
C S& data
S C& data
Spoofed !ntrusion$
% S& S'"(IS")*+ S,C - C
S C& S'"(IS"s*+ AC.(IS")*
% S& AC.(IS"s*+ S,C - C
% S& AC.(IS"s*+ S,C - C+ #alicious #essaes
,;
sequence number generating algorithm+ a hac'er could predict the neBt sequence
number that (ill be used by S&
At this point+ S can be intruded upon& The hac'er sends a SY" pac'et to S (ith
the trust host as source !% address& E*en though the SY"0A4 pac'et (ill not
return to the hac'er+ he7she can still finish the connection establishment by
sending out the final A4 pac'et (ith the guessed sequence number from the
pre*ious step&
The *ictim host S all along concludes a *alid connection request from trusted host
& Then+ the hac'er could send data from host @&
One thing that needs to be clarified in this intrusion is+ (hen the hac'er from host @
masquerades himself as a trusted client and sends out a SY" pac'et+ the returned SY"0A4
pac'et from the *ictim host (ill go to the real host & As mentioned in the pre*ious chapter+
the host (ill immediately respond (ith a RST pac'et and the intrusion (ill fail because the
intended connection (ill be shutdo(n (hen the *ictim host S recei*ed this pac'et&
Therefore+ SY" flooding is al(ays performed as a preparing step in H!% SpoofingI& As a
result+ the returned SY"0A4 pac'et (ould not reach the destination host but gets lost on
the (ay& The reason is that the host is busy in dealing (ith large amounts of bogus
requests and runs out of system resources&
!% spoofing is a typical eBample of client?side spoofing intrusion& All the applications
(ith loose authentication mechanism based on !% address also face the threats from this type
of intrusion&
1&3&1&1 Ser*er?Side Spoofing
Ser*er?side spoofing employs a similar idea& :o(e*er+ the goals and the methods
used are a bit different& For the client?side spoofing+ as (e mentioned in the eBample of
HrloginI+ a hac'er impersonates an authori6ed user and then gains data from an information
pro*ider& 8y contrast+ the ser*er?side spoofing is eBecuted in the re*erse (ay& !n order to
obtain confidential information from indi*idual clients+ a hac'er masquerades as a real
ser*ice pro*ider and steals sensiti*e information from ser*ice users&
,.
The idea behind ser*er?side spoofing intrusion can be properly eBpressed by a real
life eBample& Suppose that some one creates a machine that loo's eBtremely li'e an ATM but
does not pro*ide the real functionality of a normal ATM& !nstead+ it records the number of an
ATM card and its holder/s %!" D%ersonal !dentification "umberE+ then reports some error
message to mislead the user that this machine has temporary mechanical problem& !f such a
machine (ere placed at the entrance of a shopping mall+ the result (ould be disastrous& A
user may lose large amounts of money #ust because he7she once used an out?of?order ATM
se*eral days before&
Same idea is employed in (eb spoofing& First step is to put some :TMC D:yperteBt
Ma'eup CanguageE lin's in some popular (eb pages& Ahen a *ictim *isits that page and
clic's that lin'+ all the follo(ing connections is hi#ac'ed by a malicious ser*er+ (hich hides
itself bet(een the user bro(ser and the real (eb ser*er& "o sophisticated technique is used
in this attac'& Some simple Ja*a script applet+ together (ith a little :TT% D:yperteBt
Transfer %rotocolE and G! Dommon Gate(ay !nterfaceE 'no(ledge+ is sufficient to hi#ac'
such connections& Aith the gro(ing popularity of electronic commence+ this type of
intrusion becomes e*en more dangerous& A malicious ser*er can easily grab personal
information from a (eb shopper+ such as credit card information&
2.3.3 Ser/ice Specific Intrusions
!n this section+ (e sur*ey some ser*ice specific intrusions+ such as finger daemon
attac'+ routing infrastructure intrusion+ 2"S misuse and se*eral attac's to "FS D"et(or'
File SystemE or @?Aindo(s system&
1&3&3&, Finger 2aemon Attac'
All the intrusions discussed abo*e (ere either attac's to the protocols themsel*es or
intrusions that result due to (ea' implementations of protocols& :o(e*er+ there are still a
number of intrusions that do not ha*e much relation (ith communication systems& 8ut (e
could still detect their eBistence by eBamining the related net(or' traffic&
,-
The (ell?'no(n !nternet (orm program fits into this category+ (hich eBploits some
fla(s in se*eral utility programs under )"!@ systems& One problematic utility+ (hich it
found+ (as fingerd& The fingerd program (as intended to run as a daemon+ or bac'ground
process+ to ser*ice remote requests using finger protocol F>G& The fingerd used a system?call
gets+ (hich does not chec' the length of the buffer used for reading& So+ a hac'er could
deliberately form a finger request to o*er(rite the buffer in fingerd, and this buffer o*er(rite
can be used to eBecute arbitrary command at the target system& A detailed description of this
attac' can be found in F,-G& Technically+ this intrusion is a buffer o*erflo( type of attac'&
8ut (e can still detect it by chec'ing the net(or' pac'ets to a finger daemon+ for a normal
request only has a small length of data+ (hile a malicious intrusion pac'et contains a large
amount of data intending to o*er(rite the system buffer&
1&3&3&1 Routing !nfrastructure !ntrusions
As described at the beginning of this thesis+ all the T%7!% ser*ices are built on a
connectionless pac'et deli*ery system F;G& Aith a layered protocol stac' in mind+ e*ery
message is transferred in the form of !% pac'et+ (hich is the basic unit of data tra*eling
among distributed net(or' de*ices& !n a large?scale and heterogeneous net(or'
en*ironment+ li'e the !nternet+ deli*ering a pac'et to the right destination is the tas' of
routing infrastructures&
!nternet adopts a hierarchical routing architecture+ (hich relie*es a single router from
storing huge amount of path information& A router ma'es routing decision of an !% pac'et
based on a data structure called routing table+ (hich 'eeps the status of each path lin'ed to
that router& !f R!% DRouting !nformation %rotocolE is used+ a router (ill periodically generate
CS) DCin' State )pdatesE that describe the latest status of the lin's to the router and
disseminate those updates to the other neighboring routers& Then+ based on CS) recei*ed+
routers update their o(n routing tables and cooperate in for(arding the !% pac'ets from
source to destination F.G&
%otential threats to the routing infrastructures come mainly from the spoofing
intrusions and some of them can lead to the results of denial of ser*ice& A faulty router can
1=
modify the pac'ets passing through it or discard the pac'ets at all& This may bring some
net(or's or hosts unreachable& Furthermore+ a malicious or compromised router can send
bogus routing control pac'ets+ li'e CS)+ to other routers+ (hich may in turn cause all the
pac'ets s(itch to itself and it can then ea*esdrop the content (ithin the pac'ets& Another
scenario is that a router sends bogus CS)/s that ma'es other routers thin' that some
reachable hosts are unreachable&
1&3&3&3 2"S Misuse
2"S is not a part of T%7!% protocol suite (hen it (as first proposed& :o(e*er+ (ith
millions of net(or's and hosts interconnected by the !nternet+ !% address becomes
incon*enient for an end?user to ma'e connections& An alternati*e approach is to map lo(?
le*el !% addresses into meaningful hostnames+ (hich is the main moti*ation of using 2"S&
2"S is a distributed database system+ (hich handles mapping high?le*el host names
into lo(?le*el !% addresses+ or *ice *ersa& Much li'e routing infrastructures+ 2"S is
composed by a large number of name ser*ers in a distributed hierarchical architecture+ (hile
each indi*idual name ser*er handles requests from a limited number of domains& !f a name
ser*er does not 'no( ho( to resol*e a particular query+ it may for(ard the query to another
name ser*er+ (hich either has much more information or is more specific to that particular
domain&
Most 2"S implementations adopt )2% as the transportation layer protocol& So+
besides the *ulnerabilities of 2"S+ security fla(s from )2%+ li'e lac' of state information
and (ea'ness in user authentication are also inherited& Aith a similar architecture as the
routing infrastructures+ 2"S faces the same threats from spoofing type intrusions& A misused
name ser*er could be easily used by a hac'er to masquerade himself as from any host+ for a
hac'er?controlled name ser*er can intercept a resol*er query and can respond (ith (hate*er
!% address a hac'er intends to be& A recently found bug in Ja*a class *erifier has a tight
relation (ith this 'ind of intrusion+ in (hich a malicious applet could connect (ith any host
other then the host from (hich it (as do(nloaded&
1,
aching is (idely used by 2"S to impro*e the system performance& !n 2"S
specifications+ there is a little concern for the data integrity and consistency of caching&
Therefore+ an intrusion by sending spoofed information to a name ser*er in a straightfor(ard
(ay (ill not (or'& !nstead+ a hac'er uses another approach called HAs' MeI to poison a
name ser*er/s cache by malicious data items F,=G&
!magine there is a hac'er on host @+ (ho has full control of name ser*er 8 and
intends to pro*ide the follo(ing (rong mapping information to name ser*er A$
!% of host @ "ame of host A
"ame of host A !% of host @
As "S 8 cannot directly send this malicious mapping to "S A+ it as's "S A to resol*e
a mapping that can only be handled by "S 8 itself& As a result+ "S A (ill for(ard this
request bac' to "S 8& "S 8 then appends the abo*e incorrect mapping information at the
response to "S A& Aith this little tric'+ the cache of name ser*er A (ill be poisoned by a
malicious record& After this point+ the hac'er on host @ can go ahead and launch some more
serious intrusions+ for instance+ an intrusion to(ards address?based applications li'e Hr?
utilitiesI&
!n addition to the li'ely results mentioned in the routing infrastructure intrusions+
such as misleading the pac'et flo(+ a 2"S intrusion can greatly facilitate the attac's aimed at
address?based applications& !n sum+ a combined intrusion on the 2"S system and the routing
mechanism can be catastrophic F1G&
1&3&3&5 "FS
"FS D"et(or' File SystemE (as created by S)" Microsystems and is the
predominant distributed file system in use today F-G& "FS allo(s files to be shared among
multiple hosts+ and it (or's in a (ay transparent to the user applications&
A "FS ser*er host eBports one or more file systems to be used by client hosts+ (hich
ha*e the permission to use those file systems& The "FS ser*er grants the access based
merely on the !% address of client machine+ (hich is not difficult to be spoofed& !n fact+ "FS
stores the configuration on the ser*er side+ including the information li'e (hich host is
11
authori6ed to use this system+ (hat le*el of access right a particular user has+ for eBample+
read only or read?(rite& The need for user authentication is left for client host& So+ from a
compromised host+ a malicious user could impersonate anyone and get access to the file
system& The entire "FS protocol is specified as a set of R%s DRemote %rocedure allE+
(hich is built on top of )2%& As (e mentioned earlier in this thesis+ )2% is connectionless
and is tri*ial to spoof&
At the application le*el+ "FS adopts a stateless approach+ (hich means the ser*er has
no idea about (hich file the remote clients open or (hat part of a file is being accessed& The
benefit of this approach is that the system is easy to reco*er from communication failures&
The do(nside is that a client can 'eep the file handle+ (hich is pre*iously assigned by the
ser*er+ and continue to use it+ e*en though the ser*er no longer trusts that client host&
1&3&3&9 @?Aindo(s
@?Aindo(s is a client7ser*er application mainly used for display management under
single or multiple )"!@ systems& An @ ser*er resides at a user machine Da&'&a& client
machineE and handles client inputs+ (hich may come from the local machine or from a
remote host& Then the ser*er dispatches these inputs to the corresponding applications
running at same machine& @ pro*ides three good features$ location independence+ hard(are
independence and operating system independence F-G&
:o(e*er+ @?Aindo(s has security (ea'ness in its access control and authentication
mechanism& There is no limitation on (hich process at that host can access the @ ser*er& So+
a hac'er can spoof as an @ client and recei*e inputs from other user processes on the same
host or e*en from other hosts& Furthermore+ the hac'er can modify the message not belong to
himself or bloc' the message from reaching the real user process& On the other hand+ an @
ser*er can also be spoofed+ (hich (ill lead to display of information from other users/
applications&
13
C"A#TER . %NTR'S%ON &ETECT%ON AN& #AC,ET F%LTER%N/
From the discussion in the pre*ious chapter+ (e can find most intrusions ta'e
ad*antages of *ulnerabilities in the system design and implementation& :o(e*er+ it is
impractical for us to eliminate all the errors in the eBisting systems or replace all the old
systems (ith ne( error?free systems gi*en the established base of soft(are& An alternati*e
approach in protecting a system from intrusion is to detect and isolate the problem before it
can impact the system performance or functionality& !n this chapter+ (e re*ie( some basic
techniques in enhancing system security and address some general issues in real?time pac'et
filtering+ (hich is important for net(or' intrusion detection&
.$ Current Tec6ni>ues in Network Security
A number of techniques ha*e been in*ented in the past fe( years to help a system
administrator in strengthening the security of a single host or the (hole computer net(or'&
Ae re*ie( a couple of most (idely used techniques&
3.1.1 Audit Trails
As defined by the "ational omputer Security enter in its Rainbo( series of system
security guide+ an audit trail is
#A chronological record of system activities that is sufficient to enable the
reconstruction, revie$ing, and e%amination of the se&uence of environments and activities
surrounding or leading to an operation, a procedure, or an event in a transaction from its
inception to final results.' ()*
Audit trail can be used in determining (hether an uneBpected or unauthori6ed
beha*ior has occurred in a system& Therefore+ it can be in*aluable to a system administrator
for net(or' management and security analysis& !n practice+ almost e*ery operating system
15
used today pro*ides auditing and logging utilities& Most of them are in the form of log files+
(hich record information from a user/s most recent login time and a user/s originating host+
to e*ery message generated by operating system 'ernel&
The hac'er+ (ho 'no(s (here to find the log files and ho( to modify their contents+
can easily ma'e a system+ loo' as if nothing has happened& Generally spea'ing+ auditing is a
'ind of post e*ent protection mechanism& !n other (ords+ by the time an intrusion is logged+
the hac'er may ha*e already bro'en into the system& Therefore+ audit trail may not be *ery
useful in terms of protecting a system from brea'?ins& Moreo*er+ an eBperienced hac'er can
typically defeat or circum*ent the auditing mechanisms& "e*ertheless+ system auditing can
be an effecti*e deterrent for ineBperienced hac'ers+ since it pro*ides a mechanism to trace
their acti*ities&
3.1.2 0ire1all
A recent trend in net(or' security enhancement in*ol*es the use of fire(all+ (hich is
a collection of filters and gate(ays that shield trusted net(or's (ithin a locally managed
security perimeter from the eBternal untrusted net(or's F,G&
3&,&1&, Screening Router
Screening router is a router+ (hich in addition to for(arding pac'ets li'e a normal
router+ also eBamines data in the pac'ets+ and applies some predefined access control policies
on the pac'ets to determine (hether they can be for(arded to the neBt hop or should be
discarded&
The pac'et?filtering function performed by a screening router is implemented by
eBamining a small portion of data in the header part of each pac'et+ such as source and
destination address in the !% header and port number in the T% or )2% header& Mean(hile+
some security policies are tested against each pac'et by the screening router+ mostly for the
purpose of access control& Aith carefully configured policies+ the screening router can be
effecti*e in pre*enting some classes of net(or' intrusions& For eBample+ (e can set up the
policy on a screening router as follo(s$
19
onfigure the eBternal interface of the router to bloc' incoming pac'ets that ha*e
source !% address from the internal net(or'L
onfigure the internal interface of the router to bloc' outgoing pac'ets that ha*e
source !% address from the eBternal net(or'&
This is effecti*e in pre*enting most !%?Spoofing based attac's aimed at or launched from
(ithin the local hosts&
:o(e*er+ since a screening router does not chec' the information other than protocol
headers+ it is unable to pre*ent attac's that depend on pac'et content& For instance+ it is not
capable of detecting attac's such as 2"S cache poisoning&
3&,&1&1 Application Gate(ay
2ue to the abo*e?mentioned limitations of a screening router+ *arious application
gate(ays are created to implement high?le*el policies in a fire(all strategy& As the name
implies+ an application gate(ay (or's at the le*el of application layer protocols rather than
being limited to !% or T% le*el& Application gate(ays pro*ide one or more of the follo(ing
functionality$ relay+ proBy and ser*er filter&
Relay gate(ay+ passes the data bet(een the t(o sides of a fire(all system& !n some
special en*ironments+ li'e a company using HlocalI !% addresses Di&e& *isible only (ithin the
companyE for internal net(or'+ a relay gate(ay should also pro*ide the function for
translating these addresses before they are sent out&
%roBy is of most importance to a fire(all system+ for most of access control policies
are enforced through application proBies& )sually+ a proBy gate(ay is application specific&
Ahen a client program inside the fire(all requires a connection (ith an outside ser*er+ an
application proBy on the fire(all (ill handle the request first& !t applies some security
policies against the connection request& !f the connection request is granted+ the proBy (ill
ma'e real connection to the ser*er outside the fire(all& 8eyond this point+ a proBy gate(ay
acts no more than a relay gate(ay&
Ser*er filter (or's in the opposite direction as an application proBy& !t handles the
incoming connection requests from eBternal net(or' to the internal ser*ers& Similar to inetd
1>
under most )"!@ systems+ a ser*er filter acts as a proBy for multiple internal application
ser*ers& Ahen recei*ing a connection request+ the ser*er filter dispatches it to the
corresponding application ser*er& The benefit obtained from using ser*er filter is that (e are
able to perform access control (ithout changing too much for the original application
ser*ers&
As an application gate(ay eBamines more data in a net(or' pac'et than a screening
router does+ it pro*ides more po(er in net(or' intrusion detection and pre*ention& On the
do(nside+ it requires more system resources and more processing time& As a tradeoff+
modern fire(all security systems al(ays adopt a combination of screening router and
application gate(ays DFigure 3&,E& )sually+ a screening router is placed as the first line of
defense+ (hich is used to filter out in*alid net(or' traffic by applying the policies against !%
address+ T% or )2% port+ and so on& Then+ the pac'ets left are for(arded application
gate(ays+ (hich implement higher?le*el security policies&
Firewall System
Figure .$ Screening Router and Application /ateway
.( #acket Filtering
%ac'et filtering technique (as in*ented for diagnostic and analysis purpose in
net(or' management& Cater on+ it began to be used by the net(or' security systems& As (e
mentioned earlier+ it forms the foundation for the fire(all strategy& "either screening router
%nternal
Networ
k
EAterna
l
Networ
k
Screening
Router
Bastion
"ost
9Application
/ateways:
1;
nor application gate(ay can li*e (ithout pac'et filters& At present+ the pac'et?filtering
technique also plays an important role in net(or' intrusion detection&
3.2.1 2eneral Issues
The 'ey issues on building a pac'et filter are$
"eal-time performance$ the pac'et filter should be able to quic'ly capture a ra(
pac'et from data lin' layer and process it in a short period of time&
No pac+et dropping$ no pac'et dropping is allo(ed+ especially for a net(or'
intrusion detection system& The information missed from dropped pac'ets can
ma'e the (hole detection scheme fail&
,le%ibility$ the specification of pac'et patterns can be modified easily to support
different communication protocols&
calability$ in terms of a system for net(or' intrusion detection+ ne( intrusion
signatures can be added into the pac'et filter (ithout degrading performance&
3.2.2 3)istin Pac4et50ilterin Syste#s
The three common methods for accessing the data lin' layer and retrie*ing ra(
pac'ets under )"!@ are the 8S2 %ac'et Filter D8%FE+ the S<R5 2ata Cin' %ro*ider
!nterface D2C%!E and the CinuB SOCK_PACKET interface F,,G&
3&1&1&, CinuB SO40%A4ET
CinuB has an elementary mechanism for pac'et capture+ namely SOCK_PACKET& !t
pro*ides the capacity for a user process to directly fetch pac'et from data lin' layer&
:o(e*er+ there is no buffering and filtering done in the 'ernel space for this mechanism&
Therefore+ it means most of pac'ets (ill be copied into the user space for further processing&
The o*erhead to a pac'et filter is ob*ious in this implementation&
1.
3&1&1&1 2ata Cin' %ro*ider !nterface
2C%! is a protocol?independent interface designed by ATMT& !t pro*ides a user
application (ith the ser*ice to the data lin' layer under most S<R5 systems F,,G& As sho(n
in Figure 3&1+ there are t(o main components (ithin 2C%!$ pfmod (hich filters the pac'et
stream before it goes to the user space+ and bufmod+ (hich buffers filtered pac'ets bet(een
data lin' layer and user process& The primary ad*antage of 2C%! o*er CinuB SOCK_PACKET is
part of pac'et filtering can be performed inside the 'ernel space (ithout the o*erhead
brought by cross boundary copy of pac'ets&
Figure .( &L#% System Arc6itecture
The filter model used by 2C%! is straightfor(ard and can be described as a boolean
eBpression tree& Figure 3&3 sho(s an eBample filter built by 2C%! to collect all the pac'ets
coming from Hhost fooI+ no matter the pac'et is an !% pac'et or an AR% or RAR% pac'et&
3&1&1&3 8S2 %ac'et Filter
8%F (as originally created for 8S2 )"!@ and has been ported to many )"!@
fla*ors& !t is an elegant solution for pac'et filtering and pro*ides better performance than
application application
7ufmod
97uffer:
pfmod
9filter:
pfmod
9filter:
7ufmod
97uffer:
protocol stack
data
link
process
kernel
1-
Figure .. &L#% Filter 3odel
other pac'et filtering systems described abo*e&
8%F consists of t(o main components$ net(or' tap and pac'et filter F,=G& The
net(or' tap is responsible for copying ra( pac'ets from de*ice dri*ers and mo*ing them to
the listening user processes& The filters are sitting bet(een the de*ice dri*er and user process
as sho(n in Figure 3&5& They apply the filtering patterns and determine (hether a pac'et
should be deli*ered to the upper?le*el component or it should be discarded in the 'ernel
space&
kernel
packets
network
Figure .0 B#F System Arc6itecture
user process user process
filter filter
B#F dri!er
de!ice dri!er
protocol stack
OR
A"
2
A"
2
OR OR OR
ether&typ
e
NAR%
arp&ds
t
Nfoo
arp&sr
c
Nfoo
ether&typ
e
NRAR%
ether&typ
e
N!%
ip&sr
c
Nfoo
ip&ds
t
Nfoo
EApression Tree Filter Function for B6ost fooC
3=
Although 8%F has a similar architecture (ith 2C%!+ it adopts a different approach in
the filter model design& Other than using the boolean eBpression tree+ 8%F builds a pac'et
filter (ith a directed acyclic control flo( graph DFGE& Figure 3&9 sho(s a sample 8%F filter
(ith the same function as that of the pre*ious 2C%! filter&
Figure .- B#F Filter 3odel
omparing the abo*e t(o filters+ (e can clearly see the ad*antage of 8%F model o*er
2C%! model& !n the 2C%! filter+ an eBpression may be computed redundantly& For instance+
Hether.type==ARPI (ill be e*aluated+ e*en if the tested for Hether.type==IPI has been
satisfied& On the other hand+ each path in the 8%F filters eBclusi*ely represents a comparison
procedure needs to be done for a particular pac'et pattern& !n other (ords+ the pac'et
information is HrememberedI in the filter& Once an eBpression is e*aluated+ it need not be
computed again e*ery time it is referenced&
3.2.3 Pac4et Capture 6i$rary
The library+ libpcp+ is a set of implementation?independent A%!s for pac'et
capturing and filtering& !t can be built on top of either one of the three pac'et capture
facilities (e mentioned before+ CinuB SOCK_PACKET+ 2C%! and 8%F&
The libpcp pro*ides a po(erful language for pac'et pattern specification& The
patterns (ritten in this language can be translated into a filter function+ (hich may either use
ether&typeN!
%
ip&srcNfo
o
ether&typeNRAR%
arp&srcNfo
o
arp&dstNfo
o
TR)E FACS
E
ether&typeNAR%
ip&dstNfo
o
CF/ Filter Function for B6ost fooC
3,
an eBpression tree or a FG+ depending on (hich underlying pac'et capture mechanism is
being used&
Follo(ing are t(o sample pac'et patterns that can be recogni6ed by libpcp$
tcp port 25
icmp[0] != 8 and icmp[0] != 0
The first one describes all the pac'ets to and from T% port 19& The second one intends to
capture all the !M% pac'ets (ithout !M% echo requests or echo replies&
As illustrated by the eBample patterns+ this specification language supports basic
protocol types of T%7!%+ such as !%+ T% and !M%+ etc& Furthermore+ it also facilitates the
pattern specification by abstracting se*eral frequently used fields in the protocol headers+ li'e
source and destination address in !% header+ T% and )2% port+ etc& !n case a user needs to
access a field not in any predefined protocol+ a pac'et can be simply treated as a plain byte
stream& Any reference to a pac'et field can be con*erted to byte offset+ as illustrated in the
second eBample&
.. BroD An %ntrusion &etection System Based on #acket Filtering
%ac'et filtering can be *ery important to a net(or' intrusion detection system& !n this
section+ (e introduce 8ro+ a stand?alone system de*eloped by the "et(or' Research Group
in Ca(rence 8er'eley "ational Caboratory for detecting net(or' intruders in real?time by
passi*ely monitoring a net(or' lin' o*er (hich the intruder/s traffic transits F,1G&
3.3.1 7ro Architecture
The design of 8ro is through a hierarchical and modular approach& onceptually+
there are three components in a 8ro system$ a pac'et capturing and filtering module
implemented by libpcp+ an e*ent engine and a policy script interpreter DFigure 3&>E&
The pac'et?filtering module of 8ro captures the ra( pac'et stream from data lin'
layer and applies filter function to choose the pac'ets need to be passed to the higher layer
component& The use of libpcp in the implementation may bring significant performance
impro*ement if a 'ernel filter can be used+ li'e 8%F&
Network
%ac'et stream
31
Figure .4 Bro Arc6itecture
Ahen the filtered pac'et stream arri*es at the 8ro e*ent engine+ it (ill be further
processed for the con*enience of high?le*el analysis& For instance+ a T% pac'et (ill be
chec'ed for connection status+ if either of SY"+ F!" or RST flag is disco*ered in the pac'et
header& After the pac'et processing+ the e*ent engine (ill chec' (hether an e*ent is
generated& !f so+ it (ill trigger the corresponding e*ent handler specified in the policy script&
%olicy script captures the response to be ta'en (hen an e*ent is detected& !n the
policy script+ a specification language is used to specify the e*ent handler+ (hich can be
*arious commands supported by 8ro runtime system+ such as generating ne( e*ent+ logging
notifications+ recording data to dis'+ or modifying internal state for sequent e*ents F,1G& An
interpreter is used to bind the e*ent (ith the rele*ant handlers&
3.3.2 7ro 6anuae
2ifferent from the specification language used by the libpcp+ in (hich the patterns
for the pac'ets to be captured are described+ 8ro language is a domain?specific language
mainly used to (rite e*ent handlers& Simply put+ an e*ent handler describes the processes
need to be done after a pattern has been obser*ed&
#olicy Script %nterpreter
E!ent Engine
li7pcap
Real?time notificaition
E*ent stream
Filtered pac'et stream
%olicy script
E*ent control
%ac'et Filter
33
!n the 8ro system+ pac'et structures are totally hidden from the security policy
(riters& 2ifferent pac'et patterns are encapsulated through a number of predefined OO
classes& Ta'e finger ser*ice as an eBample& !n the 8ro e*ent engine+ there is a class
!ingerConn+ (hich is deri*ed from the general purpose TCP_Connection class& Ahene*er a
ne( connection is encountered (ith ser*ice port ;-+ a !ingerConn ob#ect (ill be
instantiated+ instead of a TCP_Connection ob#ect for an unrecogni6ed port F,1G& Therefore+
for each specific ser*ice+ a corresponding class needs to be implemented for application
specific process&
For a security policy (riter+ (hat he7she needs to do is to specify the
finger_re"uest+ (hich is the e*ent handler for a finger connection& Follo(ing is the entire
policy script for the finger ser*ice&
globl hot_nmes = $%root%& %lp%& %uucp%& %opertor%& %dmin%& %system%'(
globl finger_log = open) geten*)%+RO_I,%- == %% . %finger.log% /
fmt)%finger.0s%& geten*)%+RO_I,%-- -(
e*ent finger_re"uest)c/connection( re"uest/ string( full/ bool(-
1
locl id = c2id(
if ) byte_len)re"uest- 3 45 -
1
re"uest = fmt)%0s...%& sub_bytes)re"uest& 6& 45--(
77c2hot(
8
if ) re"uest in hot_nmes -
77c2hot(
locl re" = re"uest == %% . %A99% / fmt)%:%0s:%%& re"uest-(
if ) c2ddl ;= %% -
< This is n dditionl re"uest.
re" = fmt)%)0s-%& re"-(
if ) full -
re" = fmt)%0s )=>-%& re"-(
locl msg = fmt)%0s 3 0s 0s%& id2orig_h& id2resp_h& re"-(
if ) c2hot 3 5 -
log fmt)%finger/ 0s%& msg-(
35
print finger_log& fmt)%0.?f 0s%& c2strt_time& msg-(
c2ddl = c2ddl == %% . re" / fmt)%@0s& 0s%& c2ddl& re"-(
8
The first line in the script defines a set of strings+ (hich represents the login names
of some system users& The second statement specifies a file to store finger logs& !n 8ro
language+ record field access is using 2 to a*oid ambiguity (ith constants used for hostname
or !% addresses& So+ the eBpression c2id is the same as c.id in & Follo(ing is the e*ent
handler+ (hich is eBecuted (hen a finger request arri*es& The e*ent handler chec's (hether
the request is eBcessi*ely long and (hether the request corresponds to any of the entries in
the hot_nme set& Finally+ it logs the request into a log file& The detailed description can be
found in F,1G&
.0 #acket Filtering for Network %ntrusion &etection
8esides the requirements for general?purpose pac'et filtering+ a net(or' pac'et filter
for intrusion detection faces many ne( challenges+ li'e the large number of pac'et patterns
and more sophisticated pac'et patterns& Therefore+ both the eBisting pac'et?filtering facilities
and 8ro system suffer limitation (hen they are applied to a large?scale intrusion detection
frame(or'&
The primary concern in a pac'et filter+ li'e 8%F+ is to reduce the number of pac'ets
copied from 'ernel space to user space& The reason is that most reaction logic to a net(or'
pac'et is located (ithin the user space& Minimi6ing the pac'ets that need to be copied is the
main goal of 8%F to impro*e the entire system performance& The time needed for that the
user process in computing the reaction logic is not considered& :o(e*er+ this is not the case
for a pac'et filter for net(or' intrusion detection purpose+ for the reaction to a pac'et needs
to be ta'en right a(ay& For instance+ if a pac'et filter detects a pac'et used by a
:ARGE"7E:O attac'+ it is better to discard this pac'et (ithout passing it to the high?
le*el system component& !n addition+ another difference of the pac'et filter for intrusion
detection is that the filter needs to report all the patterns matched by a pac'et instead of
indicating (hether the pac'et should be copied or not&
39
As described in the pre*ious section+ the language of 8ro system is layered on top of
8%F+ and thus suffers from the dra(bac' mentioned abo*e& 8lind copying (ould necessitate
that the high?le*el process perform the matching operations again in order to identify (hich
rules are applicable& Moreo*er+ lo(?le*el pac'et patterns are captured by a set of user?
de*eloped OO classes in the 8ro system& So+ fleBibility becomes an issue (hen the same
security polices need to be applied for different protocols or ne( pac'et patterns are to be
added into the eBisting system& The reason is that the user needs to re(rite the entire OO
class hierarchy or pro*ide ne( deri*ed class for each ne( pac'et pattern&
Therefore+ a po(erful specification language is required for describing *arious
intrusion patterns& The ideal language should be protocol?independent and easy to change for
different pac'et structures& !n addition+ the approach must be robust in the face of any attac'
on the pac'et filter itself& An important concern in this conteBt is type and memory safety+
(hich focus a typed language that ensures that adequate type chec'ing can be performed Dat
compile time or runtimeE to ensure safety of all memory accesses& !n order to ma'e a
language memory safe and type safe+ automatic type chec'ing is highly desired& For
eBample+ before the !% source address in a pac'et can be tested+ the pac'et should ha*e been
*erified as an !% pac'et& Moreo*er+ performance is especially important to a pac'et filter for
intrusion detection& For instance+ a hac'er may o*erburden the filter before he7she launches
the real attac'& As far as the filter model is concerned+ a highly simplified boolean function
is not strong enough for an intrusion detection system+ because it needs to distinguish *arious
patterns in the net(or' pac'ets&
3>
C"A#TER 0 %NTR'S%ON #ATTERN S#EC%F%CAT%ON LAN/'A/E
"et(or' intrusion detection brings a number of ne( challenges to a pac'et filtering
system+ such as fleBibility+ scalability and robustness& 8oth the eBisting pac'et?filtering
facilities and 8ro system discussed in chapter 3 suffer from limitation (hen they are faced
(ith the requirements for a net(or' security system& !n this chapter+ (e present a no*el
approach in the design of a domain?specific language for net(or' intrusion detection& ASC is
our specification language for describing intrusion patterns and reaction& This language is
protocol independent+ eBtensible and type safe&
0$ ASL SyntaA
ASC is made up by the rules of the form
pattern - condition reaction
The pattern specifies the e*ent that initiates a rule& !n terms of a pac'et filtering system+ a
pattern is al(ays an arri*ing net(or' pac'et& The condition part usually includes simple tests
on the components of the pattern& As far as a net(or' pac'et is concerned+ a condition
typically consists of a series of comparisons in*ol*ing pac'et fields& Once the condition is
satisfied+ the actions specified by the reaction part (ill be triggered&
The discussion of reaction is beyond the scope of this thesis+ as (e only focus on the
pattern matching for the net(or' pac'ets& !t (ill suffice to assume that reaction is simply an
identification of the rule that matched
0( #acket Structure &escription
An ob*ious benefit from using pac'et structures is con*enience for describing a
pac'et pattern& Suppose (e do not use any structure for pac'et?field access& !nstead+ (e
treat a pac'et as a byte stream& Then+ a reference to the protocol field inside an Ethernet
3;
header (ill loo' li'e this+ H.short/p(01*I& A dra(bac' of this approach is the type
information for each filed is lost and type casting needs to be used e*ery(here+ for a filter
cannot 'no( (hether you (ant to use a t(o?byte short *alue or a four?byte integer&
To capture the nested structure of protocol headers+ ASC employs a language
mechanism similar to inheritance& Therefore+ an !% header can be considered as an eBtension
of Ethernet header (ith eBtra options for !% protocol& Similarly+ a T% header is inherited
from !% header (ith entire data members from !% header and Ethernet header& Follo(ing are
some code snippets for the definition of these data structures$
6.<define ETAER_9EB ?
C.struct ether_hdr 1
D. byte e_dst$ETAER_9EB'( =@ Ethernet destintion ddress @=
E. byte e_src$ETAER_9EB'( =@ Ethernet source ddress @=
F. short e_type( =@ protocol of crried pcGet @=
?.8(
H.struct ip_hdr / ether_hdr 1 =@ ether_hdr plus folloIing fields @=
4. bit *ersion$E'( =@ ip *ersion @=
J. bit ihl$E'( =@ heder length @=
65. byte tos( =@ type of ser*ice @=
66. short tot_len( =@ totl length @=
6C K
6D. byte protocol( =@ highLle*el protocol @=
6E. K
6F.8(
6?.struct tcp_hdr / ip_hdr 1 =@ ip_hdr plus folloIing fields @=
6H. short tcp_sport( =@ source port number @=
64. short tcp_dport( =@ destintion port number @=
6J. int tcp_se"( =@ se"uence number @=
C5. int tcp_cGse"( =@ cGnoIledge number @=
C6. K
CC.8(
CD.struct udp_hdr / ip_hdr 1 =@ ip_hdr plus folloIing fields @=
CE. byte udp_sport( =@ source port number @=
CF. byte udp_dport( =@ destintion port number @=
C?. byte udp_len( =@ heder 7 dt length @=
3.
CH. byte udp_csum( =@ checGsum for heder M dt @=
C4.8(
%ac'et structure description is a no*el (ay in pro*iding a pac'et filter (ith fleBibility
and eBtensibility& A ne( protocol such as !M% can be easily supported by the system (ith a
set of ne(ly defined pac'et structures& !n addition+ the (hole system can be con*eniently
ported to another protocol suite+ li'e !%*>&
0. Constraint C6ecking
Another requirement for ASC is to be type safe for each pac'et operation& For
instance+ before any access to the field tcp_sport+ a pac'et must be *erified as a T%
pac'et& :o(e*er+ among the pre*iously defined pac'et structures+ it is not easy to
distinguish bet(een the structure tcp_hdr and udp_hdr+ for both of them are deri*ed from
ip_hdr& Ae then strengthen our data structure definition by introducing the concept of
constraint chec'ing& A direct obser*ation from T%7!% specification is that+ a field in high?
le*el protocol header al(ays has dependence on some particular *alue in the lo(?le*el
header& For eBample+ only if the e_type field in the ether_hdr equals to =B=.==+ the
follo(ing part of a pac'et can be an !% header& 8ased on this obser*ation+ (e modify the
definition of !% header as$
<define ETAER_IP 5N5455
struct ip_hdr / ether_hdr Iith e_type == ETAER_IP 1
bit *ersion$E'( =@ ip *ersion @=
bit ihl$E'( =@ heder length @=
byte tos( =@ type of ser*ice @=
short tot_len( =@ totl length @=
K
byte protocol( =@ highLle*el protocol @=
8(
Follo(ing the ne( 'ey(ord Iith is the constraint of this pac'et structure& Ae can thin' in
this (ay+ an !% header is an Ethernet header (ith its e_type field equal to a specific *alue
ETAER_IP& Follo(ing the same rule+ (e can change the structures of tcp_hdr and udp_hdr
as follo(s$
3-
<define IP_TCP 5N555?
<define IP_O,P 5N5566
struct tcp_hdr / ip_hdr Iith protocol == IP_TCP 1
short tcp_sport( =@ source port number @=
short tcp_dport( =@ destintion port number @=
int tcp_se"( =@ se"uence number @=
int tcp_cGse"( =@ cGnoIledge number @=
K
8(
struct udp_hdr / ip_hdr Iith protocol == IP_O,P 1
byte udp_sport( =@ source port number @=
byte udp_dport( =@ destintion port number @=
byte udp_len( =@ heder 7 dt length @=
byte udp_csum( =@ checGsum for heder M dt @=
8(
Sometimes+ dependence may also eBist among different fields in the same pac'et structure&
)sually+ a field occurring in the later portion of a pac'et depends on the *alue of a field
occurring earlier& :ere is an eBample$
struct ip_pGt / ip_hdr 1
byte ip_dt$' Iith siPeof)ip_dt- == tot_len L ihl(
8(
!n this case+ the length of data segment of an !% pac'et is calculated by t(o fields+ tot_len
and ihl+ all of (hich are fields of ip_hdr& 8asically+ (hene*er a field inside a pac'et
structure is to be accessed+ all of the constraints included by the structure and its parent
structures need to be eBamined first&
There is a bit concern on ho( to determine the type for a pac'et *ariable& !n ASC+ (e
declare the type for the pac'et *ariable p occurred in the pcGet)p- e*ent specification as
the base structure ether_hdr& The reason is that the pcGet)p- e*ent can be used in
multiple rules+ (hile different rule is able to use it at different protocol le*el+ li'e !%+ T% or
:TT%& As a result+ p can be of different types& :o(e*er+ (e declare it as the base structure&
Once a field of p is referenced in a rule+ (e need to do the name loo'up inside the structure
ether_hdr or its child structures& Then (e can obtain the type information for the *ariable
5=
p& !n turn+ (e are able to locate all the required constraints for p+ (hich are already defined
in the pac'et structure description& Therefore+ a requirement for pac'et structure design is to
uniquely name each pac'et field across all the pac'et structures&
00 Sample #atterns
Ae illustrate ho( to use ASC to (rite the pac'et pattern rules by a couple of simple
eBamples& First+ (e re*isit the Hhost fooI filter+ in (hich a pac'et filter is required to capture
all the pac'ets coming from host HfooI& Supposing H%%.yy.22.$$I is the !% address of host
HfooI+ the corresponding rule loo's li'e$
pac+et.p/ - .p.s3addr 44 %%.yy.22.$$/ message.#host foo'/5
The pac+et is a predefined pattern+ representing a ra( pac'et obtained from a de*ice
dri*er& The condition part is simply a boolean function+ in (hich+ the Hp.s3addr' refers to
the !% source address of the pac'et p& As (e mentioned earlier in this thesis+ e*ery pac'et
operation must be type safe& So+ before the p.s3addr can be accessed+ (e need to *erify that
the pac'et belongs to !% protocol& Thus+ this pattern is equi*alent to$
pac+et.p/ - .p.e3type 446786"3I!/ 99 .p.s3addr 44 %%.yy.%%.$$/5
The neBt eBample is a real (orld net(or' intrusion+ %ing of 2eath& As (e described
in chapter 1+ the nature of this intrusion is ma'ing sum of the offset and pac'et si6e of the last
fragment in an !% fragment series+ eBceed the maBimum si6e of an !% pac'et& The matching
rule is$
pac+et.p/ - .p.tot3len:p.frag3off ; MA<3I!3I=6/ message.#!ing of Death'/5
Ae can easily understand the pattern of this intrusion from the abo*e rule&
The :ARGE" and E:O attac' discussed in hapter 1 can be another eBample of
using ASC to describe the intrusion patterns& This attac' is straightfor(ard and its
corresponding ASC rule is as follo(s$
pac+et.p/ - .p.udp3sport 44 !>"736C8>/ 99 .p.udp3dport 44 !>"73C8A"?6N/
message.#C8A"?6N@6C8> Attac+'/5
As (e mentioned earlier in this thesis+ finger daemon attac' is a buffer o*erflo( type
intrusion& :o(e*er+ (e can also (rite a rule in ASC to capture the pattern of this attac'&
pac+et.p/ - .p.tot3len A p.ihl A p.tcp3len ; MA<3,IN?6"3B6N?78/
5,
message.#,inger Cuffer >verflo$'/5
!n this rule+ p.tot3len A p.ihl A p.tcp3len is used to calculate the length of the data portion in a
finger request pac'et+ (here p.tot3len is the total length of an !% pac'et+ p.ihl and p.tcp3len is the
length of !% header and T% header separately&
51
C"A#TER - S2STE3 &ES%/N AN& %3#LE3ENTAT%ON
The ASC?%FM DASC %ac'et Filtering ModuleE plays a fundamental role in our entire
net(or' intrusion detection system& !t facilitates the intrusion detection process by pro*iding
a po(erful pac'et pattern matching function& !n this chapter+ (e mainly focus on the design
and implementation issues in constructing this pac'et filter module&
-$ System Arc6itecture
The pac'et patterns (ritten in ASC (ill be translated into a runtime filtering system
DFigure 9&,E& Then the incoming net(or' pac'ets (ill be fed into the runtime ASC?%FM
directly from data lin' layer& Once a pattern is matched+ the ASC?%FM (ill trigger the
reaction part in the rule& The reaction (ill typically be to print a message that identifies the
matched patternDsE&
Figure -$ ASL=#F3 Arc6itecture
E!ent Engine
Runtime ASL=#F3
tcpdump
file
&ata link
layer
packet stream
e!ent stream
53
For the purpose of system diagnosis and performance analysis+ (e also pro*ide ASC?
%FM (ith the ability to fetch the data from files& !n addition+ a data generation module has
been de*eloped to produce test data&
-( #acket Offset Calculation
The pac'et structure represented in the language ASC+ can facilitate rule (riting and
pac'et type chec'ing& :o(e*er+ the runtime system cannot recogni6e any data structures but
only access data using a byte offset+ e&g& p.e_type field of a pac'et p can only be tested in
the form of p$6C'+ in (hich 6C is the offset corresponding to the field e_type&
Most of the offset calculation can be done at compile time+ because the corresponding
offset is a constant& For eBample+ the offset for e_type is 6C+ (hile the offset for e_src is ?
all the time& Follo(ing is the definition of the structure rp_hdr and the offset calculation
procedure for the field r_op$
struct rp_hdr / ether_hdr Iith e_type == ETAER_ARP 1
short r_hrd( =@ !ormt of hrdIre ddress @=
short r_pro( =@ !ormt of protocol ddress @=
byte r_hln( =@ 9ength of hrdIre ddress @=
byte r_pln( =@ 9ength of protocol ddress @=
short r_op( =@ ARP opcode )commnd- @=
8(
offset)r_op- = offset)rp_hdr- 7 siPeof)short-@C 7 siPeof)byte-@C
= offset)ether_hdr- 7 siPeof)ether_ddr-@C
7 siPeof)short- 7 siPeof)short-@C 7 siPeof)byte-@C
= 5 7 ?@C 7 C 7 C@C 7 6@C
= C5
:o(e*er+ there are some offset calculations that cannot be done prior to the runtime+
li'e the starting offset for the data segment of an !% pac'et (ith eBtra options& The data
begins right after the header part of an !% pac'et& !n other (ords+ offset)ip_dt- equals
offset)ip_hdr-7siPeof)ip_hdr-& The length of an !% header is specified by the field ihl
55
of structure ip_hdr& 8ut you can only obtain the *alue of ihl at runtime& So+ the computing
of offset)ip_dt- (ill be delayed to the time at (hich a pac'et is being parsed&
-. Filter 3odel for Single Rule
The design of filter model adopted by the ASC pac'et filtering module gains much
benefit from the FG model of 8%F& The FG model is good at 'eeping the parsing
information for already parsed pac'et+ (hile reducing the possibility of redundant testing& !n
addition+ it can facilitate type chec'ing required by the ASC matching rules&
The Hhost fooI eBample can be used as (ell to sho( ho( the FG model fits for
building a filter for matching a single rule& The first step is to (rite a matching rule and
patching proper type chec'ing conditions+ as (e ha*e done in pre*ious section$
pac+et. p/ - .p.s3addr 44%%.yy.22.$$/ message.#host foo'/5
pac+et. p/ - .p.e3type 446786"3I!/ 99 .p.s3addr 44%%.yy.22.$$/ message.#host foo'/5
"eBt+ (e can establish the filter tree (ith the reference to FG model&
As sho(n in Figure 9&1+ a little difference from the pre*ious FG filter happens at the
rule?matching node& !nstead of returning a boolean *alue+ ASC filter is required to in*o'e the
corresponding reactions described in the rule& !n the abo*e filter tree+ (e notice that each
comparison node can only ha*e t(o outgoing branches& Sometimes+ it may degrade the
(hole structure of the tree& For eBample+ there is an intrusion pattern called !% )n'no(n
%rotocol attac'& The idea of this attac' is using an in*alid *alue at the protocol field in the !%
header of a net(or' pac'et to crash a poorly implemented T%7!% stac'& Assuming only four
protocols are supported in T%7!% specification+ i&e& T%+ )2%+ !M% and !GM%& Ae can
describe this intrusion as$
p&e0typeNNET:ER0!%
p&s0addrNNBB&yy&66&((
True P message DHhost
fooIE
False
59
Figure -( Sample Filter for B"ost fooC
pac+et. p/ - .p.protocol D4I!37C!/ 99 .p.protocol D4 I!3ED!/
99 .p.protocol D4 I!3ICM!/ 99 .p.protocol D4 I!3I?M!/
message.#I! En+no$n !rotocol'/5
The corresponding filter tree is sho(n in Figure 9&3& Ae can optimi6e the tree
structure by pulling the *alue out of the comparison node and con*erting it into an output
branch& The immediate benefit is the decrease on the si6e of the tree+ for a comparison node
can ha*e multiple branches& The impro*ed *ersion of filter tree for the intrusion pattern H!%
)n'no(n %rotocolI is sho(n is Figure 9&5&
Figure -. Sample Filter for B%# 'nknown #rotocolC
Figure -0 %mpro!ed Filter for B%# 'nknown #rotocolC
p&e0typeNNET:ER0!%
p&protocol QN !%0T%
p&protocol QN !%0)2%
p&protocol QN
!%0!M%
p&protocol QN
!%0!GM%
True P message DH!% )n'no(n %rotocolIE
Fals
e
!%0T%
ET:ER0!%
p&e0typ
e
p&protocol
True P message DH!% )n'no(n %rotocolIE
Fals
e
!%0!M%
!%0!GM%

!%0)2%
5>
-0 Filter %ntegration
!f (e can build an efficient filter for a single rule+ does it mean (e can build an
equally efficient pac'et filtering system for multiple rulesK The ans(er is no&
An ideal pac'et filtering system should ha*e the follo(ing properties$
All the matching rules can be identified in a single scan& !t means that a pac'et
can be processed (ithout repeated tests on the same fields&
The time required for rule matching is insensiti*e to the number of rules&
8asically+ there are t(o approaches in building a pac'et filer for multiple rules& One
is to simply put together all the filters for indi*idual rules+ then test these filters one after
another& learly+ this is not a good choice& For eBample+ a pac'et (ill be tested against each
rule separately+ (hile e*ery elementary comparison common to the multiple rules (ill be
chec'ed more than once& Ci'e the eBpression Hp.e3typeI+ it is going to be compared (ith the
same *alue by all the rules for the !% protocol in the system& More o*er+ the number of
matching rules largely affects the performance of a filtering system& !n other (ords+ a
filtering system (or'ing fine for fi*e rules may become unacceptably slo( for one hundred
rules&
Another approach in building a pac'et filtering system is through a 2FA
D2eterministic Finite AutomatonE li'e automaton+ (hich can rapidly select the matching?
patterns in a single scan of input F,3G& A typical scenario in fulfilling this approach is to
preprocess all the patterns into a 2FA?li'e automaton+ then scan the pac'et fields in a left to
right manner& !n the paper (ritten by R& & Se'ar+ R& Ramesh+ and !&<& Rama'rishnan F,3G+ a
ne( concept named HAdapti*e %attern MatchingI is proposed& The basic idea is to adapt the
tra*ersal order to suit the input patterns& Simply put+ instead of bro(sing the information
from the input one by one+ (e can impro*e the system performance by s'ipping o*er those
fields that are irrele*ant for matching any pattern& A detailed discussion of this algorithm can
be found in F,3G&
The adapti*e pattern matching is a good fit for the pac'et filtering system& First+ in a
net(or' pac'et+ most of the critical information is stored in the *arious protocol headers+ li'e
!% header+ T% header or :TT% header+ etc& Aithin a protocol header+ (e may only care
5;
about a small part of fields+ e&g& source address in the !% header+ SY" flag in the T% header&
Therefore+ many fields in the protocol headers and almost the entire data portion of a pac'et
are al(ays useless for the pattern matching purpose+ because most 'no(n intrusion patterns
can be disco*ered through chec'ing partial number of fields in a pac'et& So+ by s'ipping
most irrele*ant data and eBamining only a limited number of fields of a pac'et+ (e can gain
significant performance impro*ement o*er traditional pac'et filtering approach&
!n the conteBt of pac'et filtering for net(or' intrusion detection+ an intrusion pattern
is described as a part of a matching rule in our ASC system& A direct obser*ation is more
than one rule can be matched simultaneously& For instance+ a rule Hp.s3addr44%%.yy.22.$$I can
be matched at the same time another rule H.p.s3addr44%%.yy.22.$$/ 99 .p.protocol44I!3ICM!/I is
matched& This is the difference of our filter model from that of 8%F or any other pac'et
filter& !n those filters+ only one *alue is returned by the filter function to indicate (hether the
pac'et should be captured or discarded& 8y contrast+ our filter needs to report all the rules
matched by a pac'et& Therefore+ (e need a data structure to represent either the rules already
matched or the rules to be matched at each parsing stage&
An algorithm for filter automaton construction is sho(n in Figure 9&9&
%rocedure 8uild is recursi*e and the entire automaton can be established by in*o'ing
8uild DrootE+ (here the root is a node (ith an empty matching rule set and a candidate rule
set containing all of the rules specified& !n line 1+ the candidate rule set is eBamined& !f no
potential rule can be matched+ the procedure (ill be terminated& Other(ise+ the construction
process continues from line 5 to line ,9& !n terms of pac'et filtering+ the neBt position to be
inspected refers to an offset in a pac'et& !t is computed by select function+ (hich can be
implemented through different strategies& Ae (ill discuss this function later& Once the offset
has been set up for the current node+ (e need to create a ne( node for each *alue appearing
at that offset and mar' the transition by the *alue& !n addition+ (e construct a branch
representing a *alue other than all the *alues appearing at the offset in any of the patterns in
& !f a rule can be matched after the comparison at this step+ (e can put this rule into the
matched rule set of the ne( node& Other(ise+ (e put it into the candidate rule set of the ne(
node& !f a rule (ith inequality comparison has been matched+ (e need to put that
5.
Procedure +uild )*- 1
6. * is node in utomton 7R eBtra information are attached to each node$ p is the
offset to be inspected+ m is the set of already matched rules and c is the set of
candidate rules R7
C. if )*.c is empty-
D. stop 7R if no candidate rule+ terminate the procedure R7
E. *.p = select)*.c- 7R select the neBt offset to inspect R7
F. crete ll the possible brnches of node * 7R each branch has a edge to it
from *+ (ith corresponding *alue R7
?. for ech rule r in *.c
H. if r hs test rele*nt to *.p
4. if test for e"ulity
J. dd r into mtched rule set if r cn be mtched
fter this test& otherIise dd r into cndidte rule set of the
brnch Iith corresponding *lue
65. if test for ine"ulity
66. dd r into mtched rule set if r cn be mtched
fter this test& otherIise dd r into cndidte rule set of ll
brnches eNcept the brnch Iith corresponding *lue
6C. else 7R all the test in r are irrele*ant to *&p R7
6D. dd r into cndidte rule set of ll brnches
6E. for ech brnch *Q
6F. +uild)*Q- 7R recursi*ely call 8uild for */ R7
8
Figure -- Algorit6m for Automaton Construction
rule into e*ery ne(ly created node+ eBcept the one (ith the equal *alue& Same method
applies to the candidate rule set& !f the offset does not appear in any comparison of a rule+
that rule should be added into the candidate rule set for all the branches& The reason is that
you cannot rule out the possibility that the rule (ill be matched after this comparison&
5-
The easiest (ay to illustrate the algorithm is through an eBample& onsider the
follo(ing matching rules for our ASC pac'et filtering system&
"0: pac+et.p/ - .p.e3type 44 6786"3I!/ 99 .p.protocol D4I!37C!/
99 .p.protocol D4 I!3ED!/ 99 .p.protocol D4 I!3ICM!/
99 .p.protocol D4 I!3I?M!/
"1: pac+et.p/ - .p.e3type 44 6786"3I!/ 99 .p.protocol 44I!3ED!/
99 .p.udp3sport 44 6C8>3!>"7/ 99 .p.udp3dport 44 C8A"?6N3!>"7/
"F: pac+et.p/ - .p.e3type 44 6786"3I!/ 99 .p.protocol 44I!37C!/ 99 .p.tcp3flag 44 GN/
"H: pac+et.p/ - .p.e3type 44 6786"3I!/ 99 .p.protocol 44I!37C!/ 99 .p.tcp3flag 44 ACI/
Ae remo*ed the reaction part from each rule intending to focus only on the condition part+
(hich consists of the pac'et patterns (e need to match& Rule , is the eBact description of !%
)n'no(n %rotocol attac'+ (here all of the necessary type chec'ing related conditions ha*e
been added& A pre*iously mentioned denial of ser*ice intrusion :ARGE" and E:O is
mapped by rule 1& !t is one instance of this intrusion& Other instances can be obtained by
changing the source and destination )2% ports& The third and forth rule consist partly of the
intrusion pattern of SY" flooding+ in (hich+ a SY" pac'et or an A4 pac'et is matched
separately& The corresponding automaton for abo*e rules is constructed by applying the
procedure 8uild as sho(n in Figure 9&>&
As (e mentioned earlier+ se*eral strategies can be employed in implementing the
function elect+ (hich is used to determine the neBt offset in a pac'et to inspect&
S,+1+3+5T
ET:ER0!% p&e0typ
e
S,+1+3+5T
ST
S3+5T
S.T S0T ST

!%0T%
S1T
!%0)2%
!%0!M%
p&protocol
!%0!GM%

ST ST S$T SY"
A4

p&udp0spor
t
p&tcp0fla
g

E:O0%ORT
ST

:ARGE"0%ORT
p&udp0dpo
rt
ST S(T
S,+ (T ??
, U candidate rule+
( U matched rule&
S1T
9=
Figure -4 Sample Automaton for R$=R0
Select an offset such that the number of distinct *alues appearing at the node is
minimi2ed& Aith this criterion+ (e attempt to minimi6e the number of transitions
out of a state&
Select an offset such that the number of distinct *alues appearing at the node is
ma%imi2ed& The moti*ation here is that the candidate sets associated (ith each
transition is li'ely to be smaller&
Select an offset such that the number of rules that in*ol*e a test at this offset is
maBimi6ed&
Select an offset such that the number of duplicated rules after the current
comparison is minimi6ed& This strategy (ill reduce the possible redundant
comparisons performed in future&
!n the sample automaton+ (e adopt the third strategy+ (hich tries to in*ol*e as many as
possible rules in one offset chec'ing& :o(e*er+ this can be easily changed for an indi*idual
problem&
-- Rule #reprocessing
Some preprocessing needs to be done for an ASC rule+ before it can be used in
automaton constructing&
8.8.1 ,ule Deco#position
8efore any t(o rules can be processed into a single automaton+ (e need to filter out
each indi*idual comparison eBpression& Other(ise+ (e are not able to find the pac'et offset
chec'ed by more than one rule+ or e*en among different comparison eBpressions (ithin the
same rule& Ta'e the follo(ing t(o rules as an eBample$
"0: pac+et.p/ - .p.protocol 44I!37C!/ 99 .p.tcp3flag 44 GN/
"1: pac+et.p/ - .p.protocol 44I!37C!/ 99 .p.tcp3flag 44 ACI/
9,
E*ery pac'et offset eBamined in R, also appears in R1& And+ the con#unction among the
comparison eBpressions are all HMMI& So+ (e can decompose the conditions of R, and R1
into four sub?rules$
"00: .p.protocol 44I!37C!/ "01: .p.tcp3flag 44 GN/
"10: .p.protocol 44I!37C!/ "11: .p.tcp3flag 44 ACI/
!t is tri*ial to pro*e that R, can be satisfied if and only if both R,, and R,1 are satisfied&
Things (ill become a little bit complicated+ if the con#unction HPPI is used in the rule&
"e*ertheless+ (e can a*oid using HPPI in a rule condition by mo*ing HPPI out of a rule and
ma'ing a single rule become multiple rules at the highest le*el& So+ (ithin a single rule+ all
the comparison eBpression are connected by HMMI&
The follo(ing simple recursi*e algorithm can be employed to turn a single rule into
multiple sub?rules&
Procedure !ctor)c- 1
6. c is the condition inside rule to be fctored 7R in reality+ a rule is al(ays
represented by an eBpression tree R7
C. if )c is in the form of Rc6 MM cCS- 1
D. !ctor)c6-( 7R recursi*ely call Factor for c, R7
E. !ctor)cC-( 7R recursi*ely call Factor for c1 R7
F. 8 else if )c is in the form of ReNpr6 opertor eNprCS- 1 7R operator can be
VNN/+ VQN/+ etc& R7
?. crete neI subLrule for the rule being fctored
H. 8
8
8.8.2 Constraint Stac4 Construction
After the decomposition step is finished+ a rule becomes a collection of comparison
eBpressions& For eBpressions in*ol*ing a pac'et *ariable+ (e need to ta'e care of constraint
chec'ing for each field access&
As (e mentioned earlier in this chapter+ (hen a field access (ithin a pac'et is
accessed+ (e need to chec' the type of the *ariable+ such as an ether_hdr or an ip_hdr&
91
This can be done through recursi*e name loo'up in all child structures of the base structure
ether_hdr+ as long as each filed name is unique throughout the pac'et structure hierarchy&
After type chec'ing+ (e are able to locate the parent structure of the *ariable and e*en its
parent/s parent structure through the inheritance relationship among the pac'et structures&
Follo(ing the parent lin's+ (e can easily collect all the constraints for the *ariable and then
build them into a corresponding constraint stac'&
An algorithm in pseudo code is used to sho( the procedure for constraint stac'
construction&
Procedure Cons+uild)p& f- 1
6. p is pcGet *rible Iith correct type informtion( f is field
Iithin p
C. s = structof)p-( 7R structofDpE returns the corresponding structure type for p R7
D. for ech field fQ of s tht ppers before f 7R iterate in re*erse order R7
E. if fQ hs constrint
F. push the constrint into the stcG 7R add constraint for the
field in the same structure R7
?. if s hs constrint
H. push the constrint into the stcG 7R add constraint for the (hole
structure R7
4. pQ = prentof)p- 7R parentofDpE returns the parent structure type for pL if p is the
base structure+ returns null R7
J. fQ = lstfieldof)pQ- 7R lastfieldofDpE returns the last field inside a structure p R7
65. if )pQ ;= null-
66. Cons+uild)pQ.fQ-( 7R recursi*ely call ons8uild for p/f/ R7
8
Applying this algorithm to the eBpression Hp.tcp3flag 44 GNI+ (e can obtain the
constraint stac' sho(n as follo(s DFigure 9&;E&
p.e_type == ETHER_IP
p.protocol == IP_TCP
93
Figure -1 Sample Constraint Stack for Bp.tcp9fla -- S'"C
-4 Automaton Construction
)p to this point+ all the pac'et?matching rules ha*e been con*erted into lists of
eBpression stac's (ith necessary constraints inside the stac'& "eBt step is to build the
automaton (ith the algorithm (e de*eloped at the Section 9&5&
8.:.1 ;ffset Selection
!n the process of automaton construction+ choosing the right *ariable or offset of a
pac'et *ariable to be tested at a comparison step is crucial in reducing the automaton si6e or
matching time for a pac'et pattern& The function elect is created for this purpose& !n the
implementation of ASC?%FM+ (e support all the heuristics mentioned in Section 9&5&
One heuristic (e al(ays use is to minimi6e the number of duplicated rules after a
comparison+ (hile preferring fields that lead to creation of small number of branches& The
obser*ation of this heuristic is that it can significantly reduce the si6e of the created
automaton& A rule (ill be duplicated into all the branches after a comparison node+ if the
*ariable or offset that is chec'ed in the comparison node is irrele*ant to the rule& Ae then
define the duplicate indeB of a *ariable as the product of the number of rules not in*ol*ed in
eBamining it and the number of branches of the *ariable& For instance+ (e ha*e 9 rules in
total+ (hile 3 of (hich do not chec' the *ariable % and the left 1 rules ha*e the eBpression
%440 and %441 separately& Then (e can calculate the duplicate indeB of B as$
,I)N- = D @ D = J
!n the elect function+ (e compute the duplicate indeB for each *ariable and choose the
*ariable (ith the minimal *alue& !n case t(o *ariables ha*e the same duplicate indeB+ (e
(ill select the one (ith less number of branches& See the follo(ing three rules D(e ha*e
omitted the reaction partE$
"0: .% 44 0/ 99 .y 44 0/
p.tcp_la! == "#$
95
"1: .% 44 1/ 99 .y 44 0/
"F: .% 44 F/ 99 .y 44 0/
8oth *ariable % and y ha*e the same duplicate indeB =& Ae can build the automaton in either
(ay sho(n in Figure 9&.& :o(e*er+ it is ob*ious that choosing *ariable y first can reduce the
si6e of target automaton&
C6oose y first; t6en A
Figure -@ Sample Automaton
Most operations to a net(or' pac'et are equality and inequality comparisons+ (hich
can be handled *ery (ell by the elect function& For the comparisons li'e W+ WN+ X+ or XN+ (e
can consider the (hole eBpression as a single boolean *ariable (ith t(o branches , and =+
representing true and false separately& Then (e can apply the elect function in the same
(ay as to the *ariable (ith equality and inequality comparisons& :o(e*er+ there is another
'ind of operation (ith a pac'et *ariable+ bit?(ise operation+ especially bit?and& A typical
situation happens (hen (e (ant to eBamine the net(or' address of a pac'et+ li'e in
p.d_ddrM5N!!!!!!55 == 5N?FEDC655& !n the eBpression+ the heBadecimal *alue
5N!!!!!!55 is the net(or' mas' for an !% address+ assuming (e are in a class net(or'&
y y
,
3 1
y
B

,
, , ,
y
B
, y
,
3 , 3

1 1
B
C6oose A first; t6en y
99
The bit?and operation brings some difficulty (hen (e apply the elect function& Suppose (e
ha*e follo(ing t(o rules$ one has an eBpression p.s_ddr == 5N4H?FEDC6+ (hile the other
one includes an eBpression p.s_ddrM5N!!!!!!55 == 5N4H?FED55&
A solution for this problem is to further decompose the eBpressions& !n the abo*e
case+ (e can con*ert the eBpression p.s_ddr == 5N4H?FEDC6 into t(o sub?eBpressions as
follo(s$
Dp.s_ddrMoN!!!!!!55==5N4H?FED55-MM)p.s_ddrM5N555555!!==5N555555C6-
After doing this+ (e can apply the elect function again and obtain the common offset
s_ddrMoN!!!!!!55&
8.:.2 Su$5Auto#aton Sharin
)nder certain circumstance+ some redundancy may eBist among different sub?
automata in a large automaton& onsider the follo(ing automaton built for the t(o simple
rules listed belo( DFigure 9&-E$
"0: pac+et.p/ - p.icmp3type 44 6C8>3"6J
"1: pac+et.p/ - p.s3addr 44 K%)LMNHF10
Figure -E 'noptimi8ed Automaton
S,+1T
S$T
S(T
S1T
S,+1T
S,+1T ST
S$+(T ST S(T
ET:ER0!%

p&e0typ
e
!%0!M%

E:O0REY
=B.;>9531,
S$+1T
p&s0add
r
S1T

=B.;>9531,
p&icmp0typ
e
p&protoco
l
p&s0add
r
p&s0add
r
ST

=B.;>9531,
! !!
!!!
9>
You may ha*e noticed that the same automaton after the testing for field s_ddr
occurred at different place of the automaton& !n a larger automaton+ this duplication may be
e*en (orse& The reason is that the Cuild procedure is eBecuted at different node recursi*ely&
Each run is independent and is not a(are that the same automaton has been established
some(here else&
One (ay to sol*e this problem is that (hene*er a procedure needs to create a ne(
node+ let it first loo's up all the eBisting nodes in order to a*oid duplicate& :o(e*er+ this
loo'up is not tri*ial& The comparison of the same offset of a pac'et *ariable is not strong
enough in deciding (hether t(o nodes are identical& As sho(n in the Figure 5&-+ although
the node ! has the same offset compared as that of node !! and !!!+ it cannot be substituted by
either node !! or !!!+ for it has different matching rule set and candidate rule set& A sufficient
condition for t(o nodes to be equi*alent is both of them ha*e the same comparing *ariables+
matching rule sets and candidate rule sets& !n addition+ for a rule in both nodes/ candidate
sets+ it must ha*e the same tests remaining for each rule& All of these ma'e the eBact input
for our Cuild procedure and elect function& Therefore+ the same input guarantees the node
to be created (ill be same as (ell& An optimi6ed *ersion of automaton is sho(n as follo(s
DFigure 9&,=E&
Figure -$F Optimi8ed Automaton
S,+1T
S$T S(T
S1T
S,+1T
S,+1T ST
S$+(T
ET:ER0!%

p&e0typ
e
!%0!M%

E:O0REY
=B.;>9531,
S$+1T
p&s0add
r

p&icmp0typ
e
p&protoco
l
p&s0add
r
ST
=B.;>9531,
9;
-1 Code /eneration
The ultimate result from our ASC?%FM is an eBecutable code snippet+ (hich can be
used in de*eloping the runtime pac'et filter&
Ahat a pac'et filter does is almost the same as an automaton chec' through an input
stream& A portion of a pac'et (ill be compared (ith some *alue at a node and the automaton
(ill then mo*e to the branch (ith matched *alue label& Ahen a left node is reached+ no
further comparison (ill be made& !nstead+ the reaction specified in the rule definition (ill be
triggered for all matched rules& The code generated by ASC?%FM should eBactly reproduce
this process& Ta'e the follo(ing t(o rules as an eBample+ (e (ill see (hat the generated
code loo's li'e&
EBample rules$
"0: pac+et.p/ - .p.e3type44K%)KK/ 99 .p.protocol40/ 99 .p.icmp3type44)/
message.#ICM! 6C8> "e&uest'/5
"1: pac+et.p/ - .p.e3type44K%)KK/ 99 .p.protocol40/ 99 .p.icmp3type44K/
message.#ICM! 6C8> "eply'/5
EBample ode generated by ASC?%FM$
*oid smple_code)const u_chr @p- 1
if ))@))unsigned short @-)p76C--- == 4- 1
if ))@))unsigned chr @-)p7CD--- == 6- 1
if ))@))unsigned chr @-)p7DE--- == 4- 1
messge)%ICTP ECAO Re"uest%-(
8
else if ))@))unsigned chr @-)p7DE--- == 5- 1
messge)%ICTP ECAO Reply%-(
8
else 1
8
8
else 1
8
8
else 1
8
9.
8
There are a number of issues need to be mentioned around code generation& First of
all+ an incoming pac'et is treated as a byte stream& This is a natural design and can be easily
implemented through *arious pac'et capture facilities& The type information of a pac'et field
is enforced through proper pointer casting as sho(n in the sample code& One thing needs to
be ta'en into consideration for a pac'et field larger than . bits is byte order& "ormally+ (e
need to con*ert the filed in a pac'et from net(or' order into host order before it can be
compared (ith any *alue specified in the rule definition& :o(e*er+ in order to sa*e the
precious time at runtime+ (e should ma'e the re*erse con*ersion on the predefined *alues&
!n other (ords+ (e can change those *alues from host order into net(or' order at compile
time&
An important issue that may affect the runtime performance of the future pac'et filter
is the (ay (e handle the comparison and do the branching& !n the abo*e sample code+ (e
simply used if?else statement pro*ided in & !t may (or' fine for the *ariable+ (hich has the
number of branches less than 3& !n case there are a large number of branches for a *ariable+
this scheme may greatly degrade the efficiency for a runtime pac'et filter& An alternati*e is
binary search+ in (hich the branches of a *ariable need to be sorted before code generation&
!n our implementation+ (e adopt binary search for each node (ith the number of branches
more than 3& At runtime+ this 'ind of node can be processed in log1
"
time+ (here " is the
number of branches& A further impro*ement can be achie*ed by using #ump?table or hash?
table if the number of branches become *ery large&
-@ &ata /eneration
Static pac'et data can be *ery helpful in the design phase for a real?time pac'et?
filtering system& !t can be used in testing and performance analysis& An ideal data source
should meet the follo(ing requirements&
Interpretable by other systems& !f the generated data can be fed into other pac'et?
filtering system+ li'e tcpdump+ (e can then easily compare our implementation
(ith others&
9-
6asy to specify& On one hand+ a user is able to setup e*ery field inside a pac'et&
On the other hand+ data integrity must be guaranteed+ i&e& there should be no
conflict among dependent fields and all of the required fields must be filled in
ad*ance automatically& For instance+ (hene*er a )2% pac'et is specified+ the
e_type field of ether_hdr must be set to ETAER_IP as default&
Controllable distribution& Ahen a large amount of data needs to be produced+ (e
may (ant a statistically controlled distribution of *arious pac'et 'inds+ li'e 91Z
T% pac'ets+ 13Z)2% pac'ets+ as (ell as 19Z !M% and !GM% pac'ets&
The first requirement can be easily achie*ed (ith the help of libpcp& After some
modification of libpcp A%!+ (e are able to produce pac'et data file (ith the compatible
format as the file used by tcpdump&
!n terms of field dependence and type chec'ing+ the constraint stac' concept in our
ASC language design and pointer type casting in the code generation can be a potential
solution& As (e ha*e described in hapter 3+ (hene*er an automaton is constructed+ all the
required fields are chec'ed before a pac'et field is accessed& Therefore+ (e propose to
con*ert each reading operation on a pac'et field into (riting a certain *alue on a pseudo
pac'et+ for the purpose of data generation&
8efore *arious pac'ets can be generated+ a user should ha*e a con*enient (ay in
specifying the distribution percentage for each pac'et 'ind& Ae then eBtend our ASC
language specification intending to pro*ide a (eight field for each rule& Follo(ing are some
eBamples$
"0: .% 44 0/ O (K.H*
"1: .% 44 1/ O (K.M*
"F: .y 44 0/ O (K.1*
"H: .y 44 1/ O (K.)*
8ecause (e are focusing on the data generation+ both pattern and reaction part ha*e been
omitted from rule definition& The number at the end of each rule is the (eight field+ (hich is
the eBpected percentage for this pattern in the final data generation& There is a little
difference (hen a rule is specified for the purpose of pac'et generation+ i&e& all the fields
>=
occurring in the rule must be eBplicitly specified& !t means if you ha*e a rule
p.s_ddr==p.d_ddr+ you ha*e to change it to )p.s_ddr==5N4H?FEDC6- MM
)p.d_ddr==5N4H?FEDC6-& The reason for doing that is to facilitate the pac'et generation&
!n order to achie*e the distribution specified in the rule definition+ (e need to build
the (eight into the nodes of the automaton and apply it (hen the code is generated& The (ay
(e calculate the (eight is as follo(s$
,or a left node$ the (eight is computed by multiplying all the (eight specified by
the matched rules& This is simply based on the obser*ation that the possibility of
matching t(o rules is the product of the possibility of matching each single rule&
,or an internal node$ the (eight is the sum of all the (eight of its child nodes&
Figure 9&,, sho(s the (eighted automaton built for the pre*ious eBample rules&
Figure -$$ *eig6ted Automaton
B
y
,
, 1 , 1
1
,&= N =&5O=&>
6
=&5N=&=.R=&31 =&>N=&,1O=&5.
=&5.N=&>R=&. =&,1N=&>R=&1 =&31N=&5R=&. =&=.N=&5R=&1
>,
C"A#TER 4 E5#ER%3ENTAL RES'LTS AN& CONCL'S%ON
!n this thesis+ (e presented a con*enient+ eBpressi*e and safe language for describing
intrusion patterns and reactions& The primary contribution of this language to a pac'et filter
construction is independence from the communication protocol and type safety in pac'et
manipulation operations& Ae de*eloped an algorithm and implementation for pac'et filtering
that is based on adapti*e pattern matching technique+ (hich is important for impro*ing the
pac'et filter performance and scalability&
4$ %ntrusion &etection 'sing ASL
At the end of chapter 5+ (e once listed some sample intrusion patterns (ritten in
ASC& !n the real system+ (e eBtended those eBamples and created a number of ne( patterns
for the attac's such as Smurf+ Cand+ and so on& More eBamples can be found in AppendiB 8&
@P murf .Intermediate ite/ P@
pac+et.p/ - .p.icmp3type 44 ICM!36C8>37G!63"6JE67/
99 .p.d3addr 44 B>CAB3C">ADCA73ADD"/
message.#murf'/5
@P Band Attac+ P@
pac+et.p/ - .p.s3addr 44 p.d3addr/
message.#Band attac+'/5
Our pac'et filter can fetch a pac'et either from a tcpdump file or directly from real
net(or' traffic& Ae also performed testing on some data files generated by our ASC0%FM
itself& This greatly facilitates our testing (or'+ for some attac's are not easy to be simulated
through attac'ing program or by capturing real net(or' traffic& The final result sho(s that
our pac'et filter can efficiently detect DpotentialE net(or' intrusions&
>1
4( #reliminary #erformance Testing
%erformance is a 'ey criterion in e*aluating a real?time net(or' pac'et filter& !n this
section+ (e did some preliminary performance study through a simple utility program+ (hich
is used to collect the number of pac'ets arri*ed for each net(or' ser*ice& Ae compared our
runtime pac'et filter and 8%F pac'et filter for this eBample&
:.2.1 S,<STAT& Ser/ice Statistics
4no(ing the number of pac'ets arri*ed for each net(or' ser*ice+ e&g& TEC"ET+
SMT%+ or :TT%+ can be helpful for the net(or' management purpose+ li'e chec'ing the
traffic load for each machine in the net(or'& !n addition+ it may also useful in detecting
some malicious net(or' beha*iors& For eBample+ if a large number of connection?request
pac'ets are detected at login ser*ice port in a short period of time+ it is possible that some
hac'er is trying to brea' into your system through pass(ord guessing& Follo(ing are some
code snippets in the ASC rule definition for SR<STAT$
pac+et.p/ - -; p+t3total ::5
pac+et.p/ - .p.icmp3type 44 ICM!36C8>37G!63"6JE67/
-; icmp3echo3re&uest ::5
pac+et.p/ - .p.tcp3dport 44 !>"73877!/
-; tcp3http ::5
pac+et.p/ - .p.udp3dport 44 !>"73N,/
-; udp3nfs ::5
O
The idea is straightfor(ard& Ahene*er a pac'et to(ards a ser*ice is captured+ a
corresponding counter for that ser*ice (ill be incremented& !n addition+ a counter for all the
recei*ed pac'ets (ill be incremented as (ell+ no matter the incoming pac'et is an !M%
request or a message to the "FS ser*er& )sually+ pac'et filtering is ser*ice specific& This
means a pac'et filter can be used to collect ho( many login attempts ha*e been recei*ed
from host foo or ho( many times the finger ser*ice is required& !n SR<STAT+ (e #ust count
the number of pac'ets recei*ed by each ser*ice&
An obser*ation from this utility program is that there may be a large number of
patterns for a pac'et filter+ (hile the number of pac'et fields co*ered by these patterns are
>3
quite small& For instance+ SR<STAT can ha*e o*er one hundred patterns specified if (e
(ant to monitor all of normal )"!@ net(or' ser*ices& :o(e*er+ these patterns may only
access a small number of fields in a pac'et+ such as tcp_dport and udp_dport& !n this case+
the processing time of our pac'et filter (ill not change too much+ because it largely
dependent on the number of pac'et fields accessed instead of the number of pac'et patterns&
:.2.2 Perfor#ance Co#parison& AS6 /s. 7P0
Ae also performed some preliminary performance testing on our ASC pac'et filter
and 8%F pac'et filter& Our focus in this thesis is mainly on the filter model+ especially (hen
the number of pac'et patterns becomes relati*ely large& Therefore+ (e do not consider the
issues+ li'e 'ernel buffering or 'ernel filtering+ (hich are implementation techniques that can
be applied to all filter models&
!n our test for the filter performance+ (e compared our runtime ASC filter (ith 8%F
under the conditions+ (here different numbers of patterns are specified for a filter& To
eliminate the random factors introduced by the li*e net(or' traffic+ (e used static tcpdump
files as the data source for both filters&
First+ (e tested our ASC filter (ith different branching mechanism used& Figure >&,
sho(s that the processing time for our filter+ (hen sequential comparison is used increases
significantly as more patterns are specified for the filter& This is to be eBpected as the number
of comparison operations introduced by sequential comparison increases linearly (ith the
number of patterns& 8inary search can impro*e the filter performance& An efficient hashing
function can impro*e the filter performance e*en more (hen the number of branches are
moderate D5 or moreE&
!n the neBt test+ (e run both of our ASC filter and 8%F o*er the same set of pac'ets&
Figure >&1 displays the final results& !n terms of absolute performance+ 8%F is mush slo(er+
partly because the 8%F filter eBecutes interpreter code *ersus the compiled code in our
approach& !n addition+ (e found that the processing time for our ASC filter does not change
too much as the number of patterns increased&
>5
Figure 4$ Branc6ing 3ec6anism Comparison for ASL Filter
Figure 4( #erformance Comparison
4. Conclusion
The methods used by net(or' intrusions can be different from one to another&
:o(e*er+ the nature of most net(or' intrusions is based on HmaliciousI net(or' traffic+
ASL vs. BPF
0
0.5
1
1.5
2
2.5
3
3.5
4
1 4 8 16 32
Number of Patterns
P
r
o
c
e
s
s
i
n
g

T
i
m
e

f
o
r

1

P
a
c
k
e
t

(
M
i
c
r
o
s
e
c
o
n
d
)
BPF
ASL
Brancing Mecanism !om"arison
0.000
0.200
0.400
0.600
0.800
1.000
1.200
1 4 8 16 32
Number of Patterns
P
r
o
c
e
s
s
i
n
g

T
i
m
e

f
o
r

1

P
a
c
k
e
t

(
M
i
c
r
o
s
e
c
o
n
d
)
Sequential
Comparison
Binary Sear!
"as!in# $a%le
>9
(hich either has in*alid *alue inside a field of a pac'et+ or features incorrect combination or
sequence of pac'et segments& Aith this obser*ation+ (e can use pac'et?filtering technique in
building net(or' intrusion detection systems&
:o(e*er+ pac'et filter faces ne( challenges for the intrusion detection purpose+ li'e
high?*olume data+ no pac'et dropping+ and requirement for system fleBibility and scalability&
!n this thesis+ (e borro(ed some idea from the adapti*e pattern matching technique and
applied it to our pac'et?filtering module for a large?scale intrusion detection frame(or'&
One of the 'ey components in our approach is a specification language ASC+ (hich is
used to describe the patterns for the pac'ets to be captured& ASC pro*ides a number of
features in facilitating pattern (riting and filter construction+ such as pac'et structure
description and automatic pac'et type chec'ing&
8uilding an efficient filter is crucial to the o*erall system performance& Ae presented
an elegant algorithm in filter construction& The main concern is ho( to select a *ariable or an
offset inside a pac'et to be inspected at a node of a filter automaton& The primary goal is to
minimi6e pattern matching time and the si6e of the automaton&
ASC?%FM pro*ides a no*el and elegant (ay in pac'et filter design and
implementation for net(or' intrusion detection systems& :o(e*er+ there are still many
topics need to be in*estigated in the field of pac'et filtering and net(or' intrusion detection+
such as in 'ernel integration of pac'et filtering module&
>>
A##EN&%5 A #AC,ET &ATA STR'CT'RES FOR ASL
Ethernet Aeder/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
<define ETAER_9EB ?
struct ether_hdr
1
byte e_dst$ETAER_9EB'(
byte e_src$ETAER_9EB'(
short e_type(
8
ARP/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
<define ETAER_IP 5N5455
<define ETAER_ARP 5N545?
struct rp_hdr / struct ether_hdr Iith e_type == ETAER_ARP
1
short r_hrd( =@ !ormt of hrdIre ddress @=
short r_pro( =@ !ormt of protocol ddress @=
byte r_hln( =@ 9ength of hrdIre ddress @=
byte r_pln( =@ 9ength of protocol ddress @=
short r_op( =@ ARP opcode )commnd-. @=
8
=@ ARP protocol AAR,>ARE identifiers @=
<define ARPAR,_ETAER 6 =@ Ethernet 65Tbps @=
=@ ARP protocol PROTOCO9 identifiers @=
<define ARPPRO_IP 5N5455 =@ IP @=
=@ ARP protocol opcodes @=
<define ARPOP_REUOEST 6 =@ ARP re"uest @=
<define ARPOP_REP9V C =@ ARP reply @=
<define ARPOP_RREUOEST D =@ RARP re"uest @=
<define ARPOP_RREP9V E =@ RARP reply @=
struct ether_ip_rp / struct rp_hdr Iith
)r_hrd == ARPAR,_ETAER- MM )r_pro == ARPPRO_IP-
1
byte rp_sh$ETAER_9EB'(=@ sender hrdIre ddress @=
int rp_sp( =@ sender protocol ddress @=
byte rp_th$ETAER_9EB'(=@ trget hrdIre ddress @=
int rp_tp( =@ trget protocol ddress @=
8
IP/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
struct ip_hdr / struct ether_hdr Iith e_type == ETAER_IP MM ihl == F
1
bit *ersion$E'( =@ ip *ersion @=
>;
bit ihl$E'( =@ heder length @=
byte tos( =@ type of ser*ice @=
short tot_len( =@ totl length @=
short id( =@ identifiction @=
bit flg$D'( =@ flgs @=
bit frg_off$6D'( =@ frgment offset @=
byte ttl( =@ time to li*e @=
byte protocol( =@ protocol @=
short checG_sum( =@ heder checGsum @=
ip_ddr s_ddr( =@ source ip ddress @=
ip_ddr d_ddr( =@ destintion ddress @=
8
struct ip_pGt / struct ip_hdr
1
byte ip_dt$tot_len L ihl'(
8
ICTP/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
=@ IP protocol PROTOCO9 identifiers. @=
<define IP_ICTP 5N5556 =@ ICTP @=
<define IP_IWTP 5N555C =@ IWTP @=
<define IP_TCP 5N555? =@ TCP @=
<define IP_O,P 5N5566 =@ O,P @=
struct icmp_hdr / struct ip_hdr Iith protocol == IP_ICTP
1
byte icmp_type( =@ icmp messge type @=
byte icmp_code( =@ icmp messge code @=
short icmp_csum( =@ checGsum for entire messge @=
8
struct icmp_pGt / struct icmp_hdr
1
byte icmp_dt$tot_len L ihl L siPeof)icmp_hdr-'(
8
<define ICTP_ECAO_TVPE_REUOEST 4
<define ICTP_ECAO_TVPE_REP9V 5
<define ICTP_ECAO_CO,E 5
struct icmp_echo_re"uest / struct icmp_hdr Iith
)icmp_type == ICTP_ECAO_TVPE_REUOEST-
MM )icmp_code == ICTP_ECAO_CO,E-
1
byte icmp_echoid( =@ identifier @=
byte icmp_echose"( =@ se"uence number @=
byte icmp_echodt$tot_len L ihl L siPeof)icmp_hdr- L C'(
8
struct icmp_echo_reply / struct icmp_hdr Iith
)icmp_type == ICTP_ECAO_TVPE_REP9V- MM )icm_code == ICTP_ECAO_CO,E-
1
>.
byte icmp_echoid( =@ identifier @=
byte icmp_echose"( =@ se"uence number @=
byte icmp_echodt$tot_len L ihl L siPeof)icmp_hdr- L C'(
8
<define ICTP_,ESOBREA_TVPE D
struct icmp_unrech / struct icmp_hdr Iith
icmp_type == ICTP_,ESOBREA_TVPE
1
short icmp_reser*ed(
ip_hdr icmp_iphdr(
byte icmp_dt$4'(
8
<define ICTP_SRCUOEB_TVPE E
<define ICTP_SRCUOEB_CO,E 5
struct icmp_s"uench / struct icmp_hdr Iith
icmp_type == ICTP_SRCUOEB_TVPE
1
short icmp_reser*ed(
ip_hdr icmp_iphdr(
byte icmp_dt$4'(
8
O,P/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
struct udp_hdr / struct ip_hdr Iith protocol == IP_O,P
1
byte udp_sport( =@ source port number @=
byte udp_dport( =@ destintion port number @=
byte udp_len( =@ heder 7 dt length @=
byte udp_csum( =@ checGsum for heder M dt @=
8
struct udp_pGt / struct udp_hdr
1
byte udp_dt$udp_len L siPeof)udp_hdr-'(=@ dt @=
8
TCP/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
struct tcp_hdr / struct ip_hdr Iith protocol == IP_TCP
1
short tcp_sport( =@ source port number @=
short tcp_dport( =@ destintion port number @=
int tcp_se"( =@ se"uence number @=
int tcp_cG( =@ cGnoIledge number @=
bit tcp_hlen$E'( =@ heder length @=
bit tcp_reser*ed$?'( =@ reser*ed @=
bit tcp_urg( =@ flgs @=
bit tcp_cG(
bit tcp_psh(
bit tcp_rst(
>-
bit tcp_syn(
bit tcp_fin(
short tcp_Iin( =@ IindoI siPe @=
short tcp_csum( =@ checGsum for heder M dt @=
short tcp_urp( =@ urgent pointer @=
8
struct tcp_pGt / struct tcp_hdr
1
byte tcp_dt$tot_len L ihl L tcphlen'(
8
,BS/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
<define ,BS_PORT FD
struct dns_hdr / struct udp_hdr Iith
)udp_sport == ,BS_PORT- XX )udp_dport == ,BS_PORT-
=@ either to dns port or from dns port @=
1
short dns_id( =@ identifier @=
short dns_flgs( =@ flgs @=
short dns_n"ues( =@ Bo. of "uestions @=
short dns_nns( =@ Bo. of nsIers RR @=
short dns_nuth( =@ Bo. of uthority RRs @=
short dns_ndd( =@ Bo. of dditionl RRs @=
8
struct dns_"ues
1
string dns_"nme( =@ "uery nme @=
short dns_"type( =@ "uery type @=
short dns_"clss( =@ "uery clss @=
8
struct dns_rr_hdr
1
string dnme( =@ domin nme @=
short type( =@ RR type @=
short clss( =@ RR clss @=
int ttl( =@ time to li*e @=
8
<define ,BS_UOERV_A 6
struct dns_rr_A / struct dns_rr_hdr rrhdr Iith rrhdr.type == ,BS_UOERV_A
1
short rdlen( =@ resource dt length @=
ip_ddr rdt$rdlen'(
8
struct dns_pGt_A / struct dns_hdr dnshdr
1
struct dns_"ues d"ues$n"ues'( =@ dns "uestions @=
struct dns_rr_A dns$nns'( =@ dns nsIer RRs @=
;=
struct dns_rr_A duth$nuth'( =@ dns uthority RRs @=
struct dns_rr_A ddd$ndd'( =@ dns dditionl RRs @=
8
RIP/
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
<define RIP_PORT FC5
struct rip_hdr / struct udp_hdr Iith
)udp_sport == RIP_PORT- XX )udp_dport == RIP_PORT-
=@ either to rip port or from rip port @=
1
byte rip_commnd( =@ rip commnd @=
byte rip_*ersion( =@ rip *ersion @=
short rip_Pero( =@ must be Pero @=
8
struct rip_rec
1
short rip_fid( =@ ddress fmily identifier @=
short rip_Pero( =@ must be Pero @=
int rip_ipddr( =@ ip ddress @=
int rip_Pero$C'( =@ must be Pero @=
int rip_metric( =@ metric @=
8
strcut rip_pGt / struct rip_hdr
1
rip_rec riprec$)udp_len L siPeof)struct rip_hdr--
= siPeof)struct rip_rec-'(
8
;,
A##EN&%5 B %NTR'S%ON #ATTERN SA3#LES
,& !% )n'no(n %rotocol
Todule IPOnGnoInProtocol)- 1
pcGet)p-X)p.protocol ;= IP_TCP- MM )p.protocol ;= IP_O,P-
MM )p.protocol ;= IP_ICTP- MM )p.protocol ;= IP_IWTP-
messge)RIP OnGnoIn ProtocolS-(
8
1& %ing of 2eath
Todule Pingof,eth)- 1
pcGet)p-X)p.tot_len 7 p.frg_off 3 TAY_IP_SIZE-
messge)RPing of ,ethS-(
8
3& hargen+ Echo+ Time+ 2aytime
Todule CET,)sPort& dPort- 1
pcGet)p-X)p.udp_sport == sPort- MM )p.udp_dport == dPort-
messge)RCET, AttcGS-(
8
5& Finger 8uffer O*erflo(
<define !IBWER,_PORT HJ
Todule !ingerO*erfloI)- 1
pcGet)p-X)p.tcp_dport == !IBWER,_PORT-
MM )p.tot_len [ p.ihl [ p.tcp_hlen- 3 TAY_!IBWER_9EB-
messge)R!inger +uffer O*erfloIS-(
8
9& CA"2 Attac'
Todule 9AB,)- 1
pcGet)p-X)p.s_ddr == p.d_ddr- messge)R9AB, AttcGS-(
8
;1
>& Smurf D!ntermediate SiteE
Todule Smurf)- 1
pcGet)p-X)p.icmp_type == ICTP_ECAO_TVPE_REUOEST-
MM )p.d_ddr == 9OCA9_BET_+ROA,CAST_A,,R-
messge)RSmurf AttcGS-(
8
;& !% Spoofing
=@ for eNternl interfce @=
Todule IPSENternl)- 1
pcGet)p-X)p.s_ddrMIBSI,E_BET_TASK == IBSI,E_BET_A,,R-
messge)RIPS ENternlS-(
8
=@ for internl interfce @=
Todule IPSInternl)- 1
pcGet)p-X)p.s_ddrMIBSI,E_BET_TASK ;= IBSI,E_BET_A,,R-
messge)RIPS InternlS-(
8
.& SY" Flooding
Todule SVB!looding)- 1
pcGet)p-X)p.tcp_flg == SVB-
nSyn = nSyn 7 6(
e*ery predefined inter*l& checG
if )nSyn 3 SVB_TARESAO9,-
messge)RSVB !loodingS-(
else
nSyn = 5(
pcGet)p-X)p.tcp_flg == ACK-
if )nSyn 3 5-
nSyn = nSyn [ 6(
8
;3
-& %ing Flooding
Todule Ping!looding)- 1
pcGet)p-X)p.icmp_type == ICTP_ECAO_TVPE_REUOEST-
nPing = nPing 7 6(
e*ery predefined inter*l& checG
if )nPing 3 PIBW_TARESAO9,-
messge)RPing !loodingS-(
else
nPing = 5(
8
,=& R!% Trace
<define D
Todule RIPTrce)- 1
pcGet)p-X)p.rip_commnd == RIP_TRACE_COTTAB,-
messge)RRIP Trce Commnd OnS-(
8
;5
REFERENCES
F,G Carry J& :ughes+ Jr& Actually )seful !nternet Security Techniques+ "e( Riders
%ublishing+ !ndianapolis+ !"+ ,--9&
F1G R& :eady+ G& Cuger+ A& Maccabe+ and 8& Mu'her#ee& A Method To 2etect !ntrusi*e
Acti*ity in a "et(or'ed En*ironment& !n !roceedings of the 0H
th
National Computer
ecurity Conference+ pages 3>1?3;,+ October ,--,&
F3G Abdela6i6 Monn#i& Canguages and Tools for Rule?8ased 2istributed !ntrusion
2etection+ %h2 thesis+ Facultes )ni*ersitaires+ "otre?2ame de la %aiB+ 8elgium+
September ,--;&
F5G A& R& Ste*ens& T%7!% !llustrated <ol& , U The %rotocols+ Addison?Aesley %ublishing
ompany+ !nc& Reading+ MA+ ,--5&
F9G S& M& 8ello*in& Security %roblems in the T%7!% %rotocol Suite, Computer
Communications "evie$+ <ol& ,-+ "o& 1+ pp& 31?5.+ April ,-.-&
F>G Morris R& A Aea'ness in the 5&1 8S2 )"!@ T%7!% Soft(are, Computer cience
7echnical "eport No 00L+ ATMT 8ell Caboratories+ Murray :ill+ "J+ ,-.9&
F;G ERT& T% SY" Flooding and !% Spoofing Attac's+ arnegie Mellon )ni*ersity+
%ittsburgh+ %A+ September ,-->&
F.G & obb and S& obb& 2enial of Ser*ice+ ecure Computing+ pp&9.?>=+ July ,--;&
F-G & C& Schuba+ !&<& 4rsul+ Ma'us G& 4uhn+ E&:& Spafford+ A& Sundaram+ 2& [amboni&
Analysis of a 2enial of Ser*ice Attac' on T%+ %urdue )ni*ersity+ Aest Cafayette+
!"+ ,-->&
F,=G S& 2ash& !ntegration of 2"SSE D'ey?ser*erE (ith Ssh Application+ MS thesis+ !o(a
State )ni*ersity+ Ames+ !A+ ,--;&
F,,G A& R& Ste*ens& )"!@ "et(or' %rogramming <ol& , U "et(or' A%!s$ Soc'ets and
@T!+ Second Edition+ %rentice :all %TR+ )pper Saddle Ri*er+ "J+ ,--.&
F,1G <ern %aBson& 8ro$ A System for 2etecting "et(or' !ntruders in Real?Time+ Ca(rence
8er'eley "ational Caboratory+ 8er'eley+ A+ ,--.&
;9
F,3G R& & Se'ar+ R& Ramesh+ !& <& Rama'rishnan& Adapti*e %attern Matching+ 8ellcore+
Morristo(n+ "J+ ,--3&
F,5G Ste*en Mcanne+ <an Jacobson& The 8S2 %ac'et Filter$ A "e( Architecture for
)ser?le*el %ac'et apture+ Ca(rence 8er'eley Caboratory+ 8er'eley+ A+ ,--1&
F,9G 8is(anath Mu'her#ee+ C& Todd :eberlein+ 4arl "& Ce*itt& "et(or' !ntrusion
2etection+ I666 Net$or++ pp&1>?5,+ May7June ,--5&
F,>G Frederic' 8& ohen& A "ode on 2istributed oordinated Attac's+ omputer M
Security+ pp&,=3?,1,+ *,9+ ,-->&
F,;G Ste*en heung+ 4arl "& Ce*itt& %rotecting Routing !nfrastructures from 2enial of
Ser*ice )sing ooperati*e !ntrusion 2etection+ )ni*ersity of alifornia+ 2a*is+ A+
,--;&
F,.G hristoph C& Schuba& Addressing Aea'ness in the 2omain "ame System %rotocol+
OAST Caboratory+ %urdue )ni*ersity+ Aest Cafayette+ !"+ ,--3&
F,-G Eugene :& Spafford& The !nternet Aorm !ncident+ 7echnical "eport CD-7"-QQF+
%urdue )ni*ersity+ Aest Cafayette+ !"+ September ,-+ ,--,&
;>
AC,NO*LE&/E3ENTS
! (ish to eBpress my sincere appreciation to 2r& R& & Se'ar+ my ma#or professor+ for
his full support+ *aluable ad*ice and assistance to carry out and complete this research&
! (ould li'e to than' 2r& Johnny Aong for his help in my graduate study& :is
encouragement and support ma'es my t(o years study at !o(a State )ni*ersity the most
re(arding time in my life&
The contribution of 2r& 2oug Jacobson as committee member is greatly
ac'no(ledged&
Many than's to %remchand )ppuluri+ Ra*i <an'amamidi and Yong ai for their help
to my research and study&
Finally+ than's for the lo*e from my parents and my (ife+ Ci Fang& Aithout their
support+ ! (ould not ha*e completed this research&
This pro#ect is supported by 2efense Ad*anced Research %ro#ect Agency\s
!nformation Technology Office D2AR%A?!TOE under the !nformation System Sur*i*ability
program+ under contract number F3=>=1?-;??=155&
;;