USF Libraries
USF Digital Collections

Grouper :

MISSING IMAGE

Material Information

Title:
Grouper : a packet classification algorithm allowing time-space tradeoffs
Physical Description:
Book
Language:
English
Creator:
Kuhn, Joshua Adam
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Algorithms
Computer Networks
Network Algorithms
Network Security
Security
Dissertations, Academic -- Computer Science -- Masters -- USF   ( lcsh )
Genre:
bibliography   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: This thesis presents an algorithm for classifying packets according to arbitrary (including noncontiguous) bitmask rules. As its principal novelty, the algorithm is parameterized by the amount of memory available and can customize its data structures to optimize classification time without exceeding the given memory bound. The algorithm thus automatically trades time for space efficiency as needed. The two extremes of this time-space tradeoff (linear search through the rules versus a single table that maps every possible packet to its class number) are special cases of the general algorithm we present. Additional features of the algorithm include its simplicity, its open-source prototype implementation, its good performance even with worst-case rule sets, and its extendability to handle range rules and dynamic updates to rule sets. The contributions of this thesis first appeared in 1.
Thesis:
Thesis (M.S.C.S.)--University of South Florida, 2011.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Joshua Adam Kuhn.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 34 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E14-SFE0004919
usfldc handle - e14.4919
System ID:
SFS0028167:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 22 Ka 4500
controlfield tag 007 cr-bnu---uuuuu
008 s2011 flu ob 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0004919
035
(OCoLC)
040
FHM
c FHM
049
FHMM
090
XX9999 (Online)
1 100
Kuhn, Joshua Adam.
0 245
Grouper :
h [electronic resource] /
b a packet classification algorithm allowing time-space tradeoffs
by Joshua Adam Kuhn.
260
[Tampa, Fla] :
University of South Florida,
2011.
500
Title from PDF of title page.
Document formatted into pages; contains 34 pages.
502
Thesis
(M.S.C.S.)--University of South Florida, 2011.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
520
ABSTRACT: This thesis presents an algorithm for classifying packets according to arbitrary (including noncontiguous) bitmask rules. As its principal novelty, the algorithm is parameterized by the amount of memory available and can customize its data structures to optimize classification time without exceeding the given memory bound. The algorithm thus automatically trades time for space efficiency as needed. The two extremes of this time-space tradeoff (linear search through the rules versus a single table that maps every possible packet to its class number) are special cases of the general algorithm we present. Additional features of the algorithm include its simplicity, its open-source prototype implementation, its good performance even with worst-case rule sets, and its extendability to handle range rules and dynamic updates to rule sets. The contributions of this thesis first appeared in [1].
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
590
Advisor:
Ligatti, Jay .
653
Algorithms
Computer Networks
Network Algorithms
Network Security
Security
690
Dissertations, Academic
z USF
x Computer Science
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.4919



PAGE 1

Grouper:APacketClassicationAlgorithmAllowingTime-SpaceTradeoffs by JoshKuhn Athesissubmittedinpartialfulllment oftherequirementsforthedegreeof MasterofScienceinComputerScience DepartmentofComputerScienceandEngineering CollegeofEngineering UniversityofSouthFlorida MajorProfessor:JayLigatti,Ph.D. AdrianaIamnitchi,Ph.D. KenChristensen,Ph.D. DateofApproval: March22,2011 Keywords:NetworkAlgorithms,Security,NetworkSecurity,ComputerNetworks, Algorithms Copyright c 2011,JoshKuhn

PAGE 2

ACKNOWLEDGEMENTS Iwouldliketoacknowledgemymajorprofessor,JayLigatti,forhisguidanceandsupport. I'dalsoliketothankChrisGage,EthanFinkel,andBhargavaKondaveetifortheirassistance withtheimplementationofGrouper.IwouldliketothankGirlTalkandRadioheadfortheir musicwhichIlistenedtoonrepeatwhileimplementingGrouper.Onthetechnicalside,thanks toLenovoformakingthelegendarilysolidThinkpadlineoflaptops,RichardStallmanfor creatingEmacsandGCCandtheGPLwhichwelicensedGrouperunder,andnallyLinus TorvaldsforcreatingLinux.Finally,IwouldliketothanktheNSF,whosupportedmywork withgrantsCNS-0716343andCNS-0742736.

PAGE 3

TABLEOFCONTENTS LISTOFFIGURESii ABSTRACTiii CHAPTER1INTRODUCTION1 1.1RelatedWork1 1.2Contributions5 CHAPTER2THEGROUPERALGORITHM7 2.1PossibilitiesfortheNumberofLookupTables10 2.2MemoryUse,Table-buildTime,andClassicationTime11 2.3OptimizingClassicationTimewithoutExceedingaGivenMemoryBound12 2.4ASimpleExample13 CHAPTER3EMPIRICALANALYSIS16 3.1MotivationforImplementation16 3.2Implementation16 3.3ExperimentalSetup17 3.4Results18 CHAPTER4FINALREMARKS26 4.1DiscussionofExtensions26 4.2Summary27 LISTOFREFERENCES28 i

PAGE 4

LISTOFFIGURES Figure2.1LayoutoftherulesetsGrouperoperateson7 Figure2.2DiagramoftheGrouperclassicationtables8 Figure2.3BasicAlgorithmfortablebuilding9 Figure2.4Algorithmforpacketclassication10 Figure2.5Algorithmtodeterminetheminimumtablesforagivenmemorybound13 Figure2.6AsimpleexampleofhowGrouperworks14 Figure3.1Maximumandminimumclassierthroughputs19 Figure3.2Maximumandminimumtable-buildtimes20 Figure3.3Throughputsfor1,000rules21 Figure3.4Throughputsfor10,000rules22 Figure3.5Throughputsfor100,000rules23 Figure3.6Throughputsfor320bitsclassied,with100,000rules24 Figure3.7Throughputsfor12,000bitsclassied,with10,000rules25 ii

PAGE 5

ABSTRACT Thisthesispresentsanalgorithmforclassifyingpacketsaccordingtoarbitraryincluding noncontiguousbitmaskrules.Asitsprincipalnovelty,thealgorithmisparameterizedbythe amountofmemoryavailableandcancustomizeitsdatastructurestooptimizeclassication timewithoutexceedingthegivenmemorybound.Thealgorithmthusautomaticallytrades timeforspaceefciencyasneeded.Thetwoextremesofthistime-spacetradeofflinearsearch throughtherulesversusasingletablethatmapseverypossiblepackettoitsclassnumberare specialcasesofthegeneralalgorithmwepresent.Additionalfeaturesofthealgorithminclude itssimplicity,itsopen-sourceprototypeimplementation,itsgoodperformanceevenwithworstcaserulesets,anditsextendabilitytohandlerangerulesanddynamicupdatestorulesets.The contributionsofthisthesisrstappearedin[1]. iii

PAGE 6

CHAPTER1 INTRODUCTION Packetclassiersareessentialcomponentsofmanynetworkutilities,includingroutersand securityserviceslikerewalls,packetlters,andintrusion-detectionsystems.Onceanetwork utilityclassiesapacketoroftenjusttherstinaowofpackets,theutilitycanperform someactionsspecictothatclassofpackets,suchasforwardingthepackettoaparticular destination,droppingthepacket,updatingsomeinternalstate,orlogginginformationabout thepacket. Apacketclassierinputsalistofrules,eachspecifyingaclassofpacketsmatchedbythat rule.Forexample,arulemightspecifythatitmatchesallTCPpacketswithanysourceIP address,anydestinationIPaddressoftheform131.247.*.255,sourceport118,andanyoddnumbereddestinationportgreaterthan1023.Givenalistofsuchrules,theclassiertypically preparessomedatastructuresthatprovideamappingfromanyincomingpacket p totheset ofclassesormorecommonly,thehighest-priorityclassthat p matches.Thus,thepacket classier'sjobistoinputpackets,andforeverypacketinput,outputaclassnumber.Typically, byoutputtingclassnumber n forinputpacket p ,aclassierindicatesthatthe n th ruleinitsrule lististherstonetomatch p classiersnormallyassumeanalcatch-allruletoensurethat everyinputpacketmatchesatleastonerule. 1.1RelatedWork Manysoftwarealgorithmsexistforpacketclassication,andseveralarticlesandbooks surveytheelde.g.,[2,3,4,5].Alargenumberofalgorithmstakerulesdenedbyrange 1

PAGE 7

andprexpatternssincetheselendthemselvestoefcientsearch.Forexample,Rovniaginand WooldescribeanalgorithmtheycallGeometricEfcientMatchingGEM,whichclassies packetsbasedonrangepatterns[6].Itworksbyconsideringeachpacketdimensionasaseparatecoordinateina d -dimensionalspacewhere d isthenumberofpacketdimensionstherules arespeciedover,andpartitionsthespaceintoareaswheredifferentrulesmatch.GEMhasa dimensiondependentclassicationtimethatis O d log n ,andaworstcasespacecomplexity thatis O n 4 .BaboescuandVargheseproposeanalgorithmcalledAggregatedBitVectors ABVsin[7]thatisanimprovementoftheLucentbitvectorschemedescribedbyLakshman andStiliadisin[8].Thesealgorithmstakeprexandrangerulesasinputandusebitmapintersectionofpartialmatchesforeachpacketeldtodeterminewhichclassapacketbelongs to.ABVclaimsclassicationratescapableofhandlinglinespeedsequivalenttoOC-48,even thoughclassicationspeedistechnicallylinearinthenumberofrules. Qietal.describein[9]analgorithmcalledHyperSplitwhichisbasedonmappingthe possiblepacketvaluesintoamulti-dimensionalspaceanddividingthatspaceupintoregions sothatallpointsrepresentedbyapacketvalueinaregionmatchaspecicrule.HyperSplit isalsodenedbyrangeandprexpatternsandgainsefciencyoverthesimilarstrategyof HiCuts[10].Bothalgorithmsdividethesearchspaceforincomingpacketsintoaheirarchy. Interestingly,classicationtimeforbothHiCutsandHyperSplitis O d ,andtheirspaceusage is O n d meaningtheclassicationtimeisdimensiondependentunlikeGrouper,asChapter3 discusses. Unfortunately,handlingonlyrange/prexpatternsisproblematicforrulesthatcouldbe speciedmoresimplywithbitmasksbitmaskpatternscannotingeneralbetranslatedefcientlyintorange/prexpatterns.Forexample,tomatchIPaddressesoftheform131.247.*.255 e.g.,allhostsnumbered255onanysubnetinthe131.247networkwouldrequire256range/prex patternsbecausethewildcardbitsdonotappearattheendofthepattern.Similarly,tomatch all16-bitportnumbersthatareoddandgreaterthan1023wouldrequireonly6bitmaskpatternsbut32,256range/prexpatterns.Ingeneral,asingle b -bitbitmaskpatterncanrequire 2

PAGE 8

upto 2 b )]TJ/F22 7.9701 Tf 6.586 0 Td [(1 range/prexrulesinordertomatchanequivalentsetofinputsspecicallywhen convertingabitmaskpatternoftheform*1or*0intorange/prexpatterns.Thisisnotjusta theoreticalproblem;GuptaandMcKeownfoundthatabout10%ofrulestheysurveyedinreal classierscontainednoncontiguousbitmaskpatterns[11]. Ontheotherhand,wecanconvertrange/prexpatternsrelativelyefcientlyintobitmask patterns.Prexpatternsarealreadybitmaskpatterns,andeveryrangepatternover b bitscanbe automaticallyconvertedintoatmost 2 b )]TJ/F15 11.9552 Tf 9.298 0 Td [(2 bitmask/prexpatternstheworst-caseconversion occursforrangesoftheform00...01...10[12].BasedonGuptaandMcKeown'ssurvey ofpracticalpatternusageinclassicationrules,particularlytheirndingthatabout10%of rulesusedarangepatternbutabout90%ofallrangepatternsjustspeciedportnumbers greaterthan1023[11],wemightexpectthatconvertingarange-patternrulelistintoabitmaskpatternrulelistwouldtypicallyinatethenumberofrulesbyabout50%.Also,theworst-case linearinationwhenconvertingrangetobitmaskpatternsistractable,whiletheworst-case exponentialinationwhenconvertingbitmasktorange/prexpatternsisnot.Bitmaskpatterns arethereforemoreefcientlyexpressive,ingeneral,thanrange/prexpatternsalone. Somesoftwareclassicationsolutionscanhandlenoncontiguous-bitmaskpatterns,such asRecursiveFlowClassicationRFCdevelopedbyGuptaandMcKeown[11].RFCisa heuristicthatexploitsthestructureofcommonrulesets.Itusesafeedbackmechanismto recursivelyclassifyapacketintosmallerandsmallersetsofpossiblerulesthatcanmatchthe incomingpacket.Theauthorsreport,however,thatRFCusesaprohibitivelylargeamountof memoryforrulesetsofmorethan6,000rules.Inaddition,thememoryusageisnottunable andgrowsexponentiallyinthenumberofdimensionstherulesetisdenedover. Ternarycontent-addressablememoriesTCAMsarespecializedhardwarethatcanalso classifypacketsusingbitmaskrules[13,14].Currently,theyarethedefactoindustrystandard forclasscationduetotheirhighthroughputbecausetheycancompareincomingpacketsto allrulepatternsinparallel[15].TCAMSareexpensive,however,costingupto$250per Mbofmemory[15].Theyalsoconsumefrom15-30wattsperMb,leadingmostofthechips 3

PAGE 9

commerciallyavailabletobelimitedto128Mbsizeorlessusually1-2Mbiscommon[15,16]. Finally,TCAMsarelimitedinthelengthofthebitmaskruletheycanspecify.Thestandard lengthis144bits[15],butthisisinadequateforclassifyingIPv6headerswhichhave320bitsat aminimum.WewillseethatnotonlycanGrouperclassifyrulesetsofhundredsofthousandsof rules,itcanalsohandlerulesspeciedovermanythousandsofbitsi.e.,Groupercanclassify packetsbasedonrulesetsmuchlargerthan128Mb,evenoncommodityhardware. Otherclassicationalgorithmshandlepatternsmoreexpressivethanbitmasks,including fullregularexpressionse.g.EFSAs[17],XFAs[18],andBDDs[19].However,allalgorithmsinthiscategorysufferfromworst-caseexponentialinthenumberofpacketbits classiedmemoryrequirementsand/orclassicationtimes,duetothearbitrarilycomplexset ofpacketsspeciableinasinglerule. Anothergroupofclassiersthathandlemoreexpressivepatternsthanbitmasksarethe Snortintrusiondetectionsystem[20],andtheopensourcerewalliptables[21].Bothofthese haveveryexpressiverulelanguages,whichareconvenientfordescribingcomplicatedrewall policies.Unfortunately,bothoftheseimplementationsrelyonalinearsearchthroughthe rulelisttoclassifyeachpacket.Linearclassicationtimeinthenumberofruleslimitstheir applicationinenvironmentswherehighthroughputandlargerulesetsarerequiredbecausethe classierbecomesabottleneckofthesystem. Allclassicationalgorithmsmakesometime-spacetradeoffbetweentwoextremes.Atone extreme,aclassiercouldusenospacebeyondthatoftherulelistbuthavetoclassifyeach packetbyperformingalinearsearchthroughthelistofrules.Inthiscase,boththespace usedandpacket-classicationtimesare O nb=w ,becauseeachofthe n rulesmayspecify b packetbitsthathavetomatchthe b bitsstoredin O b=w machinewordswhere w isthe wordsizeinbits;weanalyzespaceusageandclassicationtimeintermsofmemorywords stored/accessed.Attheotherextreme,aclassiercouldmaintainasingletablethatmaps eachofthe 2 b possibleinputpacketstoitsclassnumber.Inthiscase,thespacerequiredfor storing 2 b entriesofclassnumberseachhavingsize O lg n =w is O b lg n =w ,whilethe 4

PAGE 10

classicationtimeisonly O lg n =w .Linearsearchisspaceefcientbutruntimeinefcient, whileasingletableisruntimeefcientbutspaceinefcient. 1.2Contributions ThisthesispresentsanalgorithmcalledGrouperdescribedindetailinChapter2for classifyingpacketsaccordingtobitmaskrules.Thealgorithmpartitionsthebitsbeingclassied intoapproximatelyequal-sizedgroupsanduseseachvalueofgroupedbitsinapackettolook upabitmapofrulesmatchingthatgroupvalue.Itcomputesthesetofrulesmatchingany packetbyintersectingthesetsofrulesmatchingeachofthatpacket'sgroupedbits.Thus, Grouperusesthecommontechniqueofintersectingsetsofmatchedrules[8,11,7],butunlike anyrule-set-intersectionalgorithmsweareawareof,Grouperclassiesaccordingtoarbitrary includingnoncontiguousbitmaskruleswhileexhibitinggoodperformanceevenonlargerule setshavingmanythousandsofrules. Bycontrollingthesizesofbitgroupings,Groupercancontroltheamountofmemory neededforitsbitmap-lookuptables;largergroupsizesimplylargeramountsofmemoryconsumedbutfasterclassicationtimes.Thus,thealgorithmcancustomizeitsdatastructuresto optimizeclassicationtimewithoutexceedingagivenmemorybound.Thisabilitytoautomaticallytradetimeforspaceefciencyisthealgorithm'sprincipalnovelty.Besidesautomaticallytradingtimeforspaceandclassifyingaccordingtoarbitrarybitmaskpolicies,Grouper features:simplicity,anopen-sourceprototypeimplementation[22],goodperformanceeven withworst-caserulesets,andextendabilitytohandlerangerulesanddynamicupdatestorule sets. AsdescribedinChapter3,theexperimentsperformedhaveshownthatGrouperiscapable, whenimplementedinsoftwareonacommoditylaptopusingabout2GBofmemory,ofclassifyingentire320-bitIPv6headersintooneof1,000respectively100,000randomlygenerated classesat579,397,774pps.Whenclassifyingaccordingtoonly100randomlygenerated rules,butwitheachrulespecifyingabitmaskoverafull12,000bitsofEthernetpayload,we 5

PAGE 11

observedclassicationthroughputsof25,271ppsi.e.,severalhundredMbps,againinasoftwareimplementation.Groupercanclassifyhundreds/thousandsofpacketbitsefciently,in partbecauseitoperatesindependentlyofthenumberofpacketelds/dimensionsbeingclassied. 6

PAGE 12

CHAPTER2 THEGROUPERALGORITHM Grouperuses t lookuptablestoclassify b packetbitsaccordingto n rulesseeFigure2.1. Eachlookuptablemapseither b b=t c or d b=t e referredtoasoorandceilinggroupsrespectivelyofthe b packetbitstoan n -lengthbitmapindicatingwhichofthe n rulesmatch those b b=t c or d b=t e packetbits.Wesaythe b b=t c or d b=t e bitsusedtoindexatableare grouped together,withagroupsizeof b b=t c or d b=t e bits.Everytablemapsagroupofbitsto an n -lengthbitmapseeFigure2.2. Figure2.1.LayoutoftherulesetsGrouperoperateson Grouperusesthelookuptablesasfollows.Given b bitsofaninputpackettoclassify, Grouperdividesthose b bitsinto t groupsandusesthevaluesofthebitsineachgrouptoindex intoatabletolookupthe n -lengthbitmapofrulesmatchingthatgroupofbits.Byintersecting i.e.,bitwiseANDingallbitmapsofrulesmatchingeverygroup,Grouperendsupwithan n -lengthbitmapofrulesmatchingtheentireinputpacket.Therstsetbitinthatnalbitmap 7

PAGE 13

Figure2.2.DiagramoftheGrouperclassicationtables indicatesthelowest-numberedhighest-priorityrulematchingtheoriginal b inputbits.The algorithminFigure2.4formallydescribeshowGrouperclassiespackets. BecauseGrouperdoesnotguaranteethatitwillgroupanytwoparticularbitstogether,it mayformgroupsofarbitrarybitsfromthe b -bitinput.Oneconsequenceofthisarbitrarinessin bitgroupingsisthatGrouperoperatesindependentlyofpacketelds/dimensions;thealgorithm simplyviewsitsinputas b packetbits,regardlessofhigher-levelcategorizingofthosebitsinto elds.Asecondconsequenceofgroupingarbitrarybitstogetheristhatotherpacketbitscannot inuencewhataparticularbitvaluematches;itisthisconstraintthatlimitsGrouperinitsbasic versiontoclassifyingaccordingtobitmaskrules. 8

PAGE 14

Grouperdoeshoweverguaranteethatitwillpartitionthe b packetbitsinto t groupshaving asequalofsizeaspossiblei.e.,either b b=t c or d b=t e bits.Eveningoutthegroupsizesin thiswayevensoutthenumberofbitsusedtoindexeachtable,thuspreventingspace inefcienciesthatarisewithdisproportionatelylargetablesandtimeinefcienciesthat arisewithdisproportionatelysmalltables.Sincethebitmapisthesamelengthforeachtable itisproportionalto n ,nottothenumberofbitsthatindexthetable,disproportionatelysmall tablesincurthesameruntimehitasalargetable,butprovidelessinformationforclassication. ThealgorithminFigure2.3showshowGrouperconstructsitstables. Input :Thenumberoftablestobuild, t Input :Thenumberofrelevantbitsinarule, b Input :Thetotalnumberofrules, n Input : Rules ,aseriesofbitmaskrulesindexablebybits.Example:Figure2.1 Output :AseriesoftablesthatcanbeusedbytheclassicationalgorithminFig.2.4 begin Tables azeroedseriesoftableswiththestructureshowninFig.2.2 for i =1 t )]TJ/F15 11.9552 Tf 11.955 0 Td [( b mod t do L i )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 b b = t c +1 H L + b b = t c)]TJ/F15 11.9552 Tf 19.925 0 Td [(1 for j =0 2 b b = t c )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 do for k =1 n do if bits L through H of Rules [ k ] match j then Setbit k intable i ,row j in Tables offset t )]TJ/F15 11.9552 Tf 11.955 0 Td [( b mod t for i = t )]TJ/F15 11.9552 Tf 11.956 0 Td [( b mod t t do L i )]TJ/F40 11.9552 Tf 11.955 0 Td [(offset )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 d b = t e + offset b b = t c +1 H L + d b = t e +1 for j =0 2 d b = t e )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 do for k =1 n do if bits L through H of Rules [ k ] match j then Setbit k intable i ,row j in Tables return Tables Figure2.3.BasicAlgorithmfortablebuilding 9

PAGE 15

Input :AsetofGrouper Tables createdbythealgorithminFig.2.3 Input :ThenumberofGroupertables, t Input :Thenumberofrelevantbitsinarule, b Input :Thetotalnumberofrules, n Input :Abitmap p oflength b ,theincomingpacketbitstobeclassied Output :Theclassnumbertheincomingpacket p belongsto begin OutVector azeroedbitmapoflength n bits for i =1 t do if i t )]TJ/F15 11.9552 Tf 11.956 0 Td [( b mod t then s b b = t c i offset 0 b offset 0 else s d b = t e i offset t )]TJ/F15 11.9552 Tf 11.955 0 Td [( b mod t b offset i offset b b = t c L i )]TJ/F40 11.9552 Tf 11.955 0 Td [(i offset )]TJ/F15 11.9552 Tf 11.956 0 Td [(1 s + b offset +1 H L + s )]TJ/F15 11.9552 Tf 11.955 0 Td [(1 rownum bits L through H inclusiveof p OutVector OutVector bitwiseANDedwithrow rownum ,table i in Tables return Positionoftherstsetbitin OutVector or0ifnobitisset Figure2.4.Algorithmforpacketclassication 2.1PossibilitiesfortheNumberofLookupTables Asaspecialcase,when t =1 ,Grouperdoesnothavetoperformanybitmapintersections, soitsonelookuptable,indexedbyall b packetbits,canbeoptimizedtostorenotbitmapsbut theactualclassnumbersmatchingallpossible b -bitvalues.Hence,thespecialcaseof t =1 correspondstooneextremeinthetime-spacetradeoffofpacketclassication,inwhichasingle tablemapsallpossible b -bitvaluestotheirclassnumbers. Asanotherspecialcase,when t = b ,everylookuptablemapsasinglebitoftheinput packettoabitmapindicatingwhichrulesmatchthatbitvalue.InthiscaseGrouperclassiesby iteratingthrougheverybitofinputandintersectingthebitmapforeachinputbittodetermine whichrulesmatchallinputbits.Thisapproachisconceptuallythesameasalinear-search classicationalgorithm:bothapproachesiteratethroughallpossiblepairingsofinputbitsand 10

PAGE 16

rulenumberstondwhichrulenumbersmatchalltheinputbits;bothapproachesclassifyin time O bn=w using O bn=w space.Hence,thespecialcaseof t = b correspondstotheother extremeinthetime-spacetradeoffofpacketclassicationinwhichthealgorithmperformsa linearsearch. Ingeneral,setting t toalowervaluecausesGroupertousemorespacebutclassifypackets morequicklyfewertableshavetobequeriedandfewerbitmapshavetobeintersected.The specialcasesof t =1 and t = b correspondtoextremesofthetime-spacetradeoffinpacket classication.However,itturnsoutthatitnevermakessensetoset t> d b= 2 e becauseanysuch t valuesavesnospacecomparedtosetting t = d b= 2 e .Toseewhy,considerthehypothetically mostspace-savingsettingof t to b ;inthiscaseeachofthe b tablesstorestwo n -lengthbitmaps. Wecanalwaysreplacetwosuchtablesconsumingatotalof 4 d n=w e spacewithasingle tablethatmaps2bitsoftheinputpackettofourpossible n -lengthbitmapsalsoconsuming atotalof 4 d n=w e space.Thus,itonlymakessensetouseGrouperwith t valuesranging from 1 correspondingtothesingle-lookup-tablealgorithmto d b= 2 e correspondingtothe linear-searchalgorithm. 2.2MemoryUse,Table-buildTime,andClassicationTime Grouperuses t tables,eachhaving O b=t entries,witheachentrybeingan n -lengthbitmap consuming O n=w machinewords.Thetotalmemorywordsusedistherefore O b=t tn =w whereagain, t canrangefrom 1 to d b= 2 e .Moreprecisely,thefollowingequationgivesthe numberofbits m requiredtostoreGrouper'stables. m = 8 > > > < > > > : t )]TJ/F15 11.9552 Tf 11.955 0 Td [( b mod t 2 b b=t c n + b mod t 2 d b=t e n if 2 t d b= 2 e 2 b d lg n e if t =1 .1 Equation2.1partitionsthe b inputbitsinto t groupssuchthateverygrouphasasuniform aspossibleofasize: b mod t groupswillcontain d b=t e bits,while t )]TJ/F15 11.9552 Tf 12.523 0 Td [( b mod t groupswill contain b b=t c bits.Weconsequentlyhave b mod t tablesof 2 d b=t e entriesand t )]TJ/F15 11.9552 Tf 10.944 0 Td [( b mod t tables 11

PAGE 17

of 2 b b=t c entries.Buildingthefulllookuptablesfromscratchmayrequiretimeproportionalto theirsize,withGrouperiteratingoverandsettingeverytableentry. Grouper'sspacerequirementsareexponentialinthenumberofbitsineachgroup,whichis differentfrommostothercomparableclassicationalgorithmswhosespaceusageisdenedin termsofthexednumberofpacketdimensionsitsrulesetisdenedoverusuallydenoted d .Forexample,boththecross-productingtechnique[12],andRFC[11]usememorythatis O n d ,where N isthenumberofrules[4]. ClassicationtimeforGrouperis O tn=w becauseitquerieseveryoneofthe t tables toobtainabitmapconsuming O n=w memorywords,andasGrouperfetcheseachofthose bitmaps,theygetintersectedwithanypreviouslyfetchedbitmaps.Althoughthisclassication timeislinearin n thenumberofrules,Grouper,likeotherbitmap-intersectionalgorithms, benetsfromstoringruleinformationinbitmapstodividethe n factorintheclassication timebythewordsize,andspatiallocalityofbitsfetchedinbitmaps,resultingingoodcache performance.ThisisincontrasttosomeotherpacketclassierslikeSnortandiptableswhose bestcaseclassicationtimeislinearinthenumberofrulesi.e. O n [20,21]. 2.3OptimizingClassicationTimewithoutExceedingaGivenMemoryBound MinimizingGrouper's O tn=w classicationtimerequiresminimizing t numberoftablesand n numberofrulesandmaximizing w wordsize.Therulesetdictates n and hardwaredictates w ,makingtheseparametersbeyondGrouper'scontrol.Groupercanhoweverminimize t suchthatitstablestwithinagivenmemoryconstraint.Totakeadvantage ofthisability,weparameterizeGrouperbynotonlyaclassicationpolicy,butalsoamemoryconstraint;Grouperwillautomaticallyduringtablepreprocessingtradetimeforspace efciencytomakeitslookuptablesasruntimeefcientaspossiblewhileobeyingthegiven memoryconstraint. Equation2.1alreadyshowshowtocalculate m thenumberofbitsneededforGrouper's lookuptableswhengiven t b ,and n .Tocalculateaminimum t whengivenamaximum m 12

PAGE 18

andaclassicationpolicywhichdeterminesthevaluesof b and n ,Groupersimplychecks whetherasinglelookuptableconsumeslessmemorythanthegiven m value;ifnotthen performsabinarysearchbetweenallpossible t valuesfrom 2 to d b= 2 e tondthesmallest onethat,whenpluggedintoEquation2.1withthegiven b and n valuesandrounding n tothe nextmultipleof8,producesamemoryrequirementnogreaterthanthegivenmaximum m valueseeFigure2.5.Thisbinary-searchalgorithmproducestheoptimal t in O lg b time. Input :Maximummemoryallowedinbits, m Input :Numberofrules, n Input :Numberofrelevantbits, b Output :Minimumnumberoftablesthattwithinthegivenmemorybound,or-1if thereisnopossiblevalueoftthatrespectsthisbound begin if m < 2 n b or n < 1 or b < 1 then return )]TJ/F15 11.9552 Tf 9.299 0 Td [(1 if m log n 2 b then return 1 low d b = 2 e high 1 while low high > 1 do mid low + high = 2 FloorTablesMem mid )]TJ/F15 11.9552 Tf 11.955 0 Td [( b mod mid 2 b b = mid c n CeilingTablesMem b mod mid 2 d b = mid e n memNeededForTables FloorTablesMem + CeilingTablesMem if m < memNeededForTables then high mid else low mid return low Figure2.5.Algorithmtodeterminetheminimumtablesforagivenmemorybound 2.4ASimpleExample InordertobetterunderstandhowGrouperworks,itcanbehelpfultohaveanexample. ConsidertherulesetinFigure2.6.Forsimplicity,thisrulesethasonlytworulesofsevenbits each n =2 b =7 .Inaddition,theexamplelimitsthealgorithmtousingonly24bitsfor 13

PAGE 19

Figure2.6.AsimpleexampleofhowGrouperworks theclassicationtables M =24 .Withanumberofrelevantbits, b ,thissmall,thereareonly 3valuesfor t thatmakesense:1,2or3tablesseeSection2.1.Ifwecomputethememory usagewithequation2.1usingtheseparameters,wegetmemoryrequirementsof256,48and 16bits,respectively.Since16bitsistheonlyresultthattswithinthememorylimit, t =3 withsofewpossibilitiesfortables,itiseasytocalculatealloftheirmemoryrequirements,but thealgorithminFigure2.5performsthiscalculationfasterforlargervaluesof b Next,GrouperbuildstheactualclassicationtablesasshowninFigure2.6.Inthediagram, thenumbersalongthetopofthetablesrepresenttherulenumbers.Thenumbersalongtheleft sideofthetablesrepresenttherownumberexpressedinbinaryforeaseofcomparingwiththe bitmaskrules. Figure2.6containstwoexampleinputs,A 1011110 andB 0011111 .Grouperbreaks theinputintothreegroups,twoof2bitsandoneof3bits,andretrievesthebitmapfromthe correspondingrowofthecorrespondingclassicationtable.ThenitperformsabitwiseAND. InA,bothrulesmatchtheinput,butGrouperreturnsonlythehighestpriorityruletheleast 14

PAGE 20

signicantsetbitinthebitmap,sorule1wouldbeoutput.InB,neitherrulematchesthe input,soGrouperwouldoutputthedefault0. 15

PAGE 21

CHAPTER3 EMPIRICALANALYSIS 3.1MotivationforImplementation WhilewecandeterminethetheoreticalperformanceofGrouper,it'shelpfulgaugethe realworldperformanceofthealgorithminordertoplaceitinsomecontext.Forexample, itisusefultocomparethethroughputofthealgorithmtothemaximumthroughputsofvariouscommonnetworkcapacitiessuchas10gigabitEthernet.Inaddition,wealsowantedto measurehowGrouperbehavesbothunderrealisticconditionsandextremestressconditions bothwithrulesetsconcernedwiththousandsofbitsandwithrulesetscontainingverylarge numbersofrules. Finally,whiletheGrouperalgorithmshouldscalesmoothlyinperformancewhenchangingtheamountofmemoryavailabletoit,inpractice,onrealmachinestherearenumerous complicatingfactorssuchasvirtualmemory,multi-levelprocessorcaches,branchprediction, andOScontextswitchesthatcandecreasethereliabilityofGrouper'sperformance.Inour implementationandtesting,weattempttominimizeoraccountfortheseeffectsasmuchas possible,butit'snotpossibletoeliminatethementirely. 3.2Implementation WeimplementedaprototypeofGrouperin1093linesofCcode.Thesourcecodeand benchmarkingscriptsareavailableonline[22].Wecompiledtheprogramforthex86-64architecture,whichadherestotheAMD64specicationandincludes16128-bitmultimediaregisters.Whencompiledwith gcc 's -O3 option,ourprototypeperformsbitmapintersectionsin 16

PAGE 22

these128-bitregisters.Ourprototypealsomitigatestheinefciencyofaddressingindividual bitsonabyte-addressablemachinebypaddingallbitmapstocoincidewithbyteboundaries henceusing d b= 8 e 8 bitsinsteadof n bitsasthebitmaplength. Ourimplementationmultithreadsthetable-buildpreprocessingoperationtospeedthis operationupinproportiontothenumberofprocessorcores.Likeallbitmap-intersectionalgorithms,webelieveGrouper'spacket-classicationoperationsareamenabletoparallelization orpipelining.Webrieyexperimentedwithperformingclassicationsintwothreadsone foreachofthetwoprocessorcoresonourtestmachinebutfoundthecontext-switchingcosts outweighedthebenetsofconcurrencyinthiscase,sowerevertedtoasingle-threadimplementation. 3.3ExperimentalSetup WetestedGrouper'sperformanceonaDellLatitudeD630with2GHzIntelCore2Duo processors,runningaminimalversionofArchLinux.Althoughthelaptophad4GBofmemory,welimitedthememoryusedbyGroupertoabout2GBtopreventGrouperfromcontending withanysystemsoftwareformemory. Ourexperimentshadthreeindependentvariables b n ,and t andtwodependentvariables throughputandtable-buildtimes.The b valuestestedwere:32correspondingtoclassicationbasedonandIPv4addressorsourceplusdestinationport,104correspondingtoclassicationbasedonan8-bitprotocolnumber,sourceanddestinationportnumbersat16bitseach, andsourceanddestinationIPv4addresses,320correspondingtoclassicationbasedonan entirexedportionofanIPv6header,and12,000correspondingtoclassicationbasedon theentirecontentsofamaximum-sizedEthernetv2payload.The n valuestestedwere:100, 1K,10K,100K,and1Minthisthesis,K,M,andGreferto 10 3 10 6 ,and 10 9 .The t values testedwere:everyvaluefromthemaximumof b= 2 tables,downtotheminimumnumberof tablespossiblewithoutexceedingabout2GBofmemorytheminimum t overalltestswas2, whichwaspossiblewith b =32 and n 100K.Theoneexceptiontothisuniverseofindepen17

PAGE 23

dentvariablesisthatwecouldnottestGrouper'sperformanceclassifying12Kbitsaccording to1Mrulesbecausedoingsowouldrequireabout3GBofmemory,evenusingthemaximum t valuepossible.Hence,wereportnoresultsforthiscaseof b = 12Kand n = 1M. Foreverycombinationof b n ,and t values,wemeasuredthroughputandtable-buildtime forarandomlygeneratedruleset.Wemeasuredthethroughputbycreatingaleof500K random b -lengthpacketstheexceptionbeingthatweonlyused10Krandompacketswhen b was12K,startingareal-timetimerjustbeforetherst b bitswerereadfromthatle,having Grouperinputandclassify b -lengthpacketsoneatatimefromthele,andstoppingthetimer justafterGroupernishedclassifyingallpacketsinthele;thisprocessproducedappsmeasurementbasedonrealincludingleI/Otime.Inthefollowingsection,wecalculatedall measurementsreportedintermsofbpsfromtheoriginalppsmeasurementusingaxedpacket sizeof12Kbits.Inaddition,allthetable-buildtimesarewallclocktime.Weperformedall teststhreetimesthedatapointsinthegraphshereareaveragesofthethreetrials. 3.4Results ThegraphsinFigures3.1.2summarizeourexperimentalresults.Thesegurespresent throughputsandtable-buildtimesforgivenvaluesof n and b .Thethroughputsarerepresented inbothabsolutebitspersecondontherightaxes,andpacketspersecondontheleftaxes. Foreachcombinationof n and b ,thegraphsdisplaytwopoints:anupperpointcorrespondingtothethroughputortable-buildtimewithGrouperusingthemaximumamountof memoryavailabletoit,uptoabout2GB,andalowerpointcorrespondingtothethroughput ortable-buildtimewithGrouperusingtheminimumamountofmemorypossiblei.e.,with t = b= 2 .Forexample,withall2GBofmemoryavailable,Grouper'sthroughputwas25Kpps Mbpsfor b = 12Kand n = 100,140Kpps.68Gbpsfor b = 320and n = 10K,and1.1M pps.2Gbpsfor b = 104and n = 1K.Figure3.1showsthatourprototypesoftwareimplementationperformswell,particularlygiventhatrulesetsoftenhavefewerthan1Krules[11,23]. Figures3.1.2whosegraphshavelog-scaleaxesalsoillustratethatimprovingclassication 18

PAGE 24

throughputbyaconstantfactorrequiresexponentiallygreatermemory,implyingexponentially greatertable-buildtimes. Figures3.3.5xthenumberofrulesat1K,10K,and100K,sowecanviewtheclassicationthroughputsintermsofmemoryconsumption.Thehighthroughputwhen t = 2ledus tobreakthey-axesinthesegraphs.Also,thethroughputdipsthatoccurinFigure3.3,evenas theamountofmemoryusedincreases,areduetoourtestmachine's4MBL2cachesize. Figure3.1.Maximumandminimumclassierthroughputs Finally,Figures3.6.7depictGrouper'sperformanceinacoupleofextremecaseswhere Grouperisclassifyingeitheralargenumberofbitsorisclassifyingusingaverylargeruleset. Thesegraphsillustratetheinverserelationshipbetweenthroughputy-axisandnumberof tablesx-axis,whichresultsfromGrouper's O tn=w classicationtimeperpacket. 19

PAGE 25

Figure3.2.Maximumandminimumtable-buildtimes 20

PAGE 26

Figure3.3.Throughputsfor1,000rules 21

PAGE 27

Figure3.4.Throughputsfor10,000rules 22

PAGE 28

Figure3.5.Throughputsfor100,000rules 23

PAGE 29

Figure3.6.Throughputsfor320bitsclassied,with100,000rules 24

PAGE 30

Figure3.7.Throughputsfor12,000bitsclassied,with10,000rules 25

PAGE 31

CHAPTER4 FINALREMARKS 4.1DiscussionofExtensions WeareconsideringtwoextensionstoGrouper.Therstisaddingtheabilitytospecify rangesintherulesets.Severalpaperse.g.[24,25]discussalgorithmstoexpandaclassicationrulespeciedasarangeintoaseriesofbitmaskrulesusuallyinthecontextofTCAMs. Intheworstcase,asinglerangewillrequireanumberofbitmaskrulesproportionaltothe numberofbitsoverwhichtheruleisspeciede.g.sixteenbitmaskrulesforarangeoversixteenbitsoftheinput.Unfortunatelywhilethisisaniceupperboundforasinglerange,many commonclassicationrulesrequirespecifyingmorethanonerangeperrule.Forexample, thishappenswhenspecifyingonerangeforthesourceportandonerangeforthedestination port.Inthiscase,werequiretheCartesianproductoftheindividualrangeexpansionsinorder tocorrectlymatchallpossiblecombinationsofmatches.Thismeansthatrangeexpansionis exponentialinthenumberofrangesspeciedintherule. Fortunately,Grouperoffersanadditionalwaytohandlerangerules.Insteadofconverting therangerulesdirectlytoaseriesofbitmaskrules,wesimplygroupallbitsofeachrangeinto theirowntable.Wecanthenbuildthetablesbysettingtheentriesinthetablethatfallwithin therange'sbounds.Thislosestheexibilityofbeingabletomakethegroupsizesverysmall sinceallthebitsineachtherangemustbeinthesamegroup,buthastheadvantagethatnow thememoryrequireddoesnotscaleexponentiallywiththenumberofrangesspecied.Which strategyresultsinlessmemoryusagedependsonthedetailsoftheruleset,soideallyGrouper cancomparethememorycostsbeforebuildingitstablesandadjustitsstrategyaccordingly. 26

PAGE 32

Thesecondextensionwouldhandledynamicupdatestorulesets,forthosecaseswhere rebuildinglookuptablesfromscratchtakestoolongcf.Figure3.2.Tohandledynamic ruledeletions,Groupercouldidentifyruleswithinternalnumbers,whichmaydifferfromthe externallydenedclassnumbers.Forexample,afterdeletingRule0,Groupercouldcontinue toidentifythenewRule0internallyasRule1.Thisdecouplingofinternalrulenumbersfrom externalclassnumbers,combinedwithafewadditionaldatastructuresanextrabitmapwith 0sinpositionsofdeletedinternalrulesandamapfrominternaltoexternalnumbers,enables Groupertoprocessruledeletionsefciently.Tohandledynamicruleadditions,Groupercould allocatelargerbitmapsthanitneedsinitially.Ifthisextraspaceevergetsexhausted,Grouper coulduseanotherthreadtorebuildthetableswithouttakingtheclassicationthreadofine. 4.2Summary ThisthesishaspresentedGrouper,analgorithmforclassifyingpacketsaccordingtoarbitrarybitmaskrules.Grouperisparameterizedbytheamountofmemoryavailableforitslookuptablesandautomaticallytradestimeforspaceefciencyasneededtotwithinagivenmemory bound.ExperimentswithGrouper'sopen-sourceprototypeimplementationonacommodity laptophavedemonstrateditsgoodperformance,particularlywhenclassifyingbasedonlarge numbersofpacketbitse.g.,300Mbpswith b = 12Kand n = 100,1.68Gbpswith b = 320and n = 10K,and13.2Gbpswith b = 104and n = 1K.Becausethebitmasksusedinarulesethave noeffectonGrouper'sperformance,ourexperimentalresultsonrandomlygeneratedrulesets demonstrateGrouper'sperformanceonworst-caserulesets.Inaddition,Grouperlendsitself tobeingextendedtohandlerangerules.Givenitsexibilityandperformance,Grouperisa compellingpacket-classicationalgorithm. 27

PAGE 33

LISTOFREFERENCES [1]JayLigatti,JoshKuhn,andChrisGage.Apacket-classicationalgorithmforarbitrary bitmaskrules,withautomatictime-spacetradeoffs.In ProceedingsoftheInternational ConferenceonComputerCommunicationNetworksICCCN ,August2010. [2]DavidE.Taylor.Surveyandtaxonomyofpacketclassicationtechniques. ACMComput. Surv. ,37:238,2005. [3]SatajSahni,KunSukKim,andHaibinLu.IProutertables.InDineshMehtaandSartaj Sahni,editors, HandbookofDataStructuresandApplications ,chapter48.Chapman& Hall/CRC,2005. [4]PankajGupta.Multi-dimensionalpacketclassication.InDineshMehtaandSartaj Sahni,editors, HandbookofDataStructuresandApplications ,chapter49.Chapman& Hall/CRC,2005. [5]DeepankarMedhiandKarthikeyanRamasamy. NetworkRouting:Algorithms,Protocols, andArchitectures .MorganKaufmannPublishersInc.,SanFrancisco,CA,USA,2007. [6]D.RovniaginandA.Wool.Thegeometricefcientmatchingalgorithmforrewalls.In ProceedingsoftheIEEEConventionofElectricalandElectronicsEngineersinIsrael 2004. [7]FlorinBaboescuandGeorgeVarghese.Scalablepacketclassication. IEEE/ACMTrans. Netw. ,13:2,2005. [8]T.LakshmanandD.Stiliadis.High-speedpolicy-basedpacketforwardingusingefcient multi-dimensionalrangematching. SIGCOMMComput.Commun.Rev. ,28:203, 1998. [9]YaxuanQi,LianghongXu,BaohuaYang,YiboXue,andJunLi.Packetclassication algorithms:Fromtheorytopractice.In ProceedingsofInfocom ,2009. [10]PankajGupta,,PankajGupta,andNickMckeown.Packetclassicationusinghierarchicalintelligentcuttings.In inHotInterconnectsVII ,pages34,1999. [11]PankajGuptaandNickMcKeown.Packetclassicationonmultipleelds.In ProceedingsofSIGCOMM ,1999. [12]V.Srinivasan,G.Varghese,S.Suri,andM.Waldvogel.Fastandscalablelayerfour switching. SIGCOMMComput.Commun.Rev. ,28:191,1998. 28

PAGE 34

[13]KarthikLakshminarayanan,AnandRangarajan,andSrinivasanVenkatachary.AlgorithmsforadvancedpacketclassicationwithternaryCAMs.In ProceedingsofSIGCOMM ,2005. [14]ChadR.Meiners,AlexX.Liu,andEricTorng.Bitweaving:Anon-prexapproach tocompressingpacketclassiersinTCAMs.In ProceedingsoftheIEEEInternational ConferenceonNetworkProtocols ,pages93,October2009. [15]ChadR.Meiners,AlexX.Liu,andEricTorng.Tcamrazor:Asystematicapproach towardsminimizingpacketclassiersintcams. NetworkProtocols,IEEEInternational Conferenceon ,0:266,2007. [16]IncCiscoSystems.Ciscocatalyst6500seriesswitch. http://www.cisco.com/en/US/products/hw/switches/ps708/ products white paper09186a00800c9470.shtml#wp39459. [17]R.SekarandP.Uppuluri.Synthesizingfastintrusionprevention/detectionsystemsfrom high-levelspecications.In ProceedingsoftheUSENIXSecuritySymposium ,1999. [18]RandySmith,CristianEstan,andSomeshJha.XFA:Fastersignaturematchingwith extendedautomata.In ProceedingsoftheIEEESymposiumonSecurityandPrivacy pages187,2008. [19]ScottHazelhurst,AdiAttar,andRaymondSinnappan.Algorithmsforimprovingthedependabilityofrewallandlterrulelists.In ProceedingsoftheInternationalConference onDependableSystemsandNetworks ,pages576,2000. [20]Sourcere,Inc. Snort .http://www.snort.org/. [21]NetlterCoreTeam.Iptableswebpage.http://www.netlter.org/projects/iptables/. [22]JoshKuhn,JayLigatti,andChrisGage.Thegrouperwebpage. http://www.cse.usf.edu/ligatti/projects/grouper/. [23]AvishaiWool.Aquantitativestudyofrewallcongurationerrors. Computer ,37:62 67,2004. [24]BaruchSchieber,DanielGeist,andAyalZaks.Computingtheminimumdnfrepresentationofbooleanfunctionsdenedbyintervals. DiscreteAppliedMathematics ,1493:154173,2005.BooleanandPseudo-BooleanFunctions. [25]O.RottenstreichandI.Keslassy.Worst-casetcamruleexpansion.In INFOCOM,2010 ProceedingsIEEE ,pages1,March2010. 29