USF Libraries
USF Digital Collections

End-to-end available bandwidth estimation and monitoring

MISSING IMAGE

Material Information

Title:
End-to-end available bandwidth estimation and monitoring
Physical Description:
Book
Language:
English
Creator:
Guerrero Santander, Cesar
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Hidden Markov model
Traceband
Network measurement
Moving average
Network testbed
Dummynet
Dissertations, Academic -- Computer Science and Engineering -- Doctoral -- USF   ( lcsh )
Genre:
bibliography   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: Available Bandwidth Estimation Techniques and Tools (ABETTs) have recently been envisioned as a supporting mechanism in areas such as compliance of service level agreements, network management, traffic engineering and real-time resource provisioning, flow and congestion control, construction of overlay networks, fast detection of failures and network attacks, and admission control. However, it is unknown whether current ABETTs can run efficiently in any type of network, under different network conditions, and whether they can provide accurate available bandwidth estimates at the timescales needed by these applications. This dissertation investigates techniques and tools able to provide accurate, low overhead, reliable, and fast available bandwidth estimations. First, it shows how it is that the network can be sampled to get information about the available bandwidth. All current estimation tools use either the probe gap model or the probe rate model sampling techniques.Since the last technique introduces high additional traffic to the network, the probe gap model is the sampling method used in this work. Then, both an analytical and experimental approach are used to perform an extensive performance evaluation of current available bandwidth estimation tools over a flexible and controlled testbed. The results of the evaluation highlight accuracy, overhead, convergence time, and reliability performance issues of current tools that limit their use by some of the envisioned applications. Single estimations are affected by the bursty nature of the cross traffic and by errors generated by the network infrastructure. A hidden Markov model approach to end-to-end available bandwidth estimation and monitoring is investigated to address these issues. This approach builds a model that incorporates the dynamics of the available bandwidth. Every sample that generates an estimation is adjusted by the model.This adjustment makes it possible to obtain acceptable estimation accuracy with a small number of samples and in a short period of time. Finally, the new approach is implemented in a tool called Traceband. The tool, written in ANSI C, is evaluated and compared with Pathload and Spruce, the best estimation tools belonging to the probe rate model and the probe gap model, respectively. The evaluation is performed using Poisson, bursty, and self-similar synthetic cross traffic and real traffic from a network path at University of South Florida. Results show that Traceband provides more estimations per unit time with comparable accuracy to Pathload and Spruce and introduces minimum probing traffic. Traceband also includes an optional moving average technique that smooths out the estimations and improves its accuracy even further.
Thesis:
Dissertation (Ph.D.)--University of South Florida, 2009.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Cesar Dario Guerrero Santander.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 130 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 002220518
oclc - 645542150
usfldc doi - E14-SFE0002863
usfldc handle - e14.2863
System ID:
SFS0027180:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200397Ka 4500
controlfield tag 001 002220518
005 20100701175802.0
007 cr cnu|||uuuuu
008 100701s2009 flua sb 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0002863
035
(OCoLC)645542150
040
FHM
c FHM
049
FHMM
090
TK7885 (Online)
1 100
Guerrero Santander, Cesar.
0 245
End-to-end available bandwidth estimation and monitoring
h [electronic resource] /
by Cesar Dario Guerrero Santander.
260
[Tampa, Fla.] :
b University of South Florida,
2009.
500
Title from PDF of title page.
Document formatted into pages; contains 130 pages.
Includes vita.
502
Dissertation (Ph.D.)--University of South Florida, 2009.
504
Includes bibliographical references.
516
Text (Electronic dissertation) in PDF format.
520
ABSTRACT: Available Bandwidth Estimation Techniques and Tools (ABETTs) have recently been envisioned as a supporting mechanism in areas such as compliance of service level agreements, network management, traffic engineering and real-time resource provisioning, flow and congestion control, construction of overlay networks, fast detection of failures and network attacks, and admission control. However, it is unknown whether current ABETTs can run efficiently in any type of network, under different network conditions, and whether they can provide accurate available bandwidth estimates at the timescales needed by these applications. This dissertation investigates techniques and tools able to provide accurate, low overhead, reliable, and fast available bandwidth estimations. First, it shows how it is that the network can be sampled to get information about the available bandwidth. All current estimation tools use either the probe gap model or the probe rate model sampling techniques.Since the last technique introduces high additional traffic to the network, the probe gap model is the sampling method used in this work. Then, both an analytical and experimental approach are used to perform an extensive performance evaluation of current available bandwidth estimation tools over a flexible and controlled testbed. The results of the evaluation highlight accuracy, overhead, convergence time, and reliability performance issues of current tools that limit their use by some of the envisioned applications. Single estimations are affected by the bursty nature of the cross traffic and by errors generated by the network infrastructure. A hidden Markov model approach to end-to-end available bandwidth estimation and monitoring is investigated to address these issues. This approach builds a model that incorporates the dynamics of the available bandwidth. Every sample that generates an estimation is adjusted by the model.This adjustment makes it possible to obtain acceptable estimation accuracy with a small number of samples and in a short period of time. Finally, the new approach is implemented in a tool called Traceband. The tool, written in ANSI C, is evaluated and compared with Pathload and Spruce, the best estimation tools belonging to the probe rate model and the probe gap model, respectively. The evaluation is performed using Poisson, bursty, and self-similar synthetic cross traffic and real traffic from a network path at University of South Florida. Results show that Traceband provides more estimations per unit time with comparable accuracy to Pathload and Spruce and introduces minimum probing traffic. Traceband also includes an optional moving average technique that smooths out the estimations and improves its accuracy even further.
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
590
Advisor: Miguel A. Labrador, Ph.D.
653
Hidden Markov model
Traceband
Network measurement
Moving average
Network testbed
Dummynet
690
Dissertations, Academic
z USF
x Computer Science and Engineering
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.2863



PAGE 1

End-to-EndAvailableBandwidthEstimationandMonitoring by CesarDarioGuerreroSantander Adissertationsubmittedinpartialfulllment oftherequirementsforthedegreeof DoctorofPhilosophy DepartmentofComputerScience&Engineering CollegeofEngineering UniversityofSouthFlorida MajorProfessor:MiguelA.Labrador,Ph.D. KennethChristensen,Ph.D. TapasK.Das,Ph.D. AdrianaIamnitchi,Ph.D. RafaelPerez,Ph.D. DateofApproval: February20,2009 Keywords:HiddenMarkovModel,Traceband,NetworkMeasure ment,MovingAverage, NetworkTestbed,Dummynet c r 2009,CesarDarioGuerreroSantander

PAGE 2

Tomyfamily

PAGE 3

AcknowledgementsIwouldliketogratefullythankmymentorandadvisorDr.Mig uelLabradorforhisguidance,patience,andmostimportantly,hisfriendshipdurin gmygraduatestudies.Hegave metheopportunitytogobeyondmyacademicandresearchduti esandtomakethisa meaningfulpersonalandprofessionalexperience.Iamalso thankfultohimforsupportingmeasResearchAssistant.IwouldliketothanktheUniversityofSouthFlorida,Univer sidadAutonomadeBucaramangaandtheColombianFulbrightCommissionforsupportin gthiswork.Iwouldalso liketosincerelythankDr.Christensen,Dr.Perez,Dr.Iamn itchi,andDr.Dasforbeing partofmycommitteeandfortheirvaluablecommentsandsugg estions. IwanttoexpressmygratitudetomywifeMarcela.Herlove,pa tience,understanding, andencouragementenabledmetocompletethisdream.Thanks forSarah.Thisdissertationisdedicatedtothem.Ialsothankmyparentsandsisters ,fortheirloveandsupportat everystepofmylife.Finally,andmostimportantly,IthankGodforguidingmylif eandmydreams.

PAGE 4

TableofContentsListofTables v ListofFigures vii Abstract xiii Chapter1Introduction 1 1.1Background 2 1.1.1End-to-EndPath 3 1.1.2End-to-EndAvailableBandwidth31.1.3AvailableBandwidthEstimation5 1.2WhyistheEstimationoftheAvailableBandwidthDifcul t?6 1.2.1SystemTiming 7 1.2.2End-hostThroughput 9 1.2.3End-to-EndPathologies91.2.4QueuingBehavior 10 1.3ProblemStatement 10 1.4Contributions 11 1.5OrganizationoftheDissertation12 i

PAGE 5

Chapter2LiteratureReview 13 2.1AvailableBandwidthEstimationTechniques13 2.1.1ProbeGapModel(PGM)142.1.2ProbeRateModel(PRM)15 2.2AvailableBandwidthEstimationTools16 2.2.1Spruce 16 2.2.2Abing 17 2.2.3IGI 19 2.2.4Pathload 22 2.2.5Pathchirp 25 Chapter3EvaluationofCurrentAvailableBandwidthEstima tionTools27 3.1PerformanceMetrics 29 3.2Testbed 31 3.3Analytically-BasedAvailableBandwidthEvaluation33 3.3.1Experiments 35 3.3.2Results 36 3.3.2.1EstimationError373.3.2.2Overhead 43 3.3.2.3EstimationTime45 3.4Experimentally-BasedAvailableBandwidthEvaluation 47 3.4.1PhaseOne:The2 5 FactorialDesign48 ii

PAGE 6

3.4.2PhaseTwo:MainExperiments543.4.3Results 54 3.4.3.1VariableTightLinkCapacity543.4.3.2VariableOne-WayPropagationDelay603.4.3.3VariablePacketLossRates643.4.3.4VariableAmountofCrossTrafc673.4.3.5VariableCrossTrafcPacketSize70 3.5ApplicabilityofCurrentAvailableBandwidthEstimati onTools72 Chapter4HMMApproachtoAvailableBandwidthEstimation78 4.1DiscreteHiddenMarkovModels79 4.1.1HMMElements 80 4.2HMMtoEstimateAvailableBandwidth81 4.2.1ProbingSamplingMethod824.2.2ModelDescription 83 4.2.3ParameterEstimation874.2.4StateSequenceEstimation88 Chapter5Traceband:MonitoringAvailableBandwidth90 5.1TracebandDescription 90 5.1.1TracebandSender 91 5.1.2TracebandReceiver 92 5.2PerformanceEvaluation 92 iii

PAGE 7

5.2.1Synthetic-generatedCrossTrafc94 5.2.1.1PoissonCrossTrafcExperiments955.2.1.2BurstyCrossTrafcExperiments975.2.1.3Self-similarCrossTrafcExperiments99 5.2.2Internet-trafcBasedExperiments1015.2.3MovingAverageAlgorithm101 5.3AdditionalExperiments 107 5.3.1HurstParameter 108 5.3.2TightLinkCapacity1095.3.3NumberofStatesintheHMM109 Chapter6ConclusionsandFutureWork 111 6.1Conclusions 111 6.2FutureWork 113 ListofReferences 115 Appendices 122 AppendixA:FactorialDesignforAvailableBandwidthEvalu ation123 AppendixB:ObservationProbabilityMatrices130 AbouttheAuthor EndPage iv

PAGE 8

ListofTablesTable1.1End-hostNICachievablethroughput.9Table3.1Parametersusedbytheestimationtools.36Table3.2Factorsandlevelsinthe2 5 factorialdesign.48 Table3.3Maineffectintheperformancemetricswhenvaryin gonefactor.50 Table3.4Summaryofbest/worstperformingtools.77Table3.5Qualitativeassessmentofapplicationrequireme ntsandbesttoolsfor thesetofrepresentativeapplications.77 Table4.1AvailablebandwidthHMMvariables.87Table5.1Performanceevaluationfor30%Poissoncrosstraf c.95 Table5.2Performanceevaluationfor30%burstcrosstrafc .99 Table5.3Performanceevaluationfor30%self-similarcros strafc.101 Table5.4Performanceevaluationwithrealcrosstrafcina 100Mbpspath.101 Table5.5Estimationerrorafterapplyingmovingaverageto experimentresults withPoissontrafc. 105 Table5.6Estimationerrorafterapplyingthemovingaverag etoexperiment resultswithself-similartrafc. 107 Table5.7Tracebandperformanceforhighandlowtightlinkc apacities.109 v

PAGE 9

TableA.12 5 factorialdesignmatrix. 124 TableA.2Maineffectintheperformancemetricswhenvaryin gtwofactors.126 TableA.3Maineffectintheperformancemetricswhenvaryin gthreefactors.127 TableA.4Maineffectintheperformancemetricswhenvaryin gfourfactors.128 TableA.5Maineffectintheperformancemetricswhenvaryin gvefactors.129 TableB.1Observationprobabilitymatrixfor5statesand5o bservationsymbols. 130 TableB.2Observationprobabilitymatrixfor10statesand1 0observationsymbols. 130 vi

PAGE 10

ListofFiguresFigure1.1End-to-endcommunicationpath. 3 Figure1.2Availablebandwidthinanaveragetimescaleperi od.4 Figure1.3Narrowandtightlinks. 5 Figure1.4Singlelinkmodelforbandwidthestimation.6Figure2.1IGIturningpoint. 20 Figure2.2Pathloadgrayregion. 23 Figure2.3Chirpqueuingdelaysignature. 25 Figure3.1Testbedtoevaluatebandwidthestimationtools. 31 Figure3.2Networkofqueuesfortheevaluationtestbed.34Figure3.3Pathloadestimationwith0%crosstrafc.38Figure3.4IGIestimationwith0%crosstrafc.38Figure3.5Spruceestimationwith0%crosstrafc.38Figure3.6Pathloadestimationwith25%crosstrafc.39Figure3.7IGIestimationwith25%crosstrafc.39Figure3.8Spruceestimationwith25%crosstrafc.39Figure3.9Pathloadestimationwith50%crosstrafc.40 vii

PAGE 11

Figure3.10IGIestimationwith50%crosstrafc.40Figure3.11Spruceestimationwith50%crosstrafc.40Figure3.12Pathloadestimationwith75%crosstrafc.41Figure3.13IGIestimationwith75%crosstrafc.41Figure3.14Spruceestimationwith75%crosstrafc.41Figure3.15Pathloadrelativeerrorfor0%,25%,50%and75%c rosstrafc.42 Figure3.16IGIrelativeerrorfor0%,25%,50%and75%crosst rafc.42 Figure3.17Sprucerelativeerrorfor0%,25%,50%and75%cro sstrafc.42 Figure3.18Pathloadoverheadfor0%,25%,50%and75%crosst rafc.44 Figure3.19IGIoverheadfor0%,25%,50%and75%crosstrafc .44 Figure3.20Spruceoverheadfor0%,25%,50%and75%crosstra fc.44 Figure3.21Pathloadtimefor0%,25%,50%and75%crosstraf c.46 Figure3.22IGItimefor0%,25%,50%and75%crosstrafc.46Figure3.23Sprucetimefor0%,25%,50%and75%crosstrafc. 46 Figure3.24Maineffectontheresponsevariableswhenfacto rsarevaried.51 Figure3.25Estimationerrorat20%crosstrafcwithvariab lecapacity,10ms OWD,1%PLR. 55 Figure3.26Estimationerrorat75%crosstrafcwithvariab lecapacity,10ms OWD,1%PLR. 55 Figure3.27Convergencetimeat20%crosstrafcwithvariab lecapacity,10ms OWD,1%PLR. 56 Figure3.28Convergencetimeat75%crosstrafcwithvariab lecapacity,10ms OWD,1%PLR. 57 viii

PAGE 12

Figure3.29Overheadat20%crosstrafcwithvariablecapac ity,10msOWD, 1%PLR. 58 Figure3.30Overheadat75%crosstrafcwithvariablecapac ity,10msOWD, 1%PLR. 58 Figure3.31Estimationerrorat5Mbpsforvariableonewayde lay,1%PLR,and 75%crosstrafc. 59 Figure3.32Estimationerrorat100Mbpsforvariableoneway delay,1%PLR, and75%crosstrafc. 59 Figure3.33Estimationtimeat5Mbpsforvariableonewaydel ay,1%PLR,and 75%crosstrafc. 60 Figure3.34Estimationtimeat100Mbpsforvariableonewayd elay,1%PLR, and75%crosstrafc. 61 Figure3.35Overheadat5Mbpsforvariableonewaydelay,1%P LR,and75% crosstrafc. 62 Figure3.36Overheadat100Mbpsforvariableonewaydelay,1 %PLR,and75% crosstrafc. 63 Figure3.37Estimationerrorat5Mbpsforvariablepacketlo ssrate,10msdelay, and75%crosstrafc. 63 Figure3.38Estimationerrorat100Mbpsforvariablepacket lossrate,10ms delay,and75%crosstrafc. 64 Figure3.39Estimationtimeat5Mbpsforvariablepacketlos srate,10msdelay, and75%crosstrafc. 65 Figure3.40Estimationtimeat100Mbpsforvariablepacketl ossrate,10ms delay,and75%crosstrafc. 65 Figure3.41Reliabilityat5Mbpsforvariablepacketlossra te,10msdelay,and 75%crosstrafc. 66 Figure3.42Reliabilityat100Mbpsforvariablepacketloss rate,10msdelay,and 75%crosstrafc. 66 ix

PAGE 13

Figure3.43Estimationerrorat5Mbpsforvariable%ofcross trafc,10ms OWD,and1%PLR. 68 Figure3.44Estimationerrorat100Mbpsforvariable%ofcro sstrafc,10ms OWD,and1%PLR. 68 Figure3.45Estimationtimeat5Mbpsforvariable%ofcrosst rafc,10msOWD, and1%PLR. 69 Figure3.46Estimationtimeat100Mbpsforvariable%ofcros strafc,10ms OWD,and1%PLR. 69 Figure3.47Overheadat5Mbpsforvariable%ofcrosstrafc, 10msOWD,and 1%PLR. 70 Figure3.48Overheadat100Mbpsforvariable%ofcrosstraf c,10msOWD, and1%PLR. 70 Figure3.49Estimationerrorat5Mbpsand20%crosstrafcfo rvariablecross trafcpacketsize,10msOWD,and1%PLR.71 Figure3.50Estimationerrorat5Mbpsand75%crosstrafcfo rvariablecross trafcpacketsize,10msOWD,and1%PLR.71 Figure3.51Estimationerrorat100Mbpsand20%crosstrafc forvariablecross trafcpacketsize,10msOWD,and1%PLR.71 Figure3.52Estimationerrorat100Mbpsand75%crosstrafc forvariablecross trafcpacketsize,10msOWD,and1%PLR.72 Figure4.1AvailablebandwidthMarkovmodel.81Figure4.2HiddenMarkovmodelforavailablebandwidthesti mations.84 Figure5.1Tracebandreceiverpseudocode. 93 Figure5.2TestbedtoevaluatetheperformanceofTraceband .94 Figure5.3Pathloadestimationfora10Mbpstightlinkwith3 0%ofPoisson crosstrafc. 96 x

PAGE 14

Figure5.4Spruceestimationfora10Mbpstightlinkwith30% ofPoissoncross trafc. 96 Figure5.5Tracebandestimationfora10Mbpstightlinkwith 30%ofPoisson crosstrafc. 97 Figure5.6Pathloadestimationfora10Mbpstightlinkwith3 0%ofburstycross trafc. 97 Figure5.7Spruceestimationfora10Mbpstightlinkwith30% ofburstycross trafc. 98 Figure5.8Tracebandestimationfora10Mbpstightlinkwith 30%ofbursty crosstrafc. 98 Figure5.9Pathloadestimationfora10Mbpscapacityand30% self-similar crosstrafc. 100 Figure5.10Spruceestimationfora10Mbpscapacityand30%s elf-similarcross trafc. 100 Figure5.11Tracebandestimationfora10Mbpscapacityand3 0%self-similar crosstrafc. 100 Figure5.12Pathloadestimationfora100Mbpstightlinkwit hInternetcross trafc. 102 Figure5.13Spruceestimationfora100MbpstightlinkwithI nternetcrosstrafc. 102 Figure5.14Tracebandestimationfora100Mbpstightlinkwi thInternetcross trafc. 102 Figure5.15MovingaveragepostprocessingtoPathloadexpe rimentswithPoissoncrosstrafc. 103 Figure5.16MovingaveragepostprocessingtoSpruceexperi mentswithPoisson crosstrafc. 104 Figure5.17MovingaveragepostprocessingtoTracebandexp erimentswithPoissoncrosstrafc. 104 xi

PAGE 15

Figure5.18Movingaveragealgorithm. 105 Figure5.19MovingaveragepostprocessingtoPathloadexpe rimentswithselfsimilarcrosstrafc. 106 Figure5.20MovingaveragepostprocessingtoSpruceexperi mentswithselfsimilarcrosstrafc. 106 Figure5.21MovingaveragepostprocessingtoTracebandexp erimentswithselfsimilarcrosstrafc. 107 Figure5.22EffectoftheHurstparameterintheestimatione rror.108 Figure5.23Tracebandestimationerrorandtimefordiffere ntnumberofstates.110 xii

PAGE 16

End-to-EndAvailableBandwidthEstimationandMonitoringCesarDarioGuerreroSantanderABSTRACTAvailableBandwidthEstimationTechniquesandTools(ABET Ts)haverecentlybeen envisionedasasupportingmechanisminareassuchascompli anceofservicelevelagreements,networkmanagement,trafcengineeringandreal-ti meresourceprovisioning, owandcongestioncontrol,constructionofoverlaynetwor ks,fastdetectionoffailures andnetworkattacks,andadmissioncontrol.However,itisu nknownwhethercurrent ABETTscanrunefcientlyinanytypeofnetwork,underdiffe rentnetworkconditions, andwhethertheycanprovideaccurateavailablebandwidthe stimatesatthetimescales neededbytheseapplications.Thisdissertationinvestigatestechniquesandtoolsablet oprovideaccurate,lowoverhead,reliable,andfastavailablebandwidthestimations. First,itshowshowitisthat thenetworkcanbesampledtogetinformationabouttheavail ablebandwidth.Allcurrentestimationtoolsuseeithertheprobegapmodelorthepr oberatemodelsampling techniques.Sincethelasttechniqueintroduceshighaddit ionaltrafctothenetwork, theprobegapmodelisthesamplingmethodusedinthiswork.T hen,bothananalytical andexperimentalapproachareusedtoperformanextensivep erformanceevaluationof currentavailablebandwidthestimationtoolsoveraexibl eandcontrolledtestbed.The xiii

PAGE 17

resultsoftheevaluationhighlightaccuracy,overhead,co nvergencetime,andreliability performanceissuesofcurrenttoolsthatlimittheirusebys omeoftheenvisionedapplications.Singleestimationsareaffectedbytheburstynature ofthecrosstrafcandbyerrors generatedbythenetworkinfrastructure.AhiddenMarkovmodelapproachtoend-to-endavailableband widthestimationand monitoringisinvestigatedtoaddresstheseissues.Thisap proachbuildsamodelthat incorporatesthedynamicsoftheavailablebandwidth.Ever ysamplethatgeneratesanestimationisadjustedbythemodel.Thisadjustmentmakesitp ossibletoobtainacceptable estimationaccuracywithasmallnumberofsamplesandinash ortperiodoftime. Finally,thenewapproachisimplementedinatoolcalledTra ceband.Thetool,writtenin ANSIC,isevaluatedandcomparedwithPathloadandSpruce,t hebestestimationtools belongingtotheproberatemodelandtheprobegapmodel,res pectively.Theevaluation isperformedusingPoisson,bursty,andself-similarsynth eticcrosstrafcandrealtrafc fromanetworkpathatUniversityofSouthFlorida.Resultss howthatTracebandprovides moreestimationsperunittimewithcomparableaccuracytoP athloadandSpruceand introducesminimumprobingtrafc.Tracebandalsoinclude sanoptionalmovingaverage techniquethatsmoothsouttheestimationsandimprovesits accuracyevenfurther. xiv

PAGE 18

Chapter1:IntroductionThenewcenturyhasseenacontinuousincreasingnumberofIn ternetusersandnetwork applications.Internetusershavegrownaround300%from20 00to2008[1][2]andnetworkapplicationshavegrownfromemailtovoiceoverIP,vid eosteaming,peertopeer (P2P)letransfers,overlaynetworks,amongothers.Forso meofthesenetworkapplications,informationabouttheavailablebandwidthcanbeu sedtoimprovetheirperformance.Forexample,networkmanagementtoolsthatmonit orlargenetworkedsystemscanuseavailablebandwidthdatatoshowthecurrentuti lizationofthenetworkresources.Internetserviceprovidersanduserscanmonitora ndverifyservicelevelagreements(SLA)tomanagetheircontracts.Trafcengineeringm echanismswouldbeableto performreal-timeresourceprovisionningwhilebalancing theloadofthenetwork.Call admissioncontrolmechanismsmighttakeadvantageofavail ablebandwidthinformation toeitheradmitorrejectanewincomingconnection,avoidin gnetworkcongestionand guaranteeingthequalityofserviceofcurrentandnewconne ctions.Overlaynetworks coulddeterminethemostappropriatetopologybasedonavai lablebandwidthinformation. Transportlayerprotocolsmightdecidetochangethetransm issionrateaccordingtothe amountofbandwidthavailableinthepath,usingthenetwork resourcesefcientlywhile avoidingcongestion.Also,networkavailablebandwidthin formationcouldbeanimportantindicatortodetectnetworkfailuresandmaliciousatt acks. 1

PAGE 19

Althoughtheenvisionedusefulnessofavailablebandwidth informationisnotinquestion, currentavailablebandwidthestimationtoolscannotbeuse dbymostofthenetworkapplicationsrequiringtheestimation[3].Moreover,theyca nnotbeusedineverynetwork scenario.Forexample,whileanavailablebandwidthestima tionwitha10%errorcould beconsideredwithinacceptablevaluesforaroutingprotoc oltomakeroutingdecisions, itmaybecompletelyunacceptableforSLAverication.Simi larly,itmaybejustne foratooltotakeseveralsecondsorevenminutestoprovidea nestimateforanetwork managementsystem,butitwouldbeuselessforatransportla yerprotocoltomakerate changingdecisions.Finally,althoughitmaynotbeabigiss uetouseaveryintrusive availablebandwidthestimationtoolinanopticalnetwork, itmayconsumeveryscarce andpreciousresourcesinawirelessmobileadhocnetwork.Thisdissertationstudiescurrentavailablebandwidthest imationtechniquesandtoolsand proposesanovelaccurate,low-overhead,reliable,andfas testimationapproachthathas beenimplementedinanavailablebandwidthestimationtool called Traceband 1.1BackgroundIncomputernetworks, bandwidth isaratemeasuredenedastheamountofbitstransmittedinacommunicationchannelperunittime.Itisgenerally speciedinbitspersecond (bps).Therearetwodifferentmetricsrelatedtobandwidth .Oneisthe capacity andthe otheroneisthethe availablebandwidth .Thecapacityofalinkisthemaximumamount ofbitsthatcanbetransmittedtothelinkperunittime.That is,themaximumbandwidth. Theavailablebandwidthisthesparecapacity. 2

PAGE 20

n r Figure1.1:End-to-endcommunicationpath. 1.1.1End-to-EndPathAnend-to-endcommunicationpathisasingleroutethatconn ectstwoendhoststhrough asetofcommunicationlinksorhopsconnectedvianetworkde vices.Althoughthatroute canchange,ithasbeenshownbyPaxson[4]andZhang[5]thate nd-to-endpathsbetween Internethostsarestableonscalesrangingfromhourstoday s. AsitisshowninFigure1.1,senderandreceiverendhostsare communicatedthrough foursinglelinks(orfourhops)connectedviaroutersA,B,a ndC.Eachlinkinthepath hasaparticularcapacitydeterminedbytheNetworkInterfa ceController(NIC)attached tothecorrespondingnetworkdeviceinthepath.Forexample ,thecapacityofthelink betweenroutersBandCisdeterminedbytheNIC'smaximumtra nsmissionrateinrouter BconnectedtorouterC.1.1.2End-to-EndAvailableBandwidthTheminimumofallnon-utilizedlinkcapacitiesthroughout thecommunicationpath iscalledtheend-to-endavailablebandwidth.Thisisatime -varyingmetricrelatedto theindividualutilizationofeachlinkthroughoutthepath .Deningtasthe averaging 3

PAGE 21

t AB(t) tCapacity (C) Figure1.2:Availablebandwidthinanaveragetimescaleper iod. timescale oftheavailablebandwidth[6],theaverageutilizationofl ink i forasampleof timet,isgivenby u i = 1 tZ t +tt u i ( s ) ds ; 0 u i 1 : (1.1) Foralink i withcapacity C i ,theavailablebandwidthofthelinkintheinterval ( t ; t +t) canbedenedastheaveragenon-utilizedcapacityduringth etimet(seeFigure1.2): A i = C i [ 1 u i ] : (1.2) Foranend-to-endpathwith H hops,theavailablebandwidthduringtisgivenbythelink withtheminimumnon-utilizedcapacityofallhops,asfollo ws: A = min i = 1 :: H ( A i ) : (1.3) AsitisshowninFigure1.3,thelinkwiththeminimumcapacit yisknownasthe narrow link andthelinkwiththeminimumavailablebandwidthisknownas the tightlink ,which 4

PAGE 22

nrn Figure1.3:Narrowandtightlinks. isconsideredthebottleneckofthepathandthelinkthatdet erminestheend-to-endavailablebandwidth.1.1.3AvailableBandwidthEstimationToestimatetheavailablebandwidthinanend-to-endpathit isnecessarytosamplethe networkbysendingprobingpackets.Althoughmostoftheava ilablebandwidthtools generatethosepacketsasadditionaltrafcinthenetwork, Man etal. [7][8]proposethe useofcarefullyselectedanddelayeddatapacketstoservea sprobingpacketswithout insertingadditionaltrafctothenetwork.Fromtheanalysisofthedelaysthatprobingpacketssufferw henpassingthroughthe tightlink,theavailablebandwidthcanbedetermined.Theb ehaviorofaprobingpacket pairafterleavingthetightlinkisshowninFigure1.4.This singlelinkmodelisbasedon theassumptionofasinglequeuefollowingtherst-come-r st-serveddiscipline.Asitis shownintheFigure,iftwoconsecutivepacketsaresenttoth enetworkpath,theyarriveto thenodewithadeterminedinitialtime-separationbetween them( D in ).Afterinteracting inthetightlinkqueuewiththecrosstrafccomingfromdiff erentsources,thepairof 5

PAGE 23

Cross traffic Probing Packets in out Tight link capacity Ct out out= in out> in out out< in Figure1.4:Singlelinkmodelforbandwidthestimation. p robingpacketswillleavetherouterwithanewtime-separation( D out ).Thedifference betweenthem D out )Tj/T1_3 11.955 Tf10.92 0 Td(D in isthepacketpairdispersion. Thepacketpairdispersioncanbenegative,positive,orequaltozero.Asitisshownin Figure1.4,anegativevalue( D out < D in )occurswhentherstpacketndscrosstrafc packetsinthequeuefollowedbythesecondpacket.Apositivevalue( D out > D in )occurs whencrosstrafcpacketsareinsertedbetweentheprobingpacketpairinthequeue. Finally,avalueofzero( D out = D in )occurswhenthelinkhasnoenoughcrosstrafcto affecttheinitialpacketseparation. Basedonthetightlinkmodel,therearetwodifferentapproachestoestimatetheavailable bandwidthinanend-to-endpath:theprobegapmodel(PGM)andtheproberatemodel (PRM).PGMobservesprobingpacketpairdispersionstoestimatetheamountofcross trafc.PRMobservesvariationsintheprobingpacketonewaydelaytodeterminethe availablebandwidth.BothmodelswillbedescribedinSection2.1. 1.2WhyistheEstimationoftheAvailableBandwidthDifcult? Theestimationoftheavailablebandwidthisdifcultfortwomainreasons.Oneisthe burstnatureofthecrosstrafcandtheotherisrelatedtoerrorsgeneratedbyendhosts 6

PAGE 24

androutersalongtheend-to-endpath.Duetotheburstnatur eofthecrosstrafc,asingle pairofprobingpacketscannotcapturetheaveragetrafclo adinthesinglelinkmodel describedbefore.Todealwiththisproblem,estimationtoo lsbasedonboththePGMand PRMapproachesuseatrainofprobingpacketstogenerateasi ngleaveragedmeasurement.Thissolutionisalsousedinthetoolpresentedaspart ofthiswork. Inaddition,endhostsandroutersaresourcesoferrorstoth eestimationtools[9][10] [11].Incorrectpackettimestamps,poorNICutilization,o ut-of-orderpacketdelivery, packetreplication,packetcorruption,andchangingqueui ngbehaviorsaffecttheaccuracy oftheestimationperformedbyasinglepairofprobingpacke ts.Mostoftheseerrorscan becorrectedinacontrolledtestingenvironmentbutnotina realscenariowhereusers canruntheestimationapplicationbuthavenoknowledgeabo uthowtoconveniently setuptheirmachines.Theestimationtoolpresentedinthis workdoesnotpreventthese errorstooccurbutbuildsamodeloftheavailablebandwidth tostatisticallyadjustthe erraticmeasurements.Thefollowingsectionsprovideamor edetailedexplanationof theseerrors.1.2.1SystemTimingWhensendingprobingpacketsandmeasuringtheirgapsorrat esatthereceiver,accurate timingisrequired.Thetimeatwhichpacketsaretimestampe drightbeforea"sendto()" socketfunctionatthesenderisdifferenttothetimeatwhic hthepacketisactuallysent. Similarly,thetimeatwhichapacketisseenbythereceiver' sNICisdifferentfromthe timeitisreportedtotheestimationapplication.Severalf actorscontributetotimingerrors: 7

PAGE 25

Timerresolutions inmostoperatingsystemsarearound1 ms .Sendingandtime stampingprobingpacketswillusuallyrequireabetterreso lution.Forexample, thetransmissiontimeofa100-Bytespacketina100Mbpslink is8ms .Iftwoof thosepacketsaresentback-to-back,theirtimestampswoul dbeidenticalifa1 ms resolutiontimerisinplace. Contextswitch canaffectthegaptimebetweenpacketswhentheestimationp rocess isabruptlysuspendedbytheoperatingsystemtoassigntheC PUtoanotherprocess. Specically,mostoftheestimationtoolstransmitatraino fpacketsatdesiredtime byreadingthesystemclockinapollingloopuntilthewholet rainiscompletely sent.Ifthisloopisinterruptedbyacontextswitchingeven t,timinginsidethatloop willbeunreliable. Interruptcoalescence (IC)[12][13]isamechanismimplementedinmostofhighspeednetworkinterfacecardsthataffectsappropriatepro bingpacketstime-stamping atthereceiversideoftheestimationapplication.Thismec hanismdelaysthegenerationofaCPUinterruptionforeverypacketarrivingtotheN IC.Instead,ICstores intheNICseveralpacketsbeforeinterruptingtheCPUtonot ifytheirarrival.Therefore,allpacketsreportedinasingleinterruptionwillhav ethesameincorrecttimestampattheapplicationlevel. Systemcalldelays duetotime-querieslike"gettimeofday()"andsocketoperationslike"sendto()"or"receivefrom()"addseveralmicrosecondstothemeasurements.Formostoperatingsystemsandcomputerarchite ctures,gettimeofday()takesabout1ms andsendto()andreceivefrom()about40ms each. 8

PAGE 26

Table1.1:End-hostNICachievablethroughput. Operating CPU NICachievablethroughput System 10Mbps 100Mbps 1000Mbps Linux2.4.1 IntelP42.0GHz 8.8 77.2 323.0 Linux2.4.1 AMDAthlonXP2500+ 8.9 78.3 340.2 FreBSD4.8 IntelP42.0GHz 8.8 72.5 299.2 MacOSX10.2 PowerG41GHz 8.6 68.6 256.7 WindowsXPSP2 IntelP42.4GHz 8.8 70.6 280.2 1.2.2End-hostThroughputEstimationapplicationsrequireprobingpacketstobesent ataspecicrate.However, NICachievablethroughputisinferiortoitsrealcapacity. Table1.1showstheresults ofastudyperformedin[9]testingtheachievablethroughpu tforsixdifferentoperating systemsinstalledondifferentcomputers.Forexample,a10 MbpsNICcannotachieve morethan8.9Mbps.Infact,thegreaterthecapacitythelowe rtheNICutilization.A 1000MbpsNICcannotachievemorethan340Mbps.ThispoorNIC utilizationisinpart causedbecauseendhostsaregeneral-purposepersonalcomp uters.Network-orienteddevicessuchasroutersarehardwareandsoftwaredesignedtoa chieveveryhighthroughput. 1.2.3End-to-EndPathologiesNetworkpathologiesasnamedbyPaxsonin[10]arereferredt ounusualorunexpected networkeventslikeout-of-orderdelivery,replication,a ndcorruptionthataffectmost oftheestimationmechanisms.Out-of-orderdeliveryisani ssuesincemostestimation methodsassumethatpacketswillkeepthedeliverysequence establishedbytheFIFO queuingpolicyintherouters.However,anytimearoutechan ges,ifthenewrouteoffersa 9

PAGE 27

lowerdelaythantheoldone,thenpacketscouldbereceivedi nadifferentorder[14].The reorderedpacketswillinparticularaffectestimationbas edongapsbetweenpacketpairs sincethepairssequencewillbelost.Packetreplicationoccurswhenthenetworkdeliversmultip lecopiesofthesamepacket. Packetcorruptionoccurswhenthenetworkdeliversanincor rectcopyoftheoriginal packet.Bothpathologiesareobviouslyharmfulwhentheyoc curinprobingpacketsused tosampletheavailablebandwidth.1.2.4QueuingBehaviorThetightlinkmodelassumesasinglequeuefollowingthers t-come-rst-serveddiscipline.Thisisnotalwaysthecase.Manyroutershaveimpleme ntedweightedfairqueuing (WFQ)mechanismsthatcanchangethedeliveringorderofthe receivedpackets.Inaddition,duetotheburstbehaviorofthecrosstrafc,itissh owninFigure1.4,thattherst ofaprobingpacketpairmightndthequeuebusy.Thiseventw illproduceanegative dispersionbetweenpacketpairsthatisignoredbymostofth eestimationmechanisms. 1.3ProblemStatementEstimatingtheavailablebandwidthinanend-to-endpathis requiredbyseveralnetwork applicationstoimprovetheirperformance.However,thees timationaccuracyisaffected bytheburstnatureofcrosstrafcanderrorsassociatedtot henetworkinfrastructure. Theseissuesforcetheestimationtoolstocollectseverals amplesfromthenetworkto provideanaverageavailablebandwidthwithincertainacce ptedvaluesofaccuracy.This 10

PAGE 28

increaseinaccuracybysendingseveralprobingpacketsinc reasesalsotheoverheadand thetimespenttoreportaresult.Similartrade-offshaveto bemadetoprovidetheconvergencetime,overhead,andreliabilityrequiredbytheappli cations. Thisdissertationclaimsthataccurate,non-intrusive,re liableandfastend-to-endavailable bandwidthestimationcanbeachievedbysendingprobingpac ketpairstothenetwork. 1.4ContributionsThisdissertationmakesthefollowingcontributionscompl etelydescribedintheremaining chaptersofthismanuscript. Anextensiveevaluationofcurrentavailablebandwidthest imationtools. Previousevaluationsofavailablebandwidthestimationto ols[15][16][17]were performedoverlimitednumberoftoolsandnetworkscenario s.Thisworkpresents anevaluationofthemostimportantestimationtoolsusinga nalyticandexperimentalrepresentationsoftherealavailablebandwidthandafa ctorialdesigntechnique todeterminethemostrelevantexperimentstobeperformedo vernetworkscenarios nevertestedbefore.Inaddition,thisstudyisuniquesince itconcludesaboutthe usefulnessoftheestimationtoolsfromthenetworkapplica tionspointofview.The resultsofthiscontributionarepresentedin[18]and[3]. Anend-to-endavailablebandwidthestimationmodel.ThisistherstworkthatusesahiddenMarkovmodelapproach torepresentand estimatetheavailablebandwidth.Thisapproachcombinedw ithamovingaverage techniquereducesthenumberofsamplesrequiredandprovid esafastandaccurate estimation.Thisnovelestimationapproachhasbeenpublis hedin[19]. 11

PAGE 29

Anovelend-to-endavailablebandwidthestimationtool.Basedontheestimationmodel,anewtoolcalledTracebandis builttoprovidefast, reliable,accurateandlowoverheadestimations.Thistool istheonlytoolableto accuratelymonitortheavailablebandwidthwithagranular itynevershownbefore. Tracebanddescriptionispresentedin[20]. Aresearchandteachingnetworkinfrastructure.Afullycontrolledtestbedisbuilttoperformtheevaluatio ns.Thistestbedallowsto emulatedifferentnetworkscenariosneverstudiedinprevi ousavailablebandwidth evaluations.Thisinfrastructureiscurrentlyusedforper formanceevaluationinthe ComputerNetworksgraduateclassandhasbeenusedtoperfor mtheevaluations shownin[21],[18],and[3]. 1.5OrganizationoftheDissertationTheremainderofthedissertationisorganizedasfollows:C hapter2presentsthetwo availablebandwidthestimationtechniquesandthecurrent toolsdevelopedusingthese twoapproaches.Chapter3presentsanextensiveanalytical andexperimentalevaluation ofcurrentavailablebandwidthestimationtoolsinscenari osneverevaluatedbefore.Itis shownhowafactorialdesigntechniquehelpstoconsiderabl yreducethenumberofexperimentsrequiredintheevaluation.Chapter4presentsthehi ddenMarvovmodelapproach usedinthisdissertationastheavailablebandwidthestima tionmodeltobeimplemented inanewestimationtool.Chapter5describestheoperationa ndimplementationofanew availablebandwidthestimationtoolcalledTraceband.Cha pter6concludesthedissertationandpresentsdirectionforfutureresearch. 12

PAGE 30

Chapter2:LiteratureReviewTheproblemofbandwidthestimationhasbeenstudiedforsev eralyearsbymanyauthors. TherstapproachbyKeshavonpacket-pairowcontrol[22]r eliedonfairqueuingin allnetworkrouterstoestimatebandwidthbysendingback-t o-backprobingpackets.Jacobson[23]proposedtouseACKpacketstoestimatebandwidt hbasedonthespacing betweenthem.Carterintroducedcprobe[24]whichsendsash orttrainofICMPecho packetsbetweentwohostsandusesthespacingbetweenther standlastreturningpacket toestimatetheavailablebandwidth.Later,Dovrolis[25]p ointedoutthatwhatcprobe measuresistheasymptoticdispersionrate(ADR),whichisd ifferenttotheavailable bandwidth.AsimilarapproachwasproposedbyJin[26]inato olcalledpipechar. Thesestudieshavetriggeredthedevelopmentofavailableb andwidthestimationtools forthelastsevenyears.Thischapterintroducestheavaila blebandwidthestimationapproachescurrentlyusedandtheestimationtoolsdeveloped uponthem. 2.1AvailableBandwidthEstimationTechniquesTherearetwodifferentapproachestoestimatetheavailabl ebandwidthinanend-to-end path:theprobegapmodel(PGM)andtheproberatemodel(PRM) .PGMobservesprobingpacketpairdispersionswhilePRMobservesonewaydelay sintheprobingpackets. 13

PAGE 31

Bothapproachesutilizeatrainofprobingpacketstogenera teanaveragedestimationand copeinthatwaywiththeburstinessnatureofcrosstrafc.Althoughnotnecessaryfortheestimationtoolstowork,two assumptionsarerequiredto holdfortheanalyticalvalidityoftheestimationmodels: RoutersalongthepathexhibitaFIFOqueuingdiscipline. Thesingle-linkmodelshowninFigure1.4isonewherethecro sstrafcrateis constantduringtheaveragingtimescalet. 2.1.1ProbeGapModel(PGM)Thismodelbasestheestimationonthegapdispersionbetwee ntwoconsecutiveprobing packetsatthereceiver,whichhasastrongcorrelationwith theamountofcrosstrafcin thetightlink.Thedispersionincreaseslinearlywiththec rosstrafcrateifthequeueof thetightlink(Figure1.4)doesnotbecomeemptyafterther stpacketofthepairleaves therouterandbeforethesecondpacketarrivesattherouter [27].Therefore,theavailable bandwidthisestimatedbydeterminingtheamountofcrosstr afcandsubtractingitfrom theknowncapacityofthetightlink: A = C ( 1 e) (2.1) whereeistherelativedispersionor strain [28]denedby:e= D out D in D in : (2.2) 14

PAGE 32

Examplesofavailableestimationtoolsbasedontheprobega pmodelapproachare Spruce [29],Abing[30]and IGI [27]. 2.1.2ProbeRateModel(PRM)Thisisamodelbasedontheideaof inducedcongestion ,inwhichthetoolssendprobe packettrainsatincreasingratesandthereceiverobservev ariationsintheaveragetrain onewaydelaylookingfortheturningpoint,orthepointatwh ichthedelayoftheprobe packetsstartsincreasinginaconsistentbasis.Ifatraini ssentataratelessthanthepath availablebandwidth,thetrainwillexperiencesimilardel ays.Ontheotherhand,ifthe trainrateisgreaterthanthepathavailablebandwidth,the trainwillqueueinthetightlink routerandwillexperienceincreasingdelays(turningpoin t).Theavailablebandwidthis thenestimatedlookingattheprobepacketrateutilizedwhe ntheturningpointisfound. Atthispoint,thetrainrateisequaltotheavailablebandwi dthintheend-to-endpath. Examplesoftoolsintheproberatemodelare Pathload [31]and Pathchirp [32]. Thismethodwasinitiallyknownasthetrainofpacketpair(T OPP)mechanismasdened byMelander[33][34].Heproposedtoinjectpairsofprobepa cketsintothenetwork andobservesatthereceiverthereceptiontimesoftheprobe packetstoestimateavailablebandwidth.Thesenderstartstransmittingasetof n separatedpairsofequallysized packets L atsomerate R min .Thisrateisthenincreasedandanothertrainissent.Thisg oes onuntilthemaximumprobingrate R max isreached.Fromtherelationbetweentheinput andoutputrates,theavailablebandwidthisestimated.TOP Pwasonlysimulatedusing thenetworksimulator ns-2 [35]. 15

PAGE 33

2.2AvailableBandwidthEstimationToolsThissectiondescribestheestimationtoolsaspresentedby theirauthors.Theperformance ofallthetoolswillbeevaluatedinChapter3.Thebasicnota tionusedtoexplainthe operationofthetoolsisbasedonthesinglelinkmodelprese ntedinSection1.1.3.The rstthreetoolspresentedhereusetheprobegapmodelasest imationapproachandlas remainingtwotheproberatemodel.2.2.1SpruceSpruce[29]usestheprobegapmodelapproachtoperformthee stimation.Itsendsa Poissonsampleof1500BUDPpairsofpacketswithanintra-pa irgapequaltothenarrow linktransmissiontimeofa1500Bpacket.Thatguaranteesth atthesecondpacketarrives tothenarrowlinkqueuebeforetherstpacketleavesthatqu eue.Bysettingtheinterpairgaptotheoutputofanexponentiallydistributedfunct ion,SpruceperformsaPoisson samplingprocessthatallowsthetooltobenon-intrusive.Usingthedispersionoftheprobepacketsmeasuredattherec eiver,Sprucecalculatesthe averagerateofthetrafcthatarrivestothequeuebetweent hetwopacketsasthecapacity ofthetightlink C t multipliedbytherelativedispersionobtainedfromEquati on2.2. Theavailablebandwidthisdeterminedbysubtractingthatc rosstrafcratefromthecapacityinthetightlink.Afterperforming K samplemeasurements,thetoolreportsthe averageofalltheavailablebandwidthscalculated.Thedef aultvaluefor K is100.Spruce estimationrequiresapreviouscalculationofthetightlin kcapacity. 16

PAGE 34

Aspresentedbytheauthors,Sprucecanbedistinguishedfro motheravailablebandwidth toolsbythefollowingaspects: SpruceusesaPoissonprocessofpacketpairsratherthanpac kettrains(orchirps). ThisformofsamplingallowsSprucetobebothnon-intrusive androbust. Bycarefullychoosingthevalueoftheinitialgap,Spruceen suresthatthebottleneck queuedoesnotemptybetweenthetwoprobesinapair,whichis arequirementfor thecorrectnessofthegapmodel. Spruceseparatescapacitymeasurementfromavailableband widthmeasurement.It assumesthatcapacitycanbemeasuredeasilywithoneofthec apacitymeasurement toolsandthatcapacitystaysstablewhenmeasuringavailab lebandwidth. Sprucedoesnotoverwhelmthenarrowlinkonapathbecauseit sproberateisno moretrafcthantheminimumof240Kb/sand5%ofthecapacity ofthenarrow link. ApartfromthenumberofpairsKoverwhichtoaveragethemeas urements,Spruce doesnothaveanytunableparameters 2.2.2AbingAbing[30]isbasedontheprobegapmodel.Itsendstwentybac k-to-back1500-Byte longpacketpairswithaknownseparationof50ms.Afterpass ingthroughthetightlink, accordingtotheauthors,probingpacketscanbeseparatedb ycrosstrafc(CT)packets inanyplaceontheend-to-endpath.Separationoftherstpr obingpacket(P1)fromthe secondprobingpacket(P2)ofapaircanhappenevenwherethe reisnorealbottleneck 17

PAGE 35

orcongestion.Thetimedelay( Td )betweenP1andP2willgrowdiscretelybecauseitis causedbyCTpacketswithparticularlengthsandnallyitwi llcontainthedelaycaused byalltheCTpacketsinsertedbetweentheprobingpacketsin anyhopalongthepath. Thenaltimedelay Td betweenpacketpairswillhavetheinformationoftheamount of crosstrafcthroughoutdifferentlinkswithdifferentcap acities.Thatvaluecorrespondsto theloadonthepath.Astheloadonthepathgrows, Td alsogrows. Theauthorsobservedtwocomponentsin Td .Onecomponent Td init iscommontoall individualmeasurementsandiscausedbythenarrowlink;th eothercomponent Td var is variableandreectsqueuingchanges.Therefore,thepacke tdispersionhasalinearanda non-lineargrowth: Td = Td init + Td var : (2.3) Thelineargrowthoccurswhenthecurrenthophasahigheruti lizationfactorthanthe previoushop.Thenon-lineargrowcanbecausedbythe"stret ching"effectdepictedin Figure1.4(when D out > D in ).Theauthorsndwhattheycallaconversionfunctionthat relatestheavailablebandwidthwiththedispersionvalue Td .Inotherwords,Abinguses thesameprobegapmodelusedbyothertoolsbutisuniqueinth ewayitestimatesthe amountofcrosstrafctraversingeverylinkintheend-to-e ndpath(whichisrelatedto Td ). Abingsends40probingpacketspermeasurementtocalculate ameanvaluefor Td and then,theamountofcrosstrafcandtheavailablebandwidth inthepath. 18

PAGE 36

2.2.3IGIIGI[27]usestheprobegapmodel.Theauthorsdeveloptwopac ketpairtechniquesto characterizetheavailablebandwidth.OneisIGI(initialg apincreasing)andtheother PTR(packettransmissionrate).Thesetechniquesareusedt oexperimentallydetermine theinitialgap( D in )thatwillyieldahighcorrelationbetweenthecompetingtr afcthroughputonthetightlinkandtheoutputgap( D out )atthedestination. IGIndsaninitialprobinggapvaluesothataprobingpacket traininteractswiththe crosstrafcinanonemptynarrowlinkqueue,whichiscalled bytheauthorsthe Joint QueuingRegion (JQR).Inthatregion,thereisaproportionalrelationbetw eenthegap whenprobingpacketsleavethequeue(outputgap)andthecro sstrafc.Theauthorsnd twocomponentsinthemathematicaldenitionoftheoutputg apunderthisJQRregion: D out = g B + B C D in C t : (2.4) Therstcomponentisthetimetakentoprocesstherstpacke tP1(seeFigure1.4)denotedby g B .Thisvalueiscalledbytheauthorsthebottleneckgapsince itisthegapvalue oftwoback-to-backprobingpacketsonthebottlenecklink( whichisassumedtobethe tightlink).Thesecondcomponentisthetimetakentoproces sthecrosstrafcthatarrives betweenthetwoprobingpacketsP1andP2. B C isthecompetingtrafcthroughputfor thetimeintervalofpacketsP1andP2.Thekeytoanaccuratea vailablebandwidthestimationbyIGIistondandinputgap D in sothattheprobingpackettrainoperatesinthis JQRregion.Theotherregioncalledthe DisjointQueuingRegion (DQR)occurswhenthesecond probingpacketP2ndsthequeueempty.Thishappensifthequ eueisemptyafterP1 19

PAGE 37

0 0.2 0.4 0.6 0.8 1 1.2 x 10 -3 0 1 2 3 4 5 x 10 -4 Initial Gap (sec)Gap Difference (sec) Figure2.1:IGIturningpoint.Initialgapisequaltotheout putgapfor0.8milliseconds. leavestherouterandbeforeP2arrives.Inthatcase,theout putgap D out istheinitialgap minusthequeuingdelayforP1: D out = D in Q C t (2.5) where Q isthequeuesizewhentherstpacketarrivestotherouter.T heproblemisthat packetpairsoperatingintheDQRregionwillprovidewrongv aluesforthepurposeof relatingthecrosstrafcwiththe D out .Toestimatetheamountofcompetingtrafc,IGI focusesonincreasedgapsinaprobingpackettrainoperatin gintheJQR.Specically, consideraprobingtraininwhich M probinggapsareincreased, K areunchanged,and N aredecreased.ByapplyingEquation2.4itisobtainedthees timationofthecompeting trafcload: B C = C t Mi = 1 g +i g B Mi = 1 g +i + Ki = 1 g =i + Ni = 1 g i : (2.6) Thatis,theamountofcrosstrafcthatarrivetotherouterd uringtheprobingperioddividedbythetotalprobingtime.Increased,unchangedandde creasedgapvaluesaredenotedby g +i g =i ,and g i respectively.Equation2.6iscalledtheIGIformula. 20

PAGE 38

UsingthesameIGIformulanotation,if L istheprobingpacketsize,theaveragetransmissionrateofthepackettraincanbeestimatedbythePTRformu la: A = ( M + K + N ) L Mi = 1 g +i + Ki = 1 g =i + Ni = 1 g i : (2.7) Whentheinitialgapisincreasedandequaltotheoutputgap, theavailablebandwidthon thetightlinkisequaltotheaveragerateofthepackettrain .Afterthatpoint,calledbythe authorstheturningpoint(seegapof0.8millisecondsinFig ure2.1),thenarrowlinkwill beoverowedbytheprobingpackets.BothIGIandPTRalgorithmssendtothedestinationasequenc eofpackettrainswith increasinginitialgap.Theymonitorthedifferencebetwee ntheaverage D in and D out gaps untilthatdifferencebecomeszero.Atthatpoint,thepacke ttrainisoperatingattheturningpointandtheIGIandPTRformulasareappliedtocomputet henalmeasurement. Theavailablebandwidthisobtainedbysubtractingtheesti matedcompetingtrafcthroughputfromthevalueof C t measuredbyanycapacityestimationtool. AlthoughakeyelementinIGIistheselectionoftheinitialg ap,therearetwomorefactorsthataffecttheaccuracyofthetool.Therstfactorist heselectionoftheprobing packetsize.Measurementsusingsmallprobingpacketsarev erysensitivetointerference. Theotherfactoristhenumberofprobingpacketssent.Sendi ngtoomanypacketscan causequeueoverowandpacketlosses,increasetheloadont henetwork,andlengthen thetimeittakestogetanestimate.Byexperimentation,theauthorsshowthatthequalityofthe estimatesisnotverysensitive totheprobingpacketsizeandthenumberofpackets,andthat thereisafairlylargerange ofgoodvaluesforthesetwoparameters.Forexample,a700-B ytepacketsizeand60 21

PAGE 39

packetspertrainworkwellontheInternet.Itisshownbythe authorsthatinthecaseof multiplehopsandsignicantcrosstrafcfollowingthetig htlink,theaccuracyofIGIsuffers.Asimilarsituationisfoundwhenthetightlinkisnott henarrowlink.Otherauthors havefoundthatIGIwasunresponsivetovariationsincrosst rafcatGbpsspeeds[17]. 2.2.4PathloadPathload[31]usestheSelf-LoadingPeriodicStream(SLoPS )[36]techniquewhichfollowsthesameprincipleoftheproberatemodel.Ingeneralte rms,SLoPSisbasedonthe factthattheonewaydelayofaperiodicpacketstreamincrea seswhentherateofthe probingtrafcishigherthantheavailablebandwidthinthe path.Otherwise,thereisno increaseinthedelaymeasured.Aeetofstreams(ofaxednu mberofpacketseach)are sentatvaryingratesandtheonewaydelaytrendofeachstrea misthencharacterizedat thereceiveraseitherincreasingordecreasing.Whenthatd elayisina grayregion where thereisnotclearlyincreasingnordecreasingtrend(seeFi gure2.2),themethodology presentsavariationrangeoftheavailablebandwidth.AmoredetaileddescriptionofSLoPSisthefollowing.Suppo seasendertransmitting asinglestreamofpacketstothereceiver.Everypacket i istimestampedbythesender beforeitistransmittedanditsarrivaltimeiscalculateda tthereceiver.Thedifference ofbothtimesistherelativeonewaydelayofthepacketdenot edby D i .Afterreceivingtheentirestreamofpackets,theOWDsvaluesareinspect edtocheckwhetherthe transmissionrateofthestream R islargerthantheavailablebandwidth A .When R > A ,therelativeOWDsofthe K packetsinthestream f D 1 ; D 2 ; ; D K g areexpectedto havean"increasing"trend.Thisisbecausethestreamcreat esashort-termoverloadin 22

PAGE 40

0 10 20 30 40 50 0 0.5 1 1.5 2 2.5 3 3.5 4 x 10 -3 Packet NumberOne Way Delay (sec) Figure2.2:Pathloadgrayregion.Thereisnotclearlyincre asingordecreasingtrendinthe onewaydelaybetweenpackets35and40.thetightlink.Duringthatperiodthetightlinkqueuebuild supandthequeuingdelay ofpacket i inthestreamisexpectedtobelargerthatthequeuingdelayo fpacket j with i > j .Thiseffectiswhattheauthorscallasself-loadingofthep eriodicstream.When R < A ,therelativeOWDsofthe K packetsareexpectedtohavea"non-increasing"trend. Theavailablebandwidthisgivenbytherateatwhichan"incr easing"trendinthestream startstobeobserved.Todetectthe"increasing"trendintheOWDsofastream,thea lgorithmimplementedin Pathloaddoesthefollowing.The K OWDsmeasurementsaredividedin G = p K groups. Foreachgroup,itiscalculatedthemedianOWD ˆ D i ofthegroup. Twostatisticsareusedtodetermineifthestreamshowsan"i ncreasing"trend.Oneis calledthepairwisecomparisontest(PCT)whichforeveryst reamiscalculatedby: S PCT = Gk = 2 I ˆ D k > ˆ D k 1 G 1 (2.8) 23

PAGE 41

where I isoneif ˆ D k > ˆ D k 1 ,andzerootherwise.Astrong"increasing"trendintheOWDs willbedetectedwhen S PCT isclosetoone.InPathloadan"increasing"trendisreporte d if S PCT > 0 : 55,a"non-increasing"trendif S PCT < 0 : 45,andan"ambiguous"trendotherwise.Theothermetriciscalledthepairwisedifferencetest(PDT )whichforeverystreamis calculatedby: S PDT = ˆ D G ˆ D 1 Gk = 2 ˆ D k > ˆ D k 1 (2.9) Astrong"increasing"trendintheOWDswillbedetectedwhen S PDT isclosetoone.In Pathloadan"increasing"trendisreportedif S PCT > 0 : 66,a"non-increasing"trendif S PCT < 0 : 54,andan"ambiguous"trendotherwise. Todeterminewhetherastreamischaracterizedbyan"increa sing"trendornot,Pathload doesthefollowing.Ifoneofthe S PCT and S PDT valuesreports"increasing"trend,while theotheriseither"increasing"or"ambiguous",thestream ischaracterizedastype-I("increasing").Ifonemetricreports"non-increasing"trendw hiletheotheriseither"nonincreasing"or"ambiguous",thestreamischaracterizedas type-N("non-increasing"). Finally,ifbothmetricsreport"ambiguous",orwhenoneis" increasing"andtheotheris "non-increasing",thestreamisdiscarded.Asexplainedbe fore,whenthestreamisina grayregion wherethereisnotclearly"increasing"nor"decreasing"tr endintheOWDs, Pathloadreportsavariationrangeoftheavailablebandwid th. Pathloadsendsperiodicpacketstreams(eets)ofUDPtraf candusesaTCPconnection tosendtrendresultsbacktothesender.Givenadesiredstre amrate R ,Pathloadsetsthe packetinter-departuretime T at100msandcalculatesthenecessarypacketsize L to satisfy R = L = T .Ifthe L islessthan96bytes,Pathloadusesthisminimumvalueand calculates T instead. 24

PAGE 42

0 0.005 0.01 0.015 0.02 0 0.5 1 1.5 2 2.5 3 x 10 -3 Packet Sending Time (sec)Queueing Delay (sec) Figure2.3:Chirpqueuingdelaysignature.Excursionsendw henthequeuingdelay returnstozero(between3and7.5ms)orwhenthereisanincre asingqueuingdelay(after 10ms)2.2.5PathchirpPathchirp[32]alsousestheproberatemodel.Insteadofsen dingapackettrain(orstream) ataspecicrateasPathloaddoes,Pathchirpincreasesthep robingratewithineachtrain inanexponentialmanner.Bydoingthat,Pathchirpcaptures delaycorrelationinformation usingasmallernumberofprobingpackets.SimilartoPathlo ad,PathchirpusesinformationoftherelativeOWDsofprobepackets.Thetoolsendsseveralpacketchirpstothereceiver.Eachch irphas N exponentiallyspaced packets,eachofsize P .Therearethreemainadvantagesonusingchirps.First,the chirp has N 1packetspacingsthatwouldnormallyrequire2 N 2packetsusingpacketpairs. Second,exponentiallyspacedpacketsrequireonly log ( G 2 ) log ( G 1 ) packetstoprobe thenetworkovertherangeofrates [ G 1 ; G 2 ] Mbps.Finally,chirpscapturecriticaldelay correlationinformationthatpacketpairsdonot. 25

PAGE 43

Tobetterdescribeachirp,Figure2.3showswhattheauthors calla"queuingdelaysignature"ofachirp m .Burstsofcrosstrafccausean"excursion"whichendswhen the queuingdelayreturnstozero(seeexcursionfrom3to7.5msi ntheFigure).Thatoccurs whenthechirprate R k islessthanthetightlinkcapacity C t .Anexcursioncanalsoend withincreasingqueuingdelaysasshowninthelastexcursio nafter10ms.intheFigure. Thatoccurswhenthechirprate R k isgreaterthanthetightlinkcapacity C t ,whichcauses thechirppacketstollupintermediatequeues.Witheverysignature,Pathchirpmakesanestimate E ( m ) k oftheper-packetavailablebandwidth.Toobtaintheper-chirpavailablebandwidth D ( m ) ,theper-packetvaluesareaveragedusingthefollowingequation: D ( m ) = N 1 k = 1 E ( m ) k D k N 1 k = 1 D k (2.10) where D k istheinter-spacingtimebetweenpackets k and k + 1.Theaverageofallperchirpavailablebandwidthvaluesisreportedbythetoolast henalestimation. 26

PAGE 44

Chapter3:EvaluationofCurrentAvailableBandwidthEstim ationTools Currentavailablebandwidthestimationtoolshavebeeneva luatedbydifferentauthors. However,thenetworkscenariosandmetricsusedintheevalu ationsarelimitedandtheir analysisabouttheapplicabilityofthetoolsinrealnetwor kapplicationsisabsent.An additionalissueisthattheseevaluationsdonotincludeth eamountofexperimentsneeded toprovidestatisticallyvalidconclusions.Forexample,in[17],Shriram etal .utilizeahigh-speedtestbedtoevaluateSpruce,Abing, Pathchirp,andPathload.Theyusepassivemonitorstoverif ytheactualloadlevelofthe generatedtrafcandtestthetoolsusinglinksofOC-48and1 Gbpscapacities.Theproblemwiththeseexperimentsisthattheyjustprovideapartia lpictureoftheevaluation, astheresearchersdonothavethecapabilitytoworkwithlin ksofdifferentcapacities. In[16],Lee etal. describeproblemswithsomebandwidthestimationtoolswhe nused onthePlanetlab[37]infrastructure.Sincethecapacityof thelinksisunknowntothe researchers,theyusePathrate[38]tomeasuretheend-to-e ndcapacityofthelinks.The problemisthattheassociatederrorincurredbyPathratein theestimationofthelinkcapacitiesintroduceserrorsinthenalestimationoftheava ilablebandwidth.In[15],Angrisani etal. evaluateIGI,IperfandPathloadoveralocalareanetworkan duseMGEN [39]asatrafcgenerator.Thisevaluationsufferssimilar exibilityproblemsliketheones foundin[17].Also,theauthorsdonotmeasuretheoverheadg eneratedbythetools. 27

PAGE 45

Previousworksdonotanalyzeseveralissuesthathavetobec onsideredbeforeimplementinganytoolinanetworkapplicationrequiringtheesti mation.Areavailablebandwidthtoolsreadytobeusedinallnetworkapplications?Can theyprovideestimates atthegranularityrequiredbyspecicnetworkapplication s?Further,cantheybeused regardlessofwhethertheapplicationisrunoveralowbandw idthwirednetwork,ora wirelessmobileadhocnetwork,orasatellitenetwork,orev enahighbandwidthand cleanopticalnetwork?Forexample,whileanavailableband widthestimationwitha10% errorcouldbeconsideredwithinacceptablevaluesforarou tingprotocoltomakerouting decisions,itmaybecompletelyunacceptableforSLAveric ation.Similarly,itmaybe justneforatooltotakesecondsorevenminutestoprovidea nestimateforanetwork managementsystem,butitwouldbeuselessforatransportla yerprotocoltomakerate changingdecisions.Finally,althoughitmaynotbeabigiss uetouseaveryintrusive activeprobingestimationtoolinanopticalnetwork,itmay consumeveryscarceand preciousresourcesinawirelessmobileadhocnetwork.Inordertoanswerthequestionsabove,thischapterpresent sanevaluationofthemain currentavailablebandwidthestimationtools.Theestimat ionprovidedbythetoolsis comparedwiththerealvalueoftheavailablebandwidth.Two approachesareusedto performtheevaluationdependingonthewaythisrealavaila blebandwidthisdetermined. Therstapproachusesananalyticalandthesecondapproach usesanexperimentalvalue. Thisevaluationisnovelinseveralways.First,aexiblean dlow-costtestbedisbuilt toincludescenariosandnetworkconditionsnotconsidered beforesuchasusinglow, medium,andhighlinkcapacities;packetlossratestosimul atelossylinks,orveryclean linkslikeopticalbers;crosstrafcloadanddistributio ntoexperimentwithdifferent levelsofnetworkcongestion;propagationdelaystosimula teeitherlocalarea,widearea, orsatellitenetworks;etc.Second,therstevaluationapp roachpresentsananalytical 28

PAGE 46

methodneverusedbeforetodeterminethetheoreticalvalue oftheavailablebandwidth andcompareitwiththeestimationgivenbythetools.Third, theapproachthatusesan experimentalvalueoftherealavailablebandwidthperform sacomprehensivesetofexperimentsdenedbyafactorialdesignneverconsideredbef ore.Fourth,theevaluation includesanewmetriccalled"reliability"notevaluatedbe fore.Finally,itispresenteda uniqueanalysisofthetoolsutilityfromthenetworkapplic ationspointofview. 3.1PerformanceMetricsThethreemainmetricstraditionallyusedandincludedinth ischaptertoevaluateavailablebandwidthestimationtoolsare: estimationerror overhead ,and estimationtime Theestimationerrororaccuracymetricprovidesaquantita tivevaluethatcomparesthe estimationoftheend-to-endavailablebandwidth,asprovi dedbythetoolunderconsideration,withtherealvalue,whichisinthisworkcalculateda nalyticallyandexperimentally. Theestimationerrorisgivenbyapercentageerror.Theover headisrelatedtotheamount ofprobepacketsthatthetoolneedstoinjectintothenetwor kinordertoperformthe estimation.Mostavailablebandwidthestimationtoolsare activeprobingmeasurement mechanismsandassuchtheysamplethesystemsendingprobep ackets.Theoverheadis denedasthepercentageoftooltrafcrespecttothecapaci tyofthetightlink.Finally, theestimationtimesayshowlongittakesthetooltoprovide theestimate,anditisusuallygiveninseconds.Inthiswork,fortheevaluationusinganexperimentalobtai nedvalueoftherealavailable bandwidth,afourthmetricisadded:the reliability ,whichprovidesinformationaboutthe robustnessofthetoolinprovidingestimations.Thereliab ilityisgivenbythepercentage 29

PAGE 47

ofteststhetoolsucceededtoprovideanestimate.Itiscalc ulateddividingthenumberof replicationsforaparticularexperimentbythenalnumber oftrialsneededtoperformto reachthatnumber.Thus,a100%reliabletoolneeded N trialstoprovide N estimations. Accordingtotheperformancemetricsdenedbefore,itcoul dbesaidthatanidealtool shouldprovideveryaccurateestimations,withveryloworn ooverhead,inalmostno time,andwitha100%reliability.However,notallapplicat ionsneedanidealtool.Some requirementscouldberelaxedortightenedaccordingtothe applicationandthenetworkingenvironmentathand.Forexample,anapplicationthatmo nitorsthecomplianceof SLAsneedstheavailablebandwidthestimationtohavehigha ccuracy,mediumoverhead, mediumorlowestimationtime,andmediumreliability.Alth oughsomeofthemetrics mightbearguable,onestrongrequirementishighaccuracy. Similarly,iftheavailable bandwidthisusedtodrivetheowandcongestioncontrolmec hanismsofatransport layerprotocol,itbetterbefastandintroducealmostnoove rhead.Inthiscase,theestimationdoesnotneedtobeveryprecise,agoodestimationm ightwork,butithasto befast,sothattheprotocolcanreactontimetorapidlychan gingnetworkconditions, andwithnooverheadbecauseotherwisethehugenumberoftra nsportlayerprotocol connectionsovertheInternetwilldrivethegoodputofthen etworktoverylowlevels.As analexample,takeanavailablebandwidthestimationtool tobeusedtodetectsecurity attacks.Inthiscase,itiseasytoseethatthemostimportan tmetricsaretheestimation timeandthereliabilityofthetool. 30

PAGE 48

Clientr Serverr 192.168.3.0/24r 192.168.2.0/24r 192.168.1.0/24r 192.168.4.0/24r 192.168.0.0/24r BET tool trafficr Cross Trafficr Link Ar Link Br Link Cr Link Dr USr Colombiar Chinar Spainr rrr r [ Testbed control packets throughout a 10.0.0.0/24 network ]r Figure3.1:Testbedtoevaluatebandwidthestimationtools. Intermediatecomputersactas routersandtheirnamesaretomimiclocationsinawideareanetwork. c r 2006IEEE. 3.2Testbed Inordertoexperimentallyevaluatetheavailablebandwidthestimationtools,thetestbed showninFigure3.1isbuilt.Thisisafullycontrolledenvironmentwithparameterizable linksintermsoflinkcapacities,packetlossrates,queuessizes,andpropagationdelays. Itcanalsobecontrolledtheamountofcrosstrafcanditsstatisticaldistributiononaper linkbasis.Thetestbedutilizeslowcostcomputersandopensourcesoftware,asdescribed next. Therearethreemaincomponentsinthetestbed:oneclient(orsender),fourintermediate routers,andoneserver(orreceiver).TheyaresixcomputerswithAMDAthlon643500+ processors,1GBRAM,and80GBharddrivecapacityinterconnectedthroughtwoprivate networks.A1Gigabit192.168.X.Xnetworkisutilizedtocarryallthetrafcrelatedto theevaluationofthetools;a100Mbps10.0.0.0networkisusedtocongureandrun theexperiments.Thesetwonetworksareestablishedtoseparatecongurationdatafrom evaluationdata.TheclientandserverareLinux-basedmachinesrunningwith10000Hz timergranularity.Theyhosttheavailablebandwidthestimationtoolsunderinvestigation. 31

PAGE 49

IntermediateroutersareimplementedbyfourFreeBSDmachi nesemulatingamultihopnetworkpath.Thekernelhasbeenrecompiledtohostapac ketshapercalledDummynet[40]andtorunatthesameendnodesgranularity.Dummy netallowstochange thecapacityofeachlinkfrom0tounlimited(orlimitedbyth ephysicalcapacity),and introducepacketlossesanddelaystoemulatelossyandlong links.Differentqueuesizes canalsobeestablishedifsodesired.Togeneratecrosstraf ctheMGEN[39]trafcgeneratorisused,whichallowstochoosetherate,packetsize, andthestatisticaldistribution ofthecrosstrafc.Inordertohaveindependentqueues,the crosstrafcisintroducedon aperlinkbasis,i.e.,entersattheinputofeachqueueandle avesthenetworkbeforethe subsequentqueue(seeFigure3.1).TheaccuracyofthisDummynet-basedtestbedisexperimenta llyveried.Using netperf and iperf itisobservedthatthetestbedmaximumthroughputiscloset o340Mbps.Accordingtoa ping command,thetestbedcanemulatedelayswith97.53%average precision; tcpdump tracesalsoverifythatthetestbedemulatespacketlossrat eswitha99.84% averageprecision.Finally, tcpdump tracesshowthatMGENgeneratedtrafcwith96.19% accuracy.Allthetoolsunderevaluationusethe gettimeofday functiontoquerythesystem'scurrenttime.APythonclient-serverapplicationisdevelopedtoautomat etheentireevaluationprocess. Thisapplicationallowsuserstoselectthebandwidthestim ationtooltobeevaluated,the linkbandwidths,thepositionofthetightlink,thetypeand rateofthecrosstrafcandthe numberofexperimentsperscenario(toallowstatisticalsi gnicanceintheresults).The applicationautomaticallyrunstheexperimentsandcollec tstheresults;itreadstesting lesplacedinaparticularfolderandwritestheresultsina notherfolderaftertheexperimentsarenished.Runningexperimentsremotelyandsharin gthetestbedwithothersis mucheasierwiththisapplication. 32

PAGE 50

3.3Analytically-BasedAvailableBandwidthEvaluationTherstevaluationusesananalyticalmodeltodetermineth erealavailablebandwidth andcompareitwiththeestimationgivenbythetools.Theide aistobuildamodelto mimicthebehaviorofthenetworkofqueuesinthetestbedNIC sandtodeterminefrom themodeltherealvalueoftheavailablebandwidth.Thismod elshowninFigure3.2 consistsofeightM/M/1queuesrepresentingthenetworkint erfacecardswherethetight linkwillbesetandevaluated(queues1,3,5,and7),whereth ecrosstrafcwillberouted outsidethesystem(queues2,4,and6),andwhereoutputtraf cofthesystemwillbe received(queue8).ThisnetworkofqueueswasstudiedbyJackson[41,42]in1957 .InJackson'smodel,if i isthenumberofnodesinthesystem( i = 1,2,..., K ),itisassumedthatnode i contains n i queues(servers).Also,itemsarrivefromoutsidethesyste morfromothernodestonode i ataPoissonrateandareservedinturnatanexponentialserv icerate.Onceservedata node,anitemgoes(instantaneously)tonode j ( j = 1 ; 2 ;:::; K )withprobabilityqij ,orout ofthesystem.Fromtheseassumptions,inthesteadystate,t heaveragearrivalratetonode j (lj )isgivenbyEquation3.1,whereqij representstheroutingprobabilityofgoingfrom node i tonode j andgj istheexternaltrafcenteringqueue j :lj =gj + K i = 1liqij ; for 1 j K : (3.1) Theunderlyingstochasticprocessofthesystemisdenedby X = f X i t : t 2 R + ; i 2 [ 1 :: 8 ] g (3.2) 33

PAGE 51

nr Figure3.2:Networkofqueuesfortheevaluationtestbed. c r 2006IEEE. where X i t isthenumberofpacketsinthequeueandserver i attime t .Itisworthnoticing thattheprobingpackettrafcrateg0 tobeobtainedfromexperimentationisutilizedin Equation3.1tocalculatetheinputrates.Thisisthereason whydifferenttheoreticalvaluesareobtainedforeachtoolinthesameexperimentaswillb eshownlater. Thevalueofqij correspondstotheroutingmatrixoneachqueue,whichisdif ferentfrom thetransitionprobabilitymatrixoftheunderlyingMarkov model.Inthisevaluation,the routingmatrixqij isgivenby:qij = 2666666666666666666664 0100000000g0 l2 00001 g0 l2 000100000000g0 l4 001 g0 l4 00000100000000g0 l6 1 g0 l6 0000000100000001 3777777777777777777775 : (3.3) 34

PAGE 52

GivenalltheM/M/1queuesinputrates,wecanestimatetheav ailablebandwidthcorrespondingtoeachqueuecanbeestimatedasthenon-utilizedc apacityofthesystemas follows: A i = 1 li mi = 1 ri (3.4) whereri isthecalculatedutilizationofeachqueue. ItisworthnoticingthatthisanalysisassumesaPoissondis tributionfortheprobingtrafc generatedbythetools.Althoughthisassumptionmightnotb etrue,theresultsobtained indicatethatthisisnotabadassumption.Areasonforthati sthelowoverheadintroduced bythetoolcomparedwiththeamountofcrosstrafc,whichpa cketinterarrivaltimesare exponentiallydistributed.3.3.1ExperimentsUsingthetestbedandtheJackson'smodeldescribedbefore, thetoolsanalyzedunderthis rstevaluationapproacharePathload,IGIandSpruceaccor dingtotheirestimationtime, overheadandestimationerrormetrics.Theestimationtime inthecaseofPathloadand IGIisprovideddirectlybythetool.InthecaseofSpruce,th eestimationtimeiscalculatedbythedifferenceoftimesbeforeandafterrunningthe tool.Theoverheadisgiven bytheratiobetweenthetrafcgeneratedbythetoolandthec apacityofthetightlink.In otherwords,itrepresentsthepercentageofthetightlinkc apacityutilizedbythetool.The estimationerroriscalculatedcomparingtheavailableban dwidthestimationgivenbythe toolwiththeexpectedvaluefromthemathematicalmodelusi ng Matlab .Plotsshowing theestimationerrorontheestimationbuseEquation3.5forthecalculations. 35

PAGE 53

Table3.1:Parametersusedbytheestimationtools. c r 2006IEEE. TOOL PacketSize Packets/stream Pathload Variable.Minimum:96B 100 IGI 500B 60to256 Spruce 1500B 100 b= m A mA mA (3.5) where m A isthevaluecalculatedfromexperimentationandmA isthevaluefromtheanalyticalmodel.Themainparametersintheevaluatedtoolsar egiveninTable3.1 Foreachtoolevaluated,28differentscenariosaredened. Eachscenariocorrespondsto variationsinthecapacityofthetightlinkfrom1Mbpsto9Mb psat1Mbpsintervals,and from10Mbpsto100Mbpsat5Mbpsintervals.Foreachscenario ,itisconsideredthe situationwherethelinksarecompletelyemptyofcrosstraf candloadedat25,50and 75percentofthecapacity.MGENisusedtogeneratePoissonp rocesseswithmeanrates equaltothegivendesiredamountofcrosstrafc.Itisworth noticingthatthecrosstrafc isgeneratedonanodebynodebasis,i.e.thetrafcgenerate datnode i loadsitsoutput queueandthelinkfromnode i tonode i + 1 ,butitdoesnotloadtheoutputqueueofnodes j 6 = i .Inthismannernotrafccorrelationsfromnodetonodearei ncludedandallnodes arecompletelyindependent.3.3.2ResultsThefollowingsectionsdescribetheresultsafterperformi ngtheexperimentsandevaluatingtheestimationerror,overheadandestimationtimemetr ics.Eachpointinthegraphs istheaverageofrunningeachexperiment35times.Thatallo wsinthecaseofIGIand 36

PAGE 54

Spruce,tocalculateandplota95%condenceintervalcalcu latedbythenormaldistributiontest.InthecaseofPathload,itisplottedtheaverager angegivenbythetool.Atotal of11760experimentsareperformed:3tools,28capacityvar iations,4crosstrafcloads and35samples.Fortheanalyticalresults,g0 inEquation3.1istheprobingtrafcrategeneratedbyeach tool.Thisvalueistheresultofdividingtheamountofprobi ngpacketbytescalculated with tcpdump bytheestimationtimeofthetool.ThisisthereasonwhytheJ acksonmodel behavesdifferentlywitheachexperimentandwitheachtool 3.3.2.1EstimationErrorFigures3.3to3.14presenttheestimationerrorofPathload ,IGIandSprucewhenthe tightlinkisloadedwith0%,25%,50%and75%ofcrosstrafc. Pathloadprovidesthe bestapproximationtotheanalyticalvalueobtainedfromth eJackson'smodel.Some Pathloadmeasurementsarenotshownbecausethetoolhascon vergenceproblemsinlow capacitylinks.However,asshowninFigure3.15,whentheto olconverges,regardless oftheamountofcrosstrafcandtightlinkcapacity,ithasa relativeerroroflessthan 20%.Inmostcasesthetooloverestimatestheavailableband width.Itiswellknownthat Pathloadisoneofthemostaccuratebandwidthestimationto ols[17]. InthecaseofIGI,thistoolpresentshighestimationerrors .Forexample,in[27],theauthorsofIGIshowthatthetoolhasanerroroflessthan20%ins cenarioswithlowround triptimevalues.Experimentsinthisevaluationalsoverif ythisconclusionalthoughthe resultsalsoindicateveryhighvariability.Figure3.16sh owsthatwhenthecrosstrafcis high,theaccurracyofthetoolisverylow.Thisisalsomenti onedbytheauthorsofIGI 37

PAGE 55

0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Pathload Figure3.3:Pathloadestimationwith0%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson IGI Figure3.4:IGIestimationwith0%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Spruce Figure3.5:Spruceestimationwith0%crosstrafc. 38

PAGE 56

0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Pathload Figure3.6:Pathloadestimationwith25%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson IGI Figure3.7:IGIestimationwith25%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Spruce Figure3.8:Spruceestimationwith25%crosstrafc. 39

PAGE 57

0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Pathload Figure3.9:Pathloadestimationwith50%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson IGI Figure3.10:IGIestimationwith50%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Spruce Figure3.11:Spruceestimationwith50%crosstrafc. 40

PAGE 58

0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Pathload Figure3.12:Pathloadestimationwith75%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson IGI Figure3.13:IGIestimationwith75%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Available Bandwidth (%) Jackson Spruce Figure3.14:Spruceestimationwith75%crosstrafc. 41

PAGE 59

0 2 4 6 8 10 x 10 7 -20 0 20 40 60 80 100 Tight Link Capacity (bps) Estimation Error (%) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.15:Pathloadrelativeerrorfor0%,25%,50%and75% crosstrafc. 0 2 4 6 8 10 x 10 7 -20 0 20 40 60 80 100 Tight link capacity (bps) Estimation Error (%) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.16:IGIrelativeerrorfor0%,25%,50%and75%cross trafc. 0 2 4 6 8 10 x 10 7 -20 0 20 40 60 80 100 Tight link capacity (bps) Estimation Error (%) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.17:Sprucerelativeerrorfor0%,25%,50%and75%cr osstrafc. 42

PAGE 60

whentheytestedlinkswithlongroundtriptimes.However,i ncontrasttotheIGIpaper, Pathloadisstillaccurateinexperimentswithhighlyloade dlinks.Spruce,ontheother hand,showsarelativeerrorsmallerthan30%inmostscenari os,whichalsoveriesthe resultspresentedbytheauthorsin[29].AsinthecaseofIGI ,Sprucealsopresentsproblemswhenestimatingoverhighcapacitylinkswithhightraf cloads.Itsestimationvarianceisalsohighoverlowcapcitylinks.Itisworthnoticing thatIGIandSprucebelong tothesameprobegapmodelcategory.Spruceshowsaparticul arhighunderestimation valuewhenthecapacityisaround70Mbps.3.3.2.2OverheadFigure3.18to3.20showtheoverheadratioofthetoolsforea chcrosstrafcload.The overheadofPathloaddoesnotexceed10%ofthetightlinkcap acity.Pathloadintroduces moreprobetrafcwhenthecrosstrafcdecreases.Thisisco mpletelyexpectedasit worksbasedontheprincipleofinducedcongestion,sotheem ptierthechannelthehigher theamountofprobetrafcthatthetoolneedstoinject.InFigure3.19itisshownthatIGIhaslowoverheadoverhighc ongestedlinks.Thisis becauseIGIndsseveralpackettrainsinthe JointQueuingRegion anddoesnotneedto sendadditionalpacketstodeterminetheturningpoint.The reare,however,somescenarioswheretheaverageoverheadgrowsupto30%ormore,suc hasthosepointsin Figure3.19wherethecapacityofthetightlinkis25Mbpsand thecrosstrafcis25or 50%ofthecapacity. 43

PAGE 61

0 2 4 6 8 10 x 10 7 0 5 10 15 20 25 30 35 40 Tight Link Capacity (bps) Overhead (%) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.18:Pathloadoverheadfor0%,25%,50%and75%cross trafc. 0 2 4 6 8 10 x 10 7 0 5 10 15 20 25 30 35 40 Tight Link Capacity (bps) Overhead (%) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.19:IGIoverheadfor0%,25%,50%and75%crosstraf c. 0 2 4 6 8 10 x 10 7 0 5 10 15 20 25 30 35 40 Tight Link Capacity (bps) Overhead (%) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.20:Spruceoverheadfor0%,25%,50%and75%crosstr afc. 44

PAGE 62

Spruceoverheadislowandconstantregardlessoftheamount ofcrosstrafc.Thisis explainedbythePoissonsamplingmethodutilizedbythetoo l.Anotherobservationis theincreaseoftrafcoverheadwhenthetoolsbasedonthepr obegapmodeloperatein lowbandwidthscenarios.Inthiscase,IGIneedstosendmore probingpacketstondthe correctprobinggapvalueandSpruceachievesonlyasmallin ter-pairgapinthePoisson samplingprocess,whichresultsinaquitemoreintrusivesa mple.Thisisalsoreectedin thehighestimationerrorvariationshownbythesetwotools inlowlinkcapacities. Inthebestcase,whenthenetworkishighlyloaded,thetools needtoinjectaround3% probetrafcofthenarrowlinkcapacitytoperformtheestim ations.Although3%sounds likealowvalue,inrealityitmaybeabignumber.Forinstanc e,inthecaseofa50Mpbs narrowlink,thetoolswouldoccupy1.5Mbps.Thisamountofo verheadcouldlimitthe utilizationofthetoolsincertainenvironments,suchaswi relessnetworkswherelink bandwidthisascarceresource.3.3.2.3EstimationTimeFigures3.21to3.23depicttheestimationtimeinsecondswh enthecrosstrafcvaries from0%to75%ofthenarrowlink.FromFigure3.21,itcanbese enthatPathloadneeds lesstimetoconvergeinthecaseof0%crosstrafcthaninthe 75%case.Tcpdumptraces providetheexplanationforthisbehavior.Whenthenetwork isslightlyloaded,thetool sendsprobetrafcmorefrequentlyandgetsfeedbackaboute achsamplefaster.Asa result,itinjectsmoretrafcandconvergesfaster.Whenthenetworkishighlyloaded,thetoolneedstoincrease thegapbetweenprobe packetsandthegapbetweentrains,whichreducestheamount ofprobetrafc.However, 45

PAGE 63

0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Estimation Time (s) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.21:Pathloadtimefor0%,25%,50%and75%crosstraf c. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Estimation Time (s) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.22:IGItimefor0%,25%,50%and75%crosstrafc. 0 2 4 6 8 10 x 10 7 0 20 40 60 80 100 Tight Link Capacity (bps) Estimation Time (s) 0% cross traffic 25% cross traffic 50% cross traffic 75% cross traffic Figure3.23:Sprucetimefor0%,25%,50%and75%crosstrafc 46

PAGE 64

thetoolhasmoreproblemsndingtheestimation,whichtran slatesintolongerestimation times.Pathloadcantakemorethan100secondstoprovidethe estimationinsomecases. ThislongestimationtimemaylimittheapplicabilityofPat hloadincertainapplications ormayprovideerroneousestimationsinthoseenvironments withfastchangingtrafc patterns.AsitisshowninFigure3.22,IGIneedsconsiderablylessamo untoftimetoconverge thanPathload.Spruceestimationtimeisdirectlyassociat edtotheamountofprobing packetssenttothenetwork,whichisconstantregardlessth eamountofcrosstrafc. Regardlessoftheamountofcrosstrafc,theevaluatedtool shavemoreproblemsconvergingwhenthenarrowlinkisalowcapacitylink.Inthecas eofIGIandSpruce,their behaviorcanbeexplainedbythedifcultyofthetoolstoset theappropriategap,which impliesmoremeasurementsandmoredelayintheestimation. InthecaseofPathload, thesmallertheavailablebandwidth,thehigherthenumbero fiterationsthetoolneedsto performtodetectthegrayregion.Thisisalsothereasonwhy insomeofthesepointsthe toolisnotabletoconverge.3.4Experimentally-BasedAvailableBandwidthEvaluationTheevaluationpresentedintheprevioussectionevidences trade-offsregardedtothe performanceofthetoolsinthedenedmetrics.Specically ,Pathloadisthemostaccuratetoolbuttheslowesttoconverge.IGI,ontheotherhand, isthefastesttoolbutthe leastaccurate.Spruceistheleastintrusivetoolwithinte rmediateestimationerrorand estimationtime.Thatevaluationbasedontheanalyticalca lculationoftherealavailable bandwidthtriggersadditionalquestionsabouttheperform anceoftheseandothertools 47

PAGE 65

Table3.2:Factorsandlevelsinthe2 5 factorialdesign. c r 2006IEEE Factor Level(-1) Level(+1) TightLinkCapacity 5Mbps 100Mbps OneWayPropagationDelay 10ms 80ms PacketLossRate 0.01 0.07 Percentageofcrosstrafc 25% 75% crosstrafcPacketSize 512Bytes 1408Bytes fromthenetworkapplicationspointofview.Moreover,ther earequestionsabouthowthe toolsperforminavarietyofnetworksscenarios.Asecondevaluationincludingadditionaltools,networksc enarios,anewmetric,andan experimentalvalueoftherealavailablebandwidthisprese ntedinthissection.Adding moretoolsandnetworkscenariosincreasesthenumberofexp erimentsthatwouldhave beennecessarytoruninordertoevaluateeachtooloverthet otalityofnetworkscenarios.Forthatreason,thisevaluationisperformedintwopha ses.Intherstphasea2 k p factorialdesign[43]describedinAppendixA,isutilizedt oreducethatlargenumberof experiments.Byusingthismethod,itispossibletoestabli shwithenoughcondencethe mostrelevantfactorsandtheircombinationsthataffectth eperformancemetricsunder evaluation.Thesecondphaseutilizestheresultsofthefac torialdesigntorunspecic experimentsandanalyzerelevantcasesdeeper.3.4.1PhaseOne:The 2 5 FactorialDesign Thirty-twosetsofexperimentsarecarriedoutinthissecti ontoperforma2 5 factorialdesignfollowingstandardstatisticalproceduresfoundinth eliterature[43].Ina2 5 factorial design,experimentsareperformedusingtwoextremevalues foreachofthevefactorsin ordertocollectonesampleofeachperformancemetricforea chtool. 48

PAGE 66

Asmanyexperimentsasneededareperformedtoobtaintenval idresultsfromeachevaluationscenario.Thisistocalculatea95%condenceinterva lusingtheStudent'st-distribution andpresentstatisticallycondentresults.Asaresult,am inimumof1600(32 10 5tools)experimentsareruninthisphase.Table3.2showsth evefactors(tightlink capacity,linkpropagationdelay,packetlossrate,percen tageoftightlinkcapacityused bycrosstrafc,andcrosstrafcpacketsize)andthelowand highvaluesselectedforeach factoraccordingtothemethodology.Inthissecondevaluation,fourperformancemetricsorresp onsevariablesaretested:the estimationerroroftheestimation,givenbythepercentage erroroftheestimatecompared withtherealvalue;theoverhead,givenbythepercentageof thetightlinkcapacityutilizedbytheprobetrafc;theestimationtimeinseconds;an dthereliabilityofthetool, givenby10estimationsdividedbythetotalnumberoftimest hetoolhadtoberunto obtainthose10values.Inalltheseexperimentsandunlessn otedotherwise,eachoutput queueinthepathhasabuffersizeequalto50slots,thecross trafctypeisPoissonsending1408-Bytelongpackets,andallthelinkcapacitiesares etto200Mbps.Toobtainthe valueoftherealavailablebandwidth,everyoutputlinkint hetestbedissniffedandtheir packetsclassiedtodifferentiateprobingtrafcfromcro sstrafc.Thisclassicationis performedbyascriptwrittenin Awk thatreadsatcpdumptraceandlterstherequired values.Fromthe2 5 factorialdesign,themaineffectsofvaryingoneorseveral factorsontheresponsevariablesisdetermined.Table3.3showsthecaseinw hichonlyonefactorisvaried.Itisworthnoticingthattheseresultsdonotsaymuchab outtheabsolutevalueofthe metric.Theyonlyprovideinformationabouttheaveragevar iationinaparticularmetric whenthefactorvalueisincreased.Forexample,inthecaseo fPathload,theeffectofincreasingthetightlinkcapacityfrom5to100Mbpsontheesti mationerroris 21 : 86%. 49

PAGE 67

Table3.3:Maineffectintheperformancemetricswhenvaryi ngonefactor. Factor ResponseVariables Error(%) Overhead(%) Time(s) Reliability(%) Pathload TightLinkCapacity -21.86 -3.12 -29.21 0.00 PropagationDelay -1.62 -1.35 28.97 0.00 PacketLossRate -88.55 -2.22 -31.92 0.00 Percentageofcrosstrafc 6.06 -0.75 12.27 0.00 crosstrafcPacketSize 4.24 -0.65 8.33 0.00 IGI TightLinkCapacity -80.43 -2.05 -1.74 0.00 PropagationDelay 121.25 -0.23 1.88 0.00 PacketLossRate -58.13 -0.40 5.27 0.00 Percentageofcrosstrafc 329.38 0.00 3.86 0.00 crosstrafcPacketSize 110.51 0.03 -1.49 0.00 Spruce TightLinkCapacity -25.60 -2.14 -0.28 -7.33 PropagationDelay 0.80 -0.12 0.61 1.13 PacketLossRate 3.47 -0.29 4.33 -26.12 Percentageofcrosstrafc 26.80 -0.02 0.22 0.05 crosstrafcPacketSize -5.06 0.008 0.50 0.91 Abing TightLinkCapacity -636.00 -14.72 0.00 -1.50 PropagationDelay -325.84 -2.04 0.28 0.05 PacketLossRate -64.18 -0.82 -0.02 -38.06 Percentageofcrosstrafc 744.01 0.01 -0.01 -3.06 crosstrafcPacketSize -311.76 -0.04 0.00 0.95 Pathchirp TightLinkCapacity -283.64 -4.72 0.16 0.00 PropagationDelay 11.11 0.06 -0.03 0.00 PacketLossRate -111.77 -0.62 -0.87 0.00 Percentageofcrosstrafc 229.73 -0.18 -1.33 0.00 crosstrafcPacketSize -40.67 0.09 1.66 0.00 50

PAGE 68

XT rPSizer % of XTr PLRr Delayr Capacityr Pathchirpr Abingr Sprucer IGIr Pathloadr Pathchirpr Abingr Sprucer IGIr Pathloadr Pathchirpr Abingr Sprucer IGIr Pathloadr Pathchirpr Abingr Sprucer IGIr Pathloadr Reliabilityr Timer Overheadr Errorr Factorsr XT rPSizer % of XTr PLRr Delayr Capacityr Pathchirpr Abingr Sprucer IGIr Pathloadr Pathchirpr Abingr Sprucer IGIr Pathloadr Pathchirpr Abingr Sprucer IGIr Pathloadr Pathchirpr Abingr Sprucer IGIr Pathloadr Reliabilityr Timer Overheadr Errorr Factorsr Some impactr Some impactr Medium impactr Medium impactr High impactr High impactr Figure3.24:Maineffectontheresponsevariableswhenfact orsarevaried. ThisresultmeansthatPathloadnegativelyvariesitsestim ationerrordownwardswith anaverage21.86%asthecapacityofthetightlinkincreases .However,thisresultdoes notmeanthattheestimationerrorimprovesordeclinesby21 .86%.Figure3.24summarizesthemaineffectofvaryingeachfactoroneachrespon sevariableforeachofthe toolsashavingsomeimpact,mediumimpact,andhighimpact. Inaddition,itisalso calculatedtheaverageinteractionoftwo,three,four,and vefactors,whichareshown inTablesA.2,A.3,A.4,andA.5.FromFigure3.24,thefollowingmainconclusionscanbemade : Thevariationofanyofthefactorsconsiderablyimpactsthe estimationerrorvariationforIGI,Abing,andPathchirp.Thisisanindicationoft heinaccuracyofthese tools.Therefore,theapplicabilityofthesetoolsmaybeli mitedtothoseapplicationsthatdonotneedtohaveverypreciseestimations.Spru ceandPathloadarethe leastaffectedtoolsinthesensethattheirestimationerro rsdonotvarymuch,independentlyofwhethertheestimatesareaccurateornot.Thea ccuracyofPathloadis 51

PAGE 69

mainlyaffectedbychangesinthepacketlossrateandthecap acityofthetightlink. Therefore,thistoolmightnotbegoodchoiceforthoseappli cationsrunningover wirelessnetworks,whichusuallyhavehigherpacketlossra tesandlowbandwidth channels.Spruce'saccuracyisaffectedbytheamountofcro sstrafc(congestion) andthecapacityofthetightlink.Additionalexperimentsn eedtobemadetobetter analyzetheestimationerrorofthesetools.Itisworthment ioningthatavailable bandwidthestimationtoolshavenotbeenanalyzedunderdif ferentpacketlossrates before. Theoverheadthatthetoolsinsertintothenetworktoperfor mtheestimationis almostunaffectedbythevariationofanyofthefactors,exc eptforAbing,which seemstobeaffectedbythecapacityofthetightlink.Theref ore,onlyexperiments withvariationsinthecapacityofthetightlinkareperform edtoobservetheoverheadshownbythetoolsinmoredetail.Thisconclusiondoesn otmeanthatthe toolsdonotinsertaconsiderableamountofoverheadbuttha ttheoverheadthat theyintroduceisfairlyconstantregardlessofthefactors WiththeexceptionofPathload,theestimationtimeoftheto olsisbarelyaffectedby anyofthefactors,meaningthatthetoolstakeasimilaramou ntoftimetoconverge. Thisisagoodfeature,asitprovidespredictabilityinthee stimationtime.TheestimationtimeofPathloadisshowntobeaffectedbythecapac ityofthetightlink, thepropagationdelay,thepacketlossrate,andtheamounto fcrosstrafc.Given theseresults,theapplicabilityofPathloadmaybelimited tocertainnon-realtime applicationsandcertainnetworktypes.Again,itmaynotbe agoodideatouse Pathloadinwirelessandsatellitenetworks,whichhavehig herpacketlossrates, lowbandwidth,andlongpropagationdelaylinks. 52

PAGE 70

Thepacketlossrateistheonlyfactorthataffectstherelia bilityofthetools,and onlyaffectsSpruceandAbing.Moreexperimentsareneededt odeterminethe reliabilitylevelofthetoolsandthepacketlossrateatwhi chthereliabilitybecomes critical. Similarly,theaverageeffectthatthecombinationoftwo,t hree,four,andallvefactors haveintheperformancemetricsisanalyzed(resultsaresho wninAppendixA).Hereare themainobservationsfromthatanalysis: TheestimationerrorofIGI,Abing,andPathchirpisverysen sitivetovariationsin thecapacityofthetightlinkandtheamountofcrosstrafci nthenetwork.More specically,whenthesetwofactorsareincreased,anegati vevariationintheestimationerrorisobtained.Asimilartrendisobservedwhenthep acketlossrateandthe amountofcrosstrafcareincreased. Whenthecapacityofthetightlinkandthecrosstrafcpacke tsizeincrease,Pathload andSprucetendtounderestimatetheavailablebandwidth. TheestimationtimeofPathloadishighlyaffectedbyvariat ionsinthecapacityof thetightlink,thepacketlossrate,andtheamountofcrosst rafc. Whenallfactorsarecombined,IGI'sestimationerroristhe mostaffectedbutits overheadisthemoststable.TheestimationtimeofAbingsho wsthemoststable behavior. 53

PAGE 71

3.4.2PhaseTwo:MainExperimentsThesecondphaseincludestheresultsoftheadditionalexpe rimentsperformedaccording totheresultsofthefactorialdesign.LookingatFigure3.2 4,itcanbeseenthatthecapacityofthetightlinkaffectstheestimationerror,overh ead,andestimationtimeofthe tools.Asaresult,therstsetofexperimentswilllookatth esemetricswhilevaryingthe capacityofthetightlinkfrom10to200Mbps,andsettingapa cketlossrateof1%,aone waypropagationdelayof10ms,andalowcongested(20%ofcro sstrafc)andhighly congested(75%)tightlink.Theseresultsshouldprovideso meguidanceastowhichtools arebettersuitedforlow,medium,orhighbandwidthnetwork s. 3.4.3ResultsThefollowingsectionsdescribetheresultsofperformingt hesignicantexperimentsand collectingdataabouttheestimationerror,overhead,esti mationtime,andreliabilityofthe evaluatedtools.Eachpointinthegraphsistheaverageofru nningeachexperiment10 timeswhichprovidesstatisticallymoresignicantresult swhencondenceintervalsare calculatedusingtheStudent'st-distribution.3.4.3.1VariableTightLinkCapacityFigures3.25to3.30showthat,ingeneralandforallthetool s,theestimationerrorgoes downwiththecapacityofthetightlink.Forthelowcongeste dscenario,thecapacityof thetightlinkdoesnotseemtohavemucheffectontheaccurac yofthetools,exceptinthe 54

PAGE 72

0.5 1 1.5 2 x 10 8 -100 -50 0 50 100 150 200 250 300 Tight Link Capacity (bps) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.25:Estimationerrorat20%crosstrafcwithvaria blecapacity,10msOWD,1% PLR. 0.5 1 1.5 2 x 10 8 -100 -50 0 50 100 150 200 250 300 Tight Link Capacity (bps) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.26:Estimationerrorat75%crosstrafcwithvaria blecapacity,10msOWD,1% PLR.caseofIGIinscenarioswithlowbandwidthlinks,andSpruce ,whichpresentsestimation problemsbeyond100Mbps.Spruce'sbehaviorinhighcapacit ylinkswasnotevidenced inFigure3.24becausethehighlevelinthefactorialdesign wasselectedtobe100Mbps. Figures3.25and3.26showthattheaccuracyofthetoolsismo stlyaffectedbythelevel ofnetworkcongestion.Itcanbeseenthatthetoolsprovidef airlystable(lowvariance) estimationsinlowcongestedscenariosregardlessoftheca pacityofthetightlinkwhile theypresenthighlyvariableestimationsinhighlycongest edscenarios. 55

PAGE 73

0.5 1 1.5 2 x 10 8 0 5 10 15 20 25 30 Tight Link Capacity (bps) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.27:Convergencetimeat20%crosstrafcwithvaria blecapacity,10msOWD, 1%PLR.Regardlessofthecongestionlevel,PathloadandSpruce(in thatorder)areshowntobe themostaccuratetools,withanestimationerroroflesstha n25%.Pathloadtendstooverestimatetheavailablebandwidthinlowcongestedscenario sandunderestimatethebandwidthinhighlycongestedones.Spruce,asmentionedbefore ,presentsestimationproblemswhenthelinkcapacitygoesbeyond100Mbps.IGIandAbin g,ontheotherhand, areshowntobehighlyinaccurate,especiallyinhighlycong estedlowcapacitylinks(with estimationerrorshigherthan100%).Finally,Pathchirpis fairlyaccurateinlowcongested scenarioswhileitpresentshighestimationerrorsinhighl ycongestedones,especially whenlowcapacitylinksareused.Pathchirpestimationerro rsareusuallylessthan50% exceptinthiscasewheretheerrorcanbeashighas200%.AsitisobservedinFigures3.27and3.28,thecapacityofthe tightlinkdoesnotseemto haveastrongimpactontheconvergencetimeofthetools.Onl yPathload,underhighly congestedscenariosseemstobeaffected,increasingtheco nvergencetimewiththecapacity.Otherthanthat,alltoolspresentfairlylowvariab ilityasthecapacityofthetight linkisvaried.Itcanbeseenthattheconvergencetimeofthe toolsalsoshowsawell knowntrend.Pathloadistheslowesttooltoconverge,follo wedbyPathchirp,Spruce, 56

PAGE 74

0.5 1 1.5 2 x 10 8 0 5 10 15 20 25 30 Tight Link Capacity (bps) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.28:Convergencetimeat75%crosstrafcwithvaria blecapacity,10msOWD, 1%PLR.IGI,andAbinginthatorder.Anotherimportantaspectistha ttheconvergencetimeof thetoolsdoesnotseemtobeaffectedbythelevelofcongesti oninanappreciableway. Regardlessofthecongestionlevel,Spruce,Pathchirp,and Abingshowlowvariation andlowerconvergencetimethanPathload.Pathchirpgivesa meanconvergencetimeof around13seconds.Spruce'sconvergencetimeisaround12se condswhileAbingisthe fastesttoolwithanalmostconstant1secondconvergenceti me.IGIisthesecondfastest toolbutitsconvergencetimedependsonthecongestionleve l;inlowcongestedscenarios IGI'sconvergencetimeisaround4secondswhileinhighcong estedcasesitmayreach15 secondsandpresentshighvariability.Withregardtotheoverhead,Figures3.29and3.30showthata sthecapacityofthetight linkisincreased,theoverheaddecreases.Thisisaclearin dicationthattheoverhead introducedbythetoolsisratherconstant,i.e.,thetoolsn eedasimilaramountofprobe packetstomaketheestimationregardlessofthelevelofcon gestionandcapacityofthe networks. 57

PAGE 75

0.5 1 1.5 2 x 10 8 0 2 4 6 8 10 Tight Link Capacity (bps) Overhead (%) Pathload IGI Spruce Abing Pathchirp Figure3.29:Overheadat20%crosstrafcwithvariablecapa city,10msOWD,1%PLR. 0.5 1 1.5 2 x 10 8 0 2 4 6 8 10 Tight Link Capacity (bps) Overhead (%) Pathload IGI Spruce Abing Pathchirp Figure3.30:Overheadat75%crosstrafcwithvariablecapa city,10msOWD,1%PLR. Fromtheresults,itisclearthatPathloadandAbingarethem ostintrusivetools.The amountofprobepacketsinPathloadtakesapproximately6%o fthetightlinkcapacity inlowcongestedscenariosand1.5%inhighlyloadednetwork s.Pathloadislessintrusiveinhighlycongestedscenariosbecauseitworksllingt heavailablepipecapacity. However,asthecapacityofthetightlinkisincreasedtheam ountofoverheadneededby thetooldecreases,i.e.,thetoolisabletondtheavailabl ebandwidthmoreefciently whentheavailablecapacityisbig.Theothertoolthatusest hesameinducedcongestion principleisPathchirp.However,asitisarguedin[32],Pat hchirpneedslessthan10%of theprobingtrafcthatPathloaduses.Pathchirp,sendsthe probepacketsusingadifferent 58

PAGE 76

10 20 30 40 50 60 70 80 -100 -50 0 50 100 150 200 250 300 One Way Propagation Delay (ms) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.31:Estimationerrorat5Mbpsforvariableonewayd elay,1%PLR,and75% crosstrafc. 10 20 30 40 50 60 70 80 -100 -50 0 50 100 150 200 250 300 One Way Propagation Delay (ms) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.32:Estimationerrorat100Mbpsforvariableonewa ydelay,1%PLR,and75% crosstrafc.gapdistributionandalgorithmthatmakesthetooltoconver gefasterand,therefore,sends fewerprobepackets.Ontheotherhand,Abinguilizesaxeda ndratherlargeamountof overheadtoperformtheestimations.Thisiswhytheoverhea ddecreaseswiththecapacityofthetightlinkandthelevelofcongestion.IGIandSpru cearethetoolsthatintroduce theleastamountofoverhead(lessthan0.5%).Theresultsfo rthisrstsetofexperiments conrmwhatisalreadyknownabouttheaccuracyofthetoolsf romdifferentauthors[32], [27],[29],and[18]. 59

PAGE 77

10 20 30 40 50 60 70 80 0 20 40 60 80 100 120 140 One Way Propagation Delay (ms) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.33:Estimationtimeat5Mbpsforvariableonewayde lay,1%PLR,and75% crosstrafc.3.4.3.2VariableOne-WayPropagationDelayThesecondsetofexperimentslookattheeffectoftheone-wa ypropagationdelay,which isvariedfrom10to80msecs.Thisisanotherscenariothatha snotbeenstudiedbefore andshouldsaywhethercurrentavailablebandwidthestimat iontechniquesareappropriate ornotinnetworkswithlongdelaylinks,suchasnetworkswit hsatelliteconnections. FromFigure3.24,itcanbeseenthatvariationsoftheone-wa ypropagationdelayonly affecttheestimationerrorandtheconvergencetimeofIGIa ndAbing,andPathload, respectively.Overheadplotsareaddedduetotheparticula rsituationofAbing,which showsasignicantoverhead(upto20%)whenthetightlinkca pacityislow(5Mbps). Experimentsareperformedwithlow(5Mbps)andhigh(100Mbp s)tightlinkcapacities andwithanend-to-endpacketlossrateof1%.Onlytheexperi mentresultsrelatedto highlycongestedscenarios(75%ofcrosstrafc)areinclud edbecausetheyarethemost challengingscenariostothetools.Withregardtotheestimationerror,whenthetightlinkisse tat5Mbps(Figure3.31), Abing'serrorisextremelyhigh(upto3000%)andincreasesw iththeone-waypropa60

PAGE 78

10 20 30 40 50 60 70 80 0 20 40 60 80 100 120 140 One Way Propagation Delay (ms) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.34:Estimationtimeat100Mbpsforvariableoneway delay,1%PLR,and75% crosstrafc.gationdelay.Pathchirpalsohasaveryhighestimationerro r(upto500%).(Thisiswhy theseresultsarenotincludedinthegraph.)SpruceandPath loadarethemostaccurate toolsinthisscenariowitha50%estimationerrorfollowedb yIGIwithanestimation errorintheorderof150%.Althoughtheincrementintheone-waydelayaffectstheaccu racyofthetoolscompared totheresultsobtainedinFigure3.26,mostofthetoolsseem tobeunaffectedbyadditionalincrements.Ingeneral,alltoolspresentoverestim ationproblemsinthisparticular scenario.Whenthetightlinkissetto100Mbps,Figure3.32s howsthattheestimationerrorofthetools,withtheexemptionofIGI,improvescompare dwiththe5Mbpsscenario, buttheone-waypropagationdelayseemstohavenomajorimpa cteither.Ingeneral,only PathloadandSpruceseemtobeadequateinthesecases,asthe estimationerrorofthe restofthetoolsisveryhigh.Here,Spruceisevenmoreaccur atethanPathloadwithan estimationerroroflessthan15%.(Noticethattheresultso fSpruceandPathloadarevery consistentwiththeonesshowninFigure3.26at100Mbps.)Regardingtheestimationtime,theonlyproblematictoolis Pathload.Whenthetightlink issetto5Mbps(Figure3.33),Pathloadpresentstheworstes timationtime,takingmore 61

PAGE 79

10 20 30 40 50 60 70 80 0 5 10 15 20 One Way Propagation Delay (ms) Overhead (%) Pathload IGI Spruce Abing Pathchirp Figure3.35:Overheadat5Mbpsforvariableonewaydelay,1% PLR,and75%cross trafc.than100secondstoconvergecomparedwithlessthan15secon dsfortherestofthetools. Whenthetightlinkissetto100Mbps,Figure3.34showsthat, althoughPathloadimprovesitsestimationtime,itstillistheworstperforming tool.ThislastFigureshowshow Pathload'sestimationtimeincreaseswiththeone-wayprop agationdelayfrom20seconds toaround60seconds.Thisbehaviorisexpectedsincethetoo ladjuststhetransmission rateaccordingtotheone-waydelayvariationshownbyaprob ingpackettrain.Thus,the longerthepropagationdelayis,theslowerthetoolreacts. Theothertoolspresentsteady estimationtimesregardlessofthecapacityofthetightlin kandtheone-waypropagation delay.Abingisdenitivelythefastesttoolwithanaverage 1secondestimationtime whileIGI,Pathchirp,andSprucepresentestimationtimeso faround12secondsorlower. Figures3.35and3.36showthatthepropagationdelayhasnom ajorimpactintheoverheadofthetools,i.e.biggerpropagationdelaysdonottran slateintomoreprobepackets. Inthecaseofatightlinkof100Mbpsalltoolsexperienceano verheadbelow1%;however,whenthetightlinkhasacapacityof5Mbpsthisvalueca nreachupto6%insome tools.TheexemptioninthislastcaseisAbing,withanoverh eadofashighas20%.This behaviorofAbingcanbeeasilyexplained.Abingsendsthesa mehighamountofxed 62

PAGE 80

10 20 30 40 50 60 70 80 0 5 10 15 20 One Way Propagation Delay (ms) Overhead (%) Pathload IGI Spruce Abing Pathchirp Figure3.36:Overheadat100Mbpsforvariableonewaydelay, 1%PLR,and75%cross trafc. 2 4 6 8 10 -100 -50 0 50 100 150 200 250 300 Packet Loss (%) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.37:Estimationerrorat5Mbpsforvariablepacketl ossrate,10msdelay,and 75%crosstrafc.sizeprobingpacketsregardlesstothepathcapacity,there fore,inlowcapacitylinks,itis expectedtobemoreintrusive.Itisworthnoticingthatthep ercentagevaluespresented thusfardonotindicatethatthetoolsintroducelowoverhea d.Noticethatevena1%of overheadinthecaseofthe100Mbpstightlinkcapacity,mean sthatthetoolintroducesan overheadof1Mbps. 63

PAGE 81

2 4 6 8 10 -100 -50 0 50 100 150 200 250 300 Packet Loss (%) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.38:Estimationerrorat100Mbpsforvariablepacke tlossrate,10msdelay,and 75%crosstrafc.3.4.3.3VariablePacketLossRatesThethirdsetofexperimentsaremeanttostudytheperforman ceofthetoolsinscenarios withdifferentpacketlossrates.Theseexperimentsareimp ortanttoconcludeaboutthe utilizationofcurrentavailablebandwidthestimationtoo lsinnetworkswithlossylinks, suchaswirelessnetworks,andtostudythereliabilityofth etools.Inthiscase,experimentsareperformedwithlow(5Mbps)andhigh(100Mbps)tigh tlinkcapacitieswith 75%ofcrosstrafcandwithone-waypropagationdelayof10m swhileincreasingthe packetlossratefrom1%to10%.SinceFigure3.24showsthatt heestimationerror,the estimationtime,andthereliabilityofthetoolsaretheres ponsevariablesaffectedby thepacketlossrate,thesewillbetheonlyplotsincludedin thispart.Withregardtothe estimationerror,whenthetightlinkissetto5Mbps(Figure 3.37),PathchirpandAbing gooutofanynormalrangeand,therefore,arenotincludedin theFigure. Pathloadpresentsasteadybehaviorwithanunderestimatio nof25%forpacketerrorrates of3%andhigher.Spruce,ontheothehand,presentsasteadya ndaccurateestimation regardlessofthepacketlossrate.Whenthetightlinkisset to100Mbps(Figure3.38), 64

PAGE 82

2 4 6 8 10 0 5 10 15 20 25 30 35 Packet Loss (%) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.39:Estimationtimeat5Mbpsforvariablepacketlo ssrate,10msdelay,and 75%crosstrafc. 2 4 6 8 10 0 5 10 15 20 25 30 35 Packet Loss (%) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.40:Estimationtimeat100Mbpsforvariablepacket lossrate,10msdelay,and 75%crosstrafc.thetoolspresentdifferentbehaviors.Forexample,theest imationerrorisabove100%for AbingandIGIwhileitisbelow 75%(underestimation)forPathchirpandPahtload,especiallyforpacketlossratesabove4-5%.Further,intheca seofPathload,theestimation errorisevenworse(closeto100%).Withregardtotheestimationtime,theresultsusingtightl inksof5and100Mbps,depictedinFigures3.39and3.40,showthatthebehaviorofthe toolsissimilartoprevious scenarios,withPathloadtakingthelongesttime.Forthe5M bpslinkcase,Pathloadgoes 65

PAGE 83

2 4 6 8 10 0 20 40 60 80 100 Packet Loss (%) Reliability (%) Pathload IGI Spruce Abing Pathchirp Figure3.41:Reliabilityat5Mbpsforvariablepacketlossr ate,10msdelay,and75% crosstrafc. 2 4 6 8 10 0 20 40 60 80 100 Packet Loss (%) Reliability (%) Pathload IGI Spruce Abing Pathchirp Figure3.42:Reliabilityat100Mbpsforvariablepacketlos srate,10msdelay,and75% crosstrafc.from131secondsto46secondswhenthepacketlossratesgofr om1%to10%.Inthis case,asthepacketlossrateincreses,Pathloadprovidesfa sterbutworseestimates.As before,SpruceandPathchirpeachtakesaround12secondsto convergeforbothtightlink capacities.Ontheotherhand,IGI'sestimationtimetendst oincreasewiththepacketloss rateandishigherthanSpruce'sandIGI'sinthe5Mbpscase.A sexpected,Abingisthe fastesttoolwithsteadyestimationtimesevenlowerthan1s econd. 66

PAGE 84

AsshowninFigure3.24,thereliabilityofAbingandSprucei saffectedonlywhenthe packetlossrateisincreased.Theresultsofthereliabilit yexperimentsareshowninFigures3.41and3.42forthe5and10Mbpstightlinkcapacitycas es.Fromthetwoplots, itisclearthatAbingandSprucearetheleastreliabletools .Spruceseemstohaveproblemsprovidinganestimatewithpacketlossratesbeyond6%w hileAbinghasproblems regardlessofthelossrate.3.4.3.4VariableAmountofCrossTrafcThenextfactorvariedistheamountofcrosstrafcinthetig htlink.Again,theexperimentsperformedincludedlow(5Mbps)andhigh(100Mbps)tig htlinkcapacitieswith aone-waypropagationdelayof10msandapacketlossrateof1 %,butthistimethe amountofcrosstrafc(congestion)isvariedinthetightli nkfrom10%to80%.AccordingtoFigure3.24,variationsintheamountofcrosstrafco nlyaffecttheestimationerror andtheestimationtimeofthetools.However,additionalplotsarealsoincludedfortheoverhea d(Figures3.47and3.48), whichagainshowsthesamehighoverheadvalueofAbinginthe 5Mbpscase,similar totheoneshowninFigure3.35.Figures3.43and3.44showthe estimationerror. Itisclearthatregardlessofthecapacityofthetightlink, theestimationerrorofthetools increaseswiththeamountofcrosstrafcandwithatendency tooverestimatethereal availablebandwidth.Itisalsoclearthatthetoolshavemor edifcultyestimatingthe availablebandwidthaccuratelywhenthetightlinkcapacit yis5Mbps.Inthiscase,it canbeseenthatSpruceandPathloadarethebesttools,intha torder.Therestofthetools presentveryinnacurateestimatesinmanycases.Inparticu lar,asisalsopresentedin[17], 67

PAGE 85

10 20 30 40 50 60 70 80 -100 -50 0 50 100 150 200 250 300 Cross Traffic in the Tight Link (%) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.43:Estimationerrorat5Mbpsforvariable%ofcros strafc,10msOWD,and 1%PLR. 10 20 30 40 50 60 70 80 -100 -50 0 50 100 150 200 250 300 Cross Traffic in the Tight Link (%) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.44:Estimationerrorat100Mbpsforvariable%ofcr osstrafc,10msOWD, and1%PLR.Abing'saccuracyshowsalsoproblemswhentheavailableban dwidthdropsbelow60%of thetightlinkcapacity.Withregardtotheestimationtime,theresultsaresimilart otheonesobtainedinpastexperiments.Pathloadagainisshowntohaveproblemsintheca seofthe5Mbpstightlink, especiallyinthemediumtohighcongestionrange.Asitissh ownin[17],Pathloadtakes about20secondstoconvergewhenthecapacityofthetightli nkishigh.Therestofthe toolspresentstablebehaviorsregardlessofthecapacityo fthetightlinkandtheamount ofcrosstrafc,andveryconsistentwithpastresults.Agai n,Abingisthefastesttool 68

PAGE 86

10 20 30 40 50 60 70 80 0 10 20 30 40 50 Cross Traffic in the Tight Link (%) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.45:Estimationtimeat5Mbpsforvariable%ofcross trafc,10msOWD,and 1%PLR. 10 20 30 40 50 60 70 80 0 10 20 30 40 50 Cross Traffic in the Tight Link (%) Estimation Time (s) Pathload IGI Spruce Abing Pathchirp Figure3.46:Estimationtimeat100Mbpsforvariable%ofcro sstrafc,10msOWD, and1%PLR.followedbyIGIandthenSpruceandPathchirpwithestimatio ntimesofapproximately 1,6,12,and13seconds,respectively.TheoverheadintroducedbythetoolsisshowninFigures3.47 and3.48.Asexplainedbefore,thebehaviorofPathloadistheexpectedone:theoverh eadreduceswiththeamount ofcongestion.Theothertoolsintroducethesameamountofo verheadregardlessofthe amountofcongestion.However,theoverheadcanbesignica ntif,asinthecaseofAbing, thetoolinsertsaconstantbuthighamountofprobingpacket sintothenetwork.Forexam69

PAGE 87

10 20 30 40 50 60 70 80 0 5 10 15 20 Cross Traffic in the Tight Link (%) Overhead (%) Pathload IGI Spruce Abing Pathchirp Figure3.47:Overheadat5Mbpsforvariable%ofcrosstrafc ,10msOWD,and1% PLR. 10 20 30 40 50 60 70 80 0 5 10 15 20 Cross Traffic in the Tight Link (%) Overhead (%) Pathload IGI Spruce Abing Pathchirp Figure3.48:Overheadat100Mbpsforvariable%ofcrosstraf c,10msOWD,and1% PLR.ple,inthecaseofthe5Mbpstightlinkcapacity,Figure3.47 showsthatAbing'soverhead isaround19.4%ofthetightlinkcapacity,or950kbpsoutofa 5Mbpslink. 3.4.3.5VariableCrossTrafcPacketSizeFinally,theeffectintheestimationerrorwhenvaryingthe crosstrafcpacketsizeis studied.Heretwodifferentcongestionlevels(20%and75%o fcrosstrafc)andtightlink 70

PAGE 88

400 600 800 1000 1200 1400 -50 0 50 100 150 200 250 Cross Traffic Packet Size (Bytes) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.49:Estimationerrorat5Mbpsand20%crosstrafcf orvariablecrosstrafc packetsize,10msOWD,and1%PLR. 400 600 800 1000 1200 1400 -50 0 50 100 150 200 250 Cross Traffic Packet Size (Bytes) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.50:Estimationerrorat5Mbpsand75%crosstrafcf orvariablecrosstrafc packetsize,10msOWD,and1%PLR. 400 600 800 1000 1200 1400 -50 0 50 100 150 200 250 Cross Traffic Packet Size (Bytes) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.51:Estimationerrorat100Mbpsand20%crosstraf cforvariablecrosstrafc packetsize,10msOWD,and1%PLR. 71

PAGE 89

400 600 800 1000 1200 1400 -50 0 50 100 150 200 250 Cross Traffic Packet Size (Bytes) Estimation Error (%) Pathload IGI Spruce Abing Pathchirp Figure3.52:Estimationerrorat100Mbpsand75%crosstraf cforvariablecrosstrafc packetsize,10msOWD,and1%PLR.capacities(5and100Mbps)arestudied.Theone-waypropaga tiondelayisof10msand thepacketlossrateof1%.BylookingatFigures3.49to3.52, itcanbeseenthatthecross trafcpacketsizehasnomajoreffectontheestimationerro rofthetools,whichisfairly constantregardlessofthepacketsize.Itisalsoclearthattheerrorissmallerandmorestableinlo wcongestedscenariosthan inhighlycongestedones,whichisalsoconsistentwithpast results.Also,itcanbeseen thatPathchirpandAbingpresentestimationproblemsinthe caseoflowcapacityandcongestedtightlinks.Thisisalsoveryconsistentwithotherr esultspresentedsofar,which leadustoconcludethatneitherPathchirpnorAbingaregood choicesforlowbandwidth andhighlycongestedchannels.3.5ApplicabilityofCurrentAvailableBandwidthEstimati onTools Thissectionpresentsgeneralandspecicconclusionstoan swertheoriginalquestions regardingtheapplicabilityofcurrentavailablebandwidt hestimationtools.Therstgeneralconclusion,andperhapsthemostimportantone,isthat currenttoolsarestillfarfrom 72

PAGE 90

havinggoodperformanceformany,ifnotmost,oftheenvisio nedapplicationsandnetworkingenvironments.Forexample,thetoolwiththebestes timationtimeisAbing, whichprovidesestimationsinaround1second.Simplyput,a estimationtimeof1secondmayleaveoutmanyreal-timeapplicationsthatwouldnee dtohaveestimatesinthe orderofmilliseconds.Oneexampleofthiscasemaybetheow controlmechanismofan availablebandwidth-basedtransportlayerprotocol.Second,noneoftheexistingtoolsintroducelowenoughover headtobeusedinthose applicationsthatwillworkonaper-connectionbasis.Thee xampleofthetransportlayer protocolcomestomindagain;ifTCPoranysubstitutetransp ortlayerprotocolwere touseanavailablebandwidthtoolforowcontrol,theamoun tofoverheadhastobe consideredgiventhelargenumberofconnectionsthatcurre ntlygothroughtheInternet. Forthesetypeofapplications,newmethodshavetobedevise dtousethedatapacketsas probepackets,reducingtheoverheadtoacceptablelevels.Third,toolsbasedonthePGMareusuallylessintrusivebutl essaccuratethantheones basedonthePRM.Fourth,alltoolspresenttheirworstbehav iorandperformanceinhighly loadedscenarios.Finally,currentavailablebandwidthtoolsaremostly“net work-unaware”andaccuracy problemsmayoccurbecausethemechanismsoftheunderlying networkingtechnology arenotbeingmodeledorincludedinthemethodologiesandto ols.Forexample,thereare wellknownmediumaccessprotocolsandback-offalgorithms inwirelessnetworksthat mayintroduceerrors.Morespecicconclusions,drawnfrom theresultsoftheperformanceevaluation,areasfollows: Mostofthetoolsarebarelyaffectedbythecapacityoftheti ghtlink.OnlyIGI andSprucepresentaccuracyproblems-IGIatlowlinkcapaci tiesandSprucebe73

PAGE 91

yond100Mbps.Ifthecapacityofthetightlinkandthelevelo fcongestionare combined,theresultsshowthatPathloadandSprucearethem ostaccuratetools. Theseresultsindicatethattheymaybethebestcandidatesf orlow,medium,and highcapacitynetworks.Ontheotherhand,IGI,Abing,andPa thchirparehighly inaccurate,especiallyinhighlycongestedscenarios.Exc eptforPathloadinhighly congestedscenarios,theestimationtimeofthetoolsisnot impactedbyvariations inthecapacityofthetightlink.Pathloadistheslowesttoo ltoconvergefollowedby Pathchirp,Spruce,IGI,andAbing,inthatorder.Abingprov idesanalmostconstant estimationtimeofapproximatelyonesecond.Intermsofove rhead,Pathloadand Abingarethemostintrusivetools;therefore,thesetoolsa renotthebestchoicesfor lowcapacityandwirelessnetworks.Pathload'soverheadva riesanddependsonthe capacityandlevelofcongestionofthenetwork.Thisisduet othefactthatPathload utilizestheprincipleofinducedcongestion.Abing'sover headisxedbutlarge. Spruce,Pathchirp,andIGIaretheleastintrusivetools. Currentavailablebandwidthestimationtoolsbehavediffe rentlytodifferentend-toendpropagationdelays.Ingeneral,furtherincreasesinpr opagationdelayshave nomajorimpactontheestimationerrorofthetools.Pathchi rp,Abing,andIGI presentveryhighestimationerrorsregardlessofthecapac ityofthenetwork.On theotherhand,PathloadandSprucearebarelyaffectedbyth epropagationdelay; bothtoolsprovideestimateswitherrorsbelow50%andlowva riability.Withregard totheestimationtime,theresultsshowthatwiththeexempt ionofPathload,the toolsseemtobeunaffectedbyincreasesinthepropagationd elay.Whilemosttools presentestimationtimesbelow20secondsandwithoutvaria tion,Pathloadhas highandincreasingestimationtimes.Finally,thepropaga tiondelaydoesnothave amajorimpactontheoverheadofthetools.Comparingallthe tools,itcanbe 74

PAGE 92

concludedthatSpruceisthebesttoolforlongpropagationd elayscenarios,asit presentsthebestcombinationofresults:Spruceisthemost accurate,oneofthe fastest,andonewiththelowestoverhead.Sprucemightbeag oodchoiceforthose applicationsrunningoversatellitelinksortransatlanti cberopticlinks.However, itsperformanceinscenarioswithlinksofmorethan100Mbps isstillinquestion, giventheresultsshowninFigure3.26.Inthosecases,Pathl oadisthebestsecond tool,ifthehighestimationtimeisnotanissue. Spruceisthemostaccurateandimmunetooltovariationsint hepacketlossrate. Therestofthetoolspresenthighestimationerrorsasthepa cketlossrateisincreased.Theestimationtimeresultsofthetoolsarealsosi milartopastresultsin termsofvariationandabsolutevalues,withPathloadprese ntingthelongestestimationtimefollowedbyIGI,Pathchirp,Spruce,andAbing,i nthatorder.Spruce's estimationtimedoesnotseemtobeaffectedbythepacketlos sratesinceitstays ataround12seconds.Thisestimationtimeisalsoseeninthe otherexperiments, makingSpruceaverypredictabletoolinthisregard.Theonl yproblemisthatin scenarioswithhighcongestionandpacketlossrateshigher than6%,Sprucefails toprovideanestimateinmanycases.Abing,althoughfaster thanSpruce,showed tobeevenmoreunreliable.TheseresultssuggestthatSpruc eisthebesttoolfor networkswithvaryingandhighpacketlossrates,suchaswir elessnetworks. Intheanalysisofthetoolsunderdifferentcongestionleve ls,itisclearthatregardlessofthecapacityofthetightlink,theestimationerroro fthetoolsincreaseswith theamountofcrosstrafcandwithatendencytooverestimat etherealavailable bandwidth.Itisalsoclearthatthetoolshavemoredifcult yestimatingtheavailablebandwidthaccuratelywhenthecapacityofthetightlin kis5Mbps.Inthis case,itcanbeseenthatSpruceandPathloadarethebesttool s,inthatorder.The 75

PAGE 93

restofthetoolsprovideveryinaccurateestimatesinmanyc ases.Withregardtothe estimationtimeandoverhead,theresultsaresimilartothe onesobtainedinpast experiments.Asaresult,itcanbeconcludedthatSpruceand Pathloadarethebest choicesforcongestedandlowcapacitynetworks,suchasDSL andcablemodem accessnetworks. Resultsshowthatthesizeofthecrosstrafcpackethasnoma joreffectonthe estimationerrorofthetools.Theestimationerrorisfairl yconstantandconsistent withpastresults.Itisalsoclearthattheerrorissmallera ndmorestableinlow congestedscenariosthaninhighlycongestedones,whichis alsoconsistentwith pastresults. Althoughpreviousconclusionsarebasedontheaveragebeha viorofthetoolsunder differentscenarios,thereareisolatedresultsspecially intheestimationerrorevaluationwhicharefarfromthataveragebehavior.Forexample,I GIshowedaverylow estimationerror(around0 : 5%)whenthelinkissetto5Mbps,10mspropagation delay,1%packetlossrate,20%ofcrosstrafcinthetightli nk,andanycrosstrafc packetsize.Therefore,anyconclusionforaparticularnet workscenariomustbe drawnfromthedetailedobservationofFigures3.25through 3.52. Table3.4summarizesalltheresultsprovidedthusfarandsh owsthebest/worstperformingtoolforeachofthefactorsandresponsevariablesstudi ed.Intermsoftheapplicabilityofthecurrentavailablebandwidthestimationtoolsint heenvisionedapplications,Table3.5includesaqualitativeassessmentoftheirrequirem entsandthebesttoolsforeach application.Theassessmentisqualiedaslow,medium,orh ighiftheapplicationneeds low,medium,orhighestimationerror,overhead,estimatio ntime,orreliability.From theTable,itcanbeseenthatdespitethecurrenteffortsint hedesignanddevelopmentof 76

PAGE 94

Table3.4:Summaryofbest/worstperformingtools. Main Sec. Best/worsttools Networking Factor Level Factor Estim.error(%) Time(s) Overh.(%) Rel.(%) Environment Low XT Pchirp-Abg/IGI Abg/Pload Spr/Pload All100 Alltypes T.Link High 20% IGI-Pload/Spr Abg/Pchirp Spr/Pload of Cap. Low XT Pload-Spr/IGI Abg/Pload Spr/Abg All100 wired High 75% Pload/IGI Abg/Pload Spr/Pload networks Low Cap. Pload-Spr/Abg Abg/Pload Spr/Abg All100 LAN,WAN, OWD High 5M Pload-Spr/Abg Abg/Pload Spr/Abg satellite XT:75% Low Cap. Spr-Pload/IGI Abg/Pload Spr/Abg All100 Highspeed High 100M Spr-Pload/IGI Abg/Pload Spr/Abg LANandWAN Low Cap. Pload-Spr/Abg Abg/Pload Spr/Abg All100 Lowand PLR High 5M Pload-Spr/Abg Abg/Pload IGI/Abg All/Abg highspeed XT:75% Low Cap. Spr/IGI Abg/Pload IGI/Pload All100 wireless High 100M Spr/IGI Abg/Pload IGI/Abg All/Abg networks Low Cap. Spr/Pchirp Abg/Pload Spr/Abg All100 DSL,cable XT High 5M Spr/Abg-Pchirp Abg/Pload Spr/Abg accessnetworks Amount Low Cap. IGI-Pload/Spr Abg/Pload Spr/Pload All100 Alltypesof High 100M Pload-Spr/IGI Abg/Pload Spr/Pload wirednetworks Table3.5:Qualitativeassessmentofapplicationrequirem entsandbesttoolsforthesetof representativeapplications. Application Accuracy Overhead Time Reliability BestTool MainIssues SLACompliance High Medium Med/High Medium Spr/Pload Overhead N.Management Medium Medium Med/High Low/Med Spr/Pload Overhead TrafcEng. Medium Low Low High Noneyet Overheadandtime FlowControl Medium Low Low High Noneyet Overheadandtime Security Low/Med Medium Low High Noneyet Estimationtime Admiss.Control Med/High Low/Med Low High Noneyet Overheadandtime newmethodologiesandtoolstoestimatetheend-to-endavai lablebandwidth,mostofthe toolsarenotentirelysuitableformostoftheenvisionedap plications.Currentavailable bandwidthestimationtoolsonlyndapplicabilityinveryr elaxedscenarioswherethe overheadandtheestimationtimeofthetoolsarenotbigissu es. 77

PAGE 95

Chapter4:HMMApproachtoAvailableBandwidthEstimationHiddenMarkovmodels(HMM)hasbeentraditionallyappliedt ospeech,handwritingand gesturerecognition,weatherprediction,andinBioinform aticstomodelDNAandprotein sequences.Theseareproblemswherefewandnoisyinformati onisusedtoestimatea stateofthemodeledsystem.Theavailablebandwidthestima tionproblemalsorequires asfewsamplesaspossiblesothatthenetworkisnotcongeste dwithadditionalprobing trafc.Inaddition,asitisshowninSection1.2,thereares everalissuesthataffectthe accuracyofeachmeasurementobtainedformeverysampleoft henetwork.Buildinga hiddenMarkovmodeloftheavailablebandwidthprovidesame chanismtoadjustthe erraticmeasurements(nottoavoidthem).AlthoughthisistherstworkthatappliesHMMintheavailab lebandwidthestimation problem,therearepreviousstudiesthatuseHMM'sinnetwor king.In[44],apacketleveltrafcHMMisusedtomodeltrafcsourcesandestimate PacketSizeandInter PacketTime.ThemodelisvalidatedusingInternettraceswi thSMTPandHTTPtrafc. Asimilarstudyperformedin[45]buildsHMMprolestoclass ifynetworkapplications fromInternettrafctraces.Someoftheapplicationsstudi edareFTP,SMTP,HTTP,and Telnet.Anotherstudy[46]usesaHMMtomodelfadingcommuni cationchannelsandto ndclosed-formsolutionsfortheprobabilitydistributio noffadedurationandnumberof levelcrossings. 78

PAGE 96

Thischapterintroducestheconceptandelementsofahidden Markovmodel,describes howthemodelisappliedtotheavailablebandwidthestimati onproblem,andpresentsthe algorithmsusedtoinfertheavailablebandwidthstatefrom welldenedobservationsof thesystem.4.1DiscreteHiddenMarkovModelsAdiscreteMarkovmodelisrepresentedbyasetof N distinctstateswherethestateat time t isdenotedby q t andtheprobabilityofgoingfromonestatetoanotherdepend son thevaluesofthepreviousstates[47]: P q t + 1 = S j j q t = S i ; q t 1 = S k ;::: : (4.1) Inarst-orderMarkovmodel,thestateattime t + 1dependsonlyonthestateattime t Thatis, P q t + 1 = S j j q t = S i ; q t 1 = S k ;::: = P q t + 1 = S j j q t = S i (4.2) Inan observable Markovmodel,atanytime t q t isknown.Therefore,thesequenceof stateswhenthesystemmovesfromonetoanotherisknown.Ina hiddenMarkovmodel, thestateisnotobservable(itis hidden )butcanbeinferredfromagivenobservationthat isaprobabilisticfunctionofthestate. 79

PAGE 97

4.1.1HMMElementsAsdescribedin[48],ahiddenMarkovmodeliscomposedbythe followingveelements: Nstatesinthemodel.Thisisanitenumberthatrepresentst henumberofstates. Thestateattime t isdenotedas q t .Thesetofstatesisdenotedby: S = S 1 ; S 2 ;:::; S N (4.3) Mdistinctobservationsymbolsperstate.Theseareallthep ossibleoutcomesofa state.Thesetofobservationsymbolsisdenotedby: V =u1 ;u2 ;:::;uM (4.4) Statetransitionprobabilitymatrix(A).Eachelementofth ismatrixhastheprobabilityoftransitioningfromonestatetoanother.Thesumof eachrowinthematrix hastobeone.Thismatrixisdenotedby: A =[ a ij ] wherea ij = P ( q t + 1 = S j j q t = S i ) for 1 i ; j N (4.5) Observationprobabilities(B).Thisisasetofprobabiliti esthatindicateshowlikely isthatattime t anobservation O t isgeneratedbyeachstatefromtheset S .Thisset denedforeachstate1 j N isdenotedby: B =[ b j ( m )] whereb j ( m )= P ( O t =um j q t = S j ) for 1 m M (4.6) 80

PAGE 98

Figure4.1:AvailablebandwidthMarkovmodel.Statesrepre sentlevelsofbandwidth availability. Initialstateprobabilities( P ).Thisisavectorwiththeprobabilitiesthateachstateis therstinthestatesequencethatgeneratedtheobservatio ns: P =[pi ] wherepi = P ( q 1 = S i ) for 1 i N (4.7) Thelastthreesetofprobabilitiesareusuallydenotedasl=(A,B, P )toindicatethecompleteparametersetofthemodel.4.2HMMtoEstimateAvailableBandwidthTheavailablebandwidthinanend-to-endpathcanbemodeled by N states,eachone representingcertainlevelofavailability.Forexample,i ntheve-staterepresentation showninFigure4.1,theavailablebandwidthcouldbeinoneo fLow(L),MediumLow (ML),Medium(M),MediumHigh(MH),andHigh(H)states.That is,itcouldbelocated inanyspareutilizationrangefrom[0,0.2),[0.2,0.4),[0. 4,0.6),[0.6,0.8),or[0.8,1].By samplingtheavailablebandwidthduringtime T ,thesequenceofstatesvisitedduring thatperiodcanbedetermined.Then,theaveragestatevisit edduring T ,calculatedas theaverageofthemiddlepointsofeachstaterange,isanest imationoftheavailable bandwidthduringthatperiodoftime. 81

PAGE 99

SincetheaveragetimescaleinEquation1.2isverysmall(mi lliseconds),itisassumed thattransitionsfromonestatetoanotherduringthatperio dgonofartherthanonestate apart.Therefore,theMarkovchainrepresentingtheavaila blebandwidthprocess,asitis showninFigure4.1,hasaone-steptransitionprobabilitym atrixdeterminingmovements betweenavailablebandwidthlevels.However,availablebandwidthstatescannotbedirectlyobs erved,theyarehidden,since theend-to-endestimatorsdonothaveinformationaboutban dwidthconsumptioninintermediaterouters.Rather,availablebandwidthestimato rssamplethenetworkpathwith probingpacketsthatconveypacketdispersioninformation ,whichcanbeusedbyahidden Markovmodeltoinferthenon-observablestates.4.2.1ProbingSamplingMethodInordertogetinformationabouttheavailablebandwidthdy namicsduringtheperiod T thenetworkissampledusingtheprobegapmodel(seeSection 2.1.1).AssumingthesingletightlinkmodelshowninFigure1.4,aprobingpacketpai renterstherouterwitha D in separation.Then,duetotheinteractionoftheprobingpack etpairswiththecrosstrafcin therouter'soutputqueue,theyleavethelinkwithadiffere ntseparation,ordispersion.It iswellknownthatthisvariationhasastrongcorrelationwi ththeamountofcrosstrafc inthequeueduringthesamplingperiod,whichcanbeusedtoe stimatetheavailable bandwidth.Therelativedispersionsufferedbytheprobing packetscanbedenedas:et =( D out D in ) = D in ; t = 1 ::: T (4.8) 82

PAGE 100

whichisameasureofthetightlinkutilizationasseenbyapr obingpacketpairattime t .Then,knowingthecapacityofthetightlink C t ,theend-to-endavailablebandwidthat time t canbeestimatedby: A t = C t ( 1 et )= C t 1 D out D in D in (4.9) SimilartootherPGM-basedtools,thevalueofthetightlink capacity( C t )canbecalculatedusingwellknownandaccuratetools,likePathrate[38 ]. 4.2.2ModelDescriptionSinceintheavailablebandwidthmodelproposedinFigure4. 1thestatescannotbedirectlyobserved,aHMMapproachcanbeusedtondthestatese quenceassociatedwith thedispersionsobservedduringthesamplingperiod.Themo del,whichisshowninFigure4.2,isahiddenMarkovmodelwithdiscretehiddenstates q representingtheavailablebandwidthlevels(ranges)anddiscreteobservationva riablesxrepresentingprobing packetpairdispersions.Aparticularobservationhasasso ciatedaprobability B ofbeing generatedbyaparticularhiddenstate.Availablebandwidt htransitionsgofromtime t = 1 totime t = T .Transitionsbetweenstatesaredeterminedbyprobabiliti esspeciedinthe transitionprobabilitymatrix A Thismodel,whichisrenedwitheverynewobservation,isus edtodeterminethemost probablestatesequence( Q = q 1 ; q 2 ; ; q T )responsibleforwhathasbeenobserved during T .Attheend,theaveragestateintheestimatedsequenceofst ateswillbethe averageavailablebandwidthduringtimeperiod T 83

PAGE 101

1 t t-1 T q1 qt-1 qt qT AA A B BB BFigure4.2:HiddenMarkovmodelforavailablebandwidthestimations. AccordingtothecomponentsofahiddenMarkovmodeldescribedinSection4.1.1,the availablebandwidthestimationmodelhasthefollowingveelements: Numberofstatesinthemodel( N ). Thebiggerthenumberofstates N ,thelongerthetimeneededtoprovideanestimation.Thesetofstatesisdenedby S = f S 1 ; S 2 ;:::; S N g wheretheavailable bandwidthlevelgrowsfrom S 1 (low)to S N (high).Thestateattime t isdenoted by q t .Thedefaultnumberofpossiblestatesintheestimationtooldevelopedinthis workisten,representingavailablebandwidthrangesof[0,0.1),[0.1,0.2),...,and [0.9,1]. Numberofdistinctobservationsymbolsperstate( M ). Theseareallthepossibleoutcomesofastate.Thatis,thesetofsymbolsassociated withobserveddispersionsfromtheprobingsamplingmethod.Thedefaultnumber ofdistinctobservationsymbolsintheestimationtoolpresentedinthisworkisten, anditisdenotedby V = fu1 ;u2 ;:::;u10 g .Eachsymbolisadecimalnumberfrom 1to10.Thesetensymbolsrepresentobservedrelativedispersionvalueseinthe rangesu1 [0,0.1),u2 [0.1,0.2),...,andu10 [0.9,1].Therefore,everysingle observationattime t hastobeconvertedtoadiscretevaluext associatedwitha symboluby: 84

PAGE 102

xt = d M j 1 et je (4.10) Thatis,Equation4.10determinesthediscreteobservation valuext thatcorresponds toacontinuousobservationet ,wherext 2 V .Thereisalsoarelationbetweenstates (availablebandwidth)andobservations(associatedwithe)asitisdenedinEquation4.9. Statetransitionprobabilitymatrix( A ). A =[ a ij ] where a ij = P ( X t + 1 = S j j X t = S i ) ,1 i ; j N .Sinceonlyone-step transitionsbetweenstatesarepossible,thenumberofunkn ownelementsinthe matrixisreducedtothethreemaindiagonals: A = 266666666664 a 1 ; 1 a 1 ; 2 0 0 a 2 ; 1 a 2 ; 2 a 2 ; 3 0 ... 0 . . . . 0 ...0 a N 1 ; N 2 a N 1 ; N 1 a N 1 ; N 0 0 a N ; N 1 a N ; N 377777777775 (4.11) Observationprobabilities( B ). Asexplainedbefore,thisisasetofprobabilitiesthatindi cateshowlikelyitisthat attime t anobservationsymbolxt isgeneratedbyeachstatefromtheset S .More specically, B =[ b j ( m )] where b j ( m )= P (xt =um j X t = S j ) for1 m M ,1 j N ,and Mm = 1 b j ( m )= 1: 85

PAGE 103

B = 266666666664 P (u1 j S 1 ) P (u2 j S 1 ) P (uM j S 1 ) P (u1 j S 2 ) P (u2 j S 2 ) P (uM j S 2 ) P (u1 j S 3 ) P (u2 j S 3 ) P (uM j S 3 ) ... ... ... ... P (u1 j S N ) P (u2 j S N ) P (uM j S N ) 377777777775 (4.12) Itisexpectedthatsmallvaluesofxaretheresultofahighlyloadednetworkand thereforemorelikelygeneratedbyaloworderstate(oneind icatinglowavailable bandwidth)andconversely.Basedonthis,probabilityvalu escanbeassignedand xedinthemodelasshownbelowinthecaseoftenstatesandte nobservation symbols.Notethat0.35and0.25arehighprobabilityvalues assignedtomore likelystates: B = 266666666664 0 : 350 : 250 : 150 : 100 : 050 : 030 : 030 : 020 : 010 : 01 0 : 200 : 350 : 200 : 100 : 050 : 030 : 030 : 020 : 010 : 01 0 : 080 : 200 : 350 : 200 : 070 : 030 : 030 : 020 : 010 : 01 ... ... ... ... ... ... ... ... ... ... 0 : 010 : 010 : 020 : 030 : 030 : 050 : 100 : 150 : 250 : 35 377777777775 (4.13) Initialstateprobabilities( P ). Ithastheprobabilitiesforeachstatetobetherstinthest atesequencethatgeneratedtheobservations. P =[pi ] wherepi = P ( X 1 = S i ) for1 i N : P =[p1p2 pN ]=[ P ( X 1 = S 1 ) P ( X 1 = S 2 ) P ( X 1 = S N )] (4.14) Table4.1summarizesallthevariablesusedintheestimatio nmodel. 86

PAGE 104

Table4.1:AvailablebandwidthHMMvariables. Variable Description C Tightlinkcapacity D in Packetpairseparationbeforethetightlink D out Packetpairseparationafterthetightlink N Numberofstatesrepresentingavailablebandwidthlevels S Setofstates(lowtohigh): S = S 1 ; S 2 ;:::; S N M Numberofdistinctobservationoutcomes V Setofobservations: V =u1 ;u2 ;:::;uM A Statetransitionprobabilitymatrix B Observationprobabilities P Initialstateprobabilities T Samplingperiod. t = 1 ::: T et Relativetimedispersionattime t q t Stateattime t Q Statesequence: Q = X 1 ; X 2 ;:::; X T xt Observationsymbolattime t O Observationsequence: O =x1 ;x2 ;:::;xT 4.2.3ParameterEstimationGivenanobservationsequence O =x1 ;x2 ;:::;xT ,thatis,asetofsamplesfromthenetworkduring T ,itisdesiredtoestimatethemodellthatmostlikelygeneratedthatsequence,i.e.themodell=(A,B, P )forwhichthe P ( O jl) ismaximized.Thesolution tothisproblemisgivenbyaniterativeprocedureformulate dintheBaum-Welchalgorithm[49].Theestimationtoolpresentedinthisworkhasimplementeda modiedversionofthe Baum-WelchalgorithmwritteninCbyTapasKanungo[50].The rearetwomainmodicationstothealgorithm.Therstoneisthattheinitialt ransitionprobabilitymatrix A israndomlygeneratedtobeaone-steptransitionmatrix.Th erefore,onlythethree maindiagonalsinthematrixhaveprobabilityvalues.These condmodicationisthatthe 87

PAGE 105

observationprobabilitymatrix B isxedsothattheprobabilitiesofobservationsbeing generatedbythestatesdonotchange.Thisisduetothefactt hatitisexpectedthathigh congestedlinkswillincreasethedispersionbetweenpacke tsandviceversa.Indeed,a linkwithzerocrosstrafcgenerateszero(orclosetozero) dispersionbetweenthepairof packets.Thealgorithm,whichhasatimecomplexityof O ( N 2 T ) ,runsasfollows: Settheinitialmodell0 witharandomlygeneratedone-steptransitionmatrix A 0 andinitialstateprobabilityvector P 0 .Matrix B 0 isinitializedasexplainedinthe previoussection. Calculateanew bl=( b A ; B 0 ; b P ) basedonl0 andtheobservationsequence O .See[48] formoredetails. if logP ( O = bl) logP ( O =l0 ) < 0 : 001thenstop elsel0 blandgotostep2. Notethat B = B 0 allthetimeasexplainedbefore. 4.2.4StateSequenceEstimationWithanupdatedl,thenextproblemistondthestatesequence Q = q 1 ; q 2 ; ; q T that maximizesthelikelihoodof P ( q 1 ; q 2 ; ; q T j O ;l) .Thatstatesequenceisusedtocalculatetheaverageavailablebandwidthduring T .Thisisdonebyusinganotheriterative algorithm,theViterbialgorithm[51],whichalsohasa O ( N 2 T ) timecomplexity.The algorithmselectsthemostlikelypathfromaparticularsta tetoallpossiblepathsand doesthesameforeachstate.See[48]or[50]forimplementat iondetailsofthealgorithm. Thenalmostlikelypathrepresentsthelevelsofavailable bandwidththattheprobing 88

PAGE 106

samplingpacketshaveobservedduringthesamplingtime.As denedinEquation1.1, thenalestimationisbasedontheaverageutilizationobse rvedduring T .Therefore,the availablebandwidthiscalculatedastheaverageofallstat esinthesequence: A = Q N C t (4.15) 89

PAGE 107

Chapter5:Traceband:MonitoringAvailableBandwidthThehiddenMarkovmodelrepresentationoftheavailableban dwidthestimationprocess hasbeenimplementedinanewestimationtoolcalledTraceba nd.Thischapterdescribes andpresentsaperformanceevaluationofthetool.Traceban discomparedwithSpruce andPathloadwhicharethemostrepresentativetoolsofthep robegapandproberate Modelrespectively.Theperformanceevaluationisperform edinanetworktestbedwith Poisson,burstandself-similarsyntheticcrosstrafcand inarealnetworkpathatUniversityofSouthFlorida.5.1TracebandDescriptionTracebandisasender-receiver(client-server)toolwritt eninANSICthatusesthedescribedhiddenMarkovrepresentationoftheavailableband widthdynamicstoprovide fast,continuous,lowoverhead,reliable,andaccurateava ilablebandwidthestimates.The toolhasbeendevelopedforLinux-basedoperatingsystemsa ndcanbedownloadedfor evaluationpurposes[52].Atrainofprobingpacketspairsissentfromthesenderappli cationtothereceiveratthe end-to-endtightlinkrate.Afterinteractingwithcrosstr afcinthetightlink,everypacket pairinthetrainwillprovideasingledispersionvaluethat constitutesoneobservation 90

PAGE 108

inthehiddenMarkovmodelsequence.Thereceiverapplicati onprocesstheobservation sequenceandafterestimatingthesequenceofstatesthatge neratestheobservations,providesasingleaveragedestimationoftheavailablebandwid th.Thedetailedoperationof thesenderandreceiverapplicationisexplainednext.5.1.1TracebandSenderThesenderrunsincyclesoftenestimations.Intherstesti mationthetoolsends50UDP packetpairs1498byteslong.Thenineremainingestimation sareperformedwith30UDP packetpairseach.ThisreductionispossiblesincetheHMMi sabletolearntheavailable bandwidthdynamicswithaninitialsampleandkeepthemodel updatedwithsamplesof reducedsize.Itisfoundfromexperimentationthatre-lear ningeverytenestimationsis enoughtomaintaingoodaccuracywithlowoverhead.Tracebandutilizesdifferentvaluesfortheintra-gapandi nter-gaptimesofpacketpairs. Theintra-gapreferstothetimebetweenthetwopacketsofea chpacketpair.Theintragapor D in issetequaltothetransmissiontimeofasingleprobingpack etinthetightlink. Inthatway,thepacketpairwillbeabletocapturecrosstraf cinthequeue,ifany.The inter-gapreferstothetimebetweenpairsofprobingpacket s,i.e.thetimebetweenthe secondpacketofprobingpair i 1andtherstpacketofprobingpair i .Thesetimesare obtainedusingthegettimeofday()function,anditsvaluesaresenttothereceiverinthe packetpayload.SimilartoSpruce[29],TracebandperformsaPoissonsampli ngprocessoftheavailable bandwidthofthepathbyusingexponentiallydistributedin ter-gaptimes.Inordertokeep theoverheadcontrolledandlow,themeaninter-gaptimeval ueiscalculatedsothatthe 91

PAGE 109

maximumoverheadintroducedbythetoolis5%ofthetightlin kcapacity.Thisallowsthe tooltobelessintrusivewithconsistentaccuracy.5.1.2TracebandReceiverAtthereceiverside,thetooltimestampseachreceivedprob ingpacketatthekernellevel. Thismethodhelpstoreducedelaysgeneratedbythegettimeofday()functionasis mentionedinSection1.2.Packetsarenumberedtodeterminewhichpacketsareinthesa mepairandcalculatethe correctrelativetimedispersion(e)betweenthem.ByapplyingEquation4.10,thecorrespondingobservationsymbolfortheHMMisdeterminedforea chpacketpair. TheHMMmoduleinTracebandreceiverreadsthevaluesofN,Ma ndBfromaleto computethemodellbasedonthe50(orless)observations.Themodelisusedtode terminethemostlikelysequenceofstatesthatgeneratedtheob servations.Foreverynew estimation,theinitialmodell0 istheoutputofthepreviousestimation.Asitisdened inEquation4.15,thesequenceofstatesisthenaveragedand multipliedbythetightlink capacitytoprovideanalavailablebandwidthestimation. ThemainTracebandalgorithm runningatthereceiverisshowninFigure5.1.5.2PerformanceEvaluationTheperformanceofTracebandisevaluatedandcomparedwith PathloadandSpruce, whicharethecurrentmostrepresentativetoolsfortheprob eratemodelandprobegap model,respectively.Thesetoolsalsoshowthebestoverall performanceintheevaluation 92

PAGE 110

nnrnrr r nrrr # $" %" r & (&)$! r &*+' ' ,r %(-n.(% n nr %(/0*#.(% 1 r n 1%(2 r %((% rrrr Figure5.1:Tracebandreceiverpseudocode. performedinChapter3.BothtoolsandTracebandareimpleme ntedandusedwithout anymodicationoftheirdefaultparameters.Thecrosstraf cisarticiallygeneratedina testbedandtakenfromarealpathatUniversityofSouthFlor ida. SimilartotheevaluationperformedinChapter3,theperfor mancemetricsusedtoevaluateTracebandareestimationerrororaccuracy,overhead, andestimationrate(similar toestimationtime).Theestimationerrormetriccomparest heestimationprovidedbythe toolwiththerealaveragevalueobtainedfromthe tcpdump trace,duringthetoolestimationperiod.Theestimationerrorisgivenbytherelativeer roraccordingtoEquation3.5. Theoverheadisrelatedtotheamountofprobingpacketsthat thetoolneedstoinjectinto thenetworkinordertoperformtheestimation.Itistheperc entageoftooltrafcrate(tool trafcdividedbythetoolrunningtime)withrespecttothec apacityofthetightlink.Finally,theestimationrateshowshowoftenthetoolisableto provideanestimate.Thisrate isgiveninestimationsperminute.Thehigherthisvaluethe bettertheestimationtimeof thetool.PathloadandTracebanddirectlyreporttheestima tiontime.Spruceestimation 93

PAGE 111

n n rn r Figure5.2:TestbedtoevaluatetheperformanceofTraceban d. c r 2008IEEE. timeisrecordedusingascripttocalculatethedifferenceo ftimesbeforeandafterrunning thetool.Experimentsareperformedusingsyntheticcrosstrafcgen eratedoveratestbedandwith realnetworktrafctransmittedoverapathatUniversityof SouthFlorida. 5.2.1Synthetic-generatedCrossTrafcTheinitialsetofexperimentsareperformedinthetestbeds howninFigure5.2.Thisisa controlledenvironmentwitha10/100Mbpstightlinkcapaci ty.Crosstrafcisgenerated fromthehostcalled US tothehostcalled China andtheestimationisperformedfrom Sender to Receiver .ThetrafcgeneratorMGENisusedtogeneratePoissonandbu rst crosstrafcexperiments;itallowstosendcrosstrafcatd ifferentratesandwithdifferent probabilitydistributions.Self-similarcrosstrafcisg eneratedusingaCapplicationthat sendspacketsfromatracelegeneratedby[53].Acomputeru sing tcpdump sniffsthe outputlinkintherouterandrecordsatracewiththejoinedc rossandprobingtrafc.This traceisusedtocalculatetheaveragelinkutilizationever y1/10seconds. 94

PAGE 112

Table5.1:Performanceevaluationfor30%Poissoncrosstra fc. Tool EstimationError Estimations/min Overhead Pathload 6 : 71% 1 : 17% 1 : 754 0 : 066 6 : 57% 0 : 20% Spruce 7 : 77% 0 : 98% 5 : 579 0 : 059 1 : 41% 0 : 02% Traceband 8 : 83% 0 : 43% 11 : 645 0 : 132 1 : 96% 0 : 03% Thetoolsareevaluatedasiftheywereperformingacontinuo usnetworkmonitoringtask duringaperiodof200seconds.InthecaseofPathloadandSpr uce,itisnecessaryto runthetoolsinaloop.InthecaseofTraceband,thetoolhasa noptiontosettheestimationperiod.Foreveryexperiment,theoutputofthetoolisr edirectedtoaloglethatis processedtoextractinformationaboutthetime,amount,an dvaluesoftheestimations. Thetightlinkisloadedwith3Mbps(30%ofitscapacity)with Poisson,bursty,andselfsimilar(Hurstparameter=0.8)crosstrafc.5.2.1.1PoissonCrossTrafcExperimentsFigures5.3to5.5showthetools'estimationswhenthetight linkisloadedwithanaverageof3MbpsPoissoncrosstrafc.Themeanvalueforthere alavailablebandwidth iscalculatedastheaverageofallrealavailablebandwidth valuesobservedbetweentwo estimationsofeachtool.Thisisdoneinthatwaysincetheto olsalsoprovideanaverage overtheestimationperiod.Forcomparisonpurposes,Pathl oadsinglepointsarecalculatedasthemidpointoftherangereportedbythetool.IntheexperimentresultsshowninFigure5.3,Pathloadmake s1.86estimationsperminute, inserts6.86%ofthetightlinkcapacityastooloverhead,an dpresentsanaverageestimationerrorof6.92%.Spruce,asseeninFigure5.4,performs5 .49estimationsperminute, inserts1.42%ofthetightlinkcapacityastooloverhead,an dhasanaverageestimation 95

PAGE 113

0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.3:Pathloadestimationfora10Mbpstightlinkwith 30%ofPoissoncrosstrafc. 0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.4:Spruceestimationfora10Mbpstightlinkwith30 %ofPoissoncrosstrafc. errorof8.54%.Finally,TracebandasshowninFigure5.5per formsanaverageof11.42 estimationsperminute,inserts1.90%ofthetightlinkcapa cityastooloverhead,and presentsanaverageestimationerrorof8.40%.Theseresult scorrespondtoonesingle experiment.Table5.1shows95%condenceintervalsforeac hperformancemetricasa resultofrunningtheexperimentsvetimes.InthisPoissoncrosstrafcscenario,thethreetoolsunder evaluationshowestimation errorsbelow10%,whichaccordingtoevaluationsperformed byotherauthorslikein[17] canbeconsideredasofhighaccuracy.ComparedwithSpruce, Tracebandperformstwice 96

PAGE 114

0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.5:Tracebandestimationfora10Mbpstightlinkwit h30%ofPoissoncross trafc. 0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.6:Pathloadestimationfora10Mbpstightlinkwith 30%ofburstycrosstrafc. thenumberofestimationsperminutewithsimilartotalover head.Pathloadhasshownto bemorethanthreetimesmoreintrusiveandmorethansixtime sslowerthanTraceband. 5.2.1.2BurstyCrossTrafcExperimentsFigure5.6to5.8showtheresultsofrunningthetoolswhenth enetworkisloadedwith3 Mbpsofburstycrosstrafc.Theburstsoccuratrandominter valswithanaverageinterval 97

PAGE 115

0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.7:Spruceestimationfora10Mbpstightlinkwith30 %ofburstycrosstrafc. 0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.8:Tracebandestimationfora10Mbpstightlinkwit h30%ofburstycross trafc.fromthestartofoneburstuntilthestartofthenextof10.0s econds.Thedurationofeach burstisofexponentialstatisticswithanaverageburstdur ationof5.0seconds. Inthiscase,asitisshowninTable5.2,Tracebandshowsthem inimumestimationerror withthemaximumnumberofestimationsperminute.AsintheP oissoncrosstrafccase, theamountofoverheadintroducedbyTracebandisconsidera blylowerthanPathload. AsitcanbeobservedfromFigure5.8,sinceTracebandperfor msmoreestimationsper minute,thetoolisabletoaccuratelyreacttoperiodswhere thetightlinkhasnocross 98

PAGE 116

Table5.2:Performanceevaluationfor30%burstcrosstraf c. Tool EstimationError Estimations/min Overhead Pathload 12 : 06% 2 : 45 7 : 65% Spruce 8 : 21% 5 : 46 1 : 34% Traceband 4 : 12% 11 : 26 1 : 98% trafc.Further,duringthoseemptyperiods,theHMMprovid es100%accuracysetting theestimationstothestaterepresentingthehighestavail ability. 5.2.1.3Self-similarCrossTrafcExperimentsFigures5.9to5.11showtheresultsofrunningthetoolswhen thenetworkisloadedwith 3Mbpsofself-similarcrosstrafcwithHurstparameterequ alto0.8.Togeneratethis trafc,aBoundedParetoburstsizeandexponentialinterbu rsttimetraceiscreatedusing syntraf1a.c[53].Thetraceisgeneratedwithatargetutili zationof30%fora10Mbpslink capacity.Itcontainstheinterarrivaltimeandpacketsize ofeachpacket. Toplaybackthetraceintotherealnetwork,anapplicationc alled"udpreply"hasbeen created.Thisapplicationreadsthetraceleandsendsthet rafctothedestination. Sinceitisassumedthattheavailablebandwidthcanbemodel edbyahiddenMarkov model,thememorylesspropertyholdswhenthecrosstrafci sPoissonbutnotself-similar. Therefore,asexpected,Tracebandshowsahigherbutstilll owestimationerror.The95% condenceintervalscalculatedforthethreeevaluatedtoo lsunderself-similarcrosstrafc areshowninTable5.3.Overheadandestimationratesarever ysimilartoresultsobtained inthePoissoncrosstrafcscenario(Table5.1).Theestima tionerrorishoweverhigher butstillinalowrange(around10%)withlowvariability. 99

PAGE 117

0 50 100 150 200 0 2 4 6 8 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.9:Pathloadestimationfora10Mbpscapacityand30 %self-similarcrosstrafc. 0 50 100 150 200 0 2 4 6 8 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.10:Spruceestimationfora10Mbpscapacityand30% self-similarcrosstrafc. 0 50 100 150 200 0 2 4 6 8 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.11:Tracebandestimationfora10Mbpscapacityand 30%self-similarcross trafc. 100

PAGE 118

Table5.3:Performanceevaluationfor30%self-similarcro sstrafc. Tool EstimationError Estimations/min Overhead Pathload 4 : 68% 1 : 29% 1 : 600 0 : 092 6 : 31% 0 : 22% Spruce 7 : 72% 1 : 57% 5 : 482 0 : 060 1 : 36% 0 : 04% Traceband 10 : 48% 1 : 33% 12 : 543 0 : 051 2 : 09% 0 : 02% Table5.4:Performanceevaluationwithrealcrosstrafcin a100Mbpspath. Tool EstimationError Estimations/min Pathload 13 : 89% 4 : 20 Spruce 11 : 07% 5 : 91 Traceband 10 : 95% 109 : 85 5.2.2Internet-trafcBasedExperimentsToperformexperimentswithInternettrafc,itisusedapat hconnectingacomputer fromtheInformationSystemsLab[54]toalocationinCUTR[5 5]throughalayer-3 switchconnectedtoInternet.Thepathhasa100Mbpstightli nk.Sincethisisnotafully controlledenvironment,thetrafctraceshavebeenprovid edbyanetworkadministrator intheuniversityandhaveagranularityof10seconds.TheaveragevaluesplottedinFigures5.12to5.14aresummar izedinTable5.4.Here Tracebandisasexpectedfasterthantheothertools.Having agreaternumberofestimationsperminuteallowsTracebandtohaveabetteraverageac curacywhencomparedwith a10-secondsgranularityrealtrafctrace.5.2.3MovingAverageAlgorithmInthissection,itisdescribedamovingaveragealgorithmm eanttoimprovetheestimationerrorandvariabilityofTraceband.Theideaofthealgo rithmissimilartotheone 101

PAGE 119

0 10 20 30 40 50 60 70 0 2 4 6 8 10 x 10 7 Time (s)Available Bandwidth (bps) Mean Real AvBw Estimated AvBw Figure5.12:Pathloadestimationfora100Mbpstightlinkwi thInternetcrosstrafc. 0 10 20 30 40 50 60 70 0 2 4 6 8 10 x 10 7 Time (s)Available Bandwidth (bps) Mean Real AvBw Estimated AvBw Figure5.13:Spruceestimationfora100Mbpstightlinkwith Internetcrosstrafc. 0 10 20 30 40 50 60 70 0 2 4 6 8 10 x 10 7 Time (s)Available Bandwidth (bps) Mean Real AvBw Estimated AvBw Figure5.14:Tracebandestimationfora100Mbpstightlinkw ithInternetcrosstrafc. 102

PAGE 120

0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.15:MovingaveragepostprocessingtoPathloadexp erimentswithPoissoncross trafc.proposedin[56]tolteroutabruptchangesinthereceiveds ignalstrengthofwireless devices.Itcalculatesthemeanavailablebandwidth A andthestandarddeviation S of vecontinuousestimationstocalculatea95%condenceint ervalusingthet-student distribution: D A = S Q n = 5 ; 0 : 95 p 5 (5.1) Interval = A D A ; A + D A (5.2) where Q n = 5 ; 0 : 95 isthe95%quantileontheStudent'st-distributionforn=5a vailable bandwidthestimations.Ifthenextsingleestimationliesa boveorbelowtheupperor lowerlimitscalculatedusingEquation5.2,thatestimatio nisconsidereda“peak”(avery raresample)anditischangedtotheintervalupperorlowerl imitvalue.Thenanewcondenceintervaliscalculatedwiththelastveestimations (awindowofveestimations iscontinuouslyshiftedonceeverytime).Thesmoothedesti mationisthereforetheresult ofaveragingthelastvemeasurementsafteradjustingthos eoutofthecondentinterval limits.ThealgorithmisshowninFigure5.18.Itisworthnot icingthatthistechniqueis 103

PAGE 121

0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.16:MovingaveragepostprocessingtoSpruceexper imentswithPoissoncross trafc. 0 50 100 150 4 5 6 7 8 9 10 x 10 6 Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.17:MovingaveragepostprocessingtoTracebandex perimentswithPoisson crosstrafc.generalandcouldbeappliedandincorporatedintoanyother availablebandwidthestimationtool.ThisoptionalmovingaveragealgorithmavailableinTraceb andisevaluatedusingPoisson andself-similarcrosstrafc.InthePoissoncase,theresu ltsshowninFigures5.3to5.5 aresmoothedusingthealgorithm.TheresultsareshowninFi gures5.15to5.17.From Figure5.17,itcanbeobservedthatsinceTracebandestimat ionsaremoresymmetricover themeanvaluethanPathload'sandSpruce's,afterapplying themovingaverage,thetool 104

PAGE 122

nnrnrn n nnn !"# !" $$r#%$nn$n & n & nr(n)*+ n n ( n$ Figure5.18:Movingaveragealgorithm. showsthebestaccuracyandthelowestvariability.Itiswor thnoticingthatgiventhesmall estimationrateofPathload,usingthislteringalgorithm thetoolisnotabletoperform therstestimatebefore150seconds.Asbefore,fortheseto fveexperiments,a95% condenceintervaliscalculated.Theoverheadandestimat ionratearethesameasin Table5.1buttheestimationerrorresultsareshowninTable 5.5. Inthecaseofself-similarcrosstrafc,inwhichTraceband 'sperformanceworsenswith theHurstparameter,themovingaveragealgorithmmakesimp ortantimprovementsin Table5.5:Estimationerrorafterapplyingmovingaveraget oexperimentresultswith Poissontrafc. Tool EstimationError Pathload 4 : 88% 2 : 13% Spruce 3 : 84% 1 : 92% Traceband 2 : 93% 1 : 42% 105

PAGE 123

0 20 40 60 80 100 120 140 160 180 200 0 1 2 3 4 5 6 7 8 9 10 x 10 6 Estimation Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.19:MovingaveragepostprocessingtoPathloadexp erimentswithself-similar crosstrafc. 0 20 40 60 80 100 120 140 160 180 200 0 1 2 3 4 5 6 7 8 9 10 x 10 6 Estimation Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.20:MovingaveragepostprocessingtoSpruceexper imentswithself-similar crosstrafc.both,theestimationerroranditsvariability.Figures5.1 9to5.21showtheresultsofapplyingthemovingaveragealgorithmtothesamedatausedtop lotFigures5.9to5.11 withaHurstparameterof H = 0 : 8.Asbefore,forcomparisonpurposes,thetechniqueis alsoappliedtoPathloadandSpruce.FromtheFigure,itcanb eobservedthatTraceband's estimationerrorisreducedconsiderablyusingthismethod ology,asitisitsvariability. Further,thealgorithmimprovesPathload'sandSpruce'spe rformanceaswell.The95% condenceintervalsareshowninTable5.6,whichshowsthat nowTracebandisthetool withthelowestvariability. 106

PAGE 124

0 20 40 60 80 100 120 140 160 180 200 0 1 2 3 4 5 6 7 8 9 10 x 10 6 Estimation Time (s)Available Bandwidth (bps) Real AvBw Mean Real AvBw Estimated AvBw Figure5.21:MovingaveragepostprocessingtoTracebandex perimentswithself-similar crosstrafc.5.3AdditionalExperimentsThissectionpresentstheresultsofexperimentsperformed tostudythebehaviorofTracebandundervariationsinnetworkconditionsandvariations initsownparametersrelated tothehiddenMarkovmodel.Resultspresentedheremotivate afactorialdesignanalysis tobeperformedaspartofthefuturework.Table5.6:Estimationerrorafterapplyingthemovingavera getoexperimentresultswith self-similartrafc. Tool EstimationError Pathload 3 : 90% 1 : 72% Spruce 4 : 67% 1 : 24% Traceband 5 : 12% 0 : 49% 107

PAGE 125

0.5 0.55 0.6 0.65 0.7 0.75 0.8 0 2 4 6 8 10 12 14 Hurst ParameterEstimation Error (%) Traceband Pathload Spruce (a)Withoutmovingaverage. 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0 2 4 6 8 10 12 14 Hurst ParameterEstimation Error (%) Traceband Pathload Spruce (b)Withmovingaverage. Figure5.22:EffectoftheHurstparameterintheestimation error. 5.3.1HurstParameterOneadditionalexperimentisperformedusingthesametestb edshowninFigure5.2to lookattheeffectoftheHurstparameterintheestimationer rorofthetools.Figure5.22 showstheseresultswithandwithoutusingthemovingaverag ealgorithmandhavingthe Hurstparametervariedfrom0 : 5to0 : 8.Foreachpointinthegraph,a95%condence intervalisplotted.FromFigure5.22(a),itisclearthatPa thloadisthemostaccuratetool followedbySpruceandTraceband,inthatorder.Asexpected ,theselfsimilaritylevelaffectstheaccuracyoftheevaluatedtools,andinparticular ,theperformanceofTraceband, whichnotonlyincreasesitsestimationerrorbutalsoitsva riability. Whenthemovingaveragealgorithmisapplied,Figure5.22(b )nowshowsthatallthe toolspresentfairlysimilarresults,butbettercomparedw iththecasewithoutthemoving averagealgorithm. 108

PAGE 126

Table5.7:Tracebandperformanceforhighandlowtightlink capacities. Capacity EstimationError Estimationsperminute 10Mbps 10.48% 12.54 100Mbps 10.95% 109.85 5.3.2TightLinkCapacityAsitisshowninTable5.7,Tracebandestimationerrorhasno signicantvariationwhen thetightlinkcapacitychanges.Ontheotherhand,sincethe toolisabletosendprobing packetsataveryfastrate,thenumberofestimationspermin uteincreaseswiththetight linkcapacity.5.3.3NumberofStatesintheHMMThenumberofstatesisalsovariedfrom5to20withthecorres pondingnumberofobservationsymbols(thatis,from5to20).Asexpected,Figur e5.23(b)showshowthe morestatesandobservations,thereceivingsideoftheappl icationhastoperformmore calculationsandthereforethenumberofestimationspermi nutedecreases.Withregards totheestimationerror,asitisshowninFigure5.23(a),the reisnovariationtrendwhen thenumberofstateschanges.Thisisanaspectthatneedsfurtherresearchbutitisprobab lyduetothedenitionofthe observationprobabilitymatrix B .AppendixBshownthedenedmatricesaccordingto thefollowingpolicy: 25%ofthemostlikelystatesresponsibleforanobservation accumulate70%ofthe probabilities 109

PAGE 127

5 10 15 20 5 10 15 Number of statesEstimation Error (%) (a)Error. 5 10 15 20 85 90 95 100 105 110 115 120 125 Number of statesEstimations per minute (b)Time. Figure5.23:Tracebandestimationerrorandtimefordiffer entnumberofstates. 50%ofthemostlikelystatesresponsibleforanobservation accumulate90%ofthe probabilities 110

PAGE 128

Chapter6:ConclusionsandFutureWorkThisdissertationstudiestheproblemofavailablebandwid thestimationandproposesa noveltechniqueandtooltoaccuratelyandnon-intrusively monitortheavailablebandwidthofandend-to-endpath.Thissectionpresentstheconc lusionsdrawnfromthecontributionspresentedinSection1.4andoutlinesfuturedir ectionsofthiswork. 6.1Conclusions Regardingtheperformanceevaluationofavailablebandwid thestimationtools,the mainconclusionisthatcurrenttoolsarestillfartoprovid ethehighaccuracy,low estimationtime,lowoverhead,andreliabilityrequiredby thenetworkapplications. Table3.5showsthatonlynetworkmanagementandSLAcomplia nceapplications couldbenetfromtoolslikeSpruceandPathload(inthatord er).Othernetwork applicationsstillneedabetterperformanceofthecurrent tools. Ontheotherhand,Table3.4summarizesthetoolsthatperfor mbestandworstin specicnetworkscenarios.Thisinformationisusefultode cidewhichtoolwould bethemostsuitableforanetworkapplicationrunninginapa rticularnetworkscenario.Ingeneral,thestudyclearlyshowswhichtoolsmight bethebestchoicesfor particularapplications,networks,andnetworkcondition s,andwhichaspectsneed 111

PAGE 129

furtherresearchinthisarea,hopingthatthiswilltrigger moreinteresttopushthe state-of-the-artinthiseld. AhiddenMarkovmodeloftheavailablebandwidthisabletoad justobservations affectedbythebustynatureofcrosstrafcandbyerrorsass ociatedtothenetwork infrastructure.Thismodelallowstokeepthenumberofobse rvationslowwith acceptableaccuracyintheestimation.Byusingamovingave ragetechniqueitis possibletosmooththeestimateandimprovetheaccuracyeve nfurther. ThehiddenMarkovmodelimplementedinTracebandprovidesa novelapproach toobtainaccurateestimationswithaconsiderableimprove mentintheestimation timeofthetool.Thisimprovementmakesthetooluniquesinc eitistheonlyone abletoaccuratelymonitortheavailablebandwidthwithagr anularitynevershown before.TracebandcomparedwithPathloadandSpruce,noton lyprovidesbetter performanceresultsoverall,butitisalsoabletoreactand accuratelyestimatethe availablebandwidthunderabruptchangesincrosstrafc.E xperimentalresults usingPoisson,bursty,andself-similarcrosstrafcshowt hatTracebandprovides moreestimationsperunittimewithcomparableaccuracytoP athloadandSpruce andwithlessprobingtrafc.ThetooltestedinaUniversity ofSouthFloridanetworkpathshowssimilarresultasinthetestbedexperiments Thetestbedinfrastructurebuiltforthisworkisabletoacc uratelyemulatedifferent networkscenariosandconditionsasseenintheInternet.Th isinfrastructuresupportsseveralresearchprojectsincludingclassassignmen tsinthegraduatecourseof computernetworks.ThetestbedcanbeaccessedovertheInte rnetwhichmakesit availabletootherresearchers. 112

PAGE 130

6.2FutureWorkTheresearchincludedinthisdissertationcanbeextendedi nseveralways.Someofthem are: Intheanalyticalevaluationofavailablebandwidthestima tiontoolsamoreprecise estimationoftherealavailablebandwidthrequirestostud ytheexactdistributionof thetools'probingtrafc.ThiscouldimplytheuseofaG/M/1 networkofqueuesif thedistributionisdifferentthanPoisson. Allthetrafcinthenetworktestbedisarticiallygenerat ed.AmorerealisticapproachistousetracestakenfromdifferentInternetsource s[57][58][59][60] andtoreplaythemintothenetworktestbed.Thiscanbedoneu singatoolcalled tcpreplay[61]. Tracebandrequiresapreviousestimationofthetightlinkc apacity.Thisestimation canbeincludedinthetoolbysendingback-to-backpacketsa ndusingthosewhose onewaydelaysareminimumtoestimatetheratevariationthe ysufferbecauseof thetightlinkcapacity. Thereareadditionalnetworkscenariostoevaluateandcomp arewithothertoolsthe performanceofTraceband.Oneofthemisover1or10Gbpslink s. RegardingthehiddenMarkovmodelapproachanditsimpacton theestimation tool,moreworkhastobedonetostudytheeffectofvariation sinthedenitionof theobservationprobabilitiesforagivennumberofstatesa ndobservationsymbols. Giventhelargenumberofcombinations,afactorialdesigna nalysissimilartothe oneshowninAppendixAcanbeconductedtoreducethenumbero fexperiments. 113

PAGE 131

Anotheranalysiscanbeperformedwithdifferentvalues(x edorlearned)ofthe observationprobabilities. 114

PAGE 132

ListofReferences [1]InternetWorldStats,“WorldInternetUsageandPopulat ionStatistics,”2008. [Online].Available:http://www.internetworldstats.co m/ [2]CentralIntelligenceAgency,“TheWorldFactbook,”200 8.[Online].Available: https://www.cia.gov/library/publications/the-worldfactbook/index.html [3]C.D.GuerreroandM.A.Labrador,“OntheApplicabilityo fAvailableBandwidth EstimationTechniquesandTools,” SubmittedtoIEEECommunicationsMagazine 2009. [4]V.Paxson,“MeasurementsandAnalysisofEnd-to-EndInt ernetDynamics,”Ph.D. dissertation,UniversityofCaliforniaatBerkeley,1997. [5]Y.Zhang,N.Dufeld,V.Paxson,andS.Shenker,“OntheCo nstancyofInternet PathProperties,”in Proceedingsofthe1stACMSIGCOMMWorkshoponInternet Measurement .SanFrancisco,California,USA:ACM,2001,pp.197–211. [6]R.Prasad,C.Dovrolis,M.Murray,andK.Claffy,“Bandwi dthEstimation:Metrics, MeasurementTechniques,andTools,” IEEENetwork ,vol.17,no.6,pp.27–35,2003. [7]C.L.T.Man,G.Hasegawa,andM.Murata,“Availablebandw idthmeasurement viaTCPconnection,”in Proceedingsofthe2ndIEEE/IFIPWorkshoponEnd-to-End MonitoringTechniquesandServices(E2EMON2004) ,2004. [8]——,“ICIM:AnInlineNetworkMeasurementMechanismforH ighspeed Networks,”in Proceedingsofthe4thIEEE/IFIPWorkshoponEnd-to-End MonitoringTechniquesandServices ,2006,pp.66–73. 115

PAGE 133

[9]H.Zhou,Y.Wang,X.Wang,andX.Huai,“DifcultiesinEst imatingAvailable Bandwidth,”in IEEEInternationalConferenceonCommunications ,vol.2,2006,pp. 704–709. [10]V.Paxson,“End-to-EndInternetPacketDynamics,” IEEE/ACMTransactionson Networking ,vol.7,no.3,pp.277–292,1999. [11]F.MichautandF.Lepage,“Application-orientedNetwo rkMetrology:Metricsand ActiveMeasurementTools,” IEEECommunicationsSurveysandTutorials ,vol.7, no.2,pp.2–24,2005. [12]R.Prasad,M.Jain,andC.Dovrolis,“EffectsofInterru ptCoalescenceonNetwork Measurements,”in Proceedingsofthe5thPassiveandActiveMeasurement(PAM) Workshop ,2004. [13]G.JinandB.L.Tierney,“SystemCapabilityEffectsonA lgorithmsforNetwork BandwidthMeasurement,”in Proceedingsofthe3rdACMSIGCOMMconference onInternetmeasurement .MiamiBeach,FL,USA:ACM,2003,pp.27–38. [14]J.C.Mogul,“ObservingTCPDynamicsinRealNetworks,” in ProceedingsofACM SIGCOM ,1992,pp.305–317. [15]L.Angrisani,S.D'Antonio,M.Vadursi,andG.Ventre,“ PerformanceComparison ofDifferentTechniquesforAvailableBandwidthMeasureme ntinPacketSwitched Network,”in VirtualEnvironments,Human-ComputerInterfacesandMeas urement Systems,2003.VECIMS'03.2003IEEEInternationalSymposi umon ,2003,pp. 212–217. [16]S.J.Lee,P.Sharma,S.Banerjee,S.Basu,andR.Fonseca ,“MeasuringBandwidth BetweenPlanetLabNodes,”in Proceedingsofthe6thPassiveAndActiveNetwork Measurement:Workshop ,Boston,MA,USA,2005. [17]A.Shriram,M.Murray,Y.Hyun,N.Brownlee,A.Broido,M .Fomenkov,and K.Claffy,“ComparisonofPublicEndtoEndBandwidthEstima tionToolsonHigh SpeedLinks,”in Proceedingsofthe6thPassiveandActiveMeasurementsWork shop 2005,pp.306–320. 116

PAGE 134

[18]C.D.GuerreroandM.A.Labrador,“ExperimentalandAna lyticalEvaluationof AvailableBandwidthEstimationTools,”in ProceedingsoftheIEEELocalComputer Networks ,2006,pp.710–717. [19]——,“AHiddenMarkovModelApproachtoAvailableBandwi dthEstimation andMonitoring,”in ProceedingsoftheInternetNetworkManagementWorkshop Orlando,FL,2008. [20]——,“Traceband:AFast,LowOverheadandAccurateToolf orAvailableBandwidth EstimationandMonitoring,” SubmittedtoComputerNetworks ,2009. [21]A.Shipalov,C.D.Guerrero,M.A.Labrador,andM.Alzat e,“OntheImplementation ofaCapacityEstimatorforWirelessAdHocNetworks,”in ProceedingsoftheIEEE SoutheastCon ,Atlanta,2009. [22]S.Keshav,“AControl-TheoreticApproachtoFlowContr ol,” ACMSIGCOMM ComputerCommunicationReview ,vol.25,no.1,pp.188–201,1995. [23]V.JacobsonandM.J.Karels,“CongestionAvoidanceand Control,” SIGCOMM ComputerCommunicationReview ,vol.25,no.1,pp.157–187,1995. [24]R.CarterandM.Crovella,“MeasuringBottleneckLinkS peedinPacket-Switched Networks,”Tech.Rep.,1996,bprobeandcprobe. [25]C.Dovrolis,P.Ramanathan,andD.Moore,“WhatdoPacke tDispersionTechniques Measure?”in TwentiethAnnualJointConferenceoftheIEEEComputerand CommunicationsSocieties.INFOCOM ,vol.2,2001,pp.905–914. [26]G.Jin,G.Yang,B.Crowley,andD.Agarwal,“NetworkCha racterization Service(NCS),”in Proceedingsofthe10thIEEEInternationalSymposiumonHig h PerformanceDistributedComputing ,2001,pp.289–299. [27]N.HuandP.Steenkiste,“EvaluationandCharacterizat ionofAvailableBandwidth ProbingTechniques,” IEEEJournalonSelectedAreasinCommunications ,vol.21, no.6,pp.879–894,2003. 117

PAGE 135

[28]S.Ekelin,M.Nilsson,E.Hartikainen,A.Johnsson,J.E .Mangs,B.Melander, andM.Bjorkman,“Real-TimeMeasurementofEnd-to-EndAvai lableBandwidth usingKalmanFiltering,”in NetworkOperationsandManagementSymposium,2006. NOMS2006.10thIEEE/IFIP ,2006,pp.73–84. [29]J.Strauss,D.Katabi,andF.Kaashoek,“AMeasurementS tudyofAvailable BandwidthEstimationTools,”in Proceedingsofthe3rdACMSIGCOMMconference onInternetMeasurement .MiamiBeach,FL,USA:ACMPress,2003,pp.39–44. [30]J.NavratilandR.L.Cottrell,“ABwE:APracticalAppro achtoAvailableBandwidth Estimation,”in Proceedingsofthe4thPassiveandActiveMeasurementWorks hop PAM2003 ,2003. [31]M.JainandC.Dovrolis,“Pathload:AMeasurementToolf orEnd-to-EndAvailable Bandwidth,”in Proceedingsofthe3rdPassiveandActiveMeasurementsWork shop vol.11,2002,pp.14–25. [32]V.J.Ribeiro,R.H.Riedi,R.G.Baraniuk,J.Navratil,a ndL.Cottrell,“pathChirp: EfcientAvailableBandwidthEstimationforNetworkPaths ,”in Proceedingsofthe 4thPassiveandActiveMeasurementsWorkshop ,vol.2,2003. [33]B.Melander,M.Bjorkman,andP.Gunningberg,“ANewEnd -to-EndProbingand AnalysisMethodforEstimatingBandwidthBottlenecks,”in ProceedingsoftheIEEE GlobalTelecommunicationsConference ,vol.1,SanFrancisco,CA,USA,2000,pp. 415–420. [34]——,“Regression-BasedAvailableBandwidthMeasureme nts,”2002. [35]“TheNetworkSimulator(ns-2),”2000.[Online].Avail able: http://www.isi.edu/nsnam/ns/ [36]M.JainandC.Dovrolis,“End-to-EndAvailableBandwid th:Measurement Methodology,Dynamics,andRelationwithTCPThroughput,” IEEE/ACM TransactionsonNetworking ,vol.11,no.4,pp.537–549,2003. [37]“PlanetLab.”[Online].Available:https://www.plan et-lab.org/ 118

PAGE 136

[38]C.Dovrolis,P.Ramanathan,andD.Moore,“Packet-disp ersionTechniquesanda Capacity-estimationMethodology,” IEEE/ACMTransactionsonNetworking ,vol.12, no.6,pp.963–977,2004. [39]B.AdamsonandS.Gallavan,“MGEN,”1997.[Online].Ava ilable: http://cs.itd.nrl.navy.mil/work/mgen/index.php [40]L.Rizzo,“Dummynet:ASimpleApproachtotheEvaluatio nofNetworkProtocols,” SIGCOMMComputerCommunicationsReview ,vol.27,no.1,pp.31–41,1997. [41]J.R.Jackson,“NetworksofWaitingLines,” OperationsResearch ,vol.5,no.4,pp. 518–521,1957. [42]——,“JobShopLikeQueuingSystems,” ManagementSciences ,vol.10,no.1,pp. 131–142,1963. [43]R.Jain, TheArtofComputerSystemsPerformanceAnalysis .JohnWiley&Sons, 1991. [44]A.Dainotti,A.Pescape,P.S.Rossi,G.Iannello,F.Pal mieri,andG.Ventre,“An HMMApproachtoInternetTrafcModeling,”in IEEEGLOBECOM06 ,2006. [45]C.Wright,F.Monrose,andG.M.Masson,“HMMProlesfor NetworkTrafc Classication,”in Proceedingsofthe2004ACMworkshoponVisualizationandda ta miningforcomputersecurity ,WashingtonDC,USA,2004,pp.9–15. [46]W.TurinandR.VanNobelen,“HiddenMarkovModelingofF latFadingChannels,” IEEEJournalonSelectedAreasinCommunications ,vol.16,no.9,pp.1809–1817, 1998. [47]E.Alpaydin, IntroductionToMachineLearning .MITPress,2004. [48]L.R.Rabiner,“ATutorialonHiddenMarkovModelsandSe lectedApplicationsin SpeechRecognition,” ProceedingsoftheIEEE ,vol.77,no.2,pp.257–286,1989. 119

PAGE 137

[49]L.E.Baum,T.Petrie,G.Soules,andN.Weiss,“AMaximiz ationTechnique OccurringintheStatisticalAnalysisofProbabilisticFun ctionsofMarkovChains,” TheAnnalsofMathematicalStatistics ,vol.41,no.1,pp.164–171,1970. [50]T.Kanungo,“UMDHMM:HiddenMarkovModelToolkit,”199 9.[Online]. Available:http://www.kanungo.com/software/software. html [51]G.D.ForneyJr,“TheViterbiAlgorithm,” ProceedingsoftheIEEE ,vol.61,no.3, pp.268–278,1973. [52]C.D.GuerreroandM.A.Labrador,“Traceband.”[Online ].Available: http://www.csee.usf.edu/guerrerc/traceband/soft.htm [53]K.Christensen,“TrafcGenerator,”2005.[Online].A vailable: http://www.csee.usf.edu/christen/tools/syntraf1a.c [54]“InformationSystemsLab.”[Online].Available:http ://www.csee.usf.edu/islab/ [55]“CenterforUrbanTransportationResearch.”[Online] .Available: http://www.cutr.usf.edu/ [56]P.Wightman,D.Jabba,andM.A.Labrador,“AnRSSI-base dFilterforMobility ControlofMobileWirelessAdHoc-basedUnmannedGroundVeh icles,”in ProceedingsofSPIE ,2008. [57]“MPEG-4andH.263VideoTracesforNetworkPerformance Evaluation.”[Online]. Available:http://www.tkn.tu-berlin.de/research/trac e/trace.html [58]“TheInternetTrafcArchive.”[Online].Available:h ttp://ita.ee.lbl.gov/index.html [59]“MAWIWorkingGroupTrafcArchive.”[Online].Availa ble: http://tracer.csl.sony.co.jp/mawi/ 120

PAGE 138

[60]“InternetMeasurementDataCatalog.”[Online].Avail able: http://imdc.datcat.org/Home [61]A.Turner,“Tcpreplay.”[Online].Available:http:// tcpreplay.synn.net/trac/ 121

PAGE 139

Appendices 122

PAGE 140

AppendixA:FactorialDesignforAvailableBandwidthEvalu ation Therearethreemainapproachesusedinperformanceevaluat ion:experimental(usinga realsystem),analytic(usingmathematics),andsimulatio n(usingcomputertoolsimplementingasystemmodel).Inanyoftheseapproaches,experim entshavetobeperformed toevaluate,compareoranalyzethebehaviorofthesystem.E xperimentalDesignhelpsto obtainthemaximuminformationusingtheminimumnumberofe xperiments. Theexperimentaldesignusedinthisworkismeanttoevaluat etheperformanceofseveral bandwidthestimationtechniquesundervariablepathchara cteristicsandcrosstrafc. Designanexperimentimplytodenethefollowing: Responsevariablesormetrics:theyaretheperformancemet ricsoroutcomesof theexperiment.Fortheavailablebandwidthevaluation,th esearetheresponse variables:estimationerrororaccuracyoftheavailableba ndwidth[Mbps],estimationtimeorresponsetime[s],overheadwhichisdeneas( tooltrafcrate 100)/(Tightlinkcapacity)[%],andreliabilitydenedas( Numberofestimations)/ (Numberoftrials)[%] Experimentalfactors:Theyarethevariablesthataffectth eresponsevariables. Fortheavailablebandwidthevaluation,theyarethefollow ing:capacity[Mbps], propagationdelay[ms],packetlossrate[0..1],crosstraf camount[%ofthelink capacity],andcrosstrafcpacketsize[Bytes] Factorlevels:theyarethedifferentvaluestobestudiedfo rthefactors.Forexample,inthecaseofthecapacityvaluesof10,20,30,...,200M bps. 123

PAGE 141

AppendixA:(continued) Replication:Isthenumberofexperimentrepetitionsforea chsetoflevels.Inthe caseoftheavailablebandwidthevaluation10successfulre petitionsareusedto calculatea95%condenceintervalusingat-test. Forasystemwith k factors,withthe i th factorhaving n i levels,thenif r replicationsare performedforeachlevelandeverycombinationofalllevels ofallfactorsisstudied,the TNR(totalnumberofrepetitions)isgivenbythisFullFacto rialDesign: TNR = r k i = 1 n i (A.1) Thiswillleadtoalargenumberofexperimentsbecauseallth elevelsforeveryfactorare considered.Amoresimpliedmodelisdenotedby2 k factorialdesign.Theideaistouse only2levelvaluesforeachfactoridentiedas-1(forthesm aller)and+1(forthelarger) andstudythepossiblecombinations.Thiswillleadtoanumb erofrepetitionsgivenby: TNR = r 2 k (A.2) TableA.1:2 5 factorialdesignmatrix. Exp. Factors Response Capacity Delay PLR %XTrafc XTpacketsize (Error) 1 5000(-1) 10(-1) 0.01(-1) 25%(-1) 512(-1) y 1 2 100000(+1) 10(-1) 0.01(-1) 25%(-1) 512(-1) y 2 3 5000(-1) 80(+1) 0.01(-1) 25%(-1) 512(-1) y 3 ... ... ... ... ... ... ... 30 100000(+1) 10(-1) 0.01(+1) 75%(+1) 1408(+1) y 30 31 5000(-1) 80(+1) 0.07(+1) 75%(+1) 1408(+1) y 31 32 100000(+1) 80(+1) 0.07(+1) 75%(+1) 1408(+1) y 32 124

PAGE 142

AppendixA:(continued)Todothat,ithastobedenedrsta2 k factorialdesignmatrixasshowinTableA.1where y i isdeterminedbyaveragingthe r replicationsofexperiment i .Usingthedesignmatrix, theMEF(maineffectofvaryingafactor)isdenedastheaver agechangeintheresponse variableduetomovingthefactorfromits-1toits+1level(i ncreasingit)withallother factorsbeingconstant.Itisdenedas: MEF j = [ columnj ] T response : column 2 k 1 (A.3) AlowMEFmeansthatincreasingthefactorlevelhasnoeffect intheresponsevariable. Table3.3showsthemaineffectintheperformancemetricsof veestimationtoolswhen varyingonefactor.Itisalsopossibletodeterminetheaverageinteractioneff ectoftwoormorefactors.Inthe caseoftwofactors,IEFisdenedastheaveragechangeinthe responsevariablewhen thetwofactorsareatthesamelevelandwhentheyareatoppos itelevels.Itisdenedas: IEF ij = [ columni columnj ] T response : column 2 k 1 (A.4) AlowIEFmeansapoorinteractionbetweenthetwofactors.Th eaverageinteractionof two,three,four,andvefactors,whichareshowninTablesA .2toA.5 125

PAGE 143

AppendixA:(continued) TableA.2:Maineffectintheperformancemetricswhenvaryi ngtwofactors. Factor ResponseVariables Error(%) Overhead(%) Time(s) Reliab.(%) Pathload Capacity+Delay 0.54 0.39 14.27 0.00 Capacity+PLR 6.24 1.13 14.00 0.00 Delay+PLR 1.36 0.78 -16.25 0.00 Delay+%XTrafc -1.61 0.45 -2.70 0.00 PLR+%XTrafc 0.30 0.62 -5.85 0.00 %XTrafc+XTpacketsize 4.63 -0.48 7.42 0.00 IGI Capacity+Delay -133.04 0.20 -0.35 0.00 Capacity+PLR 69.82 0.36 0.58 0.00 Delay+PLR -50.41 0.03 0.47 0.00 Delay+%XTrafc 122.91 0.00 -0.69 0.00 PLR+%XTrafc -58.10 -0.05 0.05 0.00 %XTrafc+XTpacketsize 110.88 -0.03 -0.38 0.00 Spruce Capacity+Delay 6.32 0.12 -0.50 5.19 Capacity+PLR -0.88 0.27 -0.43 -7.95 Delay+PLR 2.19 0.06 -0.48 -4.69 Delay+%XTrafc 1.18 -0.02 0.14 0.78 PLR+%XTrafc 3.72 -0.05 -0.05 -5.78 %XTrafc+XTpacketsize 3.67 0.10 -1.18 9.94 Abing Capacity+Delay 324.09 1.86 -0.01 -2.64 Capacity+PLR 94.92 0.76 -0.01 1.57 Delay+PLR 83.72 0.16 0.01 -0.75 Delay+%XTrafc -317.66 0.10 0.01 -1.81 PLR+%XTrafc -76.44 -0.06 0.01 -2.26 %XTrafc+XTpacketsize -307.68 -0.04 0.00 -0.25 Pathchirp Capacity+Delay -11.06 -0.07 1.10 0.00 Capacity+PLR 33.52 0.55 -0.56 0.00 Delay+PLR -0.58 -0.07 0.81 0.00 Delay+%XTrafc 9.38 0.06 -0.52 0.00 PLR+%XTrafc -55.18 0.17 -1.21 0.00 %XTrafc+XTpacketsize -42.69 -0.02 0.77 0.00 126

PAGE 144

AppendixA:(continued) TableA.3:Maineffectintheperformancemetricswhenvaryi ngthreefactors. Factor ResponseVariables Error(%) Overhead(%) Time(s) Reliab.(%) Pathload Capacity+Delay+PLR -1.24 -0.07 -2.62 0.00 Capacity+Delay+%XTrafc -0.27 -0.18 0.66 0.00 Capacity+Delay+XTpacketsize 4.25 -0.31 0.29 0.00 Delay+PLR+XTpacketsize -3.01 -0.12 -3.28 0.00 Delay+%XTrafc+XTpacketsize 2.35 0.30 -5.49 0.00 PLR+%XTrafc+XTpacketsize 0.20 0.28 -4.35 0.00 IGI Capacity+Delay+PLR 46.34 -0.03 -0.23 0.00 Capacity+Delay+%XTrafc -131.97 0.00 -0.58 0.00 Capacity+Delay+XTpacketsize -141.01 0.02 -0.89 0.00 Delay+PLR+XTpacketsize -57.43 0.03 -1.25 0.00 Delay+%XTrafc+XTpacketsize 133.34 -0.05 0.77 0.00 PLR+%XTrafc+XTpacketsize -55.95 0.01 -0.60 0.00 Spruce Capacity+Delay+PLR -5.20 -0.06 -0.17 1.64 Capacity+Delay+%XTrafc 4.78 0.03 -0.95 -4.00 Capacity+Delay+XTpacketsize -0.93 0.00 -0.09 8.59 Delay+PLR+XTpacketsize -4.49 -0.03 0.01 -1.49 Delay+%XTrafc+XTpacketsize -0.48 -0.06 0.69 -5.43 PLR+%XTrafc+XTpacketsize -0.33 0.05 -1.05 13.49 Abing Capacity+Delay+PLR -97.70 -0.16 0.01 -3.44 Capacity+Delay+%XTrafc 323.01 -0.11 0.01 6.26 Capacity+Delay+XTpacketsize -176.39 0.01 0.00 -1.65 Delay+PLR+XTpacketsize -9.60 0.01 0.00 -1.53 Delay+%XTrafc+XTpacketsize 177.93 -0.03 0.00 5.34 PLR+%XTrafc+XTpacketsize -30.55 -0.02 0.00 -7.11 Pathchirp Capacity+Delay+PLR 0.99 0.07 0.65 0.00 Capacity+Delay+%XTrafc -10.57 -0.08 -0.48 0.00 Capacity+Delay+XTpacketsize 7.49 -0.02 0.83 0.00 Delay+PLR+XTpacketsize 7.14 0.07 1.10 0.00 Delay+%XTrafc+XTpacketsize -10.45 -0.04 -1.36 0.00 PLR+%XTrafc+XTpacketsize -1.94 -0.11 0.27 0.00 127

PAGE 145

AppendixA:(continued) TableA.4:Maineffectintheperformancemetricswhenvaryi ngfourfactors. Factor ResponseVariables Error(%) Overhead(%) Time(s) Reliab.(%) Pathload Cap.+Delay+PLR+%XTraf. -1.39 -0.02 6.40 0.00 Cap.+Delay+PLR+XTpsize -6.52 -0.01 5.25 0.00 Cap.+Delay+%XTraf.+XTpsize 0.84 -0.21 3.11 0.00 Cap.+PLR+%XTraf.+XTpsize 10.60 -0.28 11.16 0.00 Delay+PLR+%XTraf.+XTpsize -3.51 -0.13 1.78 0.00 IGI Cap.+Delay+PLR+%XTraf. 48.62 -0.01 -0.49 0.00 Cap.+Delay+PLR+XTpsize 57.29 -0.03 -0.32 0.00 Cap.+Delay+%XTraf.+XTpsize -138.66 0.05 0.00 0.00 Cap.+PLR+%XTraf.+XTpsize 50.22 -0.01 -0.72 0.00 Delay+PLR+%XTraf.+XTpsize -59.39 0.00 0.82 0.00 Spruce Cap.+Delay+PLR+%XTraf. -5.58 0.01 -0.56 3.72 Cap.+Delay+PLR+XTpsize -5.48 0.03 -0.08 14.41 Cap.+Delay+%XTraf.+XTpsize -1.60 0.06 -0.31 3.90 Cap.+PLR+%XTraf.+XTpsize 0.59 -0.05 0.27 8.71 Delay+PLR+%XTraf.+XTpsize -3.85 0.00 0.27 -4.81 Abing Cap.+Delay+PLR+%XTraf. -99.97 -0.09 -0.01 -0.60 Cap.+Delay+PLR+XTpsize -1.43 -0.02 0.00 0.67 Cap.+Delay+%XTraf.+XTpsize -172.15 0.02 0.00 -0.96 Cap.+PLR+%XTraf.+XTpsize 19.39 0.02 0.00 -1.95 Delay+PLR+%XTraf.+XTpsize -5.69 0.06 0.00 6.14 Pathchirp Cap.+Delay+PLR+%XTraf. 2.78 -0.01 -0.03 0.00 Cap.+Delay+PLR+XTpsize -4.93 -0.09 1.31 0.00 Cap.+Delay+%XTraf.+XTpsize 11.30 0.04 -1.75 0.00 Cap.+PLR+%XTraf.+XTpsize -1.52 0.11 -0.23 0.00 Delay+PLR+%XTraf.+XTpsize 6.01 0.00 -1.53 0.00 128

PAGE 146

AppendixA:(continued) TableA.5:Maineffectintheperformancemetricswhenvaryi ngvefactors. Factor ResponseVariables Error(%) Overhead(%) Time(s) Reliab.(%) Pathload All -0.64 0.161 -0.20 0.00 IGI All 56.67 0.002 1.14 0.00 Spruce All -5.67 0.008 -0.26 -6.09 Abing All 3.39 -0.074 0.00 -0.16 Pathchirp All -4.56 0.007 -1.62 0.00 129

PAGE 147

AppendixB:ObservationProbabilityMatricesTablesB.1andB.2showtheobservationprobabilitiesasdes cribedinSection4.2.2and chosenbyTracebandaccordingtothenumberofstatesandobs ervationsymbols. TableB.1:Observationprobabilitymatrixfor5statesand5 observationsymbols. 0.500.250.150.050.05 0.250.500.150.050.05 0.050.200.500.200.05 0.050.050.150.500.25 0.050.050.150.250.50 TableB.2:Observationprobabilitymatrixfor10statesand 10observationsymbols. 0.350.250.150.100.050.030.030.020.010.01 0.200.350.200.100.050.030.030.020.010.01 0.080.200.350.200.070.030.030.020.010.01 0.030.080.200.350.200.070.030.020.010.01 0.020.030.080.200.350.200.070.020.020.01 0.010.020.020.070.200.350.200.080.030.02 0.010.010.020.030.070.200.350.200.080.03 0.010.010.020.030.030.070.200.350.200.08 0.010.010.020.030.030.050.100.200.350.20 0.010.010.020.030.030.050.100.150.250.35 130

PAGE 148

AbouttheAuthorMr.CesarGuerreroisaPh.D.candidateinthedepartmentofC omputerScienceand EngineeringattheUniversityofSouthFlorida.Hereceived hisM.S.degreeinComputer SciencefromITESM(Mexico)andUNAB(Colombia)in2002.Hea lsoreceivedhis M.S.degreeinComputerEngineeringfromUSF(US)in2007.He isaFulbrightscholar whoworkswithUniversidadAutonomadeBucaramanga(Colomb ia).HisresearchinterestsincludeBandwidthEstimationandNetworkMeasurement