USF Libraries
USF Digital Collections

Detection of marine vehicles in images and video of open sea

MISSING IMAGE

Material Information

Title:
Detection of marine vehicles in images and video of open sea
Physical Description:
Book
Language:
English
Creator:
Fefilatyev, Sergiy
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Ship detection
Tracking
Horizon detection
Computer vision
Buoy camera
Dissertations, Academic -- Computer Science -- Masters -- USF   ( lcsh )
Genre:
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: This work presents a new technique for automatic detection of marine vehicles in images and video of open sea. Users of such system include border guards, military, port safety, flow management, and sanctuary protection personnel. The source of images and video is a digital camera or a camcorder which is placed on a buoy or stationary mounted in a harbor facility. The system is intended to work autonomously, taking images of the surrounding ocean surface and analyzing them for the presence of marine vehicles. The goal of the system is to detect an approximate window around the ship. The proposed computer vision-based algorithm combines a horizon detection method with edge detection and postprocessing. Several datasets of still images are used to evaluate the performance of the proposed technique. For video sequences the original algorithm is further enhanced with a tracking algorithm that uses Kalman filter. A separate dataset of 30 video sequences 10 seconds each is used to test its performance. Promising results of the detection of ships are discussed and necessary improvements for achieving better performance are suggested.
Thesis:
Thesis (M.S.C.S.)--University of South Florida, 2008.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Sergiy Fefilatyev.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 63 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 002021425
oclc - 428424441
usfldc doi - E14-SFE0002609
usfldc handle - e14.2609
System ID:
SFS0026926:00001


This item is only available as the following downloads:


Full Text

PAGE 1

DetectionofMarineVehiclesinImagesandVideoofOpenSea by SergiyFelatyev Athesissubmittedinpartialfulllment oftherequirementsforthedegreeof MasterofScienceinComputerScience DepartmentofComputerScienceandEngineering CollegeofEngineering UniversityofSouthFlorida MajorProfessor:DmitryB.Goldgof,Ph.D. LawrenceO.Hall,Ph.D. SudeepSarkar,Ph.D. DateofApproval: June24,2008 Keywords:shipdetection,tracking,horizondetection,co mputervision,buoycamera, Kalmanlter,machinelearning,performanceevaluation c r Copyright2008,SergiyFelatyev

PAGE 2

DEDICATION Thisthesisisdedicatedtomyparents,LarisaandNikolayFe latyev,forsupport duringmyacademicyears.

PAGE 3

ACKNOWLEDGEMENTS Iwouldliketoexpresswordsofsinceregratitudetomymajor professorDr.Dmitry Goldgofforgivingmeopportunitytoworkintheeldofcompu tervisionunderhissupervision.Thankyou,Dr.LawrenceHallandDr.SudeepSarka r,forassistingmeduring theyearsofgraduateschoolandprovidingconstantscienti cinputandfeedback.Iwould alsoliketoextendmygratitudetoMr.LarryLangebrakeandM r.ChadLembkefromthe CenterofOceanTechnologyforgivingtheideafortheprojec t,consultingintheeldof marinescienceandprovidingimageandvideodataforexperi mentalwork.

PAGE 4

TABLEOFCONTENTS LISTOFTABLES iii LISTOFFIGURES iv ABSTRACT v CHAPTER1INTRODUCTION 1 1.1Overview 1 1.2PreviousWork 2 1.2.1MarineVehiclesDetection 2 1.2.2HorizonDetection 5 CHAPTER2BACKGROUND 7 2.1EdgeDetection 7 2.2ConnectedComponentsAlgorithm 8 2.3KalmanFilter 9 2.4TextureinImages 11 CHAPTER3ALGORITHMS 16 3.1Overview 16 3.2DetectionofMarineVehiclesinSingleImages17 3.2.1ImageAcquisition 18 3.2.2EdgeDetection 19 3.2.3HorizonDetection 20 3.2.4PostprocessingSteps 20 3.2.5LabelingComponentsandOutput21 3.3DetectionofMarineVehiclesinVideo22 3.3.1UseoftheKalmanFilter 24 3.3.2TrackingofMarineVehicles26 CHAPTER4DATAANDPERFORMANCEEVALUATION29 4.1Overview 29 4.2MetricsforHorizonDetectionEvaluation304.3MetricsforMarineVehicleDetectionEvaluation324.4Datasets 34 i

PAGE 5

CHAPTER5COMPARISONOFHORIZONDETECTIONALGORITHMS38 5.1Overview 38 5.2HorizonDetection:UnsupervisedApproach405.3HorizonDetection:SupervisedApproach425.4HorizonDetectionPerformanceontheDatasetwithoutSh ips43 5.5HorizonDetectionPerformanceontheDatasetofImagesw ithFloating ObjectsPresent 46 5.6SelectionofHorizonDetectionAlgorithm50 CHAPTER6RESULTSONMARINEVEHICLEDETECTION51 6.1PerformanceofAlgorithmonSingleImages516.2PerformanceofAlgorithmonVideoSequences52 CHAPTER7CONCLUSIONS 58 REFERENCES 60 ii

PAGE 6

LISTOFTABLES Table5.1Accuracyofthehorizondetectionalgorithmsonth eHORIZONDATASET 1accordingtometric(4.1). 45 Table5.2Accuracyofthehorizondetectionalgorithmsonth eHORIZONDATASET 1accordingtometric(4.2). 45 Table5.3Runningtimeofthehorizondetectionalgorithmso nHORIZONDATASET 1inrelativetimeunitsperimage. 45 Table5.4AccuracyofthehorizondetectionalgorithmsonHO RIZONDATASET 2accordingtometric(4.1). 48 Table5.5AccuracyofthehorizondetectionalgorithmsonHO RIZONDATASET 2accordingtometric(4.2). 48 Table5.6Runningtimeofthehorizondetectionalgorithmso nHORIZONDATASET 2inrelativetimeunitsperimage. 49 Table6.1Resultofmarinevesseldetectioninsingleimages .52 Table6.2SFDAmetricsfortwodierentsettings:onindivid ualframesandon videosequencewithtracking. 53 iii

PAGE 7

LISTOFFIGURES Figure1.1Proposedbuoy-basedsea-tracmonitoringsyste m.2 Figure2.1Signal-rowgraphrepresentationofalineardisc rete-timedynamical system. 11 Figure2.2Textureimages. 15 Figure3.1Basicstructureofshipdetectionalgorithm.18Figure3.2Intermediateresultsduringshipdetection.19Figure3.3Horizondetectionexamples. 23 Figure3.4Examplesofmultipledetectionofasingleobject .23 Figure3.5Outlineofmarinevehicletrackingalgorithm.26Figure3.6Exampleoftrackingmarinevehiclesinvideosequ ence.28 Figure4.1Horizonrepresentation. 31 Figure4.2Examplesofimagesfromdatasetsusedfortesting horizondetection algorithms. 35 Figure4.3ExampleimagefromSHIPDATASET1usedfortesting marinevehicles detectionalgorithm. 37 Figure5.1Exampleoffailureofhorizondetection.39Figure5.2AccuracyofhorizondetectionalgorithmsonHORI ZONDATASET1, metric(4.2)isused. 46 Figure5.3StepsofUNSUPERVISED-SLICEalgorithm.47Figure5.4AccuracyofhorizondetectionalgorithmsonHORI ZONDATASET2, metric(4.2)isused. 49 Figure6.1Resultsofmarinevehicledetectioninsingleima ges.55 Figure6.2Examplesofshiplocalizationfragmentation.56Figure6.3Examplesofshipdetectioninvideo. 57 iv

PAGE 8

DETECTIONOFMARINEVEHICLESINIMAGESANDVIDEOOF OPENSEA SergiyFelatyev ABSTRACT Thisworkpresentsanewtechniqueforautomaticdetectiono fmarinevehiclesinimages andvideoofopensea.Usersofsuchsystemincludebordergua rds,military,portsafety, rowmanagement,andsanctuaryprotectionpersonnel.Theso urceofimagesandvideo isadigitalcameraoracamcorderwhichisplacedonabuoyors tationarymountedin aharborfacility.Thesystemisintendedtoworkautonomous ly,takingimagesofthe surroundingoceansurfaceandanalyzingthemforthepresen ceofmarinevehicles.The goalofthesystemistodetectanapproximatewindowaroundt heship.Theproposed computervision-basedalgorithmcombinesahorizondetect ionmethodwithedgedetection andpostprocessing.Severaldatasetsofstillimagesareus edtoevaluatetheperformance oftheproposedtechnique.Forvideosequencestheoriginal algorithmisfurtherenhanced withatrackingalgorithmthatusesKalmanlter.Aseparate datasetof30videosequences 10secondseachisusedtotestitsperformance.Promisingre sultsofthedetectionofships arediscussedandnecessaryimprovementsforachievingbet terperformancearesuggested. v

PAGE 9

CHAPTER1 INTRODUCTION 1.1Overview Shipmonitoringisanessentialprocessforanumberofappli cationsofpracticalimportance.Itisutilizedbyborderguardsandmilitary,isanimp ortantelementinsecuring portsandsanctuaryprotection.Thisworkpresentsanalgor ithmforanautomatedcomputersystemthatdetectsshipsonthehorizon.Suchasystem canbeequippedwitha digitalcamera,locatedonabuoyorstationarymountedinap ortfacilityandwillworkin anautonomousmodetakingimagesofthesurroundingarea.Af terprocessingtheimage informationthesystemwouldonlysendimagesofthefoundob jectstoahumanoperator forfurtherevaluationandaction(seeFigure1.1). Theobjectiveofthisprototypesystemliesineectivedete ctionofthepresenceofships onthehorizon,isolatingshipsintheimageandpreparingan image-resulttobetransmittedtoacontrolcenterforareviewbyahumanevaluator.Oneo ftheconstraintsforsucha systemisthelowcommunicationbandwidthavailablewhiler eportingtheresults.Hence, oneoftherequirementsliesinecientcompressionoftheob tainedvisualresultsbefore theyaresenttothecontrolcenter.Anotherconstraintislo wpowerconsumption.The latterrequirementsuggeststhatalgorithmsutilizedbyth esystemshouldhaveminimum complexitywhileprocessingimagedata.Thisworkdescribe ssoftwaresystemcomponents thatcomprisebasicalgorithmfordetectingshipsinimages oftheopenseatakenbya digitalcamera.Relationshipsbetweenthesecomponentsis discussedandtheperformance ofthealgorithmisshownontwodatasetsofstillimages:(a) 100independentimageswith multipleshipspresent;(b)adatasetconsistingof9000fra mesfrom30videosequences. Inordertoenhanceresultsonvideodataatrackingalgorith misintroducedanditsper1

PAGE 10

Figure1.1.Proposedbuoy-basedsea-tracmonitoringsystem.formanceisshown.Insummarysomeimprovementsformorerel iableandrobustmarine vesseldetectionalgorithmaresuggested.1.2PreviousWork1.2.1MarineVehiclesDetection Mostofthejobinseamonitoringisdoneusingradioradars[1 ].Theyrelyonrerection ofelectromagneticwavesfromachangeinthedielectricord iamagneticconstants.A radartransmitteremitsradiowaves,whicharererectedbyt hetarget.Rerectedradio wavesaredetectedbyaradarreceiver.Targetobjectsarede tectedbasedonthelatency betweentransmittedandreceivedsignals.Arangeofobject s,includingweatherformations, groundandseasurfacesrerectradiowavesand,thus,maybed etected.However,theradio rerectionisparticularlyhighforelectricallyconductiv ematerials,suchasmetalandcarbon ber,thematerialwidelyusedforframesofthemarinevesse lsandaircraft. 2

PAGE 11

Radar'sabilitytodetectmetalobjectsofvarioussizesdep endsonthechoiceofoperatingfrequency.High(microwave)frequencymarineradars areemployedinmostofthe currentvessels,aircraftandgroundstationsforshipsdet ection.Theyhavepropertiesof reliabledetectionofevenrelativelysmall(withradarcro sssectionofabout1 m2objects. However,theoperatingrangeofthoseradarsislimitedbyli neardistancetothetarget objectsandusuallydoesnotexceed10-20milesormoreforai rborneinstalledsystems. Anothercategoryofradars,groundwaveover-the-horizonr adars(GWOTHR),use lowerfrequenciesofradiowaves,andthus,canpropagatesi gnalalongthecurvedsurface. Theyalsousethefactofrefractionofradiowavesinionosph ere.ThisprovidesGWOTHR radarswithsurveillancecapabilitiesoververylargearea s.Forexample[2]reportsresults fortechnicalcapabilitiesofNorthernRadar'sCapeRaceGr oundWaveRadarsystem whichhadthepotentialtoprovidesurveillanceofover160, 000squarekilometers.Its rangeoftargetsincludedships,icebergs,aswellasenviro nmentalparameterssuchas surfacecurrentsandseastates.Thiscategoryofradarsisa bletodetectlow-ryingaircraft onbigdistancesbutdetectionofmarinevehiclesonbigdist ancesislimitedtoduethe smallradialDopplervelocityofthevessels:thetargets'D opplerusuallysharesthesame spectralregionwiththecontinuumofhighorderseaclutter [3]. Anotherapproachtoshipdetectionisdescribedin[4].This methodusesmagneticelds aroundmagnetictargetstodetectrelativelylargeshipson thesurfaceorevensubmarines inasubmergedstate.Amagnetometeronboardanaircraftmea suresanomaliesinthe magneticeldintheareasurroundingtheaircraft.Althoug hconceptuallyproventodetect articialmetalobjects,thisexoticmethod,however,ismo resuitableforgeology,fornding depositsofironoreorothermetalresources.Detectionofs eatargetislimitedbythe distancefrommagnetometertothetarget.Thenatureofmagn eticeldrequiresthe aircraftcarryingthesystemtobeintheclosevicinitytoth etarget. Thenextcategoryofradars,SAR(SyntheticApertureRadar) hasmanyapplications inremotesensingandmapping.Theapproachusessophistica tedpostprocessingofradar datatoproduceaverynarroweectivemicrowavebeam[5].SA Rcanonlybeused onmovingplatformstoconductsensingoverrelativelyimmo biletargets.Thetwomost 3

PAGE 12

usedplatformsforSARaresatellites(forspaceborneSAR)o raircraft(forairborneSAR). MarinevehiclesandcoastlinedetectionbySARhasbecomeat opicofconsiderableinterest sincetheupsurgeofdemandforthiskindofinformationinth ecommercialmarket.SAR capabilitiesincludeallweatherandalldaydetection,hig hresolutionandwidecoverage (around100x100km).Theresultofsensingusingtheradaris animageoftheocean surface.Currentsystemsusedforshipdetectionvaryinthe computervisionapproaches todetecttheobjectsofinterest.Manytechniques[6],[7]s earchforfeaturesofthewake ofashipinaSARimage.In[8]theRadontransformofSARimage sofshipwakesand oceanwaveswascomputedandanenhancementoperatorwithin thetransformspacewas applied. InmoreadvancedworkusingSARimagesinstancesofshipsare assignedaclassor category(i.e.destroyer,aircraftcarrier).In[9]radari mageswereusedtoclassifyships basedonglobalfeatures.Morphologicalanalysisoftheobj ectboundaryfromSARimages wasutilizedin[10].Classicationoftheshipsandtheirmo tionparametersisdescribed in[11]and[12]. Otherimagingsensorswereusedforshipdetectionaswell.F orwardLookingInfraRed (FLIR)camerasisoftenagoodsensorofchoicebecausetheim agestheyprovideareinsensitivetolightingconditions.Thisimagingsensorismored esirableinmilitaryapplications becauseitdoesnotrevealthelocationoftheimagingsystem .Manyapproachescanbe foundtheintheliteratureforshipclassicationinFLIRim ages.In[13]theyusedthe principalcomponentglobalfeaturesandsimilaritymatchi ngtoclassifyshipsilhouettes. Theauthorsof[14]usedneuralnetworksapplieddirectlyto imagepixels.Recentworkon shipclassicationin[15]usesa k NearestNeighborclassieronshapefeaturesextracted withMPEG-7region-basedshapedescriptor. AnotabledisadvantageofFLIRsystemsisthelowresolution oftheimagesobtained whichrequiresarelativelysmallviewingdistanceoftheim agingsystemtothetarget. Itisalsomorepowerconsumingand,thus,islesscapablefor autonomousoperation. Theapproachdescribedin[16]usessolelycomputervisiona lgorithmsandimagedata fromregulardigitalcameras(inthevisualspectrum)todet ectmarinevehicles.Itis 4

PAGE 13

aimedforautonomousoperation,underpowerandcommunicat ionbandwidthconstraints. Theapplicationsforitsuseincludelocalseatracmonitor inginsanctuariesandport zones.Imagesofopenseatakenfromaforwardlookingcamera installedonabuoy.After detectinghorizonlinethesystemlooksforobjectslyingon it.Thesystemisintended tomonitoronlylocaltrac,withinvisibledistance.Howev er,theabsenceofradarsand relianceonsimplecomputervisionalgorithmsmakesthepot entialsystemcheapandeasy indeploymentandmaintenance.Thisdocumentenhancesprel iminaryresultsdescribed in[16].Itaddsadditionalrobustnesstohorizondetection forsingleimages.Italsouses trackingtechniquestoimprovedetectioninvideosequence s. 1.2.2HorizonDetection Anothercategoryofrelatedworkthatshouldbementionedin cludeshorizondetection inimages.Accuratehorizondetectiongetsconsiderableat tentioninthisthesis.According totheliteraturesurveyperformedforthisworkmostofhori zondetectionapproaches weredesignedfornavigationofmicroairvehicles(MAV).Wh ilelargeraircraftmayhave high-precisiongyrostosenseangularratesoracceleratio n,smallerMAVhaveverystrict constraintsforpayloadcapacity,dimensions,andelectri calpower.MAVautonomousright controlsysteminmanycasesreliesoncheapandlighton-boa rdimagingsensorssuchas cameras,which,however,providerichinformationcontent .Inadditiontosurveillance tasksthathavebeenconsideredastheprimarymissionofMAV ,suchimagingsensors allowestimationofnavigationparameters.Pitchangleand rollangleofaMAVcanbe extractedbasedonthehorizonlineinthevideoimages.Seve ralvisionsystemshavebeen reportedtoprovidevision-basedhorizon-tracking. Oneresearchgroup,veryactiveinvision-basednavigation ,haspublishedanumber ofpapersrelatedtohorizondetection[17,18,19,20].Thei rbasicapproach,astatistical horizondetectionalgorithm,attemptstominimizetheintr a-classvarianceoftheground andskydistribution.Thehorizonisconsideredalinewithh ighlikelihoodofseparating theskyfromground(ornon-sky)regions. 5

PAGE 14

Otherapproachesdevelopedbydierentauthors[21,22]rel yonmoresimpleandless computationallyintensivealgorithms.Thehorizonanglei sfoundasafunctionofthe averagecoordinatesforskyandnon-skyclasses.Theangleo fthehorizonisdetermined bytakingtheperpendicularofthelinejoiningskyandnon-s kycentroids.Skyandnonskyregionsaredistinguishedbyasimplethresholdingcrit erionbasedonthebrightnessof pixelsinagrayscaleimage. Aconsiderablydierenttechniqueisdescribedin[23].Itu sesanideasimilartoskew detectionindocumentimageanalysis,i.e.detectingthesk ewofthescanneddocuments. Theoriginalimageispreprocessedandedgesareextracted. Projectionprolesofedgesin theimagefordierentanglesareobtainedandthehorizonis foundintheprolecorrespondingtothelargestpeakofsuchaprojection. Theapproachdescribedin[24]wasdesignednotonlyforhori zondetectionbutalsoto avoidobstaclesduringMAVrights.Theauthorsclassifyreg ionsintheimageintoskyand ground/obstaclesandobtainboundariesofthesafeareaahe adoftheaircraftaswellas angularparametersoftheaircraftdeterminedfromthehori zonline.Anotherclassication approach,notrelateddirectlytheMAVnavigation,isdescr ibedin[25].Thehorizonis modeledasalineseparatingskyandnon-skypixelandisfoun dafterimagecontentis classiedusingnumerousfeaturesintotwoclasses-skyand non-sky. SeveralotherworksrelatedtoMAVnavigationreportmorein tegrationwithhardware, oftenallowingon-boardprocessingofthedata.Oneresearc hgroupdevelopedhorizon detectionequipmentthatislightenoughtobeairborne[26] .Theyuseathermalimaging cameraandscannedlineararray.Similarapproachesthatus etheinfra-redspectrumare usedinaerospaceapplicationforstabilizingaircraftand aredescribedin[27]. Chapter5willcontinuethediscussionabouthorizondetect ionoptimalneededforthis work.Thementionedhorizondetectionalgorithmswillbere viewedinmoredetailand severalwillbeevaluatedforperformance. 6

PAGE 15

CHAPTER2 BACKGROUND 2.1EdgeDetection Edgedetectionisaresearcheldwithinimageprocessingan dfeatureextractionand therearemanydierentapproachestoit.Thegoalofedgedet ectionistomarkthepoints inanimageatwhichtheintensitychangessharply.Sharpcha ngesinpixelsintensityinan imageusuallyrerectimportanteventsandchangesinworldc orrespondingtotheimage. ExamplesofedgedetectorsincludeSobel,LaplaceandCanny andotheredgedetectors [28],[29],[30].ThisworkusesCannyedgedetector[31]. ThealgorithmusedinCanny'smethodfollowsalistofcriter iatoimproveedgedetection incomparisontoothermethods.Itsadvantagesincludelowe rrorrate,well-localizededge pointswithminimumdistancebetweentheactualedge,andsi ngleedgeresponse.The algorithmconsistsofthefollowingsixsteps: Noiseintheoriginalimageislteredoutbeforelocatingan ddetectinganyedges. BecausetheGaussianlter[28],[32]canbecomputedusinga simplemask,itis usedexclusivelyintheCannyalgorithm.Onceasuitablemas khasbeencalculated, Gaussiansmoothingisperformedusingstandardconvolutio nmethods. Theedgestrengthisfoundbytakingthegradientoftheimage .2-Dspatialgradient measurementisperformedbyconvolvingtwo(vertical,andh orizontal)3x3kernels aroundeachpixelintheimage.Then,theapproximateabsolu tegradientmagnitude (edgestrength)ateachpointcanbefound.TheSobeloperato rusesapairof3x3 convolutionmasks,oneestimatingthegradientintheXdire ction(columns)andthe otherestimatingthegradientintheYdirection(rows).The magnitude,oredge 7

PAGE 16

strength,ofthegradientisthenapproximatedusingthefor mula: j G j = j Gxj + j Gyj (2.1) EdgedirectionisfoundfromthegradientmagnitudeintheXa ndYdirectionusing thefollowingformula: =arctan( Gx Gy )(2.2) Thenon-maximumsuppression,usedtotracealongtheedgein theedgedirection,is appliedaftertheedgedirectionsareknown.Thisresultsin athinlineintheoutput image. Hysteresisisusedasameansofeliminatingstreaking.Stre akingisthebreakingup ofanedgecontourcausedbytheoperatoroutputructuatinga boveandbelowthe threshold.Ifasinglethreshold,T1isappliedtoanimage,a ndanedgehasanaverage strengthequaltoT1,thenduetonoise,therewillbeinstanc eswheretheedgedips belowthethreshold.Equallyitwillalsoextendabovetheth resholdmakinganedge looklikeadashedline.Toavoidthis,hysteresisusestwoth resholds,high(T1)and low(T2).AnypixelintheimagethathasavaluegreaterthanT 1ispresumedto beanedgepixel,andismarkedassuchimmediately.Then,any pixelsthatare connectedtothisedgepixelandthathaveavaluegreatertha nT2arealsoselected asedgepixels. 2.2ConnectedComponentsAlgorithm Extractingandlabelingofvariousdisjointandconnectedc omponentsinanimageis animportantpostprocessingstepofshipdetection.Thecon nectedcomponentslabeling algorithm[33]isappliedonabinaryimagetogroupitspixel sintocomponentsbasedon pixelconnectivity,i.e.,allpixelsinaconnectedcompone ntsharesimilarpixelintensity values(zeroorone)andareinsomewayconnectedwitheachot her.Onceallgroups 8

PAGE 17

havebeendetermined,eachpixelislabeledwithanumberacc ordingtothecomponentit wasassignedto.Thealgorithmworksbyscanninganimage,pi xel-by-pixel(fromtopto bottomandlefttoright)inordertoidentifyregionsofadja centpixelswhichsharethe samesetofintensityvalues V Connectedcomponentlabelingworksonbinaryimagesandtwo dierentmeasuresof connectivityarepossible:4-connectivityand8-connecti vity,dependingonthenumber ofneighborpixelpossibleforconnection.Forthecurrentp roject8-connectivityisused, however,forthepurposeofsimplicityonly4-connectivity isdescribedhere.Thelabeling operatorscanstheimagebymovingalongarowuntilitcomest oapoint p ,where p denotes thepixeltobelabeledatanystageinthescanningprocessfo rwhich V =1.Whenthis istrue,itexaminesthefourneighborsof p whichhavealreadybeenencounteredinthe scan(i.e.theneighbors(I)totheleftof p ,(II)aboveit,and(IIIandIV)thetwoupper diagonalterms).Basedonthisinformation,thelabelingof p occursasfollows: ifallfourneighborsare0,assignanewlabelto p ,else ifonlyoneneighborhas V =1,assignitslabelto p ,else ifmorethanoneoftheneighborshave V =1,assignoneofthelabelsto p andmake anoteoftheequivalences. Theequivalentlabelpairsaftercompletingthescanaresor tedintoequivalenceclasses andauniquelabelisassignedtoeachclass.Asanalstep,as econdscanismadethrough theimage,duringwhicheachlabelisreplacedbythelabelas signedtoitsequivalence classes.Fordisplay,thelabelsmightbedierentgrayleve lsorcolors. 2.3KalmanFilter TheKalmanlter[34],[35]providesarecursivesolutionto thelinearoptimalltering problem.Itsdomainofitsapplicationsincludesstationar yandnon-stationaryenvironments.Thesolutionisrecursive;eachupdatedestimateoft hestateiscomputedfromthe previousestimateandthenewinputdata.TheKalmanlterne edstostoreonlythepre9

PAGE 18

viousestimate,eliminatingtheneedforstoringtheentire pastobserveddata,and,thusis computationallymoreecientthancomputingtheestimatef romtheentirepastobserved dataateachstepofthelteringprocess. TheblockdiagramshowninFigure2.1describesalinear,dis crete-timedynamicsystem. Thestatevector,denotedby xk,isdenedastheminimalsetofdatathatuniquely describestheunforceddynamicalbehaviorofthesystem;th esubscript k denotesdiscrete time.Thestatevectoristheleastamountofdataonthepastb ehaviorofthesystem thatissucienttopredictitsfuturebehavior.Ongeneral, thestate xkisunknown.To estimateit,asetofobserveddata,denotedbythevector yk,isused.Processnoise wkand measurementnoise vk,showninthediagram,arerepresentedbycovariancematric es Q and R ,andareconsideredconstant.Thefollowingsetofequation sdescribesthedynamic systemmathematically: Processequation xk +1= Fk +1 ;kxk+ wk(2.3) where Fk +1 ;kisthetransitionmatrix,whichrelatesthestate xkfromtime k totime k +1intheabsenceofnoise.Theprocessnoise wkisassumedtobeadditive,white, withnormalprobability.Itsprobabilitydistributionhas zeromeanwithacovariance matrixdenedby E [ wnwk T]= 8>><>>: Qkfor n = k 0for n 6 = k (2.4) where T denotesmatrixtransposition.Thedimensionofthestatesp aceisdenoted by m Measurementequation yk= Hkxk+ vk(2.5) 10

PAGE 19

Figure2.1.Signal-rowgraphrepresentationofalineardiscrete-time dynamicalsystem.where ykistheobservableattime k and Hkisthemeasurementmatrix.Themeasurementnoise vkisassumedtobeadditive,white,andnormalprobability.It s probabilityhaszeromeanandwithcovariancematrixdened by E [ vnvk T]= 8>><>>: Rkfor n = k 0for n 6 = k (2.6) Themeasurementnoise vkisuncorrelatedwiththeprocessnoise wk.Thedimension ofthemeasurementspaceisdenotedby n TheKalmanlteringproblem,namely,theproblemofjointly solvingtheprocessand measurementequationsfortheunknownstateinanoptimumma nnermaynowbeformally statedasfollows:usetheentireobserveddata,consisting ofthevectors y1, y2,..., yk, tondforeach k 1theminimummean-squareerrorestimateofthestate xk.The problemiscalledlteringif i = k ,predictionif i>k andsmoothingif1 i k .Detailed techniquesforsolvingKalmanlterproblemisshownin[34] 2.4TextureinImages Theconceptoftextureisoftenusedtodescriberegionprope rtiesinanimage.Texture providesmeasuresforpropertiesforaregioninanimagesuc hassmoothness,coarseness, andregularityandisscaledependant.Textureconsistsoft extureprimitives-texture 11

PAGE 20

elements.Thethreemostusedapproachestodescribethetex tureofaregioninanimage arestatistical,structuralandspectral.Statisticalapp roachesshowcharacterizationsof texturesassmooth,coarse,grainy,andsoon.Structuralte chniquesdealwiththearrangementoftextureelements,suchasthedescriptionoftexture basedonregularityspaced parallellines.Spectraltechniquesarebasedonpropertie softheFourierspectrumandare usedprimarilytodetectglobalperiodicityinanimagebyid entifyinghighenergyorpeaks inthespectrum. Becauseoftheirsimplicityindescribingtexturethisproj ectusesstatisticalapproaches. Thoseincludestatisticalmomentsofthegraylevelhistogr am,uniformityandentropy calculatedforasmallregioninanimage.Let z bearandomvariabledenotinggraylevels andlet p ( zi), i =0 ; 1 ; 2 ;:::;L 1 ; bethecorrespondinghistogram,where L isthenumber ofdistinctgraylevels.The n thmomentof z ,aboutthemeanis n( z )=L 1Xi =0( zi m )np ( zi)(2.7) where m isthemeanvalueof z (theaveragegraylevel): m =L 1Xi =0zip ( zi)(2.8) Thesecondmoment 2( z )isameasureofgraylevelcontrastthatcanbeusedtoestabl ish descriptorsofrelativesmoothness.Forexample,themeasu re R =1 1 1+ 2( z ) (2.9) is0forareasofconstantintensity(thevarianceiszerothe re)andapproaches1forlarge valuesof 2( z ).Becausevariancevaluestendtobelargeforgrayscaleima geswithvalues, itisnormalizedtotheinterval[0,1]forusein(2.9).Thisi sdonebysimplydividing 2( z ) by( L 1)2in(2.9).Thestandarddeviation, ( z ),alsoisusedfrequentlyasameasureof texturebecausevaluesofthestandarddeviationtendtobem oreintuitivetomanypeople. 12

PAGE 21

Thethirdmoment, 3( z )=L 1Xi =0( zi m )3p ( zi)(2.10) isrelatedtotheskewnessofthehistogramwhilethefourthm omentisameasureofits relativeratness.Thefthandhighermomentsarenotsoeasi lyrelatedtohistogramshape, buttheydoprovidefurtherquantitativediscriminationof texturecontent. Someusefulmeasuresbasedonhistogramsincludeameasureo funiformitygivenby U =L 1Xi =0P2( zi) ; (2.11) andanaverageentropymeasure,whichisdenedas e = L 1Xi =0p ( zi)log2p ( zi)(2.12) Becausetheprobabilitieshavevaluesintherange[0,1]and theirsumequals1,the measureUismaximumforanimageinwhichallgraylevelsaree qual(maximallyuniform), anddecreasesfromthere.Entropyisameasureofvariabilit yandis0foraconstantimage. Ingeneralcoarsetexturesarebuiltfromlargerprimitives (textureelements),netextureshavesmallerprimitives.Coarsetexturesarecharact erizedbylowerspatialfrequencies,netexturebyhigherspatialfrequencies. Figure2.2illustratesdierenttexturemeasures.Foreach ofthetexturemeasuresa textureimageoftheresolution256x192hasbeenreceived.T hemethodofgettingthese textureimagesisthefollowing: Obtainapatchofanoriginalimagewiththesizeof11x11arou ndeachpixelinthe grayscaleimageobtainedfromtheoriginalcolorimage. Calculateatexturemeasureforthispatchaccordingto(2.7 )-(2.12). Assigntheresultofthetexturemeasureasavalueofthepixe linthetextureimage. 13

PAGE 22

Normalizeallvaluesinthetextureimageequallyintherang ebetween0and255. Thus,foragrayscaleimageobtainedfromanoriginalimaget herearesixtextureimages. 14

PAGE 23

Figure2.2.Textureimages.(a)originalcolorimage;(b)correspondin ggrayscaleimageofthe redchannel;(c)textureimageusingmeanmeasure;(d)textu reimageusingstandarddeviation measure;(e)textureimageusingsmoothnessmeasure;(f)te xtureimageusingthirdmoment measure;(g)textureimageusinguniformitymeasure;(h)te xtureimageusingentropymeasure.15

PAGE 24

CHAPTER3 ALGORITHMS 3.1Overview Theliteraturesectionofthisdocumentdescribedvariousm ethodsfordetectionmarinevehicles.Thesemethodsdierintechnologicalapproa chesandpotentialcapabilities. However,mostofthemaresimilarinoneimportantaspect:mo stofthesemethodsare characterizedbythehighcostoftheequipmentandmaintena nce.Thiscreatesavery narrowcategoryofusersforsuchsystems.Inaddition,rapi ddeploymentofsuchsystems formonitoringpurposesoflocalareasisdicult,itmayreq uirearadarinstallationor accesstosatellitedata. Thisworkfocusesondetectionofmarinevehiclesusingimag ingsensorsuchasdigital camerasorcamcorders.Thescopeofsurfaceareaformonitor ingislimitedbyvisual distancefromsomepointintheoceanorshore.Thismaybebea ppropriateforsometasks suchasportsecurityorsanctuaryprotection.Inessence,t helimitationandeectivenessof theapproacharesimilartoperiscopeofasubmarine:marine vehiclesarevisuallylocated inimagedataobtainedfromacamerastickingoutofthewater surface.Anoceanbuoy isconsideredaprimaryplatformforsuchasystem.Anexampl eofappropriateplatform canbeshownonBSOP-theBottomStationingOceanProler[36 ].Itisanautonomous bouy-platformdesignedtocarryasensorpayload,collecto ceanicdataandstoreor/and transmitthedatathroughabi-directionalRFsatellitelin ktoacontrolcenter.Ifequipped withaproposedvisionunitsuchasystemcouldperformvisua lsurveillanceforpassingsea trac. Thefollowingsectionsenhancethepreviousworkonalgorit hmicsolution[16]forsuch asystem.Inadditiontodetectionofmarinevehiclesinsing leimages(Section3.2)other 16

PAGE 25

settingsareexplored.Section3.3addsrobustnesstothede tectionapproachforvideodata. Suchimportantaspectsasdetection,ltering,andtrackin garealsodescribed. 3.2DetectionofMarineVehiclesinSingleImages Imagesegmentationandobjectdetectionisabroadtopicofr esearchincomputervision. Usually,aparticularapplicationrequirescertainassump tionsabouttheenvironmentand targetsfordetection.Approachessuchasbackgroundsubtr actionorappearancesimilarity arenotappropriatefordetectionofmarinevehiclesinimag eswithhighlydynamicocean surface.Ourassumptionfortheproblemofmarinevehiclesd etectionisarelativeposition betweenthesoughtmarineobjectsandtheoceanhorizonline .Thatassumptionisvalid forthecasewhenimagesorvideotakenfromthecamerawithim ageaxisparalleltothe oceansurfaceandwhenthetargetmarinevehiclesarelocate dwithinvisualdistance.Such assumptionsgreatlyfacilitatelocalizationofmarineveh iclesorotherroatingobjectsin imagedata.Thisisbecausetheoceansurfacepartoftheimag eisrichininformation contentandisdiculttoprocessintermsofimageanalysis. Ontheotherhandthesky background,infrontofwhichthesoughtmarinevehiclesare expectedtobelocated,is relativelyhomogeneousincolorandhaslittletexture,and thus,iseasytoprocess. Figure3.1showsthebasicstructureandorderofstepsinthe algorithmtodetect marinevehiclesinsingleimages.Thealgorithmconsistsof sixsteps.Duringtheimage acquisitionstepimagedataisobtained.Somepreprocessin gsuchasbasicnoiseremoval isalsodoneduringthatstep.Preprocessedimageisfedinto twocomponents-edge detectorandhorizondetector,toobtain,correspondingly ,edgesfromtheoriginalimage andtheparametersofthehorizonline.Thepostprocessings tepofthealgorithmcombines resultsfromedgeandhorizondetectorsandprocessesonlye dgesthatarelocatedabove thehorizonline.Theconnectedcomponentsblockofthealgo rithmprovidesthelocation oftheboundingboxesaroundthefoundmarinevehicles.Theo utputresultconsistsofa regionfromtheoriginalimageinsidetheboundingboxaroun dfoundobjects.Figure3.2 showsasequenceofstepsinshipsdetectiononasampleimage 17

PAGE 26

Figure3.1.Basicstructureofshipdetectionalgorithm.DetailedimplementationofthealgorithmisdescribedinSu bsections3.2.1{3.2.4. 3.2.1ImageAcquisition Theimageacquisitionstepusesadigitalcamerainstalledo nabuoytoacquireimages ofthesurroundingarea.Theimagedatafromthecameraispro videdinRGBformat.The focusofthecameraissettoinnityandthus,capturesonlyf ar-lyingobjectswhichare expectedtobeabovethehorizonline.Thedrivingfactorsof suchanassumptionare thefollowing:atlongdistancesthelinebetweencameraand objectofinterest(aship) becomesparalleltothesealevelandthereforeallobjectso finteresthavetoexistabove thehorizon.Theheightabovethewateronwhichthecamerais mountedisamatterof consideration.Theeectiverangeofdetectionisbiggerwh encameraismountedhigher. However,acloselylocatedshipmaynotbeexactlyabovetheh orizonlinedetectedfrom suchacamera.Forsomesettingstheheightofcamerainstall ationisalsoamatterofwhat ispractical.Forexample,buoysystemsmaynothaveamastta llenoughtoinstallthe cameraontherequiredheight.Incontextofthisworkthedep endencybetweentheheight ofinstallationanddetectionrangeofthesystemisnotexpl ored.Ideasabouttheoptimal heightofthecamera-mastforapossibleshipsurveillances ystemmaybetakenfromthe designofsubmarineperiscopesystems[37]. 18

PAGE 27

Figure3.2.Intermediateresultsduringshipdetection.(a)Originali mage.(b)Detectedhorizon line.(c)Edge-image.(d)Edgestakenforconsideration(ab ovehorizonline).(e)Resultof postprocessingstep.(f)Boundingboxoverdetectedship.3.2.2EdgeDetection Theoutputofthisstepisabinarymapofedgesfoundintheori ginalimage.Edges intheimagesrerecttheregionsintheimageswherepixelint ensitychangesquickly.For apossiblemarinevehiclewithdistinctappearanceitmeans sharpchangesinintensityat leastaroundthecontourofthetargetinfrontofskybackgro und.Chapter2describedthe Cannyedgedetectorthatwaschosenforthiswork.Goodedges shouldincludethecontour ofamarinevehicles:anedgemapthatcreatesaconvexhullar oundtheobjectandexclude edgesnotbelongingtotheobject.Edgesalsoneedtobeconsi stent,possiblyhavingas fewbrokenlinesbelongingtothesameobjectaspossible.Th eCannyedgedetectorhas threeparameters: sigma (0.5-5.0), low (0.0-1.0)and high (0.0-1.0).Automaticsearchfor optimalparameterswasnotconsideredinthecontextofthis workbecauseofthediculties increatingprecisepixelwisegroundtruth.Manualsearchf orparametersoftheCannyedge detectorwasconductedonaseparatedatasetof27images.Th eparameterswereadjusted tocaptureonlywell-denedlongedgesthatarecommonforve sselswithextensivecontour 19

PAGE 28

lineslikeyachtsorbarges.Figure3.2(c)showsanexampleo fedgemapdetectedfor originalimageinFigure3.2(a).3.2.3HorizonDetection Horizondetectionstepisanessentialstepinthealgorithm .Thefoundhorizonlineis usedtoeliminatealledgesintheedge-mapthatwouldnotbel ongtoroatingobjects-we expectthatinanimageallobjectsofinterest.Failuretoid entifythehorizoncorrectly resultsinalackorabundanceofedgesthatareusedinfurthe rpostprocessing,andconsequently,resultsinincorrectshipidentication.Therefo reitisveryimportanttochoosean algorithmthatwouldprovideaccuratehorizonlinedetecti onfordierentenvironmental conditionspossibleintheocean.Chapter5ofthethesispre sentsadetailedanalysisof horizondetectionalgorithmsbrierydiscussedinChapter1 .Itcomparesdierentalgorithms,showstheirperformanceanddescribestheiradvant agesanddisadvantagesforthe applicationofshipdetection.Figure3.2(d)showstheappl icationofthefoundhorizon lineperformedinpostprocessingstepofthealgorithm:edg esbelowthehorizonlineare removedfromconsideration.3.2.4PostprocessingSteps Thepurposeofthepostprocessingstepistocreateaprelimi narysegmentationof objectsbycreatingbinarymapswherethelocationofpossib letargetobjectsisdenedby non-zeroelementsofthemap.Theinputofthestepareparame tersofthehorizonline andabinaryedgemapfromtheedgedetector.Postprocessing stepszeroesallnon-zero elementsintheedgemapthatarelocatedbelowthehorizonli ne,thuseliminatingfrom resultsregionsoftheimagewheremarinevehiclescannotbe presents.Sinceallnon-zero pixelsareconsideredtobetargetpixelsitisrequiredtoha veallpixelsthatbelongto thesameobjectconnected.Furtherinthealgorithmconnect edregionsareconsideredto belongtooneobject,thusmakingthediscriminationofmult ipletargetspossible. Inordertoconnectcloselylyingseparateedgesbelongingt oasingleobjectmorphology operationsoftheerosionanddilationareused[28],[32].I nitialuseoferosionoperation 20

PAGE 29

allowslteringoutsomesmalledgesthatdonotbelongtotar getobject:partsofuneven horizonline,dierentobjectsessentialtotheoceanenvir onment-ryingbirds,cloudsetc. Furtheruseoferosionanddilationwithabiggerstructural elementallowsconnectionof binaryregionsthatliewithinacertaindistancerelativet othesizestructuralelement.The sizeandshapeofthestructuralelementforthisworkwerede terminedempiricallyonatest datasetof27images.Thediskshapewaschosenforthestruct uralelement.Thediameter ofthediskwasequalto2pixelsforlteringoperationsfoll owedbydiskwithdiameter7 toconnectcloselylyingpixelregions.3.2.5LabelingComponentsandOutput Theconnectedcomponentsalgorithm(describedinChapter2 )isusedtodetectboundingboxesaroundthefoundobjectsandgroundtruthobjects( whenperformanceisevaluated),tondoutthesizeoftheobjectsandtolteroutsomeo fthesegmentationresults. Everynon-zeropixelfromtheoutputofthepostprocessings tepisassignedalabel bytheconnectedcomponentsalgorithm.Pixelsthathavethe samelabelareconnected andbelongtoonedetectedobject.Thus,thissteptransform spixelwiserepresentationof thetargettolabelwise.Byhavingthepixelsoftheobjectsl abeleditispossibletocheck thelocationofthefurthestpointsoftheobjectinthetop,b ottom,right,andleftsides. Locationsofthosepixelsconstitutesthepositionofthesi destheboundingboxaround thefoundobject.Objectsthathaveintersectingboundingb oxesaremergedbyassigning asinglelabeltothepixelscomprisingthem.Afterthemerge stepboundingboxesare computedagainandtheirsizeiscalculated. Allfoundobjectsarecheckedfortheproximitytothehorizo n.Onlyregionswithin10 pixelsofthehorizonlinearefurtherconsideredtobetarge ts.Thisoperationreducesfalse alarmscausedbybirds,clouds,andaircraftwhicharealsol ocatedabovethehorizonline. Intheexperimentobjectswhichhadsizelessthen6pixelswe reeliminatedfromoutput. Thatisjustiedbythefactthatshipsintheseawithsuchsma llsizearelessimportantfor ahumanevaluatorandareusuallylocatedfarenoughfromthe placeofinterest.Filtering ofthesmallroatingobjectsfurtherreducesfalsealarmrat ewhenthesmallregionsabove 21

PAGE 30

thehorizonlinerepresentpartsofunevenhorizoninthesea ,birdsonthewatersurface, etc. Thealgorithmoutputsthepartsoftheoriginalimagecorres pondingtothefound boundingboxesinconnectedcomponentsstep(seeFigure3.2 (f)).Accuracyofdetectionofmarinevehiclesisdescribedbasedonhowwellthefou ndboundingboxesmatch groundtruthboxes.Evaluationandperformancemetricsfor suchdetectionareshownin Chapter4.3.3DetectionofMarineVehiclesinVideo Suchheuristicsasvicinitytothehorizonlineorsegmentat iontechniquesgreatlyreduce thealarmrateandincreasetheoverallaccuracyofdetectio n.Still,somestepsofthe algorithmarepronetoerrorswhichcauseincorrectresults .ExampleinFigure3.3shows oneofthetypicalfailuresduringthehorizondetectionste p.Thehorizonisdetected incorrectlyduetoawavesurfintheimage.Thepostprocessi ngstepcombinesedgesabove theincorrectlydetectedhorizonandconsidersmanymoreob jectsforpossibletargets.Bad performanceforshipdetectionmayalsostemfromtheedgede tectionstepwhenthesame objectisrepresentedbyseveraldisjointedges.Thathappe nsforexamplewhenthecamera isill-focusedandtheimageisblurred.Inthatcasetheinte nsityaroundthecontourdoes notshowsignicantchange,edgesarenotcrispandtheedged etectorndsonlydisjoint regionsofthesameobject.Laterinthealgorithmthatmaycr eateamissresultforthe originalshipandfalsealarmfortheattempt.Fortheexampl einFigure3.4thecamera wasnotfocusedwellenoughandthenalresultwastwoobject sfoundintheleftimage andevenmoreintherightimage. Videoinformationallowsustouseredundantinformationto increasetheaccuracyof shipdetection.Theassumptionusedhereisthattheobjectp resentinthecurrentframe shouldbepresentinthenextframeinclosevicinitytothecu rrentlocation.Thesize ofthedetectedobjectsshouldalsoshowconsistencybetwee ntheframes.Sharpchanges inthelocationandsizeoftheobjectsbetweentheframesmay beconsideredasignof 22

PAGE 31

Figure3.3.Horizondetectionexamples.Leftimagesshowscorrecthori zonidentication,inthe rightimage,becauseoftheseasurfthehorizonwasdetected incorrectly. Figure3.4.Examplesofmultipledetectionofasingleobject.afalsepositiveandsuchresultscanbedisregarded.Theopp ositesituationmayoccur whenatargetinaframeisoccludedbyobstacles.Forexample apartoftheshipmaybe occludedbywaterregionsintheimageorothershipsintheoc ean.Insuchasituation localizationofthemarinevehiclemaynotbepossiblebecau sesmallobjectsaredisregarded fromconsiderationinthebasicalgorithm. Inordertotacklesuchproblemsvideotrackingofthedetect edtargetsisperformed.To tracktheobjectsfromframetoframetheideaofanobservati onandastateofanobjectis used.Observationandstateofanobjectaretime-relateden tities.Thestateoftheobjects rerectsthetruepositionoftheobjectsintheenvironmenta taparticulartime,orframe t .Observationshowsthelocationofthetargetobjectdetect edbyanalgorithm.Having 23

PAGE 32

severalconsecutiveobservationsitispossibletodecidet hefollowingaboutthestateofthe object:itslocationandbehavior.Itisalsopossibletopre dictthenextstateoftheobject basedonahistoryofobservations.Trackingoftheobjectis consideredsuccessfulwhenthe predictionmatchestheactualmeasurement,i.e.whenbased onthehistoryofobservations oftheobjectapredictionaboutthepossiblenextlocationi smadeandconrmedbythe realdetectionoftheobjectinthatlocationinthenextfram e.Inrelationtomarinevehicle trackingitmeans,forexample,thatbasedona10frameseque ncethelocationofthe marinevehicleispredictedforthe11thframe.Iflater,int he11thframe,adetection algorithmshowsthelocationofthemarinevehiclethatpred ictedlocationthentheobject istrackedsuccessfully. Trackingallowshandlingsituationswithfalsealarmsandm isseddetections.Ifthe historyofobjectbehaviordoesnotsupportthemeasurement ofitslocation,thensuch detectionofanobjectinthecurrentframecanbedisregarde d.Thisreducesthefalsealarm ratecausedbythepossiblepresenceofnoiseinimages.Onth eotherhand,detectionofan objectcanbeconsideredforacertainlocationinaframeint hevideoevenifthedetection algorithmdoesnotshowpresenceoftheobject.Thatmayhapp enwhenthehistoryofthe objectfromthepreviousandfollowingframesshowsconsist entbehavior. TheKalmanlter,describedinChapter2,isagoodframework toimplementvision trackingofobjectsinvideo.Thenextsubsectionshowsdeta ilsofalterimplementation. 3.3.1UseoftheKalmanFilter Forthisworktwodierentltersaredenedtotrackthefoll owingtwoentities:the positionofthecenteroftheboundingbox(centroid)andthe boundingboxdimensions. Eachofthesetwolterstracksthebehaviorofthetwovariab les.Twoauxiliaryvariables foreachlterdescribethespeedofchangethemainvariable s.Forexamplefortherst lterthepairofvariablesthatdescribethepositionofthe centroid-namelythe x and y coordinatesaresupplementedbyauxiliaryvariablesthatd escribethespeedofmovementof thiscentroidalongthe x and y axes;horizontalandverticaldimensionsoftheboundingbo x 24

PAGE 33

W and H aresupplementedwiththespeedofchangingofthesevariabl esinthehorizontal andverticaldirections. Equations(3.1){(3.2)showtheimplementationoftheKalma nlterforvisualtracking usedinthethesis.Thetransformationmatrixin(3.1)estab lishestherelationbetween themainandauxiliaryvariablesinthecurrentandnextfram es.Thatrelationrerects thelinearnatureofthemotionofthemodeledobject:thepre dictedvalueofthemain variable(suchaslocationofthecorner)inthenextstate k +1isdierentfromthe previousstate k ontheamountofvalueofthecorrespondingauxiliaryvariab le xk: xk +1= xk+ xk.Themeasurementmatrixin(3.2)showscorrespondencebetw eenthe statevectorandmeasurementvector.Otherimportantvaria blesusedinthemodelare state wkandmeasurement vknoiseswhicharedescribedbyanormaldistribution. 266666664 xk +1yk +1 xk +1 yk +1377777775 = 1010010100100001 266666664 xkyk xk yk377777775 + wk(3.1) 264 xm kym k375 = 10000100 266666664 xkyk xk yk377777775 + vk(3.2) Here x and y representthemainvariablessuchasthepositionofthecent erordimension oftheboundingbox. x and y areauxiliaryvariables. TheKalmanltercantoleratebigdiscrepancies,socalledo utliersinmeasurementor, asitisoftencalledincomputervision,smallocclusions.I nthecasethatatrackingobject isnotfoundinacertainneighborhoodofthepredictedstate itisconsideredthattheobject ishiddenandwillnotusethemeasurementcorrectionbutins teadonlyprediction. 25

PAGE 34

Figure3.5.Outlineofmarinevehicletrackingalgorithm.(a)Detectio nofmarinevehiclesin singleimageorframe.(b)Trackingofdetectedobjectsthro ughvideosequence.3.3.2TrackingofMarineVehicles OuralgorithmforobjecttrackingissimilartotheMultiple HypothesisTracking(MHT) algorithm[38][39],butwithsignicantmodications.The featuresofthealgorithmincludetrackinitiation,continuation,andterminationfor multipleobjects.Modications introducedforthepurposeofthisworkarerelatedtotheres olutionofdataassociation uncertaintyaswellascomputationalspeedup. Figure3.5showstheoutlineofthealgorithmfortrackingde tectedobjects.Thealgorithmstartstheiterationforthecurrentframebyanalyz inghistoryofthepreviously trackedobjects.Itpredictsthepossiblelocationandsize ofboundingboxesaroundobjects byusingalinearKalmanlterasdescribedabove.Independe ntlythecurrentframeisa subjectoftargetdetectionbyusingthealgorithmdescribe dinSection3.2.Parametersof detectedobjectsarecomparedwithpredictions.Thosetrac ksthathadtheirobjectfound inthevicinityofthepredictedlocationareupdatedtorere ctthedetectioninthecurrent frame.Tracksthatdidnotreceiveevidenceoftheobjectint hecurrentframeusethe predictionsinsteadofdetectionstoavoidpossibleocclus ionsintheimage.Newtracksare initiatedforobjectsthatentertheframeorinthebeginnin gofthevideosequence.Tracks areterminatedwhenobjectsleavetheframeandalsowhentra cksarenotlongenough. Objectsareconsideredtobemarinevehiclesiftheyhavesub stantialtrackinghistory. 26

PAGE 35

Associationbetweenthedetectedobjectsandtracksiscons ideredaone-to-onemapping. Theclosestobjectintermsofeuclidiandistanceisassigne dtothetrackifitwithinthe validationregionforthattrack. Thetrackforanewobjectisinitiatediftheobjectthatdidn otbelongtoanytrackwas detectedintwoconsecutiveframesandtheboundingboxesof suchanobjectinthesetwo framesintersect.Valuesforthestateandmeasurementvect orsforthattrackareinitiated fromthesetwoframes.Suchasimpleinitiationofatrackcre atesmanyfalsealarms,but thosefalsetracksareterminatedquicklyifthedetectiond oesnotshowconsistencyinthe followingframes.Thetrackforanobjectisterminatedifth enumberofvaliddetections inthehistoryofthetrackislessthanhalfofthenumberofco nsecutiveframesforwhich thetrackexists.Objectswithatracklengthofmorethat20f ramesareconsideredmarine vehicles.Figure3.6shows3-frameexamplefromavideosequ encewithfalsealarms,true targetsandtheircorrespondingtracks. 27

PAGE 36

Figure3.6.Exampleoftrackingmarinevehiclesinvideosequence.Seaw avesarethemaincause offalsepositivesinbasicdetectionalgorithm.Theyarel teredoutifthedetectionsdonotshow consistencyinbetweentheframes.28

PAGE 37

CHAPTER4 DATAANDPERFORMANCEEVALUATION 4.1Overview Thischapterofthethesisdescribesquantitativemeasures toevaluatetheresultsof marinevehicledetectionanddatasetsusedtoevaluatethea lgorithms.Oneoftheissues withtheevaluationofshipdetectionperformanceingenera listhatitisoftenasubjective task:itishardtosaywith100%condenceifdetectionofans hipwascarriedoutornot. Thereforemethodsforevaluationofperformanceareaveryi mportantpartofanysuch systemandshouldbedenedinordertodeterminethesuccess orfailureofanalgorithm, measureimprovements,orproduceausefuldescriptionofth eresults. Theaccuracyofdetectionofmarinevehiclesinthisthesisi smeasuredbycomparing outputofthealgorithm,orthecandidatedata,withthegrou ndtruth,ortargetdata. Groundtruthisasetofresultswhicharecreatedbyahuman.F orthisworkgroundtruth annotationswerecreatedbytheresearchersforbothhorizo nandmarinevehiclesdatain ordertoevaluationperformanceofalgorithmsforbothcate gories. Quantitativecomparisonfordierentdetectionalgorithm sispossiblebyusingmechanismofmetrics.Metricsprovideadescriptionofhowwellt hecandidatedatamatches thetargetdatainaparticularsetting.Dierentmetricspr ovidedierentaspectsofperformanceofalgorithmsandeachmetricusuallyhasitsadvan tagesanddisadvantages.In contentofthisworkmetricsforboth,horizondetection,an dforthenalresult,marine vehicledetection,aredenedforasingleimageandforaset ofimages. 29

PAGE 38

4.2MetricsforHorizonDetectionEvaluation Thehorizonismodeledasasinglestraightlinewithzeropix elthicknessthatseparates two,assumablyhomogeneous,disjointregionsofpixelsint heimage-skyandoceansurface. Everylineischaracterizedbyitstwoparametersinpolarco ordinates: distancetothe originofthecoordinatesystemandanglebetweenthenorma ltothehorizonlineandthe verticalaxis(seeFigure4.1(a)).Givensucharepresentat ionofthehorizonitisdicult toquantitativelycomparelinesinthetargetandcandidate datajustbythematching correspondingparameters.Inordertoanswerthequestiono fhowclosetwodescription ofhorizonlineareitwaschosentousepixelwisecomparison ofthecandidateandtarget data.Thegroundtruthimageforhorizondetectionconsists ofblackandwhitepixelswhich areseparatedbyenvisionedhorizonline.Whitepixelsinsu chanimagecorrespondtosky regionsoftheimageandblackpixelsdenotetheoceansurfac e.Figure4.1(b)showsan exampleofsuchgroundtruth. Twometricswerechosentorerecttheaccuracyofthehorizon linedetection.First accuracyperformancemetricsrepresentsthepercentageof pixelsintheimage(throughout thewholedataset)correctlyseparatedintooceansurfacea ndnon-oceansurfaceregionsby thefoundline: A1= 1 kkXi =1Ni c Ni(4.1) where k isthenumberofimagesinthedataset, Ni c-numberofpixelscorrectlyseparated bythefoundhorizonlineinimage i Ni-numberofpixelsintheimage i .Thismetric providesareferencetoageneralperformanceofanalgorith monadatasetofimages.Its maindisadvantageisthatbadvisualperformanceofhorizon detectiononsomeimagesmay stayunnoticedbythesuchmetricoutputifthemajorityofot herimagesinthedatasetare processedwell. Secondmetricsisaimedtorerectperformanceofthealgorit hmoneachoftheimage ofthedataset.Thehorizonlineinanimageisconsideredtob edetectedcorrectlyifthe 30

PAGE 39

Figure4.1.Horizonrepresentation.(a)Thehorizonisrepresentedasa straightlineinpolar coordinateswithparameters and.(b)Groundtruthforanimage.Thewhitecolorregions correspondstothesky(abovethehorizon),blackregionsco rrespondtotheoceansurface.percentageofpixelsintheimagecorrectlyseparatedbythe horizonlineisaboveaspecied threshold.Thepercentageofsuchimagesinthedatasetwill denethesecondaccuracy metric: A2= nc k (4.2) where k isthetotalnumberofimagesinthedatasetand ncisthenumberofimagesinthe datasetwherethehorizonlinewasdetectedwithaccuracyab ovethegiventhreshold: Ni c Ni T (4.3) where T isthethresholdvalue. Anotherimportantcharacteristicforwhichthehorizondet ectionalgorithmswereevaluatedwastherunningtimeofthealgorithms.Themainrequir ementforfastperformance maycomefromsystemsthatdorealtimeprocessingofvideoda taorsystemswherethe numberofCPUcyclesusedfordetectionisrequiredtobeatth eminimallevelinorder satisfypowerconsumptionconstraints. 31

PAGE 40

4.3MetricsforMarineVehicleDetectionEvaluation Detectionofmarinevehiclesismorecomplicatedthanhoriz ondetectionandthus,more aspectsneedtobeevaluated.Insingleimagesdetectionimp liesspatiallocalizationoffound ships(allowingmultiplevesselsperimage).Invideo,comb inationofspatialandtemporal dimensionsrequirestrackingthetargetfromframetoframe ,thustemporallocalizationof thesameobjectsthroughouttheframesequenceshouldbeper formed. Marinevehiclesconsideredfordetectioninimagesandvide oare,inmostcases,compact objectsandthereforecanbecoveredbysimpleboundingshap es.Itwaschosentouse rectangularboundingboxasashapetomarkthegroundtrutha ndoutputdataofthe algorithm.Forsimplicityofrepresentation,thebounding boxisalwaysorientedsothatits sidesareparalleltotheaxesoftheimageplane.Eachground truthimageisabinaryimage wherewhiteregionscorrespondtobackgroundandblackrect angularregionscorrespond tothetargetobjects. Processofcreatinggroundtruthformarinevehiclesistedi ouswhenthenumberof groundtruthimagesissubstantial.Tofacilitatecreation ofgroundtruththeauthorused softwareViPER[40]tocreateboundingboxesaroundtargets inthedata.ViPERtoolwas createdtoproviderepeatabilityandcomparabilityinperf ormanceevaluation.Attributes ofdescriptors,suchasboundingboxes,mayberecordedfora rbitrarysetsofconsecutive framesinthevideowhichmakessuchtoolveryconvenientfor theresearcher. Detectionandtrackingmeasuresforperformanceevaluatio nwereadoptedfrom[41]. Thesecomprehensivemetricsaccountforimportantmeasure sofsystemperformancesuch asnumberofobjectsdetectedandtrackedmissedobjects,fa lsepositives,fragmentation inbothspatialandtemporaldimensions,andlocalizatione rrorofdetectedobjects.The followingnotationsareusedtodenethemetrics: Gidenotes ithtargetobjectatthesequenceleveland G( t ) idenotesthe ithtarget objectinframe t Didenotes ithcandidateobjectatthesequenceleveland D( t ) idenotesthe ithdetected objectinframe t 32

PAGE 41

G( t )isthesetoftargetsinasingleimage t D( t )isthesetofcandidatesfromthealgorithm. NGand NDdenotethenumberofuniquetargetsandnumberofuniquecand idates forthesequenceofframes. Nframesdenotesthenumberofframesinthesequence. Nmappedisthenumberofmappedtargetandcandidatepairs. FrameDetectionAccuracy(FDA)isthemetricthatrerectsac curacyofdetection localization.Itcalculatestheoverlapbetweenthetarget andcandidateobjectsasaratio ofthespatialintersectionbetweenthetwoobjectsandthes patialunionofthem.The sumofalloftheoverlapsisnormalizedovertheaverageofth enumberoftargetsand candidates.Foraframe t with N( t ) Gtargetsand N( t ) Dcandidates FDA ( t )isdenedas follows: FDA ( t )= OverlapRatio [N ( t ) G + N ( t ) D 2] (4.4) where;OverlapRatio =N ( t ) mappedXi =1j G( t ) iT D( t ) ij j G( t ) iS D( t ) ij (4.5) SequenceFrameDetectionAccuracymetricshowstheperform anceofdetectionforthe wholeframesequence.Thismetricsaccountsforbothmissed detectsandfalsepositives inonescore.ItisexpressedasFDAcalculatedoverallofthe framesinsequenceand normalizedtothenumberofframesinthesequencewheretarg etsorcandidatesexisted: SFDA = Pt = N frames t =1FDA ( t ) Pt = N frames t =19 ( N( t ) GORN( t ) D) (4.6) Nextmetricsaresimilartothehorizonmetric(4.2).Theyta keharddecisionfor eachtargetobject:detectedornotdetected.Atargetobjec tisconsidereddetectedif minimumproportionofitsareaiscoveredbythecandidate.T hresholdedOverlapRatio 33

PAGE 42

andThresholdedFrameDetectionAccuracy(FDA-T)aredene dasfollows: ThresholdedOverlapRatio = FDA T j G( t ) iS D( t ) ij (4.7) where FDA T = 8>>>><>>>>:j G ( t ) i S D ( t ) i j ;if j G ( t ) i T D ( t ) i j j G ( t ) i S D ( t ) i j Threshold j G ( t ) i T D ( t ) i j ;if j G ( t ) i T D ( t ) i j j G ( t ) i S D ( t ) i j
PAGE 43

Figure4.2.Examplesofimagesfromdatasetsusedfortestinghorizonde tectionalgorithms.(a) HORIZONDATASET1-noroatingobjects,clearhorizon;(b)HO RIZONDATASET2-marine vehiclespresentsonthehorizonline.Dataofthesamenatur ewasusedinSHIPDATASET2horizondetectionalgorithmsintermsofaccuracyofhorizo ndetectiononthetwodatasets ofimages. Therstdataset,furtherreferredasHORIZONDATASET1,was chosentotestthe generalcharacteristicsofthehorizondetectionalgorith m:abilitytocorrectlydetecthorizon lineinimageswhereonlyseaandskyregionwerepresent.Thi sdatasetconsistedof160 frameschosenrandomlyfromvideosequencestakenbyacamer ainstalledonabuoyinthe opensea.Aseparatesetof10similarframeswasusedfortrai ningpurposes.Allimages (framesofthevideo)werecolorimagesofresolutionof720x 480. Theseconddataset,referredtoastheHORIZONDATASET2,rep resented150frames pickedinasimilarwayfromvideosequencestakenbythesame buoycamera.Manyof theseimagescontainedships,roatingobjects,orwavestha tmadethehorizonlinelook uneven.Examplesofimagesinbothhorizondatasetsareshow ninFigure4.2. GroundtruthforbothsetswascreatedasdescribedinSectio n4.2.Forimageswherethe waves,ships,orotherobjectswerepresentthisgroundtrut hlinerepresentedthehorizon envisionedwithoutanyobjects.Thislocationofhorizonli newouldprovidethebestcue forfurthersegmentationoftheimageinordertolocatethes oughtmarinevehicles. Forevaluationofperformanceofthemarinevehiclesdetect ionalgorithmdatasetsof stillimagesandvideosequenceswereused.Hereafter,SHIP DATASET1referstothe 35

PAGE 44

datasetof100imagestakenbyadigitalcameraindaylightco nditions.Thecamerawas installedonshoreandtheimagescontainedonlyoceansurfa ce,roatingobjectsincluding ships.Theimagesdidnotcontainanycoastalobjects.Allim agesinthedatasetwerein resolutionof1280x960.Figure4.3showsasampleimagefrom thisdataset. Theseconddatasetformarinevehicleevaluation,furtherr eferredtoasSHIPDATASET 2,consistedof30videosequences.Thevideoforthedataset wastakenfromadigital camcordermountedonabuoyinopenoceanindaylight.Thecon tentofthevideoisthe oceansurfacewithpossiblesingleshippresentandnocoast alobjects.Thementioned marinevehicleinthevideosequencecouldbepresentthewho letime,couldenteror leavetheframeatsomepointoftime,orbeabsentthroughout thevideosequence.Each videosequencewas10secondslongandcontained300frames( 30fps)andwascreated ataresolutionof720x480.Thetotalnumberofframesinthed atasetis9000.Original videosequencesfromthecamcorderwererecordedinMPEG-1f ormatbutthedatafor thealgorithmwerefedframebyframe.Animportantaspectof theSHIPDATASET2is thefactthat,although,thenatureofthedataissequential ,thealgorithmsweretestedto detectmarinevehiclesinbothsettings.Intherstsetting theframesofthevideoswere consideredindependentlyfromneighboringframestotestd etectionofthebasicalgorithm asinsingleimages.Inthesecondsettingconsideredtracki ngamarinevehiclefromframe toframeand,thus,wasworkingontheseframesaswithordere dsequence. Imagesforhorizondetectiondatasetswereofthesamenatur easSHIPDATASET2, soFigure4.2canserveforillustrationpurposesofdatainS HIPDATASET2aswell. Groundtruthforthebothshipdatasetswascreatedasdescri bedinSection4.3.Chapter6 discussesresultsfordetectionofmarinevehicles. 36

PAGE 45

Figure4.3.ExampleimagefromSHIPDATASET1usedfortestingmarineveh iclesdetection algorithm.37

PAGE 46

CHAPTER5 COMPARISONOFHORIZONDETECTIONALGORITHMS 5.1Overview Detectionashipinthevicinityofthehorizonlineismuchea sierandreliablethan tryingtosegmentoutaship-bloboutofthewholehorizonima ge.Horizondetectionis morerobustanditspreciseidenticationreducesthepossi blesearchspaceformarine vehicles.Alsosegmentationofashipoutoftheskybackgrou ndcanbeperformedwith simpleimageprocessingtechniquescomparedtocomplexmet hodsofobjectsegmentation outofthewholesky-seaimage. Thischapterinvestigatestheperformanceofthehorizonde tectionalgorithmsdescribed inChapter1ontwodierentdatasetsofhorizonimages(seeS ection4.4).Therstset, HORIZONDATASET1,containsonlyhorizonimagesofclearoce ansurfacewithoutmarinevehicles(seeFigure4.2(a)).Thesecondtestset,HORI ZONDATASET2,introduces shipsandotherroatingobjectsonthehorizonwhichgeneral lyinruencehorizondetection (SeeFigure4.2(b).Somemodicationstotheoriginalalgor ithms,introducedforthe purposeofbetterperformanceof"shipimages"weretestedo ntheseconddataset. Particularre-implementationsofhorizondetectionalgor ithmsandtheirmodication willbenamed. ThehorizondetectionalgorithmsdescribedinChapter1wer edevelopedbyteamsof researchersmostlyforprojectsrelatedtonavigationofun mannedaerialvehicles(UAV). Requirementsforthesealgorithmswerequitedierentfrom therequirementsforthealgorithmdescribedinthiswork.WhilenavigationofUAVneedst hehorizonlineinorderto avoidobstaclesthatareassumedtobebelowthehorizonoura pplicationneedsthehorizon lineforreliablemarinevehiclesegmentationinimagesand videosequenceswhichareas38

PAGE 47

Figure5.1.Exampleoffailureofhorizondetection.Assumptionthatth eimageconsistsoftwo regions-skyandnon-skyleadstothedetectionofhorizonli neabovethevessel.sumedtobelocatedabovetheenvisionedhorizonline.Outpu tofthealgorithmsthatmay besuitablefornavigationofUAVmaynotbeappropriateford etectionofmarinevehicles. Figure5.1showstheresultofthehorizondetectionalgorit hmdescribedin[18].Because ofthevesselpresentintheimagethealgorithmfoundthelin ethatbestseparatesskyand non-skyimageliesabovethesoughtship,aresultnotaccept ableforourapplication. Theliteraturereviewsectionofthisdocumentcontainsref erencestovecategoriesof approachestodetecthorizoninimages.Someoftheseapproa ches[26,27]relyonspecic hardwaretoobtainimages,and,thus,arenotappropriatein thecontextofouralgorithm. Theapproachdescribedin[21,22]wasdesignedtousethehor izonasanestimatorforroll andpitchangle,andprecisionofthehorizonlineintheimag eitselfwasnottheprimary target.Theangleofthehorizonisdeterminedbytakingthep erpendicularoftheline joiningskyandnon-skycentroids.Incaseofrectangularim ages(sourceofourdata), thelinebetweenthecentroidsdoesnotturnoutexactlyperp endiculartothehorizon.In additiontheprocessofassigninglabelstopixelsintosky/ non-skyclassesbasedjustonthe grayscaleintensityvalueisinitselfveryinaccurate.Ani nterestingandfastapproach[23] toapplyaprojectionproletoanedge-imageisgoodenoughf orstraighthorizonlineswith littletexture(sourceofedges)comingfromtheoceansurfa ce.Butasitwastestedwith someimagesfromourevaluationdatasetthismaynotbetheca se,andsomeanomaliesin horizondetectionareobservedevenforsimpleimages. 39

PAGE 48

Itwasdecidedtoconcentrateontwomaincategoriesofappro achesforhorizondetection.Therstcategory,describedinanumberofpapers[17, 18,19,20],isastatistical approachthattriestominimizeintra-classvariancebetwe entwocategoriesofpixels.This approachdoesnotassumeanytrainingandisreferredtoasun supervised.Thesecondcategoryofalgorithms,describedin[24,25],assumestrainin gofaclassieronsomelabeled dataandisreferredassupervisedapproach.Thenexttwosec tionsdescribeindetailthese originalalgorithms.Thefollowingsectionsshowsomemodi cationstothesealgorithms duringourre-implementation,aswellasacomparisonofper formanceofthesealgorithms. 5.2HorizonDetection:UnsupervisedApproach Theunsupervisedapproach[18]tohorizondetectionalgori thmusestwobasicassumptions: thehorizonlineappearsintheimageapproximatelyasastra ightline;and thehorizonlineseparatestheimageintotworegionsthatha vedierentappearance -skyandnon-sky.Intheskyregionpixelswilllookmorelike otherskypixels,and lesslikenon-sky,andviceversa. Therstassumptionreducesthesearchspaceforallpossibl ehorizonstoatwo-dimensional searchinline-parameterspace.Foreachpossiblelineinth attwo-dimensionalspaceaspecialcriteriondetermineshowwellthatparticularlineagr eeswiththesecondassumption orinotherwordshowwellthecorrecthorizonlineseparates theimageintotworegions thathavedierentappearance.Themethodutilizesthenorm alrepresentationofaline (seeFigure4.1(a)forillustration): x cos+ y sin= (5.1) where( x;y )arethecoordinatesofthepointsontheline,istheangler epresentingthe rotationoftheline,and istheperpendiculardistancefromthelinetotheorigin. 40

PAGE 49

TheRGBcolorwaschosenasameasureofappearance.Foranygi venhypothesized horizonline,thepixelsabovethelinearelabeledassky,an dpixelsbelowthelineare labeledasnon-sky.Allhypothesizedskypixelsaredenoted as, xi s=[ ri sgi sbi s] ;i 2f 1 ;:::;nsg (5.2) where ri sdenotesthevalueforintensityoftheredchannelinRGB, gi sdenotesthegreen channelvalueand bi sdenotesthebluechannelvalueofthe i -thskypixel.Allthehypothesizednon-skypixelsarenotedas, xi g=[ ri g;gi g;bi g] ;i 2f 1 ;:::;nsg (5.3) Giventhesepixelgroupings,theassumptionthatskypixels looksimilartoothersky pixels,andthatnon-skypixelslooksimilartoothernon-sk ypixelsisquantied.One measureofthisisthedegreeofvarianceexhibitedbyeachdi stribution.Theproposed optimizationcriterionforsuchvarianceisthefollowing: J ( ; )= 1 s+g(5.4) basedonthecovariancematricesandofthetwopixeldistrib utions, s= 1 ns 1n sXi =1( xi s s)( xi s s)T(5.5) g= 1 ng 1n gXi =1( xi g g)( xi g g)T(5.6) where nsisthenumberofsky-pixels, ng-numberofnon-skypixels,and sand gare meanvectorsforcolor-intensityoftheskyandnon-skypixe ldistributionsandaredened as s= 1 ns n sXi =1xsi;g= 1 ng n gXi =1xgi(5.7) 41

PAGE 50

Assumingthatthemeansoftheactualskyandgrounddistribu tionsaredistinct,theline thatbestseparatesthetworegionsshouldexhibitthelowes tvariancefromthemean.Ifthe hypothesizedhorizonlineisincorrect,somenon-skypixel swillbemistakenlygroupedwith skypixelsandviceversa.Theincorrectlygroupedpixelswi llliefartherfromeachmean, consequentlyincreasingthevarianceofthetwodistributi ons.Theincorrectlygrouped pixelswillskeweachmeanvectorslightly,contributingfu rthertoincreasedvarianceinthe distributions.5.3HorizonDetection:SupervisedApproach Theapproachesdescribedin[24]and[25]usemachinelearni ngtechniquestoclassify eachpixelintheimageintoskyandnon-skyclassbasedonits colorandtextureofa regionaroundit.Theclassicationoftheimageintoskyand non-skyin[24]isperformed usingasupportvectormachine(SVM)classier[42]thatass ignsacategoryforeachpixel basedonitscolorintheRGBcolorspace,aftersmoothingthe imagewithaGaussian ltertoreducetheeectsofnoise.Theapproachdescribedi n[25]comparesmultiple classiersandusesmultipletexturefeaturesalongwithth ecolorofpixels.Theauthorsof thepaperconcludethatSVMperformsbetterthanotherclass iersduetothefactthat SVMisinherentlyabinaryclassier. Althoughideaofpixelclassicationissimilarinbothappr oachesmethodsofnding theactualhorizonlinefromtheclassiedimagearedieren t.In[24]thehorizonisfound bysearchingforthelinethatbestrepresentstheboundaryb etweenbinaryskysegmented imageintoskyandground.Therststepistoapplystandardm orphologyoperationsof erosionanddilation[28],[32]tothebinarysegmentedimag etoremoveanysmallsectionsof misclassiedpixels.Theborderbetweenskyandgroundregi onsisthenfoundbysmoothing thebinaryimageandclassifyingallpixelswithvaluesnear 0.5asboundarypixels.The horizonlinedetectionisperformedusingtheHoughtransfo rm[28]ontheborderimage withanormalrepresentationofaline.Eachlinein( X;Y )spacecanberepresentedbya pointin( ; )space.Conversely,eachpointinthe( X;Y )spacerepresentsasetoflines 42

PAGE 51

in( ; )space,whichcorrespondtoallofthepossiblelinesthatpa ssthroughthatpoint. Thus,foreachpointin( X;Y )space,avoteisplacedineachbinofadiscretized( ; ) spacethatcorrespondstoallpossiblelinesthatpassthrou ghthatpoint.Binsin( ; ) spacethatreceivealargenumberofvotescorrespondtoprob ablelines.Thebesthorizon ischosenasthecandidatelinewhichminimizesthecostfunc tion: J ( ; )= Xi;j~e ( ~xij)(5.8) ~e ( ~xij)= 8>>>>>>>>>><>>>>>>>>>>: 0if f ( ~xij)=1 ; ( i;j )isabovehorizon if f ( ~xij)= 1 ; ( i;j )isabovehorizon 1if f ( ~xij)=1 ; ( i;j )isbelowhorizon 0if f ( ~xij)= 1 ; ( i;j )isbelowhorizon (5.9) wheref(x)istheclassicationoutput,1forsky,-1forgrou nd, xijisthecolorvectorfrom thepixelat( i;j )intheimage,and isapositiveconstant.Thus, J ( ; )isaweighted sumofallofthepixelsabovethecandidatehorizonclassie dasnon-skyandallofthe pixelsbelowthecandidatehorizonclassiedassky. Theothermethod[25]inordertondhorizonlineinaclassi edimageusesan approachsimilartotheonedescribedinSection5.2.Butins teadofcolorinformationused todescribeeachpixel,binaryinformation(classofapixel )isused. 5.4HorizonDetectionPerformanceontheDatasetwithoutSh ips Thealgorithmsthatweretestedonthisdatasetofimagesinc ludedthehorizondetection algorithmdescribedinSection5.2,aderivationofitthati ncludestexture,andanalgorithm [24]describedinSection5.3.Thealgorithm[25]whichuses classicationofpixelsinthe imageintoseasurface/non-seasurfacebasedonmanytextur efeatureswasnotconsidered 43

PAGE 52

becauseofmanycomputationally-intensivetexturemeasur ementsneededtobeperformed, operationsthatmaketherun-timeprohibitivelybig. Thealgorithmfrom[18],furthercalledUNSUPERVISED,wasi mplementedbasedon thepaper'sdescriptionasthesearchforthebestparameter sintwosteps.Duringthe rststepthealgorithmusedtheimagescaledto1/5oftheori ginaldimensioninorder tondapproximateparameters'values.Scalingoftheimage isdoneforthepurposeof speedingupthecomputations.Thesecondstepsearchesfort hehorizonparametersinthe originalimage(withoutscaling)butusestheresultofthe rststepasastartingpointand thesearchisperformedinthelimitedvicinityofthesearch resultoftherststep,thus, reningtheresultoftherststep.Featuresusedtodisting uishskyandnon-skyregions areRGBintensityofthepixelsthatcomprisethemavailable directlyfromtheinputimage. AderivationoftheUNSUPERVISEDalgorithmthatusestextur einformation,further calledUNSUPERVISED-TEXTURE,usesthesametwo-stepappro achtondthehorizon parameters,butalongwithRGBcolorsitalsoincorporatedt exturemeasurementofuniformity(see(2.11)andFigure2.2(g)forillustration).Un iformitywaschosenoutofthe numberoftexturefeaturesdescribedinSection2.4because itprovidesmoredistinctionfor valuesinseasurface/non-seasurfaceregions.Thistextur emeasurementwasassignedto eachpixelintheimageandwascalculatedasameasurementof uniformityinthegrayscale patchof10x10pixelsaroundthetargetpixels. Thelastalgorithmconsideredforevaluation[24],describ edinSection5.3,isbasedon classicationofpixelsintoseasurface/non-seasurface. Furtherinthetextthisalgorithm isreferredasSUPERVISEDalgorithm.Totraintheclassier 10imagesofthetrainingset wereused.Thetotalof1%pixelsineachoutof10trainingima geswaschosenrandomly tocreateasetoftrainingvectors(about38,000vectors).T heclassicationresultswere followedbymorphologyoperationsoferosionanddilationw ithastructuralelementofdisk withradiusequalto7pixels. TheTable5.1showstheperformanceresultsforthethreealg orithmsobtainedonthe HORIZONDATASET1withthemetricdescribedby(4.1).Theres ultsareshownforthe 44

PAGE 53

accuracyrangeof82%andabove,wherethedescribedhorizon detectionalgorithmshow thebiggestdierenceinperformance. Table5.1.AccuracyofthehorizondetectionalgorithmsontheHORIZON DATASET1 accordingtometric(4.1). UNSUPERVISED SUPERVISED UNSUPERVISEDTEXTURE AccuracyofDetectionAccordingto(4.1) 99.26% 98.89% 97.94% Table5.2showstheperformanceofthesamealgorithmsusing thesecondmetrics describedby(4.2).ThechartinFigure5.2showsthesameres ultsvisually. Table5.2.AccuracyofthehorizondetectionalgorithmsontheHORIZON DATASET1 accordingtometric(4.2). THRESHOLD Algorithm 84% 86% 88% 90% 92% 94% 96% 97% 98% 99% UNSUPERVISED 98.75% 98.75% 98.75% 98.75% 98.75% 98.75% 98.75% 98.75% 97.50% 95.00% SUPERVISED 98.75% 98.75% 98.12% 98.12% 96.88% 95.63% 95.00% 94.37% 92.50% 91.25% UNSUPERVISEDTEXTURE 99.38% 99.38% 98.75% 97.50% 96.88% 93.13% 86.25% 76.88% 65.63% 46.88% Table5.3showsrunningtimeinsecondsperimagefortheseal gorithms.Thealgorithms wereimplementedinMATLABenvironmentandtestedonAMDAth lon64X2Dual3GHz processor.FortheSUPERVISEDalgorithmtherunningtimedo esnotincludetraining time.Trainingofthealgorithmissupposedtobedoneoine. Table5.3.RunningtimeofthehorizondetectionalgorithmsonHORIZON DATASET1in relativetimeunitsperimage. UNSUPERVISED SUPERVISED UNSUPERVISEDTEXTURE Runningtime(timeunits) 17.1 1.4 44.4 TheresultofcomparisonexcludestheUNSUPERVISEDTEXTURE algorithmfrom furtherconsiderationbecauseofitsinferioraccuracyand signicantrunningtime. 45

PAGE 54

84% 86% 88% 90% 92% 94% 96% 97% 98% 99% 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 ThresholdAccuracy UNSUPERVISED SUPERVISED UNSUPERVISED TEXTURE Figure5.2.AccuracyofhorizondetectionalgorithmsonHORIZONDATASE T1,metric(4.2)is used.5.5HorizonDetectionPerformanceontheDatasetofImagesw ithFloating ObjectsPresent ThealgorithmstestedinthisexperimentincludedUNSUPERV ISED,SUPERVISED algorithms,andanotherderivativeofUNSUPERVISEDalgori thm.Asmentionedbefore oneoftheproblemswithhorizondetectionalgorithms,espe ciallywithUNSUPERVISED, isthattheoptimallineforhorizonmaybeskewedbypossible objectspresentonthe horizonline.Amarinevehiclemaybeoneoftheseobjects. AderivativeoftheUNSUPERVISEDalgorithm,furtherreferr edasUNSUPERVISED SLICE,appliesthesameideaforhorizondetectionastheori ginalUNSUPERVISEDalgorithm:thesoughthorizonlineisthelinethatbestseparate shomogeneouslycoloredtwo distributions,seasurface/non-seasurface.Thealgorith mdividedtheoriginalimageinto n disjointverticalslices(inthisparticularimplementati on n waschosentobe20)and ndshorizonlineineachofthese.Thehorizonlineforthewh oleimageisfoundasthe combinationofthe"smallhorizons"intheseslicesoftheor iginalimage.Theideabehind thistechniqueistodisregardthosesliceswheretheroatin gobjectsskewtheresultforthe 46

PAGE 55

Figure5.3.StepsofUNSUPERVISED-SLICEalgorithm.(a)horizonlinesa refoundforeachof theslicesintheoriginalimage.(b)Houghtransformapplie donsynthesizedimagecontaining onlyhorizonlinesfortheslices.Redlineshowsthehorizon found.(c)Foundhorizon superimposedontheoriginalimage.wholeimage.Possibletechniquesforsuchcombinationof"s mallhorizons"mayinclude themedianlter,whereonlyapartofthevaluesforhorizoni sused,theHoughtransform forimageofcombinedhorizons,oratechniquesimilartowei ghted-majorityalgorithm[43]. UNSUPERVISEDSLICEalgorithmevaluatedinthisexperiment usedHoughtransform. Itwasappliedtoabinaryimageofcombinedhorizonresultsf romslicesoftheoriginal image.Figure5.3showsstepsoftheUNSUPERVISEDSLICEalgo rithmasanexample. Itshowsthatthehorizonlinesfoundforslices,wherethesh ipwasobserved,havebeen disregardedanddidnotcontributetowardsthenalhorizon line. 47

PAGE 56

Table5.4.AccuracyofthehorizondetectionalgorithmsonHORIZONDAT ASET2accordingto metric(4.1). UNSUPERVISED SUPERVISED UNSUPERVISEDSLICE AccuracyofDetectionAccordingto(4.1) 98.41% 99.29% 99.41% Table5.5.AccuracyofthehorizondetectionalgorithmsonHORIZONDAT ASET2accordingto metric(4.2). THRESHOLD Algorithm 84% 86% 88% 90% 92% 94% 96% 97% 98% 99% UNSUPERVISED 96.67% 95.33% 94.67% 94.67% 94.67% 94.67% 92.00% 92.00% 88.00% 82.67% SUPERVISED 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% UNSUPERVISEDSLICE 100% 100% 100% 100% 100% 100% 100% 98.67% 95.33% 88.00% TheperformanceofthethreealgorithmsontheHORIZONDATAS ET2forboth metricsofaccuracyandrunningtimeisshowninTables5.4,5 .5,5.6andFigure5.4. 48

PAGE 57

Table5.6.RunningtimeofthehorizondetectionalgorithmsonHORIZON DATASET2in relativetimeunitsperimage. UNSUPERVISED SUPERVISED UNSUPERVISEDSLICE Runningtime(timeunits) 17.1 1.4 25.1 84% 86% 88% 90% 92% 94% 96% 97% 98% 99 0.8 0.825 0.85 0.875 0.9 0.925 0.95 0.975 1 UNSUPERVISED UNSUPERVISED SLICED SUPERVISED Figure5.4.AccuracyofhorizondetectionalgorithmsonHORIZONDATASE T2,metric(4.2)is used.49

PAGE 58

5.6SelectionofHorizonDetectionAlgorithm Thehorizondetectionalgorithmsevaluatedinthischapter areimportantforaccurate shipsegmentation.Forourapplicationperformanceofthea lgorithmsontheHORIZON DATASET2ismoreofconsiderationbecausetherealisticdat aforafutureapplication mayincluderoatingobjects.Twoalgorithmsperformedwell onthatdataset-theUNSUPERVISEDSLICEandSUPERVISED.TheUNSUPERVISEDSLICEis thehorizon detectionalgorithmofourchoiceintheoveralldetections cheme. ItisworthmentioningthatwhiletheUNSUPERVISEDSLICEisb etterintermsof accuracy,ithasanotabledisadvantage.TheUNSUPERVISEDa lgorithmanditsderivativesworkasunsupervised-classierswhichalwaystrytod rawalinebetweenthetwo mostdistinctregionsintheimage,i.e.twoclusters:seasu rfaceandnon-seasurface.For somesituationswhenanimagemaycontainonlyskyoronlysea surfacepixels,ordierentregionsoftheseasurfacehavemoredistinctionthenmaj orcategoriesofseaandsky suchclasterizationmayleadtopoorperformanceresults.T heSUPERVISEDalgorithm ismoreadjustableforsuchsituationbecauseitsclassier canbetrained(however,the performancemaysuerifthetrainingdataislimited).Inad ditionclassicationofpixels andsubsequentHoughtransformisdonemuchfastercompared tothesearchinparameterspaceperformedbytheUNSUPERVISEDalgorithmanditsde rivatives.Withfuture modicationsandheuristicsaderivativeoftheSUPERVISED algorithmmaymatchThe UNSUPERVISEDSLICEintermsofperformance. 50

PAGE 59

CHAPTER6 RESULTSONMARINEVEHICLEDETECTION Thischapterofthethesisevaluatesthenalresultsofship detectionusingthealgorithm describedinpreviouschapters.Theimageandvideodataset saswellasperformance metricsdescribedinChapter4wereusedtoevaluatetheprop osedshipdetectiontechnique. TheUNSUPERVISEDSLICEhorizondetectionalgorithm,descr ibedinChapter5,was usedtodetectthehorizonline.6.1PerformanceofAlgorithmonSingleImages DuringthisexperimenteachimageinSHIPDATASET1wasproce ssedseparately.No trackingwaspossibleandthusonlyabasicdetectionscheme describedinSection3.2and showninFigure3.1wasused.Thefollowingparameters(foun dempirically)andsettings wereused: Theshapeofastructuralelementformorphologyoperations usedisdisk. Thesizeofthestructuralelementforinitialerosionis2pi xels. Thesizeofthestructuralelementforthefollowingdilatio nis7pixels. Minimumsizeofamarinevesselinanydimensionshouldbe6pi xels. Afoundobjectsshouldbewithin10pixelsfromthehorizonli neinordertobe consideredaship. Table6.1showstheperformanceofthealgorithmonSHIPDATA SET1recordedusing SequenceFrameDetectionAccuracymetricforbinaryandnon -binarythresholdoptions (SFDABT,SFDANBT).TheregularSFDAmetricshows75%accura cyvalue.There 51

PAGE 60

werenomissedobjects.Therateoffalsealarmswas3%.Figur e6.1showsexamplesof thedetectionofshipsinsingleimagesfromaforward-looki ngcamera. Figure6.2showstypicalfailuresindetectionlocalizatio nonSHIPDATASET1.Most ofthetimefalsealarmspointoutpartoftherealtargets,wh ichhavealreadybeendetected (fragmentationproblem). Table6.1.Resultofmarinevesseldetectioninsingleimages. ThresholdValue SFDANBT SFDABT 0.2 94.02% 88.80% 0.4 86.96% 75.87% 0.6 53.26% 41.57% 0.8 27.17% 15.88% 6.2PerformanceofAlgorithmonVideoSequences Theperformanceoftheshipdetectionalgorithmonvideowas evaluatedontheSHIP DATASET2whichwasdescribedinSection4.4.Theexperiment wasconductedwiththe samesettingsandparametersforthebasicdetectionpartas fortheexperimentwithsingle images.Theparametersandsettingsusedforthetrackingpa rtofthealgorithminclude thefollowing: Atrackisinitiatediftheobjectispresentintwoframesand itsboundingboxesin twoframesspatiallyintersect. TheparametersoftheKalmanlterforthetrackareinitiali zedfromthesetwoframes Thevalidvariationoflocationandsizeoftheboundingboxi sobtainedfromcovariancematrixforthestatevector.Thedetectionoftheobject inaframeisconrmedif itiswithinthevalidationrangeforit'slocationandsizep redictedfromtheprevious measurement. Iftheobjecthasnotbeendetectedinaframethepredictioni susedastheactual measurementinordertocontinuetracking. 52

PAGE 61

Table6.2.SFDAmetricsfortwodierentsettings:onindividualframe sandonvideosequence withtracking. IndividualFramesWithoutTracking VideoSequencesWithTracking Threshold SFDA SFDANBT SFDABT SFDA SFDANBT SFDABT 0.2 33.18% 53.68% 52.17% 68.85% 92.43% 92.29% 0.4 45.26% 38.72% 88.65% 80.18% 0.6 34.92% 28.67% 83.42% 67.07% 0.8 26.88% 19.22% 72.64% 48.16% Atrackisterminatedifthenumberofvaliddetectionsinatr acksequenceisless thanhalfofthelengthofthetracksequence. Theobjectisconsideredamarinevehicleandisshowninalgo rithm'soutputifthe trackhasalengthofnolessthan20frames.Thisparameterwa schosenmanuallyin ordertoavoiddetectionofseawaves.Fordierentframe-pe r-secondsettingofthe videosequencethisparametersshouldbechanged. TheSequenceFrameDetectionAccuracy(SFDA),SequenceFra meDetectionAccuracyNon-BinaryThreshold(SFDANBT),andSequenceFrameDe tectionAccuracyBinaryThreshold(SFDABT)metricswerecalculatedonthedata setofvideosequencesfor twosettings:fortheresultsofdetectionformarinevehicl esindependentlyineachframe andfortheresultsofmarinevehiclesdetectedandtrackedi nvideosequences.Sucha setupoftheexperimentallowsevaluationofimprovementwh ichgivestheadditionofthe trackingalgorithmtothebasicdetectionalgorithm.Themi ssrateandfalsealarmrate werecalculatedforthesecondsetting. Table6.2showstheperformanceaccordingtothosemetrics. Thevalueforeachsetting rerectsSFDAperformanceaveragedfromall30videosequenc espresentinthedataset. SFDA-thresholdsettingsshowtheperformanceofthealgori thmaccordingtofourthreshold valueschosentorerectthespatiallocalizationaspectoft healgorithm.Resultsforthe settingwheredetectedobjectsweretrackedthroughthevid eosequencesshowsignicant improvementoverindependentdetectioninsingleframes.T hereasonforimprovementisin substantialreductionoffalsealarms.Overall,thevisual resultsofdetectionarepromising: targetsweredetectedandtrackedinallvideosequencesoft hedataset.Missedobjectsat 53

PAGE 62

the2.27%rate.Therateoffalsealarmswas11%.Figure6.3sh owsexamplesofdetection ofshipsinvideo. Itisworthmentioningthattheperformanceofthealgorithm onsingleimagesforthe SHIPDATASET2issignicantlyworsethanontheSHIPDATASET 1.Thenatureof theSHIPDATASET2(camerainstalledonabuoywhichissubjec ttomotion,aswellas thelowheightofthecameraabovethesurface)makesitharde rforthebasicalgorithm todetectthehorizonandsubsequentlytheship.Occlusiono ftheshipbyoceanwavesin frontofthecameraisanotherreasonforsuchperformance.W ithouttrackingpartofthe algorithmthedetectionofshipsinsuchconditionisveryli mited. Theworstresultsintrackinginvideowasobservedonsequen ceswherethesizeofthe shipwassmall(lessthan20pixels).Inthatcasevalidation rangeforthedetectionisalso smallanditiseasiertomissthetrackoftheshipbetweenthe framesinapresenceofthe strongbuoy(andcameraattachedtoit)motionorvisualoccl usionoftheshipbythesea waves. Thedatasetforvisualtrackingofmarinevehiclescontaine donlyoneornoshipspresent inthevideo.Incasewhenmultipleshipsarepresentontheho rizonperformanceofthe trackingmaysuerfromtargetIDswitchwhenatrackinitial lyassignedtooneobjectmay bereassignedtoanotherdetectedtarget.However,iftheID oftargetsisnotimportantand onlythelocationofthetargetsneedstobereportedtheprop osedschemeshouldprovide resultswithintheaccuracyrangeshownabove. 54

PAGE 63

Figure6.1.Resultsofmarinevehicledetectioninsingleimages.55

PAGE 64

Figure6.2.Examplesofshiplocalizationfragmentation.Someofthesi ngle-objectshipswere detectedmultipletimescausingfalsealarms.56

PAGE 65

Figure6.3.Examplesofshipdetectioninvideo.Thepictureonthebotto mshowsanexampleof failureofdetectionduetotemporalfragmentationofthetr acks.57

PAGE 66

CHAPTER7 CONCLUSIONS Anewtechniqueforautomaticdetectionandtrackingofmari nevehiclesinimagesand videoofopenseawaspresented.Theproposedcomputervisio n-basedalgorithmcombined ahorizondetectionmethodwithedgedetectionandpostproc essing.Severaldatasetsof stillimagesandvideosequenceswereusedtoevaluatethepe rformanceoftheproposed technique.Forvideosequencestheoriginalalgorithmwase nhancedbyusingtheKalman lterandatrackingalgorithmderivedfromMultipleHypoth esisFramework.Because thedetectionalgorithmdependsonaccuratehorizondetect ion,severalhorizondetection algorithmswereevaluatedforaccuracyofhorizondetectio n. Experimentsconductedonimageandvideodatashowedpromis ingresultsformarine vehicledetectionandtracking.Highdetectionratewitham oderatealarmrateforlow thresholdvalueswasrecordedforstillimages.Thevisualr esultsfortheexperimentsrerect thefactthatatypicalfailureforshipdetectioninasingle frameoccursduetomultiple regionrepresentationofsingleobjects.Thiscreatesamis sresultfortheoriginalshipand afalsealarmfortheattempt.Theresultssuggestthatthema incauseoffailuresandfalse alarmsareedge-detectionandpostprocessingstepsintheo riginalalgorithmwhenasingle objectintheseaisrepresentedbyseveraldisjointregions inthepostprocessedimage. Themainissueswithbadtrackingperformancearerelatedto temporalfragmentation oftracks.Reducingtemporalfragmentationoftracksmaylo werthefalsealarmrate andincreasetrackingaccuracyinthefuture.Temporalfrag mentationintrackingusually occurswhenonetrackisprematurelyterminatedbecauseoft helongframe-rangeocclusion andanewtrackisstartedwhenthewatercraftisagainvisual lyavailable.Introducing 58

PAGE 67

recognitionofobjectsinseparatetracksmaybeoneoftheap proachestoreducesuch temporalfragmentation. Infutureworkbasicnoiseremovalandcolorenhancementmay beappliedforamore reliablehorizondetectionwhichfollowsimageacquisitio n.Anintroductionofatraining stepinordertoidentifytheparametersoftheCannyedgedet ectormayincreasetheinitial segmentationaccuracyandmakeitadaptableforvariousday lightconditions.Thesegmentationaccuracywillbeimprovedwithanadditionof"mer ge"procedure,whichwould connectcloselylocatedsegmentsbelongingtooneobject.O neofthepossibledirections hereisconnectingintermittentlinesofthecontourofmari nevehiclesusingsegmentnding techniquesdescribedin[44]. Thecurrentdetectionschemeassumesdeploymentofabuoyin openseawatersand coastallinewasnotconsideredtobepresentonthehorizon. Furtherdirectionofthe researchmayconsiderthis,morecomplicatedenvironment. Someresultsonmaritime surveillanceincoastalareacanbefoundin[45].Detection ofmarinevehiclescanbealso pairedwithmarinevehiclesclassication.Oncedetectedi nanimageorvideosequencea shipcanbeclassiedintoanumberofcategorieswithmethod ssimilartothosementioned intheChapter1forFLIR-basedimages.Determinationofthe distancetothetargetmay beanotherdesiredfunctionalityfollowingvisualdetecti on. 59

PAGE 68

REFERENCES [1]M.I.Skolnik. Introductiontoradarsystems .McGraw-HillNewYork,1980. [2]R.Khan,B.Gamberg,D.Power,J.Walsh,B.Dawe,W.Pearso n,andD.Millan. Targetdetectionandtrackingwithahighfrequencygroundw averadar. Oceanic Engineering,IEEEJournalof ,19(4):540{548,1994. [3]B.Root.HFradarshipdetectionthroughcluttercancell ation. RadarConference, 1998.RADARCON98.Proceedingsofthe1998IEEE ,pages281{286,1998. [4]M.Hirota,T.Furuse,K.Ebana,H.Kubo,K.Tsushima,T.In aba,A.Shima,M.Fujinuma,andN.Tojyo.Magneticdetectionofasurfaceshipby anairborneLTSSQUID MAD. AppliedSuperconductivity,IEEETransactionson ,11(1Part1):884{887,2001. [5]J.C.CurlanderandR.N.McDonough.Syntheticaperturer adar-Systemsandsignal processing(Book). NewYork:JohnWiley&Sons,Inc. ,1991. [6]M.T.Rey,J.K.Tunaley,J.T.Folinsbee,P.A.Jahans,J.A .Dixon,andM.R.Vant. ApplicationOfRadonTransformTechniquesToWakeDetectio nInSeasat-ASAR Images. GeoscienceandRemoteSensing,IEEETransactionson ,28(4):553{560,1990. [7]D.R.ThompsonandJ.R.Jensen.Syntheticapertureradar interferometryapplied toship-generatedinternalwavesinthe1989LochLinnheexp eriment. Journalof GeophysicalResearch ,98(C6):10259{10270,1993. [8]L.M.Murphy.Linearfeaturedetectionandenhancementi nnoisyimagesviathe Radontransform. PatternRecognitionLetters ,4(4):279{284,1986. [9]D.Gibbins,DAGray,andD.Dempsey.Classifyingshipsus inglowresolutionmaritime radar. SignalProcessingandItsApplications,1999.ISSPA'99.Pr oceedingsoftheFifth InternationalSymposiumon ,1,1999. [10]F.LesageandL.Gagnon.Experimentinglevelset-based snakesforcontoursegmentationinradarimagery. ProceedingsofSPIE ,4041:154,2000. [11]P.Valin,Y.Tessier,andA.Jouan.Hierarchicalshipcl assierforairbornesynthetic apertureradar(SAR)images. Signals,Systems,andComputers,1999.Conference RecordoftheThirty-ThirdAsilomarConferenceon ,2,1999. [12]C.YuanandD.Casasent.Compositeltersforinversesy ntheticapertureradar classicationofsmallships. OpticalEngineering ,41:94,2002. 60

PAGE 69

[13]V.GouaillierandL.Gagnon.Shipsilhouetterecogniti onusingprincipalcomponents analysis. Proc.SPIEConf.onApplicationsofDigitalImageProcessin gXX ,3164:59{ 69. [14]C.Alippi.Real-timeanalysisofshipsinradarimagesw ithneuralnetworks. Pattern Recognition ,28(12):1899{1913,1995. [15]Q.Luo,TMKhoshgoftaar,andA.Folleco.Classication ofShipsinSurveillance Video. InformationReuseandIntegration,2006IEEEInternationa lConferenceon pages432{437,2006. [16]S.Felatyev,D.B.Goldgof,andL.Langebrake.Towards detectionofmarinevehicles onhorizonfrombuoycamera. ProceedingsofSPIE ,6736:67360O,2007. [17]S.Todorovic. StatisticalModelingandSegmentationofSky/GroundImage s .PhD thesis,Masterthesis,UniversityofFlorida,2002. [18]S.M.Ettinger,M.C.Nechyba,P.G.Ifju,andM.Waszak.V ision-guidedrightstability andcontrolformicroairvehicles. AdvancedRobotics ,17(7):617{640,2003. [19]S.Todorovic,MCNechyba,andPGIfju.Sky/groundmodel ingforautonomousMAV right. RoboticsandAutomation,2003.Proceedings.ICRA'03.IEEE International Conferenceon ,1,2003. [20]S.TodorovicandMCNechyba.Avisionsystemforintelli gentmissionprolesofmicro airvehicles. VehicularTechnology,IEEETransactionson ,53(6):1713{1725,2004. [21]T.CornallandG.Egan.MeasuringHorizonAnglefromVid eoonaSmallUnmanned AirVehicle,2004. [22]T.CornallandG.Egan.CalculatingAttitudefromHoriz onVision. EleventhAustralianInternationalAerospaceCongress,Melbourne ,2005. [23]G.Q.Bao,S.S.Xiong,andZ.Y.Zhou.Vision-BasedHoriz onExtractionforMicroAir VehicleFlightControl. InstrumentationandMeasurement,IEEETransactionson 54(3):1067{1072,2005. [24]TGMcGee,R.Sengupta,andK.Hedrick.ObstacleDetecti onforSmallAutonomous AircraftUsingSkySegmentation. RoboticsandAutomation,2005.Proceedingsofthe 2005IEEEInternationalConferenceon ,pages4679{4684,2005. [25]S.Felatyev,V.Smarodzinava,L.O.Hall,andD.B.Gold gof.HorizonDetectionUsing MachineLearningTechniques. MachineLearningandApplications,2006.ICMLA'06. 5thInternationalConferenceon ,pages17{21,2006. [26]L.Fety,M.Terre,andX.Noreve.Imageprocessingforth edetectionofthehorizon anddevicefortheimplementionthereof. UnitedStatesPatent5,214,720,Thompson TRTDefense ,1991. [27]B.Taylor,C.Bil,S.Watkins,andG.Egan.HorizonSensi ngAttitudeStabilisation: AVMCAutopilot. InternationalUAVSystemsConference,Bristol,UK ,2003. 61

PAGE 70

[28]M.Sonka,V.Hlavac,andR.Boyle.ImageProcessing,Ana lysis,andMachineVision. 2007. [29]MCShin,DBGoldgof,andKWBowyer.ComparisonofEdgeDe tectorPerformance throughUseinanObjectRecognitionTask. ComputerVisionandImageUnderstanding ,84(1):160{178,2001. [30]M.Shin,D.Goldgof,andKWBowyer.Anobjectivecompari sonmethodologyofedge detectionalgorithmsusingastructurefrommotiontask. ProceedingsofCVPR ,pages 190{195. [31]J.Canny.Acomputationalapproachtoedgedetection. IEEETransactionsonPattern AnalysisandMachineIntelligence ,8(6):679{698,1986. [32]R.C.GonzalezandR.E.Woods.DigitalImageProcessing [33]R.Jain,R.Kasturi,andB.G.Schunck. Machinevision .McGraw-Hill,Inc.NewYork, NY,USA,1995. [34]G.WelchandG.Bishop.AnIntroductiontotheKalmanFil ter. ACMSIGGRAPH 2001CourseNotes ,2001. [35]E.Cuevas,D.Zaldivar,andR.Rojas.Kalmanlterforvi siontracking. [36]LCLangebrake,CELembke,RHWeisberg,RHByrne,DRRuss ell,G.Tilbury,and R.Carr.Designandinitialresultsofabottomstationingoc eanproler. Oceans'02 MTS/IEEE ,1,2002. [37]S.J.Pratt.Submarineperiscopesystems,July181989. USPatent4,848,886. [38]D.Reid.Analgorithmfortrackingmultipletargets. AutomaticControl,IEEETransactionson ,24(6):843{854,1979. [39]I.J.Cox,S.L.Hingorani,etal.Anecientimplementat ionofReid'smultiplehypothesistrackingalgorithmanditsevaluationforthepurposeo fvisualtracking. IEEE TransactionsonPatternAnalysisandMachineIntelligence ,18(2):138{150,1996. [40]D.DoermannandD.Mihalcik.Toolsandtechniquesforvi deoperformanceevaluation. PatternRecognition,2000.Proceedings.15thInternation alConferenceon ,4,2000. [41]RKasturi,D.Goldgof,P.Soundararajan,V.Manohar,JG arofolo,B.Bowers, M.Boonstra,V.Korzhova,andJ.Zhang.FrameworkforPerfor manceEvaluation ofFace,Text,andVehicleDetectionandTrackinginVideo:D ata,Metrics,and Protocol. IEEETransactionsonPatternAnalysisandMachineIntellig ence ,page doi.ieeecomputersociety.org/10.1109/TPAMI.2008.57. [42]C.J.C.Burges.ATutorialonSupportVectorMachinesfo rPatternRecognition. Data MiningandKnowledgeDiscovery ,2(2):121{167,1998. [43]N.LittlestoneandMKWarmuth.Theweightedmajorityal gorithm. Foundationsof ComputerScience,30thAnnualSymposiumon ,pages256{261. 62

PAGE 71

[44]NCRowe,LLGrewe,U.S.N.P.Sch,andCAMonterey.Change detectionforlinear featuresinaerialphotographsusingedge-nding. GeoscienceandRemoteSensing, IEEETransactionson ,39(7):1608{1612,2001. [45]D.D.Morris,B.R.Colonna,andF.D.Snyder.Image-base dmotionstabilizationfor maritimesurveillance. ProceedingsofSPIE ,6497:64970F,2007. 63


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 002021425
003 fts
005 20090729113638.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 090729s2008 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0002609
035
(OCoLC)428424441
040
FHM
c FHM
049
FHMM
090
QC76 (Online)
1 100
Fefilatyev, Sergiy.
0 245
Detection of marine vehicles in images and video of open sea
h [electronic resource] /
by Sergiy Fefilatyev.
260
[Tampa, Fla] :
b University of South Florida,
2008.
500
Title from PDF of title page.
Document formatted into pages; contains 63 pages.
502
Thesis (M.S.C.S.)--University of South Florida, 2008.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
3 520
ABSTRACT: This work presents a new technique for automatic detection of marine vehicles in images and video of open sea. Users of such system include border guards, military, port safety, flow management, and sanctuary protection personnel. The source of images and video is a digital camera or a camcorder which is placed on a buoy or stationary mounted in a harbor facility. The system is intended to work autonomously, taking images of the surrounding ocean surface and analyzing them for the presence of marine vehicles. The goal of the system is to detect an approximate window around the ship. The proposed computer vision-based algorithm combines a horizon detection method with edge detection and postprocessing. Several datasets of still images are used to evaluate the performance of the proposed technique. For video sequences the original algorithm is further enhanced with a tracking algorithm that uses Kalman filter. A separate dataset of 30 video sequences 10 seconds each is used to test its performance. Promising results of the detection of ships are discussed and necessary improvements for achieving better performance are suggested.
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
590
Advisor: Dmitry B. Goldgof, Ph.D.
653
Ship detection
Tracking
Horizon detection
Computer vision
Buoy camera
690
Dissertations, Academic
z USF
x Computer Science
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.2609