USF Libraries
USF Digital Collections

Blogs

MISSING IMAGE

Material Information

Title:
Blogs balanced local and global search for non-degenerate two view epipolar geometry
Physical Description:
Book
Language:
English
Creator:
Brahmachari, Aveek Shankar
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Similarity
Joint feature distributions
Jump diffusion
Degeneracy
Epipolar geometry
Dissertations, Academic -- Computer Science -- Masters -- USF   ( lcsh )
Genre:
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: The problem of epipolar geometry estimation together with correspondence establishment in case of wide baseline and large scale changes and rotation has been addressed in this work. This work deals with cases that are heavily noised by outliers. The jump diffusion MCMC method has been employed to search for the non-degenerate epipolar geometry with the highest probabilistic support of putative correspondences. At the same time, inliers in the putative set are also identified. The jump steps involve large movements guided by a distribution of similarity based priors while diffusion steps are small movements guided by a distribution of likelihoods given by the Joint Feature Distribution (JFD). The 'best so far' samples are accepted in accordance to Metropolis-Hastings method.The diffusion steps are carried out by sampling conditioned on the 'best so far', making it local to the 'best so far' sample, while jump steps remain unconditioned and span across the correspondence and motion space according to a similarity based proposal distribution making large movements. We advance the theory in three novel ways. First, a similarity based prior proposal distribution which guide jump steps. Second, JFD based likelihoods which guide diffusion steps allowing more focused correspondence establishment while searching for epipolar geometry. Third, a measure of degeneracy that allows to rule out degenerate configurations. The jump diffusion framework thus defined allows handling over 90% outliers even in cases where the number of inliers is very few. Practically, the advancement lies in higher precision and accuracy that has been detailed in this work by comparisons.In this work, BLOGS is compared with LO-RANSAC, NAPSAC, MAPSAC and BEEM algorithm, which are the current state of the art competing methods, on a dataset that has significantly more change in baseline, rotation, and scale than those used in the state of the art. Performance of these algorithms and BLOGS are quantitatively benchmark for a comparison by estimating the error in the epipolar geometry given by root mean Sampson's distance from manually specified corresponding point pairs which serve as a ground truth. Not just is BLOGS able to tolerate very high outlier rates, but also gives result of similar quality in 10 times lesser number of iterations than the most competitive among the compared algorithms.
Thesis:
Thesis (M.S.C.S.)--University of South Florida, 2009.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Aveek Shankar Brahmachari.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 46 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 002064191
oclc - 567793654
usfldc doi - E14-SFE0003084
usfldc handle - e14.3084
System ID:
SFS0027400:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200385Ka 4500
controlfield tag 001 002064191
005 20100323161826.0
007 cr mnu|||uuuuu
008 100323s2009 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003084
035
(OCoLC)567793654
040
FHM
c FHM
049
FHMM
090
QA76 (Online)
1 100
Brahmachari, Aveek Shankar.
0 245
Blogs :
b balanced local and global search for non-degenerate two view epipolar geometry
h [electronic resource] /
by Aveek Shankar Brahmachari.
260
[Tampa, Fla] :
University of South Florida,
2009.
500
Title from PDF of title page.
Document formatted into pages; contains 46 pages.
502
Thesis (M.S.C.S.)--University of South Florida, 2009.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
520
ABSTRACT: The problem of epipolar geometry estimation together with correspondence establishment in case of wide baseline and large scale changes and rotation has been addressed in this work. This work deals with cases that are heavily noised by outliers. The jump diffusion MCMC method has been employed to search for the non-degenerate epipolar geometry with the highest probabilistic support of putative correspondences. At the same time, inliers in the putative set are also identified. The jump steps involve large movements guided by a distribution of similarity based priors while diffusion steps are small movements guided by a distribution of likelihoods given by the Joint Feature Distribution (JFD). The 'best so far' samples are accepted in accordance to Metropolis-Hastings method.The diffusion steps are carried out by sampling conditioned on the 'best so far', making it local to the 'best so far' sample, while jump steps remain unconditioned and span across the correspondence and motion space according to a similarity based proposal distribution making large movements. We advance the theory in three novel ways. First, a similarity based prior proposal distribution which guide jump steps. Second, JFD based likelihoods which guide diffusion steps allowing more focused correspondence establishment while searching for epipolar geometry. Third, a measure of degeneracy that allows to rule out degenerate configurations. The jump diffusion framework thus defined allows handling over 90% outliers even in cases where the number of inliers is very few. Practically, the advancement lies in higher precision and accuracy that has been detailed in this work by comparisons.In this work, BLOGS is compared with LO-RANSAC, NAPSAC, MAPSAC and BEEM algorithm, which are the current state of the art competing methods, on a dataset that has significantly more change in baseline, rotation, and scale than those used in the state of the art. Performance of these algorithms and BLOGS are quantitatively benchmark for a comparison by estimating the error in the epipolar geometry given by root mean Sampson's distance from manually specified corresponding point pairs which serve as a ground truth. Not just is BLOGS able to tolerate very high outlier rates, but also gives result of similar quality in 10 times lesser number of iterations than the most competitive among the compared algorithms.
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
590
Advisor: Sudeep Sarkar, Ph.D.
653
Similarity
Joint feature distributions
Jump diffusion
Degeneracy
Epipolar geometry
690
Dissertations, Academic
z USF
x Computer Science
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.3084



PAGE 1

BLOGS:BalancedLocalandGlobalSearchforNon-Degenerate TwoViewEpipolarGeometry by AveekShankarBrahmachari Athesissubmittedinpartialful“llment oftherequirementsforthedegreeof MasterofScienceinComputerScience DepartmentofComputerScienceandEngineering CollegeofEngineering UniversityofSouthFlorida MajorProfessor:SudeepSarkar,Ph.D. RangacharKasturi,Ph.D. DmitryGoldgof,Ph.D. DateofApproval: June12,2009 Keywords:Similarity,JointFeatureDistributions,JumpDiusion,Degeneracy,Epipolar Geometry c Copyright2009,AveekShankarBrahmachari

PAGE 2

DEDICATION Tomylovingfamily

PAGE 3

ACKNOWLEDGEMENTS IwouldliketoexpressmysincerethankstoProf.SudeepSarkarforgivingmean opportunitytoworkwithhimonsuchatopicofbroadinterestincomputervision.The courseshehastaughtandtheresearchadvicehehasdeliveredwouldcertainlyhavea strongimpactonmeforyearstocome.OveraperiodoftwoyearsIhavealwaysfelt superblyguidedandadvisedbyhim.His”owsofthoughtshavealwaystriggeredmine. IamgratefultoProf.RangacharKasturiandProf.DmitryGoldgofforinstillingan insightintocomputervisionrelatedsubjectsthroughtheirwonderfulcourses.Iamalso thankfultothemforbeingmycommitteemembers.Wideknowledge,downtoearthand compassionatenatureofProf.Kasturi,Prof.GoldgofandProf.Sarkarhasalwaysdrawn myimmenserespect. IwanttothankVasantManoharforhistechnicalhelpwheneverneededandforalarmingmewithinformationsattherighttime.Iwouldalsothankthetechnicalsupportteam ofthedepartmentfortheirpromptactions.Iamthankfultoallmyfriendswhosepresence madelifeandworkeasier. Finally,Iwouldwanttoexpressmydeepestgratitudetomyparents,mysisterandmy “anceforboththecoolshadeandwarmthoftheirloveandaection.Ithankthemfor standing“rmlybymeandencouragingmeinmytechnicalaspirations.

PAGE 4

TABLEOFCONTENTS LISTOFTABLESiii LISTOFFIGURESiv ABSTRACT v CHAPTER1INTRODUCTION1 1.1CorrespondenceandEpipolarGeometry2 1.2Motivation4 1.3OverviewofOurApproach5 CHAPTER2ROBUSTSTATISTICSANDEPIPOLARGEOMETRY7 2.1RANSAC7 2.2EpipolarGeometryModelGeneration8 2.2.1Normalized8-pointAlgorithm8 2.2.27-pointAlgorithm9 2.3ModelQualityEstimation10 2.3.1SampsonsDistance10 2.4Degeneracy10 2.5StoppingCriterion10 2.6LearningtheEpipolarGeometry11 CHAPTER3STATEOFTHEART12 3.1LeastMedianofSquares12 3.2M-estimatorandpbM-estimator12 3.3MINPRAN13 3.4NAPSAC13 3.5MLESAC,GuidedMLESAC,MAPSAC14 3.6LO-RANSAC15 3.7DEGENSAC15 3.8PROSAC16 3.9WMMbasedGuidanceforEpipolarGeometryEstimation16 3.10BEEM17 CHAPTER4BLOGS:BALANCEDLOCALANDGLOBALSEARCH19 4.1TheProblemDe“nition:OurModel19 4.2JointFeatureDistributions22 4.3TheAlgorithm23 4.3.1QualityMeasureoftheCorrespondel25 4.3.2DegeneracyMeasure25 4.3.3JumpDiusion27 i

PAGE 5

4.3.3.1JumpusingSimilarity-basedProposalDistribution28 4.3.3.2DiusionusingJFD-basedProposalDistribution28 4.3.4AcceptanceofSample29 CHAPTER5EXTENSION30 5.1ExtensiontoMultiplePutativeSets30 CHAPTER6EXPERIMENTS32 6.1TestData32 6.2PerformanceMeasure33 6.3Results33 CHAPTER7DISCUSSIONANDCONCLUSION39 REFERENCES41 ii

PAGE 6

LISTOFTABLES Table6.1ComparativeperformanceanalysisofLO-RANSAC,NAPSAC,MAPSAC,BEEMandBLOGS(ourmethod).34 iii

PAGE 7

LISTOFFIGURES Figure2.1Figureshowingthe2Dpointsu,v,3DpointX,opticalcentersO, O,epipolarlinesl,l,epipolese,eandepipolarplane.8 Figure4.1Illustrationofrepresentationofcorrespondencesasavector.19 Figure4.2Similaritymatrix S fromwhichbestmutualmatchesarefound,brighter onesbeingbettermatches.21 Figure4.3Imagesshowinghighprobabilitycorrespondencesearchregionsgiven byJFDs.24 Figure4.4Flowchartofthesamplingprocessusedto“ndthebestsetofcorrespondences.26 Figure6.1ComputedepipolarlineonKmsm,CorridorandBlunaimagepairs.35 Figure6.2ComputedepipolarlineonFlags,SteelMeshandPillarimagepairs.36 Figure6.3ComputedepipolarlinesonBuilding,UniversityandStonesimage pairs.37 Figure6.4ComputedepipolarlinesonParking,CafeteriaandCarsimagepairs.38 iv

PAGE 8

BLOGS:BalancedLocalandGlobalSearchforNon-Degenerate TwoViewEpipolarGeometry AveekShankarBrahmachari ABSTRACT Theproblemofepipolargeometryestimationtogetherwithcorrespondenceestablishmentincaseofwidebaselineandlargescalechangesandrotationhasbeenaddressed inthiswork.Thisworkdealswithcasesthatareheavilynoisedbyoutliers.Thejump diusionMCMCmethodhasbeenemployedtosearchforthenon-degenerateepipolargeometrywiththehighestprobabilisticsupportofputativecorrespondences.Atthesame time,inliersintheputativesetarealsoidenti“ed.Thejumpstepsinvolvelargemovementsguidedbyadistributionofsimilaritybasedpriorswhilediusionstepsaresmall movementsguidedbyadistributionoflikelihoodsgivenbytheJointFeatureDistribution (JFD)[64].ThebestsofarsamplesareacceptedinaccordancetoMetropolis-Hastings method.Thediusionstepsarecarriedoutbysamplingconditionedonthebestsofar, makingitlocaltothebestsofarsample,whilejumpstepsremainunconditionedand spanacrossthecorrespondenceandmotionspaceaccordingtoasimilaritybasedproposal distributionmakinglargemovements.Weadvancethetheoryinthreenovelways.First, asimilaritybasedpriorproposaldistributionwhichguidejumpsteps.Second,JFDbased likelihoodswhichguidediusionstepsallowingmorefocusedcorrespondenceestablishment whilesearchingforepipolargeometry.Third,ameasureofdegeneracythatallowstorule outdegeneratecon“gurations.Thejumpdiusionframeworkthusde“nedallowshandling over90%outliersevenincaseswherethenumberofinliersisveryfew.Practically,the advancementliesinhigherprecisionandaccuracythathasbeendetailedinthisworkby comparisons.Inthiswork,BLOGSiscomparedwithLO-RANSAC[10],NAPSAC[40], v

PAGE 9

MAPSAC[59]andBEEM[21]algorithm,whicharethecurrentstateoftheartcompeting methods,onadatasetthathassigni“cantlymorechangeinbaseline,rotation,andscale thanthoseusedinthestateoftheart.PerformanceofthesealgorithmsandBLOGSare quantitativelybenchmarkforacomparisonbyestimatingtheerrorintheepipolargeometrygivenbyrootmeanSampsonsdistancefrommanuallyspeci“edcorrespondingpoint pairswhichserveasagroundtruth.NotjustisBLOGSabletotolerateveryhighoutlier rates,butalsogivesresultofsimilarqualityin10timeslessernumberofiterationsthan themostcompetitiveamongthecomparedalgorithms. vi

PAGE 10

CHAPTER1 INTRODUCTION Overmanyyears,correspondenceestablishmentandepipolargeometryestimationhas beenresearchedandnowtheproblemisexpandingitshorizontowiderbaseline,large scalechangesandrotation.However,theproblemstilldemandsalotofresearch.Currentresearchispushingthelimitofminimalpossibleinformationsharedbyanimagepair neededtocomeupwithmeaningful3Dstructureestimateofthescene.Threethingsare importantforepipolargeometryestimation:correspondencegeometry,thecamerageometryandthescenegeometry.Indeed,theobjectiveistoknowthemotionofthecameras andthe3Dscenestructure.Correspondencesarerequiredtoestimatestructureandmotion,andknowledgeofstructureandmotioncanhelpestablishcorrespondences.Thus, coupledupdateapproachesformanaturalsolutiontothisproblem.Consequently,startingwithoutpriorknowledgeofthefeaturecorrespondences,featuredistancesorfeature correlationsarefoundtocomeupwithasimilarityorcon“dencemeasurebasedputative featurecorrespondencestobootstrapthesearchofcorrespondenceandepipolargeometry.Amongfeatures,SIFT[31]andGLOH[37]pointfeatureshaverecentlybecomevery popularalmostreplacingHarriscorners[25]. Epipolarconstraintisthestrongestconstraintonthesearchforepipolargeometry, althoughweaknessoftheepipolarconstraintliesinthefactthatitdoesnotdiscernbetweencorrespondencesalongtheepipolarline.Othercommonconstraintsareuniqueness, similarityandproximity.Epipolarconstrainttogetherwithappearancebasedsimilarity betweenfeaturesisthusmostoftenused.Further,deviancefromtheepipolarconstraint isthenegativeloglikelihoodofcorrectnessofeitherthemotionestimateorthecorrespondencesinconsideration[62].Wecanre“necameraandscenegeometrybyminimizingthere-projectionerrorsfortheinferred3Dscenegeometry.Bundleadjustmentisa 1

PAGE 11

non-lineartechniquetominimizethere-projectionerroriterativelybyre“ningthemotion estimates,the3Dscenestructureandtheintrinsiccameracalibrationparameters,allat thesametime.Re-projectionerrorgivesthenegativeloglikelihoodofcorrectnessof3D scenestructure,motionparametersandintrinsiccameracalibrationparameterstakentogether[12,7,34,65].Again,agoodestimateoftheepipolargeometrycouldeectively bootstrapthisiterativeestimationprocess.Thus,estimationofepipolargeometry,ascapturedbythefundamentalmatrix,iscentraltomotionandstructureestimation.Current researchworksonthisproblemconsidersituationswithwidebaseline,whichresultin signi“cantamountoffeaturesinthescenewithnocorrespondence. Fromhereon,thestructureofthethesisisasfollows.Section1.1,1.2givesabrief overviewoftheresearchworkspertainingtocorrespondenceandepipolargeometryestimationandourmotivationrespectively.Section1.3givesabriefdiscourseofthenovelty inourapproach.Chapter2ingeneraltalksabouttheRANSACrobustestimationalgorithminsection2.1anddetailsissuesrelatedtoepipolargeometryestimationalgorithm, thataremodelgeneration,modelqualityestimation,degeneracy,stoppingcriterionand learninginsection2.2,2.3,2.4,2.5,2.6respectively.Chapter3laysforththecentralidea ofvariousstateoftheartalgorithms.Chapter4startswithdescribingtheproblemmodel insection4.1,theJointFeatureDistributioninsection4.2,the”owchartoftheBLOGS algorithminsection4.3followedbydetailsofourqualitymeasureinsection4.3.1,proposed degeneracymeasureinsection4.3.2,thejump-diusionmethodinsection4.3.3andacceptanceofthesamplein4.3.4.Chapter5talksaboutourextensiontomultipleputative sets.Chapter6talksaboutourtestdata,experimentsandresultsinsection6.1,6.2,6.3. Chapter7endsupwithdiscussionandconclusion. 1.1CorrespondenceandEpipolarGeometry Therearesigni“cantamountofworksonfundamentalmatrixestimationandstructure frommotion[27,28,43,44,32,24].Shapiro etal. [51]proposedaclassiceigenspace approachtothecorrespondenceproblem.Salvi etal. [1]comparestheperformanceoffundamentalmatrixestimationalgorithmsclassi“edaslinear,iterativeandrobustalgorithms. 2

PAGE 12

Amongthelinearalgorithms,sevenpointalgorithmandthenormalizedeightpointalgorithm[30,26]havebeenthemostpopular.Theselinearmethodsareusedinalmostall robustalgorithmstogetanestimateofthefundamentalmatrix. RobustalgorithmslikeRANSAC[15]havebeenverypopularforfundamentalmatrix estimation.Algorithmsinvolvingrandomsamplingandconsensusworkingeneralbyrandomlypickingupaminimalsetofcorrespondences,estimatingmotionforalargenumber ofsuchrandomselections,and“ndingthemotionthatbest“tstheentiresetofputative correspondences.InRANSAC,thebest“tisfoundasperthecardinalityofthesupportset ofthemotion.Otherrobustalgorithmshaveothercriterionofbest“t.Manysimilarrobustalgorithms[27]suchasLMedS,M-estimator[6],pbM-estimator[47],MINPRAN[52], NAPSAC[40],MLESAC[62],MAPSAC[59],Guided-MLESAC[56]haveevolvedoverthe years.PROSAC[8]isanothersuchalgorithmthatrandomlysamplesfromprogressively largersetsofcorrespondencesrankedinorderofhighertolowersimilarityscoresbetween SIFTfeatures.IMPSAC[60]proposedhierachicalmatching.Fewgoodalgorithmsreported thathavealsolaidtheepipolargeometryandcorrespondenceprobleminaprobabilistic frameworkare[4,13].Epipolargeometryestimationalgorithmsthathaveclaimedtosolve theepipolargeometryproblematveryhighoutlierratenearly90%are[39,21,20].Tensor voting[55]appliedtotheeipolargeometryproblemleadtoresolvingproblemswithas muchas60%outliers.Globaloptimalitywithfeasiblecomputationtosolvethecorrespondenceproblembetweensparsesetofpointswasclaimedin[33].OptimallyRandomized RANSAC[9]isanotherrecentpaperontheproblem. IthasbeenobservedthatRANSACneedsmoreiterationsthantheoreticallyexpected. ThisleadsustoLO-RANSAC[10]whichisinspiredbythefactthatasetofcorrespondencesuncontaminatedbyoutliersmightnotleadtoacorrectepipolargeometry,thus increasingthenumberofiterations.Thisisbecausemanyofthecorrespondencesinthe uncontaminatedsetmightlieonthesameplane.Suchplanesarecalleddegenerateplanes orcriticalsurfaces.Chum etal. [11]cameupwithagoodmethodofdetectingplanardegeneracyandestimatingtheepipolargeometryinpresenceofadominantplane.Polleyfeys etal. [17]alsoaddressedtheproblemofdegeneracy.Otherresearchworksadressingdegeneracyare[63,61,46].Theepipolargeometryproblemhasalsobeenlookedatbyusing 3

PAGE 13

3correspondences[36,38]atatimeby“ndingmaximallystableextremalregions[35]. Weakmotionmodels[21]andBEEM[20]algorithmshavealsotakensimilarapproach. Recently,epipolargeometrywasestimatedusing2LAFcorrespondences[45].TheBEEM algorithmpaper[20]alsoproposesestimationofepipolargeometryusing2SIFTcorrespondences.ItusesthedominantangleofSIFTfeaturestoproduce4correspondences from1.Thisresultsin8correspondencesthatarefedtothenormalizedeightpointsalgorithmtogetanepipolargeometry.Thetheoreticallyminimumpointsrequiredare5. The5-pointalgorithmproposedbyNister etal. [42]hasbeenverypopularforneeding minimalpointcorrespondences.Otherrelatedworksonstructureandmotioncanbefound in[49,58,19,53,29,57,41,18,16,3,5]. 1.2Motivation TheapproachadoptedinthisworkissimilarinschoolofthoughtastheMaximum LikelihoodEstimationSAmplingConsensus(MLESAC)[62]approach.MLESACmodels theresidualerrordistributionofcorrespondencesinaputativesetofcorrespondences, givenamotionhypothesis,basedonanassumedsetofcorrespondences,asamixtureof Gaussianinliererrordistributionanduniformoutliererrordistribution.Theseconditional probabilitiesduetoindividualresidualerrorsareassumedtobeindependent.Theproduct ofalltheconditionalprobabilitiesleadstoameasureoflikelihoodofthecorrespondence setgiventhemotionhypothesis.Foreachmotionhypothesis,maximumloglikelihood andtheprobabilityofvalidityofmatchesthatmaximizestheloglikelihoodarefoundby expectationmaximization.MLESAClooksforthemotionhypothesisthatmaximizesthe likelihoodoftheputativecorrespondenceset.MAPSAC[59]andGuided-MLESAC[56] aretwopopularvariantsofMLESAC.MAPSACistheBayesianversionofMLESACwhich improvesuponitbymaximizingtheaposterioriprobabilityinsteadoflikelihood.GuidedMLESACextendsonMLESACbyusingpriorknowledgeofvalidityofcorrespondences. TherearethreeaspectsofMLESAC-schoolofapproachthatformthebackground againstwhichweadvancethestateoftheart.First,isrelatedtothemodelsusedforinlier andoutliercorrespondences.Whiletheinliererrordistributioncanbequitecon“dently 4

PAGE 14

modeledasGaussian,assumingthatoutliererrorsexhibituniformdistributionisarguable. Thenatureofnoisemightbequitestructuredsuchasinthepresenceofrepeatedpattern. Second,MLESACdoesnotassumeanypriorknowledgeoftheprobabilityofvalidityof acorrespondencematchandallmatchesaregiventhesameprobabilityofvalidityfor asinglehypothesis.Third,inliersareassumedtobemutuallyindependent,butmutual independenceoftheinliersmightbeanotbeacorrectassumption.Ouralgorithmseeksto improveontheseprobableshortcomingsofMLESACandreturnnon-degenerateepipolar geometry. 1.3OverviewofOurApproach Letussaythatsearchesarebroadlyoftwotypes:globalandlocal.Theglobalsearch isdonebyjumpstepsandlocalsearchisdonebydiusionsteps.Weseektomaximizea costfunctionbyrandomglobalandlocalsearches.Theglobaljumpsearchhelpsusarrive atdierentparametersandlocaldiusionsearchesaredoneto“netunetheseparameters toseeifanearbysolutionisbetter.Inouralgorithm,globalsearchesaredoneusinga distributionofsimilaritybasedweightsandlocalsearchesaredoneusingTriggs[64]Joint FeatureDistributionwhichessentiallyimposestheepipolarconstraintinamuchuni“ed way.Werandomlychooseineachiterationwhetherto“ndasimilarityguidedsampleor aJFDguidedsample.JFDguidedsamplesaredrawnfromadistributionofconditional probabilitiesofputativecorrespondencesconditionedonbestknowncorrespondelsofar. Theminimalsetofcorrespondences,e.g.8correspondences,neededforepipolargeometry estimationisreferredtointhisworkasacorrespondel.Thus,ourguidancestrategy necessarilyfollowsaMonteCarloMarkovChain.WeemploytheMetropolis-Hastings MCMCmethodinouralgorithm. LikeMLESAC,MAPSACandGuided-MLESAC,werandomlysamplefromaprobabilitydistribution,butunlikethem,wedonotcharacterizetheoutlierandinliererrordistributionseparately.Wechoosetodoconditionalcharacterizationoftheprobabilityspace ofcorrespondencebyusingTriggsJointFeatureDistributionandatthesametimeletting itbothcompeteandbene“tfromsimilarityguidedsamples.Themotivationofconditional 5

PAGE 15

characterizationcomesfromtheideaofconditionaldependenceofcorrespondenceswhich alsosubsumeslocalityofvalidcorrespondencesasinNAPSAC[40].NAPSAC(N-Adjacent PointsrandomSAmplingConsensus)isinspiredbytheideaoflocalityofvalidcorrespondences.Seekinglocalityinvalidcorrespondencesmightleadustocorrespondenceson criticalsurfacesordegenerateplanes.Instead,weusedameasureofnon-degeneracyofa correspondel(8correspondencepairs). Qualityofafundamentalmatrixisanotherimportantaspectthatneedstobequanti“ed.Cardinalityofsupportsethasbeenapopularmeasureforthis.However,deciding uponthethresholdoftheerrorisamajorproblem.Wemeasurethequalityofacorrespondelusingtheprobabilisticsupportgivenbythetotalsumofnegativeloglikelihoods. 6

PAGE 16

CHAPTER2 ROBUSTSTATISTICSANDEPIPOLARGEOMETRY Inthischapter,wediscusstherobuststatisticstechniqueRANSACthathasbeen themostrudimentaryapproachforepipolargeometryestimation.Allotherapproaches arederivedorsimilartoRANSAC.Moreover,alltheimportantfactorsthatin”uence theperformanceofstateoftheartepipolargeometryestimationalgorithmsarebrie”y introducedinthischapter.WebeginwithexplainingRANSAC. 2.1RANSAC RANSACstandsforRandomSampleConsensus.RANSACisoneofthemostfamous methodsemployedinrobuststatistics.ThestepsdoneinRANSACare: 1.Randomlydrawasampleofminimalsize s ofdatapointsneededtoformamodel hypothesis. 2.Generateamodelhypothesisfromthesample. 3.Verifythequalityofthemodelusingallthedatapoints. 4.Storethebestsofarmodelanditssupportset. 5.Repeatsteps1through4for initerations. 6.Bestsofarmodelanditssupportsetarereturnedasresult. HowmanyiterationsdoesinRANSACneedtodetectthemodelcorrespondingtothe inlierdata?Letusseehowthenumberofiterationsisderivedbyprobabilistically.Let pobetheprobabilitythatasampledcorrespondenceisanoutlier.Let pdbetheprobability ofgettingade-noisedminimalsamplesetofcorrespondencesand s bethenumberof correspondencesinminimalsampleset.Given po, pdand s ,wecan“nd in,thenumberof iterationsrequiredtoattainasuccessrateof pdwherethesamplesarenoisedby pooutlier 7

PAGE 17

OOee' l l' u v XEpipolarPlaneFigure2.1.Figureshowingthe2Dpointsu,v,3DpointX,opticalcentersO,O, epipolarlinesl,l,epipolese,eandepipolarplane. rate.Probabilityofnotgetting s -tupleofinliersatleastoncein insamplesof s -tuplesis (1 Š (1 Š po)s)in=1 Š pd. ThenumberofiterationsneededbyRANSACisthusgivenby: in= log(1 Š pd) log(1 Š (1 Š po)s) (2.1) 2.2EpipolarGeometryModelGeneration 2.2.1Normalized8-pointAlgorithm Writethe3 3matrix F intheform f =[ f11,f12,f13,f21,f22,f23,f31,f32,f33]T.Let thehomogeneouscoordinatesin u =[ u1,u2, 1]Tand v =[ v1,v2, 1]T. x = v u =[ v1u1,v1u2,v1,v2u1,v2u2,v2,u1,u2, 1](2.2) A =[ x1,x2, xn]T(2.3) 8

PAGE 18

A isa2 n 9matrix.Minimumvalueof n is8.Thepointsneedtobenormalized beforeapplyingthealgorithm.So,itiscalledanormalizedeightpointalgorithm.The normalizationisdonebytranslationandscaling.Translationisdonebytranslatingthe centroidofthepointstotheoriginofthecoordinatesandscalingisdonesuchthatthe RMSvaluesofthedistancesofthepointsfromtheoriginis 2. Wewant Af =0.So,weneedtominimize Af .Itiswellknowninlinearalgebrathat the f thatminimizes Af isgivenbytheeigenvectorcorrespondingtotheleasteigenvalue of ATA Oneotherpropertyofthefundamentalmatrixisthatitissingularandofrank2. Thus,arank2constraintneedstobeimposedon f vector.Thisisdonebyreshaping f to3 3matrix F ,followedbyaneigen-decompositionandreconstructionusingthetwo mostsigni“canteigenvectorswithlargesteigenvalues. 2.2.27-pointAlgorithm Fundamentalmatrixhas7degreesoffreedom.Thus,7pointsaresucienttogeta fundamentalmatrixbutthesolutionisnon-linear.Notethatthemethodislinearinusing theDirectLinearTransform,butnon-linearissolvingforpolynomialindegreethree. Let F1and F2begivenbyreshapingtheeigenvectorscorrespondingtothetwolowest eigenvaluesin ATA F = F1+ F2(2.4) Therank2constraintisimposedas det F = a33+ a22+ a1 + a0=0(2.5) Theaboveequationisapolynomialofdegree3,so3valuesof give3fundamental matrices.Ignoringthecomplexsolutionsmightleadtoonly1fundamentalmatrix.If auniquesolutionisnotobtained,morepointsareneededto“ndthetruefundamental matrix.Allthesolutionsareingeneralusedtogeneratemodelhypothesis. 9

PAGE 19

2.3ModelQualityEstimation Themostimportantaspectoftheepipolargeometryestimationalgorithmsishowthe algorithmestimatesthequalityofthemodelhypothesis.Fewmethodsare: 1.MaximumcardinalityofsupportsetoutofallputativesetsasinRANSAC,LMedS, MINPRAN. 2.MaximumlikelihoodoftheputativecorrespondencesasinMLESAC. 3.MaximumaposteriorioftheputativecorrespondencesasinMAPSAC. 4.MaximumweightedsupportoftheentireputativesetasinM-estimators. 2.3.1SampsonsDistance TheSampsonsdistanceisameasureofthequalityofthemodelaswellasthecorrespondencewhicheverisjudged.TheSampsonsdistanceofthepoint x isgivenby: d ( x = v u F )=ni =1( vT iF ui)2 ( F ui)2 1+( F ui)2 2+( vT iF )2 1+( vT iF )2 2(2.6) 2.4Degeneracy Degeneracyleadstolossofinformationleadingtounstableestimates.Iftwopointsare requiredforestimatingalineandtheyarealmostcoinciding,aminorerrorinprecision andaccuracyofthelocationofthepointswouldmaketheestimateoftheequationof thelinejoiningthetwopointsveryunstable.Similarlyaplanecannotbedeterminedby threecollinearoralmostcollinearpoints.Thus,thesevenoreightpointsfedformodel generationtoasevenoreightpointalgorithmshouldcarrydistinctinformationaboutthe structureofthescene. 2.5StoppingCriterion Numberofiterationshasbeenthemostcommonstoppingcriterioninrandomorguided samplingalgorithms.ThepaperonRANSAC[15]cameupwithanestimateofthenumber ofiterationsthatwouldbeneededto“ndanepipolargeometrysupportedbyallinliers.It wasfoundin[10]thatthenumberofiterationsneededtogetgoodepipolargeometryis 10

PAGE 20

notsucient.Theothercommonstoppingcriterionisnumberofiterationspassedwithout updatingthebestsofar. 2.6LearningtheEpipolarGeometry Dellaert etal. [12]proposedMCMCguidanceforcorrespondenceandepipolargeometry estimation.Guided-MLESAC[56]alsoproposesMCMCguidance.MCMCguidanceisa methodtolearnfrompreviouslymadehypothesis,whichpropelsthealgorithminaguided fashion. 11

PAGE 21

CHAPTER3 STATEOFTHEART Inthischapter,tenofthemostpromisingepipolargeometryalgorithmshavebeen brie”yexplained.Anattempthasbeenmadetoorderthealgorithmsbothchronologically andbytheircentralideas.Thecoreideasofallthealgorithmsarebrie”ypointedout tohelpthereaderunderstandthestateoftheartwithlittlereading.FourofthealgorithmsexplainedherethatareLO-RANSAC,NAPSAC,MAPSACandBEEMhavebeen comparedinchapter6.LetusstartourdiscussionwithLMedS. 3.1LeastMedianofSquares IntheLMedSalgorithm,thequalityofthemodelisestimatedbythemedianof squarederrorsforeachdatapoint.Themodelthatminimizesthemedianofsquared errorsmin { medi{ ri 2}} where riistheerror,issoughtandoutput.Evidently,LMedSdoes nothandlemorethan50%outliers. 3.2M-estimatorandpbM-estimator M-estimator“ndstheweightedmeansquareerrorofallthedatapointsforeachmodel hypothesis.Themodelhypothesisforwhichtheweightedmeansquareerrorisminimum ischosenasbestsofarineachsamplingiteration.Weightederrorisnegativeloglikelihood,andM-estimatorslookformaximumlikelihood.Thus,M-estimatorsareMaximum likelihoodestimators.DierentweightsgivedierentM-estimators.So,inM-estimators weneedtosolveformin { n i =0 ( ri 2) } where riistheerrordistancefromthemodelof eachpointand istheweightfunctionoftheerror. 12

PAGE 22

pbM-estimatoris projectionbased M-estimatorwherethelikelihoodmaximizationproblemisgivenaformofkerneldensityestimator.pbM-estimatorcomesupwithaweight basedontheoptimalParzenwindowbandwidthforscalingtheerrorestimate. 3.3MINPRAN MINPRANstandsforMINimumProbabilityofRANdomness.AnimportantassumptioninMINPRANisthattheoutliererrordistributionisuniform.Minimumresidual errorforeachdatapointisfound.Theprobabilityof“ndingatleast k pointsamongtotal n pointssuchthattheyarewithinanerrordistanceof Š r/ + r fromthemodelhypothesis, giventhatalloutliersareuniformlydistributedbetween Š Z/ + Z isgivenby F ( r,k,n )=ni = k nCi( r Z0)i(1 Š r Z0)n Š i(3.1) Itcanbeeasilyseenthattheprobabilitywouldincreaseif r increasesanddecrease if k increases.ThisisanalogoustothemotivationofstandardRANSACwhichlooksfor maximuminliersataminimumthresholddistance.Iftheprobabilityisveryless,the distributionislesslikelytobeuniformandthemodelhypothesisisbetter. r and k that minimizestheprobabilityofrandomnessisfound. p min1 j S, 1 i NF ( rj,i,i,n )
PAGE 23

distributionofdatainhighdimensionalspaceforselectionofthehypothesismodel.This ishowNAPSACworkstogenerateahypothesismodel. 1.Selectaninitialpoint x0randomlyfromthesetofallpoints. 2.Findallpointswithinahyper-sphereofradius r with x0asthecenter. 3.Ifnumberofpointsfoundislessthantheminimumrequiredformodelgeneration, failure. 4.Include x0anduniformlysamplerestofminimalnumberofpointsfromallthepoints found. NAPSACisespeciallyforproblemsinhighdimensionalspacewhereeveniftheoutlier rateislow,itisdiculttogetanuncontaminatedsample.Thissamplingstrategyworks fastinthosecasesaswell.Ontheotherhand,lookingforproximityinpointsmightlead toselectionofadegeneratemodel. 3.5MLESAC,GuidedMLESAC,MAPSAC MAPSACistheBayesianversionofMLESAC.MLESACstandsforMaximumLikelihoodEstimateforSAmplingConsensus.MLESACmodelstheputativecorrespondence setasamixtureofoutliersandinliers,wheretheerrordistributionofoutliersisassumed uniformandthatofinliersisGaussian.ExpectationMaximizationisdonetocomeupwith aprobabilityofvalidityofcorrespondenceandalsothemaximumlikelihoodoftheentire putativesetgivenamodelhypothesis.Thehypothesisforwhichlikelihoodismaximum isaccepted.MAPSAClooksforamaximumaposterioriestimateratherthanmaximum likelihood. p ( ri| Mh)= 1 2 2 eŠ r i 2 2 2+ 1 w (1 Š p ( vi))(3.3) log( p ( R | Mh))=ni =1log 1 2 2 eŠ r i 2 2 2+ 1 w (1 Š p ( vi)) (3.4) Qh=maxp ( v )(log( p ( R | Mh)))(3.5) 14

PAGE 24

Equation3.3showstheprobabilityoftheresidualerrorofacorrespondencegivena modelhypothesis.Equation3.4assumesconditionalindependenceofthecorrespondences multipliesprobabilitiesofeachcorrespondenceandcomesupwithaprobabilityoftheentire putativesetandalogarithmoftheentireexpressionisdonetoarriveatlikelihood.Asin equation3.5,EMalgorithmisusedto“ndthemaximumlikelihoodandtheprobability ofvaliditythatmaximizesthelikelihood.Thesameprocessisrepeatedforallmodel hypothesesandthemodelthatgeneratesthemaximumlikelihoodisreturned. Guided-MLESACgetsridoftheexpectationmaximizationstepbyusingfeaturecorrelationbasedpriorsreplacingprobabilityofvalidityinMLESACsformulation.GuidedMLESACalsoextendstomultipleputativesets.MAPSACusesapriorwhichisnot appearancebased. 3.6LO-RANSAC LOinLO-RANSACstandsforLocallyOptimized.LO-RANSACismotivatedfroman observationthatasampleuncontaminatedbyoutliersmightnotleadtogoodmodel. Localoptimizationworksasfollows: 1.Letthelargestsupportsetfoundsofarbe Su. 2.Drawthenextsamplefrom Su. 3.Letthesamplesizebe min (Su 2, 14)incaseofepipolargeometryestimation.Estimation ofepipolargeometryusinglargersamplesizegivesmorestableestimates. 4.Usethisestimateto“ndallinliersandrepeatstep3untilnoimprovementisachieved. 5.Repeatstep2throughstep5 Iknumberoftimes. 6.Finallywegetstableepipolargeometryandastablesupportset. 7.Thelargeststablesupportsetandcorrespondingepipolargeometryisaccepted. 3.7DEGENSAC DEGENSAClooksfornon-degenerateepipolargeometry.Theseven-pointalgorithm isusedinDEGENSAC.21distincttripletscanbechosenfromthese7correspondences. Only5ofthemneedtobetestedforhomography.Theyare { 1,2,3 } { 4,5,6 } { 1,4,7 } 15

PAGE 25

{ 2,3,7 } { 5,6,7 } .No4pointsshouldlieonthesameplane.Theideaissimpleandeasyto implement. 3.8PROSAC PROSACstandsforProgressiveSamplingConsensus.PROSACusesthedistance ratiobetweentheclosestandsecondclosestSIFTfeaturesarrangedinanordertaking progressivelylargersetofputativecorrespondencesstartingfromminimalsamplesize. PROSACcomesupwithastrategytoincreasethenumberofcorrespondencesindescending orderofsimilarityscoreoneatatimeanddrawsanumberofsamplesfromtheprogressively growingsetsuchthatitmaintainsthesamesamplingdistributionasinsimpleRANSAC. ThoughPROSACexploitstherankorderoftheputativecorrespondence,itdoesnot exploitthesimilarityscore. 3.9WMMbasedGuidanceforEpipolarGeometryEstimation WeakMotionModelsareusedtoroughlyapproximatethemotionbetweentwoimages. Nw,WMMsarefoundandthegeometricdistanceofpointsfromtheseWMMsaremodeled asamixtureofGaussianinliererrordistributionanduniformoutliererrordistribution. WMMsarebasicallyanetransformationswhichrequire3correspondencestobede“ned. Anoutlineofthealgorithmisasfollows: 1.Generateanoutliersamplefromthecorrespondencesapartfromtheputativeset. 2.Assumedierentoutlierratesinset graduallyuptoauserspeci“edmaximum jwhere j isthenumberofoutlierratesspeci“ed. 3.GeneraterandomWMMsfortheoutlierrate. 4.Finetunetheestimateoftheoutlierrate,astheassumedonesarenotaccurate. 5.Estimateinlierprobabilities. 6.Estimatenumbersofiterations NsneededbyLO-RANSACstep. 7.Guidethealgorithmusingtheinlierprobabilitiesif Nsislessthanthemaximum speci“edbyuser,otherwisegotothenextoutlierratein 8.Ifthestoppingcriterionismet,stop. 16

PAGE 26

Ifthenumberofinliersindicatedbythemixturemodelisapproximatelysameasthe numberofinliersfoundbytheguidedLO-RANSACstep,thenstop. 3.10BEEM BalancedExplorationandExploitationModel(BEEM)searchforecientepipolargeometryestimationproposedfewnovelideas.BEEMapproachisclosesttoourapproachin havingalocalaswellasglobalsearch,butourapproachismoreuni“edandprobabilistic andsharesitsmotivationwithMAPSAC.IntheBEEMalgorithm,therearetwokinds ofexplorationandonekindofexploitation.Whileexplorationisbothlocalandglobal, exploitationisonlylocal.BEEMusesthedistanceratiobetweentheclosestandsecond closestSIFTfeaturetosamplefrom.Forratiosabove0.8assuggestedin[31],thedistributionisuniform.SamplingfromthisdistributionformsBEEMsglobalsearchstrategy. Theycallitglobalexploration.BEEMemploystwolocalsearchstrategies.BEEMalgorithmestimatesepipolargeometryusingonly2SIFTcorrespondences.Itmaintainsaset ofinliersandsamplesonecorrespondencefromtheinliersetandanothercorrespondence fromoutsidetheinliersetorinsidetheinliersetwithagivenprobabilityinordertoescape degeneracy.Thisiscalledlocalexploration.Anotherlocalsearchisthelocaloptimization stepasinLO-RANSACanditiscalledexploitation.Intheexplorationsteps,ifamodel withlargersupportsetisfound,thenextstepisexploitationandthereaftertheinliersetis maintainedasthebestsofar.Ifamodelwithlargersupportsetisnotfound,thequality ofthemodelisestimated.Thequalityofthemodelistheprobabilitythatthebestmodel foundsofarisnotanoutliermodel.Ifthestoppingcriterionisnotyetmetinthequality estimationstep,localexplorationischosenwiththeprobabilityfound,otherwiseglobal searchischosen.Thisprobabilityisalsousedtoescapedegeneracyinlocalexploration step.Thestoppingcriterionismetifthebestsofarhypothesisisnotupdatedfora numberofiterations. Whilewe“xtheprobabilityofchoosinglocalandglobalsearchesunbiasedinBLOGS, BEEMpreferstodynamicallyallocatethisprobabilitybythequalityofthemodelitis samplingateveryiteration.ThemajornovelapproachinBEEMistheabilitytocompute 17

PAGE 27

epipolargeometryusingjust2SIFTcorrespondences.BEEMexploitsthedominantangle informationinSIFTfeaturestocomeupwith3extrapointsperfeatureontheplane containingtheSIFTfeature.Asmentionedin[27],2distincthomograhiesinascenecan yieldanepipolargeometry.InBEEM,eightpointsthusfoundaresuppliedtoeightpoints algorithmtogetanestimateoftheepipolargeometry.Samplingjusttwocorrespondences togetamodelmakesBEEMextremelyfastcomparedtootheralgorithms. 18

PAGE 28

CHAPTER4 BLOGS:BALANCEDLOCALANDGLOBALSEARCH 4.1TheProblemDe“nition:OurModel Giventwoimagesofascene,withoutlossofgenerality,letusdenotetheimagewith smallernumberofdetectedfeaturestobe I1,containing a features: f1=[ q1, q2, qa]. Tothefeaturesetoftheotherimages, I2,addaNULLfeature r0suchthat f2= [ r0, r1, rb].Thecorrespondenceproblemistomap a featuresto b +1featuresbased ontheimagelocationandlocalphotometricattributes.Anynumberoffeaturesin f1can correspondtotheNULLfeaturein f2.Thismapping,ingeneral,isaNP-hardproblem. z1z2z3z4Camera position1 Camera position 2 q1q2q3r1r3r2C = 014 z5r4a = 3 b = 4 ObjectFocal plane 1Focal plane 2Figure4.1.Illustrationofrepresentationofcorrespondencesasavector. 19

PAGE 29

Howmanysuchdistinctmapsarepossible?ThenumberofNULLcorrespondences couldbefrom0to a .WeneedtochooseanyonenumbergivenbyaCi.Thenumberof non-NULLcorrespondenceswouldbenumberofpermutationsoftherestgivenbybPa Š i. Thus,numberofpossiblemapsisasummationoftheproductofthetwo. nmap=ai =0 aCibPa Š i(4.1) Onlyoneoutof nmapmappingsiscorrect.Forlargevaluesof a and b nmapwouldbe veryhigh.Combinatorialexplosionisverycostly.Thus,approximationalgorithmssuch asrandomsamplingbecomeanaturalsolution. Eachfeaturein f1and f2hasadescriptoroflength l andapositionoflength3in homogeneouscoordinatesystem.Letthedescriptorset D ofafeature f be D ( f )= [ d1( f ) ,d2( f ) ,.....,dl( f )].Wecan“ndthesimilaritybetweeneachfeatureof f1and f2using theirdescriptors.Let S bethesimilaritymatrixwith a rowsand b columns.Forvarious possiblesimilaritymeasuresthatcanbeused,thereaderisreferredto[48].Similarity constraintisobtainedfromthedescriptorsandepipolarconstraintisobtainedfromthe positions. Avector C ofsize a isusedtorepresentthemappingbetweenthetwofeaturesets.The nullmapsaredenotedby0andnon-nullmapsarevaluesfrom1to b .Thisisillustrated inFig.4.1.Ourobjectiveistocorrectlyassignthesevaluesto C andalsoestimatethe correctepipolargeometry. Weinitialize C inagreedyfashionwiththeindexofthehighestsimilarityvaluesin eachoftherowsof S .Ofthesemaps,thematchpairs { ( uk, vk) | k =1 ,n } thatexhibit highestsimilaritymeasurein S bothinitsrowandcolumn,areselectedtobetheputative correspondencesset X .Werepresenteachputativematchpairasa9componenttensor xk= vk uk.Thus, X =[ x1, x2,....., xn].Theseputativecorrespondencesformthe kernelfromwhichrestofthecorrespondenceswillbebuilt.Randommodelsaredrawn fromtheputativeset.Thosethatdonot“tintothemostconsistentmodelwillbemapped toNULL. 20

PAGE 30

Figure4.2.Similaritymatrix S fromwhichbestmutualmatchesarefound,brighterones beingbettermatches. Witheachputativematchpair,weassociateacon“dencemeasurewhichwerefertoas similarityweightsinourpaper.Letthehighestsimilarityinarowandcolumnofamatch xkbe mk.Let mkrbethesecondhighestsimilarityinitsrowand mkcbethesecond highestinitscolumn.Weconstructaweight tkforthecorrespondence xkas tk=(1 Š expŠ mk)2(1 Š mkr mk)(1 Š mkc mk)(4.2) The“gure4.2isfromthefamousseminalpaperbyScottandLonguet-Higgins[50]on spectralcorrespondences.Weusethe“guretoshowthatputativecorrespondencesarethe bestmutualmatch. TheJointFeatureDistributions,whicharediscussednext,areusedtosamplecorrespondelandguidetheMCMCaswell.TheuseoftheconditionalJFDalleviatestheneed 21

PAGE 31

forassumingthatcorrespondencesareindependentofeachother,acommonassumption inmanyrandomsamplingmethods. 4.2JointFeatureDistributions Triggs[64]proposedtheconceptofJointFeatureDistributions(JFDs)toprovidea ”exibleandrobustalternativetothestrictanddeterministicgeometricconstraintsused forprojectivematching.Inourcontext,weareparticularlyinterestedintwo-camera2D to2Depipolarconstraint.Simplyput,JFDsarethejointprobabilitydistributionsover theparametersofthecorresponding2Dto2Dfeatures.Theysummarizethestatistics ofagivensetofcorrespondencesanddoesnotrigidlyconstrainthemtoadeterministic geometry.Thatiswhytheyareanidealformalismtoaccountforsmallnon-rigiddistortions anderrorsthatwillinevitablybepresentinanycamera. Wecanmodelthenoisymappingofthe2Dfeatures, u ,intothecorresponding v bythe probability p ( v | u ).Theformforthisconditionalprobabilitywillbecenteredaroundthe underlying,deterministic,2Dto2Depipolarconstraintwhere F3x3isthe3x3fundamental matrix. vTF3x3u = 0 (4.3) Thisequationcanbelinearizedbyconsideringthetensorproductofthecorresponding points, x = v u ,withdimension9by1andexpressedintheform AF9x1=0,where F9x1isreshapedformof F3x3.ThislinearformimpliesthattheJFDmodelsareGaussian inthetensorspace, p ( xk) exp Š Lk 2 (4.4) wherethenegativelog-likelihoodfunction, Lk,isgivenby Lk= xk TWxk(4.5) where k variesfrom1to n .Thus,theJFDisparameterizedbythehomogeneousinformationtensor, W ,whichissymmetricpositivede“nite9by9matrixgeneralizingthe homogeneousinformation.Wecanestimatethisfromsamplecorrespondencesasfollows. 22

PAGE 32

Letusdrawarandomsampleofcorrespondences sofsize s from X .Lettheindices ofthesampledmatchpairsbe h .Thenwebuildtheir9by9homogeneousscattermatrix V =1 ssxhixT hi=1 sss T,where s=[ xh1, xhs]isthe9by s measurementmatrix and i variesfrom1to n .Thismeasurementmatrixalsoappearsinlinearmatchingtensor estimation.Triggs[64]hasfoundthattheinverseofthismatrixisagoodestimateoftheinformationtensor W VŠ 1.Inpractice,wehavetocompute W ( V +diag( ,, 0))Š 1toregularizetheinversion. The conditionalprobability ofanymatchpair( uk, vk)in X ,givenasetofcorrespondences swhichisasampleofsize s drawnfrom X isgivenbythemultivariateGaussian distributionfunctionasfollows p ( xk| s)= | ( V ( s)+ )Š 1 (2 )4 5| exp Š xk T( V ( s)+ )Š 1xk 2 (4.6) where V ( s)isa9by9matrixconstructedfrom s,asspeci“edearlier.Wewillusethis conditionalprobabilityfunctiontosamplefromthecorrespondencespace. In“gure4.3,“rstanimageisshownwithapoint.Next,highprobabilitycorrespondenceregionoversecondimagefoundusingJFDbasedonentireputativesetisshown, followedbyanotherimageshowinghighprobabilitycorrespondenceregionoversecondimagefoundusingJFDbasedonthebestcorrespondelfound.Notethatthehighprobability regionellipseinbdoesnotcoverthecorrespondingpointwhilethatincdoes.Alsonote thatJFDforaccuratesetofcorrespondenceinnarrower. 4.3TheAlgorithm Westartwiththe”owchartofouralgorithm.Allthenecessarystepsinthe”owchart aredetailedinthesectionsofthischapter.TheoverallapproachofBLOGSshownin Fig.4.4.Thealgorithmbeginsbyrandomlydrawingasamplefromthedistributionof similarityweightsoftheputativecorrespondences.Thisiscontinuedtillwegetanondegeneratecorrespondelasperourthresholdonthemeasureofnon-degeneracy,discussed later.Afterthis,dependingonaparameter ,eitheraJFDguidedsampleorasimilarity guidedsampleisdrawnateachiteration. isacontrolparameterusedtostrikeabalance 23

PAGE 33

a) b) b) c) Figure4.3.ImagesshowinghighprobabilitycorrespondencesearchregionsgivenbyJFDs. betweenthetwokindofguidanceforsearch.Similarityguidedsamplesareindependent andexploremoreglobalregionswhereasJFDguidedsamplesareconditionallydependent 24

PAGE 34

onthebestknownsamplesofarandthuseectivelysearchlocalregions. shouldbe suchthatitallowsglobalregionssothattheMCMCdoesnotconvergetoalocaloptimal solutionandshouldalsoallowlocalsearchesneartheoptimalbeforeitisdiscardedbya bettersolutionviaglobalsearch.Gettingabettersolutionanywaysisacceptable,butwe wanttohaveabalancedlocalandglobalsearchsothatwearelesslikelytomisssamples thatarepotentiallyoptimal.Inourexperiments,weuseda“xed =0.5withoutclaiming ittobethebestchoice.LikeinMetropolis-HastingsMCMCmethod,wedecidewhetherto acceptthenewlydrawnsample,giventhepreviousbestknownsampledrawnsofar.The decisiontoacceptismadeusingacorrespondencehypothesisqualitymeasure,adegeneracy measure,andeectiveproposalprobability.Thealgorithmendsafter g iterations.Inthe followingsubsections,weoutlinethevariousaspectsofthealgorithm. 4.3.1QualityMeasureoftheCorrespondel Sampsonserrordistanceforapointcorrespondencecanbetakenas(negativelog) likelihoodofthepointcorrespondencegivenacorrespondel.We“ndthelikelihoodof eachoftheputativecorrespondencesgivenacorrespondel.Weconstructameasureofthe qualityofthecorrespondelbythesumoftheselikelihoods.Summationismorerobustto thepresenceofoutliersthanproduct.Lettheerror,i.e.distancefromepipolarline,of the k thputativecorrespondencebe k.Thequalityofthecorrespondelortheassociated motionisgivenby ( 8)=nk =1exp Š k. 2 (4.7) Weset =104inourexperiments. 4.3.2DegeneracyMeasure Let sbeasampledcorrespondelofsize s whichis8inourcase.8-pointalgorithm failswhenpointslieonthecriticalsurfaceordegenerateplanes.Wecanidentifyifthe correspondencesthatlieonadegenerateplane. Foreachputativecorrespondencetensor,wedoaneigenvaluedecompositiontoget theeigenvalues.Weshouldhavetwonon-zeroeigenvaluesforboththecorrespondencesto 25

PAGE 35

Figure4.4.Flowchartofthesamplingprocessusedto“ndthebestsetofcorrespondences. 26

PAGE 36

beinformative.Inpractice,theratioofthesmallereigenvaluewithrespecttothelarger eigenvalueshouldbeaboveacertainthresholdvalue .Ifallthepairsofcorrespondencesin thecorrespondelareabovethethreshold,thecorrespondelisacceptedasnon-degenerate. ( 8)iseither0or1implyingdegenerateandnon-degeneraterespectively. ( 8)=i =7i =1 j =8j>i 2ij 1ij> (4.8) where 1ijand 2ijaretheeigenvaluesobtainedbytheeigendecompositionofthe i thand j thcorrespondencetensorsinthesampledcorrespondel.Theexpressioninbracketiseither 1or0forallpairsofcorrespondencesdependingonthethreshold .Inourexperiments, wechoosethe =0 25. 4.3.3JumpDiusion MCMCsamplinghasbeenpopularincorrespondenceandepipolargeometryestimation[12].JumpdiusionprocessbyintroducedbyGrenanderandMiller[23].Greens[22] papermorespeci“callydealswithimageprocessingandvisionrelatedresearch.Han et al. [14]hasusedjump-diusionframeworkforrangeimagesegmentationmorerecently.In theliterature[54],manyhybridsamplershavebeenreported.Jump-diusionisonesuch hybridsampler.Awidediscourseonjumpdiusioncanbefoundin[2].Whilejumpisa globalmove,diusionisalocalmove.Weuseappearancebasedpriordistributiontodraw jumpmovesamplesandepipolarconstraintbasedJFDforadistributionoflikelihoodsis usedtodrawdiusionmovesamples.Weseepriorsasacuetoglobaljumpbasedsearch andlikelihoodsasacuetolocaldiusionbasedsearch.Thebestsofarfoundsampleis improvedbybothjumpanddiusionmovesunderseveralMCMCiterations. 1.Jump:Jumpmovesexplorethespaceofcorrespondenceandmotionunbiasedonany previouslearning.Thismakesjumpaglobalsearchprocess.Jumpisaglobalexploration move. 2.Diusion:Diusionmovesexplorethespaceofcorrespondenceandmotionbiasedon thebestlearntmotionsofar.Thismakesdiusionasearchprocesslocaltothebestfound motionsofar.Diusionisalocalexplorationmoveandexploitationmoveaswell. 27

PAGE 37

3.Balancebetweenjumpanddiusion:Coordinationcanbedoneusingaparametersuppliedtothealgorithmorcanbedynamicallysetasthealgorithmproceeds.Wepreferto“x itasaparameter.Asthequalityofthemotionincreases,bothjumpmovesanddiusion movesgraduallyareunabletoimprovethequalityofmotionanyfurther.Whilediusion movesaremorelikelytogetstuckinamaxima,jumpstepsovertimeattempttoupstage thequalityofmotionwithwhichdiusionmovescancompeteandbene“t,providingbetter results.Diusionstepsalwayskeepontryingtoimprovethebestsofarbyusingitto drawmoresamples,evenwhenjumpstepsdonotleadtoimprovements.Since,wesee thatjumpanddiusionareequallyimportant;wekeeptheparameterunbiasedforboth jumpanddiusion. Putativecorrespondencesaregivenby X = { x1, x2,...., xn} .Let 8beasampleof 8-tupleofcorrespondencesdrawnfrom X .Wede“netheimportancefunctionasfollows. ( 8)= ( 8) ( 8) ( ( 8) ( 9)) (4.9) where ( 8)isthequalityofthesampleof8-tuple 8and ( 8)isthedegeneracymeasure. 4.3.3.1JumpusingSimilarity-basedProposalDistribution Wesamplecorrespondelsfromthedistributionofsimilarityweightvector t 4.2.The proposaldistributioninthiscaseisgivenby ek= tk n k =1tk(4.10) 4.3.3.2DiusionusingJFD-basedProposalDistribution Ourproposaldistributionisgivenby ek= p ( xk| 8) n k =1p ( xk| 8) (4.11) The p ( xk| 8)isobtainedfromequation4.6 28

PAGE 38

4.3.4AcceptanceofSample Wesamplefrom X basedon E = { e1.....en} ,constructedeitherbasedonJFDorsimilarity.Let { j1.....jn} betheindexesoftherandomsamplesdrawnfrom X .Wede“nethe eectiveproposalprobabilityofasample ( s| i 8)astheproductofeectiveprobabilities from E withoutreplacementofthedrawnsamples. Wedrawthenextsamplebyusingtheinformationfromthebestknownsample.Let usde“nethe8-tuplesampledrawnatiteration i as i 8.Similarly,the8-tuplesampledrawn atiteration i +1wouldbe i +1 8.Wealsode“nethebestsampleknownsofaras 8.To startotheprocess, 8isinitializedto 1 8.Eectiveproposalprobabilityatiteration i +1 isthusgivenby ( i +1 8| 8)= ej1ej2 ej8 (1 Š ej1)(1 Š ej1Š ej2) (1 Š ej1Š ej7) (4.12) TheMetropolis-Hastingssamplingstepisnowgivenby w ( i +1 8)= ( i +1 8| 8) ( 8| i +1 8)y ( i +1 8)= ( i +1 8) ( 8)(4.13) Wecanalsowrite y ( i +1 8)as y ( i +1 8)= ( i +1 8) ( i +1 8) ( 8) ( 8) (4.14) If w ( i +1 8) > 1and y ( i +1 8) > 1 8= i +1 8(4.15) i +1 8isacceptedasoptimalifboth w ( i +1 8)and y ( i +1 8)aregreaterthan1. 8istheoptimal 8-tupleofcorrespondencesfoundsofar.The“rst8-tupleissampledusing t andthereafter MCMCsamplingistriggered.Thisisrepeatedtoover g numberofiterations. 29

PAGE 39

CHAPTER5 EXTENSION 5.1ExtensiontoMultiplePutativeSets Intheprevioussection,wesawthatthetwoimportantthingsthatmakeacorrespondencemoreprobableofbeingcorrectareitssimilarityvalueanditsdistinctnessinitsrow andcolumn.Inthissection,wedescribehowweapplythesecriteriawithout“xingthe putativeset.Westartwithasetofcorrespondencethatexhibithighsimilaritiesabovea certainthreshold.Allnon-zerorowsandcolumnsremaininginthesimilaritymatrixafter thresholdingareconsideredwithoutapplyingtheuniquenessconstraintthatallowsonly diagonalpermutations.Foreach-nonzerosimilarityvalueinthematrix,apenaltybased onthesumofthenumberofothernon-zerosimilarityvaluesinitsrowandcolumnis imposed,totakecareofthedistinctnesscriterion.Aproportionalsamplingprobability isderivedandwearriveatadistributionofsimilarityweights.Whilesamplingisdone fromthedistributionofsimilarityweightsinthejumpsteps,diusionstepsareguidedby JFDbaseddistributionofprobabilitiesforallcorrespondencesthathavenon-zerosimilarityvaluesinthesimilaritymatrixafterthresholding.Initially,wethresholdthesimilarity valuesusingthestandarddeviationateachrowandcolumninthesimilaritymatrixand aspeci“edconstantassumedtobesameforallimages.Wesuggestthisconstanttobe =1 Š 1 /e =0 63accordingtotheformofoursimilaritymeasure,althoughitisan arbitrarychoice. Letthesimilaritymatrixbe S withsize( a,b ).Let Sijbeanelementof S Sij=(1 Š expŠ1 D ij)(5.1) 30

PAGE 40

where Dijisthedistancebetweenthedescriptorsof i thand j thfeatures. Sijissubject toathresholdtomeetthehighsimilaritycriterion.Toapplythedistinctnesscriterionwe comeupwithamatrix G whoseelements Gijarede“nedasfollows. Sij=0 ifSij< 3 ( Ri)+ orSij< 3 ( Cj)+ (5.2) Gij= Sij log { Ni+ Nj} (5.3) where Niand Njarethenumberofnon-zerosimilarityvaluesin i throwand j thcolumn respectively. Thejumpstepusesthedistributionofweightsaccordingto G andfordiusionstep conditionalprobabilityvaluesbasedonJFDsforthecorrespondenceswithnon-zerovalues in G areestimated.Thedistributionoftheseconditionalprobabilityvaluesisusedforthe diusionsteps.Theseconditionalprobabilityvaluesarestoredin G. Samplesaredrawnfrom G and G.While G doesnotchange, Gkeepsonchanging asperthesamplecorrespondelthatgeneratedthebestmodelsofar.Aqualitymeasure isfoundforeachnon-zerosimilarityfeaturein G or Gwhicheverisusedinthejumpor diusionsteprespectively.Asumofnegativeloglikelihoodofcorrespondencesgiventhe modelisfoundasaqualitymeasure.Onetoonemappingoruniquenessislaterimposed byapplyingHungarianalgorithmonG,orbysummingingreedyfashionwhiletakingcare ofmutualexclusionofrowsandcolumns,orbyconsideringthesumofonlythosevalues whicharemaximumintheirrowandcolumn.Summingingreedyfashionhasbeenachoice inourcase,duetoitssimplicity.Moreover,complexityoftheHungarianalgorithmisfar toohighandourneedtodistinguishinliersfromoutliersratherthanmaximizingthesum makesgreedystrategyabetterchoice. 31

PAGE 41

CHAPTER6 EXPERIMENTS WecompareouralgorithmwithLO-RANSAC[10],NAPSAC[40],MAPSAC[59]and BEEM[21]whicharethecurrentstateoftheartcompetingmethods.WehaveusedLORANSACimplementationasintheWBSImageMatcher(executableonly),availableon theinternet.NAPSACandMAPSACimplementationshavebeenusedfromTorrsToolkit forStructurefromMotion.BEEM[21]implementationisalsoavailableontheShimshonis website.WehaveusedSIFT[31]features.Generatingaputativecorrespondencesetisa partofouralgorithm.Ourputativecorrespondencesetisasetoflessthan200correspondenceswithhighestsimilarity.Foreachimage,ourputativecorrespondencesetisalsoused asinputtoNAPSAC,MAPSACandBEEM,toremovevariabilityduetofeaturechoice. However,forLO-RANSACwehadtousethesetofputativecorrespondencescomputedby itbecausewehadaccesstojusttheexecutable.ForNAPSACandMAPSACalgorithms were-computedthefundamentalmatricesusingthenormalized8-pointalgorithmonall thecomputedinliers.Wedidthistomakesurethatwegetanunscaledfundamentalmatrixfromthesealgorithms.WeusedthefundamentalmatrixgivenbytheLO-RANSACs implementationdirectlyforcomparisonafter“ndingthatitisunscaled. 6.1TestData Wehavebenchmarkedperformanceon12imagepairsincludingtwoimagesfromanotherwork.TheBlunaimagepairandKMsmimagepairweretakenfromdatasetalong theWBSImageMatcherswebsite.TheBlunaimagepairandKMsmimagepairareof size480x360,whereastherestoftheimageshaveasizeof712x534.Thetestdatacontains imagepairsthathaveaverywidebaseline,scalechanges,rotationandocclusion.Such imagepairsarenotsucientlyaddressedintheliterature.Wehavemanuallyquanti“ed 32

PAGE 42

thenumberofinliersandtheoutlierrateintheputativecorrespondencesetforeachimage pairtogiveanideaaboutthehardnessoftheepipolargeometrysearchrequiredforeach method.Thesearelisted,sortedaccordingtodicultylevel,inTable.6.1.Notethatthe lasttwopairsinthetableareparticularlychallengingimagepairs.Theytestthelimitsofcurrentapproaches,includingours,andhelpmotivatefutureresearchtosolvesuch problems. 6.2PerformanceMeasure Foreachimagepair,wemanuallymarked16correspondences,dierentfromtheSIFT featuresusedtoestimatetheepipolargeometry.Theseserveasthegroundtruth.The rootmeanSampsonsdistanceofthese16handmarkedcorrespondencesserveasthequantitativeperformancemeasure.Forthesakeofpropercomparison,weaveragedtheerror over100executionsofthealgorithm.LO-RANSACreturnsthesameepipolargeometry eachtime,sowedidnotneedtodothisforit.BEEMconvergesonmeetingitsstopping criterion.Averageerrorandaverageiterationswereusedfortoevaluatetheperformance ofBEEM. TheSampsonsdistanceisgivenby: d ( x = v u F )=ni =1( vT iF ui)2 ( F ui)2 1+( F ui)2 2+( vT iF )2 1+( vT iF )2 2(6.1) 6.3Results WetestNAPSAC,MAPSACandBLOGSfor500,1000and5000iterationsorsamples, whileLO-RANSACandBEEMexecuteuptotheirconvergence.WhileLO-RANSACalwaysconvergesatthesamenumberofiterations,BEEMconvergesafterdierentnumberof iterationseachtime.Notethatouruseofmanuallyspeci“edgroundtruthcorrespondences isamovementofacurrentevaluationmethodologytowardamorerigorousone. Inthetable6.1,the“rstcolumnliststheimages.Thesecondcolumnnotesthenumber ofinliercorrespondencesandoutlierratesforeachimage.Thiswasmanuallydetermined. ItcapturesthehardnessŽofeachimagepair.Theimagesaresortedinthetablebased 33

PAGE 43

Table6.1.ComparativeperformanceanalysisofLO-RANSAC,NAPSAC,MAPSAC, BEEMandBLOGS(ourmethod). Image Pair Putative Correspondence Quality (Inliers, Outlier Rate) Typical Sampsons Distance (pixel errors) from 16 hand marked correspondences on executing LO-RANSAC and samples needed to converge Root Mean Square Sampsons Dist ance (pixel errors) from 16 hand marked correspondence in 10 0 executions of each algorithm NAPSAC iterations MAPSAC iterations BEEM(pixel, average error iterations) BLOGS iterations500 1000 5000500100050005001000 5000 KMsm 171, 1.16 1.01, 69 0.53 0.53 0.53 0.53 0.530.53 0.59 164 0.54 0.540.54 Corridor 44, 47.659.98, 11918.29 13.40 6.78 18.47 14.975.5922.89 57 13.30 9.25 3.27 Bluna 66, 63.3 1.07 94762.67 50.03 38.88 42.13 26.985.7459.83, 510 7.48 6.563.17 Flags 47, 75.5 85.47, 20000046.50 31.20 9.56 22.34 17.326.7816.83 2805 8.65 5.27 1.72 Steelmesh 31, 84.03.08, 450688.96 5.92 4.80 5.41 4.824.57 3.24 360 3.90 3.67 2.64 Pillars 27, 85.474.77, 20000040.48 40.44 36.02 39.98 35.4233.2028.91, 377 30.89 25.53 15.34 Building 22, 88.3 1.11 ,2359741.98 35.41 14.10 42.22 44.1038.12 3.69 448 19.29 16.82 6.72 University 19, 89.9123.32, 103319.89 10.11 6.32 25.46 19.9510.8213.14,813 6.66 6.35 6.03 Stones 15, 92.365.11, 1615865.06 58.60 55.62 53.61 52.4650.63didnot converge 43.29 42.36 24.85 Parking 14, 92.7123.60, 12853830.95 31.45 31.39 37.14 34.8929.8127.73, 619 26.81 22.82 13.34 Cafeteria 11, 94.380.47, 20000087.59 86.01 85.93 76.02 74.3867.04didnot converge 69.42 65.68 59.17 Cars 11, 94.474.65 5817784.53 84.02 79.75 81.00 75.5875.39didnot converge 76.44 74.29 60.61 onthis.ForeachalgorithmwelisttheRootMeanSampsonserrorforthe16ground truthcorrespondencesfordierentnumberofiterations.ForLO-RANSAC,wedidnot havethe”exibilitytochangethenumberofiterations.Thebestresultforeachimageis showninbold.ForBEEM,we“ndtheaverageofthepixelerrorandaverageiterationsif itconvergeswithin200000iterations. Thequanti“edperformancesareshowninTable6.1.WeobservethatLO-RANSAC performswellinfewcasesanddoesnotdosoinrestofthecases.MAPSACperformsbetter thanbothNAPSACandLO-RANSACinmostofthecases.WefoundthatMAPSACcan handlehighoutlierratesaswell.MAPSAChandlingsuchhighoutlierratesisunreportedto ourknowledge.OuralgorithmconsistentlyperformedbetterthanNAPSACandMAPSAC, whileproducingalittlemorepixelerrorsononeoccasion.BEEMconvergesfastonmeeting itsownstoppingcriterion,buttheresultwasimprovedonlyonceandwasalmostthesame ononeotherimagepair.Itdidnotconvergeevenafter200000iterationson3image pairs.Verylittledierencesmightbeignoredduetopossibleinaccuracyinhand-marked points,althoughpointsweremarkedwithutmostcare.Evenonthehardimages(the lasttwo),ourperformanceisbetterthantheothers.Theresultsshowthatweattain almostthesameaccuracyin500iterationsasMAPSACattainsin5000iterationsand thatBLOGSiscapableofgainingmorefrombarelysucientnumberofinliersthatmight 34

PAGE 44

Kmsmimage pair Blunaimage pair Corridor image pairFigure6.1.ComputedepipolarlineonKmsm,CorridorandBlunaimagepairs. includedegenerateinliersaswell.Ouralgorithmisabletopushtothelimitofover90 percentoutlierswithacceptablepixelerrorsin5000iterationsinfewcases.However,what 35

PAGE 45

Flags image pair Steel mesh image pair Pillars image pairFigure6.2.ComputedepipolarlineonFlags,SteelMeshandPillarimagepairs. valueofpixelerrorisacceptabledependsontheapplication.Thetwochallengeimage pairsremainunsolved,motivatingustocomeupwithevenbettersolutions. 36

PAGE 46

Buidingimage pair Stones image pair University image pairFigure6.3.ComputedepipolarlinesonBuilding,UniversityandStonesimagepairs. 37

PAGE 47

Parking image pair Cafeteria image pair Cars image pairFigure6.4.ComputedepipolarlinesonParking,CafeteriaandCarsimagepairs. 38

PAGE 48

CHAPTER7 DISCUSSIONANDCONCLUSION Theproblemoffeaturecorrespondenceandmotionestimationstillremainsachallenge. Despitemanyyearsofresearch,theaccuracy(correctness)andprecision(convergenceto thesamesolutiononrepetitions)ofthestateoftheartalgorithmsforfeaturecorrespondenceandmotionestimationincaseofwidebaseline,highlyscaledandrotatedimage pairsisfarfrombeingpracticallyexploitable.Inliercorrespondencesmaynotyieldcorrectepipolargeometryiftheinliercorrespondencesaredegenerate.Ontheotherhand, correctepipolargeometrymightnotbesupportedbyonlyinliercorrespondences.Outlier correspondencesalongtheepipolarlinesareacceptedasinliers.Theironyisthatoutlier mightbeusedtocomeupwithagoodmotionmodelbutinliermightnot.TriggsJoint FeatureDistributioncomesinpicturehereasitgivesahighprobabilitynarrowsearch ellipticalregionwithitsmajoraxisalignedalongtheepipolarline.Andwealsoweedout degeneratecorrespondelsbyadimensionalitychecksuchthatnopairofcorrespondences inthesampledcorrespondelhasadimensionalityof1ratherthan2.JFDbasedguidance wouldtendtosamplemoreofcorrectcorrespondencesthantheconventionalapproaches. WeusedthisstrengthofJFDsandaddresstheweaknessesbynotusingitasameasureof qualityandalsoweedingoutthedegeneracy.JFDbasedguidanceisonlyascorrectasthe sampledcorrespondelusedtoestimatetheJFDanditsharesitsstrengthandweaknesses withthe8-pointalgorithmexceptforbeingfocusedonaregionratherthanalineandnot requiringtherank2constraintbeingexplicitlyimposed. ThesuccessoftheBLOGSapproachcanbeattributedtothreeaspects:thesimilarityweights-guidedglobalsearch,theJFDguidedsampling,andthedegeneracycriterion toweedoutdegeneratecorrespondences.Thesimilarity-basedandJFD-basedsampling strategiescomplementeachother,theformerinducessearchoverlargerglobalregionsand 39

PAGE 49

thelaterisalocalsearch.Onemajorproblemwhichisstillextensivelyresearchedisthat anallinliercorrespondelcanhaveaveryhighsupportseteveniftheylieonadegenerate planethatleadstowrongepipolargeometry.Inthispaper,weproposedandusedanovel strategytoruleoutcorrespondelswithcorrespondencesfromadegenerateplane.This takesusastepfurthertheoreticallyandexperimentally.Theproposedalgorithmtakes careofallknownaspectsofepipolargeometryestimationinasimpleanduni“edway. WecomparedouralgorithmtoBEEM,MAPSAC,NAPSACandLO-RANSAConmany imagepairswithheavilynoisedcorrespondencesandfoundthatouralgorithmperformed betterinalmostallthecases. 40

PAGE 50

REFERENCES [1]X.ArmangueandJ.Salvi.Overallviewregardingfundamentalmatrixestimation. ImageandVisionComputing ,21(2):205…220,2003. [2]N.G.ArnaudDoucet,NandodeFreitas. SequentialMonteCarlomethodsinpractice Springer,NewYork,NY,USA,2001. [3]A.M.Baumberg.Reliablefeaturematchingacrosswidelyseparatedviews.In ProceedingsofCVPR ,pagesI:774…781,2000. [4]S.Brandt.Ontheprobabilisticepipolargeometry. ImageandVisionComputing 26(3):405…414,March2008. [5]M.BrownandD.G.Lowe.Recognisingpanoramas.In Procee dingsofICCV ,page 1218,2003. [6]H.F.ChenandP.Meer.Robustregressionwithprojectionbasedm-estimators.In Procee dingsofICCV ,pages878…885,2003. [7]H.ChuiandA.Rangarajan.Anewalgorithmfornon-rigidpointmatching.In Procee dingsofCVPR ,pagesII:44…51,2000. [8]O.ChumandJ.Matas.MatchingwithPROSAC-ProgressiveSampleConsensus.In Procee dingsofCVPR ,volume1,pages220…226,2005. [9]O.ChumandJ.Matas.OptimalRandomizedRANSAC. IEEETransactionson PAMI ,30(8):1472…1482,August2008. [10]O.Chum,J.Matas,andJ.Kittler.LocallyoptimizedRANSAC.In Procee dingsof DAGM ,pages236…243,2003. [11]O.Chum,T.Werner,andJ.Matas.Two-viewgeometryestimationunaectedbya dominantplane.In Procee dingsofCVPR ,pagesI:772…779,2005. 41

PAGE 51

[12]F.Dellaert,S.M.Seitz,C.E.Thorpe,andS.Thrun.Structurefrommotionwithout correspondence.In Procee dingsofCVPR ,volume2,pages557…564,2000. [13]J.DomkeandY.Aloimonos.Aprobabilisticframeworkforcorrespondenceandegomotion.In Procee dingsofWDV06 ,pages232…242,2006. [14]Z.T.FengHanandS.C.Zhu.Rangeimagesegmentationbyaneectivejumpdiusionmethod. IEEETransactionsonPAMI ,26(9):1138…1153,2004. [15]M.A.FishlerandR.C.Boles.Randomsampleconsensus:Aparadigmformodel “ttingwithapplicationstoimageanalysisandautomatedpagescartography. CommunicationsoftheACM ,24(6):381…395,1981. [16]A.W.Fitzgibbon.Simultaneouslinearestimationofmultipleviewgeometryandlens distortion.In Procee dingsofCVPR ,volume1,page125,2001. [17]J.M.FrahmandM.Pollefeys.RANSACfor(Quasi-)Degeneratedata(QDEGSAC). In Procee dingsofCVPR ,pagesI:453…460,2006. [18]B.GeorgescuandP.Meer.Balancedrecoveryof3dstructureandcameramotionfrom uncalibratedimagesequences.In Procee dingsofECCV ,pageII:294.,2002. [19]N.GheissariandA.Bab-Hadiashar.Acomparativestudyofmodelselectioncriteria forcomputervisionapplications. ImageandVisionComputing ,26(12):1636…1649, 2008. [20]L.GoshenandI.Shimshoni.Balancedexplorationandexploitationmodelsearchfor ecientepipolargeometryestimation. IEEETransactionsonPAMI ,30(7):1230…1242, 2008. [21]L.GoshenandI.Shimshoni.Guidedsamplingviaweakmotionmodelsandoutlier samplegenerationforepipolargeometryestimation. InternationalJournalofComputerVision ,80(2):275…288,2008. [22]P.J.Green.Reversiblejumpmarkovchainmontecarlocomputationandbayesian modeldetermination. Biometrika ,82(4):711…732,1995. [23]U.GrenanderandM.Miller.Representationsofknowledgeincomplexsystems. JournalofRoyalStatisticalSociety,SeriesB ,56(4):549…603,1994. 42

PAGE 52

[24]R.M.Haralick,C.N.Lee,X.Zhuang,V.G.Vaidya,andM.B.Kim.Poseestimation fromcorrespondingpointdata.In Procee dingsofCVWS87 ,pages258…263,1987. [25]C.HarrisandM.Stephens.Acombinedcornerandedgedetection.In Procee dingsof TheFourthAlveyVisionConference ,pages147…151,1988. [26]R.I.Hartley.Indefenseofthe8-pointalgorithm. IEEETransactionsonPAMI 19(6):580…593,1997. [27]R.I.HartleyandA.Zisserman. Multipleviewgeometryincomputervision .Cambridge UniversityPress,NewYork,NY,USA,2000. [28]T.S.HuangandA.N.Netravali.Motionandstructurefromfeaturecorrespondences: Areview. Procee dingsoftheIEEE ,82(2):252…268,February1994. [29]D.Keren.Aprobabilisticmethodforpointmatchinginthepresenceofnoiseand degeneracy. JournalofMathematicalImagingandVision ,33(3):338…346,2009. [30]Longuet-Higgins.Acomputeralgorithmforreconstructingfromtwoprojections. Nature ,293:133…135,1981. [31]D.G.Lowe.Distinctiveimagefeaturesfromscale-invariantkeypoints. International JournalofComputerVision ,60:91…110,2004. [32]Q.T.LuongandO.D.Faugeras.Thefundamentalmatrix:Theory,algorithms,and stabilityanalysis. InternationalJournalofComputerVision ,17(1):43…75,January 1996. [33]J.MacielandJ.Costeira.Aglobalsolutiontosparsecorrespondenceproblems. IEEE TransactionsonPAMI ,25(2):187…199,February2003. [34]A.Makadia,C.Geyer,S.Sastry,andK.Daniilidis.Radon-basedstructurefrom motionwithoutcorrespondences.In Procee dingsofCVPR ,pagesI:796…803,2005. [35]J.Matas,O.Chum,M.Urban,andT.Pajdla.Robustwide-baselinestereofrom maximallystableextremalregions. ImageandVisionComputing ,22(10):761…767, 2004. [36]J.Matas,S.Obdrzalek,andO.Chum.Localaneframesforwide-baselinestereo. In Procee dingsofICPR ,pages363…366,2002. 43

PAGE 53

[37]K.MikolajczykandC.Schmid.Aperformanceevaluationoflocaldescriptors. IEEE TransactionsonPAMI ,27(10):1615…1630,2005. [38]K.Mikolajczyk,T.Tuytelaars,C.Schmid,A.Zisserman,J.Matas,F.Schaalitzky, T.Kadir,andL.J.VanGool.Acomparisonofaneregiondetectors. International JournalofComputerVision ,65(1-2):43…72,November2005. [39]L.MoisanandB.Stival.Aprobabilisticcriteriontodetectrigidpointmatchesbetween twoimagesandestimatethefundamentalmatrix. InternationalJournalofComputer Vision ,57(3):201…218,2004. [40]D.R.Myatt,P.H.S.Torr,S.J.Nasuto,J.M.Bishop,andR.Craddock.NAPSAC: Highnoise,highdimensionalrobustestimation-itsinthebag.In Procee dingsof BMVC ,volume2,pages458…467,2002. [41]D.Nist er.PreemptiveRANSACforLiveStructureandMotionEstimation.In ProceedingsofICCV ,page199,2003. [42]D.Nister.Anecientsolutiontothe“ve-pointrelativeposeproblem. IEEETransactionsonPAMI ,26(6):756…777,June2004. [43]J.Oliensis.Acritiqueofstructure-from-motionalgorithms. ComputerVisionand ImageUnderstanding ,84(3):407…408,December2001. [44]J.OliensisandY.Genc.Fastandaccuratealgorithmsforprojectivemulti-image structurefrommotion. IEEETransactionsonPAMI ,23(6):546…559,June2001. [45]M.Perdoch,J.Matas,andO.Chum.Epipolargeometryfromtwocorrespondences. In Procee dingsofICPR ,pages215…219,2006. [46]M.Pollefeys,F.Verbiest,andL.J.V.Gool.Survivingdominantplanesinuncalibrated structureandmotionrecovery.In Procee dingsofECCV ,pages837…851,2002. [47]S.RozenfeldandI.Shimshoni.TheModi“edpbM-EstimatorMethodandaRuntime AnalysisTechniquefortheRANSACFamily.In Procee dingsofCVPR ,pagesI:1113… 1120,2005. [48]S.SantiniandR.Jain.Similaritymeasures. IEEETransactionsonPAMI ,21(9):871… 883,1999. 44

PAGE 54

[49]F.SchaalitzkyandA.Zisserman.Multi-viewmatchingforunorderedimagesets, orŽhowdoiorganizemyholidaysnaps?Ž.In Procee dingsofECCV ,pages414…431, 2002. [50]G.ScottandH.LonguetHiggins.Analgorithmforassociatingthefeaturesoftwo images. RoyalP ,B-244:21…26,1991. [51]L.S.ShapiroandJ.M.Brady.Feature-basedcorrespondence:Aneigenvectorapproach. ImageandVisionComputing ,10(5):283…288,June1992. [52]C.V.Stewart.MINPRAN:Anewrobustoperatorforcomputervision. IEEETransactionsonPAMI ,17(10):925…938,1995. [53]H.Stew enius,D.Nist er,F.Kahl,andF.Schaalitzky.Aminimalsolutionforrelative posewithunknownfocallength. ImageandVisionComputing ,26(7):871…877,2008. [54]L.Tierney.Markovchainsforexploringposteriordistributions. AnnalofStatistics 22:1701…1728,1994. [55]W.S.Tong,C.K.Tang,andG.Medioni.Simultaneoustwo-viewepipolargeometry estimationandmotionsegmentationby4dtensorvoting. IEEETransactionson PAMI ,26(9):1167…1184,September2004. [56]B.J.Tordo.Guided-MLESAC:FasterImageTransformEstimationbyUsingMatchingPriors. IEEETransactionsonPAMI ,27(10):1523…1535,2005.Member-DavidW. Murray. [57]P.H.S.Torr. OutlierdetectionandMotionSegmentation .Dept.ofEngineering Science,UniversityofOxford,1995. [58]P.H.S.Torr.Anassessmentofinformationcriteriaformotionmodelselection.In Procee dingsofCVPR ,page47,1997. [59]P.H.S.Torr.Bayesianmodelestimationandselectionforepipolargeometryand genericmanifold“tting. InternationalJournalofComputerVision ,50(1):35…61,2002. [60]P.H.S.TorrandC.Davidson.IMPSAC:SynthesisofImportanceSamplingand RandomSampleConsensus. IEEETransactionsonPAMI ,25(3):354…364,2003. 45

PAGE 55

[61]P.H.S.Torr,A.W.Fitzgibbon,andA.Zisserman.Theproblemofdegeneracy instructureandmotionrecoveryfromuncalibratedimagesequences. International JournalofComputerVision ,32(1):27…44,1999. [62]P.H.S.TorrandA.Zisserman.MLESAC:anewrobustestimatorwithapplicationto estimatingimagegeometry. ComputerVisionandImageUnderstanding ,78(1):138… 156,2000. [63]P.H.S.Torr,A.Zisserman,andS.J.Maybank.Robustdetectionofdegenerate con“gurationswhileestimatingthefundamentalmatrix. ComputerVisionandImage Understanding ,71(3):312…333,1998. [64]B.Triggs.Jointfeaturedistributionsforimagecorrespondence.In Procee dingsof ICCV ,pages201…208,2001. [65]B.Triggs,P.McLauchlan,R.Hartley,andA.Fitzgibbon.Bundleadjustment…a modernsynthesis.In VisionAlgorithms:TheoryandPractice ,volume1883of Lecture NotesinComputerScience ,pages298…372,2000. 46