USF Libraries
USF Digital Collections

Fuzzy ants as a clustering concept

MISSING IMAGE

Material Information

Title:
Fuzzy ants as a clustering concept
Physical Description:
Book
Language:
English
Creator:
Kanade, Parag M
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Hard C Means Algorithm
Fuzzy C Means Algorithm
Ant Colony Optimization
swarm intelligence
cluster analysis
Dissertations, Academic -- Computer Science -- Masters -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: We present two Swarm Intelligence based approaches for data clustering. The first algorithm, Fuzzy Ants, presented in this thesis clusters data without the initial knowledge of the number of clusters. It is a two stage algorithm. In the first stage the ants cluster data to initially create raw clusters which are refined using the Fuzzy C Means algorithm. Initially, the ants move the individual objects to form heaps. The centroids of these heaps are redefined by the Fuzzy C Means algorithm. In the second stage the objects obtained from the Fuzzy C Means algorithm are hardened according to the maximum membership criteria to form new heaps. These new heaps are then moved by the ants. The final clusters formed are refined by using the Fuzzy C Means algorithm. Results from experiments with 13 datasets show that the partitions produced are competitive with those from FCM. The second algorithm, Fuzzy ant clustering with centroids, is also a two stage algorithm, it requires an initial knowledge of the number of clusters in the data. In the first stage of the algorithm ants move the cluster centers in feature space. The cluster centers found by the ants are evaluated using a reformulated Fuzzy C Means criterion. In the second stage the best cluster centers found are used as the initial cluster centers for the Fuzzy C Means algorithm. Results on 18 datasets show that the partitions found by FCM using the ant initialization are better than those from randomly initialized FCM. Hard C Means was also used in the second stage and the partitions from the ant algorithm are better than from randomly initialized Hard C Means. The Fuzzy Ants algorithm is a novel method to find the number of clusters in the data and also provides good initializations for the FCM and HCM algorithms. We performed sensitivity analysis on the controlling parameters and found the Fuzzy Ants algorithm to be very sensitive to the Tcreateforheap parameter. The FCM and HCM algorithms, with random initializations can get stuck in a bad extrema, the Fuzzy ant clustering with centroids algorithm successfully avoids these bad extremas.
Thesis:
Thesis (M.S.C.S.)--University of South Florida, 2004.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Parag M. Kanade.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 85 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001479812
oclc - 56389785
notis - AJS3943
usfldc doi - E14-SFE0000397
usfldc handle - e14.397
System ID:
SFS0025089:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001479812
003 fts
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 040817s2004 flua sbm s000|0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0000397
035
(OCoLC)56389785
9
AJS3943
b SE
SFE0000397
040
FHM
c FHM
090
QA76 (ONLINE)
1 100
Kanade, Parag M.
0 245
Fuzzy ants as a clustering concept
h [electronic resource] /
by Parag M. Kanade.
260
[Tampa, Fla.] :
University of South Florida,
2004.
502
Thesis (M.S.C.S.)--University of South Florida, 2004.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 85 pages.
520
ABSTRACT: We present two Swarm Intelligence based approaches for data clustering. The first algorithm, Fuzzy Ants, presented in this thesis clusters data without the initial knowledge of the number of clusters. It is a two stage algorithm. In the first stage the ants cluster data to initially create raw clusters which are refined using the Fuzzy C Means algorithm. Initially, the ants move the individual objects to form heaps. The centroids of these heaps are redefined by the Fuzzy C Means algorithm. In the second stage the objects obtained from the Fuzzy C Means algorithm are hardened according to the maximum membership criteria to form new heaps. These new heaps are then moved by the ants. The final clusters formed are refined by using the Fuzzy C Means algorithm. Results from experiments with 13 datasets show that the partitions produced are competitive with those from FCM. The second algorithm, Fuzzy ant clustering with centroids, is also a two stage algorithm, it requires an initial knowledge of the number of clusters in the data. In the first stage of the algorithm ants move the cluster centers in feature space. The cluster centers found by the ants are evaluated using a reformulated Fuzzy C Means criterion. In the second stage the best cluster centers found are used as the initial cluster centers for the Fuzzy C Means algorithm. Results on 18 datasets show that the partitions found by FCM using the ant initialization are better than those from randomly initialized FCM. Hard C Means was also used in the second stage and the partitions from the ant algorithm are better than from randomly initialized Hard C Means. The Fuzzy Ants algorithm is a novel method to find the number of clusters in the data and also provides good initializations for the FCM and HCM algorithms. We performed sensitivity analysis on the controlling parameters and found the Fuzzy Ants algorithm to be very sensitive to the Tcreateforheap parameter. The FCM and HCM algorithms, with random initializations can get stuck in a bad extrema, the Fuzzy ant clustering with centroids algorithm successfully avoids these bad extremas.
590
Adviser: Hall, Lawrence O.
653
Hard C Means Algorithm.
Fuzzy C Means Algorithm.
Ant Colony Optimization.
swarm intelligence.
cluster analysis.
690
Dissertations, Academic
z USF
x Computer Science
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.397



PAGE 1

FuzzyAntsasaClusteringConcept by ParagM.Kanade Athesissubmittedinpartialful“llment oftherequirementsforthedegreeof MasterofScienceinComputerScience DepartmentofComputerScienceandEngineering CollegeofEngineering UniversityofSouthFlorida MajorProfessor:LawrenceO.Hall,Ph.D. DmitryB.Goldgof,Ph.D. SudeepSarkar,Ph.D. DateofApproval: June17,2004 Keywords:ClusterAnalysis,SwarmIntelligence,AntColonyOptimization,FuzzyCMeans Algorithm,HardCMeansAlgorithm c Copyright2004,ParagM.Kanade

PAGE 2

DEDICATION TomyparentsandmylovingsisterPallavi.

PAGE 3

ACKNOWLEDGEMENTS IwouldliketothankDr.LawrenceO.Hallforgivingmetheopportunitytoworkwithhim.His guidanceandsupportwasinvaluable.IwouldliketothankDr.DmitryGoldgof,andDr.Sudeep Sarkarfortakingthetimeandefforttoensurethatmyworkwasofquality.Iamgratefultomy family,friendsandcolleagueswhogavemeinspirationandhelpedmeineverystep.

PAGE 4

TABLEOFCONTENTS LISTOFTABLES iii LISTOFFIGURES vi ABSTRACT viii CHAPTER1INTRODUCTION1 1.1Clustering 1 1.1.1HardClustering2 1.1.2FuzzyClustering3 1.2SwarmIntelligence3 1.2.1SelfOrganizationinSocialInsects4 1.2.2Stigmergy5 1.2.3CemeteryOrganizationandBroodSortinginAnts5 CHAPTER2DATASETS7 2.1IrisPlantDataset8 2.2WineRecognitionDataset8 2.3GlassIdentificationDataset13 2.4MultipleSclerosisDataset14 2.5MRIandBritishTownsDatasets14 2.6ArtificialDatasets14 CHAPTER3FUZZYANTSALGORITHM15 3.1PickingupanObject20 3.2DroppinganObject20 3.3TheSecondStage21 3.4Results 23 3.4.1IrisDataset23 3.4.2WineRecognitionDataset24 3.4.3GlassIdentificationDataset25 3.4.4ArtificialDatasets26 3.5EffectofParameterVariation29 3.6VariationsintheAlgorithm29 3.7Discussion36 CHAPTER4AUTOMATICALLYSETTINGTHEPARAMETERS38 4.1NewMetricforMergingHeaps38 4.2ResultsUsingtheNewMetric38 4.3FuzzyHypervolume40 i

PAGE 5

CHAPTER5FUZZYANTCLUSTERINGWITHCENTROIDS44 5.1Introduction44 5.2ReformulationofClusteringCriteriaforFCMandHCM44 5.3Algorithm45 5.4Results 49 5.5HardCMeans51 5.6ExecutionTime53 5.7Conclusions55 CHAPTER6CONCLUSIONSANDFUTUREWORK57 6.1CombinedAlgorithm57 6.2Contributions59 6.3FutureWork59 REFERENCES 60 APPENDICES 63 AppendixADatasets64 A.1ParametersUsedtoGeneratetheGaussianDatasets64 A.2PlotsoftheDierentDatasets66 ii

PAGE 6

LISTOFTABLES Table2.1Datasets 7 Table3.1ValuesoftheParametersUsedintheFuzzyAntsExperiments23 Table3.2ResultsfromFCM24 Table3.3ResultsforIrisDataset24 Table3.4ResultsforIrisDataset(2Class)25 Table3.5ResultsforWineRecognitionDataset25 Table3.6ResultsforGlassIdentificationDataset25 Table3.7ResultsforGlassIdentificationDataset(2Class)26 Table3.8ResultsforGauss-1Dataset27 Table3.9ResultsforGauss-2Dataset27 Table3.10ResultsforGauss-3Dataset27 Table3.11ResultsforGauss-4Dataset27 Table3.12ResultsforGauss-5Dataset28 Table3.13ResultsforGauss500-1Dataset28 Table3.14ResultsforGauss500-2Dataset28 Table3.15ResultsforGauss500-3Dataset28 Table3.16ResultsforGauss500-4Dataset29 Table3.17ResultsforGauss500-5Dataset29 Table3.18EffectofVariationofIterationsonIrisDataset30 Table3.19EffectofVariationofIterationsonWineDataset30 Table3.20ResultsfortheIrisDataset(3DBoard)31 Table3.21ResultsfortheIrisDataset(2Class)(3Dboard)31 Table3.22ResultsfortheWineDataset(3DBoard)32 Table3.23ResultsfortheGlassDataset(3Dboard)32 iii

PAGE 7

Table3.24ResultsfortheGlass(2Class)Dataset(3DBoard)33 Table3.25ResultsfortheGauss500-1Dataset(3DBoard)34 Table3.26ResultsfortheGauss500-2Dataset(3DBoard)34 Table3.27ResultsfortheGauss500-3Dataset(3DBoard)35 Table3.28ResultsfortheGauss500-4Dataset(3DBoard)35 Table3.29ResultsfortheGauss500-5Dataset(3DBoard)36 Table3.30TypicalExampleofVarianceinDataPartitionsObtainedwiththeWineDataset with Tcreateforheap=0.09and3000Iterations37 Table4.1ResultsfromtheNewMetricforIrisDataset39 Table4.2ResultsfromtheNewMetricforWineDataset39 Table4.3EffectofVariationofIterationsonIrisDataset40 Table4.4EffectofVariationofIterationsonWineDataset40 Table4.5 TcreateforheapandFuzzyHypervolume42 Table5.1ParameterValues49 Table5.2ResultsforFuzzyCMeans50 Table5.3NewParameters50 Table5.4ResultsforFCMObtainedfromModifiedParameters51 Table5.5ResultsforHardCMeans52 Table5.6ResultsfortheBritishTownsDataset52 Table5.7ResultsfortheWineDataset53 Table5.8ResultsfortheMRIDataset53 Table5.9Variationof RmwiththeNumberofAntsforSlice#35ofMRIDataset53 Table5.10ExecutionTime54 Table5.11FrequencyofDifferentExtremafr omFCM,forBritishTownsandMRIDatasets55 Table5.12FrequencyofDifferentExtremafromHCM,forGlass(2Class)andIrisDatasets56 Table5.13FrequencyofDifferen tExtremafromHCM,MRIDataset56 Table6.1ResultsfromtheCombinedAlgorithm58 Table6.2ResultsfromtheCombinedAlgorithm(Errors)58 TableA.1ParametersUsedtoGeneratetheGaussDatasets64 iv

PAGE 8

TableA.2ParametersUsedtoGeneratetheGauss-500Datasets65 v

PAGE 9

LISTOFFIGURES Figure1.1FuzzyCMeansAlgorithm4 Figure2.1IrisDataset(Normalized)-First2PrincipalComponents9 Figure2.2IrisDataset(Normalized)-First3PrincipalComponents10 Figure2.3IrisDataset:2Classes(Normalized)-First2PrincipalComponents11 Figure2.4IrisDataset:2Classes(Normalized)-First3PrincipalComponents12 Figure3.1TheAntBasedAlgorithm16 Figure3.2Algorithm18 Figure3.3Algorithm(cont...)19 Figure3.4AlgorithmtoPickupanObject20 Figure3.5AlgorithmforDroppinganObject21 Figure3.6TheTwo-stageAlgorithm22 Figure4.1TcreateforheapvsFuzzyHypervolume42 Figure4.2TcreateforheapvsFuzzyHypervolume(Log)43 Figure5.1PictorialViewoftheAlgorithm47 Figure5.2FuzzyAntClusteringwithCentroidsAlgorithm48 FigureA.1WineDataset(Normalized)-First2PrincipalComponents66 FigureA.2WineDataset(Normalized)-First3PrincipalComponents66 FigureA.3GlassDataset(Normalized)-First2PrincipalComponents67 FigureA.4GlassDataset(Normalized)-First3PrincipalComponents67 FigureA.5GlassDataset:2classes(Normalized)-First2PrincipalComponents68 FigureA.6GlassDataset:2classes(Normalized)-First3PrincipalComponents68 FigureA.7MultipleSclerosisDataset(Norm alized)-First2PrincipalComponents69 FigureA.8MultipleSclerosisDataset(Norm alized)-First3PrincipalComponents69 FigureA.9Gauss-1Dataset(Normalized)70 vi

PAGE 10

FigureA.10Gauss-2Dataset(Normalized)70 FigureA.11Gauss-3Dataset(Normalized)71 FigureA.12Gauss-4Dataset(Normalized)71 FigureA.13Gauss-5Dataset(Normalized)72 FigureA.14Gauss500-1Dataset(Normalized)72 FigureA.15Gauss500-2Dataset(Normalized)73 FigureA.16Gauss500-3Dataset(Normalized)73 FigureA.17Gauss500-4Dataset(Normalized)74 FigureA.18Gauss500-5Dataset(Normalized)74 vii

PAGE 11

FUZZYANTSASACLUSTERINGCONCEPT ParagM.Kanade ABSTRACT WepresenttwoSwarmIntelligencebasedapproachesfordataclustering.Thefirstalgorithm, FuzzyAnts ,presentedinthisthesisclustersdatawithouttheinitialknowledgeofthenumberof clusters.Itisatwostagealgorithm.Inthefirststagetheantsclusterdatatoinitiallycreateraw clusterswhicharerefinedusingtheFuzzyCMeansalgorithm.Initially,theantsmovetheindividual objectstoformheaps.Thecentroidsoftheseh eapsarere“nedbytheFuzzyCMeansalgorithm. InthesecondstagetheobjectsobtainedfromtheFuzzyCMeansalgorithmarehardenedaccording tothemaximummembershipcriteriatoformnewheaps.Thesenewheapsarethenmovedbythe ants.ThefinalclustersformedarerefinedbyusingtheFuzzyCMeansalgorithm.Resultsfrom experimentswith13datasetsshowthatthepartitionsproducedarecompetitivewiththosefrom FCM.Thesecondalgorithm, Fuzzyantclusteringwithcentroids ,isalsoatwostagealgorithm,it requiresaninitialknowledgeofthenumberofclustersinthedata.Inthefirststageofthealgorithm antsmovetheclustercentersinfeaturespace.Th eclustercentersfoundbytheantsareevaluated usingareformulatedFuzzyCMeanscriterion. Inthesecondstagethebestclustercentersfound areusedastheinitialclustercentersfortheFu zzyCMeansalgorithm.Resultson18datasetsshow thatthepartitionsfoundbyFCMusingtheantinitializationarebetterthanthosefromrandomly initializedFCM.HardCMeanswasalsousedinthesecondstageandthepartitionsfromtheant algorithmarebetterthanfromrandomlyinitializedHardCMeans.The FuzzyAnts algorithmis anovelmethodto“ndthenumberofclustersinthedataandalsoprovidesgoodinitializationsfor theFCMandHCMalgorithms.Weperformedsensitivityanalysisonthecontrollingparameters andfoundthe FuzzyAnts algorithmtobeverysensitivetothe Tcreateforheapparameter.TheFCM andHCMalgorithms,withrandominitializationscangetstuckina bad extrema,the Fuzzyant clusteringwithcentroids algorithmsuccessfullyavoidsthese bad extremas. viii

PAGE 12

CHAPTER1 INTRODUCTION Moderntechnologyprovidesuswithefficientandlow-costtechniquesfordatacollection.Raw data,however,isoflimitedusefordecisionmakingandintelligentanalysis.Machinelearningaims tocreateautomaticorsemiautomatictoolsfortheanalysisofrawdatatodiscoverusefulpatterns andrules.Clusteringisoneofthemostimportantunsupervisedlearningtechniques. 1.1Clustering Theaimofclusteranalysisistofindgroupingsorstructureswithinunlabeleddata[19].The partitionsfoundshouldingeneralhavethefollowingproperties Homogeneity: Thedatathatareassignedtothesameclustershouldbesimilar. Heterogeneity: Thedatathatareassignedtodiffer entclustersshouldbedifferent. Inmostcasesthedataisintheformofreal-valuedvectors.Theeuclideandistanceisasuitable measureofsimilarityforthesedatasets.Thepartitionsshouldthenbesuchthattheintra-cluster distanceisminimizedandtheinter-clusterdistanceismaximized. Theclusteringtechniquescanbebr oadlyclassifiedasfollows[16]: € Incomplete/Heuristic: Clustersaredeterminedbyheuristicmethodsbasedonvisualization afterdimensionalityreductionofthedata.Thedimensionalityisreducedbyusinggeometrical methods(PCA)orprojectiontechniques. € Deterministiccrisp: Eachdatumisassignedtooneandonlyonecluster. € Overlappingcrisp: Eachdatumcanbesimultaneously assignedtoseveralclusters. € Probabilistic: Aprobabilitydistributionspecifiestheprobabilityofassignmentofthedatum toacluster. 1

PAGE 13

€ Possibilistic/Fuzzy: Thesearepurefuzzyclusteringtechni ques.Degreesofmembershipindicatetheextenttowhichthedatumbelongstothecluster.Thesumofmembershipsofeach datumacrossalltheclustersmaynotbe1forpossibilisticclustering. € Hierarchical: Thesetechniquesarebasedoneitherdivi dingthedataintomorefine-grained classesorcombiningsmallclasse stomorecoarse-grainedclasses. € Objectivefunctionbased: Thesetechniquesarebasedonanobjectiveorevaluationfunction thatassignsaqualityorerrorvaluetoeachpossiblepartitionorgroupofclusters.The partitionthatobtainsthebestevaluationisthechosensolution. € Clusterestimation: Thesetechniquesuseheuristicfunctionstobuildpartitionsandestimate theclusterparameters. 1.1.1HardClustering HardCMeans(HCM)isoneofthesimplestunsupervisedclusteringalgorithmstoclusterdata intoafixednumberofclusters.Thebasicideaofthealgorithmistoinitiallyguessthecentroids oftheclustersandthenrefinethem.Clusterinitializationisverycrucialbecausethealgorithmis verysensitivetothisinitialization.Agoodchoicefortheinitialclustercentersistoplacethemas farawayfromeachotheraspossible.Thenearestneighboralgorithmisthenusedtoassigneach featurevectortoacluster .Usingtheclustersobtained,newclustercentroidsarecalculated.The abovestepsarerepeateduntilthereisn osignificantchangeinthecentroids. TheobjectivefunctionminimizedbythehardCMeansalgorithmisgivenin1.1 J =ci =1 nk =1Dik( xk,i)(1.1) where € c 2 : Numberofclusters € n: Numberofdatapoints € i: The ithclusterprototype € Dik( xk,i) : Distanceof xkfrom ithclustercenter 2

PAGE 14

1.1.2FuzzyClustering Hardclusteringalgorithmsassigneachdatumtooneandonlyonecluster.Thismodelis inappropriateforrealdatasetsinwhichtheboundariesbetweentheclustersmaybefuzzy.Fuzzy algorithmscanassigndatatomultipleclusters.Fuzzyalgorithmsarebasedonfuzzylogicwhich wasdevelopedbyLotfiZadeh[38].Determinist icmembershipfunctionsassigneachdatumtoa particularcluster,thatis,themembershipfunc tionsmapthemembershipofthedatuminacluster toeither0or1.Fuzzymembershipfunctions,on theotherhand,mapthemembershipstothereal interval[0.01.0].Thedegreeofmembershipintheclusterdependsontheclosenessofthedatum totheclustercenter.Highmembershipvaluesi ndicatelessdistancebetweenthedatumandthe clustercenter. TheFuzzyCMeansalgorithm(FCM),developedbyBezdek[3],allowsthedatumtobea partialmemberofmorethanonecluster.TheFCMalgorithmisbasedonminimizingtheobjective function1.2 Jm( U, )=ci =1 nk =1Um ikDik( xk,i)(1.2) where € c 2 : Numberofclusters € n: Numberofdatapoints € i: The ithclusterprototype € Dik( xk,i) : Distanceof xkfrom ithclustercenter € Uik: Membershipofthe kthobjectinthe ithcluster € m 1 : Thedegreeoffuzzification TheFuzzyCMeansalgorithmisshowninFigure1.1 ThedrawbackofclusteringalgorithmslikeFCMandHCM,whicharebasedonthehillclimbingheuristicis,priorknowledgeofthenumberofclustersinthedataisrequiredandtheyhave significantsensitivitytoclustercenterinitialization. 1.2SwarmIntelligence Researchinusingthesocialinsectmetaphorforsolvingproblemsisstillinitsinfancy.The systemsdevelopedusingswarmintelligenceprincipl esemphasizedistributiveness,directorindirect 3

PAGE 15

1.InitializetheinitialclustercentersandcalculatetheUmatrix U(0)2.Atthe tthstep,calculatethenewclustercenters i= n k =1um ik xk n k =1um ik3.UpdatetheUmatrix, Ut,U( t 1)uik=1 c j =1D ik ( x k i ) D ij ( x k j )1 1 Š m4.If | UtŠ Ut 1| < thenSTOP;otherwisegotostep2 Figure1.1.FuzzyCMeansAlgorithm interactionsamongrelativelysimpleagents,”exibilityandrobustness[7].Successfulapplications havebeendevelopedinthecommunicationnetworks,roboticsandcombinatorialoptimizationfields. 1.2.1SelfOrganizationinSocialInsects SelfOrganizationisthemechanismofformingglobalstructuresfromtheinteractionoflowerlevelcomponents.Therulesspecifyingtheintera ctionsamonglower-levelcomponentsarebased onlocalinformation,withoutreferencetoglobalinformation.Theglobalpatternisanemergent propertyofthesystem.Selforganizationisbasedonfourbasicprinciples[7]: € Positivefeedback: Itisthebasisofmorphogenesis.Examplesofpositivefeedbackarerecruitmentandreinforcement. € Negativefeedback: Itcounterbalancespositivefeedbackandisresponsibleforthestabilization oftheemergentpattern.Negativefeedbackmaybeintheformofsaturation,exhaustion,or competition. € Amplicationofuctuations: Amplificationoffluctuationsisanimportantfactorforself organization.Randomnessiscrucial,sinceitenablesdiscoveryofnewsolutions.Fluctuations canactasseedsforthegro wthofnewstructures. € Multipleinteractions: Selforganizationisaresultofinteractionswithinmutuallytolerant individuals.Individualsmakeuseoftheirownactivitiesaswellasofothersactivities.For 4

PAGE 16

examplesthetrailslaidbyantsappeartoself-organizeandbeusedcollectivelyifindividuals useotherspheromone. Thekeycharacteristicsofaselforganizedsystemare[7]: € Structures: Selforganizedsystemscreatespatiotemporalstructuresinaninitiallyhomogeneousmedium.Nestarchitectures,foragingtrails,andsocialorganizationaretheexamples ofspatiotemporalstructures. € Coexistence: Coexistenceofseveralstablestates ispossible.Theemergentsystemisthe resultofamplificationofrandomdeviations,dependingontheinitialconditionsthesystem convergestooneamongsever alpossiblestablestates. € Bifurcations: Dependingonthevariationintheparameters,thesystemmaybifurcate.The behaviorofthesystemchangesdrasticallyatbifurcations. 1.2.2Stigmergy Self-organizationinsocialinsectsistheresultofdirectandindirectinteractions.Examplesof directinteractionsarevisualcontact,chemicalcontact,antennation,trophallaxis,andmandibular contact[7].Indirectinteractionsoccurwhenoneindividualchangestheenvironmentandtheothers respondtothenewenvironmentatalatertime.Thissubtleinteractionisanexampleofstigmergy. Stigmergyhelpsinreplacingcoordinationthroughdirectcommunicationsbyindirectinteractions,reducingthecommunicationamongagents.Stigmergyalsoallowstheemergentsystemto be”exible.Theinsectsrespondtoanexternalperturbationasifitwereamodificationofthe environmentcausedbythecolonysactivities.Tha tis,thecolonycollectivelyrespondstotheperturbation[7].Artificialagentsdesignedusingthisprinciplecanrespondtoaperturbationwithout beingreprogrammedtodealwitht hatparticularp erturbation. 1.2.3CemeteryOrganizationandBroodSortinginAnts Manyspeciesofantsclusterdeadbodiestoform cemeteries,andsortthelarvaeintoseveral piles.Thisbehaviorcanbesimulatedusingasimplemodelinwhichtheagentsmoverandomlyin spaceandpickupanddeposititemsonthebasisoflocalinformation.Theclusteringandsorting behaviorofantscanbeusedasametaphorfordesigningnewalgorithmsfordataanalysisand graphpartitioning.Theobjectscanbeconsider edasitemstobesorted.Objectsplacednextto 5

PAGE 17

eachotherhavesimilarattributes.Thissortingtakesplaceintwo-dimensionalspace,offeringa low-dimensionalrepresentationoftheobjects. Thecemeteryorgan izationoftheant Lasiusniger and Pheidolepallidula hasbeenwellstudied [7].Theantsformpilesofcorpses, cemeteries,tocleanuptheirnest s.Itisobservedthatifcorpses arerandomlydistributed,thewo rkersformcemeteryclusters.If theareaisnotlargeorifthere arespatialheterogeneities,theclustersareformedalongtheedgesorfollowingtheheterogeneities [7].Thisaggregationphenomenonisduetotheattractionbetweenthedeaditemsmediatedbythe workerants.Smallclustersofitemsgrowbyattractingworkerstodepositmoreitems,thispositive feedbackleadstotheformationoflargerandlargerclusters. Broodsortingisalsowidespreadinants.Itisobservedintheant Leptothoraxunifasciatus [7].Workerantsgatherlarvaeaccordingtotheir size,alllarvaeofthesamesizetendtocluster together.Anitemisdroppedbytheantifitissurroundedbyitemssimilartotheitemtheantis carrying,anitemispickedupbytheantwhenit perceivesitemsintheneighborhoodwhichare dissimilartotheitemstobepickedup.In Leptothoraxunifasciatus thesmalllarvaearelocatedin thecenterandlargerlarvaearelocatedtowardtheperiphery.Theamountofspaceallocatedtoa brooditemvarieswiththetypeofbrood.Eggsandmicro-larvaeareclusteredcompactlyatthe center,witheachindividualitemgivenlittleindividualspace.Individualspacetendstoincrease towardtheperiphery.Thelargestlarvae,whicharelocatedattheperiphery,areallocatedmore individualspace[7]. 6

PAGE 18

CHAPTER2 DATASETS Sixrealdatasetsandtenartificialdatasetswer etestedinthisthesis.Thedatasetstestedwere € IrisPlantDataset € WineRecognitionDataset € GlassIdentificationDataset € MultipleSclerosisDataset € MRIDataset € BritishTowns'Dataset € Gauss1-5Datasets € Gauss5001-5Datasets ThedatasetsaredescribedinTable2.1andthefollowingsections. Table2.1.Datasets Numberof Numberof Numberof Dataset Examples ContinuousAttributes Classes Iris 150 4 3 Wine 178 13 3 Glass 214 9 6 MRI 65536 3 3 MultipleSclerosis 98 5 2 BritishTowns 50 4 5 Gauss1-5 1000 2 5 Gauss500-1-5 500 2 5 7

PAGE 19

2.1IrisPlantDataset Thisisprobablythemoststudieddatasetinthefieldofpatternrecognition.Thedatasethas informationaboutdifferenttypesofIrisflowers[6]. Thedatasetcontains3classes, IrisSetosa,IrisVersicolour,andIrisVirginica, with50instances ofeachclass.Oneclassislinearlyseparablefromtheothertwo.Theremainingclassesarenot linearlyseparablefromeachother.Therearefour numericpredictiveattributesforeachinstance, theyare € PetalLength € PetalWidth € SepalLength € SepalWidth Tovisualizethedataset,thePrincipalComponentAnalysis(PCA)algorithm[11]isusedto projectthedatapointstoa2Dand3Dspace.Figures2.1and2.2showthescatterofthepoints, afterPCA,in2Dand3Drespectively.Oneclassislinearlyseparablefromtheothertwo.For clusteringpurposes,theirisdatasetcanbeconsideredhavingonly2clusters.Thedatasetwith2 clustersisshowninFigures2.3and2.4. 2.2WineRecognitionDataset ThedataaretheresultsofachemicalanalysisofwinesgrowninthesameregioninItalybut derivedfromthreedifferentcultivars.Quantitiesof13differentconstituentsfoundineachofthe threetypesofwineswasmeasured[6].Theconstituentsare: € Alcohol € Malicacid € Ash € Alcalinityofash € Magnesium € Totalphenols 8

PAGE 20

0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 1st Principal Component2nd Principal Component Irissetosa Irisversicolor Irisvirginica Figure2.1.IrisDataset(Normalized)-First2PrincipalComponents Note:Allvaluesareingenericunits 9

PAGE 21

1 0.5 0 0.5 1 1 0.5 0 0.5 1 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 1st Principal Component 2nd Principal Component3rd Principal Component Irissetosa Irisversicolor Irisvirginica Figure2.2.IrisDataset(Normalized)-First3PrincipalComponents Note:Allvaluesareingenericunits 10

PAGE 22

0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 1st Principal Component2nd Principal Component Irissetosa Irisversicolor & Irisvirginica Figure2.3.IrisDataset:2Classes(Normalized)-First2PrincipalComponents Note:Allvaluesareingenericunits 11

PAGE 23

1 0.5 0 0.5 1 1 0.5 0 0.5 1 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 1st Principal Component 2nd Principal Component3rd Principal Component Irissetosa Irisversicolor & Irisvirginica Figure2.4.IrisDataset:2Classes(Normalized)-First3PrincipalComponents Note:Allvaluesareingenericunits 12

PAGE 24

€ Flavanoids € Nonavanoidphenols € Proanthocyanins € Colorintensity € Hue € OD280/OD315ofdilutedwines € Proline Theclassdistributionis class1: 59 class2: 71and class3: 48.Theprojectionofthedata pointsin2Dand3Dspaceisshowninappendi xA,inFiguresA.1andA.2respectively. 2.3GlassIdentificationDataset Thestudyoftypesofglasswasmotivatedbyitsimmensepotentialincriminology.Theglassat thesceneofcrime,ifidentifiedcorrectly,canbe usedasevidence.Thedat asethas214examplesof sixdifferenttypesofglass.Eachexamplehas9attributes,givingthequantityofdifferentchemical elementspresentintheglas s[6].Theattributesare: € RefractiveIndex € Na:Sodium € Mg:Magnesium € Al:Aluminum € Si:Silicon € K:Potassium € Ca:Calcium € Ba:Barium € Fe:Iron 13

PAGE 25

Theprojectionofthedatapointsin2Dand3Dspa ceisshowninFiguresA.3andA.4respectively. Theglassdatasetcanbeconsideredashavingtwoclasses windowglass and non-windowglass .The projectionofthedataconsideringtwoclassesisshowninFiguresA.5andA.6. 2.4MultipleSclerosisDataset TheMultipleSclerosisdatasethas98instanceseachhaving5numericcontinuousattributes. Theprojectionofthedatapointsin2Dand3Dspa ceisshowninFiguresA.7andA.8respectively. 2.5MRIandBritishTownsDatasets TheMRIdatasethas65,536instances,eachwith3numericcontinuousattributes,from3classes. TheBritishTownsdatasethas50instances,eachwith4numericcontinuousattributes,from5 classes. 2.6ArtificialDatasets TenartificialdatasetsweregeneratedusingGaussiandistributions.AmixtureoffiveGaussians wasusedtogeneratethedata.Theprobabilitydistributionacrossallthedatasetsisthesamebut themeansandstandarddeviationsoftheGaussiansaredifferent.Ofthetendatasets,fivedatasets had500instanceseachandtheremainingfivedatasetshad1000instanceseach.Eachexamplehad twoattributes.ThedatasetsareshowninFiguresA.9…A.18.Theparametersusedtogeneratethe datasetsareshownintheAppendixA.1. 14

PAGE 26

CHAPTER3 FUZZYANTSALGORITHM The FuzzyAnts algorithmisbasedon SwarmIntelligence principlesappliedtoclusterdata. Dataisclusteredwithouttheinitialknowledgeofthenumberofclusters.Weuseantbased clusteringtoinitiallycreaterawclustersandth entheseclustersarerefinedusingtheFuzzyC Meansalgorithm.Initiallytheantsmovetheindividualobjectstoformheaps.Thecentroidsof theseheapsaretakenastheinitialcluster centersandtheFuzzyCMeansalgorithmisusedto refinetheseclusters.Inthesecondstagetheo bjectsobtainedfromtheFuzzyCMeansalgorithm arehardenedaccordingtothemaximummembershipcriteriontoformnewheaps.Thesenewheaps arethensometimesmovedandmergedbytheants. Thefinalclustersformedarerefinedbyusing theFuzzyCMeansalgorithm. InpastresearchtheKmeansclusteringalgorithmhasbeenusedonthecentersobtainedfrom theantbasedalgorithmasintroducedin[26];herewestudytheeffectofusingtheFuzzyCmeans approachontheclustercentersobtainedfromtheantbasedalgorithm.In[26]theFuzzyK-means algorithmwasusedtorefinetheclustersfoundbytheants.In[4]theantsystemandtheK-means algorithmwereusedfordocumentclustering. Thegeneraloutlineoftheantbasedalgorithmusedinthisstudywasproposedin[26].Initially theobjectsarescatteredrandomlyonadiscrete2Dboard.Theboardcanbeconsideredamatrix of m m cells.Thematrixistoroidalwhichallowstheantstotravelfromoneendtoanother easily.Thesizeoftheboardisdependentonthenumberofobjects.Wehaveusedaboardof m m suchthat m2=4 n where n isthetotalnumberofobjectstob eclustered.Initiallytheants arerandomlyscatteredthroughouttheboard.Therearen 3ants,where n isthetotalnumberof objectstobeclustered. Theantsclustertheobjectstoformheaps.A heapisdefinedasacollectionof2ormore objects.Aheapisspatiallylocatedinasinglecell. 15

PAGE 27

Consideraheap H with nHobjects,thenwedefinethefollowingparameters: € Themaximumdistancebetweentwos-dimensionalobjectsintheheap Dmax( H )= maxXi,XjHD ( Xi,Xj) Where D istheeuclideandistancebetweentheobjects. € Thecenterofmassofalltheobjectsintheheap Ocenter( H )= 1 nHOiHOi€ Themostdissimilarobjectintheheap Odissim( H ):Itistheobjectwhichisthefarthestfrom thecenteroftheheap. € Themeandistancebetweentheobjectsof H andthecenterofthemassoftheheap Dmean( H )= 1 nHOiHD ( Oi,Ocenter( H )) ThemainantbasedclusteringalgorithmispresentedinFigure3.1[26]. 1.Randomlyplacetheantsontheboard.Randomlyplaceobjectsontheboardat mostonepercell 2.Repeat 3.ForeachantDo 3.1Movetheant 3.2Iftheantdoesnotcarryanyobjectthenifthereisanobjectinthe8neighboringcellsoftheant,theantpossiblypicksuptheobject, 3.3Elsetheantpossiblydropsacarriedobject,bylookingatthe8neighboring cellsaroundit. 4.Untilstoppingcriteria. Figure3.1.TheAntBasedAlgorithm Initiallytheantsarescatteredrandomlyonthe2Dboard.Theantmovesontheboardand possiblypicksupanobjectordropsanobject.Themovementoftheantisnotcompletelyrandom. Initiallytheantpicksadirectionrandomly,thentheantcontinuesinthesamedirectionwitha 16

PAGE 28

probability Pdirection,otherwiseitgeneratesanewrandomdirection.Onreachingthenewlocation ontheboardtheantmaypossiblypickupanobjectordropanobject,ifitiscarryingone.The heuristicsandtheexactmechanismforpickingupordroppinganobjectareexplainedbelow.The stoppingcriterionfortheants,here,istheupperlimitonnumberoftimesthroughtherepeatloop. ThedifferentstepsofthealgorithmareshowninFigures3.2and3.3. 17

PAGE 29

Figure3.2.Algorithm 18

PAGE 30

Figure3.3.Algorithm(cont...) 19

PAGE 31

3.1PickingupanObject Whentheantisnotcarryinganyobject,itsearc hesforpossibleobjectstopickupbyexamining theeightneighboringcellsarounditscurrentposition.Ifanobjectorheapisfoundthentheant possiblypicksupanobject.Theheuristicforpickingupanobjectdependsonthenumberof objectsintheheap.Threecasesareconsidered :onlyoneobject,aheapoftwoobjectsandaheap ofmorethantwoobjects.Ifasingleobjectispresentthentheanthasafixedprobabilityofpicking itup.Ifthereisaheapoftwoobjectsthenwithaprobability Pdestroytheantdestroystheheap bypickingarandomobjectfromtheheap.Inthethirdcasetheantpicksupthemostdissimilar objectfromtheheapifthedissimilarityisaboveagiventhreshold Tremove.Thealgorithmfor pickingupanobjectisgiveninFigure3.4[26]. 1.Markthe8neighboringcellsaroundtheantas unexplored' 2.Repeat 2.1Considerthenextunexploredcellaroundtheant 2.2Ifthecellisnotemptythen 2.2.1Ifthecellcontainsasingleobject X ,thentheobject X ispickedupwith aprobability Pload,else 2.2.2Ifthecellcontainsaheapoftwoo bjects,thentheheapisdestroyedby pickinguparandomobjectwithaprobability Pdestroyelse 2.2.3Ifthecellcontainsaheap H ofmorethan2objects,thenthemost dissimilarobject, Odissim( H ),of H isremovedonlyif D ( Odissim( H ) ,Ocenter( H )) Dmean( H ) >Tremove2.3Labelthecellas explored' 3.Untilalltheneighboringcellshavebeenexploredoroneobjecthasbeenpicked up. Figure3.4.AlgorithmtoPickupanObject 3.2DroppinganObject Whentheantiscarryinganobject,itexaminesthe8cellssurroundingitscurrentlocation. Threecasesareconsidered:thecellisempty,thecel lcontainsoneobjectonly,andthecellcontains 20

PAGE 32

aheap.Inthefirstcasetheanthasaconstantprobabilityofdroppingtheobject.Inthesecond caseaheapiscreatediftheobjectcarriedissuffi cientlysimilartotheonealreadyinthecell.In thethirdcasetheantwilladditsobjecttotheheapiftheobjectiscloserto H scenterthanthe mostdissimilar objectof H .ThealgorithmfordroppingtheobjectisgiveninFigure3.5[26]. 1.Markthe8neighboringcellsaroundtheantas unexplored' 2.Repeat 2.1Considerthenextunexploredcellaroundtheant 2.1.1Ifthecellisemptythentheobjectcarriedbytheant, X ,isdroppedwith aprobability Pdropelse 2.1.2Ifthecellcontainsasingleobject X' thenaheapoftwoobjectsiscreated bydropping X on X' onlyifD ( X,X) Dmax
PAGE 33

Inthesecondstageweconsidertheheapsform edbythefirststageandmovetheentireheap onthe2Dboard.Theantscarryanentireheapofobjects.Thealgorithmforpickingupaheap isthesameasthatfortheobjects.Antswillpicktheheapwiththesameprobability Pload.Ants dropaheap H1ontoanotherheap H2providedthat: D ( Ocenter( H1) ,Ocenter( H2)) Dmax
PAGE 34

Table3.1.ValuesoftheParametersUsedintheExperiments Parameter Value Niterations 1000 Tcreate 0.5 Pdrop 0.2 Pdestroy 0.3 Pload 0.3 Tremove(Iris) 1.5 Tremove(Wine) 3.0 Tremove(Glass) 2.0 Tremove(Gauss) 1.5 3.4Results Thereportedresultsareaveragedfor50runsoftheexperiments.Ineachrun,theantsand theobjectswereinitiallyrando mlyplacedatdifferentpositionsontheboard.Themovementof antsandthepickinganddroppingoftheobjectsalsohadastochasticcomponent.Weperformed experimentsusingdifferentparametervalues .Byvaryingtheparameterswecouldcontrolthe numberofheapsobtained.Intheexperimentsallbutoneparameterisfixed.Theparameter Tcreateforheapisvaried.Thisparameteristhethresholdforthemaximumdissimilarityallowed whilemergingtheheaps.TheoriginalparametersareshowninTable3.1,theresultsfordifferent parametervaluesareshownbelow.Theaverageresultsfor50runsoftheFCMalgorithmwith randominitializationsareinTable3.2.Wereporterrorswhichmaynotbethefairestmeasure giventhealgorithmisoptimizingafunctionwhichindirectlyrelatestoerrors. 3.4.1IrisDataset ResultsfortheIrisDatasetareshowninTable3.3.As Tcreateforheapisincreasedfewerclusters arefound.Everytimetwoclustersarefound,whichis3timeswiththevalueof0.12,19times withthevalueof0.14,and41timeswiththevalueof0.18,atleast50errorsareobserved.Thisis becausetheseparableclassisalmostalwayscorrectlysplitoffwiththeotherclusterconsistingof amixtureoftheothertwoclusters. IntheIrisdatasetfeaturespaceoneclassislinearlyseparablefromtheothertwo,sostrictly speakingonecouldcometothedet erminationthatthereareonlytw oclusters.Hence,theresults 23

PAGE 35

Table3.2.ResultsforFCMAlgorithm(Avg.from50randominitializations) Dataset Classes Errors Iris 3 16 Iris 2 3 Wine 3 9 Glass 6 96 Glass 2 20 Gauss1 5 0 Gauss2 5 0 Gauss3 5 0 Gauss4 5 25.32 Gauss5 5 7 Gauss500-1 5 1 Gauss500-2 5 0 Gauss500-3 5 1 Gauss500-4 5 1.94 Gauss500-5 5 0 fortheIrisdatasetconsidering2classesareshowninTable3.4.Inthiscaseifweallowuptofour classestobefound,whileoverclusteringisdone,wealwaysgethomogeneousclusters. Table3.3.ResultsfortheIrisDataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.18 2.18 49.76 43.88 0.14 2.76 45.92 29.42 0.12 3.74 36.34 19.42 0.10 5.12 25.56 15.16 3.4.2WineRecognitionDataset TheresultsfortheWineRecognitiondatasetareshowninTable3.5.Wegetmoreerrorsthan randomlyinitializedFCMbecausetheantssometimesfindonly2clustersandtheerrorsincrease dramaticallybecauseofthisunderclustering.Also,whenmorethan4clustersarefoundtheerrors shootup. 24

PAGE 36

Table3.4.ResultsfortheIrisDatasetConsidering2Classes Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.20 2.02 3.74 2.94 0.18 2.14 1.16 2.58 0.12 3.74 0 0.18 Table3.5.ResultsfortheWineRecognitionDataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.085 4.46 13.94 9.36 0.09 3.9 14.3 9.02 0.1 3.76 23.08 10.4 3.4.3GlassIdentificationDataset Thisisperhapsthemostdifficultdatasettocluster.TheresultsfortheGlassIdentification datasetareshowninTable3.6.Ifweallowoverclusteringthentheerrorsarealittlebetterthan therandomlyinitializedFCM,butasweclosei nonthecorrectnumberofclusters,sometimes underclusteringoccursandtheerrorsshootup. Theglassdatasetcanbeconsid eredasatwoclusterdataset, windowglass and non-window glass .TheresultsforthismodifieddatasetareshowninTable3.7.Forthisdatasetbothover clusteringandunderclusteringincreasetheerrors,forthisreasontheerrorsfortheantalgorithm arealwaysgreaterthanforrandomlyinitializedFCM. Table3.6.ResultsfortheGlassIdentificationDataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.075 9.36 102.38 88.2 0.085 8.08 103.76 93.34 0.105 5.04 111.52 99.72 0.12 4.54 115.72 102.9 0.16 2.56 136.82 118.94 25

PAGE 37

Table3.7.ResultsfortheGlassIdentificationDatasetConsidering2Classes Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.12 4.54 22.52 21.26 0.135 2.62 40.64 25.36 0.15 2.22 49.7 27.3 0.16 2.56 49.84 26.22 0.18 2.16 49.96 28.04 0.20 1.78 50.12 31.58 3.4.4ArtificialDatasets Tenartificialdatasets,eachwith2attributes ,weregeneratedusingaGaussiandistribution. TheresultsforthedatasetsareshowninTables3.8…3.17. FortheGauss-1,Gauss-2andGauss-3datasets,overclusteringandexactclusteringgivesus perfectclustering,thatistherearenoerrors.Buttheerrorsincreasedramaticallyonunderclustering. TheGauss-4dataset,with5clustershastwoextremaonewith0errorsandonewith211errors. Whenoverclusteringoccurswesometimesget 211errors,becauseofthiswedontgetperfect clusteringforthisdataset. TheGauss-5dataset,with5clustershas7errors.O verclusteringresultsincomparableerrors, butunderclusteringdramaticallyincreasestheerrors. FortheGauss500-1datasetoverclusteringandexactclusteringresultsin1error,butbecause ofunderclusteringtheerrorsshootup. FortheGauss500-2datasetwegetperfectclusteringifweoverclusterorgetanexactclustering, butunderclusteringincreasestheerrors. FortheGauss500-3dataset,overclusteringdoe sntdramaticallyincr easetheerrors,butunder clusteringdoes.Also,wedontgetperfectclusteringforthisdataset. FortheGauss500-4andGauss500-5datasetswegetperfectclusteringifweoverclusterorget anexactclustering,butbecauseofunderclusteringtheerrorsshootup. 26

PAGE 38

Table3.8.ResultsfortheGauss-1Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.19 8.26 130.26 0 0.22 7.44 181.14 9.5 0.235 6.56 268.16 2.64 0.25 5.84 348.58 25.34 Table3.9.ResultsfortheGauss-2Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.19 9.76 27.08 0 0.225 7.96 136.92 0 0.25 6.96 212.68 5.28 0.28 5.86 369.68 15.84 0.295 4.60 492.08 118.14 0.30 4.48 560.24 136.08 Table3.10.ResultsfortheGauss-3Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.25 7.44 191.9 0 0.265 6.56 276.18 13.2 0.28 5.66 380.96 46.98 Table3.11.ResultsfortheGauss-4Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.19 6.32 318.3 32.18 0.225 6.1 327.08 40.62 0.25 6.42 328.04 27.96 0.27 6.24 327.42 33.76 0.28 5.86 327.96 37.98 27

PAGE 39

Table3.12.ResultsfortheGauss-5Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.19 8.1 188.42 8.06 0.205 6.8 252.96 10.34 0.215 6.3 329.4 20.56 0.22 5.94 375.28 58.22 0.25 4.4 552 119.84 Table3.13.ResultsfortheGauss500-1Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.185 5.7 54.3 13.04 0.19 5.68 63.28 8.56 0.20 5 79.38 25 Table3.14.ResultsfortheGauss500-2Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.17 6.1 9.76 0 0.19 5.64 37.36 7.6 0.21 5.04 75.68 30.4 Table3.15.ResultsfortheGauss500-3Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.155 5.96 17.98 3.82 0.17 5.42 50.48 12.92 0.185 5.10 72.22 20.46 0.19 5.02 72.9 21.96 28

PAGE 40

Table3.16.ResultsfortheGauss500-4Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.19 6.36 1.48 0 0.195 6.34 3.62 0 0.22 5.18 62.3 32.14 0.25 4.02 153.18 90.8 0.28 3.94 175.16 97.3 Table3.17.ResultsfortheGauss500-5Dataset Classes Errorsafter Errorsafter Tcreateforheap found antstage FCM 0.19 5.86 11.84 3.04 0.195 5.78 20.96 6.08 0.20 5.96 23.32 4.56 3.5EffectofParameterVariation Theaboveresultsshowthesensitivityofthealgorithmtothe Tcreateforheapparameter.The algorithmisnotverysensitivetotheotherparameters.Weshowtheresultsforthesensitivityof thealgorithmtothenumberofiterationsinTables3.18…3.19. FortheIrisdataset,thenumberofclustersfoundincreasesfortwovaluesof Tcreateforheapand decreasesforonevalueof Tcreateforheap.Thenumberofclustersdifferbecausetheantsgetmore timetomoveandcreatenewheapsordumpobjectstoexistingheapsinthefirststage,andmerge heapsinthesecondstage.FortheWinedatasetthenumberofclustersfounddecreasesforallthe differentvaluesof Tcreateforheap. 3.6VariationsintheAlgorithm Themergingandcreationofnewheapsdependonthenumberofneighborsoftheant.Ifthe numberofneighborsincreasesthentheanthasahigherprobabilityoffindingaheapamongits neighbors.Ifweconsidera3Dboard,insteadofa2Dboard,thenumberofneighborsincreases from8to26,thisgivestheantahigherprobabilityoffindingaheap. 29

PAGE 41

Table3.18.EffectofVariationofIterationsonIrisDataset Classes Errorsafter Errorsafter Tcreateforheap Iterations found antstage FCM 0.10 1000 5.12 25.56 15.16 0.10 2000 5.14 26.10 14.26 0.10 3000 5.30 26.56 13.68 0.12 1000 3.74 36.34 19.42 0.12 2000 3.64 35.76 17.68 0.12 3000 3.78 36.50 18.50 0.14 1000 2.76 45.92 29.42 0.14 2000 2.82 46.54 33.36 0.14 3000 2.92 45.14 27.76 0.18 1000 2.18 49.76 43.88 0.18 2000 2.12 49.90 46.24 0.18 3000 2.10 49.92 46.92 Table3.19.EffectofVariationofIterationsonWineDataset Classes Errorsafter Errorsafter Tcreateforheap Iterations found antstage FCM 0.085 1000 4.46 13.94 9.36 0.085 2000 4.38 13.88 9.52 0.085 3000 4.22 14.88 9.38 0.09 1000 3.9 14.3 9.02 0.09 2000 3.8 15.48 10.32 0.09 3000 3.54 15.84 8.86 0.10 1000 3.76 23.08 10.04 0.10 2000 3.56 25.52 11.52 0.10 3000 3.46 26.76 10.16 Wetriedtwoapproachesforthe3Dboard,inthefirstapproach,weconsidereda3Dboardwith z=3,thatisthe3Dboardcanbeconsideredasthreeseparate2Dboards.Theantscanmovefrom one2Dboardtotheother.Inthesecondapproach,allthethreedimensionsareequal.Apointto benotedisthatasthez-dimensionincreases,thesizeofthecorresponding2Dboarddecreases. TheresultsareshowninTables3.20…3.29.Thevaluesofalltheparameters,excepttheboard dimension,arethesameastheoriginal2Dboard.Forallthedatasets,exceptIris(2class),the 30

PAGE 42

numberofclustersfoundbyusingthe3Dboardislessthanthatfoundbyusingthe2Dboard. Thisistobeexpected,becauseasthedimensionincreases,thenumberofneighborsincrease,which inturnincreasestheprobabilityoffindingahea p.FortheGauss500datasets,exceptGauss500-3 dataset,the3Dboardbasedalgorithmalwaysfindsperfectclustering,whichwasnotpossiblewith the2Dboard. For5outofthe10datasets,the3Dboardwithz=3foundfewerclustersthanthe3Dboard withequaldimensions.Fortheremaining5datasets,mostofthetimethethe3Dboardwithz=3 foundfewerclusters.FortheGauss500-3andtheGauss500-5datasetsthe3Dboardwithequal dimensionsfoundfewerclustersthanthe3Dboardwithz=3. Table3.20.ResultsfortheIrisDataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.12 3.74 36.34 19.42 3D(z=3) 0.12 3.38 36.38 22.56 3D(x=y=z) 0.12 3.46 36.92 20.4 2D 0.14 2.76 45.92 29.42 3D(z=3) 0.14 2.72 45.78 30 3D(x=y=z) 0.14 2.72 46.92 30.08 3D(z=3) 0.13 2.84 43.14 31.32 3D(x=y=z) 0.13 3.06 43.16 25.28 Table3.21.ResultsfortheIrisDataset(2Class)(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.12 3.74 0 0.18 3D(z=3) 0.12 3.38 0 0.42 3D(x=y=z) 0.12 3.46 0 0.24 2D 0.18 2.14 1.16 2.58 3D(z=3) 0.18 2.06 0.74 2.82 3D(x=y=z) 0.18 2.1 0.7 2.7 2D 0.2 2.02 3.74 2.94 3D(z=3) 0.2 2.02 4.44 3.82 3D(x=y=z) 0.2 2.12 3.78 2.64 31

PAGE 43

Table3.22.ResultsfortheWineDataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.085 4.46 13.94 9.36 3D(z=3) 0.085 3.68 14.26 8.78 3D(x=y=z) 0.085 3.8 16.00 8.76 2D 0.09 3.9 14.30 9.02 3D(z=3) 0.09 3.2 15.26 8.86 3D(x=y=z) 0.09 3.24 16.44 8.76 2D 0.1 3.76 23.08 10.04 3D(z=3) 0.1 3.16 25.02 12.58 3D(x=y=z) 0.1 3.12 26.66 10.12 Table3.23.ResultsfortheGlassDataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.075 9.36 102.38 88.2 3D(z=3) 0.075 8.7 104.22 90.98 3D(x=y=z) 0.075 8.92 103.82 89.58 2D 0.08 8.08 103.76 93.34 3D(z=3) 0.085 6.42 106.46 95.36 3D(x=y=z) 0.085 6.44 106.44 95.74 2D 0.105 5.04 111.52 99.72 3D(z=3) 0.105 3.9 112.6 107.42 3D(x=y=z) 0.105 3.96 112.56 107.12 32

PAGE 44

Table3.24.ResultsfortheGlass(2Class)Dataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.12 4.54 22.52 21.26 3D(z=3) 0.13 2.92 33.7 23.36 3D(x=y=z) 0.13 2.84 32.56 23.46 3D(z=3) 0.135 2.48 40.36 23.74 3D(x=y=z) 0.135 2.6 38.36 24.22 3D(z=3) 0.14 2.1 47.3 28.96 3D(x=y=z) 0.14 2.26 46.14 29.22 2D 0.15 2.22 49.7 27.3 3D(z=3) 0.15 1.64 50.5 33.76 3D(x=y=z) 0.15 1.86 50.28 31.7 2D 0.16 2.56 49.84 26.22 3D(z=3) 0.16 1.56 50.86 35 3D(x=y=z) 0.16 1.84 50.42 32.32 2D 0.18 2.16 49.96 28.04 3D(z=3) 0.18 1.24 50.88 44.92 3D(x=y=z) 0.18 1.3 50.62 44.42 33

PAGE 45

Table3.25.ResultsfortheGauss500-1Dataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 3D(z=3) 0.17 4.98 26.76 2.5 3D(x=y=z) 0.17 5 23.66 1 3D(z=3) 0.175 4.94 31.24 5.5 3D(x=y=z) 0.175 4.98 30.72 4 3D(z=3) 0.18 4.8 43.18 16 3D(x=y=z) 0.18 4.9 39.44 8.5 2D 0.185 5.7 54.3 13.04 3D(z=3) 0.185 4.66 51.92 28 3D(x=y=z) 0.185 4.86 45 11.5 2D 0.19 5.68 63.28 8.56 3D(z=3) 0.19 4.62 59.42 29.5 3D(x=y=z) 0.19 4.64 56.52 28 2D 0.2 5 79.38 25 3D(z=3) 0.2 4.26 76.94 56.5 3D(x=y=z) 0.2 4.24 77.84 58 Table3.26.ResultsfortheGauss500-2Dataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.17 6.1 9.76 0 3D(z=3) 0.17 5 9.58 0 3D(x=y=z) 0.17 5 8.18 0 2D 0.19 5.64 37.36 7.6 3D(z=3) 0.19 4.7 37.42 22.8 3D(x=y=z) 0.19 4.76 35.44 19.76 2D 0.21 5.04 75.68 30.4 3D(z=3) 0.21 4.1 74.72 70.36 3D(x=y=z) 0.21 4.18 74.46 63.84 34

PAGE 46

Table3.27.ResultsfortheGauss500-3Dataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 3D(z=3) 0.155 4.92 15.98 7 3D(x=y=z) 0.155 4.92 16.5 7 3D(z=3) 0.16 4.84 23.88 13 3D(x=y=z) 0.16 4.8 27.1 16 3D(z=3) 0.165 4.66 32.62 26.5 3D(x=y=z) 0.165 4.58 40.12 32.5 2D 0.17 5.42 50.48 12.92 3D(z=3) 0.17 4.48 46.08 40 3D(x=y=z) 0.17 4.48 48.9 40 2D 0.185 5.1 72.22 20.46 3D(z=3) 0.185 4.08 73.2 70 3D(x=y=z) 0.185 4.06 73.22 71.5 2D 0.19 5.02 72.9 21.96 3D(z=3) 0.19 4 76.08 76 3D(x=y=z) 0.19 4.06 75.82 71.5 Table3.28.ResultsfortheGauss500-4Dataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 2D 0.195 6.34 3.62 0 3D(z=3) 0.195 5 4.1 0 3D(x=y=z) 0.195 5 3.66 0 2D 0.22 5.18 62.3 32.14 3D(z=3) 0.22 4.38 66.74 59.72 3D(x=y=z) 0.22 4.62 42.44 36.02 3D(z=3) 0.235 3.8 116.28 110.52 3D(x=y=z) 0.235 4 101.72 94.48 2D 0.25 4.02 153.18 90.8 35

PAGE 47

Table3.29.ResultsfortheGauss500-5Dataset(3DBoard) Board Classes Errorsafter Errorsafter type Tcreateforheap found antstage FCM 3D(z=3) 0.18 5.02 2.4 0 3D(x=y=z) 0.18 5 4.32 0 3D(z=3) 0.185 4.98 6.6 3.04 3D(x=y=z) 0.185 4.98 7.3 1.52 2D 0.19 5.86 11.84 3.04 3D(z=3) 0.19 4.96 10.46 4.56 3D(x=y=z) 0.19 4.92 11.88 7.6 2D 0.195 5.78 20.96 6.08 2D 0.2 5.96 23.32 4.56 3D(z=3) 0.2 4.7 29.64 22.8 3D(x=y=z) 0.2 4.78 23.84 16.72 3D(z=3) 0.215 4.1 79.3 70.5 3D(x=y=z) 0.215 4.1 85.08 70.5 3.7Discussion Theuseofantsfortheclusteringprocessisonewaytodeterminethenumberofclusters. However,theantsareclearlysensitivetothet hresholdfordecidingwhentomergeheaps.The originalworkinthisareaprovidedvalues,butnojustificationorwaytosetthem.Wehave exploredarangeofvalues(albeitforoneparameter)andshownhowtheresultsdiffer.Wehave notyetfoundasystematicwaytosetthevaluesoftheparameters.Thefinalpartitionfoundwith 3classesfortheIrisdatasetisalwaysequivalenttowhatwegetwithFCM. Essentially,theantsarefindingthenumberofclustersandaninitializationforFCM.They arenotreallyproducingafinalpartition.Adifficultywithtakingwhattheyproduceasafinal partitionisthatnothingcanberemovedfromah eapwhenheapsarebeingcombined.Thiscanbe problematic. Forcomparisonpurposes,Table3.30showsthenumberoftimesapartitionofeachclasssize wasfoundforparticularsettingofparametersforthewinedataset.Itcanbeseenthatthesecond applicationofFCMusually,butnotalwaysimprovesthepartition. 36

PAGE 48

TheresultsfortheIrisdataset,whenconsider edasatwoclusterdataset,werebetterthanthe randomlyinitializedFCMalgorithmbecausesom etimesoverclusteringoccursandwegetperfect clustering.AlsotheantsfindabetterpartitionthanFCM,becausetheantsareabletosplitoff thelinearlyseparableclusters. Also,wefoundthattheclusteringwashighlysensitiveto Tcreateforheap. Tcreateforheapclearly hasastrongin”uenceonhowmanyfinalclustersareobtained. In[4],asimplerapproachthantheonewediscussisusedtoclusterdocuments.Theauthors utilizek-meansaftertheantsfindtheinitialclustercentersandfindthenumberofcenters.It wasstartlingtousthat,over10runs,theantsfindonaverageanon-fractionalnumberofcluster centers.Utilizingtheantbasedinitialization,theaccuracyofthefinalclusterpartitionwasbetter thanjustusingK-means.Theyindicatedthattheirimplementationofk-meanswassensitiveto theorderofdata,whichsuggestsanonstandardimplementation. Anotheravenuewehavepursuedistoallowtheantstorelocateclusterscentroidsinfeatures space.Theformulationisthesameataveryhigh-levelaswasdonein[14],butantsareutilized ratherthanageneticapproach. Table3.30.TypicalExampleofVarianceinDataPartitionsObtainedwiththeWineDatasetwith Tcreateforheap=0.09and3000Iterations Clusters Errorsafter Errorsafter Found Frequency Antstage FCM 3 13 8.9230 9.00 4 18 14.6111 8.00 5 15 17.3333 11.00 6 3 24.00 11.00 7 1 33.00 10.00 37

PAGE 49

CHAPTER4 AUTOMATICALLYSETTINGTHEPARAMETERS Theantalgorithmisverysensitivetothethresholdfordecidingwhentomergeheapsand removetheitemsfromtheheap.Resultsfromthepreviouschaptershowthesensitivityofthe algorithmtothe Tcreateforheapparameter.Thealgorithmislesssensitivetotheotherparameters. 4.1NewMetricforMergingHeaps Wetrieddifferentapproachestoautomaticallysetthe Tcreateforheapparameter. Theoriginalapproachformergingheapsis Ifthedistancebetweenthecentroidoftheheapcarriedbytheantandthecentroidoftheheapon theboardislessthanaconstant Tcreateforheapthentheheapsaremerged. Antsdropaheap H1ontoanotherheap H2providedthat: D ( Ocenter( H1) ,Ocenter( H2)) Dmax
PAGE 50

thesame percentagevalue doesntworkacrossthedatasets.FortheIrisdataset,2.40isthe best percentagevalue .Butforthewinedatasetthevalueof2.40istoohigh,itresultsinonlyonecluster. Thevalueof1.25isthebestforthewinedataset. Table4.1.ResultsfromtheNewMetricforIrisDataset Classes Errorsafter Errorsafter Percent Iterations found AntStage FCM 2.25 1000 4.18 47.16 21.14 2.30 1000 3.72 47.02 22.08 2.35 1000 3.56 47.64 21.68 2.40 1000 3.28 49.22 25.12 2.45 1000 2.88 49.70 29.90 Table4.2.ResultsfromtheNewMetricforWineDataset Classes Errorsafter Errorsafter Percent Iterations found AntStage FCM 2.40 1000 1 107 107 2.00 1000 1 107 107 1.25 1000 3.44 31.18 22.06 1.00 1000 4.62 13.12 9.42 Theeffectofvaryingthenumberofiterations,usingthenewmetric,isshowninTables4.3and 4.4.ForboththeIrisandWinedatasetsasthenumberofiterationsincrease,theclustersfound decrease.Thisistobeexpected,becauseasthenu mberofiterationsincr easetheantsgetmore timetosearchtheboardandmergetheheaps. 39

PAGE 51

Table4.3.EffectofVariationofIterationsonIrisDataset Classes Errorsafter Errorsafter Percent Iterations found AntStage FCM 2.25 1000 4.18 47.16 21.14 2.25 4000 3.48 48.94 21.60 2.30 1000 3.72 47.02 22.08 2.30 4000 3.22 49.16 23.42 2.35 1000 3.56 47.64 21.68 2.35 4000 3.12 49.28 28.20 2.40 1000 3.28 49.22 25.12 2.40 3000 2.90 49.52 29.36 2.40 4000 2.94 49.40 29.40 2.45 1000 2.88 49.70 29.90 2.45 4000 2.60 51.00 33.42 Table4.4.EffectofVariationofIterationsonWineDataset Classes Errorsafter Errorsafter Percent Iterations found AntStage FCM 2.45 1000 1.00 107.00 107 2.45 3000 1.00 107.00 107 2.00 1000 1.00 107.00 107 2.00 3000 1.00 107.00 107 1.25 1000 3.44 31.18 22.06 1.25 3000 3.04 37.24 28.28 1.00 1000 4.62 13.12 9.42 1.00 3000 4.32 14.40 9.26 4.3FuzzyHypervolume Traditionalalgorithmsfordeterminingthenumberandstructureofclustersincorporatesome formof PartitionEntropy [9,13,25,32,21,22,24,33].Themembershipmatrix U isusedto computetheentropyofthepartition.Themembershipoftheobjectincluster uijisinterpreted astheprobabilityfortheobject j tobeincluster i [25].Thenumberofclustersisvariedandthe 40

PAGE 52

partitionentropyiscomputed.Theoptimumnumberofclustersistheonecorrespondingtothe minimumpartitionentropy. In[12]theauthorshaveusedtheFuzzyHypervolumeanddensitycriteriatofindthenumberof clusters.Theexponentialdistancemeasure, d2 e( Xj, Vi),basedonthemaximumlikelihoodestimationisused.Theexponentialdistanceisusedincalculationoftheposteriorprobabilityofselecting the ithclustergiventhe jthfeaturevector h ( i | Xj). h ( i | Xj)=1 d2 e( Xj,Vi) K k =1 1 d2 e( Xj,Vk)(4.1) d2 e( Xj,Vi)= [ det ( Fi)]1 2 Piexp ( XjŠ Vi)TF 1 i( XjŠ Vi) 2 (4.2) where € Fi: Fuzzycovariancematrixofthe ithcluster € Pi:aprior probabilityofselectingthe ithcluster Thefuzzycovariancematrixofthe ithclusteriscomputedas: Fi= N j =1h ( i | Xj)( XjŠ Vi)( XjŠ Vi)T N j =1h ( i | Xj) (4.3) TheFuzzyHypervolumeisdefinedas: FHV=Ki =1[ det ( Fi)]1 2(4.4) Thefuzzyhypervolumeofthedatasets,assuminguniformpriorprobabilities,wascomputedto findabasisofsettingthe Tcreateforheapparameter.Thehypothesisisthatthereisacorrelation between Tcreateforheapandthefuzzyhypervolume. Tcreateforheapcouldbemodeledasafunction ofthehypervolumeandcouldbeautomaticallysetatruntime.Thefuzzyhypervolumeforthe datasetswiththebestvaluefoundfor TcreateforheapisshowninTable4.5. 41

PAGE 53

Table4.5. TcreateforheapandFuzzyHypervolume Dataset Tcreateforheap FuzzyHypervolume FuzzyHypervolume(log) Iris 0.12 3.5186E-04 -3.453633681 Wine 0.09 1.5980E-11 -10.79641912 Glass 0.085 5.7413E-10 -9.240991303 Gauss-1 0.235 9.4408E-02 -1.024993149 Gauss-2 0.25 1.0947E-01 -0.960688101 Gauss-3 0.265 1.0682E-01 0.971361128 Gauss-4 0.25 1.0775E-01 -0.967596587 Gauss-5 0.21 7.1804E-02 -1.143849094 Gauss500-1 0.2 7.8007E-02 -1.107865773 Gauss500-2 0.19 9.0652E-02 -1.0426242 Gauss500-3 0.185 8.5193E-02 -1.069593575 Gauss500-4 0.195 8.6286E-02 -1.064060967 Gauss500-5 0.195 7.9956E-02 -1.097149734 Theplotsof TcreateforheapvsFuzzyHypervolumeand TcreateforheapvsFuzzyHypervolume (log)areshowninFigure4.1and4.2.Fromtheaboveresultswecanconcludethatthetwo parametersarenotcorrelated. 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0 0.02 0.04 0.06 0.08 0.1 0.12 TcreateforheapFuzzy Hypervolume Figure4.1.TcreateforheapvsFuzzyHypervolume 42

PAGE 54

0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26 0.2 8 12 10 8 6 4 2 0 TcreateforheapFuzzy Hypervolume (log) Figure4.2.TcreateforheapvsFuzzyHypervolume(Log) 43

PAGE 55

CHAPTER5 FUZZYANTCLUSTERINGWITHCENTROIDS 5.1Introduction Previousantbasedclusteringalgorithmsclusterdatabymovingtheobjectsina2Dspaceand mergingthemtoformclusters.Similarobjectsten dtomergetogethertoformheaps.Thismerging iscontrolledbyathresholdwhichdictatesthep ermissibledissimilaritybetweentheobjectsina cluster.Theantbasedclusteringalgorithmsareverysensitivetothisthreshold[19].Thealgorithm presentedinChapter3findsthenumberofclust ercentersandgoodinitialclustercentersforthe FuzzyCMeansalgorithm.Thealgorithmisverysensitivetothe Tcreateforheapparameter. Intheproposedalgorithmthestochasticpropertyofantswassimulatedtoobtaingoodcluster centers.Theantsmoverandomlyinthefeaturespacecarryingafeatureofaclustercenterwith them.Afterafixednumberofiterationstheclust ercentersareevaluatedusingthereformulation ofFCMwhichleavesoutthemembershipmatrix[1 5].Aftertheantstagethebestclustercenters obtainedareusedastheinitialclustercentersfortheFuzzyCMeansandHardCMeansalgorithms. 5.2ReformulationofClusteringCriteriaforFCMandHCM In[15]theauthorshaveproposedareformulationoftheoptimizationcriteriausedinacouple ofcommonclusteringobjectivefunc tions.Theoriginalclusteringfunctionsminimizetheobjective function5.1tofindgoodclusters. Jm( U, )=ci =1 nk =1Um ikDik( xk,i)(5.1) where € c 2 : Numberofclusters € n: Numberofdatapoints € i: The ithclusterprototype € Dik( xk,i) : Distanceof xkfrom ithclustercenter 44

PAGE 56

€ Uik: Membershipofthe kthobjectinthe ithcluster € m 1 : Thedegreeoffuzzification Thereformulationrepla cesthemembershipmatrix U withthenecessaryconditionswhichare satisfiedby U .Thereformulatedversionof Jmisdenotedas Rm. FortheHardclusteringcasethe U optimizationisoveracri spmembershipmatrix.The necessaryconditionfor U isgiveninequation5.2.Equation5.3givesthethenecessaryconditions for U ,forthefuzzycase.Thedistance Dik( xk,i)isdenotedas Dik. Uik=0if Dik> min( D1 k,D2 k,D3 k, ,Dck) =1otherwise(5.2) Uik= D1 1 Š mik c j =1D1 1 Š mjk (5.3) Thereformulationsforhardandfuzzyoptimizationfunctionsaregiveninequations5.4and 5.5respectively.Thefunction R dependsonlyontheclusterprototypeandnotonthe U matrix,whereas J dependsonboththeclusterprototypeandthe U matrix.The U matrixforthe reformulatedcriterioncanbeeasilycomputedusingequation5.2or5.3. R1( )=nk =1min ( D1 k,D2 k, ,Dck)(5.4) Rm( )=nk =1ci =1D1 1 Š mik1 m(5.5) 5.3Algorithm Theantsco-ordinatetomoveclustercentersinfeat urespacetosearchforoptimalclustercenters. Initiallythefeaturevaluesarenormalizedbetw een0and1.Eachantisassignedtoaparticular featureofaclusterinapartition.Theantsnever changethefeature,clusterorpartitionassigned tothem.ApictorialviewisgiveninFigure5.1whereeachverticallineisadimensioninparallel coordinates[17,2].Afterrandomlymovingthecl ustercentersforafixednumberofiterations, 45

PAGE 57

calledanepoch,thequalityofthepartitionisevaluatedbyusingthereformulatedcriterion5.4 or5.5.Ifthecurrentpartitionisbetterthananyofthepreviouspartitionsintheantsmemory thentheantremembersthispartitionelsetheant,withagivenprobabilitygoesbacktoabetter partitionorcontinuesfromthecurrentpartition.Thisensuresthattheantsdonotremembera badpartitionanderaseapreviouslyknowngoodpartition.Eveniftheantschange good cluster centersto unreasonable clustercenters,theantscangobacktothe good clustercentersastheants haveafinitememoryinwhichtheykeepthecurre ntlybestknownclustercenters.Therearetwo directionsfortherandommovementoftheant.Thepositivedirectioniswhentheantismovingin thefeaturespacefrom0to1,andthenegativedirectioniswhentheantismovinginthefeature spacefrom1to0.Ifduringtherandommovementtheantreachestheendofthefeaturespacethe antreversesdirection.Afterafixednumberofepochstheantsstop. Thedataispartitionedusingthecentroidsobtainedfromthebestknown Rmvalue.Thenearest neighboralgorithmisusedforassignmenttoacluster.Theclustercenterssoobtainedarethen usedastheinitialclustercentersfortheFuzzyCMeansortheHardCMeansalgorithm.Theant basedalgorithmispresentedinFigure5.2. 46

PAGE 58

Figure5.1.PictorialViewoftheAlgorithm 47

PAGE 59

1.Normalizethefeaturevaluesbetween0and1.Thenormalizationislinear.Theminimum valueofaparticularfeatureismappedto0andthemaximumvalueofthefeatureis mappedto1. 2.Initializetheantswithrandominitialvaluesandwithrandomdirection.Therearetwo directions,positiveandnegative.Thepositivedirectionmeanstheantismovinginthe featurespacefrom0to1.Thenegativedirectionmeanstheantismovinginthefeature spacefrom1to0.Cleartheinitialmemory.Theantsareinitiallyassignedtoaparticular featurewithinaparticularclusterofaparticularpartition.Theantsneverchangethe feature,clusterorthepartitionassignedtothem. 3.Repeat 3.1Foroneepoch/*Oneepochis n iterationsofrandomantmovement*/ 3.1.1Forallants 3.1.1.1Withaprobability Presttheantrestsforthisepoch 3.1.1.2Iftheantisnotrestingthenwithaprobability Pcontinuetheantcontinues inthesamedirection,elseitchangesdirection 3.1.1.3Withavaluebetween Dminand Dmaxtheantmovesintheselecteddirection 3.2Thenew Rmvalueiscalculatedusingthenewclustercenters 3.2.1Ifthepartitionisbetterthananyoftheoldpartitionsinmemorythentheworst partitionisremovedfromthememoryandthisnewpartitioniscopiedtothe memoriesoftheantsmakingupthepartition 3.2.2Ifthepartitionisnotbetterthananyoftheoldpartitionsinmemory Then Withaprobability PContinueCurrenttheantcontinueswiththecurrentpartition Else Withaprobability0.6theantchoosestogobacktothebestknownpartition, withaprobability0.2theantgoesbacktothesecondbestknownpartition, withaprobability0.1theantgoestothethirdbestknownpartition,witha probability0.075theantgoestothefourthbestknownpartitionandwitha probability0.025theantgoestotheworstknownpartition UntilStoppingcriteria Thestoppingcriterionisthenumberofepochs. Figure5.2.FuzzyAntClusteringwithCentroidsAlgorithm 48

PAGE 60

ThevaluesoftheparametersusedinthealgorithmareshowninTable5.1. Table5.1.ParameterValues Parameter Value Numberofants 30Partitions1 Memoryperant 5 Iterationsperepoch 50 Epochs 1000 Prest 0.01 Pcontinue 0.75 PContinueCurrent 0.20 Dmin 0.001 Dmax 0.01 5.4Results Thealgorithmwasappliedtosixrealdatasetsandtenartificialdatasets:theIrisPlantdataset, GlassIdentificationdataset,WineRecognitiondataset,MRIdataset,MultipleSclerosisdataset, theBritishTownsdatasetandthe10Gaussiandatasets. TheresultsobtainedforthedatasetsareshowninTable5.2.Allresultsareanaveragefrom50 randominitializations.TheresultsfortheFCMandHCMaretheaverageresultsfrom50random initializations.Theglassdatasethasbeensimplifiedtohavejust2classes windowglass and nonwindowglass .TheresultsforthismodifieddatasetarealsoshowninTable5.2.Theagefactor playsanimportantroleintheMultipleSclerosisdataset;theresultsconsideringtheagefeature andignoringtheagefeaturearealsoshown.Note,the Rmvalueisalwayslessthanorequaltothat fromrandomlyinitializedFCMexceptforGlass(6classes).Thirteendatasetshaveasingleextrema fortheFCMalgorithm.Theyconvergetothesameextrema,forallinitializationstriedhere.This isre”ectedinTable5.2wherewehavethesamevaluesincolumns3and4forthethirteendatasets. Theparameters Numberofepochs Dminand Dmaxplayanimportantroleindetermining thequalityoftheclustersfound.Byperformingmanualsearch,newparameters,whichgave betterresults,werefound.ThevaluesofthenewparametersareshowninTable5.3andthe resultsobtainedbyusingthesemodifiedparametersareshowninTable5.4.Clearimprovements wereobservedfor3datasets.FortheBritishTownsdatasettheaveragevaluefor RmafterFCM 11partition=numberofclusters numberoffeaturespercluster49

PAGE 61

Table5.2.ResultsforFuzzyCMeans Min Rm RmfromFCM, RmfromFCM, Dataset foundbyAnts AntInitialization RandomInitialization BritishTowns 1.6828 1.6033 1.6039 Iris 5.4271 5.2330 5.2330 Wine 33.0834 28.7158 28.7158 Glass(6classes) 11.3827 7.2937 7.2917 Glass(2classes) 25.8531 24.3932 24.3932 MultipleSclerosis(withage) 6.9456 6.8538 6.8538 MultipleSclerosis(ignoringage) 3.5704 3.5319 3.5319 MRI 311.2397 302.1302 303.289 Gauss-1 6.1588 5.5481 5.5481 Gauss-2 4.6786 4.0646 4.0646 Gauss-3 4.8764 4.2120 4.2120 Gauss-4 2.5536 1.9156 2.7458 Gauss-5 8.6553 8.2035 8.2035 Gauss500-1 4.4866 4.2559 4.2559 Gauss500-2 3.6328 3.3681 3.3681 Gauss500-3 2.3564 2.0560 2.0560 Gauss500-4 1.9937 1.6880 1.7834 Gauss500-5 1.9031 1.5848 1.5848 decreasedto1.5999from1.6033,similarlyfortheGlass(6classes)datasettheaveragevaluefor RmafterFCMdecreasedto7.2897from7.2937.Thisaveragevalueisbetterthanthatobtainedfrom randomlyinitializedFCM.Theaveragevaluefor RmafterFCMfortheMRIdatasetdecreasedto 301.9198from302.1302. Table5.3.NewParameters Parameter OldValue NewValue Epochs 1000 2000 Dmin 0.001 0.0001 Dmax 0.01 0.001 50

PAGE 62

Table5.4.ResultsforFCMObtainedfromModifiedParameters Min Rm RmfromFCM, RmfromFCM, Dataset foundbyAnts AntInitialization RandomInitialization BritishTowns 1.6051 1.5999 1.6039 Glass(6classes) 9.2284 7.2897 7.2917 MRI 302.1188 301.9189 303.2894 5.5HardCMeans TheantalgorithmwasappliedtogetherwiththeHardCMeansalgorithm.Theantsfindthe clustercentersandthesecentersareusedasthe initialcentersfortheHardCMeansalgorithm. TheparametervaluesarethoseshowninTable5.1. FromTable5.5weseethatthealgorithmgivesbetterresultsthanrandomlyinitializedHCMfor 15ofthe18datasetstested.Changingtheparametervaluescanimprovetheresults.Byperforming asearchintheparameterspacewegotparametervaluesthatresultedinbetterpartitions.Tables 5.6and5.7showthevariationintheresultsobtainedbychangingthenumberofantsperpartition andepochsfortheBritishTownsandWinedatasets.Fromthetablesweseethatasthenumber ofepochsincrease,theminimum Rmfoundbytheantsdecreases,th isistobeexpectedbecause asthenumberofepochsincrease,theantsgetmoretimetorefinethecentroidsfound.Alsoas thenumberofantsincrease,better Rmvaluesarefound.Table5.8showstheresultsobtainedfor differentMRIslices,theparametervaluesusedfortheMRIdatasetaretabulatedinTable5.3and theantsperpartitionwere50.Weca nseethepartitionsallhavelower Rmvalues. 51

PAGE 63

Table5.5.ResultsforHardCMeans Min Rm RmfromHCM, RmfromHCM, Dataset foundbyAnts AntInitialization RandomInitialization BritishTowns 5.5202 3.6260 3.4339 Iris 7.0055 6.9981 8.2516 Wine 52.8098 50.4573 48.9792 Glass(6classes) 28.1317 24.3770 21.1165 Glass(2classes) 34.2488 34.1352 36.9132 MultipleSclerosis(withage) 10.2201 10.2016 10.3548 MultipleSclerosis(ignoringage) 4.6406 4.6381 4.7759 MRI 433.2752 432.7499 452.384 Gauss-1 7.1391 6.4856 11.0962 Gauss-2 5.1055 4.4725 11.0645 Gauss-3 5.2625 4.6624 12.7167 Gauss-4 2.7273 2.0386 12.4133 Gauss-5 11.6107 10.9422 11.9558 Gauss500-1 5.8333 5.4921 6.0718 Gauss500-2 4.3405 4.0029 7.7512 Gauss500-3 2.6558 2.3140 6.3784 Gauss500-4 2.1678 1.8465 7.3442 Gauss500-5 2.0758 1.7314 7.0953 Table5.6.ResultsfortheBritishTownsDataset Antsper Min Rm RmfromHCM, RmfromHCM, partition Epochs foundbyAnts AntInitialization RandomInitialization 50 2000 5.3658 3.5812 50 4000 4.5048 3.5701 75 2000 5.1691 3.6134 3.4339 100 2000 3.0835 3.0661 52

PAGE 64

Table5.7.ResultsfortheWineDataset Antsper Min Rm RmfromHCM, RmfromHCM, partition Epochs foundbyAnts AntInitialization RandomInitialization 50 2000 52.8003 49.2405 50 4000 51.0631 49.2604 75 2000 50.1879 49.2604 75 3000 49.9230 48.9748 48.9792 100 2000 49.6415 48.9716 100 4000 49.2076 48.9697 Table5.8.ResultsfortheMRIDataset Slice Min Rm RmfromHCM, RmfromHCM, # foundbyAnts AntInitialization RandomInitialization 20 853.7991 851.8342 882.5732 35 919.8082 917.6636 927.6961 45 839.1622 838.0175 851.3756 46 842.5583 841.3414 847.0730 47 796.8415 795.6057 834.1028 5.6ExecutionTime Thevariationoftheminimum RmfoundbytheantsbychangingtheantsperpartitionforMR slice#35isshowninTable5.9.Asthenumberofantsincreasestheminimum Rmfounddecreases, butatthecostofincreasedexecutiontime.Asthe numberofantsincrease,moresearchspaceis exploredandwegetbetter Rmvalues. Table5.9.Variationof RmwiththeNumberofAntsforSlice#35ofMRIDataset Antsper Min Rm RmfromHCM, RmfromHCM, partition foundbyAnts AntInitialization RandomInitialization 50 919.8082 917.6636 75 912.0365 909.9324 927.6961 100 910.0054 907.9996 53

PAGE 65

TheexecutiontimefortheBritishTownsandtheMRIdatasetisshowninTable5.10.These twodatasetswerechosenbecauseBritishTowns datasetisthesmallest(intermsofnumberof examples)datasetandtheMRIdatasetisthelargestdatasetusedinthestudy.Thevaluesare anaveragefrom20000epochsand5experimentsfortheBritishTownsdatasetandfrom6000 epochsand3experimentsfortheMRIdataset.Oneexperimentconsistsoftheantstageandthe followingFuzzyCMeansortheHardCMeansstage.Thetimerequiredforoneepochforthe BritishtownsdatasetwasmorethanthatfortheMRIdatasetastherearemoreclasses,andhence moreants,intheBritishtownsdataset.Thetimerequiredfortheentireexperimentwasmore fortheMRIdatasetastherearemoreexamples intheMRIdataset.Thetimerequiredforthe FCMalgorithmisalsoshowninTable5.10.FromTable5.10weseethat,fortheBritishTowns dataset,approximately9100randomFCMinitializationscanbeperformedinthetimerequiredfor oneantexperiment.Similarlyapproximately225randomFCMinitializationscanbeperformedfor theMRIdataset. Weperformed11300( 225 50)randominitalizationsfortheMRIdatasetandaveragedthe best50 Rmvalues.Theaverage Rmvaluewas301.9189andtheaverage Rmvaluefoundbyants from50randominitalizationswas301.9189.SimilarlyfortheBritishTowns,datasettheaverage ofbest50 Rmvaluesover20000iterationswas1.5999andtheaverage Rmvaluefoundbyants, from50randominitalizations,was1.5999. TheexperimentswereperformedonanIntelPe ntium42.53GHzprocessorwith512KBcache and2GBofmainmemory. Table5.10.ExecutionTime Ants Timefor Timefor Timefor Dataset per oneEpoch oneExperiment FCM Partition (milliseconds) (seconds) (seconds) 50 14.9645 97 BritishTowns 75 22.2210 108.8380 0.01318 100 29.7060 120.896 50 6.7333 540.72 MRI 75 10.1117 811.0133 2.3929 54

PAGE 66

5.7Conclusions Thealgorithmisbasedonrelocatingthecluste rcentroidsinthefeaturespace.Theantsmove theclustercenters,nottheobjects,infeatures pacetofindagoodpartitionforthedata.The algorithmdoesnotusetheobjectmergingcriterion,whichmakesitindependentofthethreshold formergingtheobjects.Alsotherearelesscontrollingparametersthanthepreviousantbased clusteringalgorithms[19]. Resultsfrom18datasetsshowthesuperiorityofouralgorithmovertherandomlyinitializedFCM andHCMalgorithms.Forcomparisonpurposes,Tables5.11,5.12and5.13showthefrequencyof occurrenceofdifferentextremafortheantinitializedFCMandHCMalgorithmsandtherandomly initializedFCMandHCMalgorithms.TheantinitializedFCMalgorithmalwaysfindsthebetter extremafortheMRIdatasetandfortheBritishTownsdatasettheantinitializedalgorithmfinds thebetterextrema49outof50times.TheantinitializedHCMalgorithmfindsthebetterextrema fortheIrisdatasetallthetimeandfortheGlass(2class)datasetmajorityofthetime.Forthe differentMRIslices,theantinitializedHCMal gorithmfindsthebetterextremamostofthetime. In[14],aGeneticprogramingapproachwasusedtooptimizetheclusteringcriteria,thegenetic approachforHardCMeans,foundbetterextrema64%ofthetimefortheIrisdataset.Theant initializedHCMfindsbetterextremaallthetime. Thenumberofantsperpartitionisanimportantparameterofthealgorithm.Thequalityof thepartitionimprovesasnumberofantsincrea se,buttheimprovementcomesattheexpenseof increasedexecutiontime.Futureworkshouldfocusonautomaticallyfindingthenumberofants perpartitionandthenumberofclusters.Inthisdirectionthealgorithmproposedin[19]canbe usedtofindthenumberofclusters. Table5.11.FrequencyofDifferentExtremafromFCM,forBritishTownsandMRIDatasets FrequencyFCM, FrequencyFCM Dataset Extrema AntInitialization RandomInitialization 1.5999 49 16 British 1.6037 1 18 Towns 1.6081 0 16 301.9195 50 37 MRI 307.1898 0 13 55

PAGE 67

Table5.12.FrequencyofDifferentExtremafromHCM,forGlass(2Class)andIrisDatasets FrequencyHCM, FrequencyHCM Dataset Extrema AntInitialization RandomInitialization 34.1320 19 3 34.1343 11 19 Glass 34.1372 19 15 (2class) 34.1658 1 5 6.9981 50 23 7.1386 0 14 Iris 10.9083 0 5 12.1437 0 8 Table5.13.FrequencyofDifferentExtremafromHCM,MRIDataset FrequencyHCM, FrequencyHCM Slice# Extrema AntInitialization RandomInitialization 841.3414 50 44 20 889.1043 0 6 930.1677 45 30 35 951.4871 5 15 1003.492 0 5 838.0175 50 41 45 912.2289 0 9 841.3414 50 45 46 889.1043 0 5 795.3043 35 27 47 796.2459 15 13 970.5483 0 10 56

PAGE 68

CHAPTER6 CONCLUSIONSANDFUTUREWORK 6.1CombinedAlgorithm The Fuzzyants algorithmautomaticallyfindsthenumberofclustersandgoodinitialcluster centers.Thedrawbackofthealgorithmisthelargenumberofcontrollingparametersandthe absenceofgoodmethodstosetthoseparameters. The Fuzzyantclusteringwithcentroids algorithmshaslesscontrollingparametersandless sensitivitytotheparameters.Thedrawbackofthealgorithmisitrequirestheinitialknowledgeof thenumberofclustersindata.TheresultsaregenerallybetterthenwithFCM/HCM.Onewould needalargenumberofrandominitializationtobecompetitive.Also,aparallelversionoftheants algorithmcouldoperatemuchfasterthanthecurrentsequentialimplementation,therebymaking itaclearchoiceforminimizingthechanceof“ndingapoorextrema. The Fuzzyants algorithmcanbeusedtofindthenumberofclustersinthedataandtheresults ofthealgorithmcanbegiventothe Fuzzyantclusteringwithcentroids algorithm. Weimplementedtheaboveideaandtheresultsarepromising.Tables6.1and6.2showthe resultsfromthecombinedalgorithm.Theanalysiscanbeperformedusingtwometrics: Numberof errors andminimum Rm.Forcomparison,theresultsfromrandomlyinitializedFCMalgorithm, withknownnumberofclusters,arealsogiven. Inthesecondantstagetheantsoptimizethe Rmcriteria,thiscanbeobservedinTable6.1. The RmvaluefortheIrisdatasetafterthe1ststageFCMis4.5220andafterthefinalFCMstage thevalueis4.4738.Buttheminimum Rmvaluesometimesdoesntcorrespondtotheminimum errors,thisisobservedintheIrisdataset,theerrorsafterFCM1ststageare15.92andthoseafter thefinalFCMstageare18.52. 57

PAGE 69

Table6.1.ResultsfromtheCombinedAlgorithm Classes Rmafter Rmfoundby Rmfrom Rmfrom Dataset found FCM1ststage Ants(2ndstage) FCM(final) FCM(random) Iris 3.64 4.5220 4.7658 4.4738 5.2330 Wine 4.46 20.5878 22.9632 20.5878 28.7158 Glass 5.5 8.4098 9.9975 8.4098 7.2917 BritishTowns 6.74 1.1452 1.2429 1.1453 1.6039 MultipleSclerosis 2.76 4.6844 4.8632 4.6844 6.8538 Gauss-1 5.84 6.2395 6.6336 6.1234 5.5481 Gauss-2 5.86 4.4591 5.1016 4.3724 4.0646 Gauss-3 6.34 4.1108 4.7385 4.1313 4.2120 Gauss-4 6.16 2.0323 2.7307 2.0462 2.7458 Gauss-5 6.30 7.1248 7.5398 7.0827 8.2035 Gauss500-1 5.68 4.0874 4.2854 4.0554 4.2559 Gauss500-2 5.64 3.3677 3.7338 3.3395 3.3681 Gauss500-3 5.02 2.6648 2.8838 2.6754 2.05560 Gauss500-4 5.18 2.7631 3.0501 2.7611 1.7834 Gauss500-5 5.78 1.6849 1.9576 1.6513 1.5848 Table6.2.ResultsfromtheCombinedAlgorithm(Errors) Classes Errorsafter Errorsafter Errorsafter Errorsafter Dataset found Ants1ststage FCM(1ststage) FCM(final) FCM(Random) Iris 3.64 35.76 15.92 18.52 16 Wine 4.46 13.94 9.36 9.36 9 Glass 5.5 109.8 98.02 98.02 96 MultipleSclerosis 2.76 20.86 17.86 17.86 17 Gauss-1 5.84 348.58 25.34 29.08 0 Gauss-2 5.86 369.68 15.84 15.84 0 Gauss-3 6.34 318.44 10.56 18.24 0 Gauss-4 6.16 328.98 7.92 7.92 25.32 Gauss-5 6.30 329.40 19.64 20.12 7 Gauss500-1 5.68 63.28 8.5 16.22 1 Gauss500-2 5.64 37.36 7.6 11.48 0 Gauss500-3 5.02 72.9 19 18.98 1 Gauss500-4 5.18 62.3 22.12 29.88 1.94 Gauss500-5 5.78 20.96 6.08 6.08 0 58

PAGE 70

6.2Contributions The FuzzyAnts algorithmisanovelmethodto“ndthenumberofclustersinthedataand alsoprovidesgoodinitializationsfortheFCMandHCMalgorithms.Weperformedsensitivity analysisonthecontrollingparametersandfoundthe FuzzyAnts algorithmtobeverysensitiveto the Tcreateforheapparameter. The Fuzzyantclusteringwithcentroids algorithmshaslesscontrollingparametersandless sensitivitytotheparameters.Thedrawbackofthealgorithmisitrequirestheinitialknowledgeof thenumberofclustersindata.TheresultsaregenerallybetterthenwithFCM/HCM.Onewould needalargenumberofrandominitializationtobecompetitive.Also,aparallelversionoftheants algorithmcouldoperatemuchfasterthanthecurrentsequentialimplementation,therebymakingit aclearchoiceforminimizingthechanceof“ndingapoorextrema.Thecombinedalgorithm“nds thenumberofclustersindataandgoodpartitionforthedata. 6.3FutureWork Thefutureworkshouldfocusonsettingthe Tcreateforheapparameterautomatically.Thealgorithmisverysensitivetothisparameter.Wehavetriedacoupleofapproachestosettheparameter automatically,buttheresultswerenotencouraging. Inthe Fuzzyantclusteringwithcentroids algorithm,thenumberofantsisanimportantparameter,andalsotheinitialnumberofclustersinthedataisrequired.Futureworkshouldfocus onsettingthesevaluesautomatically.Wecomparetheresultsfromthealgorithmswithrandomly initializedFCMandHCM.InfuturetheresultscanbecomparedwithcleverlyinitializedFCM andHCMalgorithms.Theinitialplacementoftheantsisalsorandom,futureworkshouldfocus ongivingcleverinitializationstotheants.Thestoppingcriteriaiscurrentlybasedonthenumber ofiterations,futureworkshouldconcentrateonautomaticallystoppingtheantsbasedontheir progress.Trailscanbeusedtoguidethemovementoftheants,futureworkshouldconcentrateon incorporatingtrailsinthealgorithm. 59

PAGE 71

REFERENCES [1]A.Abraham,J.R.delSolar,andM.K¨ oppen,editors. SoftComputingSystems-Design, ManagementandApplications,HIS2002 ,volume87of FrontiersinArticialIntelligenceand Applications ,Santiago,Chile,Dec2002.IOSPress. [2]M.BertholdandL.Hall.Visualizingfuzzypointsinparallelcoordinates. IEEETransactions onFuzzySystems ,11(3):369…374,June2003. [3]J.C.Bezdek.Patternrecognitionwit hfuzzyojectivefunctionsalgorithms. Plenumpress 1981. [4]W.Bin,Z.Yi,L.Shaohui,andS.Zhongzhi.Csim:adocumentclusteringalgorithmbased onswarmintelligence. EvolutionaryComputation,2002.CEC'02.Pro ceedingsofthe 2002 Congresson ,1:477…482,May2002. [5]W.BinandS.Zhongzhi.Aclusteringalgorithmbasedonswarmintelligence. Info-techand Info-net,2001Pro ceedings.ICII 2001 ,3:58…66,2001. [6]C.BlakeandC.Merz.UCIrepositoryofmachinelearningdatabases,1998. [7]E.Bonabeau,M.Dorigo,andG.Theraulaz. SwarmIntelligneceFromNaturaltoArticial Systems .OxfordUniversityPress,NewYork,NewYork,1999. [8]E.Bonabeau,A.Sobkowski,G.Theraulaz,andJ.-L.Deneubourg.Adaptivetaskallocationinspiredbyamodelofdivisionoflaborinsocialinsects.InD.Lundh,B.Olsson,and A.Narayanan,editors, BiocomputingandEmergentComputation ,pages36…45.WorldScienti“c,1997. [9]C.-H.Cheng. EntropyBasedSubspaceClusteringforMiningNumericalData .PhDthesis, ChineseUniversityofHongKong,1999. [10]M.Doriga,V.Maniezzo,andA.Colorni.Antsys tem:Optimizationbyacolonyofcooperating agents. IEEETransactionsonSystems,Man,andCybernetics-PartB:Cybernetics ,26(1):29… 41,Feb1996. [11]R.DudaandP.Hart. PatternClassicationandSceneAnalysis .JohnWileyandSons,New York,1973. [12]GathandA.B.Geva.Unsupervisedoptimalfuzzyclustering. IEEETransactionsonPattern AnalysisandMachineIntelligence ,11(7):773…781,Jul1989. [13]F.GolchinandK.Paliwal.Minimum-entropyclusteringanditsapplicationtolosslessimage coding. Proc.IEEEInternationalConferenceonImageProcessing ,Oct1997. [14]L.O.Hall,I.B. ¨ Ozyurt,andJ.C.Bezdek.Clusteringw ithageneticallyoptimizedapproach. IEEETrans.onEvolutionaryComputation ,3(2):103…112,1999. [15]R.J.HathwayandJ.C.Bezdek.Optimizationofclusteringcriteriabyreformulation. IEEE TransactionsonFuzzySystems ,3(2):241…245,May1995. 60

PAGE 72

[16]F.H¨ oppner,F.Klawonn,R.Kruse,andT.Runkler. FuzzyClusterAnalysis .JohnWiley,West Sussex,England,1999. [17]A.Inselberg.Theplanewithparallelcoordinates. SpecialIssueonComputationalGeometry, TheVisualComputer ,(1):69…97,1985. [18]K.Jajuga,A.Soko lowski,andH.-H.Bock,editors. InternationalFederationofClassication Societies ,number8thinStudiesinClassi“cation,DataAnalysis,andKnowledgeOrganization, Carcow,Poland,Jul2002.Springer…Verlag. [19]P.M.KanadeandL.O.Hall.Fuzzyantsasaclusteringconcept. NorthAmericanFuzzy InformationProcessingSociety,NAFIPS2003,22ndInternationalConferenceofthe ,pages 227…232,2003. [20]P.M.KanadeandL.O.Hall.Fuzzyantsclusteringwithcentroids. acceptedforpublication inFUZZIEEE2004conference ,2004. [21]N.Karayiannis.Maximumentropyclusteringalgorithmsandtheirapplicationinimagecompression. Systems,Man,andCybernetics,1994.'Humans,InformationandTechnology'.,1994 IEEEInternationalConferenceon ,1:337…342,Oct1994. [22]N.Karayiannis.Meca:maximumentropyclusteringalgorithm. FuzzySystems,1994.IEEE WorldCongressonComputationalIntelligence.,ProceedingsoftheThirdIEEEConference on ,1:630…635,Jun1994. [23]J.KennedyandR.C.Eberhart. SwarmIntelligence .MorganKaufmann,SanDiego,California, 2001. [24]R.-P.LiandM.Mukaidono.Amaximum-entropyapproachtofuzzyclustering. FuzzySystems, 1995.InternationalJointConferenceoftheFourthIEEEInternationalConferenceonFuzzy SystemsandTheSecondInternationalFuzzyEngineeringSymposium.,Proceedingsof 1995 IEEEInternationalConferenceon ,4:2227…2232,Mar1995. [25]A.Lorette,X.Descombes,andJ.Zerubia.Fullyunsupervisedfuzzyclusteringwithentropy criterion. PatternRecognition,2000.Pro ceedings.15thInternationalConferenceon ,3:986…989, Sep2000. [26]N.Monmarch e,M.Slimane,andG.Venturini.Antcla ss:discoveryofclustersinnumericdata byanhybridizationofanantcolonywiththekmeansalgorithm,Jan1999. [27]N.Monmarch e,M.Slimane,andG.Venturini.Onimpro vingclusteringinnumericaldatabases witharti“cialants.InD.Floreano,J.Nicoud,andF.Mondala,editors, 5thEuropeanConferenceonArticialLife(ECAL'99),LectureNotesinArticialIntelligence ,volume1674,pages 626…635,Lausanne,Switzerland,Sep1999.Springer-Verlag. [28]S.OuadfelandM.Batouche.Mrf-basedimagesegmentationusingantcolonysystem. Electronic LettersonComputerVisionandImageAnalysis ,2(2):12…24,2003. [29]R.S.Parpinelli,H.S.Lopes,andA.A.Freitas.Dataminingwithanantcolonyoptimization algorithm. IEEETransactionsonEvolutionaryComputing ,6(4):321…332,Aug2002. [30]V.RamosandF.Almeida.Arti“cialantcoloniesindigitalimagehabitats-amassbehaviour eectstudyonpatternrecognition. Proc.ofANTS'2000-2ndInt.WorkshoponAntAlgorithms(FromAntColoniestoArticialAnts) ,pages133…116,2000. [31]V.RamosandJ.J.Merelo.Self-organizedstigmergicdocumentmaps:Environmentasmechanismforcontextlearning. Procs.ofAEB02-SpanishConferenceonEvolutionaryandBioInspiredAlgorithms ,pages284…293,2002. 61

PAGE 73

[32]S.Roberts,R.Everson,andI.Rezek .Minimumentropydatapartitioning. Proc.of9th InternationalConferenceonArticialNeuralNetworks ,pages844…849,1999. [33]D.TranandM.Wagner.Fuzzyentropyclustering. FuzzySystems,2000.FUZZIEEE2000. TheNinthIEEEInternationalConferenceon ,4:152…157,May2000. [34]C.-F.Tsai,H.-C.Wu,andC.-W.Tsai.Anewdataclusteringaproachfordatamininginlarge databases. ParallelArchitectures,AlgorithmsandNetworks,2002.I-SPAN'02Pro ceedings. InternationalSymposiumon ,pages278…283,2002. [35]G.Weiss,editor. MultiagentSystemsAmordernApproachtoDistributedArticialIntelligence MITPress,Massachusetts,1999. [36]T.White,B.Pagurek,andF.Oppacher.ASGA:Improvingtheantsystembyintegration withgeneticalgorithms.InJ.R.Koza,W.Banzhaf,K.Chellapilla,K.Deb,M.Dorigo,D.B. Fogel,M.H.Garzon,D.E.Goldberg,H.Iba,andR.Riolo,editors, GeneticProgramming 1998:Pro ceedingsoftheThirdAnnualConference ,pages610…617,Madison,Wisconsin,1998. MorganKaufmann. [37]M.Wilson,C.Melhuish,andA.Sendova-Franks.Creatingannularstructuresinspiredby antcolonybehaviourusingminimalistrobots. Systems,ManandCybernetics,2002IEEE InternationalConferenceon ,2:53…58,Oct2002. [38]L.Zadeh.Fuzzysets. InformationandControl ,pages338…353,1965. 62

PAGE 74

APPENDICES 63

PAGE 75

AppendixADatasets A.1ParametersUsedtoGeneratetheGaussianDatasets TableA.1.ParametersUsedtoGeneratetheGaussDatasets Dataset Mean Std.Dev 2 2 1 1 8 8 1 1 Gauss-1 2 16 1 1 19 3 1 1 14 14 1 1 2 2 1 1 10 10 1 1 Gauss-2 3 20 1 1 22 5 2 2 18 18 1 1 2 4 1 1 11 11 1 1 Gauss-3 5 20 1 1 22 5 2 2 20 20 1 1 3 3 1 1 11 11 1 1 Gauss-4 10 30 1 1 32 5 1 1 20 33 1 1 2 2 1 1 6 6 1 1 Gauss-5 2 10 1 1 12 4 2 2 11 11 1 1 64

PAGE 76

AppendixA(Continued) TableA.2.ParametersUsedtoGeneratetheGauss-500Datasets Dataset Mean Std.Dev 4 4 1 1 8 8 1 1 Gauss-500-1 3 12 1 1 11 4 2 2 12 12 1 1 4 4 1 1 7 10 1 1 Gauss-500-2 4 16 1 1 13 4 1 1 14 14 1 1 4 6 1 1 12 15 1 1 Gauss-500-3 4 21 1 1 13 4 1 1 18 14 1 1 4 6 1 1 12 16 1 1 Gauss-500-4 4 21 1 1 13 4 2 2 24 14 1 1 6 8 1 1 15 15 1 1 Gauss-500-5 9 21 1 1 16 4 2 2 28 18 1 1 65

PAGE 77

AppendixA(Continued) A.2PlotsoftheDierentDatasets 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1st Principal Component2nd Principal Component Class 1 Class 2 Class 3 FigureA.1.WineDataset(Normalized)-First2PrincipalComponents 1 0.5 0 0.5 1 1 0.5 0 0.5 1 1 0.5 0 0.5 1 1st Principal Component 2nd Principal Component 3rd Principal Component Class 1 Class 2 Class 3 FigureA.2.WineDataset(Normalized)-First3PrincipalComponents 66

PAGE 78

AppendixA(Continued) 1 0.5 0 0. 5 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1st Principal Component2nd Principal Component class 1 class 2 class 3 class 4 class 5 class 6 FigureA.3.GlassDataset(Normalized)-First2PrincipalComponents 1 0.5 0 0.5 1 0.5 0 0.5 1 1.5 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1st Principal Component 2nd Principal Component 3rd Principal Component class 1 class 2 class 3 class 4 class 5 class 6 FigureA.4.GlassDataset(Normalized)-First3PrincipalComponents 67

PAGE 79

AppendixA(Continued) 1 0.5 0 0. 5 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1st Principal Component2nd Principal Component Window Glass Nonwindow glass FigureA.5.GlassDataset:2classes(Normalized)-First2PrincipalComponents 1 0.5 0 0.5 1 0.5 0 0.5 1 1.5 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1st Principal Component 2nd Principal Component3rd Principal Component Window Glass Nonwindow glass FigureA.6.GlassDataset:2classes(Normalized)-First3PrincipalComponents 68

PAGE 80

AppendixA(Continued) 0.5 0 0.5 1 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 1st Principal Component2nd Principal Component Class 1 Class 2 FigureA.7.MultipleSclerosisDataset(Normalized)-First2PrincipalComponents 0.5 0 0.5 1 1 0.5 0 0.5 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 1st Principal Component 2nd Principal Component3rd Principal Component Class 1 Class 2 FigureA.8.MultipleSclerosisDataset(Normalized)-First3PrincipalComponents 69

PAGE 81

AppendixA(Continued) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.9.Gauss-1Dataset(Normalized) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.10.Gauss-2Dataset(Normalized) 70

PAGE 82

AppendixA(Continued) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.11.Gauss-3Dataset(Normalized) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.12.Gauss-4Dataset(Normalized) 71

PAGE 83

AppendixA(Continued) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.13.Gauss-5Dataset(Normalized) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.14.Gauss500-1Dataset(Normalized) 72

PAGE 84

AppendixA(Continued) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.15.Gauss500-2Dataset(Normalized) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.16.Gauss500-3Dataset(Normalized) 73

PAGE 85

AppendixA(Continued) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.17.Gauss500-4Dataset(Normalized) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1st Attribute2nd Attribute Class 1 Class 2 Class 3 Class 4 Class 5 FigureA.18.Gauss500-5Dataset(Normalized) 74