USF Libraries
USF Digital Collections

Virtual cadaver navigation system

MISSING IMAGE

Material Information

Title:
Virtual cadaver navigation system using virtual reality for learning human anatomy
Physical Description:
Book
Language:
English
Creator:
Lothe, Abhijit V
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Virtual reality
Visible human
3d texture mapping
Gigabyte volume exploration
Direct volume rendering
Dissertations, Academic -- Computer Science -- Masters -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: The use of virtual reality (VR) for visualization can revolutionize medical training by simulating real world medical training procedures through intuitive and engaging user interface. Existing virtual reality based visualization systems for human anatomy are based on 3D surface and volumetric models and simulative systems based on model libraries. The visual impact as well as facilitation for learning are inadequate in such systems. This thesis research is aimed at eliminating such inadequacies by developing a non-immersive virtual reality system framework for storage, access and navigation of real human cadaveric data. Based on this framework, a real time software system called virtual cadaver navigation system (VCNS) is developed, that can be used as an aid for teaching human anatomy.The hardware components of the system include, a mannequin, an examination probe similar to a medical ultrasound probe, and a personal computer.The examination probe is moved over the mannequin to obtain the virtual tomographic slice from the real cadaveric3-D volume data. A 3-D binary space partitioning tree structure is defined to organize the entire volumetric data, by subdividing it into small blocks of predefined size, called as bricks that are assigned a unique address for identification. As the examination probe is moved over the mannequin, the set of bricks intersecting the corresponding tomographic slice are determined by traversing the tree structure, and only, the selected bricks are accessed from the main memory and brought into the texture memory on the graphics accelerator card for visualization. The texture memory in the graphics card and the main memory are divided into slots of size, that is a multiple of the brick size, and a tagging scheme that relates the brick addresses, texture memory slots, and the main memory blocks is developed.Based on spatial, temporal and sequential locality of reference, only the currently required bricks as well as some of the neighboring bricks are loaded from the main memory into the texture memory, in order to maintain the highest frame rates required forreal time visualization. The above framework consisting of the data organization and the access mechanism are critical in terms of achieving the interactive frame rates required for real-time visualization.The input data to the system consists of non-segmented voxel data, and the data segmented and labelled based on tissue classification. The software system includes a labeling tool, in order to display the specific tissue information at the the location of the mouse cursor. This facility is useful in both teaching anatomy and self learning.
Thesis:
Thesis (M.S.C.S.)--University of South Florida, 2005.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Abhijit V. Lothe.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 79 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001670412
oclc - 62380842
usfldc doi - E14-SFE0001288
usfldc handle - e14.1288
System ID:
SFS0025609:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001670412
003 fts
005 20051216093403.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 051130s2005 flu sbm s000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001288
035
(OCoLC)62380842
SFE0001288
040
FHM
c FHM
049
FHMM
090
QA76 (Online)
1 100
Lothe, Abhijit V.
0 245
Virtual cadaver navigation system
h [electronic resource] :
b using virtual reality for learning human anatomy /
by Abhijit V. Lothe.
260
[Tampa, Fla.] :
University of South Florida,
2005.
502
Thesis (M.S.C.S.)--University of South Florida, 2005.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 79 pages.
520
ABSTRACT: The use of virtual reality (VR) for visualization can revolutionize medical training by simulating real world medical training procedures through intuitive and engaging user interface. Existing virtual reality based visualization systems for human anatomy are based on 3D surface and volumetric models and simulative systems based on model libraries. The visual impact as well as facilitation for learning are inadequate in such systems. This thesis research is aimed at eliminating such inadequacies by developing a non-immersive virtual reality system framework for storage, access and navigation of real human cadaveric data. Based on this framework, a real time software system called virtual cadaver navigation system (VCNS) is developed, that can be used as an aid for teaching human anatomy.The hardware components of the system include, a mannequin, an examination probe similar to a medical ultrasound probe, and a personal computer.The examination probe is moved over the mannequin to obtain the virtual tomographic slice from the real cadaveric3-D volume data. A 3-D binary space partitioning tree structure is defined to organize the entire volumetric data, by subdividing it into small blocks of predefined size, called as bricks that are assigned a unique address for identification. As the examination probe is moved over the mannequin, the set of bricks intersecting the corresponding tomographic slice are determined by traversing the tree structure, and only, the selected bricks are accessed from the main memory and brought into the texture memory on the graphics accelerator card for visualization. The texture memory in the graphics card and the main memory are divided into slots of size, that is a multiple of the brick size, and a tagging scheme that relates the brick addresses, texture memory slots, and the main memory blocks is developed.Based on spatial, temporal and sequential locality of reference, only the currently required bricks as well as some of the neighboring bricks are loaded from the main memory into the texture memory, in order to maintain the highest frame rates required forreal time visualization. The above framework consisting of the data organization and the access mechanism are critical in terms of achieving the interactive frame rates required for real-time visualization.The input data to the system consists of non-segmented voxel data, and the data segmented and labelled based on tissue classification. The software system includes a labeling tool, in order to display the specific tissue information at the the location of the mouse cursor. This facility is useful in both teaching anatomy and self learning.
590
Adviser: Dr. Ranganathan.
653
Virtual reality.
Visible human.
3d texture mapping.
Gigabyte volume exploration.
Direct volume rendering.
690
Dissertations, Academic
z USF
x Computer Science
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.1288



PAGE 1

VirtualCadaverNavigationSystem:UsingVirtualRealityF orLearningHumanAnatomy by AbhijitV.Lothe Athesissubmittedinpartialfulllment oftherequirementsforthedegreeof MasterofScienceinComputerScience DepartmentofComputerScienceandEngineering CollegeofEngineering UniversityofSouthFlorida MajorProfessor:NagarajanRanganathanPh.D. DonHilbelink,Ph.D. SudeepSarkar,Ph.D. DateofApproval: June9,2005 Keywords:VirtualReality,VisibleHuman,3Dtexturemappi ng,GigabyteVolume Exploration,DirectVolumeRendering c r Copyright2005,AbhijitV.Lothe

PAGE 2

DEDICATION TomyParentsandlovingsisterGauri.

PAGE 3

ACKNOWLEDGEMENTS IwouldliketothankmymajorprofessorDr.Ranganathanwhok indlyallowedme topursuemyresearchinterestandhelpedmeateverystepasa goodfriend.Iamvery thankfultohimforthenumerousdiscussionsandmeetingswe hadtowardsdesigningan ecientVCNSsystem.MysincerethankstoDr.Hilbelinkforp rovidinghisaccesstohis labfacilities,thehardwaremotiontrackersystem,andthe guidancetowardsdevelopingthe system.Iwouldalsoliketothankhimforprovidingmeanacce ssfurnishingthehardware andthesoftwarerequiredtoimplementthesystem.Iamgrate fultoDr.Sarkarfortaking timeoutofhisbusyscheduletoreviewmythesisandprovideu sefulcommentstoimprove thesame.

PAGE 4

TABLEOFCONTENTS LISTOFTABLES iii LISTOFFIGURES iv ABSTRACT vii CHAPTER1INTRODUCTION 1 1.1DenitionofVirtualReality 1 1.2KeyElementsofVirtualRealitySystems2 1.2.1VirtualWorldorEnvironment21.2.2SenseofImmersion 2 1.2.3SensoryFeedback 2 1.2.4Interaction 3 1.3ApplicationsofVirtualReality 3 1.4VirtualRealityinMedicalTrainingandEducation31.5VirtualCadaverNavigationSystem 5 1.5.1MotivationforthisWork 5 1.6ContributionsofThesis 8 1.7OrganizationofThesis 8 CHAPTER2LITERATUREREVIEW 10 2.1SurgicalPlanningandProcedures 10 2.2VirtualEndoscopy 12 2.3MedicalEducation 12 2.4NeuropsychologicalRehabilitationandPsychology132.5IssuesandChallengesinusingVRforHealthcare142.6VisualizationofLargeVolumeData 15 2.7ContextofthisWork 16 CHAPTER3VISIBLEHUMANDATASET 17 3.1VisibleMale 17 3.2VisibleFemale 18 3.3SegmentedDataset 18 3.4DataFormat 18 CHAPTER4ARCHITECTUREOFVCNS 21 4.1ModesofOperation 21 4.2SoftwareArchitecture 23 4.2.1DataManagementModule 23 i

PAGE 5

4.2.1.1DataI/O 23 4.2.1.2ResourceManager24 4.2.2VisualizationModule 25 4.2.3TrackingModule 25 4.2.4CentroidLocatorModule 26 4.2.5LabelingModule 26 4.2.6CollisionDetectionModule 27 4.3DataandComputationalFlowofVCNS29 4.3.1TrackingMode 29 4.3.2CentroidLocatorMode 32 4.3.3LabelingMode 32 4.4HardwareComponentsoftheSystem 35 4.4.1PCIBirdMotionTracker 35 4.4.1.1Transmitter 35 4.4.1.2Sensor 35 4.4.1.3Electronics 36 4.4.1.4MeasurementTechnique36 4.5GraphicalUserInterface 37 CHAPTER5SOFTWAREIMPLEMENTATION40 5.1ClassDesign 40 5.1.1 PCIBirdInterface Class 40 5.1.2 MemoryManager Class 42 5.1.3 VolumeDataLoader Class 44 5.1.4 CollisionDetector Class 44 5.1.5 Renderer Class 45 5.1.6 LabelGenerator Class 46 5.1.7 Manager Class 46 5.2SoftwareImplementation 47 5.2.1TrackingMode 47 5.2.2CentroidLocationMode 49 5.2.3LabelingMode 49 CHAPTER6RESULTSANDDISCUSSION 52 6.1Results 52 6.2PerformanceAnalysis 52 6.3Discussion 59 REFERENCES 61 APPENDICES 66 AppendixAClassDiagramforVCNS 67 ii

PAGE 6

LISTOFTABLES Table1.1ApplicationsOfVirtualReality 3 Table4.1SpecicationsOfWorkstation 37 Table6.1CongurationsOfTwoTestMachinesUsedForPerfor manceAnalysis56 iii

PAGE 7

LISTOFFIGURES Figure1.1TheDiagramShowsHowVCNSCanBeUsedToObtainCro ssSectionalView(TomographicSlice)OfVirtualHumanBodyAtArb itrary LocationsAndOrientationDuringTheTrackingMode7 Figure3.1VisibleHumanMaleImages(a)MRIImage(b)CTImag e(c)High ResolutionColoredImage 19 Figure3.2SegmentedTransaxialSliceThroughVisibleMale Thorax19 Figure3.3ATransaxialSliceThroughVisibleFemaleHead19Figure4.1SoftwareArchitectureOfVCNSShowingDierentM odules,AndInputsAndOutputsOfTheSystem 22 Figure4.2SoftwareArchitectureOfDataManagementModule 24 Figure4.3SoftwareArchitectureOfVisualizationModule2 4 Figure4.4SoftwareArchitectureOfTrackingModule25Figure4.5SoftwareArchitectureOfCentroidLocatorModul e26 Figure4.6SoftwareArchitectureOfLabelingModule27Figure4.7BinarySpacePartitionTree 28 Figure4.8CollisionDetectionProcess 28 Figure4.9ComputationalFlowOfVCNS 30 Figure4.10ComputationalFlowChartAndDataFlowDiagramF orTracking ModeOfVCNS 31 Figure4.11ComputationalFlowChartAndDataFlowDiagramF orCentroid ModeOfVCNS 33 Figure4.12ComputationalFlowChartAndDataFlowDiagramF orLabelingMode ofVCNS 34 Figure4.13ReferenceFramesFor(a)StandardTransmitter( b)25mmsensor (Courtesy:AscensionTechnology) 36 iv

PAGE 8

Figure4.14UserInterfaceForVCNS 39 Figure5.1InteractionDiagramForSoftwareComponentsOfV CNS41 Figure5.2ConversionFromPCIBirdToOpenGLCoordinateSys tem42 Figure5.3HierarchialDistributionOfVolumeDataInVCNS4 3 Figure5.4SequenceDiagramForTrackingModeBasedOnUMLPr inciples.An UnderlineBelowAClassNameImpliesThatTheInstanceOfTha t ClassIsUsed 48 Figure5.5SequenceDiagramForLabelingModeBasedOnUMLPr inciples.An UnderlineBelowAClassNameImpliesThatTheInstanceOfTha t ClassIsUsed 50 Figure6.1ATomographicSliceOfLungInTrackingModeThrou ghVisibleMale Data 53 Figure6.2ATomographicSliceOfVertebralColumnInTracki ngModeThrough VisibleMaleData 53 Figure6.3IdenticationOfLowerLeftLobeOfLungUsingThe LabelingTool54 Figure6.4DiagramShowingTheIntersectionOfTheThreeOrt hogonalPlanes ThroughTheVisibleMaleDuringCentroidLocatorMode54 Figure6.5TransaxialSliceThroughTheVisibleMaleDataGe neratedDuring CentroidLocatorMode 55 Figure6.6SagittalSliceThroughTheVisibleMaleDataGene ratedDuringCentroidLocatorMode 55 Figure6.7CoronalSliceThroughTheVisibleMaleDataGener atedDuringCentroidLocatorMode 55 Figure6.8VariationOfFrameRateWithTextureMemorySizeO nNVIDIA QuadroFX4400(MainMemory=512MB,BrickSize=64X64X64) 56 Figure6.9VariationOfFrameRateWithTextureMemorySizeO nATIRadeon 8700(MainMemory=512MB,BrickSize=64X64X64)57 Figure6.10VariationOfFrameRateWithBrickSizeOnNVIDIA QuadroFX4400 (MainMemory=512MB,TextureMemory=64MB)57 Figure6.11VariationOfFrameRateWithBrickSizeOnATIRad eon8700(Main Memory=512MB,TextureMemory=64MB)57 Figure6.12VariationofFrameRateWithMainMemorySizeOnN VIDIAQuadro FX4400(TextureMemory=64MB,BrickSize=64X64X64)58 v

PAGE 9

Figure6.13VariationOfFrameRateWithMainMemorySizeOnA TIRadeon 8700(TextureMemory=64MB,BrickSize=64X64X64)58 FigureA.1VCNSClassDiagram 68 vi

PAGE 10

VIRTUALCADAVERNAVIGATIONSYSTEM:USINGVIRTUAL REALITYFORLEARNINGHUMANANATOMY AbhijitV.Lothe ABSTRACT Theuseofvirtualreality(VR)forvisualizationcanrevolu tionizemedicaltraining bysimulatingrealworldmedicaltrainingproceduresthrou ghintuitiveandengaginguser interface.Existingvirtualrealitybasedvisualizations ystemsforhumananatomyarebased on3Dsurfaceandvolumetricmodelsandsimulativesystemsb asedonmodellibraries.The visualimpactaswellasfacilitationforlearningareinade quateinsuchsystems.Thisthesis researchisaimedateliminatingsuchinadequaciesbydevel opinganon-immersivevirtual realitysystemframeworkforstorage,accessandnavigatio nofrealhumancadavericdata. Basedonthisframework,arealtimesoftwaresystemcalledv irtualcadavernavigation system(VCNS)isdeveloped,thatcanbeusedasanaidforteac hinghumananatomy. Thehardwarecomponentsofthesysteminclude,amannequin, anexaminationprobe similartoamedicalultrasoundprobe,andapersonalcomput er.Theexaminationprobeis movedoverthemannequintoobtainthevirtualtomographics licefromtherealcadaveric 3-Dvolumedata.A3-Dbinaryspacepartitioningtreestruct ureisdenedtoorganize theentirevolumetricdata,bysubdividingitintosmallblo cksofpredenedsize,calledas \bricks"thatareassignedauniqueaddressforidenticati on.Astheexaminationprobeis movedoverthemannequin,thesetofbricksintersectingthe correspondingtomographic slicearedeterminedbytraversingthetreestructure,ando nly,theselectedbricksare accessedfromthemainmemoryandbroughtintothetextureme moryonthegraphics acceleratorcardforvisualization.Thetexturememoryint hegraphicscardandthemain memoryaredividedintoslotsofsize,thatisamultipleofth ebricksize,andatagging schemethatrelatesthebrickaddresses,texturememoryslo ts,andthemainmemoryblocks vii

PAGE 11

isdeveloped.Basedonspatial,temporalandsequentialloc alityofreference,onlythe currentlyrequiredbricksaswellassomeoftheneighboring bricksareloadedfromthemain memoryintothetexturememory,inordertomaintainthehigh estframeratesrequiredfor realtimevisualization.Theaboveframeworkconsistingof thedataorganizationandthe accessmechanismarecriticalintermsofachievingtheinte ractiveframeratesrequiredfor real-timevisualization. Theinputdatatothesystemconsistsofnon-segmentedvoxel data,andthedata segmentedandlabelledbasedontissueclassication.Thes oftwaresystemincludesa labelingtool,inordertodisplaythespecictissueinform ationatthethelocationofthe mousecursor.Thisfacilityisusefulinbothteachinganato myandselflearning.Thus,the proposedVCNSsystemsupportsecientnavigationthrought hehumanbodyforlearning anatomyandprovidestheknowledgeofspatiallocationsand theinterrelationshipamong thevariousorgansofthebody.Aprototypesoftwaresystemh asbeendeveloped,which iscapableofachievingathroughputof30framesperseconda ndhasbeentestedwith a18-GigabytehumancadavericdataobtainedfromtheNation alLibraryofMedicine,on apersonalcomputerwith64Megabytesoftexturememoryand5 12Megabytesofmain memory. viii

PAGE 12

CHAPTER1 INTRODUCTION Humanhistoryismarkedbythedevelopmentandevolutionofd ierentcommunication media.Fromtheageoldcavepaintingstothemostrecentvirt ualreality(VR),there hasalwaysbeenaquestfornewandeectivewaystoconveythe ideas.Inthelast decade,theeldofVRhasbeenexploredbecauseofitssuitab ilityforpresentingnewand existinginformationinamoreintuitiveway.Theoriginoft heVRisbelievedtobein therightsimulatorsystemsdevelopedtotrainthepilotsby puttingthemina\real-like" environment.SowhatisVRafterall?SinceVRisanewmedium, itsdenitionisstillin rux.SeveraldenitionsoftheVRhavebeenpresentedbyther esearchersandusersfrom theirownperspective.Someofthemareasfollows.1.1DenitionofVirtualReality \Anarticialenvironmentwhichisexperiencedthroughsen sorystimuli(assightsand sounds)providedbyacomputerandinwhichone'sactionspar tiallydeterminewhathappensintheenvironment" -Merriam-Webster[4]. \Virtualrealityisamediumcomposedofinteractivecomput ersimulations,thatsense theparticipant'spositionandactionsandreplaceoraugme ntthefeedbacktooneormore senses,givingthefeelingofbeingmentallyimmersedorpre sentinthesimulation(avirtual world)" -[53]. Fromtheseandothersimilardenitionsfourkeyelementsin aVRsystemcanbe identied.Theyareasfollows[53,59]. 1

PAGE 13

1.2KeyElementsofVirtualRealitySystems1.2.1VirtualWorldorEnvironment Itisamanifestationofanimaginaryworldorarealspacetha texistselsewhere.In caseofVR,itismadeupofvirtualobjects(VO),whicharegov ernedbycertainrulesand relationshipsbetweenthem.1.2.2SenseofImmersion TheparticipantintheVRcaneithergetcompletely(Immersi ve)orpartiallyimmersed (Non-Immersive)inthevirtualworld,dependingonwhether heiscompletelyisolated fromtherealworld.Forexample,theheadmounteddisplays( HMD)providetheuser withapersonalviewofthevirtualenvironment,makingtheu serunawareofthereal world.Ontheotherhand,non-immersivesystemsleavestheu servisuallyawareofthe realworld,however,theyareabletoseethevirtualworldth roughsomedisplaydevice suchasagraphicsworkstation[59].Thehybridsystems,als oknownas\Augmented RealitySystems",superimposetherealworldviewwiththes yntheticimagesobtainedfrom thevirtualworld.SomeVRsystemsallowonlyoneusertobeim mersedinthevirtual environment,whilecollaborativesystemsinvolvemoretha noneparticipantinteracting withthesystem,orwithotherparticipantsinthevirtualwo rld. 1.2.3SensoryFeedback Unlikethetraditionalcommunicationmedia,sensoryfeedb ackisanessentialingredient ofVRsystems.Although,thevisualfeedbackhasbeenmostdo minant,othersinclude auditory[12],olfactory[30]andhaptic(touch)[29].Thef eedbackisgenerallybasedon thepositionoftheparticipant,whichnecessitatesthetra ckingoftheirmovements.The trackinginvolvescomputerizedsensingoftheposition(lo cationand/ororientation)ofthe participant'sbody,oratleastapartofhisbody. 2

PAGE 14

1.2.4Interaction Thisinvolvestheabilitytoaectthevirtualworldthrough theactionsintherealworld. Forexample,aVRsystemmayallowpickingupobjects,settin gthemdownorripping switchesetc.InteractivitygivesauthenticitytotheVRsy stems. 1.3ApplicationsofVirtualReality TherearenumerouspracticalapplicationsforwhichVRhasb eenabletoprovidemore interactivityandbetterinterpretationsofcomplexdatat hanpossiblebefore.Table1.1 liststheapplicationsindierentareastowhichtheVRhasb eenappliedsuccessfully.The scopeofVR,however,isendless. Virtualrealityisgainingrecognitionforitsenormousedu cationalpotentialparticularly inthemedicalsurgicalsimulationsandrelatedtechniques .Thefollowingsectiongivesan overviewofthestateoftheartintheVRbasedmedicaltraini ngsystems.Amoredetailed discussionispresentedinChapter2. Table1.1.ApplicationsOfVirtualReality Areas ApplicationsofVR Engineering Aero-enginedesign,SubmarineDesign,Virtualcarprototy ping, Architecturaldesignsofbuildingsandrooms Entertainment Computeranimationofcartooncharacters,VRbasedgames andtheaters,Realtimecartoonanimations Science Molecularmodeling,Telepresence-controlledRemotelyOp erated Vehicle,Ultrasoundechography,Visualizationofelectri celds Training Flightsimulations,Fireghtertraining,Virtualsurgery Colonoscopy,Militarytraining,Nuclearaccidentsimulat ion Powerplantsimulation 1.4VirtualRealityinMedicalTrainingandEducation TheVRhasbeenusedinthehealthcareinthreeareas:surgica lsimulationandplanning,medicaleducation,andrecentlyintheneuropsycholo gicalassessmentandrehabilitation.TheearliestuseofVRinmedicinedatesbacktoearly 90'sandfocusedonthe3D 3

PAGE 15

visualizationofcomplexmedicaldataforsurgeryandsurgi calplanning[15].Thesesystems havebeenaugmentedbymoresophisticatedandinteractives ystemsthatintegratesensory feedbacksuchasthehaptic(touch)feedback,movingtheuse rofthesystemclosertorealism.Already,advancedsimulationsystemsusingvarious implementationstrategiesand targetingdierentareasofthehumanbodyhavebeendevelop ed[43,49,37].Forexample, theMinimallyInvasiveSurgeryTraining-VirtualReality( MIST-VR)trainer[10,24]has beenprovedtobemoreeectiveinperformingthelaparoscop icskillsthanthetraditional methods.Virtualendoscopyisanotherareawherevirtualre alityhasbeenusedtosolve problemsfacedwithrealendoscopytrainingprocedures.Av irtualendoscopysimulator allowsthestudentstoryinsidetheorgansbyreconstructin gthevirtualsurfacemodels inrealtime.Thesurgicalplanningprocedures,whichtypic allyinvolvestudyofseriesof 2Dimagesofdierentmodalities(CT/MRI),canalsobeenhan cedbyaVRsystemthat integratesthemodalitiesfromdierentsitestoprovidean interactivethree-dimensional view[46].VRhasalsobeenhelpingtheclinicalpsychologis tsbysimulatingtherealworld, whichcanbefullycontrolledbytheuserthroughvariouspar ameters.Akeyadvantage oeredbyVRinthiscase,istoprovidethepatientanability tosuccessfullymanage situationsrelatedtohis/herdisturbances[61,20]. Complextopicssuchashumananatomy,biochemistryandmole cularbiologyhavebeen mademorecomprehensiblewiththeintuitiveandengagingVR basedteachingenvironments.Therststepinthisdirectionwasthecreationofthe VRanatomybooks,which containedimagesfromarealhumanbeingasapartofVisibleH umanprojectconducted byNationalLibraryofMedicinein1993[7,55].Throughthe3 Dvisualizationofmassive volumesofinformationanddatabases,cliniciansandstude ntscanunderstandimportant physiologicalprinciplesorbasicanatomy[9].SuchVRsyst emscanbeusedaseducational toolsallowingadeeperunderstandingofinterrelationshi psbetweentheanatomicalstructuresthatcannotbeachievedbyanyothermeans,includingc adaverdissection[43].Apart fromteachinganatomy,VRhasbeenusedforteaching12-lead ECG[28,58]. Virtualrealityhas,thus,broadenedtheoveralltraininge xperienceforthemedical studentsbyprovidingthemanabilitytoacquireprociency andcondenceinperforming 4

PAGE 16

varietyoftechniquesbeforetheycandoitclinically.Thes imulationsystemsincombination withthetrainingonrealpatientscanenhancetheacquisiti onofclinicalskillsandincrease thedepthandthebreadthofknowledgeaboutthemedicalprob lems. 1.5VirtualCadaverNavigationSystem1.5.1MotivationforthisWork Inatypicalanatomylearningsessioninvolvingcadavericd issection,aninstructor teachesthestudentsaparticularorganinisolationandthe n,inrelationtotheother partsinitsvicinityorwithrespecttosomelandmarkfeatur essurroundingit.During theevaluation,thestudentsareexpectedtoidentifythese structuresbyrelatingtothe associationstaughttothem.Thus,understandingtheinter -relationshipsbetweenvarious organsandtissuesisimportantinlearninghumananatomy. Duetothedearthinnumberofcadaversavailablefordissect ion,sometimesthestudents aretaughtonthepreviouslydissectedcadaversthataredev oidofthekeylandmarks. Inaddition,theynditdiculttolocatesomeanatomicalpa rtssuchasnerves,when presentedwithalivehumanbody.Thus,thereisaneedtodeve lopamethodbywhich thelocationsofvariousorgans,nervesandtissuesinsidet hebodyaswellastheirrelative positionscanbeunderstoodinawaybetterthancadavericdi ssections. Previously,Teistleret.al.[57]hasdevelopedasystemfor learningtheanatomyby simulatinganultrasound-likenavigationofthe3DVOXELMA Ndataset[27].Avirtual examinationprobe(analogy:medicalultrasoundprobe),is usedtogenerateobliquetomographicimagesthatarecomputedfromagivenvolumedata[57 ].Thissystem,however, doesnotidentifytheanatomicalpartsonthesliceobtained fromthetracker,whichis imperativeforlearning/teachinganatomy.Inaddition,it onlyallowstheusertoexplore thebodyonepartatatime. Toaddresstheseproblems,inthiswork,aPCbasedsoftwares ystemcalledas,virtual cadavernavigationsystem(VCNS),hasbeendeveloped,that canbeusedtoteachhuman anatomyonrealhumancadavericdataobtainedfromtheNatio nalLibraryofMedicine 5

PAGE 17

[6].Thesystemconsistsofamannequin,anexaminationprob e(partof3Dmotiontracker system),andapersonalcomputer.Thesegmentedandthenonsegmentedvolumetricdata thatconsistsofaseriesof2Dphotographicimagesofthesli cesofarealhumancadaver, formstheinputforthesoftware.Thesystemcanbeoperatedi ntracking,centroidlocation andlabelingmodes.Inthetrackingmodetheusercanexplore thecompletevirtualhuman anatomybyobtainingvirtualtomographicslicesbymovinga nexaminationprobeoverthe mannequin.Atomographicsliceshowsthecrosssectionofth ehumanbodyatagiven orientationandpositionofanimaginaryplaneoriginating fromthetracker.Theinteraction withthemannequinhelpsinunderstandingthespatiallocat ionsofvariousorgansofthe body.Inthecentroidlocationmode,thesoftwarehelpsinre gisteringthevolumetricdata withthethemannequin,inordertoobtaincorrectslicesfor givenpositionandorientation ofthetracker.Itisalsopossibletoobtainthetissuespeci cinformationbymovingthe mousecursoroverthedesiredtissueonthetomographicslic e.Thus,VCNSservesasa toolforteachingspatiallocationsandrelativepositions ofvariousbodypartsthroughthe interactionwiththemannequin,andaugmentsitwiththetis suespecicinformationfor enhancingtheinterpretations.Thesefactorsarecruciali nteachinghumananatomyand arelackingintheexistingmethods. VCNSprovidesrealtimeexplorationofverylargedatasetso fcompletehumanbody, unlikethepreviousapproachesthatallowspartofthebodyt obeexploredatagiventime. Thisisachievedbydividingthedataintosmallbricksandpa gingonlytherequiredbricks fromtheharddiskintothemainmemoryandthetexturememory forvisualization.The softwareusesonlyaxedamountofmainmemoryandthetextur ememoryduringthe execution,anddividesthemintoslotsthatarereusedtorep laceoldbrickswiththenew bricks.Spatialandtemporallocalityisusedtospeedupthe accesstothebrickdata.The systemachievesrealtimeframerates(30frames/secandhig her)onanypersonalcomputerwithtexturememoryaslowas64Megabytesandmainmemo ryof512Megabytes. Figure1.1showshowtheprobecanbeusedtointeractwiththe mannequintoobtainthe tomographicsliceduringthetrackingmode. 6

PAGE 18

Figure1.1.TheDiagramShowsHowVCNSCanBeUsedToObtainCr ossSectionalView (TomographicSlice)OfVirtualHumanBodyAtArbitraryLoca tionsAndOrientation DuringTheTrackingMode 7

PAGE 19

Thefollowingsectiongivesasummaryofthecontributionso fthethesistowardsthe developmentoftheVCNS.1.6ContributionsofThesis 1.Avirtualrealityframeworkisdevelopedforlearninghum ananatomy,basedonreal timenavigationandvisualizationofverylargecadavericd atabythewayofobtaining virtualtomographicslices,bymovingaexaminationprobeo veramannequin. 2.Anecientstoragemechanismisdevelopedfororganizing andaccessingverylarge volumetricdatabydividingitintosmallblockscalledasbr icks,andarrangingthem hierarchicallyina3-Dspatialdatastructurecalledbinar yspacepartitiontree(BSP). 3.Algorithmsaredevelopedandimplementedfor:(i)extrac tingonlytherequiredbricks fromtheBSPtreebyperforminga3Dcollisiondetectionbetw eenthetrackerplane andthebricks,basedontheframeworkproposedby[13],and( ii)obtainingthepoints ofintersectionbasedonthe3DSutherlandHodgmanclipping algorithm[56]. 4.Visualizationalgorithmsaredevelopedforcomputingno rmalizedtexturecoordinates fromthevertexcoordinatesofthecuttingplaneobtainedby clippeditagainstindividualbricks,andformapping3Dtexturedatafromthebri cksontothecutting planetoobtaintomographicslice. 5.Dataaccessalgorithmsusingtemporal,spatialandseque ntiallocalityareimplementedforecientmanagementoftexturememoryandthemain memory.The leastrecentlyusedbricksinthetexturememoryarereplace dbybringingthenew requiredbricksfromthemainmemoryandfromtheharddiskin tothemainmemory. 1.7OrganizationofThesis Thethesisisorganizedasfollows.Chapter2givesaovervie wofcurrentresearchinthe areaofvirtualrealitybasedmedicaltrainingsystems.Cha pter3explainsvisiblehuman datasetusedinthesystemfornavigation.Thearchitecture oftheVCNSispresentedin 8

PAGE 20

Chapter4followedbyadetaileddesignandimplementationo fthesysteminChapter5. Chapter6presentstheresultsobtainedunderdierentmode sofoperationoftheVCNS, andadiscussiononadvantagesofusingVCNSasateachingtoo landthefuturework. 9

PAGE 21

CHAPTER2 LITERATUREREVIEW Virtualreality(VR)techniqueshavebeenproventobeusefu linprovidingmedical traininginconvenientandeconomicwaycomparedtotheconv entionaltrainingprocedures thatusedissectionofrealhumancadavereachtime.Thischa pterprovidesanoverview ofthecurrentandpreviousresearchintheeldofvirtualre alitybasedtrainingmethods inmedicalscience.Areviewoftechniquesforvisualizatio noflargevolumedata,andthe challengesandissuesinusingthevirtualenvironmentsfor healthcare,isalsopresented. TheapplicationsofVRineldofmedicinecanbeclassiedin tofollowingcategories [43]: 1.Surgicalplanningandprocedures.2.Virtualendoscopy.3.Medicaleducation.4.Neuropsychologicalassessmentandrehabilitation. 2.1SurgicalPlanningandProcedures Studentslearningsurgicalproceduresareoftentrainedon inanimatetissuesandmodels duetocostissues.ThescienceofVRopensanentirelynewpat hforacquiringthesurgical skills,usingcomputersfortrainingandevaluation.Theea rlyeortsfocusedoncreating surgicalsimulators(forexample,theabdominal-surgerys imulatorby[50],andthelimbtraumasimulatorbyDelpet.al.[16])sueredfromproblems suchaslow-resolution graphics,lackoftactileandforcefeedbacksandthelackof realisticdeformationsoforgans. Inthelastdecade,eortshavebeendirectedtowardsdevelo pingVRbasedtrainingsystems 10

PAGE 22

toaugmentthelearningexperiencewithmoresophisticated andrealisticvisualizations andsensoryfeedbacks.Forexample,theMIST-VRtrainer[Al i02,Gor03],usedfor performingbasiclaparoscopicskills,havebeenshowntopo ssessmoretrainingecacy thanthetraditionalprocedures. Apartfromtrainingsurgicalstudents,theuseofVRenviron mentsinperformingremote surgeryisopeningnewpossibilities[64].Usingtelesurge ry,surgeonscanparticipateinthe battleeldoperationsfromremotesites;operateonapatie ntinruralareas,inanairplane, onshiporevenatspacestation,remotelyfromtheiroce[37 ].Besidesovercomingthe geographicalbarriers,telemedicinealsoresultsinthere ductionofexposuretodiseases, andreductionincostsasaresultofreducedtrauma[49].VRh asalsohelpedsurgeonsby superimposingtherealimageswiththevirtualimagesrecon structedfromtheMRIand CTdata,usingatechniqueknownasaugmentedreality.Sucht echniqueshavebeenshown tobeveryeectiveinperformingorthopaedicandtumorsurg eries[17]. VRhasbeenbenecialinimprovingtheplanningprocessdone beforetheactualsurgery. Theplanningrequiresthesurgeonstomentallyintegrateas eriesoftwo-dimensionalMR and/orCTimagestoforma3Dviewoftheanatomy.Thismentalt ransformationis dicult,sincetheanatomyisrepresentedindierentscann ingmodalitiesonseparate imageseries,usuallyfoundindierentsites/departments [43].AVR-basedsystemcan beusedtoreconstructrealistic3Dmodelsofthetraumatize dpartwithabilitytoperform grasping,clamping,cutting,andbleedingorleakingofrui ds.Thus,VRcanimprovethe waysurgeonsplanproceduresbeforesurgery[37].Onesuchs ystemistheNetra[23]used forplanningprecisionbiopsies,laser-guidedtumorresec tionsandsurgeryforParkinson's disease.AnotherexampleistheCyberscalpelsurgicalplan ningsystemdevelopedbyNASA researchers,whichhasbeensuccessfullydemonstratedtop lantheoperationoftheperson withcancerofthejaw[46]. 11

PAGE 23

2.2VirtualEndoscopy Endoscopyinvolvesdiagnosis,byinsertinginvasiveormin imallyinvasiveinstruments intothepatients.Theseproceduresareoftennotperfectan dpatientsaresubjecttocomplicationssuchasbleeding,perforationetc.Also,thecos tofendoscopyissignicant.Asa result,virtualendoscopyisbeingexploredasanalternati vebydierentresearchers[48,26]. ItinvolvesperformingstandardCTorMRIscansoftheareaof bodyofconcernandsegmentingthevariousorgansandtissues.Sophisticatedrigh tpathalgorithms,derivedfrom terraintrackingalgorithmsusedinthemilitary,areusedt orythroughtheseorgansprovidingimagescomparabletoperformingrealvideoendoscop y[35,51].Thistechniquecan beextendedtoexplorethepartsthatareinaccessibletothe realendoscopicinstruments, eitherbecauseitistoodangerous(suchaspartsinsidethee ye)ortoosmall(innerear) [49].Typicalexamplesincludecolon,stomach,esophagus, tracheo-bronchialtree,sinus bladder,ureterandkidneys,pancreasorbiliarytree[38]. Virtualendoscopicsimulators arecosteectiveandcompletelynon-invasivewithnocompl icationstothepatientunlike theirrealworldcounterpart[19].2.3MedicalEducation VRprovidesadeeperunderstandingandappreciationofthea natomicalstructureand relationshipsbyallowingthelearnertoryaroundand/orth roughthevariousorgans. Theextraordinaryperspectivesmadeavailablebytheselea rningtoolscanprovemore eectivethantheanyoftheothermeans,includingcadaverd issection.Anatomyteaching beingmainlydescriptivecanbegreatlybenetedbyVRenvir onments[18].Severalweb basedapplicationshavebeendevelopedbasedonthedataobt ainedfromthevisiblehuman dataset[55]toteachtheanatomybyobtainingcross-sectio nsthroughthevirtualhuman [6].Theavailabilityofthevisiblehumandataovertheinte rnetunderano-costlicense agreementhasspurredthecreationofhugenumberofeducati onalvirtualenvironments. Duetoshortageofcadaversrequiredfordissections,a3Dli felikedetailedvirtualhuman bodyoersaworkablereplacement.Infact,these3Dmodelsc anallowmorerealistic 12

PAGE 24

trainingthantheuseofcadavers.Forexample,itispossibl etosimulatebloodrowing outofavirtualbloodvesselwhenitisprickedunlikethecad avers,inwhichthecolor ischangedandthearteriesnolongerpulsate[37].Also,pre viouslycutpartscannotbe reattached.Inthevirtualenvironmenttheprocedurescanb erepeatedinnitely,andcan alsobestoredforanalysisforthelatergroup. Thus,VRoersdynamicenvironmentinwhichmodelsofvariou sorgansandsystems canmoveduringnormalordiseasedstates,orrespondtovari ousexternallyappliedforces providinganexperimentalanddidacticplatformforlearni nghumananatomy. 2.4NeuropsychologicalRehabilitationandPsychology VRisemergingasapromisingmediumfortreatingpatientswi thpsychologicaldisorders.Itisranked3rdoutof38psychotherapyinterventions ,thatarepredictedtoincrease inthenext10years[40].Intheeldofpsychology,VRhasbee nusedtocreaterealworld situationstailoredaccordingtothepatient'spsychologi caldisturbances.Inthevirtual environments,nothingthepatientsfearcanreallyhappent othem,givinganassurance thattheycanfreelyexplore,experimentandexperiencefee lingsandthoughts[14].This providesthemnotonlyanawarenesstodosomethingtocreate changeintheenvironment theyareimmersedin,butalsotoexperienceagreatersenseo fpersonalecacy[43].Till now,theclinicaleectivenessofVRhasbeenveriedinthet reatmentofsixpsychological disorders:acrophobia[20],spiderphobia[21],panicdiso rderswithagrophobia[60],eating disorders[44]andfearofrying[47,36,63].However,mosto fthisresearchisbasedon controlledstudiesandpilottrials,withlimitedconvinci ngevidenceabouttheecacyand practicalityoftheiruse. Although,VRhasbeenusedintheareaofcognitiverehabilit ation[52,45],thereareno clinicaltrialstosupporttheirecacyexceptintheassess mentofcognitivefunctionsinpersonswithacquiredbraininjuries[43].VRhasbeenshownasa veryeectivepsychometric toolinthiseld[65,41]. 13

PAGE 25

2.5IssuesandChallengesinusingVRforHealthcare ThereareconcernsaboutusingVRinclinicalenvironments. Thesafetyissuesthat ariseinusingVRsystemsaresymptomssuchasmotionsicknes s,strainonocularsystem, reducedsenseofpresence,anddevelopmentofresponsesina ppropriatefortherealworld, whichmightleadtonegativetraining[33]. ThesesymptomscanbereducedwithimprovedqualityofVRsim ulations.Asnotedby [42,39],inmostoftheindividualstheseeectsaretransie ntandminor,andsubsidequickly. Nonetheless,precautionsshouldbetakentoensuresafetyo fpatientsbymonitoringand controllingtheirexposuretovirtualrealityenvironment s[33]. Thetechnicalissuesincludethelackofstandardizationin thesystems,andtheperformancefactors.Currently,everyVRsystemrequiresdealing withconrictinghardwareand driversoftwareandtypicallyrequiresadedicatedstaora computertechniciantoensure itssmoothworking.Othertechnicalchallengesinimplemen tingVRsystemsincludethese [25]: 1.Highcomputationneedsofvirtualacousticdisplaysfors imulatingevenasmallnumberofsources. 2.Limitedfunctionalityoftactilefeedbackmechanisms.3.Qualityandperformancetradeosoeredbyimagegenerat orsthatcannotprovide lowdisplaylatency. 4.Inadequaterobustness,smallworkingvolumes,latencya ndpoorregistrationofpositiontrackers. 5.LimitedeldofviewsandencumberingformfactorsofHead MountedDisplays. 6.Lackofeectiveolfactorysensors. Finally,thehighcostsassociatedwiththedevelopmentand managementoftheVR systemsisalsoamajorissueinwidespreadusageoftheVRbas edsystems.Thecost 14

PAGE 26

requiredfordesigningaclinicalVRapplicationfromscrat ch,andtestingitonclinical patientmayrangebetween150,000and200,000USdollars[43 ]. 2.6VisualizationofLargeVolumeData Althoughdierentsensoryfeedbacksarevitaltovirtualre alism,visualfeedbackstill remainsthemostimportantformoffeedbackintheVRbasedsy stems.Withtheadventof newimagingtechniquessuchasMagneticResonanceImaging( MRI),ComputedTomography(CT),PositronEmissionTechnology(PET)theamounto fdatathattheclinicians needtohandleisincreasing.Inordertoproviderealisticv isualizations,itisnecessaryto presenttheoutputwithnestdetailspossibleinrealtime. Thus,itisnecessarytomanage thelargehighresolutionvolumedatarequiredfor3Drender ing.Thissectionprovidesan overviewoftheresearchinthevisualizationandlargedata managementtechniques. Volumevisualizationisaclassicalproblemincomputergra phics.Thevolumedata canbeeithersurfacerendered[34]orvolumerendered.Inca seofvolumerendering,it ispossibletoseeinsidethesurface,makingitusefulforme dicalvisualizationoforgans. Thefastestsoftwarebasedvolumerenderingalgorithmisth e\Shear-WarpFactorization", whichoperatesbyfactoringtheviewtransformationintoa3 Dshearparalleltothedata slices,aprojectiontoformanintermediatebutdistortedi mageanda2Dwarptoforman undistortednalimage[31].Newtechniquescombinehardwa resupportfor3Dtextures [22]andmulti-resolutiontechniquestodisplaythevolume oncomputerswithlowtexture memoryandatrealtimeframerates. The3Dtexturesupportfromthegraphicshardwarehasmadere altimerendering possible.Stillthesizeofthedataremainsaproblembecaus eofthelimitedamount oftexturememoryavailableonthegraphicsboards.Themult i-resolutionmethodwith texturepagingprovidesasolutiontothedatasizeproblemb ycompromisingthequality toatolerableextent.Itoperatesbydividingthevolumedat aintosmalldatabricks andarrangingthemwithdecreasinglevelofresolutionsusi nghierarchicaldatastructures [Plateet.al.2002].Inthisapproach,aparticularresolut ionisselectedforthebrick, 15

PAGE 27

dependingonitsdistancefromthecamera.Thebrickscloser tothecameraaredisplayed atthehigherresolutionthantheonefartherfromthecamera .Thistechniquemakes visualizationofdataaslargeas16GBytesfeasibleoncompu terswithtexturememoryas lowas64Mbytes.Forexample,LaMaret.al.[32]haveusedthe multi-resolutionhardware basedtexturerenderingforlargevolumevisualizationand viewingarbitrarilyorientedslices throughVisibleFemaledataset,whileVolz[62]haveusedco mbinedhardwareandsoftware techniquesforviewingseismicdatasets.2.7ContextofthisWork MostofthepreviousworkintheareaofVRbasedmedicaltrain ingisfocusedon surgicaltrainingandsimulations.Thereareonlyafewsyst emsspecicallydevotedto teachinganatomy[57][18].Themaindrawbackofthesesyste msisthat,theyaretargeted atspecicpartsofthebody,andlackintegrationofhighlev eltissuespecicinformation, whichisimperativeforlearninghumananatomy.Inaddition ,theyoerlimitationsinterms oftheamountofdatathatcanbeexploredatatime.Theworkpr esentedinthisthesis isaimedatcombiningtheadvantagesoeredbytheprevioust echniqueswithahighlevel knowledgeoftissuerelatedinformationtherebyproviding anacomprehensiveforanatomy learningsystem.Inaddition,theproblemwiththedatamana gementlimitingtheprevious systemsisovercomebyintelligenttexturepaging,andhard warebased3Dtexturemapping techniques.Thus,theproposedsystemoersacosteective comprehensiveteachingtool, whichcanbeusedforteachinganatomyaswellasforselflear ning. 16

PAGE 28

CHAPTER3 VISIBLEHUMANDATASET ThesystemhasbeentestedonthedataobtainedfromfromtheN ationalLibraryof Medicine's(NLM)VisibleHumanProject.Theaimofthisproj ectwastoobtainMRI,CT andhighresolutionRGBimagesofthehumanmaleandfemale.T hefollowingsections explainthetechniqueusedforacquiringthedatasetsandth eleformatofthedata. 3.1VisibleMale NLMawardedacontracttotheUniversityofColoradoHealthS ciencesCentertocreate thedigitalcross-sectionsofa39-yearoldconvictedmurde rer(male)whohaddonatedhis bodytoscience.MRandCTdatawerecapturedfromtheunfroze nspecimenjustafew hoursafterthedeathtoavoidthedeterioration.TheMRscan softheheadweretaken alongtheaxialplane,whiletheremainingbodyscanwasperf ormedalongthecoronal plane.Theslicethicknesswas4mminboththecases.Theimag eswerestoredas256X 256X16bitintheGeneralElectricGenesisformat[55].TheC Tscansfromheadtoneck weretakenat1mmintervals,every3mmthroughthorax,abdom enandpelvis,andevery 5mminthelowerextremities[55].AfteracquiringtheMRand CT,themalecadaverwas frozenbyplacingitinaspeciallyconstructedchambercove redwithdryiceandwasdivided intofoursectionsfromheadtotoe.Therstsectionwasfrom legtotoes,secondincluded kneesandthighs,thirdwasmadeofabdomenandpelvisandthe fourthcontainedhead, neckandthorax.Eachsectionresultedinablockwhichwasmi lledat1mmintervalusing acryomacrotome(cuttingmachine)developedatUniversity ofColoradoMedicalSchool. Aseachlayerwasexposed,acolorRGBphotographwastaken.T hisprocesscaptured theimagesin2048X2048X14bitTIFFformat.Thisrawdatawas furtherprocessedto 17

PAGE 29

obtain1800,2048X1216X24bitimagesat1mmintervalsresul tinginaRGBdataof size9GB.ThedatacanbedownloadedfromtheNLMwebsite[6]. Figure3.1showsthe MRI,CTandRGBsliceofthevisiblehumanhead.3.2VisibleFemale TheVisibleFemaleimageswereobtainedfroma59year-oldwo manwhodiedofcoronaryarterydisease.Although,theprocesswassimilartoth atoftheVisibleMale,the sliceswereobtainedat0.33mmregularintervalsasopposed tothe1mmthickness.The VisibleFemalecontainsslightlyover5,000imageswithato talof39GB.Figure3.3shows theRGBsliceofthevisiblefemalehead.3.3SegmentedDataset ThesegmentedVHmaledatasetwascreatedatUniversityofMi chigan,bymanually labelingeveryvoxelinthe16-bitgrayscaleimagesobtaine dfromtheoriginalcoloredVH maledata.The16-bitgrayscalevaluesareconvertedtoa24b itcoloredsegmentedimage byassigninguniqueR,GandBcolorvaluestoeveryclass.The reareapproximately1600 structuresclassiedandlabeledandcanbeloadedintotheV CNSusingthenamelookup le.ThisleconsistsofcommaseparatedR,G,Bvaluesandth eircorrespondingclass name.Atransaxialslicethroughthesegmentedvolumeissho wninFigure3.2. 3.4DataFormat Currently,VCNSsupportsonlythe\multipledirectorytagg edimageleformat(TIFF)", thatcanstoremultipleTIFFimageswithinasingleTIFFle. However,itispossibleto loadthedataindierentformatsdirectlyintothesystemby implementingasinglemethod ( loadDataBrick )oftheplug-inclasscalled VolumeDataLoader .TheTIFFimageisdened asasequenceofbytesthatstartswithaleheadercalledas imageledirectory (IFD) andpointerstotheactualimagedatafromthetheIFD.Incase ofthemultipledirectory TIFFletherearemorethanoneIFDsperle.TheTIFFleform atisexplainedindetail 18

PAGE 30

Figure3.1.VisibleHumanMaleImages(a)MRIImage(b)CTIma ge(c)HighResolution ColoredImage Figure3.2.SegmentedTransaxialSliceThroughVisibleMal eThorax Figure3.3.ATransaxialSliceThroughVisibleFemaleHead 19

PAGE 31

in[8].VCNSusesasoftwarelibrarycalled\libti"[3]tore adandwritetheTIFFles. ThislibraryalsoprovidesfunctionsforconvertingtheTIF Flesintoseveralotherformats suchasJPEG,PostScriptetc. WhentherawdataisloadedasaTIFFleintothesystem,theda taisconvertedinto bricksandwrittentoalecalledas\brickle",thatstores thedatacorrespondingto abrickcontiguously.Thisschemeofcontiguousdataorgani zationhelpsinreducingthe downloadtimeforthebrickdata,fromtheharddisk.Thebric kleisgeneratedforthe rawinputdataasapartofpreprocessingstep.Thislecanbe createdonce,foragiven inputdata,andcanbeusedsubsequently.Thisspeedsupthes tartuptimeforthesoftware sincethepreprocessingstepofbrickingisskipped.Thefor matofthebricklehasbeen designedspecicallyforVCNSisasfollows: //FileheaderMagicNumber:1byte(mustbe0XBF)dataformat:2bytesnumberofcomponentspervoxel:1bytenumberofbytespercomponent:1bytevoxeldimensionsinmm:3bytesnumberofrows:4bytesnumberofcolumns:4bytesnumberofbricks:4bytesbrickdimensions:4bytessizeofbricksinbytes:8bytes//ActualBrickData..... 20

PAGE 32

CHAPTER4 ARCHITECTUREOFVCNS Thischapterdescribesthesoftwarearchitectureofthepro posedsystem.Thevarious modulescomprisingthesystemaredescribedrst,followed bythecomputationalrowof thesystemaswellasthatofindividualmodules.Finally,th ehardwarecomponentsusedin thesystemaredescribedalongwiththegraphicaluserinter face(GUI)designedtointeract withthesystem.4.1ModesofOperation Thesystemisdesignedtooperateinthreedierentmodeswhi chareasfollows: 1.Trackingmode:Inthismode,therealhumancadavericdata canbeexploredby movingtheexaminationprobeoverthemannequin.Thesoftwa recomputesthe2D planarslicesfromthedatainrealtimeandtheoutputimagei sshownonthedisplay inaformsimilartotheultrasoundimage. 2.CentroidLocationorRegistrationmode:Thismodehelpst heusertoalignthe centroidsofthemannequin,andthevolumetricdata,loaded inthesoftware.Itis necessarytoalignthecentersbeforeexploringthedata,in ordertoobtaincorrect slicesinthetrackingandlabelingmode. 3.Labelingmode:Thismodeisusedforexploringthedatafor which,thesegmentation informationisavailable.Itworksinthesamewayasthetrac kingmodeexceptthat, theresultingimagecanbeusedtoobtaintissuespecicinfo rmation,suchasits name.Thismodecannotbeusedifthesegmentationdataisnot available. 21

PAGE 33

Figure4.1.SoftwareArchitectureOfVCNSShowingDierent Modules,AndInputsAnd OutputsOfTheSystem 22

PAGE 34

4.2SoftwareArchitecture Inordertoperformtheabovefunctions,thesoftwareisdivi dedintoseveralmodules. Figure4.1showsthearchitectureofthecompletesystem.Th einputstothesystemare comprisedofthesegmentedvolumedataand/orthenonsegmen tedvolumedata.Volume dataconsistsofseriesoftomographicimagesacquiredusin gtechniquessuchasmagnetic resonanceimaging(MRI),computedtomography(CT)orobtai nedfromthevisiblehuman dataset.WhentheVCNSisoperatedinoneoftheaboveexecuti onmodes,itperforms variousdatamanagementstepsbeforeobtainingthenalout put.Thisfunctionalityis dividedintodierentmodules,whichareasfollows:4.2.1DataManagementModule Thismoduleisresponsibleforhandlingallthedatarelated issues,whichincludedata organizationanddataaccess.Itiscapableofperformingma nagementofverylargedata, bydividingitintosmallblockscalledas\bricks",thatrep resentasubvolumeoftheoriginal volumetricdata.Theprocessofformingthebricksfromtheo riginalrawdataiscalledas \bricking".Itisrequiredtoconverttherawvolumetricdat aintothebrickedformbefore itcanbeusedinthesystem.Thisenablesfasteraccessanddo wnloadingofabrickfrom theharddisksincealldatacorrespondingtoabrickiswritt encontiguouslyonthedisk. Thefunctionalityofthismoduleisdividedintothefollowi ngsubmodulesasshowninthe Figure4.2:4.2.1.1DataI/O Thismodulehandlesthereadingandcreationofvolumetricb ricksfromtherawdata. Itismadeupoftwoparts:(i)DataLoaders,thatareresponsi bleforreadingthebrickdata aswellastherawdatafromtheharddisk,andcanbecustomize dtosupportdierentle formats,(ii)DataWriters,thatareusedtowritethebricke ddatageneratedasaresult ofthebrickingprocess.Thedatawritersaugmenttherawdat awiththetransparency valuesforeveryvoxelduringthebrickingprocess,sothati tcanbeusedasatextureusing 23

PAGE 35

Figure4.2.SoftwareArchitectureOfDataManagementModul e OpenGLtexturemappingroutines.Thus,thedatagenerateda saresultofthebrickingis biggerinsize,thantherawinputdata.4.2.1.2ResourceManager Thismoduleisthemostcriticalpartofthesystem,andisres ponsibleforallocating, accessing,andmanagingthemainmemoryandthetexturememo ryresourcesatruntime. Itallocatesaxedamountofthemainmemoryandthetexturem emory,anddividesthem intoslotsofsizethatisamultipleofbrickdatasize.Itmai ntainsadirectmappingbetween theslotaddressesandthebrickaddresses,andimplementsa taggingschemeinorderto locatetheminthememory.ThismodulealsoimplementsaLRU( leastrecentlyused) replacementpolicy,toreplacethebricksthatarenolonger needed,andusesthespatial andtemporallocalitytominimizetheaccesstimeforthebri ckdatafortherequiredbricks. Figure4.3.SoftwareArchitectureOfVisualizationModule 24

PAGE 36

4.2.2VisualizationModule Thevisualizationmoduleimplementsthealgorithmsforcom putingthetexturecoordinatesand3Dtexturemapping.3Dtexturesconsistsofastack oftwodimensionaltexture imagesandhavetobeloadedintothetexturememorybeforeth eycanbeusedfortexture mapping.3Dtexturemappingisatechniqueofextractingapa rtofthedataatarbitrary positionsandorientationsfromthisimagestack,andrequi resinterpolatingthetexture dataatthepointswhereitisnotavailable.Thisisusuallyi mplementedbythegraphics hardwareusingtrilinearorhigherdegreeinterpolation.T hevisualizationmoduleusesthe hardwaresupportfor3Dtextures,inordertoextractthesli cefromthevolumedata.In ordertoachievethis,itisnecessarytospecifythetexture coordinatesfortheslicetobe mapped,inanormalizedform.Theinputstothevisualizatio nmoduleinclude,thetexture dataforthebricks,andthecoordinatesofpointsofinterse ctionofthecuttingplanewith thatofthebricks.Thefunctionofthismoduleisdividedint otwoparts(Figure4.3),which are,texturecoordinategeneration,and3Dtexturemapping .Theoutputofthismoduleis thesetofverticesandtheircorrespondingtexturecoordin atesthataresentthroughthe OpenGLgraphicspipeline,toobtainthenaltexturemapped image. 4.2.3TrackingModule Thefunctionalityofthetrackingmodeisencapsulatedinth ismodule.Itismade upofseveralothermodulesasshownintheFigure4.4.Thismo dulehandlesthedata rowbetweenthevariousmodules,inordertoobtainthenalt exturemappedimagein thetrackingmode.Forexample,itcomputesthegeometryoft hevirtualcuttingplane Figure4.4.SoftwareArchitectureOfTrackingModule 25

PAGE 37

emanatingfromtheprobe,queriesthecollisiondetectormo duletondoutthebricks intersectedbytheplaneatagiventime,requeststhetextur edataforthesebricksfrom thedatamanagementmodule,andnallyprovidesthedatanec essaryforvisualizationof thetexturemappedimagetothevisualizationmodule.4.2.4CentroidLocatorModule Thismoduleencapsulatesthefunctionalityofthecentroid locatormode.Itismadeup ofseveralsubmodulesasshownintheFigure4.5andcoordina testhedatarowbetween thesemodules.Whenthesystemisoperatinginthecentroidl ocatormode,thismodule queriesthepositionsofthethreeorthogonalplanesfromth econtrolpaneldialogusingthe planepositiontrackerinterface,andcomputestheirgeome trybasedonthisinformation. Itstoresthebricksintersectedbyeachoftheselectedorth ogonalplanesandrequeststhe texturedatacorrespondingtothemfromthedatamanagement module.Alltheselected bricksaresenttothevisualizationmoduletoobtainthena limage. 4.2.5LabelingModule Thesegmentationandtissueclassicationinformationare essentialinputsforthismoduleinadditiontothenonsegmenteddata.Thetissueclassi cationinformationisprovided asamappingbetweentheiranatomicalnamesandcolorvalueo ftheclassrepresentingthe tissue.Thefunctionalityofthismoduleissimilartothato ftrackingmodulewithadierencethat,itusesthelabelgeneratormoduleforobtainingt hetissuespecicclassication informationfortheresultingslice.Inordertoobtainthel abels,thismodulegeneratesthe tomographicslicefromthesegmentedaswellasthenonsegme nteddatasetinthewaysimFigure4.5.SoftwareArchitectureOfCentroidLocatorModu le 26

PAGE 38

Figure4.6.SoftwareArchitectureOfLabelingModule ilartothetrackingmode.Thisinformationalongwiththeti ssueclassicationinformation isgivenasinputtothelabelgeneratormoduletoobtainthel abelsforthetissuesonthe slice.Thevarioussubmodulescomprisingthismoduleareas shownintheFigure4.6. 4.2.6CollisionDetectionModule Thismoduleisusedbythetracking,centroidlocatorandlab elingmodulefordeterminingthebricksthatareintersectedbytheplaneatanygiv entime.Theinputtothis moduleiseitherthecuttingplaneemanatingfromtheprobe( incaseoftrackerandlabelingmode)oroneofthethreeorthogonalplanes(incaseofcen troidlocatormode).This moduledenesabinaryspacepartition(BSP)treeforarrang ingthebrickscomprisingthe volumedata.ABSPtree(Figure4.7)isaspatialdatastructu rethatorganizesthevolume data,byrecursivelydividingitintosmallersubregions,a ndarrangingtheminabinary treestructure.Therootofthetreerepresentsthewholevol ume,internalnodesrepresent thesubregionsobtainedbydividingtheregioncorrespondi ngtotheroot,alongoneofthe threex,yorzplanes,andtheleafnodesrepresentthebricks .Thecollisiondetection moduleusesthe3Dcollisiondetectionalgorithmsproposed by[13]tosuccessivelyquery theinternalnodestodetermineintersectionwiththecutti ngplanetillitreachestheleaf nodesandreturnsthebricksintersectedbytheplaneasshow nintheFigure4.8.Itisalso determinesthepointsofintersectionoftheplanewiththee achoftheintersectingbricks 27

PAGE 39

Figure4.7.BinarySpacePartitionTree Figure4.8.CollisionDetectionProcess 28

PAGE 40

byclippingtheplaneagainsteachbrickusingSutherlandHo dgman3Dpolygonclipping algorithm[56]. Theothermodulesusedinthesystemare,trackerinterfacem oduleusedbythetracking moduletoinitializethemotiontrackersystemandtrackthe positionoftheprobetoobtain its3Dpositionandorientation,andtheplanetrackerinter facemodulethatisusedinthe centroidlocatormodetoobtainthepositionsoftheplanefr omthecontrolpaneldialog box. Thefollowingsectiongivesthecomputationalrowoftheove rallsystem,andexplains indetail,thestepsperformedineachofthemodes,andthere sultingdatarowacross variousmodules.4.3DataandComputationalFlowofVCNS Therowchartsummarizingthecomputationalrowofthecompl etesystemisshownin theFigure4.9.Initially,thesystemisintheidlestatewai tingfortheinputfromtheuser. Whentheuserloadstherawnonsegmentedvolumedata,itisr stconvertedintobricks andwrittentoanintermediateleasapartofpreprocessing step.Next,theappropriate executionpathisselecteddependinguponthecurrentmodeo foperation.Incaseof labelingmode,theuserispromptedforsegmentationandtis sueclassicationinformation beforeexecution. Thefollowingsectionsexplainthecomputationalrowandth edatarowwithinthe individualoperatingmodes.4.3.1TrackingMode Inthismode,theprobecanbemovedoverthemannequintoobta invirtualtomographic slicesfromthevolumedata.Thetrackingmodulereadsthepo sitionandorientationof theprobeandgeneratestheplanebasedonthisinformation. Next,theplanegeometry (3Dcoordinatesofthevertices)ispassedtothecollisiond etectormoduletoobtainthe intersectingbricksandthepointsofintersectionasshown intheFigure4.10.Thedata 29

PAGE 41

Figure4.9.ComputationalFlowOfVCNS 30

PAGE 42

Figure4.10.ComputationalFlowChartAndDataFlowDiagram ForTrackingModeOf VCNS 31

PAGE 43

managementmodulearrangesthethebrickssuchthatbricksa lreadypresentinthetexture memoryareprocessedbeforethebricksresidinginthemainm emory,followedbythose presentontheharddisk.Iftheresultingnumberofbricksar emorethanthemaximum numberofbricksthatcanbeaccommodatedinthetexturememo ry,thetrackingmodule processestheminbatches.Eachbatchispassedtothedatama nagementmodulefortexture binding.Theintersectiondataandthetexturedataforthes electedbricksisnallypassed tothevisualizationmoduleasshownintheFigure4.10(b).T hecomputationalrowchart ofthetrackingalgorithmisshownintheFigure4.10.Theinp uttothetrackingmodeis thenonsegmentedbrickeddataandtheoutputisanimagesimi lartoanultrasoundimage asshowninFigure4.10.4.3.2CentroidLocatorMode Thesystemisoperatedinthismodetoalignthecenterofthem annequinandthecenter ofthe3Dmotiontrackersystemwiththatofthevolumedata.T hisisdonebymovingthe threeorthogonalplanes,calledastransaxial,sagittalan dcoronal,usingtheslidercontrols onthecontrolpaneldialogboxshownintheFigure4.14.Thep ointofintersectionofthe threeslicesisconsideredascenterofthevolumeandisused formakingcorrectionstothe positionoftheprobeinthetrackingmode.Thecomputationa lrowchartforthismodeis shownintheFigure4.11.Theprocedureforobtainingthetex turemappedimageissame asthetrackingmode,andisrepeatedforeachoftheselected orthogonalplanes.Theinput tothesystemisthenonsegmentedbrickedvolumedata.Theda tarow,andtheoutput ofthesystemareshownintheFigure4.11.4.3.3LabelingMode Inthismode,thesegmentationdataisusedtoobtaintissues pecicinformationfrom thetomographicslice.Inordertoobtainthecorrectclassi cationinformation,thecutting planeemanatingfromtheexaminationprobeisusedtoobtain theslicefromsegmentation data,inadditiontothenonsegmenteddataslice.Theproced ureforobtainingtheslice issameasthetrackingmode,asshownintheFigure4.12.Inth iscase,however,the 32

PAGE 44

Figure4.11.ComputationalFlowChartAndDataFlowDiagram ForCentroidModeOf VCNS 33

PAGE 45

Figure4.12.ComputationalFlowChartAndDataFlowDiagram ForLabelingModeof VCNS 34

PAGE 46

outputofthevisualizationissavedafterobtainingthesli cefromthesegmenteddataset andisfurtherprocessedobtainthelabelsbythelabelgener ationmodule.Thelabelsare determinedforallthetissuespresentintheslicebyscanni ngthesegmentedsliceobtained intheaboveprocess,andareoverlaidonthesliceobtainedf romthenonsegmenteddata. Labelscanalsobeobtainedforasingletissue,byrightclic kingonavoxelontheresulting sliceasshownintheFigure4.12.4.4HardwareComponentsoftheSystem Followinghardwarecomponentsareusedinthesystem: 1.pciBird TM MotionTrackerSystem. 2.GraphicsWorkstation.3.Mannequin. 4.4.1PCIBirdMotionTracker ThepciBird TM isanelectromagnetic3Dmotiontrackersystemdevelopedby Ascension Technology[2].Itismadeupoffollowingthreecomponents:4.4.1.1Transmitter Atransmitterconsistsofahighpermeabilitycorewiththre esetsofwindings(X,Y,Z) placedatrightanglestoeachotherandproducemagneticel ds,whencurrentispassed throughthem[11].Thestrengthofthemagneticeldishighe stnearthetransmitter,and decreaseswithinverseofthesquareofthedistancefromthe transmitter.Figure4.13(a) showsthetransmitterandthepositionofthecartesiancoor dinatesysteminsideit. 4.4.1.2Sensor Asensor(Figure4.13(b))isaprecise3-axisringcoreruxga temagnetometerwithahigh permeabilitycoreatthecenter.Itoperatesbyperiodicall ydrivingthecoretosaturation 35

PAGE 47

Figure4.13.ReferenceFramesFor(a)StandardTransmitter (b)25mmsensor(Courtesy: AscensionTechnology)makingitspermeabilityclosetothatoftheair.Theruxgate magnatometer,then,changes thecoecientofmagneticpermeability( ,whereB= H)to2andmeasurestheEMFat theendsofthecoils,whichisproportionaltotheDCmagneti celdnearthesensor.Ring coreruxgatesoerveryhighperformanceascomparedtotheo thersensingtechnologies [11].4.4.1.3Electronics Theelectronicsconsistsofafulllength(12.23inch)32bit v2.1PCIcardandhandles transmitterdrivecircuitry,sensorsignalprocessing,da taconversion,processing,power conditioning,andhostinterfacefunctions[11].4.4.1.4MeasurementTechnique Thetrackersystemmeasurestheposition(X,Y,Z)andorient ation(Pitch,Roll,Yaw) ofthesensorineverymeasurementcycle.Thenumberofmeasu rementstakenpersecond canbeprogrammedusingthesoftwareinterface.Ameasureme ntistakenbysuccessively energizingeachofthethreecoilsofthetransmittertothep ointofstability.Atthis point,thecurrentinducedatthecoilalongcorrespondinga xisofthesensorismeasured, 36

PAGE 48

andtransmittedtotheelectronics.Theelectronicscalibr atesthereceiveddata,subtracts noise,andcomputestheangleandposition.Thenaloutputi smadeavailableatthe hostinterfaceandcanbereadusingthesoftwareinterface. Itispossibletomeasurethe positiondataindierentformatssuchasinteger,roat,and theorientationusingEuleror Quaternionmethods. ThespecicationsofworkstationareshownintheTable4.1. Thedimensionsofthe mannequinmustbeasasthatofthesubjectusedforthevolume data. Table4.1.SpecicationsOfWorkstation Workstation CPU DualIntelP43.6GHzWithHyperthreadingsupport Memory 4GByte533Mhz GraphicsCard NVIDIAQuadroFX4400512MB HardDisk 73GBSeagateUltraSCSI15000RPMNon-RAID 4.5GraphicalUserInterface Thegraphicaluserinterface(GUI)hasbeendevelopedusing MicrosoftFoundation Class(MFC)libraryundertheVisualC++developmentenviro nment.Therearetwo windowsusedforinteraction.Themainwindow(Figure4.14( a))isusedtoshowthe outputsfromvariousmodesand,controlpaneldialogbox(Fi gure4.14(b)),isusedto selectoperationmodesandlabelingoptions,performcamer asettingssuchaszoomand rotate,perform3Dor2Dviewselectionsandcontainsslider controlstomovethethree orthogonalplanesduringthecentroidlocationmode.VCNSa lsoprovidesseveralmenu optionsfordatastorageandretrieval.The File menuprovidessub-optionsforloadingthe segmentedandnonsegmentedvolumedata,andthelookuptabl efornames.The Save SettingsFile optionunderthe File menuisusedtosavecurrentstateofthesoftware. Thiscreatesasettingsle,whichcontainsinformationabo utthelocationofthecurrent volumedatale,segmenteddatale,currentviewsettingsa ndcentroidlocation.The LoadSettingsFile optionhelpsinquicklyrestoringthepreviousstateofthes oftware 37

PAGE 49

acrossmultipleinvocations.The Help menuopensupanonlinemanualdescribingthe usageofthesystem. 38

PAGE 50

Figure4.14.UserInterfaceForVCNS 39

PAGE 51

CHAPTER5 SOFTWAREIMPLEMENTATION Thischapterdescribesthedetailsofthedesignandimpleme ntationofVCNS.The systemisdevelopedinC++andusesOpenGLapplicationprogr amminginterface(API)for handlingthegraphics.Ahighlevelinteractiondiagrambet weenvariousobjectscomprising thesystemisshownintheFigure5.1.Amoredetaileddiagram showingtherelationship betweenthevariousclassesforthewholesystemisshownint heappendixA.Thefollowing sectionsexplainindetaileachoftheclassobjectsusedint hesystem. 5.1ClassDesign VCNSisdesignedusingobjectorienteddesign(OOD)princip lesresultinginarexible, andaneasilymaintainablesystem.Thevariousclassesden edbythesystemareexplained indetailbyfollowingsections.5.1.1 PCIBirdInterface Class Thisclassisapartofthetrackerinterfacesubmoduleofthe trackingmoduleand handlesthetaskofinterfacingwiththepciBird TM motiontrackersystem.Itinitializesthe trackerwithparameterssuchasthemeasurementrate,metri candhemisphereofoperation duringinitialization.Themeasurementrateisxedat30mi llisecondstomatchwiththe displayrefreshrateof30framespersecond.Thedistanceme tricisxedasmillimeters, whilethehemisphereofoperationisxedasFRONT.Thehemis pheresarisebecauseof thesymmetryinthemagneticeldaroundthetransmitter.Wh enthesensoristracked continuouslyforitsposition,itisnecessarytokeeptrack ofthetransitionofthesensor acrossthehemisphereboundary.Thisisdonebymonitoringt hesignsofX,YandZ 40

PAGE 52

Figure5.1.InteractionDiagramForSoftwareComponentsOf VCNS 41

PAGE 53

Figure5.2.ConversionFromPCIBirdToOpenGLCoordinateSy stem coordinatesofthetrackeracrosssuccessivemeasurements .Whentheboundaryiscrossed, thesignsofbothYandZmeasurementsreverse,whilethatofX remainsthesame.The nalvaluesareobtainedbyreversingthesignsofX,YandZ.T hereisnochangeinthe anglesofthesensorwhenthetransitionoccurs. The PCIBirdInterface alsoperformsnecessaryadjustmentstosynchronizethepos ition ofthetrackerwithitsvirtualcounterpart.Figure5.2show stheconversionsnecessary tomatchthetrackercoordinateswiththeOpenGLcoordinate s.Notethattheanglesare measuredinquaternionnotationtoavoid\GimbalLock",whi chreferstoasituationwhere itbecomesimpossibletorotatetheobjectinadesiredaxis[ 5].Eulermethodgivesangle ofrotationabouttheX,YandZaxesseparately,andaresensi tivetotheorderinwhich theanglesareapplied.Ontheotherhand,quaternionmeasur estherotationangleabout anarbitraryaxisofrotationandisfreefromthegimballock problem.Thepositionsand theanglesarequeriedbythe Manager objectduringthetrackingandlabelingmode. 5.1.2 MemoryManager Class Thisclassanditssubclassesarethepartofresourcemanage mentmodule.Itisresponsibleformanagingtheresourcessuchas:texturememory,gr aphicsbusbandwidth,main memoryandtheharddisk.Inordertoperformthistask,thefu nctionalityofthe Meme42

PAGE 54

Figure5.3.HierarchialDistributionOfVolumeDataInVCNS oryManager classisdividedintothreeclasses: HardDiskManager MainMemoryManager and TextureMemoryManager asshowninFigure5.1. Whenthevolumedataisloadedforthersttime,the HardDiskManager dividesit intobricks.Thesizeofthebricksisdecidedonthefactorss uchassizeoftexturememory, graphicsbusspeedanddatabandwidthaswellasthepixelll rateofthegraphicscard. Largebricksizesrequirelessseeksbutarenotinterruptib le,therebyslowingthevisualization.Smallbricksizesrequiremoreseeksbutaremoreinter ruptible.Thebricksizeof64 X64X64givesarealtimerefreshrateof30fps. Oncethedataisdividedintobricks,itiswrittentoalecal led\brickle",whichis moreecientinaccessingthebrickdatathantheordinaryvo lumedatale.Inaddition, the HardDiskManager alsomaintainsamappingbetweenthebrickIDsandtheirose ts inthebrickle.BricksareassignedIDs/addressesbythe CollisionDetector component duringinitialization.Usingthebrickleanybrickcanbea ccessedrandomlyusingonlyone seekoperation.Thecontagiouslywrittenbricksarealsone ighborsinthe3Dspacethereby providingaspatialcoherencetocertainextent.The HardDiskManager alsoaugmentsthe RGBdataforthebrickwiththeopacityvaluesbeforewriting ,sothatitcanbedisplayed usingOpenGLtexturemappingroutines.Blackpixels(R=0,G =0,B=0)areassigned anopacityof0(fullytransparent)whiletheremainingpixe lsavalueequalto1(completely opaque). 43

PAGE 55

The MainMemoryManager componentallocatesthexedamountofmemoryforthe bricksfromtheprocessaddressspace,andfragmentsitinto slotsofsizesequaltothebrick size.Theseslotsaremaintainedinthe\leastrecentlyused "(LRU)list,andcancontain onlyonebrickatatime.Thus,thereisaone-to-onemappingb etweentheslotindicesand thebrickIDs.TheLRUlistisimplementedusingtheheapdata structure,withthetopof theheappointingtotheleastrecentlyusedslotwithrespec ttoalogicaltimerimplemented asacountervariable.Thelogicaltimerisassociatedwithe achslotindexandisupdated everytimeitisaccessed. The TextureMemoryManager componentqueriesthegraphicshardwaretondoutthe maximumavailabletexturememory,anddividesitintoslots similartothemainmemory slots.TheseslotsarealsoarrangedinaLRUlistandhaveaon e-to-onemappingtothe brickIDs.Atanyinstant,theLRUbrickinthetexturememory istheMostRecentlyUsed (MRU)brickinthemainmemory.Thus,thebricksinthetextur ememoryareasubset ofthesetofbrickspresentinthemainmemoryasshownintheF igure5.3.Notethat,all thebricksinthetexturememorymaynotbeuseddependingupo ntheviewingfrustumas shownintheFigure5.3.5.1.3 VolumeDataLoader Class TheVCNSoersrexibilityofloadingthevolumedatapresent indierentimageformats throughthe VolumeDataLoader class.Inordertosupportanewleformat,thecustom dataloaderisimplemented,byinheritingfromthe VolumeDataLoader andimplementing thepurevirtualfunctioncalled loadDataBrick .Theparameterstothe loadDataBrick methodacceptthelowerandupperlimitsofthesubvolumeoft hedatatobereadfrom thele.5.1.4 CollisionDetector Class Inadditiontodividingthevolumedataintosmallerdatabri cks,itisalsonecessary todividethe3Dgeometryassociatedwithit.Typically,the numberbricksgeneratedas aresultofthisdivisionislarge(1000-2000).Whileexplor ingthevolumedatausingthe 44

PAGE 56

virtualcuttingplaneonlyafewbricksareintersectedbyth eplaneatatime.Hence, thereisaneedtoorganizethebricksmoreeciently.The CollisionDetector component alleviatesthisproblembyarrangingthebricksintheformo faBinarySpacePartition (BSP)tree(Figure4.7).Eachinternalnodeinthetreerepre sentsaconvexregionin3D space,whichissubdividedintotwochildregionsbymeansof afreelyorientedpartitioning plane[13].Thepartitioningplaneisoneoftheorthogonalp lanes(XorYorZ)centered attheoriginoftheregion.Therootofthetreecontainsthec ompletevolumegeometry, whiletheleafnodescontaintheactualbricks.Eachbrickis alsoassignedauniqueID duringthetreeconstruction. Inordertodeterminethebricksintersectedbytheplane,th e CollisionDetector recursivelydetermineswhethertheplaneintersectsaninter nalnode.Thisprocess,called collision/interferencedetection,isperformedbytherou tinesintheSOLID(SOftwareLibraryforInterferenceDetection)librarydevelopedby[13 ].Iftheregionrepresentedbythe internalnodeisintersectedbythecuttingplane,itschild renareexploredfurther;otherwise theentiresubtreerootedatthatinternalnodeisignored.O ncethebricksintersectedby thecuttingplanearedetermined,itisclippedagainstthee verybrickusingtheSutherland Hodgman3Dpolygonclippingalgorithm[56].Thisresultsin asetofpointsrepresenting thepolygonalportionofthecuttingplaneinsidethatbrick .Thepointsofintersectionare passedbacktothe Manager componentduringcollisiondetectionprocess. 5.1.5 Renderer Class Thisclassencapsulatesthefunctionalityofthevisualiza tionmodule.Itexposesthe renderingfunctionalitythroughthreechildclasses: TMRenderer CLMRenderer and LabelRenderer .Everysubclassimplementsthethreefunctionscalled beginDraw draw and endDraw .The beginDraw functionisusedtomaketheviewsettingssuchascameraposition,rotation,scaling,enabling3Dtexturingetcthata respecictoarenderer.The 3Dtexturemappingofthetomographicplaneisimplementedi nthe draw method,which takestherenderingcontextasaninputparameter.Themetho d endDraw ,ismainlyused todisable3Dtexturingandundotheviewsettingsdonedurin gtherendering. 45

PAGE 57

The TMRenderer classisfurtherdividedintotwosubclassesnamed TM2DRenderer and TM3DRenderer ,todrawthescenein2Dand3Dviewrespectively.Incaseof2D renderer,therotationandpositionofthevirtualplaneare negated,andmultipliedwith the\modelview"matrix,thatisusedinOpenGLforspecifyin gthetransformations[54]. Sincethecameraisxedatorigin,thistransformationcaus estheplanetofacethecamera. Thisprocessiscalledas\billboarding"[1]. The CLMRenderer isusedinthecentroidlocatorphase,todisplayoneorallof the threeorthogonalplanesthroughthevolume.The LabelRenderer isadirectsubclassof TM2DRenderer andextendsitsfunctionalitybyprovidingthefunctionsto readbackthe texturemappedimageandrenderlabelstrings.5.1.6 LabelGenerator Class The LabelGenerator componentgeneratesthelabelsfromthesegmentedslicebyi dentifyingthegroupsofpixelsbelongingtothesameanatomica lpart.Toenablethiscomponent,itisnecessarytoloadthelookuptableformappingc olorvalueswiththenames. The LabelGenerator readsthemappingleandcreatesamapforstoringthenamean d colorassociations.Itreturnsasetoflabelsforthevariou stissuesinthesliceduringthe labelingmode.Theselabelsaresuperimposedonthenonsegm enteddataslicebythe LabelRenderer object. 5.1.7 Manager Class Thiscomponentisresponsibleforcoordinatingalltheacti vityinthesystemandencapsulatesthefunctionalityofthetracking,centroidlocato randthelabelingmodules.When thenonsegmentedandthesegmenteddataareloadedthemanag ercreatestheinstances of MemoryManager CollisionDetector andthe Renderers class.Duringtherendering,the managercreatesa\renderingcontext"containingthebrick scurrentlyintersectedbythe tomographicplane,thepointsofintersectionoftheplanew itheverybrickandtheOpenGL textureobjects,thatpointtotheareaoftexturememoryrep resentingthetexturedata forthebrick.Thiscontextispassedtoasuitablerendererd ependinguponthecurrent 46

PAGE 58

modeofoperation.Theroleof Manager componentisdiscussedindetail,inthecontext ofsoftwareimplementation.5.2SoftwareImplementation Thissectionpresentsthelowleveldetailsofimplementati onforeachofthethree operatingmodes.5.2.1TrackingMode Inthismode,thepciBirdisinitializedandprobedperiodic allytoobtaintheposition andanglesofthesensor.Thefollowingstepsareperformedi nordertomapvirtualplane emanatingfromtheprobe,withthetexturedatafromthebric ks. Step1.The Manager readspciBird'spositionandorientation,andcomputesthe plane geometrybasedonthisdata. Step2.The Manager queriesthe CollisionDetector ,tondoutthebricksintersecting withtheplane,andstoresthemincollisionvector.The CollisionDetector obtainsthis informationbyperformingacollisiondetectionquerywith everyinternalnodeoftheBSP treeuntilitreachestheleafnodecontainingthedesiredbr ick(Figure4.8).Theintersecting bricksarereturnedalongwiththepointsofintersectionas apartof CollisionData data structure. Step3.The Manager createsbatchesofbricks,eachofsizeequaltomaximumnumb er ofbricksthatcanbeaccommodatedinthetexturememory.Eve rybatchispassedtothe MemoryManager forbindingthetexturedata.The MemoryManager rstarrangesthe brickssuchthatthebrickspresentinthetexturememoryare atthefrontofthebatch, followedbythosepresentinthemainmemory,andnallytheb ricksontheharddisk. Afterprioritizingtheprocessingorderofbricks,texture bindingisperformedforthose notinthetexturememory.Thebrickislocatedinthemainmem oryand/orthehard disk,andthetexturedataisloadedintothenextavailablet exturememoryslotbythe TextureMemoryManager 47

PAGE 59

Figure5.4.SequenceDiagramForTrackingModeBasedOnUMLP rinciples.AnUnderline BelowAClassNameImpliesThatTheInstanceOfThatClassIsU sed 48

PAGE 60

Step4.The Manager passestherenderingcontexttooneofthe2Dor3Dtracking moderenderersdependinguponthecurrentviewsettings.Th erenderernormalizesthe pointsofintersectionsbetween0and1aspertherequiremen tsofOpenGL.Thisisdone bysubtractingpositionofthecenterofthebrickandscalin gitdownbythedimensionsof thebrick.The3Dtexturemappingisperformedbyspecifying everypointofintersection oftheplaneandbrick,anditscorrespondingtexturecoordi nate. Figure5.4showsthemessagespassedbetweenvariousobject sduringtheprocessof tracking.5.2.2CentroidLocationMode Thecentroidlocationmodeisusedtoaligntheoriginofvolu medatawiththatofthe mannequin.Whenthevolumedataisloaded,theoriginofthev olumeisatthecenter oftheboxenclosingthevolume,whichmaynotbethelocation ofcenterofthepciBird transmitterandmannequin.Thecentroidlocationmodecomp utestheslicesalongthe transaxial(XY),sagittal(YZ)andcoronal(XZ)planes.The seplanescanbemovedusing slidercontrolsinthecontrolpaneldialogbox(Figure4.14 (b)),tomatchthetwocoordinate systems.Thepointwherethethreeplanesmeet,isconsidere dastheoriginofthevolume dataduringtrackingmode.Correctionsaremadetotheposit ionobtainedfromthepciBird basedonthelocationofcentroidofvolumedata.Themessage spassedbetweenvarious objectsisexactlysameasthetrackingmodeshownintheFigu re5.4,exceptthatthe positionsoftheplanesarequeriedfromthecontroldialogb oxinsteadofthetracker. 5.2.3LabelingMode Inthismode,itisnecessarytoloadthesegmentedvolumedat aandthemapping betweenthesegmentedvaluesandthenames.Thesegmentedda taisalsodividedinto brickssimilartothenonsegmenteddataandwrittentothedi skinaseparatebrickle. Thenonsegmenteddatabricks,however,sharethegeometryw iththesegmenteddata bricks.Thus,itisnecessarytoloadthenonsegmenteddatab eforeloadingthesegmented 49

PAGE 61

Figure5.5.SequenceDiagramForLabelingModeBasedOnUMLP rinciples.AnUnderline BelowAClassNameImpliesThatTheInstanceOfThatClassIsU sed 50

PAGE 62

data.Thesoftwareensuresthesequencebydisablingthe LoadSegmentedVolume menu option,untilthenonsegmenteddataisloadedusingthe LoadVolume menuoption. Inordertogeneratethelabels,itisnecessarytofreezethe motionoftheprobeby clickinginsidethewindowwiththeleftmousebutton.Thisp utsthemanageronthehold. Asaresult,themanagercapturesthesnapshotofthe2-Dslic ethroughthesegmented datasetatthefrozenpositionandorientationoftheprobe. Thecoloredsliceisobtained fromthenonsegmenteddatasetinthesamewayasthetracking mode. Dependinguponthecurrentlabelingoption,eitherallthea natomicalpartsonthe slicearelabelled,oronlythoseselectedbytheuserusingt herightmousebutton.Inthe caseofautomaticlabeling,alltheanatomicalpartsonthec urrentsliceareidentiedfrom thesegmentedsliceandlabelledbyalookupoperationinthe tablecontainingmapping betweenthenameandRGBvalues.Thismappingiscreatedfrom thelookuptablele loadedfromthe LoadLUT menuoption.Incaseof\On-Click"labelingoption,thelabe l isgeneratedonlyfortheselectedstructure.Thesequenced iagram(Figure5.5)showsthe messagesexchangedbetweenvariousobjectsduringthelabe lingprocess. 51

PAGE 63

CHAPTER6 RESULTSANDDISCUSSION Thischapterpresentstheoutputsandtheperformanceanaly sisforvariousmodesof operationofVCNS.TheadvantagesofusingVCNSasateaching tool,andthepossible amendmentstoitarediscussedintheend.6.1Results Thetomographicslicesresultingfromtheinteractionofth eprobewiththevolume dataforthevisiblemale,inthetrackingmode,areshownint heFigure6.1,Figure6.2. TheimageinFigure6.1showsthecrosssectionalviewofthel ungs,andFigure6.2shows thecrosssectionalviewofthevertebralcolumn.Thesegur esdepicthowtheprobecan beusedtoobtainthecrosssectionalslicesatvariousangle sandpositions.Figure6.3 showshowthelabelgeneratormodecanbeusedtoidentifythe lowerlobeoftheleftlung. Figures6.5throughFigure6.7showthetransaxial,sagitta landcoronalslicesthroughthe volume,whichareusedtoalignthecentroidofthevolumedat aduringthecentroidlocator process.Thepointofintersectionofthethreeorthogonalp lanesisusedtodeterminethe positionofthecentroidasshownintheFigure6.4.Thisposi tioniswherethetransmitter ofthetrackersystemisplaced.6.2PerformanceAnalysis ThefactorsaectingtheperformanceofVCNSareasfollows: 1.Sizeofthebrick2.Sizeofthetexturememory 52

PAGE 64

Figure6.1.ATomographicSliceOfLungInTrackingModeThro ughVisibleMaleData Figure6.2.ATomographicSliceOfVertebralColumnInTrack ingModeThroughVisible MaleData 53

PAGE 65

Figure6.3.IdenticationOfLowerLeftLobeOfLungUsingTh eLabelingTool Figure6.4.DiagramShowingTheIntersectionOfTheThreeOr thogonalPlanesThrough TheVisibleMaleDuringCentroidLocatorMode 54

PAGE 66

Figure6.5.TransaxialSliceThroughTheVisibleMaleDataG eneratedDuringCentroid LocatorMode Figure6.6.SagittalSliceThroughTheVisibleMaleDataGen eratedDuringCentroid LocatorMode Figure6.7.CoronalSliceThroughTheVisibleMaleDataGene ratedDuringCentroid LocatorMode 55

PAGE 67

3.Sizeofthemainmemory4.Bandwidthofthegraphicsbus Thecriteriaforperformanceisconsideredasframerate,wh ichismeasuredasthe numberofframesthatcanberenderedpersecond.Anaveraget imeforrenderingaframe iscalculatedbyaveragingthedierencebetweenthesystem timebeforeandafterrendering aframeover1000frames.Theframerateisthencalculatedas thereciprocaloftheaverage timeforrenderingaframe.Theframerateismeasuredforeac hoftheoperatingmodes separatelybyvaryingthevariousparametersmentionedabo ve.Theperformanceanalysis isperformedfortwotestmachineswithdierentcongurati onsasshownintheTable6.1. Table6.1.CongurationsOfTwoTestMachinesUsedForPerfo rmanceAnalysis Conguration Machine1 Machine2 Processor IntelPentium1.5GHz IntelPentiumDualProcessor3.6GHz MainMemory 1GB 4GB GraphicsCard ATIRadeon8700,64MB,AGP2X NVIDIAQuadroFX4400, 512MB,PCIExpress16X Storage Seagate,SCSI,7200RPM,160GB SeagateUltraSCSI,15000RPM,73GB Figures6.10,6.11showtheimpactofbricksizeonframerate forboththetestboards. Figure6.11indicatesthatforRadeon8700anincreaseinthe sizeofthebricktill64X 64X64increasestheframerate.However,beyondthissizeth eframeratedecreasesto Figure6.8.VariationOfFrameRateWithTextureMemorySize OnNVIDIAQuadroFX 4400(MainMemory=512MB,BrickSize=64X64X64) 56

PAGE 68

Figure6.9.VariationOfFrameRateWithTextureMemorySize OnATIRadeon8700 (MainMemory=512MB,BrickSize=64X64X64) Figure6.10.VariationOfFrameRateWithBrickSizeOnNVIDI AQuadroFX4400(Main Memory=512MB,TextureMemory=64MB) Figure6.11.VariationOfFrameRateWithBrickSizeOnATIRa deon8700(MainMemory =512MB,TextureMemory=64MB) 57

PAGE 69

Figure6.12.VariationofFrameRateWithMainMemorySizeOn NVIDIAQuadroFX 4400(TextureMemory=64MB,BrickSize=64X64X64) Figure6.13.VariationOfFrameRateWithMainMemorySizeOn ATIRadeon8700 (TextureMemory=64MB,BrickSize=64X64X64) 58

PAGE 70

zero,whichisduetothelimitationsoeredbythegraphicsb usbandwidth.Incaseof theNVIDIAQuadro,thislimitishigherbecauseofthehigher bandwidthoeredbythe PCIExpress16XgraphicsbusasshownintheFigure6.10.Thee ectofsizeoftexture memoryontheframerateisshownintheFigures6.8,6.9.Thei ncreaseinthetexture memoryincreasestheframeratesubstantially(almostby10 framespersecond)incase ofboththegraphicscards.Figures6.12,6.13showstheimpa ctofmainmemorysizeon theframerate.Anincreaseinthesizeofthemainmemoryfrom 512MBto1GB,only, minimallyincreasestheframerateinboththecases. Alloftheaboveguresindicatethattheframerateisloweri ncaseofthecentroid locatorthanthetrackingandlabelingmodeinmostcases.In theFigure6.10,however,for bricksizeof256X256X256theframerateishigherincaseoft hecentroidlocatormode thantheothermodes,sincethebricksareaccessedlessrand omly,therebydecreasingthe frequencyofthedownloadsforthebricks.Theframerateinl abelingmodeisalwaysin parwiththetrackingmode,althoughthesegmentationdatas etisalsousedalongwiththe nonsegmenteddata. TheperformanceanalysisshowsthatVCNSisabletoachievei nteractiveframerates (30fpsormore)onboththetestmachines,withatleast64MBo ftexturememoryand 512MBofmainmemory.6.3Discussion TheproposedVCNSsystemisacosteectivesystemcapableof exploringfullresolution visiblemaleandvisiblefemalecadavericdataonapersonal computer(PC).Theabilityto pointatthemannequinandobtaintomographicsliceatarbit rarylocationsandorientation helpsinunderstandingspatiallocationsofvariousorgans ,whichisessentialforlearning thehumananatomy.Theintegrationofthesegmentationdata helpsinobtaininglabels, makingthelearningprocessaccessibleevenforanovice.Th esystemcanbeeasilyextended tosupportdierentleformatsandcanbeusedtoexplorecli nicaldataofpatientsfor diagnosticpurposessuchasCTandMRI.Thesoftwarecanalso beusedtoteachultrasound 59

PAGE 71

imagingtechniqueforstudentsandtechnicians.Thesystem isrexible,portableandcan beinstalledeasilyonanyworkstationwithatleast64Megab ytesoftexturememoryand 512Megabytesofmainmemory. Thesystemcanbeextendedtoprovidemoreinformationtoimp rovetheunderstanding. Forexample,supplementaryinformationsuchashistologyi mages,briefdescriptionsof functionsoforgansontheslicecanbelinkedwiththeslice. 3Dmodelsoforgansand bonescanalsobeintegratedtoprovideamoreintuitiveinte rface.Thesystemcanbe easilyextendedtoincludeanevaluationtoolthatcangrade thestudent,basedhis/her abilitytolocate,andlabeltheorganscorrectlybypointin gatthemannequin. 60

PAGE 72

REFERENCES [1]http://www.3dlabs.com/tutorials/billboarding.htm [2]http://www.ascension.com.[3]http://www.libti.org/.[4]http://www.m-w.com/.[5]http://www.mathworld.com.[6]http://www.nlm.nih.gov.[7]M.J.Ackerman.Thevisiblehumanproject. JournalofBiocommunication ,18(2):14, 1991. [8]AdobeSystemsIncorporated. TIFFRevision6.0 ,June1992. [9]M.Alcaniz,C.Perpina,R.Banos,J.A.Lozano,J.Montesa ,ande.a.Botella,C. Anewrealistic3dbodyrepresentationinvirtualenvironme ntsforthetreatmentof disturbedbodyimageineatingdisorders. CyberPsychologyandBehavior ,3(3):421{ 432,2000. [10]M.R.Ali,Y.Mowery,B.Kaplan,andE.J.DeMarial.Train ingthenoviceinlaparoscopy. SurgicalEndoscopy ,16(12):1732{1736,December2002. [11]AscensionTechnology. PCIBirdTechnicalReference [12]R.Bargar,I.Choi,S.Das,andG.Camille.Model-basedi nteractivesoundforan immersivevirtualenvironment. Proceedingsofthe94InternationalComputerMusic Conference ,1994. [13]G.V.D.Bergen. CollisionDetectioninInteractive3DEnvironments .MorganKaufmann,October2003. [14]C.Botella,C.Perpina,R.M.Banos,andA.Garcia-Palac ios.Virtualreality:anew clinicalsettinglab. StudiesinHealthTechnologyandInformatics ,58:73{81,1998. [15]C.Chinnock.Virtualrealityinsurgeryandmedicine. HospitalTechnologySeries 13(18):1{48,1994. [16]S.L.DelpandF.R.Zajac.Forceandmomentgeneratingca pacityoflowerlimbmusclesbeforeandaftertendonlengthening. ClinicalOrthopaedicandRelatedResearch 284:247{259,1992. 61

PAGE 73

[17]A.M.DiGioia,B.Jaramaz,R.V.O'Toole,D.A.Simon,and T.Kanade.Medical roboticsandcomputerassistedsurgeryinorthopaedics:an integratedapproach. In: InteractiveTechnologyandtheNewParadigmforHealthcare .IOSPress.Washington D.C. ,pages88{90,1995. [18]H.D.Dobson,R.K.Pearl,C.P.Orsay,M.Rasmussen,R.Ev enhouse,andZ.Ali. Virtualreality:anewmethodofteachinganorectalandpelv icrooranatomy. Dis ColonRectum ,46(3):349{352,2003. [19]B.J.Dunkin.Flexibleendoscopysimulators. SeminLaparoscSurg ,10(1):29{35,2003. [20]P.M.Emmelkamp,M.Bruynzeel,L.Drost,andC.A.Vander Mast.Virtualreality treatmentinacrophobia:acomparisonwithexploreinvivo. CyberPsychologyand Behavior ,4(3):335{339,2001. [21]A.Garcia-Palacios,H.Homan,A.Carlin,T.A.Furness ,andC.Botella.Virtualrealityintreatmentofspiderphobia:acontrolledstudy. BhaviorresearchandTherapy 40(9):983{993,2002. [22]A.V.GelderandK.Kim.Directvolumerenderingwithsha dingviathree-dimensional textures.In VVS'96:Proceedingsofthe1996symposiumonVolumevisuali zation pages23{.,Piscataway,NJ,USA,1996.IEEEPress. [23]J.C.Goble,K.Hinckley,R.Pausch,J.W.Snell,andN.F. Kassell.Two-handed spatialinterfacetoolsforneurosurgicalplanning. IEEEComputer ,28(7):20{26,July 1995. [24]M.Gor,R.McCloy,R.Stone,andA.Smith.Virtualrealit ylaparoscopicsimulator forassessmentingynaecology. BJOG:AnInternationalJournalOfObstetricsAnd Gynaecology, ,110(2):181{187,February2003. [25]D.Gross. TechnologymanagementanduseracceptanceofVEtechnology ,chapterHandbookofVirtualEnvironments:Design,Implementat ion,andApplications. LawrenceErlbaumAssociates,Inc.,Mahwah,N.J.,2002. [26]S.HalliganandH.M.Fenlon.Virtualcolonoscopy. BritishMedicalJournal 319(7219):1249{1252,1999. [27]K.H.Hohne. VOXEL-MAN,Part1:BrainandSkullVersion1.1. Springer-Verlag ElectronicMedia,1996. [28]P.R.Jeries,S.Woolf,andB.Linde.Technologybasedv straditionalinstruction.a comparisonoftwomethodsforteachingtheskillofperformi nga12-leadecg. Nursing EducationPerpective ,24(2):70{74,2003. [29]N.W.John,N.Thacker,M.Pokric,A.Jackson,G.Zannett i,andE.e.a.Gobbetti. Anintegratedsimulatorforsurgeryofthepetrousbone. StudHealthTechnolInform 81:218{224,2001. [30]Krueger.Olfactorystimuliinvirtualrealitymedical training.Report,ARPA,1994. 62

PAGE 74

[31]P.LacrouteandM.Levoy.Fastvolumerenderingusingas hear-warpfactorizationof theviewingtransformation.In SIGGRAPH'94:Proceedingsofthe21stannualconferenceonComputergraphicsandinteractivetechniques ,pages451{458,NewYork, NY,USA,1994.ACMPress. [32]E.LaMar,M.A.Duchaineau,B.Hamann,andK.I.Joy.Mult iresolutiontechniques forinteractivetexture-basedrenderingofarbitrarilyor ientedcuttingplanes. ProceedingsofVisSym ,July1998. [33]C.H.LewisandM.J.Grin. Humanfactorsconsiderationinclinicalapplications ofvirtualreality ,chapterVirtualrealityinneuro-psycho-physiology.,pa ges35{56. Amsterdam:IOSPress,1997. [34]W.E.Lorensen.Marchingcubes:Ahighresolution3dsur facereconstructionalgorithm. SIGGRAPH ,21(4):163{169,1987. [35]W.E.Lorensen,F.A.Jolesz,andR.Kikinis.Theexplora tionofcross-sectional datawithavirtualendoscope. In:InteractiveTechnologyandtheNewParadigmfor Healthcare ,pages221{230,1995. [36]N.Maltby,I.Kirsch,M.Mayers,andG.Allen.Virtualre alityexposuretherapyfor thetreatmentoffearofrying:Acontrolledinvestigation. JournalofConsultingand ClinicalPsychology ,70(5):1112{1118,2002. [37]J.Moline.Virtualrealityforhealthcare:asurvey. VirtualRealityinNeuro-PsychoPhysiology.Amsterdam:IOSPress; ,pages3{37,1998. [38]K.Moorthy,S.Smith,T.Brown,S.Bann,andA.Darzi.Eva luationofvirtualreality bronchoscopyasalearningandassessmenttool. Respiration ,70(2):195{199,2003. [39]H.Nicolas,S.adPatel.Healthandsafetyimplications ofvirtualreality:areviewof empiricalevidence. AppliedErgonomics ,33(3):251{271,2002. [40]J.C.Norcross,M.Hedges,andJ.O.Prochaska.Thefaceo f2010:Adelphipollonthe futureofpsychotherapy. ProfessionalPsychology:ResearchandPractice ,33(3):316{ 322,2002. [41]L.Piron,F.Ceni,P.Tonin,andM.Dam.Virtualrealitya sanassessmenttoolfor armmotordecitsafterbrainlesions. StudiesinHealthTechnologyandInformatics 81:386{392,2001. [42]G.Riva.Virtualenvironmentsinneuroscience. IEEETransactionsonInformation TechnologyinBiomedicine ,2(4):275{281,19989. [43]G.Riva.Applicationsofvirtualenvironmentsinmedic ine. MethodsOfInformation InMedicine ,42(5):524{534,2003. [44]G.Riva,M.Bacchetta,M.Baru,andE.Molinari.Virtua l-reality-basedmultidimensionaltherapyforthetreatmentofbodyimagedisturbances inbingeeatingdisorders: apreliminarycontrolledstudy. IEEETransactionsonInformationTechnologyin Biomedicine ,6(3):224{234,2002. 63

PAGE 75

[45]A.A.RizzoandJ.G.Buckwalter. Virtualrealityandcognitiveassessment andrehabilitation:thestateoftheart. ,chapterVirtualrealityinneuropsycho-physiology.,pages123{146.Amsterdam:IOSPress, 1997.Online: http://www.cybertherapy.info/pages/book1.htm. [46]M.D.Ross,I.A.Twombly,C.Bruyns,R.Cheng,andS.Seng er. [47]B.O.Rothbaum,L.Hodges,S.Simth,J.H.Lee,andL.Pric e.Acontrolledstudy ofvirtualrealityexposuretherapyforthefearofrying. JournalofConsultingand ClinicalPsychology ,68(6):1020{1026,2000. [48]F.Rubino,L.Soler,J.Marescaux,andH.Maisonneuve.A dvancesinvirtualreality arewideranging. BritishMedicalJournal ,324(7337):612,2002. [49]R.SatavaandS.Jones.Currentandfutureapplications ofvirtualrealityformedcine. ProceedingsoftheIEEE ,86(3):484{489,march1998. [50]R.M.Satava.Virtualrealitysurgicalsimulator. SurgicalEndoscopy ,7:203{205,1993. [51]R.M.SatavaandS.B.Jones.Medicalapplicationsofvir tualreality.InK.M.Stanney, editor, Design,Implementation,andApplication ,pages368{391.LawrenceErlbaum AssociatesInc,2002. [52]M.T.SchultheisandA.A.Rizzo.Theapplicationofvirt ualrealitytechnologyin rehabilitation. RehabilitationPsychology. ,46(3):296{311,2001. [53]W.R.ShermanandA.B.Craig. UnderstandingVirtualReality:Interface,Application,AndDesign .MorganKaufmann,2003. [54]D.Shreiner,M.Woo,N.Jackie,andT.Davis. OpenGLProgrammingGuide:The OcialGuidetoLearningOpenGL,Version1.4 .Addison-WesleyProfessional,4th edition,2003. [55]V.Spitzer,M.J.Ackerman,A.L.Scherzinger,andD.Whi tlock.Thevisiblehumanmale:atechnicalreport. JournalofAmericanMedicalInformaticsAssociation 3(2):118{130,1996. [56]I.E.SutherlandandG.W.Hodgman.Reentrantpolygoncl ipping. Commun.ACM 17(1):32{42,1974. [57]M.Teistler,O.J.Bott,J.Dormeier,andD.P.Pretschne r.Virtualtomography:Anew approachtoecienthuman-computerinteractionformedica limaging. Proceedingsof SPIE ,5029:512{519,2003. [58]M.TeistlerandD.P.Pretschner.Visualizationofecho cardiographicdatausingvirtual scenes. ComputersInCardiology ,26:399{402,1999. [59]J.Vince. VirtualRealitySystems .SIGGRAPHSeries.Addison-Wesley,1stedition, 1995. 64

PAGE 76

[60]F.Vincelli,L.Anolli,S.Bouchard,B.K.Wiederhold,V .Zurloni,andG.Riva.Experimentalcognitivetherapyinthetreatmentofpanicdiso rderswithagrophobia:A controlledstudy. CyberPsychologyandBehavior ,6(3):312{318,2003. [61]F.Vincelli,E.Molinari,andG.Riva.Virtualrealitya sclinicaltool:immersionand three-dimensionalityintherelationshipbetweenpatient andtherapist. Studiesin HealthTechnologyandInformatics ,81:551{553,2001. [62]W.R.Volz.Gigabytevolumeviewingusingsplitsoftwar e/hardwareinterpolation.In VVS'00:Proceedingsofthe2000IEEEsymposiumonVolumevis ualization ,pages 15{22,NewYork,NY,USA,2000.ACMPress. [63]B.K.Wiederhold,D.P.Jang,S.I.Kim,andM.D.Wiederho ld.Physiologicalmonitoringasanobjectivetoolinvirtualrealitytherapy. CyberPsychologyandBehavior 5(1):77{82,2002. [64]R.Wootton. EuropeanTelemedicine ,chapterTelemedicine:anintroduction,pages 10{12.KensingtonPublicationsLtd.,London,1998/99. [65]L.Zhang,B.C.Abreu,B.Masel,R.S.Scheibel,C.H.Chri stiansen,andH.N.Virtual realityinassessmentofselectedcognitivefunctionafter braininjury. AmericanJournal ofPhysicalMedicineandRehabilitation ,80(8):597{604,2001. 65

PAGE 77

APPENDICES 66

PAGE 78

AppendixAClassDiagramforVCNS Theclassdiagramforthesystembasedonuniedmodelinglan guagedesignprinciples isshownintheFigureA 67

PAGE 79

AppendixA(Continued) FigureA.1.VCNSClassDiagram 68