USF Libraries
USF Digital Collections

Shape based stereovision assistance in rehabilitation robotics

MISSING IMAGE

Material Information

Title:
Shape based stereovision assistance in rehabilitation robotics
Physical Description:
Book
Language:
English
Creator:
Jurczyk, Michael Ulrich
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Intrinsic camera parameter
Extrinsic camera parameter
Stereopsis
Telerobotics
Scaling functions
Dissertations, Academic -- Mechanical Engineering -- Masters -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: A graphical user interface program was created along with shape models, which allow persons with disabilities to set up a stereovision system with off-the-shelf hardware and detect objects of interest, which can be picked up using a sensor assisted telerobotic manipulator. A Hitachi KP-D50 CCD camera and an Imaging Source CCD camera were used along with two Imaging Source DFG/LC1 frame grabbers to set up a stereovision system. In order to use the stereovision system, the two main problems of correspondence and reconstruction are solved using subroutines of the program created for this work.The user interface allows the user to easily perform the intrinsic and extrinsic camera calibration required for stereovision, by following a few basic steps incorporated into the user interface program, which are described in this thesis. A calibration table required for these tasks can also be easily created using the program.In order to detect the object of interest, shape models, created by the user interface program, are used to solve the correspondence problem of stereovision. The correspondence problem is that of locating corresponding points in the left eye and the right eye, which are necessary to perform the calculations to obtain the location of the object of interest with respect to the end-effector. The shape models created for some commonly available items such as a doorknob and a door handle are included in the program and used to test the stereovision system. As expected, the error of detection decreases as the stereo system is moved closer to the object of interest in the x-, y- and z-position.
Thesis:
Thesis (M.S.M.E.)--University of South Florida, 2005.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Michael Ulrich Jurczyk.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 106 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001681041
oclc - 62711929
usfldc doi - E14-SFE0001084
usfldc handle - e14.1084
System ID:
SFS0025405:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001681041
003 fts
005 20060215071144.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 051220s2005 flu sbm s000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001084
035
(OCoLC)62711929
SFE0001084
040
FHM
c FHM
049
FHMM
090
TJ145 (Online)
1 100
Jurczyk, Michael Ulrich.
0 245
Shape based stereovision assistance in rehabilitation robotics
h [electronic resource] /
by Michael Ulrich Jurczyk.
260
[Tampa, Fla.] :
b University of South Florida,
2005.
502
Thesis (M.S.M.E.)--University of South Florida, 2005.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 106 pages.
520
ABSTRACT: A graphical user interface program was created along with shape models, which allow persons with disabilities to set up a stereovision system with off-the-shelf hardware and detect objects of interest, which can be picked up using a sensor assisted telerobotic manipulator. A Hitachi KP-D50 CCD camera and an Imaging Source CCD camera were used along with two Imaging Source DFG/LC1 frame grabbers to set up a stereovision system. In order to use the stereovision system, the two main problems of correspondence and reconstruction are solved using subroutines of the program created for this work.The user interface allows the user to easily perform the intrinsic and extrinsic camera calibration required for stereovision, by following a few basic steps incorporated into the user interface program, which are described in this thesis. A calibration table required for these tasks can also be easily created using the program.In order to detect the object of interest, shape models, created by the user interface program, are used to solve the correspondence problem of stereovision. The correspondence problem is that of locating corresponding points in the left eye and the right eye, which are necessary to perform the calculations to obtain the location of the object of interest with respect to the end-effector. The shape models created for some commonly available items such as a doorknob and a door handle are included in the program and used to test the stereovision system. As expected, the error of detection decreases as the stereo system is moved closer to the object of interest in the x-, y- and z-position.
590
Adviser: Dr. Rajiv Dubey.
653
Intrinsic camera parameter.
Extrinsic camera parameter.
Stereopsis.
Telerobotics.
Scaling functions.
690
Dissertations, Academic
z USF
x Mechanical Engineering
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.1084



PAGE 1

Sh a p e B a se d St e re o vi si o n Assi st a n ce i n R e h a b i l i t a t i o n R o b o t i cs b y Mi ch a e l U l ri c h Ju rczyk A t h e si s su b mi t t e d i n p a rt i a l f u l f i l l m e n t o f t h e re q u i r e me n t s f o r t h e d e g re e o f Ma st e r o f Sci e n ce i n Me ch a n i ca l En g i n e e ri n g D e p a rt me n t o f Me ch a n i ca l En g i n e e r i n g C o l l e g e o f En g i n e e ri n g U n i ve rs i t y o f So u t h F l o ri d a Ma j o r Pro f e sso r: R a j i v D u b e y, Ph D Sh u h Ji n g Yi n g Ph D Ash o k Ku ma r, Ph D D a t e o f Ap p ro v a l : Ma rch 2 3 2 0 0 5 Ke yw o rd s: i n t ri n si c ca m e ra p a r a me t e r e xt ri n si c ca me ra p a ra me t e r, st e re o p si s, t e l e ro b o t i cs, sca l i n g f u n ct i o n s C o p yri g h t 2 0 0 5 Mi ch a e l U l r i ch Ju rc zyk

PAGE 2

D e d i c a ti o n T h i s t h e si s i s d e d i ca t e d t o my p a re n t s, D i e t e r a n d U rs u l a Ju rczyk. D i e se D i p l o ma r b e i t i st me i n e n El t e rn D i e t e r u n d U rsu l a Ju rczyk, g e w i d me t

PAGE 3

i i A c k n o w l e d g e m e n ts I w o u l d l i ke t o t h a n k D r. R a j i v D u b e y f o r g i vi n g me t h e o p p o rt u n i t y t o w o rk o n t h i s p ro j e ct f o r my MSME t h e s i s. T h a n k yo u f o r yo u r s u p p o rt p a t i e n c e a n d t h i s w o n d e rf u l o p p o rt u n i t y. I t w a s a p l e a su re t o w o rk o n t h i s t o p i c, w h i ch r e a l l y i n t e re st e d me I w o u l d a l so l i k e t o t h a n k D r. Sh u h Ji n g Yi n g f o r h i s su p p o r t a n d co n st a n t co rre ct i o n s a n d g u i d a n c e t h r o u g h o u t t h e p r o ce ss. I w o u l d l i ke t o t h a n k D r. Ash o k Ku ma r f o r t a ki n g t h e t i me t o re v i e w a n d g i ve f e e d b a ck a b o u t my w o rk. I a l so w a n t t o t h a n k S u e Bri t t e n f o r h e r co n st a n t su p p o rt a n d e n co u r a g e me n t ; a l so f o r a l l t h e h e l p s h e g a ve me i n so ma n y w a ys t h a t a re i mp o ss i b l e t o o n u me ro u s t o d e scri b e I a l so h a v e t o t h a n k D r. R o n a l d Ma n n f o r h i s su p p o rt e sp e ci a l l y i n t h e b e g i n n i n g p h a se s o f t h e t h e s i s. T h a n k yo u a l so t o a l l t h e p e o p l e i n t h e me ch a n i c a l e n g i n e e ri n g d e p a r t me n t w h i ch I ca n n o t l i st I a l so h a ve t o t h a n k m y co l l e a g u e s i n D r. D u b e y s g r o u p y o u kn o w w h o yo u a re I w o u l d a l so l i ke t o t h a n k H a mz a Be g d o u r i MSME, f o r h i s su p p o rt a n d h i s f ri e n d sh i p A l so I w o u l d l i ke t o t h a n k C h ri st i n e W o o d a r d f o r b e i n g s o u n d e rst a n d i n g a n d p a t i e n t F i n a l l y, I r e a l l y w a n t t o t h a n k my p a r e n t s a n d f a mi l y f o r t h e i r e n d l e ss su p p o r t a n d p a t i e n c e T h a n k yo u t o e ve ry o n e

PAGE 4

i T a b l e o f C o n te n ts L i st o f T a b l e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i v L i st o f F i g u re s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Ab st ra ct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi i C h a p t e r 1 I n t ro d u ct i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 Mo t i va t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 Vi si o n i n R o b o t i cs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 3 Pa st W o rk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 4 T h e si s O b j e ct i v e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 C h a p t e r 2 St e re o vi s i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 1 H i st o ry o f St e re o vi s i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 2 H a rd w a re . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 3 So f t w a re . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 4 C a me r a Mo d e l a n d T h e o ry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 2 5 I n t ri n si c C a m e ra Pa ra m e t e rs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 2 6 Ext ri n si c C a m e ra Pa ra m e t e rs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 8 C h a p t e r 3 Sh a p e Mo d e l s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 3 1 I n t ro d u ct i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 3 2 Ba ckg ro u n d o f Vi si o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2

PAGE 5

i i 3 3 C re a t i o n o f a Sh a p e Mo d e l . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 5 3 3 1 T h e O ri g i n a l I ma g e o f t h e O b j e ct o f I n t e re st . . . . . . . . . . . . . . . . 2 5 3 3 2 R e g i o n o f I n t e re st . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 7 3 3 3 C o n t ra st T h re sh o l d a n d Pyra m i d s . . . . . . . . . . . . . . . . . . . . . . . 2 8 3 3 4 O t h e r Pa ra me t e rs R e q u i r e d f o r t h e Sh a p e Mo d e l . . . . . . . . . . . 3 0 3 4 D e t e ct i o n o f Sh a p e Mo d e l s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 C h a p t e r 4 T h re e D i me n si o n a l R e co n st ru ct i o n . . . . . . . . . . . . . . . . . . . . . . . . 3 2 4 1 O ve rvi e w . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 4 2 St e re o G e o me t ry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 4 3 St e re o R e c o n st ru ct i o n C a l c u l a t i o n s . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 C h a p t e r 5 U s e r I n t e rf a ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 7 5 1 O ve rvi e w . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 7 5 2 Ma i n U s e r I n t e rf a ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 7 5 3 C a l i b ra t i o n T a b l e C re a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 9 5 4 I n t ri n si c C a m e ra Pa ra m e t e r C a l i b r a t i o n Su b ro u t i n e . . . . . . . . . . . . . 4 0 5 5 Ext ri n si c C a m e ra Pa ra m e t e r C a l i b ra t i o n Su b ro u t i n e . . . . . . . . . . . . . 4 3 5 6 Sh a p e Mo d e l C re a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 5 5 7 Ab o u t Me n u . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 6 5 8 Se ri a l D a t a T ra n sf e r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 7 C h a p t e r 6 Assi st F u n ct i o n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 9 6 1 O ve rvi e w . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 9 6 2 L i n e a r Assi st F u n ct i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 9

PAGE 6

i i i 6 3 Pl a n a r Assi st F u n ct i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 6 4 Ve l o ci t y Assi st F u n ct i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 6 5 Assi st F u n ct i o n I n t e g ra t i o n i n t o t h e U se r I n t e rf a ce . . . . . . . . . . . . . . 5 4 C h a p t e r 7 Exp e ri me n t a l D a t a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 7 1 Ph ysi ca l Exp e ri m e n t a l Se t u p . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 7 2 Exp e ri m e n t a l Pro c e d u r e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 7 7 3 Exp e ri m e n t a l D a t a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 8 C h a p t e r 8 C o n cl u si o n s a n d F u t u r e W o rk . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 8 1 C o n cl u si o n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 8 2 F u t u re W o rk a n d R e c o mme n d a t i o n s . . . . . . . . . . . . . . . . . . . . . . . . 6 5 R e f e re n c e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 6 Bi b l i o g ra p h y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 7 Ap p e n d i c e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 9 Ap p e n d i x A: G ra p h i c a l U se r I n t e rf a c e So u rce C o d e . . . . . . . . . . . . . . . . 7 0

PAGE 7

i v L i s t o f T a b l e s T a b l e 2 1 : I n t ri n s i c C a me r a Pa ra me t e r s f o r t h e I ma g i n g So u rc e C a me r a . . . . . 1 7 T a b l e 2 2 : I n t ri n s i c C a me r a Pa ra me t e r s f o r t h e H i t a ch i C a m e ra . . . . . . . . . . . . 1 7 T a b l e 2 3 : Ext ri n s i c C a me r a Pa ra me t e rs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 T a b l e 5 1 : Se r i a l Po rt Se t t i n g s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 8

PAGE 8

v L i s t o f F i g u r e s F i g u re 1 1 : R W T H Ma n u s Ma n i p u l a t o r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 F i g u re 2 1 : Ea rl y St e re sco p i c Vi e w i n g D e vi ce [ 6 ] . . . . . . . . . . . . . . . . . . . . . . . . 6 F i g u re 2 2 : Vi si o n Syst e m Sch e m a t i c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 F i g u re 2 3 : T h e Pe rs p e ct i ve C a me ra Mo d e l . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 F i g u re 2 4 : C o o rd i n a t e Syst e m Assi g n me n t . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 5 F i g u re 2 5 : C a l i b ra t i o n T a b l e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 6 F i g u re 3 1 : O ri g i n a l I ma g e o f C ro ss O b j e ct . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 6 F i g u re 3 2 : R e g i o n o f I n t e re st . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 7 F i g u re 3 3 : T h re s h o l d o f 1 0 8 0 a n d 1 0 0 R e sp e ct i ve l y (T o p t o Bo t t o m) . . . . . . 2 9 F i g u re 4 1 : St e re o Vi si o n G e o me t ry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 F i g u re 5 1 : Ma i n G ra p h i ca l U s e r I n t e rf a ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 8 F i g u re 5 2 : C a l i b ra t i o n T a b l e C r e a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 9 F i g u re 5 3 : I n t ri n s i c Pa ra me t e r C a l i b ra t i o n I n t e rf a ce . . . . . . . . . . . . . . . . . . . . . 4 1 F i g u re 5 4 : E xt e rn a l C a m e ra Pa ra m e t e r C a l i b ra t i o n Su b r o u t i n e . . . . . . . . . . . . 4 3 F i g u re 5 5 : Sh a p e Mo d e l C r e a t i o n G U I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 5 F i g u re 5 6 : Ab o u t Vi si o n Assi st I n t e rf a ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 7 F i g u re 5 7 : R S 2 3 2 Se ri a l C o n n e ct o r Pi n Assi g n m e n t . . . . . . . . . . . . . . . . . . . . 4 8 F i g u re 6 1 : I ma g e F ra me o f Vi si o n Sy st e m [ 4 ] . . . . . . . . . . . . . . . . . . . . . . . . . 5 0 F i g u re 6 2 : Ve l o ci t y Assi st F u n ct i o n S ca l e f a ct o r vs. Ve rt i ca l D i st a n ce . . . . . . . 5 3

PAGE 9

vi F i g u re 7 1 : Exp e ri me n t a l Se t U p . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 7 F i g u re 7 2 : D o o r H a n d l e D e t e ct i o n f ro m Z = 0 5 4 m . . . . . . . . . . . . . . . . . . . . . 5 9 F i g u re 7 3 : D o o r H a n d l e D e t e ct i o n f ro m Z = 0 4 4 m . . . . . . . . . . . . . . . . . . . . . 5 9 F i g u re 7 4 : D o o r H a n d l e D e t e ct i o n f ro m Z = 0 3 4 m . . . . . . . . . . . . . . . . . . . . . 6 0 F i g u r e 7 5 : D o o r H a n d l e D e t e ct i o n f ro m Z = 0 1 4 m . . . . . . . . . . . . . . . . . . . . . 6 1 F i g u re 7 6 : Erro r i n X a s a F u n ct i o n o f X f o r D i f f e re n t Z Po si t i o n s . . . . . . . . . . . 6 1 F i g u re 7 7 : Erro r i n Y P o si t i o n a s a F u n ct i o n o f Y a n d f o r D i f f e re n t Z Va l u e s . . 6 2

PAGE 10

vi i Sh a p e B a se d St e re o vi si o n Assi st a n ce i n R e h a b i l i t a t i o n R o b o t i cs Mi ch a e l U l ri c h Ju rczyk A B ST R A C T A g ra p h i ca l u s e r i n t e rf a ce p ro g r a m w a s cre a t e d a l o n g w i t h s h a p e mo d e l s, w h i ch a l l o w p e rso n s w i t h d i sa b i l i t i e s t o se t u p a st e re o vi s i o n syst e m w i t h o f f t h e sh e l f h a rd w a re a n d d e t e ct o b j e ct s o f i n t e re st w h i ch ca n b e p i cke d u p u s i n g a se n so r a ssi st e d t e l e r o b o t i c ma n i p u l a t o r. A H i t a ch i KP D 5 0 C C D ca m e ra a n d a n I ma g i n g S o u rce C C D ca me r a w e r e u se d a l o n g w i t h t w o I ma g i n g So u rc e D F G / L C 1 f ra me g ra b b e rs t o se t u p a st e re o vi si o n syst e m. I n o rd e r t o u se t h e st e re o vi si o n s yst e m, t h e t w o ma i n p ro b l e ms o f co rre s p o n d e n c e a n d re c o n st ru ct i o n a r e so l ve d u si n g s u b r o u t i n e s o f t h e p ro g r a m cre a t e d f o r t h i s w o rk. T h e u se r i n t e rf a c e a l l o w s t h e u se r t o e a si l y p e rf o rm t h e i n t ri n si c a n d e xt ri n s i c ca me ra c a l i b ra t i o n re q u i r e d f o r st e r e o vi si o n b y f o l l o w i n g a f e w b a s i c st e p s i n co rp o ra t e d i n t o t h e u se r i n t e rf a c e p ro g ra m, w h i ch a r e d e scri b e d i n t h i s t h e si s. A ca l i b r a t i o n t a b l e re q u i re d f o r t h e se t a sks ca n a l s o b e e a si l y cre a t e d u s i n g t h e p ro g ra m. I n o rd e r t o d e t e ct t h e o b j e ct o f i n t e re st sh a p e mo d e l s, cre a t e d b y t h e u se r i n t e rf a ce p r o g ra m, a r e u se d t o so l ve t h e co rr e sp o n d e n ce p ro b l e m o f st e re o vi si o n T h e co rre s p o n d e n ce p ro b l e m i s t h a t o f l o ca t i n g c o rre sp o n d i n g

PAGE 11

vi i i p o i n t s i n t h e l e f t e y e a n d t h e r i g h t e ye w h i ch a re n e c e ssa ry t o p e rf o r m t h e ca l c u l a t i o n s t o o b t a i n t h e l o c a t i o n o f t h e o b j e ct o f i n t e re st w i t h re sp e ct t o t h e e n d e f f e ct o r. T h e sh a p e mo d e l s cre a t e d f o r so me co mmo n l y a va i l a b l e i t e ms su ch a s a d o o rkn o b a n d a d o o r h a n d l e a re i n cl u d e d i n t h e p ro g ra m a n d u se d t o t e st t h e st e re o vi si o n syst e m. As e xp e ct e d t h e e rro r o f d e t e ct i o n d e cre a se s a s t h e st e re o syst e m i s m o ve d cl o se r t o t h e o b j e ct o f i n t e re st i n t h e x y a n d z p o s i t i o n T h e u se r i n t e rf a ce cre a t e d a l l o w s t h e u se r t o d e t e ct t h e o b j e ct o f i n t e re st a p p ro a c h i t w i t h a t e l e ro b o t i c ma n i p u l a t o r u si n g ve l o ci t y sca l e d a ssi st f u n ct i o n s, w h i ch sca l e t h e ve l o ci t y o f t h e t e l e ro b o t i c m a n i p u l a t o r p e rp e n d i cu l a r t o t h e d e si re d mo ve me n t a n d f i n a l l y g ra sp t h e o b j e ct T h e u se r ca n t h e n m o ve t h e o b j e ct t o a n y d e si re d l o ca t i o n

PAGE 12

1 C h a p te r 1 I n tr o d u c ti o n 1 1 Mo ti v a ti o n Pe rf o rma n c e o f t a sks ra n g i n g f ro m o p e n i n g d o o rs t o re t ri e vi n g d o cu m e n t s f ro m sh e l ve s re q u i re l i t t l e t h o u g h t a n d e f f o rt f o r mo st p e o p l e T o p e rso n s w i t h d i sa b i l i t i e s, h o w e v e r, t h e se e ve ry d a y t a sks o f t e n t i me s re q u i re a g r e a t d e a l o f e f f o rt o r a re e ve n i mp o ss i b l e w i t h o u t t h e a i d o f o t h e r p e o p l e o r d e vi c e s. T h e Ame ri c a n s w i t h D i sa b i l i t i e s Act o f 1 9 9 0 d e f i n e s d i sa b i l i t y a s a su b st a n t i a l l i mi t a t i o n i n a m a j o r l i f e a ct i vi t y. I f t h i s d e f i n i t i o n i s a p p l i e d t o t h e 1 9 9 7 C e n su s p e rf o rme d b y t h e N a t i o n a l C e n su s Bu re a u a p p ro x i ma t e l y 1 9 7 % o f t h e Ame ri ca n p o p u l a t i o n i s c o n si d e re d t o l i ve w i t h a d i s a b i l i t y [ 1 ] O f t h i s f i f t h o f t h e p o p u l a t i o n 1 2 3 % o f t h e p o p u l a t i o n w a s c o n si d e re d t o h a v e a se v e re d i s a b i l i t y, w h i l e 3 8 % o f t h e t o t a l p o p u l a t i o n re q u i re s p e rso n a l a ssi st a n c e T h i s m e a n s t h a t j u st o ve r 1 0 mi l l i o n p e rso n s i n t h e U n i t e d St a t e s a re c o n si d e r e d t o b e s e ve re l y d i sa b l e d a n d d e p e n d e n t o n p e rs o n a l a ssi st a n c e [ 2 ] T h i s n u mb e r i s a st ro n o mi c a l c o n si d e ri n g t h a t t h e r e a re n o co mm e rci a l l y a va i l a b l e r o b o t i c syst e ms t h a t i n co rp o r a t e se n so r a ss i st a n ce t o a i d p e rso n s w i t h d i s a b i l i t i e s. 1 2 Vi s i o n i n R o b o ti c s F i g u re 1 1 s h o w s a n a l t e r e d M a n u s Ma n i p u l a t o r [ 3 ] e q u i p p e d w i t h a si n g l e ca me ra a t t a ch e d t o t h e e n d o f i t s e n d e f f e ct o r. C u rre n t l y, t h e re i s n o v i si o n

PAGE 13

2 syst e m a va i l a b l e o n t h e ma rke t t h a t i s u se d f o r r e h a b i l i t a t i o n ro b o t i c a p p l i ca t i o n s. Al l vi si o n syst e ms t h a t a ct u a l l y a r e u se d i n i n d u st ry a n d re se a rch re q u i r e a n e xp e n si ve so f t w a re p a cka g e a s w e l l a s a p e rs o n c a p a b l e o f p ro g ra mmi n g F o r p e rso n s w i t h d i sa b i l i t i e s, h o w e v e r, t h i s me a n s t h a t i t i s si m p l y n o t p o ssi b l e t o u se vi si o n a ss i st a n ce f o r c o n t ro l o f t h e i r ma n i p u l a t o r, a s t h i s w o u l d co st t o o mu ch T h e ma i n f u n ct i o n o f t h e vi si o n syst e m i s t o d e t e ct co mmo n o b j e ct s su ch a s d o o rkn o b s a n d u p o n co n f i rm a t i o n p e rf o rm a si mp l e f u n ct i o n l i k e t u rn t h e kn o b F i gu r e 1 1 : R WT H M an u s M an i p u l at or I t i s e xt re me l y i n t e re st i n g t o n o t e t h a t d e sp i t e e xt e n si ve re s e a rch i n st e re o vi si o n i t se l f t h e re i s n o co mme r ci a l l y a va i l a b l e st e re o vi si o n syst e m f o r u se w i t h ro b o t i c ma n i p u l a t o rs o r e ve n f o r p e rso n a l u s e H e n ce t h i s t h e s i s a d d r e sse s t h i s g a p i n a va i l a b i l i t y i n se n so r e q u i p p e d ro b o t i c ma n i p u l a t o rs.

PAGE 14

3 Si n ce st e re o vi si o n i s n o t a n e w co n ce p t a f e w p o i n t s sh o u l d b e m a d e T h e re a re m a n y d i f f e re n t t e ch n i q u e s f o r d e t e ct i n g o b j e ct s. T h e s e r a n g e f ro m a p p l yi n g st i cke rs t o o b j e ct s o f i n t e re st t o d e t e ct i n g mo t i o n a n d u si n g t h i s d a t a t o d e t e ct a n o b j e ct H o w e v e r, t h e re i s n o t a p ro g ra m a v a i l a b l e w h i c h i n co rp o ra t e s s h a p e mo d e l s, a s d e scri b e d a n d d e v e l o p e d i n t h i s t h e si s, t o d e t e ct e ve ry d a y o b j e c t s. 1 3 Pa s t W o r k T h e re h a s b e e n so me w o rk d o n e i n t h e R e h a b i l i t a t i o n a n d En g i n e e ri n g Pro g ra m a t t h e U n i ve rsi t y o f So u t h F l o ri d a i n T a mp a F l o ri d a Be n j a mi n E. F ri t z [ 4 ] h a s se t u p a t e l e ro b o t i c syst e m w i t h h a p t i c i n p u t t o e n h a n c e ma n i p u l a t i o n ca p a b i l i t i e s o f p e rs o n s w i t h d i s a b i l i t i e s I n h i s w o rk, Be n j a m i n F ri t z u se d a ro b o t i c ma n i p u l a t o r f ro m R o b o t i cs R e se a rch C o rp o r a t i o n e q u i p p e d w i t h a s i n g l e c a me ra a n d a l a s e r ra n g e f i n d e r t o d e t e ct a n o b j e ct o f kn o w n ci rcu l a r s h a p e O n ce t h e o b j e ct w a s d e t e ct e d a se t o f a ssi st f u n ct i o n s, d e scri b e d b r i e f l y i n c h a p t e r 6 o f t h i s t h e si s, w e re e mp l o y e d t o sca l e t h e ve l o ci t y i n t h e x y a n d z d i re c t i o n s su ch t h a t t h e ma n i p u l a t o r co u l d cl o s e i n o n t h e o b j e ct o f i n t e re st w h i l e l i mi t i n g t h e mo v e me n t c a p a b i l i t i e s o f t h e u se r, su ch t h a t t re mo rs i n i n p u t co u l d b e l i mi t e d p ro vi d i n g a smo o t h t ra j e ct o ry t o w a r d s t h e o b j e ct t o b e ma n i p u l a t e d W h i l e t h e se a ssi st f u n ct i o n s h e l p t h e u s e r i n co n t ro l l i n g t h e ro b o t i c ma n i p u l a t o r, t h e si mp l e o b j e ct d e t e ct i o n ma k e s t h e syst e m i m p ra ct i ca l f o r d a i l y u s e si n ce i t i s i mp r a ct i ca l t o o n l y b e a b l e t o m a n i p u l a t e o b j e ct s w i t h a c i rcu l a r a r e a A d d i t i o n a l l y, i t i s n o t a l w a ys p o ss i b l e t o d e t e rmi n e t h e a r e a o f t h e o b j e ct o f i n t e re st a n d i t w o u l d b e

PAGE 15

4 n i ce t o b e a b l e t o d e t e ct o b j e ct s o f si mi l a r g e o me t ri e s w i t h va ryi n g si ze s. T h i s t h e si s so l v e s t h e se i ssu e s, a n d t h e y a re l i st e d i n t h e f o l l o w i n g se ct i o n W e n t a o Yu a d d r e sse s t h e d e ve l o p me n t o f a t e l e m a n i p u l a t i o n syst e m u si n g i n t e l l i g e n t ma p p i n g f ro m a h a p t i c u se r i n t e rf a ce t o a re m o t e ma n i p u l a t o r t o a ssi st i n ma x i mi zi n g t h e m a n i p u l a t i o n ca p a b i l i t i e s o f p e rso n s w i t h d i s a b i l i t i e s. T h i s ma p p i n g re f e rr e d t o a s a ssi st a n c e f u n ct i o n i s d e t e rmi n e d o n t h e b a si s o f e n vi ro n me n t a l mo d e l o r re a l t i me se n so ry d a t a t o g u i d e t h e m o t i o n o f a t e l e ro b o t i c ma n i p u l a t o r w h i l e p e rf o rm i n g a g i v e n t a sk. H u ma n i n p u t i s e n h a n ce d ra t h e r t h a n s u p e rs e d e d b y t h e co m p u t e r. T h i s i s p a rt i cu l a rl y u se f u l w h e n t h e u se r h a s re st ri ct e d r a n g e o f mo ve m e n t s d u e t o ce rt a i n d i sa b i l i t i e s su c h a s mu s cu l a r d yst ro p h y, a st ro ke o r a n y f o rm o f p a t h o l o g i c a l t re m o r [ 5 ] W e n t a o Yu s w o rk e mp l o y e d t h e st e re o vi s i o n syst e m cre a t e d f o r t h i s t h e s i s t o t e st a H i d d e n M a rko v Mo d e l d e scr i b e d i n h i s d i sse rt a t i o n t o a ssi st p e rso n s w i t h d i s a b i l i t i e s i n e n h a n ci n g ma n i p u l a t i o n ca p a b i l i t i e s w h e n e m p l o y i n g t e l e ro b o t i cs. 1 4 T h e s i s O b j e c ti v e s T h e re a r e f i v e ma i n o b j e ct i ve s o f t h i s re s e a rc h w o rk, w h i ch a re a l so e xt e n si v e l y d e scr i b e d h e re i n T h e se a re a s f o l l o w s: 1 Se t u p a st e re o vi si o n sys t e m f ro m c o mme rci a l l y a v a i l a b l e p r o d u ct s a t a l o w co st I n o rd e r t o d o t h i s, o f f t h e sh e l f ca me r a s a n d co mp u t e r e q u i p me n t sh o u l d b e u se d a n d co n n e ct e d p r o p e r l y. T h i s i s i mp o rt a n t si n ce a s d i scu ss e d b e f o re t h e r e a re n o co mme rc i a l l y a va i l a b l e

PAGE 16

5 st e re o vi si o n syst e ms f o r u se w i t h p e rso n s w i t h d i s a b i l i t i e s. H e n ce p e rso n s w i t h d i s a b i l i t i e s sh o u l d b e a b l e t o cre a t e t h e i r o w n st e re o vi si o n syst e m. 2 C re a t e a l i b ra ry o f sh a p e mo d e l s f o r e ve ry d a y i t e ms. T h e se sh a p e mo d e l s sh o u l d b e sp e c i f i c e n o u g h t o a l l o w f o r p ro p e r d e t e ct i o n o f t h e o b j e ct e ve n i n a cl u t t e re d e n vi r o n me n t b u t a l so b e b r o a d e n o u g h t o a l l o w f o r t h e o b j e ct o f i n t e re st t o b e d e t e ct e d e ve n i f t h e o b j e ct i s n o t e x a ct l y t h e sa me o b j e ct u se d t o cre a t e t h e m o d e l 3 C re a t e a u se r f ri e n d l y i n t e rf a ce t o c o n t ro l t h e a cq u i si t i o n o f i ma g e s, a s w e l l a s a n y o t h e r st e re o vi si o n p r o b l e ms. T h i s me a n s t h a t t h e p r o g ra m sh o u l d a l l o w t h e u se r t o o b t a i n i m a g e s f ro m b o t h ca m e ra s, b e a b l e t o e a si l y o b t a i n t h e i n t ri n si c a s w e l l a s t h e e xt ri n si c c a me ra p a r a me t e rs, a n d t h e u s e r s h o u l d a l so b e a b l e t o cre a t e t h e i r o w n s e t o f s h a p e m o d e l s w i t h o u t a n y p ro g r a mmi n g kn o w l e d g e 4 F i n a l l y, t h e u se r i n t e rf a ce sh o u l d b e t e st e d a l o n g w i t h t h e a cq u i re d i n t ri n si c a n d e xt ri n si c c a me ra p a ra m e t e rs t o d e t e rmi n e t h e a ccu ra cy o f t h e so f t w a re b u n d l e cr e a t e d f o r t h i s w o r k. T h e o b j e ct o f i n t e re st h a s t o b e f o u n d w i t h su f f i ci e n t h i g h a ccu r a cy, e v e n i f p a rt i a l l y co ve r e d 5 Sca l i n g a ss i st f u n ct i o n s, a s d e scri b e d b y B e n j a mi n F ri t z [ 4 ] sh o u l d b e i n t e g ra t e d i n t o t h e u s e r i n t e rf a ce a n d p r o g r a m, a l l o w i n g p e rs o n s w i t h d i sa b i l i t i e s t o a p p ro a ch o b j e ct s o f i n t e re st w i t h r e l a t i v e e a s e e ve n i f t h e u se r h a s t re mo rs i n mo ve me n t

PAGE 17

6 C h a p te r 2 Ste r e o v i s i o n 2 1 H i s to r y o f Ste r e o v i s i o n St e re o vi si o n syst e ms a r e n o t h i n g n e w t o sci e n ce a n d e n g i n e e r i n g T h e p ri n ci p l e s o f st e r e o vi s i o n g o a s f a r b a ck a s a n ci e n t G re e ce w h e n Eu cl i d f i rst e xp l a i n e d t h e p ri n ci p l e s o f st e re o vi si o n I t w a s Eu cl i d w h o sh o w e d t h a t t h e l e f t a n d ri g h t e ye s s e e d i f f e re n t i ma g e s. T h e b ra i n t h e n u se s t h e s e t w o i m a g e s a n d e sp e ci a l l y t h e d i scre p a n ci e s, o r d i f f e re n t p o si t i o n o f a n o b j e ct r e l a t i v e t o e a ch e ye t o p ro d u c e t h e p e rce p t i o n o f d e p t h F i gu r e 2 1 : E ar l y S t e r e os c op i c V i e w i n g D e vi c e [ 6]

PAGE 18

7 I n t h e f i rst t h i rd o f t h e n i n e t e e n t h c e n t u ry, Si r C h a rl e s W h e a t st o n e p re se n t e d h i s t h e o ri e s a n d d i sc o ve ri e s o n st e re o vi si o n b y sh o w i n g t w o s e p a r a t e l y d ra w n i ma g e s w i t h sl i g h t d i sp a r i t i e s t h a t w h e n vi e w e d t h ro u g h h i s st e re o sco p i c vi e w i n g a p p a r a t u s, a p p e a re d a s a t h re e d i me n si o n a l o b j e ct [ 7 ] U p u n t i l r o u g h l y t h e mi d t w e n t i e t h ce n t u ry, st e re o vi si o n w a s u se d ma i n l y a s a f o rm o f e n t e rt a i n me n t w h e re i ma g e s, o r p h o t o g ra p h s, w h i ch w e re t a k e n w i t h a p p ro xi m a t e l y 7 cm sp a ci n g w e r e vi e w e d t h r o u g h a st e r e o sco p i c vi e w i n g d e vi ce su c h a s t h a t i n F i g u re 1 1 a b o v e t o p ro d u c e t h e p e rc e p t i o n o f a t h re e d i me n si o n a l sc e n e I n 1 9 5 9 Be l a Ju l e sz d e mo n st ra t e d t h a t t h e b ra i n i s a b l e t o p e rc e i ve 3 D i n f o rma t i o n w i t h o u t a n y t ru e p e rce p t u a l d e p t h cu e s. H e i n v e n t e d t h e f i rst ra n d o m d o t st e re o g r a m i n 1 9 5 9 i n a t t e mp t s t o t e st st e re o p si s. T h e e x p e ri m e n t h e p e rf o rme d w a s w i t h t h e ra n d o m d o t i ma g e a n d b y sh i f t i n g i t sl i g h t l y t o p ro d u ce a se co n d i ma g e W h e n vi e w e d i n a st e re o sco p e t h e p a t t e rn a p p e a r e d a s a 3 D i ma g e By su rro u n d i n g t h e i m a g e s w i t h a n i d e n t i ca l ra n d o m d o t p a t t e rn t h e y a p p e a r i d e n t i c a l w h e n v i e w e d se p a r a t e l y, b u t w h e n vi e w e d st e re o sco p i ca l l y t h e ce n t ra l s q u a re p a t t e rn ( o r w h a t e v e r sh a p e i s cre a t e d ) w i l l a p p e a r t o f l o a t i n s p a ce [ 8 ] By t h e e a rl y 1 9 8 0 s, Ma rr a n d Po g g i o co me u p w i t h co mp u t a t i o n a l mo d e l s f o r d e scri b i n g d e p t h p e rce p t i o n i n r a n d o m d o t i ma g e s. I n t h e p re se n t st e re o v i si o n i s e me rg i n g i n mo re a n d mo re a re a s s u ch a s a e ri a l p h o t o g r a p h y, me d i ca l a p p l i ca t i o n s, a n d ma n y mo re a r e a s.

PAGE 19

8 St e re o vi si o n a l s o kn o w n a s st e r e o g r a p h y, h a s b e e n u se d i n c o mp u t e r v i si o n f o r su ch a p p l i c a t i o n s a s o b t a i n i n g a t h re e d i m e n s i o n a l p ro f i l e o f a s u rf a c e A co mmo n l y kn o w n e x a mp l e i s t h a t o f u si n g a sa t e l l i t e w h i c h f l i e s a b o v e t h e e a rt h s su rf a c e a n d o b t a i n s i n f o rma t i o n a b o u t t h e e a rt h s t o p o g ra p h y. W h i l e i t i s si mp l e t o s e t u p t w o ca me r a s t o o b t a i n i ma g e s s i mu l t a n e o u sl y, t h e re a re t w o ma i n p ro b l e ms e n co u n t e r e d i n st e re o vi si o n T h e se a re t h e p r o b l e ms o f co rre sp o n d e n ce a n d r e co n st ru ct i o n T h e a n sw e r t o t h e se t w o p ro b l e ms i s d i scu sse d i n t h i s ch a p t e r. F i rst t h i s ch a p t e r a d d re ss e s t h e p r e l i m i n a ry st e p s re q u i r e d t o se t u p a st e re o v i si o n syst e m. 2 2 H a r d w a r e T h e h a rd w a re u se d co n si st s o f t w o ma i n co m p o n e n t p a i rs. T h e se co mp o n e n t s a re t h e H i t a c h i KP D 5 0 C C D c a me r a a n I m a g i n g S o u rce C C D ca me ra a n d t w o I ma g i n g So u rce D F G / L C 1 f ra me g ra b b e rs. T h e t e ch n i ca l d e t a i l s o f t h e se ca me r a s ca n b e f o u n d o n t h e ma n u f a ct u re rs w e b si t e s, i n c l u d e d i n t h e re f e re n ce s s e ct i o n T h e p a r a me t e r s re q u i r e d f o r st e re o vi si o n h o w e v e r a re d i scu sse d A sch e ma t i c re p r e se n t a t i o n o f t h e s e t u p o f t h e vi si o n syst e m h a r d w a r e i s sh o w n i n F i g u re 2 2 b e l o w T h e ca m e ra s a re a t t a ch e d vi a a co a xi a l ca b l e t o t h e f ra me g ra b b e rs a n d a ss i g n e d d i f f e re n t p o rt n u mb e rs so a s t o n o t ca u se a n y i n t e rf e re n c e w h e n g ra b b i n g i m a g e s.

PAGE 20

9 Bo t h ca me ra s a re a b l e t o o b t a i n i m a g e s co n t i n u o u sl y i n t h e st a n d a rd N T SC vi d e o f o rma t a s w e l l a s t h e E u ro p e a n PAL f o rma t N T SC i s u se d f o r t h i s p r o j e ct T h e i ma g e a cq u i si t i o n s p e e d h o w e v e r, i s l i mi t e d b y t h e p ro g r a m, w h i ch g ra b s t h e i m a g e s a n d t h e n a n a l yze s t h e se Mo re a b o u t t h i s w i l l b e d i sc u sse d l a t e r o n i n t h e t h e si s. F i g u r e 2 2 : V i s i on S ys t e m S c h e m a t i c 2 3 So ftw a r e I n o rd e r t o cre a t e t h e sh a p e m o d e l s, w h i ch a r e u se d t o l o ca t e t h e o b j e ct o f i n t e re st i n b o t h c a me r a vi e w s, H a l c o n f ro m t h e c o mp a n y MVT e c i s u se d M VT e c i s sp e c i a l i ze d i n so f t w a r e so l u t i o n s f o r ma ch i n e vi s i o n a p p l i c a t i o n s u si n g st a n d a rd h a rd w a re a s t h e co mp a n y s w e b si t e st a t e s t h e i r mi ssi o n [ 8 ] T h e so f t w a re p a cka g e e mp l o ye d i n t h i s r e se a rch w o rk i s ca l l e d H a l co n W h i l e so me o f t h e t a sks u s e d i n t h i s p ro j e ct c a n b e cr e a t e d u s i n g H a l co n i t i s i m p o rt a n t t o

PAGE 21

1 0 re a l i z e t h a t t h i s i s ve ry l a b o r i n t e n si v e a n d a s su ch n o t a vi a b l e o p t i o n f o r t h e a ve ra g e u se r. H e n ce t h e Sh a p e D e t e ct i o n Pr o g ra m cre a t e d a s p a rt o f t h i s p ro j e ct i n co rp o ra t e s a l l o f t h e re q u i r e d t a sks i n o rd e r t o e n a b l e a n y p e rso n t o u se st e re o vi si o n f o r a ssi st a n ce i n ro b o t i c ma n i p u l a t i o n H a l co n co n t a i n s a l i b r a ry o f cl o s e t o 1 0 0 0 vi si o n o p e ra t o rs co ve r i n g a p p l i ca t i o n s su ch a s f a ct o ry a u t o ma t i o n q u a l i t y co n t ro l re mo t e se n s i n g a e ri a l i ma g e i n t e rp r e t a t i o n me d i ca l i ma g e a n a l ysi s, su rv e i l l a n c e t a sks, a n d ma n y mo re a re a s. I t a l s o i s a b l e t o ma t ch p a t t e r n s, a n d p e rf o rm b l o b a n a l ys i s. T h e s e t w o f u n ct i o n s a re u s e d i n t h i s r e se a rc h w o rk t o f i n d o b j e ct s o f i n t e r e st t h a t a re t o b e ma n i p u l a t e d b y t h e r o b o t i c m a n i p u l a t o r. T h e va ri o u s vi si o n syst e m a l g o ri t h ms cre a t e d a r e d i scu ss e d i n t h e n e xt ch a p t e r i n mu ch mo r e d e t a i l 2 4 C a m e r a Mo d e l a n d T h e o r y I n o rd e r t o u n d e rst a n d t h e i n t ri n si c a n d e xt ri n si c ca me r a p a ra m e t e r ca l i b ra t i o n d i scu sse d i n t h e n e xt se ct i o n i t i s e s se n t i a l t o f i rst d e scri b e s o me b a si c e q u a t i o n s re l a t e d t o o p t i cs a n d mo re sp e ci f i ca l l y t o ca me ra s. T h e mo st co mmo n mo d e l o f a ca me ra i s t h e p e rsp e ct i ve mo d e l a l s o kn o w n a s t h e p i n h o l e m o d e l w h i ch i s sh o w n i n F i g u r e 2 3 Po i n t O l i e s a t t h e c e n t e r o f t h e ca m e ra f ra m e a n d i s kn o w n a s t h e ce n t e r o r f o c u s o f p r o j e ct i o n T h e i ma g e p l a n e co n t a i n s t h e p i xe l c o o rd i n a t e s x a n d y, w h i c h re p r e se n t t h e p r o j e ct i o n o f t h e p o i n t o f i n t e re st P, o n t o t h e i ma g e p l a n e g i v i n g t h e p o i n t p T h e d i st a n c e b e t w e e n t h e c a me ra f ra me a n d t h e i m a g e p l a n e i s d e f i n e d a s t h e f o ca l l e n g t h f

PAGE 22

1 1 F i gu r e 2 3 : T h e P e r s p e c t i ve C am e r a M od e l F ro m b a si c g e o me t ry, i t ca n b e sh o w n t h a t a p o i n t p = (x, y) i n t h e i ma g e p l a n e i s re l a t e d t o a p o i n t P= (X, Y) i n t h e ca me ra p l a n e b y e q u a t i o n (2 1 ): (2 1 ) I n o rd e r t o t ra n sl a t e a p o i n t P w i n t h e w o rl d f ra m e t o a p o i n t P c i n t h e ca m e ra f ra me a ro t a t i o n m a t ri x, d e n o t e d b y R a n d a t ra n sl a t i o n ma t ri x, d e n o t e d b y T i s r e q u i r e d U s i n g t h e ro t a t i o n a n d t r a n sl a t i o n m a t ri x, o n e ca n s h o w t h a t t h e re l a t i o n b e t w e e n t h e t w o p o i n t s i s g i ve n b y e q u a t i o n (2 2 ): P c = R (P w T ) (2 2 ) w h e re t h e r o t a t i o n ma t ri x i s d e f i n e d b y e q u a t i o n (2 3 ) a s: (2 3 )

PAGE 23

1 2 N e g l e ct i n g g e o m e t ri c d i st o rt i o n s d e ri ve d f ro m o p t i cs, t h e co o rd i n a t e s o f a n i ma g e p o i n t i n p i x e l u n i t s (x i m y i m ) ca n b e re l a t e d t o t h e s a me p o i n t (x, y) i n t h e ca me ra re f e r e n ce f ra me b y e q u a t i o n ( 2 4 ): (2 4 ) w h e re ( o x o y ) a r e t h e c o o rd i n a t e s i n p i xe l s o f t h e i ma g e c e n t e r, a n d (s x s y ) i s t h e e f f e ct i ve si ze o f t h e p i xe l i n t h e h o ri zo n t a l a n d ve rt i ca l d i re ct i o n re sp e ct i ve l y. By co mb i n i n g e q u a t i o n (2 2 ) a n d e q u a t i o n ( 2 4 ) w i t h e q u a t i o n (2 1 ) i t ca n b e sh o w n t h a t t h e l i n e a r ve rsi o n o f t h e p e rsp e ct i v e p ro j e ct i o n e q u a t i o n c a n b e w ri t t e n a s e q u a t i o n (2 5 ): (2 5 ) w h e re R 1 R 2 a n d R 3 d e n o t e a 3 D ve ct o r f o rme d b y t h e 1 st 2 n d a n d 3 r d ro w re sp e ct i ve l y, o f e q u a t i o n (2 3 ). Eq u a t i o n (2 5 ) d i re ct l y l i n ks t h e p i xe l co o rd i n a t e s o f a n i m a g e p o i n t w i t h t h e w o rl d c o o rd i n a t e s o f t h e c o rre s p o n d i n g 3 D p o i n t w i t h o u t e x p l i c i t re f e re n ce t o t h e ca m e ra re f e re n c e f ra me n e e d e d H o w e ve r t h e i n t ri n si c a n d e xt ri n s i c ca m e ra p a ra me t e rs a re n o t i n d e p e n d e n t o f e a ch o t h e r T o f a ci l i t a t e c a l cu l a t i o n o f i n t ri n s i c a n d e xt ri n si c ca m e ra p a ra m e t e rs, e q u a t i o n (2 5 ) ca n b e w ri t t e n a s a m a t ri x p ro d u ct o f t h e i n t ri n si c a n d e xt ri n si c ca me ra p a ra me t e rs, yi e l d i n g e q u a t i o n s (2 6 ) a n d (2 7 ), w h i ch d e n o t e t h e i n t ri n si c a n d e xt ri n si c ca me r a p a r a me t e rs, re sp e ct i ve l y.

PAGE 24

1 3 (2 6 ) (2 7 ) N o w t h a t t h e i n t ri n s i c a n d e xt ri n s i c m a t ri ce s a n d p a r a me t e rs a r e d e f i n e d i t i s n e ce ssa ry t o a cc u ra t e l y o b t a i n t h e se va l u e s. I n a d d i t i o n t o t h e e xt r i n si c ma t ri x d e n o t e d b y e q u a t i o n ( 2 7 ) a b o ve t h e r e i s a l so a t ra n s l a t i o n ma t ri x, w h i c h d e f i n e s t h e t ra n sl a t i o n b e t w e e n t h e ca m e ra s a n d t h e e n d e f f e ct o r. T h e i n t ri n s i c ca me ra p a ra me t e rs a re o b t a i n e d b y p e rf o rm i n g t h e i n t ri n s i c ca m e ra ca l i b ra t i o n w h i l e t h e e xt ri n si c ca me r a p a r a me t e rs a re o b t a i n e d vi a t h e e xt ri n si c ca me r a ca l i b r a t i o n Bo t h o f t h e ca l i b r a t i o n p r o ce ss e s, a l o n g w i t h t h e re su l t s, a re d e scr i b e d i n t h e f o l l o w i n g s e ct i o n s. 2 5 I n tr i n s i c C a m e r a Pa r a m e te r s I n o rd e r t o u se st e r e o vi s i o n t h e i n t ri n s i c c a me ra p a ra me t e rs n e e d t o b e d e t e rmi n e d T h e i n t ri n si c c a me ra p a ra me t e rs ch a ra ct e ri z e t h e t ra n sf o rm a t i o n ma p p i n g o f a n i ma g e p o i n t f ro m c a m e ra t o p i xe l co o r d i n a t e s f o r e a ch i n d i vi d u a l ca me ra W h i l e t h e ma n u f a ct u re r o f a ca me ra g e n e r a l l y g i ve s a st a n d a rd i z e d l i st o f t h e se p a r a me t e rs, i t i s f o u n d t h a t t h e se o n l y re se m b l e a n i d e a l i ze d a v e ra g e f o r t h e se va l u e s. H e n ce i t i s i mp o rt a n t t o d e t e rmi n e t h e i n t ri n s i c ch a r a ct e ri st i cs o f

PAGE 25

1 4 e a ch c a me ra a ccu r a t e l y, a s t h e se p a ra me t e rs a r e re q u i r e d t o o b t a i n t h e t h re e d i me n s i o n a l l o c a t i o n o f a p o i n t i n t e rms o f t h e ca me ra co o rd i n a t e syst e m, f ro m w h i ch t h e p o i n t ca n t h e n b e t ra n sf o rm e d i n t o w o rl d c o o rd i n a t e s. T h e c o o rd i n a t e syst e ms a ssi g n e d t o t h e se t u p a re d e scri b e d a t t h i s p o i n t i n F i g u re 2 4 b e l o w T h i s a ssi g n me n t i s co n si st e n t t h ro u g h o u t t h i s p r o j e ct a n d sh o u l d b e re f e rr e d t o w h e n i n d o u b t o f a n e q u a t i o n a t a n y p o i n t T h e co o rd i n a t e syst e ms sh o u l d a l so b e a ss i g n e d a s su ch t o a n y r o b o t i c m a n i p u l a t o r u se d w i t h t h i s p r o j e ct s p ro g ra m. W h i l e i t se e ms b a s i c t o a ssi g n c o o r d i n a t e syst e ms, i t a ct u a l l y i s e sse n t i a l t o a ssi g n t h e m i n t h e co rre ct ma n n e r, a s t h i s ca n ma ke o r b re a k t h e u se o f st e re o vi si o n T h e i n t ri n si c ca me r a p a ra me t e rs a r e t h e f o c a l l e n g t h t h e sca l e f a ct o rs, t h e ce n t e r co o rd i n a t e s o f t h e i m a g e a n d t h e d i s t o rt i o n co e f f i ci e n t T h e sc a l e f a ct o rs g i ve i n f o rma t i o n o n t h e d i st a n c e o f a p i xe l o n t h e C C D c h i p i n t h e x a n d y d i re ct i o n T h e ce n t e r co o r d i n a t e s o f t h e i ma g e a re t h e p i xe l co o r d i n a t e s o f t h e ce n t e r o f p ro j e ct i o n w h i ch d o e s n o t n e ce ssa ri l y h a v e t o b e t h e t h e o re t i c a l c e n t e r l o ca t i o n w h i l e t h e d i st o rt i o n co e f f i ci e n t i s u se d t o co rre ct t h e i m a g e f ro m w a r p i n g w h i ch i s a n u n p l e a s a n t si d e e f f e ct o f o p t i cs, i n o rd e r t o o b t a i n a cc u ra t e d a t a F i n a l l y, t h e i m a g e h e i g h t a n d w i d t h a r e a l so u s e f u l t o kn o w t h o u g h n o t e sse n t i a l

PAGE 26

1 5 F i gu r e 2 4 : C oor d i n a t e S ys t e m A s s i gn m e n t T h e i n t e rn a l ca m e ra ca l i b r a t i o n i s p e r f o rme d u si n g a c a l i b ra t i o n t a b l e sh o w n i n F i g u re 2 4 w i t h kn o w n c o o rd i n a t e s I t ca n b e cr e a t e d u si n g t h e u se r i n t e rf a ce cre a t e d a s p a rt o f t h i s p ro j e ct w h i ch i s f u rt h e r d i scu sse d i n ch a p t e r 5

PAGE 27

1 6 F i gu r e 2 5 : C al i b r a t i on T ab l e T h e ca l i b r a t i o n t a b l e c o n si st s o f 4 9 ci r cl e s se p a r a t e d b y e q u a l d i st a n ce s. T h e ra d i u s o f e a ch ci rc l e i s co n st a n t a n d k n o w n T h e b o rd e r d i m e n si o n s o f t h e t a b l e a re a l s o kn o w n Si n ce a l l o f t h e se p a ra me t e rs a r e kn o w n w h i l e t h e d i st a n ce t o t h e ca l i b r a t i o n t a b l e i s u n kn o w n s e v e ra l i ma g e s o f t h e ca l i b ra t i o n t a b l e h a ve t o b e t a ke n b y e a ch ca me r a i n o rd e r t o p ro p e rl y a n d a ccu r a t e l y e st i ma t e t h e i n t ri n si c p a ra m e t e rs o f t h e ca me ra s u s e d T h i s i s ca l l e d a mu l t i i m a g e ca l i b r a t i o n p ro ce ss. F o r t h i s m u l t i i m a g e ca l i b r a t i o n t h e kn o w n 3 D p o i n t s o f t h e ci rc l e s a n d t h e b o rd e r o f t h e ca l i b ra t i o n t a b l e a re p ro j e ct e d i n t o t h e i ma g e p l a n e T h e su m o f t h e sq u a re d d i st a n c e s b e t w e e n t h e p ro j e ct i o n s a n d t h e co rre s p o n d i n g i ma g e p o i n t s (x i m y i m f ) i s mi n i mi ze d u n t i l t h e mi n i mi z a t i o n c o n v e rg e s. T h e e q u a t i o n s d e scri b e d i n t h e p re v i o u s s e ct i o n a r e u se d f o r t h i s p ro ce d u re F ro m t h i s, t h e i n t ri n si c ca m e ra p a ra me t e rs ca n b e o b t a i n e d F o r t h e i n t ri n s i c p a r a me t e r ca l i b r a t i o n 1 3 i ma g e s a t g re a t l y va ryi n g l o ca t i o n s w e re t a k e n I t i s i m p o rt a n t t o e n su re t h a t t h e i ma g e s o f t h e c a l i b ra t i o n t a b l e a re sh o w n a t d i f f e re n t a n g l e s a n d si z e s, so t h a t a l l o f t h e i n t ri n si c p a r a me t e rs c a n b e a sse sse d w i t h su f f i ci e n t a cc u ra cy. Al l f e a t u re s o f t h e ca l i b ra t i o n t a b l e h a ve t o b e

PAGE 28

1 7 vi si b l e i n cl u d i n g t h e b o rd e r. A d d i t i o n a l l y, i n i t i a l v a l u e s o f t h e i n t ri n si c p a r a me t e rs sh o u l d b e kn o w n Af t e r p e rf o rmi n g t h e i n t e rn a l ca m e ra ca l i b r a t i o n u si n g t h e su b ro u t i n e o f t h e u se r i n t e rf a ce d e s cri b e d i n ch a p t e r 5 a n d ru n n i n g i t e r a t i o n s u si n g t h e ca l cu l a t e d va l u e s f o r t h e i n t ri n si c p a r a me t e rs, t h e f i n a l i n t ri n si c ca me ra p a ra me t e rs a re su mma r i ze d i n T a b l e 2 1 f o r t h e I ma g i n g So u rce C a me r a a n d T a b l e 2 2 f o r t h e H i t a ch i ca m e ra b e l o w T ab l e 2 1 : I n t r i n s i c C am e r a P ar a m e t e r s f or t h e I m agi n g S o u r c e C am e r a Pa ra me t e r Va l u e U n i t F o ca l L e n g t h o f L e n s 0 0 0 8 7 2 3 6 7 [ m ] W i d t h o f C C D ch i p 1 0 2 4 9 1 0 5 [ m] H e i g h t o f C C D ch i p 1 1 1 0 5 [ m] I ma g e C e n t e r (x) 3 1 1 8 2 8 [ p i xe l ] I ma g e C e n t e r (y) 2 5 3 9 9 8 [ p i xe l ] R a d i a l D i st o rt i o n C o e f f i ci e n t 4 4 3 6 4 3 [ m 2 ] I ma g e W i d t h 6 4 0 [ p i xe l ] I ma g e H e i g h t 4 8 0 [ p i xe l ] T ab l e 2 2 : I n t r i n s i c C am e r a P ar a m e t e r s f or t h e H i t ac h i C am e r a Pa ra me t e r Va l u e U n i t F o ca l L e n g t h o f L e n s 0 0 0 7 6 4 4 3 [ m] W i d t h o f C C D ch i p 1 0 7 0 0 2 1 0 5 [ m] H e i g h t o f C C D ch i p 1 1 1 0 5 [ m] I ma g e C e n t e r (x) 3 1 7 9 1 5 [ p i xe l ] I ma g e C e n t e r (y) 2 2 4 4 9 4 [ p i xe l ] R a d i a l D i st o rt i o n C o e f f i ci e n t 4 3 3 9 8 6 [ m 2 ] I ma g e W i d t h 6 4 0 [ p i xe l ] I ma g e H e i g h t 4 8 0 [ p i xe l ]

PAGE 29

1 8 2 6 Ex tr i n s i c C a m e r a Pa r a m e te r s I n o rd e r t o su cc e ssf u l l y o b t a i n t h re e d i me n s i o n a l c o o r d i n a t e s f r o m st e re o vi si o n t h e p o si t i o n o f t h e ca m e ra s w i t h re s p e ct t o e a ch o t h e r, a s w e l l a s w i t h re sp e ct t o t h e ma n i p u l a t o r mu st b e kn o w n a cc u ra t e l y. I n o rd e r t o o b t a i n t h e se va l u e s, a s o ca l l e d h a n d e y e ca l i b ra t i o n mu st b e p e rf o rm e d T h e g o a l o f a h a n d e ye c a l i b ra t i o n i s t o a ccu r a t e l y o b t a i n t h e ro t a t i o n a n d t ra n sl a t i o n m a t ri x o f t h e ca me ra w i t h re sp e ct t o t h e ma n i p u l a t o r. An a l o g o u sl y t o t h e i n t ri n si c c a me ra c a l i b r a t i o n t h e s a me ca l i b r a t i o n t a b l e i s u se d f o r si m p l i c i t y. Ag a i n t h i rt e e n i m a g e s w e re o b t a i n e d f o r e a ch ca m e ra T h i s t i me h o w e v e r, i t i s e sse n t i a l t o a ccu r a t e l y kn o w t h e p o si t i o n o f t h e m a n i p u l a t o r w i t h re s p e ct t o t h e b a se c o o rd i n a t e s yst e m f o r e a c h i m a g e o b t a i n e d T h i s ca n e a si l y b e d o n e w h e n u si n g a ro b o t i c ma n i p u l a t o r w h o se j o i n t a n g l e s a r e k n o w n T h e p o si t i o n o f t h e c a l i b ra t i o n t a b l e s h o u l d n o t b e ch a n g e d d u r i n g t h i s p ro ce ss, a s i t i s a l so e sse n t i a l t o kn o w i t s p o si t i o n w i t h re s p e ct t o t h e b a se co o r d i n a t e syst e m. T h e h a n d e y e ca l i b r a t i o n ca n b e p e rf o rm e d b y u si n g t h e su b ro u t i n e b u i l t i n t o t h e u s e r i n t e rf a ce p ro g r a m, d i scu ss e d i n f u rt h e r d e t a i l i n c h a p t e r 5 Ad d i t i o n a l l y, t h e ro t a t i o n a n d l o ca t i o n o f t h e ca l i b ra t i o n t a b l e w i t h re sp e ct t o t h e ca me ra a re u s e f u l a s i n i t i a l st a rt i n g g u e sse s. As p a rt o f t h e i n t ri n si c c a me ra ca l i b r a t i o n t h e p o se n a me l y t h e ro t a t i o n a n d t ra n s l a t i o n ve ct o r f ro m t h e c a me ra t o t h e ca l i b r a t i o n t a b l e a r e me a su r e d a s so rt o f a b y p ro d u ct o f t h e ca l i b r a t i o n p ro ce d u re T h i s i s p o ss i b l e si n ce t h e d i me n si o n s a n d l o ca t i o n o f t h e c i rcl e s i s kn o w n G i ve n a ce rt a i n a re a o f a n o b j e ct t h e r e i s a r e l a t i o n b e t w e e n t h e a ct u a l

PAGE 30

1 9 a re a a n d t h e p r o j e ct e d a r e a g i v e n i n sq u a r e p i x e l s. T h e s e va l u e s a re b u i l t i n t o t h e e xt ri n si c ca m e ra ca l i b ra t i o n su b r o u t i n e As a n i n i t i a l g u e ss f o r t h e H i t a ch i ca me ra t h e r o t a t i o n a n d t ra n s l a t i o n f ro m ma n i p u l a t o r t o ca me ra w e r e d e t e rmi n e d a s: (2 8 ) w h i ch co rr e sp o n d s t o a ro t a t i o n o f 3 5 o a ro u n d t h e y a xi s. m (2 9 ) w h e re R d e n o t e s t h e r o t a t i o n m a t ri x a n d T d e n o t e s t h e t ra n sl a t i o n ve ct o r f ro m ma n i p u l a t o r t o ca me ra c o o rd i n a t e sys t e m. Si mi l a rl y, f o r t h e I ma g i n g So u rc e ca me ra t h e i n i t i a l g u e sse s f o r ro t a t i o n a n d t ra n sl a t i o n a r e : (2 1 0 ) w h i ch co rr e sp o n d s t o a ro t a t i o n o f 3 2 5 o a ro u n d t h e y a xi s. m (2 1 1 ) Af t e r p e rf o rmi n g t h e h a n d e y e c a l i b r a t i o n t h e f o l l o w i n g r o t a t i o n a n d t ra n sl a t i o n ma t ri ce s w e re o b t a i n e d f o r t h e H i t a ch i c a me ra : (2 1 2 )

PAGE 31

2 0 w h i ch co rre s p o n d s t o a ro t a t i o n o f 5 5 4 o 3 1 7 6 o a n d 3 5 1 9 8 o a ro u n d t h e x y a n d z a xi s, re s p e ct i ve l y m (2 1 3 ) a n d f o r t h e I ma g i n g So u rc e ca me r a t h e f o l l o w i n g w a s o b t a i n e d : (2 1 4 ) w h i ch c o rre sp o n d s t o a ro t a t i o n o f t h e ca me ra o f 3 5 1 8 5 o 3 2 5 0 9 7 o a n d 3 5 9 7 1 o a ro u n d t h e x y a n d z a xi s, re sp e ct i v e l y. m (2 1 5 ) As a l re a d y m e n t i o n e d t h e se v a l u e s a re v e ry i m p o rt a n t i n a ccu ra t e l y d e t e rmi n i n g t h e t h re e d i m e n si o n a l p o si t i o n o f t h e o b j e ct o f i n t e re st i n t h e ma n i p u l a t o r co o r d i n a t e syst e m a n d a n y sma l l ch a n g e i n e i t h e r t h e p o si t i o n o r t h e t ra n sl a t i o n o f t h e ca me ra w i t h re sp e c t t o t h e e n d e f f e ct o r w i l l b e d e t ri me n t a l t o t h e a cc u ra cy o f t h e st e re o vi s i o n re c o n st ru ct i o n p ro c e ss. T a b l e 2 3 b e l o w g i ve s a su mma ry o f t h e e xt ri n si c c a me r a p a ra me t e rs. I t sh o w s b o t h t h e i n i t i a l g u e sse s, a s w e l l a s t h e co n ve r g e d va l u e s.

PAGE 32

2 1 T ab l e 2 3 : E x t r i n s i c C am e r a P ar a m e t e r s H i t a ch i I n i t i a l H i t a ch i F i n a l I ma g i n g S o u rce I n i t i a l I ma g i n g S o u rce F i n a l X 0 1 3 m 0 1 6 9 m 0 1 3 m 0 1 2 6 m Y 0 0 1 m 0 0 4 4 m 0 0 1 m 0 0 7 5 m Z 0 0 4 6 m 0 0 1 9 m 0 0 5 m 0 0 5 7 m R o t (X) 0 o 5 5 4 o 0 o 3 5 1 8 5 o R o t (Y) 3 5 o 3 1 7 6 o 3 2 5 o 3 2 5 1 o R o t (Z ) 0 o 3 5 1 9 8 o 0 o 3 5 9 7 1 o

PAGE 33

2 2 C h a p te r 3 Sh a p e Mo d e l s 3 1 I n tr o d u c ti o n T h e p u r p o s e o f t h i s ch a p t e r i s t o d e s cri b e t h e vi s i o n a ssi st f u n ct i o n s cre a t e d f o r u se i n t h i s t h e s i s. Si n ce a l a r g e n u mb e r o f t h e se s o ca l l e d sh a p e m o d e l s w a s cre a t e d t h i s ch a p t e r o n l y d e scri b e s t h e cre a t i o n o f o n e o f t h e s e mo d e l s i n d e t a i l a s o t h e rw i se t h i s ch a p t e r w o u l d e xc e e d a n a cce p t a b l e a mo u n t o f sp a ce T h e o t h e r sh a p e mo d e l s cre a t e d h o w e ve r a re su mma ri ze d a t t h e e n d o f t h e ch a p t e r w i t h t h e ma i n sp e ci f i c a t i o n s re q u i r e d t o re p ro d u ce t h e sh a p e mo d e l e f f e ct i ve l y. A sh a p e mo d e l i s a f i l e w h i ch co n t a i n s t h e i n f o rma t i o n re q u i r e d b y t h e co n t ro l syst e m t o l o ca t e t h e o b j e ct o f i n t e re s t T h e s h a p e mo d e l a l so p ro vi d e s t h e u se r w i t h i m p o rt a n t i n f o rma t i o n o n t h e l o ca t i o n a n d o ri e n t a t i o n o f t h e o b j e ct i n t h e f i e l d o f vi si o n i n f o rm o f o u t p u t p a r a me t e rs. Al l o f t h i s i n f o rma t i o n i s d e scri b e d i n g re a t d e t a i l w i t h i n t h i s ch a p t e r. 3 2 B a c k g r o u n d o f Vi s i o n T h e o re t i ca l l y, a vi s i o n a l g o ri t h m ca n b e cre a t e d f o r a n y o b j e ct f o u n d i n t h i s w o rl d H o w e v e r, t h i s w o u l d p ro v e i mp ra ct i ca l a s t h e a mo u n t o f me m o ry re q u i re d f o r t h i s t a sk w o u l d e xc e e d a n y co m p u t i n g p o w e r kn o w n t o d a t e T h e p u r p o se o f t h e vi si o n a l g o ri t h ms i s t o p ro v i d e t h e u se r w i t h a me a n s o f l o ca t i n g a d e si re d

PAGE 34

2 3 o b j e ct i n t h e f i e l d o f v i e w Si n c e t h e re a r e p ra ct i ca l l y n o t w o o b j e ct s t h a t a re e xa ct l y a l i k e i n t h i s w o r l d t h e vi si o n a l g o ri t h m s h o u l d b e f l e xi b l e e n o u g h t o l o ca t e t h e o b j e ct o f i n t e re st e ve n i f i t d o e s n o t ma t ch t h e o b j e ct t h a t w a s u se d t o cre a t e t h e vi si o n sh a p e mo d e l i n e v e ry d e t a i l I f o n e l o o ks a t a ro u n d d o o rkn o b f o r e xa mp l e o n e w i l l n o t i ce t h a t t h e re mi g h t b e si g n s o f w e a r a n d t e a r f ro m t o u ch i n g t h e k n o b T h e d o o rkn o b mi g h t a l s o b e d i sco l o re d d u e t o d i rt o r t h e p a i n t b e i n g ch i p p e d o f f t h e s u rf a ce H o w e ve r, w h e n a h u m a n b e i n g l o o ks a t t h e d o o r, t h a t p e rso n st i l l r e co g n i z e s t h e d o o rk n o b a s a d o o rkn o b T h i s i s b e c a u se h u m a n s r e me mb e r a n d re co g n i z e o b j e ct s b a se d o n ma n y d i f f e re n t f a ct o rs. T h e se f a ct o rs i n cl u d e b u t a re n o t l i m i t e d t o t h e f o l l o w i n g : 1 Sh a p e o f a n o b j e ct 2 Si ze o f a n o b j e ct 3 C o l o r o f a n o b j e ct 4 L o ca t i o n a n d ro t a t i o n o f a n o b j e ct w i t h i n t h e f i e l d o f vi si o n 5 O t h e r d i st i n g u i sh i n g f e a t u r e s So me re m a rks sh o u l d b e ma d e t o t h e f o u r b a s i c f a ct o rs t h a t co n t ri b u t e t o t h e d e t e ct i o n o f o b j e ct s. T h e s h a p e o f a n o b j e ct i s t h e mo st b a si c w a y h u ma n s co mp a re w h a t t h e y s e e w i t h t h e i r m e mo ry a n d u se d t o d e t e rmi n e t h e f u n ct i o n o f t h e o b j e ct A p e rso n w h o se e s a ro u n d d o o rk n o b f o r e x a mp l e kn o w s i mme d i a t e l y t h a t t h i s o b j e ct i s u s e d t o o p e n a d o o r. T h e a ct i o n r e q u i re d a f t e r g ra sp i n g t h e o b j e ct i s t o t u rn t h e k n o b i n a ce rt a i n d i re ct i o n f o l l o w e d b y a p u l l i n g o r p u sh i n g a ct i o n i n o r d e r t o b e a b l e t o o p e n t h e d o o r.

PAGE 35

2 4 T h e si ze o f a n o b j e ct a l so h e l p s d i st i n g u i sh i t f ro m o t h e r i t e ms t h a t mi g h t b e i n t h e f i e l d o f vi si o n I f a n o b j e ct i s l a r g e r t h a n a n o t h e r o b j e ct l o ca t e d n e xt t o t h e f i rst t h e p e rso n o b se rvi n g t h i s kn o w s t h a t t h e l a rg e r o b j e ct w i l l mo st l i k e l y b e h e a vi e r t h a n t h e sma l l e r o b j e ct H e n ce w h e n ma n i p u l a t i n g t h i s sp e ci f i c i t e m, t h e p e rso n w i l l kn o w t o e mp l o y mo re f o rce t o su cce ssf u l l y co mp l e t e t h e d e s i re d t a sk. W h i l e t h e sh a p e a n d s i ze o f a n o b j e c t a re co n si d e re d p r i ma ry d i st i n g u i s h i n g f e a t u re s, t h e co l o r o f a n o b j e ct i s co n s i d e re d se co n d a ry. A p l a i n r o u n d d o o r kn o b f o r e xa mp l e ca n b e f o u n d i n ma n y d i f f e re n t co l o rs ra n g i n g f ro m a ma t t b r o n ze f i n i sh t o a g l o ssy g o l d e n f i n i sh Si n c e co l o r d o e s n o t a f f e ct t h e p u rp o se o f t h e o b j e ct o r d e si re d a ct i o n i t i s n o t u se d f o r t h e cre a t i o n o f vi si o n sh a p e mo d e l s. T h e l o c a t i o n o f a n o b j e ct i n t h e f i e l d o f vi si o n i s a n i m p o rt a n t f a ct o r u s e d b y h u ma n s. I n o rd e r t o b e a b l e t o ma n i p u l a t e a n y o b j e ct o n e h a s t o kn o w w h e re t h e o b j e ct i s l o c a t e d w i t h re s p e ct t o t h e h a n d T h i s p ro b l e m o f h a n d e ye ca l i b r a t i o n w h i ch i s re q u i r e d f o r t h e vi si o n sh a p e mo d e l s t o b e f u n ct i o n a l i s d i scu sse d a t t h e e n d o f t h e c h a p t e r i n d e t a i l T h e v i si o n a l g o ri t h ms a re re q u i re d t o l o ca t e t h e o b j e ct w i t h i n t h e f i e l d o f vi si o n a n d re l a t e t h i s p o si t i o n t o t h e a ct u a l p o si t i o n w i t h re s p e ct t o t h e h a n d w h i ch i s t h e g ri p p e r m o u n t e d a t t h e e n d o f t h e e n d e f f e ct o r. T h e l a st f a ct o r l i st e d a b o ve e n c o mp a sse s a w i d e va ri e t y o f f e a t u re s. T h e o ri e n t a t i o n o f a n o b j e ct m i g h t b e o f i mp o rt a n ce w h e n t ryi n g t o d i st i n g u i sh a n o b j e ct I f o n e l o o ks a t a ro u n d d o o rk n o b f o r e xa mp l e i t s a p p e a ra n ce c h a n g e s w h e t h e r i t i s vi e w e d f ro m t h e s i d e o r f ro m u p f ro n t A d o o rkn o b mi g h t h a ve a

PAGE 36

2 5 ke yh o l e o r i t mi g h t n o t T h i s f e a t u re ca n ch a n g e t h e a ct i o n t h e p e rs o n i s re q u i r e d t o d o Sh o u l d t h e r e b e a ke yh o l e i t i s ve ry l i ke l y t h a t a ke y i s re q u i re d t o b e a b l e t o t u rn t h e d o o rk n o b T h e a b se n ce o f a ke yh o l e i n d i c a t e s t h a t t h e kn o b c a n b e t u rn e d a t a n y t i me T h e s e se co n d a ry d i st i n g u i sh i n g f e a t u r e s a r e n o t co n si d e re d f o r t h e p u r p o se s o f t h i s re se a rch w o rk, a s t h e g ri p p e r mo u n t e d a t t h e e n d o f t h e e n d e f f e ct o r d o e s n o t h a ve t h e ca p a b i l i t y t o u se t h e i n f o rma t i o n co l l e ct e d t o p e rf o rm t h e re q u i r e d f u n ct i o n Al so t h e l i b ra ry o f vi si o n a l g o r i t h ms w o u l d i n cre a se d r a ma t i ca l l y i n si z e 3 3 C r e a ti o n o f a Sh a p e Mo d e l T h e g o a l o f a s h a p e mo d e l i s t o p r o vi d e t h e st e re o vi si o n w i t h a m e a n s o f a u t o ma t i c d e t e ct i o n o f t h e o b j e ct o f i n t e re st a s w e l l a s o b t a i n i n g t h e l o ca t i o n o f t h e o b j e ct i n t h e f i e l d o f vi si o n Ad d i t i o n a l l y, t h e sh a p e mo d e l i s re q u i r e d t o h a ve e n o u g h d i st i n g u i sh i n g f e a t u re s s u ch a s t o mi n i mi ze t h e o cc u rre n c e o f e rro n e o u s o b j e ct d e t e ct i o n s, w h i l e st i l l ke e p i n g t h e sh a p e mo d e l g e n e ra l e n o u g h t o b e a b l e t o d e t e ct si mi l a r o b j e ct s t h a t o n l y p o ss e ss mi n o r d i f f e re n c e s. Al l o f t h i s h a s t o b e t a ke n i n t o a cco u n t d u ri n g t h e cr e a t i o n o f a sh a p e mo d e l a n d t h e d e t a i l s a re n o w d i scu sse d 3 3 1 T h e O r i g i n a l I m a g e o f th e O b j e c t o f I n te r e s t I n o rd e r t o cr e a t e t h e v i si o n sh a p e mo d e l s, t h e su b r o u t i n e Sh a p e M o d e l C re a t i o n p a rt o f t h e U se r I n t e rf a ce w a s e mp l o y e d T h e mo st i mp o rt a n t

PAGE 37

2 6 f u n ct i o n s a n d va ri a b l e s re q u i r e d f o r t h e cre a t i o n o f a s h a p e mo d e l a re d e scr i b e d T h e f i rst o b j e ct i ve o f t h e u s e r i s t o o b t a i n a n i m a g e o f t h e o b j e ct f o r w h i c h t h e sh a p e m o d e l i s t o b e cre a t e d I d e a l l y, t h i s i ma g e sh o u l d b e a cq u i re d w i t h t h e sa me ca me ra t h a t w i l l b e u s e d l a t e r o n i n o rd e r t o a vo i d a n yt h i n g t h a t mi g h t a l t e r t h e i ma g e t a ke n T h i n g s t h a t ca n a f f e ct t h e a cq u i si t i o n o f a n i m a g e c a n i n cl u d e su ch t h i n g s a s t h e f o cu s o r l i g h t i n g o f t h e ca me ra a s w e l l a s a n y b u i l t i n d i st o rt i o n s d u e t o ma n u f a ct u ri n g e rro r s. W h i l e t h e s e f a ct o rs a re n o t n e c e s sa ri l y l i f e t h re a t e n i n g t o t h e i m a g e o r t h e c re a t i o n o f t h e s h a p e mo d e l a n i ma g e w i t h e xce l l e n t re s o l u t i o n a n d d e t a i l s h o u l d b e ca p t u re d su ch t h a t t h e d i st i n g u i sh i n g f e a t u re s re q u i re d f o r t h e sh a p e mo d e l a re sh o w n O n e o f t h e o ri g i n a l i ma g e s u se d ca n b e se e n i n F i g u re 3 1 b e l o w T h e sh a p e mo d e l t o b e cre a t e d i s f o r t h e o b j e ct w i t h t h e cro ss. T h e d e si r e d o u t p u t co o r d i n a t e s f o r t h i s o b j e ct i s t h e c e n t e r o f t h e ci rcl e F i gu r e 3 1 : O r i gi n a l I m age o f C r os s O b j e c t

PAGE 38

2 7 3 3 2 R e g i o n o f I n te r e s t I n o rd e r t o cre a t e t h e sh a p e mo d e l f o r t h i s o b j e ct o r a n y o t h e r o b j e ct a re g i o n o f i n t e r e st i n cl u d i n g t h e o b j e ct o f i n t e re st mu st b e cre a t e d T h i s i s d o n e so t h a t t h e o b j e ct f o r w h i ch t h e sh a p e mo d e l i s t o b e cre a t e d ca n b e i d e n t i f i e d mu ch mo r e e a si l y. H e n ce t h e f i r st st e p a f t e r a c q u i r i n g t h e i ma g e i s t o d e t e rmi n e a su i t a b l e a r e a o f i n t e r e st I f o n e w e re t o o ve rl a y t h e re g i o n o f i n t e re st o ve r t h e o ri g i n a l i ma g e i t w o u l d l o o k l i ke F i g u re 3 2 b e l o w F i gu r e 3 2 : R e gi on of I n t e r e s t I n t h e ca se o f F i g u r e 3 1 i t w a s d e ci d e d t h a t a ci rcu l a r re g i o n a ro u n d t h e a p p ro x i ma t e ce n t e r co o rd i n a t e s b e cre a t e d I n o rd e r t o d o t h i s, t h e c e n t e r

PAGE 39

2 8 co o rd i n a t e s a l o n g w i t h t h e d e s i re d ra d i u s i n t e rms o f p i x e l s mu st b e d e t e r mi n e d T h i s i s d o n e w i t h a n y a v a i l a b l e p h o t o so f t w a re s u ch a s Ph o t o sh o p T h e p i xe l co o rd i n a t e s o f t h e ce n t e r a r e t h e n re c o rd e d i n t e rms o f ro w a n d co l u mn a s w e l l a s t h e n u mb e r o f p i xe l s re q u i r e d f o r t h e ra d i u s. As a n a l t e rn a t i v e d e p e n d i n g o n t h e o b j e ct t h e re g i o n o f i n t e re st ca n a l so b e a re ct a n g l e o r a n y o t h e r g e o me t ri c sh a p e T h e se d e t a i l s a re l i st e d i n t h e e n d o f t h e ch a p t e r su mm a ri zi n g t h e cre a t i o n o f t h e va r i o u s sh a p e mo d e l s. 3 3 3 C o n tr a s t T h r e s h o l d a n d Py r a m i d s T h e e a si e st w a y t o d e t e rmi n e t h e c h a ra ct e ri st i cs o f a n o b j e ct i s t o u se a ce rt a i n co n t ra st t h re sh o l d va l u e By c h o o si n g j u st t h e r i g h t va l u e o f co n t ra st i t i s p o ssi b l e t o e l i m i n a t e a n y u n d e s i re d f e a t u re s, su c h a s re f l e ct i o n s o r d i rt w h i l e u si n g o n l y t h e mo st i mp o rt a n t d i st i n g u i sh i n g f e a t u re s t o cr e a t e t h e s h a p e m o d e l T h i s w a y, i t i s e n s u re d t h a t t h e o b j e c t o f i n t e re st w i l l b e d e t e ct e d i n t h e f u t u re e ve n t h o u g h t h e l i g h t i n g co n d i t i o n s mi g h t b e a l t e re d o r e ve n i f p a rt o f t h e o b j e ct i s o ccl u d e d T h e e f f e ct o f ch o o s i n g a d i f f e re n t n u mb e r f o r t h e co n t ra st ca n b e se e n i n F i g u re 3 3 T h i s f i g u re s h o w s a co n t ra st o f 1 0 8 0 a n d 1 0 0 f ro m t o p t o b o t t o m. I f t h e co n t ra st ch o se n i s t o o l o w o n e ca n se e t h a t t o o ma n y p o i n t s a re i n cl u d e d a s e ve n n a t u ra l d i sco l o ra t i o n s a re i n t e r p re t e d a s a d i st i n g u i sh i n g f e a t u re o f t h e o b j e ct o f i n t e r e st H o w e ve r, w h e n t h e co n t ra st c h o s e n i s t o o h i g h h a rd l y a n y p o i n t s a re co n s i d e re d t o b e o f a n y si g n i f i ca n c e H e n ce j u st t h e co n t ra st h a s t o b e ch o se n su ch t h a t t h e mo st i mp o rt a n t f e a t u re s a re

PAGE 40

2 9 d i st i n g u i s h e d T h e va l u e s u s e d f o r t h e va ri o u s sh a p e mo d e l s a re l i st e d a t t h e e n d o f t h i s ch a p t e r, a l o n g w i t h o t h e r i mp o rt a n t i n f o rm a t i o n r e q u i re d t o re cre a t e t h e se sh a p e mo d e l s. F i gu r e 3 3 : T h r e s h ol d of 10, 80, an d 100, R e s p e c t i ve l y ( T op t o B ot t om )

PAGE 41

3 0 F i g u re 3 3 n o t o n l y sh o w s t h e e f f e ct o f ch o o si n g d i f f e re n t co n t ra st va l u e s, b u t a l so sh o w s t h e e f f e ct t h a t f o u r d i f f e re n t so ca l l e d p yra m i d s h a ve A p yra mi d i n t h i s ca se i s si m p l y a sca l e d d o w n m o d e l o f t h e o ri g i n a l i m a g e By u s i n g d i f f e re n t p yra mi d l e ve l s, t h e p ro ce ss o f l o ca t i n g t h e o b j e ct o f i n t e r e st i n t h e f u t u re i s si mp l i f i e d si g n i f i c a n t l y. T h e sh a p e mo d e l cr e a t e d n o t o n l y st o re s t h e mo st i mp o rt a n t f e a t u re s o f t h e o b j e ct f o r t h e o r i g i n a l s i ze u se d t o cr e a t e t h e s h a p e mo d e l b u t a l s o st o re s i n f o rma t i o n f o r va ri o u s sca l e d d o w n mo d e l s. T h e f i rst p yra mi d i s t h e o ri g i n a l si ze o f t h e o b j e ct w h i l e t h e se co n d p yra m i d l e v e l i s o n e h a l f t h e t h i rd o n e t h i rd a n d t h e f o u rt h o n e f o u rt h o f t h e o ri g i n a l i ma g e T h e o re t i ca l l y, i t i s p o ssi b l e t o st o r e i n f o rma t i o n o n t e n d i f f e re n t p yr a mi d l e ve l s, b u t t h i s i s i mp ra ct i c a l a s i t t a ke s u p t o o mu c h me mo ry a n d t h e a n yt h i n g b e yo n d t h e f o u rt h p yr a mi d l e v e l i s t o o sma l l a n yw a y. H e n ce f o u r p yra mi d l e v e l s a re u se d f o r t h e cre a t i o n o f a l l sh a p e mo d e l s. 3 3 4 O th e r Pa r a m e te r s R e q u i r e d fo r th e Sh a p e Mo d e l W h i l e t h e re g i o n o f i n t e r e st a n d t h e co n t ra st o f t h e d e si re d o b j e ct o f i n t e re st i s o f g re a t i m p o rt a n ce o t h e r p a ra me t e r s h a ve t o b e sp e c i f i e d f o r i n cl u s i o n i n t h e sh a p e mo d e l f i l e T h e i n cl u s i o n o f t h e se p a ra m e t e rs i s n o t n e c e ssa ry, b u t i t w i l l sp e e d u p t h e p ro c e ss o f l o ca t i n g t h e o b j e ct o f i n t e re st w i t h i n t h e f i e l d o f v i si o n I t i s n e a rl y i m p o ssi b l e t h a t t h e o b j e ct o f i n t e re st w i l l b e ro t a t e d e xa ct l y a s i t w a s i n t h e o ri g i n a l i m a g e u s e d t o cre a t e t h e sh a p e mo d e l Ad d i t i o n a l l y, i t i s e q u a l l y u n l i k e l y t h a t t h e o b j e ct w i l l h a ve t h e e x a ct sa me a p p a re n t d i m e n si o n s.

PAGE 42

3 1 3 4 D e te c ti o n o f Sh a p e Mo d e l s O n ce a sh a p e mo d e l h a s b e e n cre a t e d f o r a n o b j e ct o r a cl a ss o f o b j e ct s w i t h si mi l a r g e o me t ri c a l ch a r a ct e ri st i cs, t h e sh a p e mo d e l ca n b e ca l l e d f ro m t h e u se r i n t e rf a ce p r o g ra m t o l o ca t e sa i d o b j e ct i n t h e f i e l d o f vi si o n o f t h e st e re o c a m e ra s. W h e n t h e s h a p e m o d e l h a s su cce ssf u l l y b e e n ma t ch e d w i t h t h e o b j e ct o f i n t e re st t h e p i xe l co o r d i n a t e s o f t h e ce n t e r o f g ra vi t y a re d e t e rmi n e d a n d st o re d i n me m o ry. T h e se co o r d i n a t e s a r e t h e n u s e d t o c a l c u l a t e t h e t h re e d i m e n s i o n a l p o si t i o n o f t h e o b j e ct o f i n t e r e st w i t h re sp e ct t o t h e e n d e f f e ct o r, b y u si n g t h e re co n st ru ct i o n t e ch n i q u e d i scu ss e d i n ch a p t e r 4

PAGE 43

3 2 C h a p te r 4 T h r e e D i m e n s i o n a l R e c o n s tr u c ti o n 4 1 O v e r v i e w O n ce t h e o b j e ct o f i n t e re st h a s b e e n f o u n d t h e f i rst o f t h e t w o st e re o vi si o n p ro b l e ms i s so l v e d t h a t o f co rre l a t i o n T h e n e xt a n d f i n a l st e p i n t h re e d i me n s i o n a l o b j e ct l o c a t i o n i s t h re e d i me n si o n a l re co n st ru ct i o n 4 2 Ste r e o G e o m e tr y I n o rd e r t o i l l u st ra t e h o w t h e p o si t i o n o f a p o i n t i s d e t e rmi n e d f ro m t w o se t s o f p i xe l co o rd i n a t e s, i t i s e sse n t i a l t o e x p l o re t h e g e o m e t ry o f a st e re o vi si o n sy st e m. A si mp l i f i e d mo d e l o f t h e g e o me t ry o f a st e re o syst e m i s sh o w n i n F i g u r e 4 1 b e l o w a n d i t i s e sse n t i a l t o u n d e rst a n d t h i s g e o me t ry b e f o re e xp l o ri n g t h e s e t s o f e q u a t i o n s n e e d e d f o r t h re e d i me n si o n a l re co n st ru ct i o n o f t h e p o i n t l o c a t i o n F i gu r e 4 1 : S t e r e o V i s i on G e om e t r y

PAGE 44

3 3 Be f o re g o i n g a n y f u rt h e r, t h e n o m e n cl a t u re u se d t h ro u g h o u t t h i s t h e si s, a s w e l l a s F i g u re 4 1 h a s t o b e cl a r i f i e d T h e su b scri p t s l a n d r d e n o t e l e f t a n d ri g h t c a me ra r e sp e ct i v e l y. T h e p o i n t s d e n o t e d b y O a re t h e ce n t e r c o o rd i n a t e s o f t h e ca me ra T h e p o i n t P d e n o t e s t h e p o si t i o n o f t h e o b j e ct o f i n t e r e st o f w h i ch t h e t h re e d i me n s i o n a l re c o n st r u ct i o n i s d e si re d T h e ve ct o r p i s t h e u n i t ve ct o r o f P, w h i c h d e n o t e s t h e ve ct o r f ro m t h e ce n t e r co o rd i n a t e o f t h e ca me ra t o t h e p o i n t o f i n t e re st I f T i s d e f i n e d a s t h e t ra n sl a t i o n v e ct o r b e t w e e n O l a n d O r a n d R d e n o t e s a ro t a t i o n m a t ri x, i t ca n b e s h o w n t h a t g i ve n a p o i n t P i n sp a ce t h e re l a t i o n b e t w e e n P l a n d P r i s g i ve n b y e q u a t i o n (4 1 ) a s: (4 1 ) I t sh o u l d b e n o t e d a t t h i s p o i n t t h a t t h e t h e o ry f o r t h e vi s i o n syst e m i s s u p e rb l y d e scri b e d b y T ru cc o [ 9 ] F o r d e t a i l e d a n d e xt e n s i ve t h e o ry, t h i s re f e r e n c e s h o u l d b e co n s u l t e d 4 3 Ste r e o R e c o n s tr u c ti o n C a l c u l a ti o n s I n o rd e r t o d e t e rm i n e t h e p o si t i o n o f t h e p o i n t P, i t i s u se f u l t o d o t h i s b y u s i n g o n e o f t h e ca m e ra s co o r d i n a t e syst e ms a s a r e f e re n c e co o rd i n a t e syst e m. T h e l e f t e ye ca m e ra n a m e l y t h e I m a g i n g So u rce ca m e ra i s ch o se n t o b e t h e re f e re n ce f ra me H e n c e t h e ri g h t e y e o r H i t a ch i c a me r a i s re l a t e d t o t h e l e f t e ye b y: (4 2 )

PAGE 45

3 4 w h e re t h e su b scri p t s R a n d L d e n o t e t h e ri g h t Eye i n t h i s ca se t h e H i t a ch i ca me ra a n d t h e l e f t e y e o r t h e I ma g i n g So u rce ca me r a re sp e ct i ve l y. O n c e t h i s i s d o n e t h e t ra n sl a t i o n v e ct o r b e t w e e n t h e t w o ca me r a s n e e d s t o b e ca l c u l a t e d Ap p l yi n g t h e f o l l o w i n g re l a t i o n sh i p d o e s t h i s: (4 3 ) T h e n e xt st e p i n r e co n st ru ct i o n i s t o d e t e rmi n e t h e u n i t ve ct o rs p L a n d p R b y a p p l y i n g f o r e a ch ca me ra t h e re s p e ct i ve va l u e s o b t a i n e d f ro m t h e i n t ri n si c ca me ra p a ra me t e r ca l i b r a t i o n : (4 4 ) w h e re x L a n d y L a r e t h e p i xe l co o rd i n a t e s o b t a i n e d f ro m t h e s h a p e mo d e l d e t e ct i o n p r o ce ss a n d a i s a d u m my va ri a b l e w h i ch c a n e i t h e r b e r o r l , d e p e n d i n g o n w h e t h e r e q u a t i o n (4 4 ) i s u se d f o r t h e ri g h t o r t h e l e f t e ye re sp e ct i ve l y. T h e se va l u e s r e p re s e n t t h e p i xe l c o o r d i n a t e s o f t h e ce n t e r o f g ra vi t y o f t h e o b j e ct o f i n t e re st T h e co n st a n t f L i s t h e f o ca l l e n g t h o b t a i n e d f ro m t h e i n t ri n si c ca m e ra ca l i b r a t i o n M i n t a i s t h e i n t ri n si c ma t ri x i s d e f i n e d a s f o l l o w s: (4 5 )

PAGE 46

3 5 w h e re a l l va l u e s a re o b t a i n e d f ro m t h e i n t ri n si c ca me ra ca l i b r a t i o n T h e d e ri va t i o n o f e q u a t i o n ( 4 5 ) i s g i ve n i n ch a p t e r 2 Si mi l a rl y, e q u a t i o n s (4 4 ) a n d (4 5 ) ca n b e u s e d f o r t h e ri g h t ca m e ra t o o b t a i n t h e u n i t ve ct o r. Po i n t P i n F i g u re 4 1 l i e s a t t h e i n t e rs e ct i o n o f t h e t w o ra ys f ro m O l t h ro u g h p l a n d f ro m O r t h ro u g h p r T h e p o i n t O a i s kn o w n a n d d e f i n e d b y t h e i n t ri n si c ca me ra p a ra me t e rs o x a n d o y w i t h t h e z p o i st i o n e q u a l t o ze ro f o r e a ch ca m e ra T h e ra y l t h ro u g h O l a n d p l a n d t h e ra y r t h ro u g h O r a n d p r c a n b e d e f i n e d a s f o l l o w s b y e e q u a t i o n s (4 6 ) a n d (4 7 ), re sp e ct i ve l y. (4 6 ) (4 7 ) w h e re a a n d b a r e si mp l y c o n st a n t sc a l a rs, w h i c h a re t o b e d e t e rmi n e d F i n a l l y, d e f i n e a ve ct o r w a s a n o rt h o g o n a l ve ct o r t o b o t h r a n d l a s: (4 8 ) w h e re c i s a g a i n a sca l a r co n st a n t t o b e d e t e rmi n e d By su mmi n g t h e v e ct o rs e q u a t i o n s (4 6 ) t h ro u g h ( 4 8 ), e q u a t i o n ( 4 9 ) c a n b e so l ve d f o r t h e co n st a n t s a b a n d c, si n ce e q u a t i o n (4 9 ) i s a se t o f t h re e e q u a t i o n s a n d t h re e u n kn o w n n a me l y t h e sca l a rs. (4 9 ) O n ce t h e sca l a rs a re kn o w n i t i s e a sy t o d e t e rmi n e t h e a ct u a l p o si t i o n o f p o i n t P b y a p p l yi n g o n e l a st e q u a t i o n Po i n t P l i e s a t t h e i n t e rs e ct i o n o f r a ys r a n d l I n o rd e r t o o b t a i n t h e p o i n t i n t h e ma n i p u l a t o r co o r d i n a t e syst e m, t h e p o i n t

PAGE 47

3 6 d e f i n e d b y t h e ve ct o rs a p l a n d T + b R T p r mu st b e t ra n sf o rm e d T h i s l e a d s t o t h e f i n a l e q u a t i o n (4 1 0 ): (4 1 0 ) w h e re X, Y, a n d Z d e n o t e t h e co o r d i n a t e s f o r t h e p o i n t P i n t h e ma n i p u l a t o r co o rd i n a t e syst e m. T h i s e q u a t i o n w i l l t h e n g i v e t h e f i n a l a b so l u t e co o r d i n a t e s o f t h e p o i n t o f i n t e re st i n t e rms o f t h e l e f t e ye ca me ra syst e m. Al l t h a t n e e d s t o b e d o n e n o w i s t o t ra n sf o rm t h e p o si t i o n o f t h e p o i n t i n t o a n y d e si re d co o rd i n a t e syst e m, f o r f u rt h e r ma n i p u l a t i o n

PAGE 48

3 7 C h a p te r 5 U s e r I n te r fa c e 5 1 O v e r v i e w I n re h a b i l i t a t i o n ro b o t i cs, i t i s e sse n t i a l t o t a i l o r e v e ryt h i n g t o t h e p e rso n w i t h d i sa b i l i t i e s. As p e rso n s w i t h d i sa b i l i t i e s o f t e n d o n o t h a ve t h e o p t i o n s a s p e r s o n s w i t h o u t d i s a b i l i t i e s, t h e u se r i n t e rf a ce u se d f o r t h e co n t ro l o f t h e ro b o t i c ma n i p u l a t o r sh o u l d b e e a sy t o u s e a n d st ra i g h t f o rw a rd T h e r e f o re sma l l o b j e ct s w e re a vo i d e d i f p o ssi b l e a n d e ve ryt h i n g w a s ma d e t o b e u se d w i t h d e f a u l t s e t t i n g t h a t w e r e d e t e rmi n e d t o w o rk f o r mo st ca se s i n v o l vi n g c a l i b ra t i o n o r sh a p e mo d e l cre a t i o n 5 2 Ma i n U s e r I n te r fa c e F i g u re 5 1 b e l o w sh o w s t h e m a i n u s e r i n t e rf a ce T h e p i ct u r e o f t h e l e f t e ye a s w e l l a s t h a t o f t h e ri g h t e ye i s sh o w n si mu l t a n e o u sl y. Ba si c f e a t u re s su ch a s co n t i n u o u s g ra b b i n g o f i m a g e s, s a vi n g i n d i v i d u a l i ma g e s a s w e l l a s a s e q u e n ce o f i ma g e s i s i n co rp o ra t e d f o r a n y d e si r e d n o n s p e ci f i e d f u n ct i o n A d ro p d o w n me n u a l l o w s t h e u s e r t o se l e ct f ro m t h e sh a p e mo d e l s st o re d w i t h i n t h e p ro g ra m a n d w h e n d o n e so t h e a b so l u t e a s w e l l a s t h e p i xe l co o rd i n a t e s o f t h e o b j e ct o f i n t e re st i f p re se n t a re sh o w n t o t h e u se r a s l o n g a s t h e st o p b u t t o n i s n o t e mp l o y e d

PAGE 49

3 8 Mo re a d va n ce d f e a t u r e s, re q u i re d f o r t h e se t u p o f a st e re o vi s i o n syst e m, a re a cce sse d v i a t h e p ro g r a m s me n u T h e se i n c l u d e i n t ri n si c ca m e ra p a ra me t e r ca l i b r a t i o n h a n d e ye o r e xt ri n s i c p a ra me t e r ca l i b r a t i o n ca l i b r a t i o n t a b l e cre a t i o n a n d o f co u rse s h a p e mo d e l cre a t i o n T h e s e f e a t u re s w e re d e si g n e d a n d u se d t h r o u g h o u t t h i s p r o j e ct a n d o p t i m i ze d s u ch t h a t t h e u s e r sh o u l d b e a b l e t o p e rf o rm t h e se t a sks ra t h e r e a si l y. Sh o u l d t h e re b e t h e n e e d h o w e ve r, t h e u se r ca n ch o o se cu st o m p a r a me t e rs su ch t h a t t h e se f u n ct i o n s a r e o p t i mi ze d f o r t h e e n vi ro n me n t i n w h i c h t h e st e re o vi si o n syst e m i s t o b e u se d O n e c a se i n w h i ch t h i s mi g h t b e o f u se i s i f t h e st e re o syst e m i s t o b e e mp l o ye d i n a n e n vi ro n me n t i n w h i c h t h e re i s e i t h e r e xce ssi ve o r n o t e n o u g h l i g h t i n g a va i l a b l e F i gu r e 5 1 : M ai n G r ap h i c al U s e r I n t e r f ac e

PAGE 50

3 9 5 3 C a l i b r a ti o n T a b l e C r e a ti o n I n o rd e r t o p e rf o rm t h e i n t ri n si c a n d e xt ri n si c ca me r a p a r a me t e r ca l i b r a t i o n a ca l i b r a t i o n t a b l e i s n e e d e d F o r co n ve n i e n ce t h i s ca l i b ra t i o n t a b l e ca n b e cre a t e d u s i n g t h e C a l i b r a t i o n T a b l e s u b ro u t i n e w h i ch ca n b e a cce sse d f ro m t h e ma i n u se r i n t e rf a ce T h e w i n d o w t h a t i s o p e n e d i s sh o w n i n F i g u re 5 2 F i gu r e 5 2 : C al i b r a t i on T ab l e C r e at i on I n t e r f ac e

PAGE 51

4 0 T h e o n l y i n p u t t h e u se r n e e d s t o g i ve t h e su b ro u t i n e i s t h e o u t si d e d i me n si o n o f t h e ca l i b r a t i o n t a b l e i n me t e rs. T h i s va l u e c a n b e o f a n y va l u e a n d o n l y d e p e n d s o n t h e p ri n t e r a v a i l a b l e I t i s, h o w e ve r, r e co mme n d e d t h a t t h e ca l i b r a t i o n t a b l e b e l a r g e r t h a n 0 2 me t e rs a n d t h a t a g o o d p ri n t e r w i t h h i g h p re ci si o n b e u s e d a s t h e ca l i b r a t i o n ma rks a re e sse n t i a l i n o b t a i n i n g t h e i n t ri n si c a n d e xt ri n s i c ca m e ra p a ra m e t e rs. T h e ca l i b ra t i o n t a b l e sh o u l d a l so b e T h e ca l i b r a t i o n t a b l e i s sa ve d i n C a l i b r a t i o n T a b l e s u b d i re ct o ry o f t h e u s e r i n t e rf a ce p ro g ra m. I t s d i m e n s i o n s a n d l o ca t i o n o f t h e ca l i b ra t i o n ma rks a re a l s o st o r e d i n t h i s su b d i re ct o ry, w h i c h i s l a t e r a u t o m a t i ca l l y ca l l e d b y t h e i n t ri n si c a n d e xt ri n si c ca l i b r a t i o n p ro ce d u re 5 4 I n tr i n s i c C a m e r a Pa r a m e te r C a l i b r a ti o n Su b r o u ti n e As a l re a d y me n t i o n e d o b t a i n i n g a c cu r a t e va l u e s f o r t h e i n t ri n si c ca m e ra p a ra me t e rs i s e sse n t i a l t o su cce ssf u l l y u se st e re o vi si o n t o d e t e rmi n e t h e p o si t i o n o f a n o b j e ct T h i s su b ro u t i n e o n l y h a s t o b e u s e d w h e n e i t h e r a n e w ca me ra i s t o b e u s e d o r i f t h e l e n s o f t h e ca m e ra h a s b e e n ch a n g e d t h u s ch a n g i n g t h e f o ca l p ro p e rt i e s. U su a l l y, h o w e ve r, t h e i n t ri n s i c ca me ra ca l i b r a t i o n n e e d s t o b e d o n e o n l y o n c e f o r a st e re o syst e m. H o w e ve r, i t sh o u l d a l s o b e p e rf o rme d i f t h e a cc u ra cy o f t h e o b j e c t o f i n t e r e st s l o ca t i o n i s d i m i n i s h e d F i g u re 5 3 b e l o w s h o w s t h e w i n d o w t h a t i s o p e n e d w h e n t h i s s u b ro u t i n e i s a cc e sse d t h ro u g h t h e ma i n u s e r i n t e rf a c e

PAGE 52

4 1 Si n ce st a rt i n g va l u e s f o r t h e i n t ri n si c p a ra me t e r ca l i b ra t i o n a re n e e d e d a s e t o f t yp i ca l i n t ri n s i c p a ra me t e rs i s u s e d T h e se va l u e s r e p re s e n t t h e w i d e v a ri e t y o f ca me ra s a va i l a b l e o n t h e ma rke t a n d ca n b e f o u n d i n t h e a p p e n d i x. F i gu r e 5 3 : I n t r i n s i c P ar am e t e r C al i b r a t i on I n t e r f ac e

PAGE 53

4 2 T h e ca me r a ca l i b r a t i o n c a n b e p e rf o r me d e i t h e r f o r a n i n d i vi d u a l ca me r a o r f o r b o t h ca m e ra s a t t h e sa me t i me T h e g e n e ra l p ro c e d u re o f p e rf o rmi n g t h e ca l i b r a t i o n i s a s f o l l o w s: 1 Se l e ct t h e ca m e ra f o r w h i ch t h e i n t r i n si c p a r a me t e r c a l i b ra t i o n i s t o b e p e rf o rme d b y ch e cki n g t h e a p p ro p ri a t e b o x. 2 U n ch e ck t h e b o x f o r t h e d e f a u l t va l u e s, i f cu st o m va l u e s f o r t h e c a me ra ca l i b r a t i o n a re t o b e u se d a n d e n t e r t h e d e si re d va l u e s. 3 C l i ck t h e b u t t o n O b t a i n a n I ma g e t o o b t a i n t h e f i rst i ma g e T h e su b ro u t i n e a u t o ma t i ca l l y l o c a t e s t h e ca l i b ra t i o n t a b l e i f p re se n t a n d o b t a i n s t h e re q u i r e d i n f o rma t i o n I f t h e ca l i b r a t i o n t a b l e i s n o t f o u n d a n e rro r b o x p o p s u p a n d t h e i m a g e i s d i s ca rd e d 4 Mo ve t h e ca m e ra t o a n o t h e r p o s i t i o n su f f i ci e n t l y d i f f e re n t f ro m t h e p re vi o u s p o s i t i o n 5 R e p e a t st e p s 3 a n d 4 u n t i l a t l e a st 1 0 i ma g e s w i t h a va l i d ca l i b r a t i o n t a b l e a re o b t a i n e d 6 Pre ss t h e C a l cu l a t e I n t ri n si c P a ra m e t e rs b u t t o n t o o b t a i n t h e i n t ri n si c p a ra me t e rs f o r t h e c a me ra u s e d T h e va l u e s a re st o re d i n a t e xt f i l e i n t h e I n t e rn a l C a me r a C a l i b r a t i o n su b d i r e ct o ry o f t h e ma i n p ro g ra m. I f t h e re i s co n t i n u o u sl y a p ro b l e m w i t h f i n d i n g t h e ca l i b r a t i o n t a b l e i n t h e i ma g e s, i t i s re co mm e n d e d t o c h a n g e t h e d e f a u l t s e t t i n g s u n t i l t h e ca l i b r a t i o n t a b l e i s l o c a t e d i n e a ch i m a g e

PAGE 54

4 3 5 5 Ex tr i n s i c C a m e r a Pa r a m e te r C a l i b r a ti o n S u b r o u t i n e Si mi l a r t o t h e i n t ri n si c c a me r a p a r a me t e r ca l i b ra t i o n t h e r e i s a s u b ro u t i n e i n t e g ra t e d i n t o t h e m a i n u s e r i n t e rf a ce o f t h e p r o g ra m. F ro m t h e m a i n me n u t h e u se r si m p l y h a s t o cl i ck o n E xt ri n si c C a me ra Pa ra m e t e r C a l i b r a t i o n t o o p e n t h e w i n d o w sh o w n i n F i g u r e 5 4 T h i s t i me h o w e ve r i t i s re co mm e n d e d t o p e rf o rm t h e ca l i b ra t i o n f o r b o t h ca m e ra s a t t h e sa me t i me n o t o n l y t o sa ve t i me b u t a l so i n o rd e r t o mo r e a ccu ra t e l y o b t a i n t h e h a n d e ye ca l i b r a t i o n F i gu r e 5 4 : E xt e r n al C am e r a P ar a m e t e r C al i b r at i on S u b r ou t i n e

PAGE 55

4 4 W h i l e p e rf o rm i n g t h e e xt ri n s i c c a m e ra p a ra m e t e r c a l i b ra t i o n so m e i t e ms h a ve t o b e co n s i d e r e d Ag a i n t h e ca l i b r a t i o n t a b l e u s e d sh o u l d b e vi si b l e i n su f f i ci e n t l y d i f f e re n t p o si t i o n s a n d h a s t o b e e n t i r e l y v i si b l e i n e a ch c a me r a At l e a st 1 0 d i f f e re n t i ma g e s sh o u l d b e t a ke n i f p o ssi b l e 2 0 T h e p o si t i o n o f t h e ma n i p u l a t o r w i t h re sp e ct t o t h e b a se o f t h e ro b o t i c ma n i p u l a t o r, o r se t u p h a s t o b e kn o w n w i t h h i g h a cc u ra cy. T h e g e n e ra l p ro ce d u r e f o r p e rf o rmi n g a su cce ssf u l c a l i b ra t i o n o f t h e e xt ri n s i c p a ra me t e rs u si n g t h i s su b ro u t i n e i s a s f o l l o w s: 1 C l i ck o n O b t a i n I ma g e i n o rd e r t o o b t a i n t h e f i rst i ma g e 2 I f t h e i ma g e s o f b o t h ca me r a s co n t a i n t h e ca l i b ra t i o n t a b l e e n t e r t h e me a su re d co o rd i n a t e s o f t h e m a n i p u l a t o r w i t h re sp e ct t o t h e b a s e o f t h e ro b o t i c m a n i p u l a t o r a cco rd i n g t o t h e l a b e l 6 d i f f e re n t va l u e s a re re q u i r e d T h e se a re t h e p o si t i o n o f t h e ma n i p u l a t o r i n C a rt e si a n c o o rd i n a t e s, a s w e l l a s t h e t h re e m a i n a n g l e s, ro w p i t ch a n d ya w I t i s a b so l u t e l y a cc e p t a b l e t o u se 0 a s a va l u e sh o u l d t h e m a n i p u l a t o r n o t p o ss e ss a n y r o t a t i o n b u t si mp l y t ra n sl a t i o n 3 R e p e a t st e p 2 u n t i l a t l e a st 1 0 su cce s sf u l i ma g e s h a ve b e e n o b t a i n e d 4 C l i ck o n t h e C a l cu l a t e Ext ri n si c P a ra me t e rs b u t t o n t o co mp l e t e t h e e xt ri n si c ca l i b ra t i o n T h e e xt ri n si c ca l i b ra t i o n re s u l t s a re sa v e d a s a t e xt f i l e i n t h e Ext e rn a l C a me r a C a l i b r a t i o n su b d i r e ct o ry o f t h e ma i n p ro g ra m. T h e f i l e c o n t a i n s si x va l u e s t h e t h re e r o t a t i o n a n g l e s, a s w e l l a s t h e t ra n sl a t i o n a l C a rt e si a n

PAGE 56

4 5 va l u e s o f e a ch c a me ra w i t h r e sp e ct t o t h e e n d e f f e ct o r. T h e se v a l u e s a r e u se d f o r t h e re co n st ru ct i o n 5 6 Sh a p e Mo d e l C r e a ti o n Sh a p e Mo d e l s ca n b e cre a t e d u si n g t h e sh a p e mo d e l cre a t i o n su b r o u t i n e w h i ch c a n b e a cc e sse d f ro m t h e Exe cu t e Me n u w i t h i n t h e M a i n U se r I n t e r f a ce T h i s w i l l o p e n a n e w w i n d o w a s se e n i n F i g u re 5 5 b e l o w F i gu r e 5 5 : S h a p e M od e l C r e at i o n G U I

PAGE 57

4 6 I n o rd e r t o cre a t e a sh a p e mo d e l t h e f o l l o w i n g st e p s sh o u l d b e f o l l o w e d : 1 T h e n a me o f t h e Sh a p e Mo d e l sh o u l d b e e n t e re d T h i s ca n re a l l y b e a n yt h i n g t h e u se r w a n t s. H o w e v e r, i t i s re co mme n d e d t h a t i t b e so me t h i n g t h a t b e re l a t e d t o t h e o b j e ct 2 T h e ca me r a f o r w h i c h t h e sh a p e m o d e l i s t o b e cr e a t e d h a s i s s e l e ct e d W h i l e i n t h e o ry, o n l y o n e sh a p e mo d e l i s n e ce ssa ry, i t i s re co mme n d e d t o cre a t e a s h a p e m o d e l f o r e a ch c a m e ra a s t h i s e n s u re s m o re a ccu ra t e d e t e ct i o n 3 T h e u se r sh o u l d ch o o se t h e t yp e o f r e g i o n o f i n t e re st T h i s ca n b e ci rc u l a r o r re ct a n g u l a r. F o r t h e ci rc u l a r re g i o n o f i n t e re st t h e u se r c l i cks o n t h e l o ca t i o n o f t h e ci rcl e a n d t h e n d r a g s t h e mo u se t o t h e ra d i u s o f t h e c i rcl e F o r t h e re ct a n g u l a r re g i o n t h e f i rst cl i ck d e f i n e s t h e f i rst co rn e r o f t h e re ct a n g l e a n d t h e s e co n d c l i ck d e f i n e s t h e s e co n d co r n e r o f t h e re ct a n g l e By ri g h t cl i cki n g t h e re g i o n i s a cc e p t e d 4 F i n a l l y, t h e b u t t o n C re a t e Sh a p e M o d e l i s c l i ck e d w h i ch re su l t s i n t h e sh a p e mo d e l b e i n g s a ve d w i t h a n r o r a n d l a s a n i d e n t i f i e r f o r ri g h t a n d l e f t ca me ra re sp e ct i v e l y. 5 7 A b o u t Me n u I n a d d i t i o n t o t h e p re vi o u sl y d i sc u ss e d su b ro u t i n e s, t h e ma i n p ro g r a m a l so h a s a n a b o u t m e n u w h i ch co n t a i n s i n f o rma t i o n o n t h e p ro g r a m i n g e n e ra l su ch

PAGE 58

4 7 a s t h e v e rsi o n n u mb e r, i n t h i s c a se ve rsi o n 1 0 1 a s p r e se n t e d i n t h i s t h e si s. F i g u re 5 6 sh o w s t h i s a b o u t me n u F i gu r e 5 6 : A b ou t V i s i on A s s i s t I n t e r f ac e 5 8 Se r i a l D a ta T r a n s f e r I n o rd e r t o e f f e ct i ve l y u se t h e o b j e ct re co g n i t i o n a se ri a l i n t e rf a c e co n n e ct i o n w a s i n co r p o r a t e d i n t o t h e m a i n u s e r p ro g ra m. Sh o u l d t h e p o si t i o n o f t h e o b j e ct b e n e e d e d t o co n t r o l a ro b o t i c ma n i p u l a t o r, f o r e xa m p l e t h e d a t a ca n b e s e n t t o a n y co m p u t e r, w h i c h h a s a se r i a l i n t e r f a ce F i g u r e 5 7 s h o w s t h e p i n a ss i g n me n t o f a t yp i ca l R S 2 3 2 se ri a l co n n e ct o r.

PAGE 59

4 8 F i gu r e 5 7 : R S 232 S e r i a l C on n e c t or P i n A s s i gn m e n t T h e se ri a l i n t e rf a ce p ro t o c o l i s i n c l u d e d i n Ap p e n d i x A, w h i c h i n cl u d e s a l l so u rce co d e f o r t h e u se r i n t e rf a ce As l o n g a s t h e re i s a s e ri a l p o rt p r o t o co l a va i l a b l e i t i s p o ssi b l e t o r e a d t h e d a t a se n t f ro m t h e u se r i n t e rf a ce a n d u se i t a s d e si re d su ch a s co n t ro l l i n g t h e C a rt e s i a n p o s i t i o n o f t h e ro b o t i c ma n i p u l a t o r. Al l t h a t i s r e q u i re d f o r a s u cce ssf u l s e ri a l co mmu n i ca t i o n a re t w o co mp u t e rs, a se ri a l c a b l e w i t h t w o 9 p i n f e ma l e c o n n e ct o rs, w h e re p i n s 2 a n d 3 a s d e f i n e d b y F i g u re 5 7 a re cro ss c o n n e ct e d F o r t h e se ri a l c o mmu n i ca t i o n t h e MSC o mm co n t ro l i n co r p o ra t e d i n t o Vi s u a l B a si c 6 w a s u se d T h e p o rt se t t i n g s r e q u i re d f o r a su cc e ssf u l s e ri a l co mm u n i ca t i o n t h e se t t i n g s l i st e d i n T a b l e 5 1 b e l o w s h o u l d b e u se d o n b o t h s i d e s o f t h e co mmu n i ca t i o n T h e se ri a l co n n e ct i o n h a s t o b e a ct i va t e d f ro m t h e Exe cu t e Me n u o f t h e Ma i n G U I T ab l e 5 1 : S e r i al P or t S e t t i n gs Ba u d 9 6 0 0 Pa ri t y N D a t a 8 St o p Bi t 1

PAGE 60

4 9 C h a p te r 6 A s s i s t F u n c ti o n s 6 1 O v e r v i e w T h e u se r o f a ro b o t i c ma n i p u l a t o r co n t ro l s i t s mo ve me n t vi a a n i n p u t d e v i c e T h i s i n p u t d e v i ce co u l d b e a j o yst i ck, a h a p t i c d e v i ce su ch a s t h e PH AN T o M, j o g co n t ro l o r a n y o t h e r i ma g i n a b l e i n p u t d e vi c e R e g a rd l e ss o f t h e i n p u t d e v i ce a u se r w i t h a d i sa b i l i t y mi g h t f i n d i t d i f f i c u l t t o co n t ro l a r o b o t i c ma n i p u l a t o r d i r e ct l y, e sp e ci a l l y i f t h e p e rso n h a s t re mo rs a s ca u s e d b y, f o r e x a mp l e Pa rk i n so n s d i se a s e As p a rt o f Be n j a mi n F ri t z s [ 4 ] t h e si s, h e i n co rp o ra t e d t h re e d i f f e re n t a ssi st f u n c t i o n s i n t o t h e c o n t ro l o f a ro b o t i c ma n i p u l a t o r. T h e se a r e d e scri b e d i n t h i s ch a p t e r, f o l l o w e d b y t h e i r a p p l i c a t i o n t o t h i s w o rk. 6 2 L i n e a r A s s i s t F u n c ti o n T h e l i n e a r a ssi st f u n ct i o n re st ri ct s t h e mo ve me n t o f t h e ro b o t i c ma n i p u l a t o r t o a l i n e a r mo ve m e n t a l o n g a l i n e co n n e ct i n g t h e e n d e f f e ct o r a n d t h e g o a l p o si t i o n T h e g o a l p o si t i o n i s d e f i n e d a s t h e l o c a t i o n o f t h e o b j e ct o f i n t e re st By u si n g t h e t h e o ry d e scri b e d i n ch a p t e r 4 t h e p o si t i o n o f t h e o b j e ct o f i n t e r e st i s kn o w n w i t h re sp e ct t o t h e e n d e f f e ct o r. W h i l e F ri t z [ 4 ] e mp l o y e d o n l y o n e ca me r a a l o n g w i t h a l a s e r ra n g e f i n d e r, t h i s t h e s i s o b t a i n s t h e t h re e d i m e n si o n a l l o ca t i o n o f t h e o b j e ct o f i n t e re st d i re ct l y. H e n ce se ve ra l c a l cu l a t i o n s d e scr i b e d b y F ri t z n e e d

PAGE 61

5 0 n o t b e e mp l o ye d a n d t h e sca l i n g ca n b e s i mp l i f i e d T h e i ma g e f ra me o f t h e vi si o n syst e m ca n b e se e n i n f i g u re 6 1 b e l o w F i gu r e 6 1 : I m age F r am e o f V i s i on S ys t e m [ 4 ] T h e ve ct o r f ro m t h e e n d e f f e ct o r p ro j e ct i o n a s d e f i n e d i n f i g u re 6 1 i s si m p l y (6 1 ) w h e re X a n d Y a re t h e va l u e s o b t a i n e d f ro m t h e r e co n st ru ct i o n c a l c u l a t i o n s. T a ki n g Z a xi s a s a u n i t ve ct o r d i re ct l y u p w a rd s, Y ca n b e d e f i n e d a s B = Z x X N o w a ssu m i n g t h a t A = Ve ct o r, a n d B a n d Z a re t h e ve ct o rs d e f i n e d a n d ca l cu l a t e d p r e vi o u sl y, o n e c a n d e f i n e i t s e l e me n t s a s: (6 2 )

PAGE 62

5 1 (6 3 ) (6 4 ) N o w d e f i n i n g t h e d i re ct i o n co si n e s a s (6 5 ) (6 6 ) (6 7 ) (6 8 ) t h e t ra n sf o rma t i o n ma t ri x b e t w e e n t h e co n t ra i n t a n d t h e e n d e f f e ct o r f ra me i s: (6 9 ) F i n a l l y, i f t h e i n p u t ve l o ci t y o f t h e c o n t r o l d e vi ce s u ch a s t h e PH A N T o M, i s g i ve n t h e ve l o c i t y o f t h e ma n i p u l a t o r ca n b e ca l cu l a t e d a s: (6 1 0 ) I f t h e ve l o ci t y sca l e ma t ri x i s d e f i n e d a s (6 1 1 ) t h e n t h e m o d i f i e d ve l o ci t y o f t h e sl a v e ca n b e ca l cu l a t e d a s (6 1 2 ) W h i ch i s t h e n e w ve l o c i t y co mma n d f o r t h e ro b o t i c ma n i p u l a t o r a n d ca n b e se n t t h e co n t ro l l e r o f t h e ma n i p u l a t o r u se d

PAGE 63

5 2 6 3 Pl a n a r A s s i s t F u n c ti o n Si mi l a r t o t h e l i n e a r a ssi st f u n ct i o n j u st d e scri b e d t h e p l a n a r a ss i st f u n ct i o n re st ri ct s t h e mo ve me n t t o a p l a n e T h re e p o i n t s t h e p o s i t i o n o f t h e e n d e f f e ct o r, t h e p o si t i o n o f t h e o b j e ct o f i n t e re st a n d f i n a l l y a n a rb i t ra ry p o i n t a l o n g t h e z a xi s o f t h e e n d e f f e ct o r d e f i n e t h i s p l a n e T h i s p ro d u ce s a v e rt i ca l p l a n e co n n e ct i n g t h e e n d e f f e ct o r w i t h t h e o b j e ct o f i n t e re st T h e ca l cu l a t i o n s n e e d e d f o r t h e p l a n a r a ss i st f u n ct i o n a re e xa ct l y t h e sa me a s t h o se d e scr i b e d b y e q u a t i o n s (6 1 ) t h ro u g h (6 1 0 ), e xce p t t h a t t h e sca l e ma t ri x i s d e f i n e d a s: (6 1 3 ) H e n ce t h e ve l o ci t y o f t h e ma n i p u l a t o r i s sca l e d su ch t h a t t h e u n d e s i re d mo t i o n i s p e r p e n d i cu l a r t o t h e co n st ra i n t p l a n e a n d i s d e scri b e d b y e q u a t i o n (6 1 4 ). (6 1 4 ) 6 4 Ve l o c i ty A s s i s t F u n c ti o n T h e ve l o c i t y a ssi st f u n ct i o n co n t ro l s t h e ve l o ci t y o f t h e ro b o t i c ma n i p u l a t o r a s a f u n ct i o n o f t h e v e rt i ca l d i st a n c e f ro m t h e e n d e f f e ct o r t o t h e o b j e ct o f i n t e re st I f t h e d i st a n c e b e t w e e n t h e t w o i s l a r g e t h e ve l o c i t y i s sca l e d su c h t h a t t h e i t i n cre a s e s. As t h e d i st a n ce g e t s sma l l e r, t h e v e l o c i t y i s sc a l e d d o w n a l l o w i n g f o r a sa f e a p p r o a ch t o t h e o b j e ct o f i n t e re st F ri t z [ 4 ] d e f i n e d t h e ve l o ci t y sca l e f a ct o rs a s sh o w n i n F i g u re 6 2 b e l o w

PAGE 64

5 3 F i gu r e 6 2 : V e l oc i t y A s s i s t F u n c t i on S c al e f ac t or vs V e r t i c al D i s t an c e O n ce a g a i n t h e ca l c u l a t i o n s r e q u i r e d a r e d e scr i b e d b y e q u a t i o n s (6 1 ) t h ro u g h ( 6 1 0 ), w h i l e t h e Sca l e ma t ri x i s d e f i n e d a s: (6 1 5 ) F i n a l l y, t h e mo d i f i e d ve l o c i t y t h a t i s se n t t o ro b o t i c ma n i p u l a t o r i s d e f i n e d b y: (6 1 6 ) U n f o rt u n a t e l y, i t w a s n o t p o ss i b l e t o i mp l e me n t t h e s e a ssi st f u n ct i o n s a t t h e t i me o f t h e si s c o mp l e t i o n d u e t o t e ch n i ca l d i f f i cu l t i e s w i t h t h e ro b o t i c ma n i p u l a t o r, b u t t h e y h a ve b e e n s h o w n t o w o rk n i c e l y a n d d a t a w a s c o l l e ct e d b y F ri t z [ 4 ] a n d d e scri b e d i n h i s t h e si s.

PAGE 65

5 4 6 5 A s s i s t F u n c ti o n I n te g r a ti o n i n t o th e U s e r I n te r fa c e I n o rd e r t o p ro p e rl y u se t h e a ssi st f u n ct i o n s d e scr i b e d w i t h i n t h i s ch a p t e r, a su b ro u t i n e w h i c h ca n b e a cc e sse d b y t h e Exe cu t e me n u o f t h e u se r i n t e r f a ce w a s cre a t e d Ag a i n t h e se r i a l p o rt i s u se d t o se n d a n d re ce i ve t h e re q u i r e d d a t a F o r d e t a i l e d i n f o rm a t i o n o n t h e se ri a l t h e re a d e r sh o u l d co n s u l t C h a p t e r 5 T h e u se r i n t e rf a ce co n t a i n s a l l t h e re q u i r e d e q u a t i o n s d e scr i b e d w i t h i n t h i s ch a p t e r a n d ca l l s t h e a p p ro p ri a t e a ssi st f u n ct i o n se l e ct e d f ro m t h e Exe cu t e me n u T h e d a t a re q u i re d f ro m t h e i n p u t d e vi c e i s a ve ct o r o f ve l o c i t i e s, a s d e f i n e d b y e q u a t i o n (6 1 7 ): (6 1 7 ) w h e re t h e e l e m e n t s o f t h e m a t ri x d e n o t e t h e ve l o c i t y o f t h e i n p u t d e vi c e i n t h e x y a n d z d i re ct i o n re sp e ct i ve l y. By u si n g e q u a t i o n s (6 1 2 ), (6 1 4 ) o r (6 1 6 ) f o r l i n e a r, p l a n a r, a n d ve l o ci t y a ssi st f u n ct i o n s, re sp e ct i ve l y, a n d o f co u rs e t h e a p p ro p ri a t e sca l i n g ma t ri x, t h e i n p u t ve l o ci t i e s a re t h e n t ra n sf o rm e d t o mo d i f i e d ve l o ci t i e s. T h e se ve l o ci t i e s a re a g a i n a ma t ri x w i t h t h re e e l e me n t s a n d a re d e f i n e d b y e q u a t i o n (6 1 8 ) a s: (6 1 8 ) T h e mo d i f i e d ve l o ci t i e s a r e t h e n t ra n smi t t e d v i a t h e s e ri a l co n n e ct i o n d e scri b e d i n ch a p t e r 5 t o t h e co n t ro l c o mp u t e r o f t h e r o b o t i c m a n i p u l a t o r. T h e se

PAGE 66

5 5 ve l o ci t i e s ca n t h e n b e u se d t o sca l e t h e ve l o ci t y o f t h e i n p u t d e vi c e su ch t h a t t h e ro b o t i c ma n i p u l a t o r i s co n st ra i n e d a s d e si re d I t sh o u l d b e n o t e d h o w e ve r, t h a t i t i s a ssu me d t h a t t h e i n p u t d e v i ce p ro v i d e s a n d co l l e ct s i n f o rma t i o n o n t h e ve l o ci t y o f t h e i n p u t Su ch a n i n p u t d e vi c e i s t h e PH AN T o M [ 1 0 ] h a p t i c i n t e rf a c e T h i s i n p u t d e v i ce h a s b e e n u se d i n t h e w o rk o f F ri t z [ 4 ] a n d h a s b e e n s h o w n t o b e e xce l l e n t f o r t h e u s e w i t h sca l i n g a ssi st f u n ct i o n T h e sca l i n g f a ct o rs u se d a re g e n e ra l l y o n e t e n t h o f t h e i n p u t ve l o c i t y, b u t i t sh o u l d a l s o b e n o t e d t h a t i t i s p o ss i b l e a n d a t t i me s a d va n t a g e o u s t o mo d i f y t h e sca l i n g f a ct o rs t o a p a rt i c u l a r a p p l i c a t i o n T h e sm a l l e r t h e sc a l i n g v a l u e t h e sl o w e r t h e o u t p u t ve l o ci t y w i l l b e a n d o f co u rse t h e o u t p u t ve l o c i t y i s i n cr e a se d f o r l a rg e r sca l i n g va l u e s. F ri t z [ 4 ] h a s a l so s h o w n su c h t a sk s a s o b st a c l e a v o i d a n c e w h i c h i s a co mb i n a t i o n o f a ssi st f u n ct i o n Ag a i n t h e re a d e r s h o u l d re f e r t o F ri t z s w o rk f o r d e t a i l e d i n f o rma t i o n o n t h e a ss i st f u n ct i o n s.

PAGE 67

5 6 C h a p te r 7 Ex p e r i m e n ta l D a ta 7 1 Ph y s i c a l Ex p e r i m e n ta l Se tu p T h e u s e r i n t e rf a ce w a s t e st e d o n t h e se t u p sh o w n i n F i g u r e 7 1 b e l o w A si mp l e m e t a l sh e l f w a s ch o s e n a s t h e ske l e t o n f o r t h e se t u p T h e ca me ra s w e re ri g i d l y mo u n t e d t o a p l a t f o rm, w h i c h ca n b e t ra n sl a t e d i n t h e x a n d y d i re ct i o n s o f t h e se t u p T w o me t a l ru l e rs a r e mo u n t e d t o t h i s p l a t f o rm, so t h a t t h e p o si t i o n o f t h e i ma g i n a ry e n d e f f e ct o r ca n b e m e a su re d w h e n o b t a i n i n g d a t a Si n c e t h e ru l e rs t e n d t o b e n d d o w n w a r d a m e t a l w i r e o n w h i c h t h e ru l e rs re st i s w ra p p e d a ro u n d t h e ske l e t o n o f t h e se t u p T h e ze ro ma rk o f t h e ru l e rs i s t a ke n t o b e t h e ce n t e r o f t h e e n d e f f e ct o r. T h e n e n d e f f e ct o r i s t a k e n t o b e 2 0 ce n t i me t e rs b e l o w t h i s p o i n t Se ve ra l o b j e ct s o f i n t e re st f o r w h i c h d a t a i s p re se n t e d l a t e r i n t h i s c h a p t e r, a re ri g i d l y mo u n t e d t o a n i n t e rm e d i a t e sh e l f o r p l a t f o rm, w h i ch c a n b e t ra n sl a t e d i n t h e se t u p z d i r e ct i o n T h e p u rp o se f o r mo u n t i n g t h e s e o b j e ct s i s si mp l y t o a l l o w f o r so m e t i me s a vi n g a s t h e p o si t i o n o f t h e o b j e ct s d o e s n o t n e e d t o b e me a su re d f o r e a ch i m a g e t a ke n

PAGE 68

5 7 F i gu r e 7 1 : E xp e r i m e n t al S e t U p 7 2 Ex p e r i m e n ta l Pr o c e d u r e Af t e r h a vi n g o b t a i n e d t h e i n t ri n si c a n d e xt ri n si c ca m e ra p a ra me t e rs, t h e u se r i n t e rf a ce i s o p e n e d An i n i t i a l i m a g e i s a cq u i re d t o ma k e su re t h a t t h e c a me ra s a re f u n ct i o n i n g p ro p e r l y. A s h a p e mo d e l f o r t h e d e si r e d o b j e ct o f i n t e r e st i s se l e ct e d f ro m t h e d ro p d o w n me n u o f t h e ma i n i n t e rf a c e a n d a ct i va t e d b y cl i cki n g o n t h e Acce p t Sh a p e Mo d e l b u t t o n Af t e r t h i s i s co mp l e t e d t h e d a t a co l l e ct i o n ca n b e i n i t i a l i ze d ; cl i cki n g o n t h e L o c a t e O b j e ct b u t t o n d o e s t h i s. N o w t h e p o si t i o n s o b t a i n e d f ro m t h e re co n st ru ct i o n a l g o ri t h m a l o n g w i t h t h e me a su re d p o si t i o n s f o r e a ch i m a g e a re re co rd e d f o r f u t u re a n a l ys i s. Sh o u l d a n o b j e ct n o t b e d e t e ct e d i n a n i ma g e t h i s p o si t i o n i s st i l l re c o rd e d so a s t o o b t a i n

PAGE 69

5 8 i n f o rma t i o n a b o u t w h e t h e r t h e re a re ce rt a i n l o c a t i o n s o f t h e ca me r a w h e r e t h e i ma g e c a n n o t b e d e t e ct e d o r w h e t h e r t h i s i s si mp l y a f l u k e T h i s i n f o rma t i o n ca n t h e n b e u s e d t o re vi se t h e s h a p e mo d e l i f n e e d e d F i ve se t s o f e xp e r i me n t s w e re c o n d u ct e d e a ch s e t f o r a d i f f e re n t sh a p e mo d e l a n d o b j e ct o f i n t e re st T h e d a t a i s p r e se n t e d i n t h e f o l l o w i n g s e ct i o n sh o w i n g t h e a cc u ra cy a n d e rro rs i n vo l ve d 7 3 Ex p e r i m e n ta l D a t a T h e f i rst se t o f d a t a i s f o r t h e sh a p e mo d e l D o o r H a n d l e A d e scri p t i o n o f t h e d o o r h a n d l e sh a p e mo d e l a s w e l l a s t h e o t h e r sh a p e m o d e l s d i scu ss e d h e re a re i n cl u d e d i n t h e a p p e n d i x. F o l l o w i n g i s a g ra p h sh o w i n g a p l a n a r me a su re m e n t o f a d o o r h a n d l e F i g u re s 7 2 t h r o u g h 7 5 sh o w t h e x a n d y co mp a ri s o n o f t h e me a su re d i e a ct u a l p o si t i o n o f t h e d o o r h a n d l e ve rsu s t h e re co n st ru ct e d o r a ssu me d p o si t i o n o f t h e d o o r h a n d l e E a ch g ra p h s h o w s a se t o f d a t a f o r a d i f f e re n t d i scre e t h e i g h t a s a l l o w e d f o r b y t h e t e st b e d

PAGE 70

5 9 F i gu r e 7 2 : D oor H a n d l e D e t e c t i on f r o m Z = 0 5 4 m F i gu r e 7 3 : D oor H a n d l e D e t e c t i on f r o m Z = 0 4 4 m

PAGE 71

6 0 F i gu r e 7 4 : D oor H a n d l e D e t e c t i on f r o m Z = 0 3 4 m F i gu r e 7 5 : D oor H a n d l e D e t e c t i on f r o m Z = 0 1 4 m

PAGE 72

6 1 F i g u re s 7 2 t h ro u g h 7 5 sh o w t h a t t h e e rro r, o r d i f f e re n ce b e t w e e n t h e a ct u a l p o si t i o n o f t h e o b j e ct w i t h re sp e ct t o t h e vi rt u a l ma n i p u l a t o r d e cr e a se s b o t h w i t h a d e cre a se d h e i g h t a s w e l l a s w i t h t h e o b j e ct o f i n t e re st b e i n g cl o se t o t h e ce n t e r p o si t i o n o f t h e ma n i p u l a t o r. T h i s c a n b e e xp l a i n e d b y t h e f a ct t h a t t h e s h a p e mo d e l w a s cr e a t e d cl o s e t o t h e c e n t e r p o i n t o f t h e ma n i p u l a t o r. D u e t o g e o me t ry, t h e o b j e ct d e t e ct e d a p p e a rs t o ch a n g e i t s sh a p e a s t h e p o si t i o n i s ch a n g e d T h i s i s si mp l y d u e t o t h e ske w i n g o f g e o me t ry f ro m d i f f e re n t p e rsp e ct i v e s. T h e e rro r i n x, y, a n d z, a s a f u n ct i o n o f x, y, a n d z a re s h o w n b e l o w i n F i g u re 7 6 t h ro u g h F i g u re 7 8 b e l o w F i gu r e 7 6 : E r r or i n X as a F u n c t i o n o f X a n d f or D i f f e r e n t Z P os i t i on s

PAGE 73

6 2 F i gu r e 7 7 : E r r or i n Y P os i t i on as a F u n c t i on o f Y an d f or D i f f e r e n t Z V al u e s F i g u re s 7 6 a n d 7 7 sh o w a t re n d t o w a rd s d e cr e a si n g t h e e rro r i n t h e x a n d y d i re ct i o n a s t h e o b j e ct t o b e d e t e ct e d a p p ro a ch e s t h e ce n t e r p o i n t o f t h e ma n i p u l a t o r. T h e l i n e a r t re n d l i n e s i n t h e g ra p h s p o rt ra y t h i s n i c e l y. Ad d i t i o n a l l y, a s t h e h e i g h t f ro m t h e o b j e ct o f i n t e re st d e c r e a se s, t h e e rro r a l so d e cr e a se s sl i g h t l y.

PAGE 74

6 3 C h a p te r 8 C o n c l u s i o n s a n d F u tu r e W o r k 8 1 C o n c l u s i o n s A st e re o vi si o n syst e m h a s b e e n cr e a t e d u si n g o f f t h e sh e l f h a rd w a re a n d so f t w a re A u se r i n t e rf a ce t o p e rf o rm a l l r e q u i re d t a sks o f st e re o vi si o n h a s b e e n cre a t e d w h i ch a d d r e sse s t h e t w o ma i n p r o b l e ms o f st e re o v i si o n t h e co rre sp o n d e n ce a n d t h e re co n st ru ct i o n p ro b l e m. Sca l i n g a ssi st f u n ct i o n s h a ve b e e n i n co rp o ra t e d i n t o t h e u se r i n t e r f a ce w h i c h c a n b e u se d t o a ssi st i n t h e t e l e ro b o t i c co n t ro l o f a ro b o t i c ma n i p u l a t o r. T h e u s e r i n t e rf a ce i s b u i l t su c h t h a t p e rs o n s w i t h d i s a b i l i t i e s ca n u s e t h e p ro g ra m w i t h o u t a n y e xt e n si ve t ra i n i n g t h u s a l l o w i n g t h e m t o u se t h e p r o g ra m w i t h re h a b i l i t a t i o n ro b o t i c ma n i p u l a t o r s t o ma n i p u l a t e o b j e ct s. W h e n a u se r u se s t h e p ro g ra m, i t i s p o ssi b l e f o r t h i s p e rs o n t o d e t e ct a n y o b j e ct o f i n t e r e st a f t e r t h e u se r cre a t e s a sh a p e mo d e l f o r t h e o b j e ct Esp e ci a l l y p e rs o n s w i t h d i sa b i l i t i e s ca n b e n e f i t f ro m t h i s p ro g ra m a n d t h e st e re o vi si o n syst e m, a s i t i s i n e xp e n si ve a n d e a sy t o se t u p w i t h vi rt u a l l y a n y ro b o t i c ma n i p u l a t o r. T h i s i s t h e f i rst t i me t o t h e kn o w l e d g e o f t h e a u t h o r, t h a t a u se r i n t e rf a ce h a s b e e n cre a t e d f o r u se w i t h t e l e ro b o t i c ma n i p u l a t i o n o f o b j e ct s. By u si n g s h a p e b a s e d o b j e ct d e t e ct i o n i t i s p o ssi b l e t o q u i ckl y a n d e a si l y l o ca t e o b j e ct s o f i n t e r e st a n d a p p ro a c h t h e o b j e ct s u si n g sca l i n g a ss i st f u n ct i o n s.

PAGE 75

6 4 T h e u se r i n t e rf a c e a l s o i n c l u d e s a su b ro u t i n e w h i ch a l l o w s t h e u s e r t o o b t a i n t h e i n t ri n si c c a me ra p a r a me t e rs. T h i s ca n e a si l y b e p e rf o rm e d u si n g t h e ca l i b r a t i o n t a b l e w h i ch c a n b e cre a t e d f ro m t h e ca l i b ra t i o n t a b l e su b ro u t i n e Si mi l a r t o t h e i n t ri n si c c a me ra c a l i b ra t i o n su b r o u t i n e a s u b ro u t i n e f o r t h e e xt ri n si c c a me ra ca l i b ra t i o n w a s i n cl u d e d By f o l l o w i n g t h e p r o ce d u re s d e s cri b e d i n t h i s t h e s i s, a n y p e rs o n sh o u l d b e a b l e t o su cc e ssf u l l y se t u p t h e i r o w n st e re o vi si o n syst e m. T h e p r o g ra m cre a t e d f o r t h i s w o rk a l l o w s t h e u s e r t o se l e ct f ro m se ve r a l sh a p e m o d e l s w h i ch w e re cr e a t e d u si n g t h e u se r i n t e rf a ce a n d a l s o g i v e s t h e u se r t h e p o w e r t o cre a t e a n d e xp a n d t h e sh a p e mo d e l l i b r a ry a s d e si r e d f o r p ra ct i ca l l y a n y o b j e ct i ma g i n a b l e T h e d e t e ct i o n o f o b j e ct s w a s t e st e d u si n g t h e so f t w a r e a n d t h e sh a p e mo d e l s i n co rp o ra t e d i n t o t h e p ro g ra m. T h e e rro rs i n d e t e ct i o n l i e w i t h i n a cce p t a b l e ra n g e a n d d e cre a se a s t h e ma n i p u l a t o r mo ve s t o w a r d s t h e o b j e ct o f i n t e re st t h e re f o re ma k i n g t h e e rr o rs a cce p t a b l e I t h a s b e e n sh o w n t h a t t h e t h re e d i m e n si o n a l p o si t i o n o f t h e o b j e ct f o u n d b y t h e sh a p e m o d e l s ca n b e u s e d w i t h t h e a ssi st f u n ct i o n s d e v e l o p e d b y F ri t z. T h i s me a n s t h a t t h e st e r e o v i si o n syst e m i s re a d y f o r u se i n t e l e r o b o t i cs a n d re h a b i l i t a t i o n a p p l i c a t i o n s. T h e o n l y p ro ce d u r e s t h a t n e e d t o b e c o mp l e t e d l i e o n t h e ro b o t i c si d e T h i s me a n s t h a t i f t h e ro b o t i c ma n i p u l a t o r c a n b e co n t ro l l e d w i t h a l mo st a n y t yp e o f i n p u t d e vi c e i t i s p o ssi b l e t o si m p l y e q u i p t h e r o b o t i c ma n i p u l a t o r w i t h a st e r e o vi s i o n syst e m a l o n g w i t h t h e vi s u a l a ssi st p ro g ra m

PAGE 76

6 5 cre a t e d f o r t h i s w o rk. Af t e r co n n e ct i n g t h e t w o co mp u t e rs w i t h a st a n d a rd se ri a l ca b l e a s d e scri b e d i n ch a p t e r 5 i t i s t h e n p o ssi b l e t o u s e t h e o u t p u t o f t h e vi si o n syst e m t o co n t ro l a n d sca l e t h e mo v e me n t o f t h e ro b o t i c ma n i p u l a t o r. 8 2 F u tu r e W o r k a n d R e c o m m e n d a t i o n s T h e sh a p e m o d e l l i b r a ry sh o u l d b e va st l y e xp a n d e d t o i n c l u d e ma n y d i f f e re n t i t e ms co mmo n l y e n co u n t e re d i n e ve r y d a y l i f e W i t h a n e xt e n si ve sh a p e mo d e l l i b ra ry, t h e st e re o syst e m a l o n g w i t h t h e u se r i n t e rf a ce co u l d b e ma rk e t e d i n co o p e r a t i o n w i t h MVT e c [ 9 ] a s t h i s t h e si s re l i e s p a rt l y o n t h e i r so f t w a re H a l co n Mo re d a t a s h o u l d b e co l l e ct e d a s t o t h e e f f e ct s o f l i g h t i n g d i f f e re n c e s, a n d t o f i x t h i s p r o b l e m, a l i g h t s o u rce co u l d b e i n co r p o ra t e d i n t o t h e st e r e o sy st e m, a l l o w i n g f o r c o n st a n t l i g h t i n g a n d h e n ce p r o p e r d e t e ct i o n o f o b j e ct s o f i n t e re st Ad d i t i o n a l l y, t h e a ssi st f u n ct i o n s d e scri b e d i n ch a p t e r 6 s h o u l d b e p ro p e rl y e va l u a t e d a n d u se d i n e x p e ri m e n t s t o f u rt h e r mo d i f y t h e se a ss i st f u n ct i o n s. O n ce t h e syst e m i s s e t u p w i t h a c o mmo n l y a v a i l a b l e r o b o t i c ma n i p u l a t o r, su ch a s t h e PU MA o r t h e Ma n u s M a n i p u l a t o r, t h e e a se o f se t t i n g u p t h e s t e re o vi si o n syst e m sh o u l d b e t e st e d a n d t ri a l r u n s w i t h p e rso n s w i t h d i s a b i l i t i e s sh o u l d b e p e rf o rme d U s i n g t h e i r f e e d b a ck, t h e p r o g ra m a n d s u b ro u t i n e s s h o u l d t h e n b e o p t i mi ze d t o b e st f i t t h e u se rs re q u i re m e n t s.

PAGE 77

6 6 R e fe r e n c e s [ 1 ] U S. C e n su s Bu re a u Ame ri c a n s w i t h D i sa b i l i t i e s H o u se h o l d Eco n o mi c St u d i e s, F e b ru a ry 2 0 0 1 p p 7 0 7 3 [ 2 ] H ka n Ef t ri n g T h e U se w o rt h i n e ss o f R o b o t s f o r Pe o p l e w i t h Ph ysi c a l D i sa b i l i t i e s, D o ct o ra l D i sse rt a t i o n I SBN 9 1 6 2 8 3 7 1 1 7 L u n d Sw e d e n 1 9 9 9 [ 3 ] G u n n a r Bo l msj o H a ka n N e ve ryd a n d H a k a n Ef t ri n g R o b o t i cs i n R e h a b i l i t a t i o n I EEE T ra n sa ct i o n s o n R e h a b i l i t a t i o n En g i n e e r i n g Vo l 3 N o 1 Ma rch 1 9 9 5 [ 4 ] Be n j a mi n E. F ri t z, D e ve l o p me n t a n d T e st i n g o f a T e l e r o b o t i c Syst e m f o r En h a n c i n g Ma n i p u l a t i o n C a p a b i l i t i e s o f Pe rso n s w i t h D i s a b i l i t i e s , MSME T h e si s, U n i ve rs i t y o f So u t h F l o ri d a 2 0 0 2 [ 5 ] W e n t a o Y u I n t e l l i g e n t T e l e ro b o t i c Assi st a n ce F o r E n h a n ci n g Ma n i p u l a t i o n C a p a b i l i t i e s O f Pe rso n s W i t h D i s a b i l i t i e s Ph D D i sse rt a t i o n U n i v e rsi t y o f So u t h F l o ri d a 2 0 0 4 [ 6 ] R a me sh Ja i n Ma ch i n e Vi si o n McG ra w H i l l I n c. N e w Yo rk, N Y, 1 9 9 5 [ 7 ] N e l l o Z u e c h U n d e rst a n d i n g a n d A p p l yi n g Ma c h i n e Vi si o n Ma rce l D e kk e r, I n c. N e w Yo rk, N Y, 2 n d e d i t i o n 2 0 0 0 [ 8 ] Ema n u e l e T ru cco a n d Al e ssa n d r o Ve rri I n t r o d u ct o ry T e ch n i q u e s f o r 3 D C o mp u t e r Vi s i o n Pre n t i ce H a l l I n c. U p p e r Sa d d l e R i v e r, N J, 1 9 9 8 [ 9 ] MVT e c AG h t t p : / / w w w mvt e c. co m/ h a l co n [ 1 0 ] Se n sAb l e T e ch n o l o g i e s, h t t p : / / w w w se n sa b l e co m/ p r o d u ct s/ p h a n t o m. h t m

PAGE 78

6 7 B i b l i o g r a p h y [ 1 ] L u c D Jo l y a n d C l a u d e An d r i o t I mp o si n g mo t i o n co n st ra i n t s t o a f o rce re f l e ct i n g t e l e ro b o t t h ro u g h re a l t i me s i mu l a t i o n o f a vi rt u a l me ch a n i sm, Pro ce e d i n g s o f t h e 1 9 9 4 I EEE I n t e rn a t i o n a l C o n f e r e n ce o n R o b o t i cs a n d Au t o ma t i o n Sa n D i e g o C A, Ma y 1 9 9 4 p p 3 5 7 3 6 2 [ 2 ] Ka zu h i ro Ko s u g e Ko j i T a ke o a n d T o sh i o Pu k u d a U n i f i e d a p p ro a ch f o r t e l e o p e ra t i o n o f vi rt u a l a n d re a l e n v i ro n me n t ma n i p u l a t i o n b a se d o n re f e r e n ce d yn a mi cs, Pro c e e d i n g s o f t h e 1 9 9 5 I EEE I n t e rn a t i o n a l C o n f e re n c e o n R o b o t i cs a n d Au t o m a t i o n N a g o y a Ja p a n Ma y 1 9 9 5 p p 9 3 8 9 4 3 [ 3 ] T h o ma s B. Sh e ri d a n T e l e r o b o t i cs Au t o ma t i o n a n d H u ma n S u p e rv i so ry C o n t ro l T h e MI T Pre ss, L o n d o n En g l a n d 1 9 9 2 [ 4 ] H u ma n Ma ch i n e C o o p e ra t i v e T e l e ro b o t i cs, T o p i ca l R e p o rt F e d e ra l En e rg y T e ch n o l o g y C e n t e r, U S. D e p a rt me n t o f En e rg y, Mo rg a n t o w n W V. N o ve mb e r 1 9 9 8 [ 5 ] Ka ra n A. Ma n o ch a D e ve l o p me n t a n d I mp l e me n t a t i o n o f T e l e o p e r a t i o n Assi st F u n ct i o n s, Ma st e r o f Sci e n ce i n Me ch a n i ca l En g i n e e r i n g U n i ve rs i t y o f So u t h F l o ri d a Au g u st 2 0 0 0 [ 6 ] N o ra l i Pe rn a l e t e D e ve l o p me n t O f A R o b o t i c H a p t i c I n t e rf a ce T o Pe rf o rm Vo ca t i o n a l T a sks By Pe o p l e W i t h D i sa b i l i t i e s, D o ct o r o f Ph i l o so p h y, El e ct ri ca l En g i n e e r i n g U n i ve rsi t y o f So u t h F l o ri d a D e ce m b e r 2 0 0 1 [ 7 ] J. Sch u yl e r, R Ma h o n e y, Asse si n g H u ma n R o b o t i c Pe rf o rma n ce f o r Vo ca t i o n a l Pl a ce me n t I EEE T ra n sa ct i o n s o n R e h a b i l i t a t i o n En g i n e e r i n g Vo l 8 N o 3 Se p t e mb e r 2 0 0 0 [ 8 ] W H a rw i n T R a h ma n R F o u l d s, A R e vi e w o f D e si g n I ssu e s i n R e h a b i l i t a t i o n R o b o t i cs w i t h R e f e r e n c e t o N o rt h Ame ri ca n R e se a rch I EEE T ra n sa ct i o n s o n R e h a b i l i t a t i o n En g i n e e ri n g Vo l 3 N o 1 Ma rch 1 9 9 5 [ 9 ] R Erl a n d so n Ap p l i ca t i o n s o f R o b o t i c/ Me ch a t ro n i c Syst e ms i n Sp e c i a l Ed u ca t i o n R e h a b i l i t a t i o n T h e ra p y a n d Vo ca t i o n a l T ra i n i n g : A Pa ra d i g m Sh i f t I EEE T ra n sa ct i o n s o n R e h a b i l i t a t i o n En g i n e e ri n g Vo l 3 N o 1 Ma rch 1 9 9 5

PAGE 79

6 8 [ 1 0 ] C St a n g e r, C An g l i n g W H a rw i n D R o mi l l y, D e vi c e s f o r Assi st i n g Ma n i p u l a t i o n : A Su mma ry o f U se r T a s k Pri o ri t i e s, I EEE T ra n sa ct i o n s o n R e h a b i l i t a t i o n En g i n e e r i n g Vo l 2 N o 4 D e ce mb e r 1 9 9 4 [ 1 1 ] St e ve n E. Eve re t t R a j i v V. D u b e y Y. I so d a a n d C D u mo n t Vi si o n Ba se d En d Ef f e ct o r Al i g n me n t Assi st a n c e f o r T e l e o p e r a t i o n Pro ce e d i n g s o f t h e I EEE I n t e rn a t i o n a l C o n f e r e n ce o n R o b o t i cs a n d Au t o m a t i o n D e t ro i t MI . Ma y 1 9 9 9 p p 5 4 3 5 4 9 [ 1 2 ] St e ve n E. Eve re t t R a j i v V. D u b e y Mo d e l b a s e d va ri a b l e p o si t i o n ma p p i n g f o r T e l e ro b o t i c Ass i st a n ce i n a C yl i n d ri ca l En vi ro n me n t Pro ce e d i n g s o f t h e I EEE I n t e rn a t i o n a l C o n f e r e n ce o n R o b o t i cs a n d Au t o m a t i o n D e t ro i t MI . Ma y 1 9 9 9 p p 2 1 9 7 2 2 0 2 [ 1 3 ] R D u b e y, K. Ma n o ch a N Pe rn a l e t e Va ri a b l e Po si t i o n Ma p p i n g Ba se d Assi st a n ce i n T e l e o p e ra t i o n f o r N u cl e a r C l e a n U p T h e I n t e rn a t i o n a l C o n f e re n ce o n R o b o t i cs a n d A u t o ma t i o n (I C R A) 2 0 0 1 Se o u l Ko re a Ma y 2 1 2 6 2 0 0 1 [ 1 4 ] R D u b e y, S. Eve re t t N Pe rn a l e t e K. Ma n o ch a T e l e o p e ra t i o n Assi st a n ce T h ro u g h V a ri a b l e Ve l o ci t y Ma p p i n g I EEE T ra n sa ct i o n s o n R o b o t i cs a n d Au t o ma t i o n O ct o b e r 2 0 0 1 [ 1 5 ] Se n sAb l e T e ch n o l o g i e s, h t t p : / / w w w se n sa b l e co m/ p r o d u ct s/ p h a n t o m. h t m

PAGE 80

6 9 A p p e n d i c e s

PAGE 81

7 0 A p p e n d i x A : G r a p h i c a l U s e r I n te r fa c e So u r c e C o d e frmAbout 1 Option Explicit Reg Key Security Options... Const READ_CONTROL = &H20000 Const KEY_QUERY_VALUE = &H1 Const KEY_SET_VALUE = &H2 Const KEY_CREATE_SUB_KEY = &H4 Const KEY_ENUMERATE_SUB_KEYS = &H8 Const KEY_NOTIFY = &H10 Const KEY_CREATE_LINK = &H20 Const KEY_ALL_ACCESS = KEY_QUERY_VALUE + KEY_SET_VALUE + KEY_CREATE_SUB_KEY + KEY_ENUMERATE_SUB_KEYS + KEY_NOTIFY + KEY_CREATE _LINK + READ_CONTROL Reg Key ROOT Types... Const HKEY_LOCAL_MACHINE = &H80000002 Const ERROR_SUCCESS = 0 Const REG_SZ = 1 Unicode nul terminated string Const REG_DWORD = 4 32 bit number Const gREGKEYSYSINFOLOC = "SOFTWARE \ Microsoft \ Shared Tools Locat ion" Const gREGVALSYSINFOLOC = "MSINFO" Const gREGKEYSYSINFO = "SOFTWARE \ Microsoft \ Shared Tools \ MSINFO" Const gREGVALSYSINFO = "PATH" Private Declare Function RegOpenKeyEx Lib "advapi32" Alias "RegOpenKeyExA" (ByVal hKey As Long, ByVal lpSubKey As String, ByVal ulOptions As Long, ByVal samDesired As Long, ByRef phkResult As Long) As Long Private Declare Function RegQueryValueEx Lib "advapi32" Alias "RegQueryValueExA" (ByVal hKey As Long, ByVal lpValueName As String, ByVal lpReserved As Long, ByRef lpType A s Long, ByVal lpData As String, ByRef lpcbData As Long) As Long Private Declare Function RegCloseKey Lib "advapi32" (ByVal hKey As Long) As Long Private Sub cmdSysInfo_Click() Call StartSysInfo End Sub Private Sub cmdOK_Click() Unload Me End Sub Private Su b Form_Load() Me.Caption = "About & App.Title lblVersion.Caption = "Version & App.Major & "." & App.Minor & "." & App.Revision lblTitle.Caption = App.Title End Sub Public Sub StartSysInfo() On Error GoTo SysInfoErr Dim rc As Long Dim SysInfoPath As Str ing Try To Get System Info Program Path \ Name From Registry...

PAGE 82

7 1 A p p e n d i x A (C o n t i n u e d ) If GetKeyValue(HKEY_LOCAL_MACHINE, gREGKEYSYSINFO, gREGVALSYSINFO, SysInfoPath) Then Try To Get System Info Program Path Only From Registry... ElseIf GetKeyValue(HKEY _LOCAL_MACHINE, gREGKEYSYSINFOLOC, gREGVALSYSINFOLOC, SysInfoPath) Then Validate Existance Of Known 32 Bit File Version If (Dir(SysInfoPath & \ MSINFO32.EXE") <> "") Then SysInfoPath = SysInfoPath & \ MSINFO32.EXE" Error File Can Not Be Found... Else GoTo SysInfoErr End If Error Registry Entry Can Not Be Found... Else GoTo SysInfoErr End If frmAbout 2 Call Shell(SysInfoPath, vbNormalFocus) Exit Sub SysInfoErr: MsgBox "System Information Is Unavailable At This Time", vbOKOnly End Sub Public Funct ion GetKeyValue(KeyRoot As Long, KeyName As String, SubKeyRef As String, ByRef KeyVal As S tring) As Boolean Dim i As Long Loop Counter Dim rc As Long Return Code Dim hKey As Long Handle To An Open Registry Key Dim hDepth As Long Dim KeyValType As Long Data Type Of A Registry Key Dim tmpVal As String Tempory Storage For A Registry Key Value Dim KeyValSize As Long Size Of Registry Key Variable -----------------------------------------------------------' Open RegKey Under KeyRoot {HKEY_LOCAL_ MACHINE...} -----------------------------------------------------------rc = RegOpenKeyEx(KeyRoot, KeyName, 0, KEY_ALL_ACCESS, hKey) Open Registry Key If (rc <> ERROR_SUCCESS) Then GoTo GetKeyError Handle Error... tmpVal = String$(1024, 0) Allocate Variable Space KeyValSize = 1024 Mark Variable Size -----------------------------------------------------------' Retrieve Registry Key Value... -----------------------------------------------------------rc = RegQueryValueEx(hKey, SubKeyRef, 0, K eyValType, tmpVal, KeyValSize) Get/Create Key Value If (rc <> ERROR_SUCCESS) Then GoTo GetKeyError Handle Errors If (Asc(Mid(tmpVal, KeyValSize, 1)) = 0) Then Win95 Adds Null Terminated String... tmpVal = Left(tmpVal, KeyValSize 1) Null Found, Ex tract From String Else WinNT Does NOT Null Terminate String... tmpVal = Left(tmpVal, KeyValSize) Null Not Found, Extract String Only End If

PAGE 83

7 2 A p p e n d i x A (C o n t i n u e d ) -----------------------------------------------------------' Determine Key Value Type For Conversion... -----------------------------------------------------------Select Case KeyValType Search Data Types... Case REG_SZ String Registry Key Data Type KeyVal = tmpVal Copy String Value Case REG_DWORD Double Word Registry Key Data Ty pe For i = Len(tmpVal) To 1 Step 1 Convert Each Bit KeyVal = KeyVal + Hex(Asc(Mid(tmpVal, i, 1))) Build Value Char. By Char. Next KeyVal = Format$("&h" + KeyVal) Convert Double Word To String End Select GetKeyValue = True Return Success rc = RegCl oseKey(hKey) Close Registry Key Exit Function Exit GetKeyError: Cleanup After An Error Has Occured... KeyVal = "" Set Return Val To Empty String GetKeyValue = False Return Failure rc = RegCloseKey(hKey) Close Registry Key End Function frmcaltab 1 Private Sub cmdcreatecaltab_Click() Dim Op As New HOperatorSetX Dim width As String Dim Tuple As New HTupleX Dim hx_BasDefWinHandle As Variant Dim Width As Double lblcaltabstatus = "Running" Width = txtcaltabwidth.Text Call Op.CreateCaltab(Width, "C : \ VisionSystem \ CalibrationTable \ caltab.descr", "C: \ VisionSystem \ Cali brationTable \ caltab.ps") lblcaltabstatus = "Calibration Table Created" End Sub frminternalcameracalibration 1 Private Sub chkdefault_Click() frameparameters.Enabled = True End Sub Priva te Sub cmdinternalcameracalibration_Click() Dim checkbutton As Integer Dim Window As HWindowX Dim valsizegauss, valmarkthresh, valmindiammarks, valstartthresh As Integer Dim valdeltathresh, valminthresh, valmincontlength, valmaxdiammarks As Integer Dim val alpha As Long Set Window = HWindowXCtrli.HalconWindow valsizegauss = valsizegauss.Text valmarkthresh = valmarkthresh.Text valmindiammarks = valmindiammarks.Text valstartthresh = valstartthresh.Text

PAGE 84

7 3 A p p e n d i x A (C o n t i n u e d ) valdeltathresh = valdeltathresh.Te xt valminthresh = valminthresh.Text valalpha = valalpha.Text valmincontlength = valmincontlength.Text valmaxdiammarks = valmaxdiammarks.Text If chkhitachi.Value = 1 Then checkbutton = 1 lblinternalcameracalibrationstatus = "Running Hitachi Calibration" End If If chkimaging.Value = 1 Then checkbutton = 2 lblinternalcameracalibrationstatus = "Running Imaging Source Calibration" End If Call InternalCameraCalibration(Window, checkbutton, valsizegauss, valmarkthresh valmindiammarks, valstartthresh, valdeltat hresh, valminthresh, valalpha valmincontlength, valmaxdiammarks) lblinternalcameracalibrationstatus = "Finished" End Sub frmshapemodelcreation 1 Private Sub cmdshapemodelcreation_Click() draw_rectangle1 (WindowHandle, R1, C1, R2, C2) Please note: In Visual Basic the zooming of images will not be adjusted automatically (like in H Develop). Therefore you have to call 'op.SetPart()' with the parameters of the image you want to display. gen_rectangle1 (HROI, R1, C1, R2, C2) gen_rectangle1 (IROI, 139, 130, 260, 258) 'Call Op.ReadImage(hx_H1, "c:/VisionSystem/ImageCapture/H.jpg") 'Call Op.ReadImage(hx_I1, "c:/VisionSystem/ImageCapture/I.jpg") 'Call Op.FindShapeModel(hx_H1, hx_HModel, 0, 3.14, 0.3, 1, 0.5, "interpolation", 0, 0.1, hx_Row1, hx_Column 1, hx_Angle1, hx_Score1) 'Call Op.FindShapeModel(hx_I1, hx_IModel, 0, 3.14, 0.3, 1, 0.5, "interpolation", 0, 0.1, hx_Row, h x_Column, hx_Angle, hx_Score) End Sub Private Sub cmdshapemodelgrab_Click() Dim Op As New HOperatorSetX Sub RunHalcon(ByRef Window As HWindowX) Dim Tuple As New HTupleX Dim Window As HWindowX Dim hx_WindowHandle As Variant, hx_R1 As Variant Dim hx_C1 As Variant, hx_R2 As Variant Dim hx_C2 As Variant, hx_HModel As Variant Dim hx_IModel As Variant, hx_Row1 As Variant Dim hx_Column1 As Variant, hx_Angle1 As Variant Dim hx_Score1 As Variant, hx_Row As Variant Dim hx_Column As Variant, hx_Angle As Variant

PAGE 85

7 4 A p p e n d i x A (C o n t i n u e d ) Dim hx_Score As Variant, hx_FGHandle As Variant Dim hx_FGHandle1 As Variant, hx_Row2 As Variant Dim hx_Column2 A s Variant, hx_Radius As Variant Dim hx_fghitachi As Variant, hx_fgimagingsource As Variant Dim hx_hrow As Variant, hx_hcolumn As Variant Dim hx_hradius As Variant, hx_irow As Variant Dim hx_icolumn As Variant, hx_iradius As Variant Dim hx_icolumn2, hx_irow 2, hx_hrow2, hx_hcolumn2 As Variant Dim hx_BasDefWinHandle As Variant Dim hx_H As HUntypedObjectX, hx_I As HUntypedObjectX Dim hx_hroi As HUntypedObjectX, hx_iroi As HUntypedObjectX Dim hx_HIR As HUntypedObjectX, hx_IIR As HUntypedObjectX Dim hx_HMI As HUn typedObjectX, hx_HMR As HUntypedObjectX Dim hx_IMI As HUntypedObjectX, hx_IMR As HUntypedObjectX Dim hx_H1 As HUntypedObjectX, hx_I1 As HUntypedObjectX Dim hx_iImage As HUntypedObjectX, hx_himage As HUntypedObjectX 'Dim hx_ROI As HUntypedObjectX, hx_hroi A s HUntypedObjectX 'Dim hx_iroi As HUntypedObjectX, hx_himage As HUntypedObjectX Set Window = HWindowXCtrl.HalconWindow hx_BasDefWinHandle = Window.HalconID Call Op.CloseAllFramegrabbers Dim Dateiname As String Dateiname = txtdateiname.Text For Left Eye I f optleft.Value = True Then Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "ntsc", "2", 0, 1, hx_fgimagingsource) For i = 1 To 2 Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Next i Call Op.DispObj(hx_iIma ge, hx_BasDefWinHandle) If optcircle.Value = True Then frmshapemodelcreation 2 Call Op.DrawCircle(hx_BasDefWinHandle, hx_irow, hx_icolumn, hx_iradius) Call Op.GenCircle(hx_iroi, hx_irow, hx_icolumn, hx_iradius) End If If optrectangle.Value = True Then Ca ll Op.DrawRectangle1(hx_BasDefWinHandle, hx_irow, hx_icolumn, hx_irow2, hx_icolumn2) Call Op.GenRectangle1(hx_iroi, hx_irow, hx_icolumn, hx_irow2, hx_icolumn2) End If Call Op.ReduceDomain(hx_iImage, hx_iroi, hx_IIR) Call Op.InspectShapeModel(hx_IIR, hx_IMI hx_IMR, 4, 120) Call Op.CreateShapeModel(hx_IIR, 5, 0, 3.14, 0.0175, "none", "use_polarity", 120, 80, hx_IMode l) Call Op.WriteShapeModel(hx_IModel, "c:/VisionSystem/" & Dateiname & "i.sm") End If For Right Eye

PAGE 86

7 5 A p p e n d i x A (C o n t i n u e d ) If optright.Value = True Then Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "ntsc", "0", 0, 1, hx_fghitachi) For i = 1 To 2 Call Op.GrabImage(hx_himage, hx_fghitachi) Next i Call Op.DispObj(hx_himage, hx_BasDefWinHandle) If optcircle.Value = True Then Call Op.DrawCircle(hx_BasDefWinHandle, hx_hrow, hx_hcolumn, hx_hradius) Call Op.GenCircle(hx_hroi, hx_hrow, hx_hcolumn, hx_hradius) End If If optrectangle.Value = True Then Call Op.DrawRectangle1(hx_BasDefWinHandle, hx_hrow, hx_ hcolumn, hx_hrow2, hx_hcolumn2) Call Op.GenRectangle1(hx_hroi, hx_hrow, hx_hcolumn, hx_hrow2, hx_hcolumn2) End If Call Op.ReduceDomain(hx_himage, hx_hroi, hx_HIR) Call Op.InspectShapeModel(hx_HIR, hx_HMI, hx_HMR, 10, 120) Call Op.CreateShapeModel(hx_HIR, 5 0, 3.14, 0.0175, "none", "use_polarity", 120, 80, hx_HMode l) Call Op.WriteShapeModel(hx_HModel, "c:/VisionSystem/" & Dateiname & "h.sm") End If End Sub Sub InitHalcon() Default settings used in HDevelop Call Op.SetSystem("do_low_error", "false") End S ub HalconForm 1 This is a template VB project to be used with HALCON programs exported from HDevelop. To use exported HDevelop code, load it with the menu Project > Add Module > Existing. Note that you will usually have to adapt the size of the HALCON ActiveX window and/or add additional HALCON ActiveX windows, depending on how many HDevelop windows your program uses. If your HDevelop program does its image processing in an infinite loop, you will have to modify the code to allow the progr am to process the events generated by interaction from the user. Examine the other VB examples that come with HALCON to see how to do this. Option Explicit Private Sub cmdshapemodelgrab_Click() Dim Window1 As HWindowX Dim Window2 As HWindowX Dim Datein ame As String Set Window1 = HWindowXCtrl.HalconWindow Set Window2 = HWindowXCtrl1.HalconWindow

PAGE 87

7 6 A p p e n d i x A (C o n t i n u e d ) Dateiname = txtdateiname.Text HalconStatus = "Grabbing Image for Shape Model Creation" HalconStatus.Refresh Call ShapeModelGrabImage(Wind ow1, Window2, Dateiname) HalconStatus = "Finished Image Capture" End Sub Private Sub cmdshapemodelcreation_Click() Dim Window1 As HWindowX Dim Window2 As HWindowX Dim Hr1, Hc1, Hr2, Hc2, Ir1, Ic1, Ir2, Ic2 As Integer Set Window1 = HWindowXCtrl.HalconWindow Set Window2 = HWindowXCtrl1.HalconWindow HalconStatus = "Please Wait Shape Model is Being Created" HalconStatus.Refresh 'Hr1 = txtHr1.Text 'Hc1 = txtHc1.Text 'Hr2 = txtHr2.Text 'Hc2 = txtHc2.Text Ir1 = txtIr1.Text Ic1 = txtIc1.Text Ir2 = txtIr2.Te xt Ic2 = txtIc2.Text Call ShapeModelCreation(Window1, Window2, Hr1, Hr2, Hc1, Hc2, Ir1, Ir2, Ic1, Ic2) HalconStatus = "Finished Shape Model Creation" End Sub Private Sub cmdshapemodelconfirm_Click() Dim Op As New HOperatorSetX Dim Window1 As HWindowX D im Window2 As HWindowX Dim i As Integer Set Window1 = HWindowXCtrl.HalconWindow Set Window2 = HWindowXCtrl1.HalconWindow Dim R(1 To 9) As Double Dim T(1 To 3) As Double Dim RT(1 To 9) As Double Dim MI(1 To 9) As Double Dim MH(1 To 9) As Double HalconForm 2 Dim sxi, syi, fi, oxi, oyi As Double Dim sxh, syh, fh, oxh, oyh As Double Imaging Source Parameters Intrinsic Parameters 'Open "C: \ Vision System \ iintrin.txt" For Input As #1 'Input #1, sxi, syi, oxi, oyi, fi 'Close #1 sxi = 1.0249 10 ^ 5 syi = 1. 1 10 ^ 5 oxi = 311.828 oyi = 253.998 fi = 0.00872367 MI(1) = sxi / fi

PAGE 88

7 7 A p p e n d i x A (C o n t i n u e d ) MI(2) = 0 MI(3) = oxi sxi / fi MI(4) = 0 MI(5) = syi / fi MI(6) = oyi syi / fi MI(7) = 0 MI(8) = 0 MI(9) = 1 Hitachi Parameters Intrinsic Parameters 'Open "C: \ Vision System \ hintrin.txt" For Input As #2 'Input #2, sxh, syh, oxh, oyh, fh 'Close #2 sxh = 1.07002 10 ^ 5 syh = 1.1 10 ^ 5 oxh = 317.915 oyh = 224.494 fh = 0.0076443 MH(1) = sxh / fi MH(2) = 0 MH(3) = oxh sxh / fh MH(4) = 0 MH(5) = sy h / fh MH(6) = oyh syh / fh MH(7) = 0 MH(8) = 0 MH(9) = 1 Rotation Parameters THIS HAS TO BE REPLACED WITH R = R(H)*TRANSPOSE(R(I)) 'Open "C: \ Vision System \ rotation.txt" For Input As #3 'For i = 1 To 9 'Input #3, R(i) 'Next i 'Close #3 R(1) = 0.376 R (2) = 0.205 R(3) = 0.904 R(4) = 0.019 R(5) = 0.973 R(6) = 0.229 R(7) = 0.926 R(8) = 0.103 R(9) = 0.362 Transpose of Rotation Matrix RT(1) = R(1) RT(2) = R(4) RT(3) = R(7) RT(4) = R(2) RT(5) = R(5) RT(6) = R(8) HalconForm 3 RT(7) = R(3)

PAGE 89

7 8 A p p e n d i x A (C o n t i n u e d ) RT(8) = R(6) RT(9) = R(9) Grab Image and Find Shapemodel Dim Tuple As New HTupleX Dim hx_fghitachi As Variant, hx_fgimagingsource As Variant Dim hx_whhitachi As Variant, hx_whimagingsource As Variant Dim hx_BasDefWinHandle As Variant Dim hx_Bas DefWinHandle2 As Variant Dim hx_himage As HUntypedObjectX, hx_iImage As HUntypedObjectX Dim hx_ModelID1 As Variant Dim hx_ModelID2 As Variant Set Window1 = HWindowXCtrl.HalconWindow Set Window2 = HWindowXCtrl1.HalconWindow Dim YH, XH, YI, XI As Variant Dim hx_hAngle, hx_hScale, hx_hScore As Variant Dim hx_iAngle, hx_iScale, hx_iScore As Variant hx_BasDefWinHandle = Window1.HalconID hx_BasDefWinHandle2 = Window2.HalconID Dim shapemodelfilei, shapemodelfileh As String Dim MPI(1 To 3) As Double Dim MPH(1 To 3) As Double Dim RTPH(1 To 3) As Double Dim X, Y, Z As Double Dim c As Integer If Not txtname.Text = "0" Then shapemodelfilei = "C: \ VisionSystem \ & txtname.Text & "i.sm" shapemodelfileh = "C: \ VisionSystem \ & txtname.Text & "h.sm" End If First, make sure all framegrabbers are closed Call Op.CloseAllFramegrabbers Open both Framegrabbers Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "0", 0, 1, hx_fghitachi) Call Op.OpenFramegrabber("DFG LC", 1, 1, 0 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "2", 0, 1, hx_fgimagingsource) If txtname.Text = "0" Then shapemodelfilei = "C: \ VisionSystem \ & cmboshapemodel.Text & "i.sm" shapemodelfileh = "C: \ VisionSystem \ & cmboshapemodel.Text & "h.sm" End If lblshapemodelname.Caption = shapemodelfilei Detect object using shape model until Stop Button is pressed Call Op.ReadShapeModel(shapemodelfilei, hx_ModelID1) Call Op.ReadShapeModel(shapemodelfileh, hx_ModelID2) Do While cmdstop.Value = False F or c = 1 To 1000 For i = 1 To 2 Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Next i Call Op.DispObj(hx_himage, hx_BasDefWinHandle2)

PAGE 90

7 9 A p p e n d i x A (C o n t i n u e d ) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Call O p.FindScaledShapeModel(hx_himage, hx_ModelID2, 0, 3.14, 0.5, 1.1, 0.3, 1, 0.9, "interpol ation", 0, 0.2, YH, XH, hx_hAngle, hx_hScale, hx_hScore) Call Op.FindScaledShapeModel(hx_iImage, hx_ModelID1, 0, 3.14, 0.5, 1.1, 0.3, 1, 0.9, "interpol ation", 0, 0.2, YI, XI, hx_iAngle, hx_iScale, hx_iScore) Calculate the Position of the Object MPI(1) = MI(1) XI + MI(1) fi MPI(2) = MI(5) YI + MI(6) fh MPI(3) = fi HalconForm 4 MPH(1) = MH(1) XH + MH(3) fh MPH(2) = MH(5) YH + MH(6) fh MPH(3) = fh RTPH (1) = RT(1) MPH(1) + RT(2) MPH(2) + RT(3) MPH(3) RTPH(2) = RT(4) MPH(1) + RT(5) MPH(2) + RT(6) MPH(3) RTPH(3) = RT(7) MPH(1) + RT(8) MPH(2) + RT(9) MPH(3) X = MPI(1) RTPH(1) + MPI(2) RTPH(3) MPI(3) RTPH(2) Y = MPI(2) RTPH(2) + M PI(3) RTPH(1) MPI(1) RTPH(3) Z = MPI(3) RTPH(3) + MPI(1) RTPH(2) MPI(2) RTPH(1) lblxpos.Caption = X 0.15 lblypos.Caption = Y 0.15 lblzpos.Caption = Z + 0.15 'Next c Loop Call Op.CloseAllFramegrabbers End Sub Private Sub cmdshapemodelrun _Click() lblshapemodel = cmboshapemodel.Text End Sub Private Sub Form_Load() Call InitHalcon End Sub Private Sub cmdimagecapture_Click() Dim i As Integer Dim Dateiname As String i = txtimagecapture.Text Dateiname = txtdateiname.Text Dim Window1 As HWindo wX Dim Window2 As HWindowX Set Window1 = HWindowXCtrl.HalconWindow Set Window2 = HWindowXCtrl1.HalconWindow HalconStatus = "Running Image Capture" HalconStatus.Refresh Call ImageCapture(Window1, Window2, i, Dateiname) HalconStatus = "Finished." End Sub Pri vate Sub mnuabout_Click() frmAbout.Show End Sub Private Sub mnucaltab_Click()

PAGE 91

8 0 A p p e n d i x A (C o n t i n u e d ) frmcaltab.Show End Sub Private Sub mnuexit_Click() Unload Me End Sub Private Sub mnuimagecapture_Click() frmimagecapture.Show End Sub Private Sub mnuint ernalcameracalibration_Click() frminternalcameracalibration.Show End Sub HalconForm 5 Private Sub mnumanual3Dcalculation_Click() frmshapemodelcreation.Show End Sub CreateShapeModel 1 ' File generated by HDevelop for HALCON/COM (Visual Basic) Version 6.1 ' This file is intended to be used with the project HDevelopTemplate which can be found in %HALCONROOT% \ examples \ vb \ HDevelopTemplate \ Option Explicit Dim Op As New HOperatorSetX Sub RunHalcon(ByRef Window As HWindowX) Dim Tuple As New HTupleX Dim hx_WindowHandle As Variant, hx_R1 As Variant Dim hx_C1 As Variant, hx_R2 As Variant Dim hx_C2 As Variant, hx_HModel As Variant Dim hx_IModel As Variant, hx_Row1 As Variant Dim hx_Column1 As Variant, hx_Angle1 As Variant Dim hx_Score1 As Variant, hx_Row As Variant Dim hx_Column As Variant, hx_Angle As Variant Dim hx_Score As Variant, hx_FGHandle As Variant Dim hx_FGHandle1 As Variant, hx_Row2 As Variant Dim hx_Column2 As Variant, hx_Radius As Variant Dim hx_fghitachi As Variant, hx_fgimagingsource As Varian t Dim hx_hrow As Variant, hx_hcolumn As Variant Dim hx_hradius As Variant, hx_irow As Variant Dim hx_icolumn As Variant, hx_iradius As Variant Dim hx_BasDefWinHandle As Variant Dim hx_H As HUntypedObjectX, hx_I As HUntypedObjectX Dim hx_hroi As HUntypedObj ectX, hx_iroi As HUntypedObjectX Dim hx_HIR As HUntypedObjectX, hx_IIR As HUntypedObjectX Dim hx_HMI As HUntypedObjectX, hx_HMR As HUntypedObjectX Dim hx_IMI As HUntypedObjectX, hx_IMR As HUntypedObjectX Dim hx_H1 As HUntypedObjectX, hx_I1 As HUntypedObjec tX Dim hx_iImage As HUntypedObjectX, hx_himage As HUntypedObjectX Dim hx_ROI As HUntypedObjectX, hx_hroi As HUntypedObjectX Dim hx_iroi As HUntypedObjectX, hx_himage As HUntypedObjectX hx_BasDefWinHandle = Window.HalconID

PAGE 92

8 1 A p p e n d i x A (C o n t i n u e d ) Call Op.Op enFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "0", 0, 1, hx_fghitachi) Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "2", 0, 1, hx_fgimagingsource) Ca ll Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.GrabImage(hx_himage, hx_fghitachi) read_image (H, 'c:/VisionSystem/ImageCapture/HHandle.jpg') read_image (I, 'c:/VisionSystem/ImageCapture/IHandle.jpg') dev_open_window(...) Please note: In Vis ual Basic the zooming of images will not be adjusted automatically (like in H Develop). Therefore you have to call 'op.SetPart()' with the parameters of the image you want to display. Call Op.DispObj(hx_himage hx_BasDefWinHandle) draw_rectangle1 (W indowHandle, R1, C1, R2, C2) Call Op.DrawCircle(hx_BasDefWinHandle, hx_hrow, hx_hcolumn, hx_hradius) Call Op.GenCircle(hx_hroi, hx_hrow, hx_hcolumn, hx_hradius) Please note: In Visual Basic the zooming of images will not be adjusted automatically (like i n H Develop). Therefore you have to call 'op.SetPart()' with the parameters of the image you want to display. Call Op.DispObj(hx_iImage hx_BasDefWinHandle) Call Op.DrawCircle(hx_BasDefWinHandle, hx_irow, hx_icolumn, hx_iradius) Call Op.GenCircle(hx_i roi, hx_irow, hx_icolumn, hx_iradius) gen_rectangle1 (HROI, R1, C1, R2, C2) gen_rectangle1 (IROI, 139, 130, 260, 258) Call Op.ReduceDomain(hx_himage, hx_hroi, hx_HIR) Call Op.ReduceDomain(hx_iImage, hx_iroi, hx_IIR) Call Op.InspectShapeModel(hx_HIR, hx _HMI, hx_HMR, 10, 120) CreateShapeModel 2 Call Op.InspectShapeModel(hx_IIR, hx_IMI, hx_IMR, 4, 120) Call Op.CreateShapeModel(hx_HIR, 5, 0, 3.14, 0.0175, "none", "use_polarity", 120, 80, hx_HModel) Call Op.CreateShapeModel(hx_IIR, 5, 0, 3.14, 0.0175, "non e", "use_polarity", 120, 80, hx_IModel) Call Op.WriteShapeModel(hx_HModel, "c:/VisionSystem/Cross Griph.sm") Call Op.WriteShapeModel(hx_IModel, "c:/VisionSystem/Cross Gripi.sm") Call Op.ReadImage(hx_H1, "c:/VisionSystem/ImageCapture/H.jpg") Call Op.ReadIma ge(hx_I1, "c:/VisionSystem/ImageCapture/I.jpg") Call Op.FindShapeModel(hx_H1, hx_HModel, 0, 3.14, 0.3, 1, 0.5, "interpolation", 0, 0.1, hx_Row1, h x_Column1, hx_Angle1, hx_Score1) Call Op.FindShapeModel(hx_I1, hx_IModel, 0, 3.14, 0.3, 1, 0.5, "interpolatio n", 0, 0.1, hx_Row, hx _Column, hx_Angle, hx_Score) End Sub

PAGE 93

8 2 A p p e n d i x A (C o n t i n u e d ) Sub InitHalcon() Default settings used in HDevelop Call Op.SetSystem("do_low_error", "false") End Sub GrabSaveImage 1 ' File generated by HDevelop for HALCON/COM (Vis ual Basic) Version 6.1 ' This file is intended to be used with the project HDevelopTemplate which can be found in %HALCONROOT% \ examples \ vb \ HDevelopTemplate \ Option Explicit Dim Op As New HOperatorSetX Sub ImageCapture(ByRef Window As HWindowX, Window 2 As HWindowX, i As Integer, Dateiname As String) Dim Tuple As New HTupleX Dim hx_fghitachi As Variant, hx_fgimagingsource As Variant Dim hx_whhitachi As Variant, hx_whimagingsource As Variant Dim hx_BasDefWinHandle As Variant Dim hx_BasDefWinHandle2 As Va riant Dim hx_himage As HUntypedObjectX, hx_iImage As HUntypedObjectX hx_BasDefWinHandle = Window.HalconID hx_BasDefWinHandle2 = Window2.HalconID First, make sure all framegrabbers are closed Call Op.CloseAllFramegrabbers Open both Framegrabbers Call Op .OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "0", 0, 1, hx_fghitachi) Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "2", 0, 1, hx_fgimagingsource) Grab an Image "a" Times in Each Framegrabber Dim a As Integer For a = 1 To i Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.DispObj(hx_himage, hx_BasDefWinHandle2) Call Op.DispObj(hx_iImage, hx_BasDef WinHandle) Call Op.WriteImage(hx_himage, "jpeg", 0, "c:/VisionSystem/ImageCapture/" & Dateiname & "Hitach iImage" & a & ".jpg") Call Op.WriteImage(hx_iImage, "jpeg", 0, "c:/VisionSystem/ImageCapture/" & Dateiname & "Imagin gSourceImage" & a & ".jpg") Next a Call Op.CloseAllFramegrabbers End Sub 'Sub InitHalcon() Default settings used in HDevelop Call Op.SetSystem("do_low_error", "false") 'End Sub

PAGE 94

8 3 A p p e n d i x A (C o n t i n u e d ) InternalCalibration 1 ' File generated by HDevelop for HALCON/COM (Visual Basic) Version 6.1 ' This file is intended to be used with the project HDevelopTemplate which can be found in %HALCONROOT% \ examples \ vb \ HDevelopTemplate \ Option Explicit Dim Op As New HOperatorSetX Sub InternalCameraCalibration(ByRef Window As HWindowX, che ck As Integer, valsizegauss As Integer valmarkthresh As Integer, valmindiammarks As Integer, valstartthresh As Integer valdeltathresh As Integer, valminthresh As Integer, valalpha As Long, valmincontlength As Integer, valmaxdiammarks As Integer) Dim hx_BasDefWinHandle As Variant hx_BasDefWinHandle = Window.HalconID Dim HTuple As New HTupleX Dim ITuple As New HTupleX Dim HRowTuple As New HTupleX Dim HColTuple As New HTupleX Dim IRowTuple As New HTupleX Dim IColTuple As New HTupleX Dim HSPTuple As N ew HTupleX Dim ISPTuple As New HTupleX Dim hx_fghitachi As Variant, hx_fgimagingsource As Variant Dim hx_hRCoord1 As Variant, hx_hCCoord1 As Variant Dim hx_IRCoord1 As Variant, hx_ICCoord1 As Variant Dim hx_hStartPose1 As Variant Dim hx_IStartPose1 As Vari ant Dim hx_hRCoord2 As Variant, hx_hCCoord2 As Variant Dim hx_IRCoord2 As Variant, hx_ICCoord2 As Variant Dim hx_hStartPose2 As Variant Dim hx_IStartPose2 As Variant Dim hx_hRCoord3 As Variant, hx_hCCoord3 As Variant Dim hx_IRCoord3 As Variant, hx_ICCoord3 As Variant Dim hx_hStartPose3 As Variant Dim hx_IStartPose3 As Variant Dim hx_hRCoord4 As Variant, hx_hCCoord4 As Variant Dim hx_IRCoord4 As Variant, hx_ICCoord4 As Variant Dim hx_hStartPose4 As Variant Dim hx_IStartPose4 As Variant Dim hx_hRCoord5 As Var iant, hx_hCCoord5 As Variant Dim hx_IRCoord5 As Variant, hx_ICCoord5 As Variant Dim hx_hStartPose5 As Variant Dim hx_IStartPose5 As Variant Dim hx_hRCoord6 As Variant, hx_hCCoord6 As Variant Dim hx_IRCoord6 As Variant, hx_ICCoord6 As Variant Dim hx_hStartP ose6 As Variant Dim hx_IStartPose6 As Variant Dim hx_hRCoord7 As Variant, hx_hCCoord7 As Variant

PAGE 95

8 4 A p p e n d i x A (C o n t i n u e d ) Dim hx_IRCoord7 As Variant, hx_ICCoord7 As Variant Dim hx_hStartPose7 As Variant Dim hx_IStartPose7 As Variant Dim hx_hRCoord8 As Varia nt, hx_hCCoord8 As Variant Dim hx_IRCoord8 As Variant, hx_ICCoord8 As Variant Dim hx_hStartPose8 As Variant Dim hx_IStartPose8 As Variant InternalCalibration 2 Dim hx_hRCoord9 As Variant, hx_hCCoord9 As Variant Dim hx_IRCoord9 As Variant, hx_ICCoord9 As Variant Dim hx_hStartPose9 As Variant Dim hx_IStartPose9 As Variant Dim hx_BasDefWinHandle As Variant Dim hx_iImage As HUntypedObjectX, hx_himage As HUntypedObjectX Dim hx_Image1 As HUntypedObjectX, hx_HCaltab As HUntypedObjectX Dim hx_ICaltab As HUntype dObjectX Dim hx_HCamParam As Variant, hx_ICamParam As Variant Dim hx_NX As Variant Dim hx_NY As Variant Dim hx_NZ As Variant Dim hx_HHCamParam As Variant, hx_HNFinalPose As Variant Dim hx_IICamParam As Variant, hx_INFinalPose As Variant Dim hx_HErrors As V ariant Dim hx_IErrors As Variant Dim i As Integer Dim a As Integer Call Op.CloseAllFramegrabbers Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "0", 0, 1, hx_fghitachi) Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "2", 0, 1, hx_fgimagingsource) For i = 1 To 20 valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiammarks Grab a set of images to get rid of init ial distortion If check = 1 Then Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(24 0, HTuple. TupleConcat(640, 480)))))))

PAGE 96

8 5 A p p e n d i x A (C o n t i n u e d ) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord1, hx_hCCoord1, hx_hStartPose1) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Cal l Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/Visi onSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTuple. TupleConcat(640, 480))))))) valstartthres h, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord2, hx_hCCoord2, hx_hStartPose2) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCalta b(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.Tu pleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTuple. InternalCalibration 3 TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlen gth, valmaxdiam marks, hx_hRCoord3, hx_hCCoord3, hx_hStartPose3) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTab le/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011

PAGE 97

8 6 A p p e n d i x A (C o n t i n u e d ) HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord4, hx_hCCoord4, hx_hSta rtPose4) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindia mmarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleC oncat(240, HTuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord5, hx_hCCoord5, hx_hStartPose5) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds (5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/Calibra tionTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathres h, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord6, hx_hCCoord6, hx_hStartPose6) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_ HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr"

PAGE 98

8 7 A p p e n d i x A (C o n t i n u e d ) HTuple.TupleConcat(0.008, H Tuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord7, hx_hCCoord7, hx_hStartPose7) Call Op.DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR" valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.00001 1, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTuple. TupleConcat(640, 480))))))) InternalCalibration 4 valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord8, hx_hCCoord8, hx_hStartPose8) Call Op .DispObj(hx_himage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_himage, hx_fghitachi) Call Op.FindCaltab(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op. FindMarksAndPose(hx_himage, hx_HCaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" HTuple.TupleConcat(0.008, HTuple.TupleConcat(0#, HTuple.TupleConcat(0.000011 HTuple.TupleConcat(0.000011, HTuple.TupleConcat(320, HTuple.TupleConcat(240, HTupl e. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_hRCoord9, hx_hCCoord9, hx_hStartPose9) Call Op.ReadCamPar("c:/VisionSystem/InternalCameraCalibration/campar.dat", hx_HCamParam ) Call Op.CaltabPoints("C:/VisionSystem/CalibrationTable/caltab.descr", hx_NX, hx_NY, hx_NZ) Call Op.CameraCalibration(hx_NX, hx_NY, hx_NZ

PAGE 99

8 8 A p p e n d i x A (C o n t i n u e d ) HRowTuple.TupleConcat(hx_hRCoord1, HRowTuple.TupleConcat(hx_hRCoord2, HRowTuple.Tupl eCon cat(hx_hRCoord3, HRowTuple.TupleConcat(hx_hRCoord4 HRowTuple.TupleConcat(hx_hRCoord5, HRowTuple.TupleConcat(hx_hRCoord6, HRowTuple.Tupl eConcat(hx_hRCoord7, HRowTuple.TupleConcat(hx_hRCoord8 hx_hRCoord9)))))))) HColTuple.TupleConcat(hx_hCCoord1 HColTuple.TupleConcat(hx_hCCoord2, HColTuple.Tupl eConcat(hx_hCCoord3, HColTuple.TupleConcat(hx_hCCoord4 HColTuple.TupleConcat(hx_hCCoord5, HColTuple.TupleConcat(hx_hCCoord6, HColTuple.Tupl eConcat(hx_hCCoord7, HColTuple.TupleConcat(hx_hCCoord8 h x_hCCoord9)))))))) hx_HCamParam HSPTuple.TupleConcat(hx_hStartPose1, HSPTuple.TupleConcat(hx_hStartPose2, HSPTuple.T upleConcat(hx_hStartPose3, HSPTuple.TupleConcat(hx_hStartPose4 HSPTuple.TupleConcat(hx_hStartPose5, HSPTuple.TupleConcat(hx_hSt artPose6, HSPTuple.T upleConcat(hx_hStartPose7, HSPTuple.TupleConcat(hx_hStartPose8 hx_hStartPose9)))))))) "all", hx_HHCamParam, hx_HNFinalPose, hx_HErrors) Call Op.WriteCamPar(hx_HHCamParam, "c:/VisionSystem/InternalCameraCalibration/hCamParam.da t") End If If check = 2 Then Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindia mmarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleC oncat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord1, hx_ICCoord1, hx_IStartPose1) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds (5) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks)

PAGE 100

8 9 A p p e n d i x A (C o n t i n u e d ) Call Op.FindMarksAndPose(hx_iImage, hx_ICa ltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord2, hx_ICCoord2, hx_IStartPose2) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep InternalCalibration 5 Call Op.WaitSeconds(5) Call Op.GrabImage(hx_i Image, hx_fgimagingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d e scr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, val alpha, valmincontlength, valmaxdiam marks, hx_IRCoord3, hx_ICCoord3, hx_IStartPose3) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/V isionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConc at(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord4, hx_ICCoord4, hx_IS tartPose4) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_iImage, hx_fgimagingsource)

PAGE 101

9 0 A p p e n d i x A (C o n t i n u e d ) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsiz egauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITupl e.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord5, hx_ICCoord5, hx_IStartPose5) Call Op.DispObj(hx_iImage, hx_BasDefWin Handle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))) )))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord6, hx_ICCoord6, hx_IStartPose6) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_iImage, hx_fgimag ingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple. TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincon tlength, valmaxdiam marks, hx_IRCoord7, hx_ICCoord7, hx_IStartPose7) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep

PAGE 102

9 1 A p p e n d i x A (C o n t i n u e d ) InternalCalibration 6 Call Op.WaitSeconds(5) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.FindCal tab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR", valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple. TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.000011, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord8, hx_ICCoord8, hx_IStartPose8) Call Op.DispObj(hx_iImage, hx_BasDefWinHandle) Beep Call Op.WaitSeconds(5) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.FindCaltab(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.DESCR" valsizegauss, valmarkthresh, valmindiammarks) Call Op.FindMarksAndPose(hx_iImage, hx_ICaltab, "c:/VisionSystem/CalibrationTable/caltab.d escr" ITuple.TupleConcat(0.008, ITuple.TupleConcat(0#, ITuple.TupleConcat(0.000011 ITuple.TupleConcat(0.00001 1, ITuple.TupleConcat(320, ITuple.TupleConcat(240, ITuple. TupleConcat(640, 480))))))) valstartthresh, valdeltathresh, valminthresh, valalpha, valmincontlength, valmaxdiam marks, hx_IRCoord9, hx_ICCoord9, hx_IStartPose9) Call Op.ReadCamPar("c:/VisionSy stem/InternalCameraCalibration/campar.dat", hx_ICamParam) Call Op.CaltabPoints("C:/VisionSystem/CalibrationTable/caltab.descr", hx_NX, hx_NY, hx_NZ) Call Op.CameraCalibration(hx_NX, hx_NY, hx_NZ IRowTuple.TupleConcat(hx_IRCoord1, IRowTuple.TupleConcat( hx_IRCoord2, IRowTuple.Tupl eConcat(hx_IRCoord3, IRowTuple.TupleConcat(hx_IRCoord4 IRowTuple.TupleConcat(hx_IRCoord5, IRowTuple.TupleConcat(hx_IRCoord6, IRowTuple.Tupl eConcat(hx_IRCoord7, IRowTuple.TupleConcat(hx_IRCoord8 hx_IRCoord9)))))))) I ColTuple.TupleConcat(hx_ICCoord1, IColTuple.TupleConcat(hx_ICCoord2, IColTuple.Tupl eConcat(hx_ICCoord3, IColTuple.TupleConcat(hx_ICCoord4

PAGE 103

9 2 A p p e n d i x A (C o n t i n u e d ) IColTuple.TupleConcat(hx_ICCoord5, IColTuple.TupleConcat(hx_ICCoord6, IColTuple.Tupl eCon cat(hx_ICCoord7, IColTuple.TupleConcat(hx_ICCoord8 hx_ICCoord9)))))))) hx_ICamParam ISPTuple.TupleConcat(hx_IStartPose1, ISPTuple.TupleConcat(hx_IStartPose2, ISPTuple.T upleConcat(hx_IStartPose3, ISPTuple.TupleConcat(hx_IStartPose4 ISPTuple .TupleConcat(hx_IStartPose5, ISPTuple.TupleConcat(hx_IStartPose6, ISPTuple.T upleConcat(hx_IStartPose7, ISPTuple.TupleConcat(hx_IStartPose8 hx_IStartPose9)))))))) "all", hx_IICamParam, hx_INFinalPose, hx_IErrors) Call Op.WriteCamPar(hx_IICamParam, "c:/VisionSystem/InternalCameraCalibration/ICamParam.da t") End If Call Op.CloseAllFramegrabbers End Sub Sub InitHalcon() Default settings used in HDevelop Call Op.SetSystem("do_low_error", "false") End Sub Module1 1 ' File generated by HDevelop for HALCON/COM (Visual Basic) Version 6.1 ' This file is intended to be used with the project HDevelopTemplate which can be found in %HALCONROOT% \ examples \ vb \ HDevelopTemplate \ Option Explicit Dim Op As New HOperatorSetX Sub RunHalcon(ByRef Window As HWin dowX) Dim Tuple As New HTupleX Dim hx_WindowHandle As Variant, hx_R1 As Variant Dim hx_C1 As Variant, hx_R2 As Variant Dim hx_C2 As Variant, hx_HModel As Variant Dim hx_IModel As Variant, hx_Row1 As Variant Dim hx_Column1 As Variant, hx_Angle1 As Variant D im hx_Score1 As Variant, hx_Row As Variant Dim hx_Column As Variant, hx_Angle As Variant Dim hx_Score As Variant, hx_FGHandle As Variant Dim hx_FGHandle1 As Variant, hx_Row2 As Variant Dim hx_Column2 As Variant, hx_Radius As Variant Dim hx_fghitachi As Var iant, hx_fgimagingsource As Variant Dim hx_hrow As Variant, hx_hcolumn As Variant Dim hx_hradius As Variant, hx_irow As Variant Dim hx_icolumn As Variant, hx_iradius As Variant Dim hx_BasDefWinHandle As Variant Dim hx_H As HUntypedObjectX, hx_I As HUntyped ObjectX Dim hx_hroi As HUntypedObjectX, hx_iroi As HUntypedObjectX Dim hx_HIR As HUntypedObjectX, hx_IIR As HUntypedObjectX

PAGE 104

9 3 A p p e n d i x A (C o n t i n u e d ) Dim hx_HMI As HUntypedObjectX, hx_HMR As HUntypedObjectX Dim hx_IMI As HUntypedObjectX, hx_IMR As HUntypedOb jectX Dim hx_H1 As HUntypedObjectX, hx_I1 As HUntypedObjectX Dim hx_iImage As HUntypedObjectX, hx_himage As HUntypedObjectX Dim hx_ROI As HUntypedObjectX, hx_hroi As HUntypedObjectX Dim hx_iroi As HUntypedObjectX, hx_himage As HUntypedObjectX hx_BasDefWinH andle = Window.HalconID Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "0", 0, 1, hx_fghitachi) Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "2", 0, 1, hx_fgimagingsource) Call Op.GrabImage(hx_iImage, hx_fgimagingsource) Call Op.GrabImage(hx_himage, hx_fghitachi) read_image (H, 'c:/VisionSystem/ImageCapture/HHandle.jpg') read_image (I, 'c:/VisionSystem/ImageCapture/IHandle.jpg') dev_open _window(...) Please note: In Visual Basic the zooming of images will not be adjusted automatically (like in H Develop). Therefore you have to call 'op.SetPart()' with the parameters of the image you want to display. Call Op.DispObj(hx_himage hx_Bas DefWinHandle) draw_rectangle1 (WindowHandle, R1, C1, R2, C2) Call Op.DrawCircle(hx_BasDefWinHandle, hx_hrow, hx_hcolumn, hx_hradius) Call Op.GenCircle(hx_hroi, hx_hrow, hx_hcolumn, hx_hradius) Please note: In Visual Basic the zooming of images will not be adjusted automatically (like in H Develop). Therefore you have to call 'op.SetPart()' with the parameters of the image you want to display. Call Op.DispObj(hx_iImage hx_BasDefWinHandle) Call Op.DrawCircle(hx_BasDefWinHandle, hx_irow, hx_icolumn, hx_iradius) Call Op.GenCircle(hx_iroi, hx_irow, hx_icolumn, hx_iradius) gen_rectangle1 (HROI, R1, C1, R2, C2) gen_rectangle1 (IROI, 139, 130, 260, 258) Call Op.ReduceDomain(hx_himage, hx_hroi, hx_HIR) Call Op.ReduceDomain(hx_iImage, hx_iroi, hx_IIR) Ca ll Op.InspectShapeModel(hx_HIR, hx_HMI, hx_HMR, 10, 120) Module1 2 Call Op.InspectShapeModel(hx_IIR, hx_IMI, hx_IMR, 4, 120) Call Op.CreateShapeModel(hx_HIR, 5, 0, 3.14, 0.0175, "none", "use_polarity", 120, 80, hx_HModel) Call Op.CreateShapeModel(hx_IIR, 5, 0, 3.14, 0.0175, "none", "use_polarity", 120, 80, hx_IModel) Call Op.WriteShapeModel(hx_HModel, "c:/VisionSystem/Cross Griph.sm") Call Op.WriteShapeModel(hx_IModel, "c:/VisionSystem/Cross Gripi.sm") Call Op.ReadImage(hx_H1, "c:/VisionSystem/ImageCaptur e/H.jpg") Call Op.ReadImage(hx_I1, "c:/VisionSystem/ImageCapture/I.jpg")

PAGE 105

9 4 A p p e n d i x A (C o n t i n u e d ) Call Op.FindShapeModel(hx_H1, hx_HModel, 0, 3.14, 0.3, 1, 0.5, "interpolation", 0, 0.1, hx_Row1, h x_Column1, hx_Angle1, hx_Score1) Call Op.FindShapeModel(hx_I 1, hx_IModel, 0, 3.14, 0.3, 1, 0.5, "interpolation", 0, 0.1, hx_Row, hx _Column, hx_Angle, hx_Score) End Sub Sub InitHalcon() Default settings used in HDevelop Call Op.SetSystem("do_low_error", "false") End Sub ObtainShapeModel 1 Option Explicit Dim Op As New HOperatorSetX Sub ObtainShapeModel(ByRef Window1 As HWindowX, Window2 As HWindowX) Dim Tuple As New HTupleX Dim hx_fghitachi As Variant, hx_fgimagingsource As Variant Dim hx_whhitachi As Variant, hx_whimagingsource As Variant Dim hx_BasDefWinHandle As Variant Dim hx_BasDefWinHandle2 As Variant Dim hx_himage As HUntypedObjectX, hx_iImage As HUntypedObjectX hx_BasDefWinHandle = Window.HalconID hx_BasDefWinHandle2 = Window2.HalconID Dim hx_FGHandle As Variant, hx_ModelID1 As Variant Dim hx_Mode lID2 As Variant, hx_ModelID As Variant Dim hx_i As Variant, hx_Row As Variant Dim hx_Column As Variant, hx_Angle As Variant Dim hx_Scale As Variant, hx_Score As Variant Dim hx_Row1 As Variant, hx_Column1 As Variant Dim hx_Angle1 As Variant, hx_Sc ale1 As Variant Dim hx_Score1 As Variant, hx_Row2 As Variant Dim hx_Column2 As Variant, hx_Angle2 As Variant Dim hx_Scale2 As Variant, hx_Score2 As Variant Dim hx_Image As HUntypedObjectX hx_BasDefWinHandle = Window.HalconID First make sure all Framegrabbers are closed Op.CloseAllFramegrabbers Now, open both Framegrabbers Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0, 0, 0, "default", 1, "default", 1, "default", "nts c", "0", 0, 1, hx_fghitachi) Call Op.OpenFramegrabber("DFG LC", 1, 1, 0, 0 0, 0, "default", 1, "default", 1, "default", "nts c", "2", 0, 1, hx_fgimagingsource) 'Call Op.ReadShapeModel("c:/Michael/Pic/Cross1.smd", hx_ModelID1) 'Call Op.ReadShapeModel("c:/Michael/Pic/Cross2.smd", hx_ModelID2) 'Call Op.ReadShapeModel("c:/Michae l/Crossobject.smd", hx_ModelID) 'Dim end_val4 As Variant 'Dim step_val4 As Variant 'end_val4 = 200 'step_val4 = 1 For hx_i = 1 To end_val4 Step step_val4

PAGE 106

9 5 A p p e n d i x A (C o n t i n u e d ) Call Op.GrabImage(hx_Image, hx_FGHandle) Call Op.FindScaledShapeModel(hx _Image, hx_ModelID2, 0, 3.14, 0.5, 1.1, 0.3, 1, 0.9, "interpol ation", 0, 0.2, hx_Row, hx_Column, hx_Angle, hx_Scale, hx_Score) If Tuple.TupleGreater(hx_Scale, 1#) Then Call Op.FindScaledShapeModel(hx_Image, hx_ModelID1, 0, 3.14, 0.5, 1.5, 0.3, 1, 0.9, "inte rpolation", 0, 0.2, hx_Row1, hx_Column1, hx_Angle1, hx_Scale1, hx_Score1) If Tuple.TupleGreater(hx_Scale1, 1#) Then Call Op.FindScaledShapeModel(hx_Image, hx_ModelID, 0, 3.14, 0.5, 1.1, 0.3, 1, 0.9, "i nterpolation", 0, 0.2, hx_Row2, hx_ Column2, hx_Angle2, hx_Scale2, hx_Score2) End If End If Please note: The call of DoEvents() is only a hack to enable VB to react on events. Please change the code so that it can handle events in a standard way. DoEvents Next ObtainShapeMode l 2 End Sub Sub InitHalcon() Default settings used in HDevelop Call Op.SetSystem("do_low_error", "false") End Sub