USF Libraries
USF Digital Collections

The construct validation of an instrument based on students university choice and their perceptions of professor effecti...

MISSING IMAGE

Material Information

Title:
The construct validation of an instrument based on students university choice and their perceptions of professor effectiveness and academic reputation at the University of Los Andes
Physical Description:
Book
Language:
English
Creator:
Montilla, Josefa Maria
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Enrollment motives
Teaching evaluation
University image
Validity
Construct validity
Dissertations, Academic -- Measurement and Evaluation -- Doctoral -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: The purpose of this study was to examine the construct validation of an instrument based on students university choice and their perceptions of professor effectiveness and academic reputation at the University of Los Andes (ULA). Moreover, a comparative analysis was carried out to determine how the selected factors that influence the students decisions and perceptions differ according to student demographic factors such as: gender and university campus. This instrument was developed with items based on the three domains formulated: university choice process, professor effectiveness, and university academic reputation. To determine the instruments appropriateness to measure the students decisions in university choice process and their perceptions about professor effectiveness and university academic reputation at the ULA, this research examined the reliability of scores by domains and factors across domains.The participants were undergraduate students who were registered in the second semester of 2002 and enrolled in the different courses by college within the ULAs main campus, which consists of ten colleges throughout the city of Merida, and within the other two university branch campuses in Tachira and Trujillo. For purposes of this research, a stratified probability sample was used to select the participants. The data show that the instrument designed has adequate internal consistency reliability estimates (all the domains exceeded .70). The confirmatory factor analysis shows that the overall fit indices revealed values at or close to the acceptable range .90, even when the model has statistically significant chi-square and demonstrates significant problems with some of the standardized residuals, which indicates that the fit of the model could possibly be significantly improved.The modified model revealed a relatively small improvement in the overall goodness of fit. These results provide supportive evidence of construct validity. Finally, the multivariate analyses of variance using gender and university campus as the predictor variables revealed a nonsignificant gender effect and a significant university campus effect, respectively. The Tukey multiple comparison test used to determine university campus differences across the domains showed approximately similar results, although they are separate and distinguishable. ULA-Merida established the highest mean scores when they are compared on the factors that influence their decisions in university choice process and their perceptions about professor effectiveness and university academic reputation, and the campus 1 (NURR-Trujillo) show the smaller mean scores.
Thesis:
Thesis (Ph.D.)--University of South Florida, 2004.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Josefa Maria Montilla.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 201 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001709542
oclc - 69019189
usfldc doi - E14-SFE0001298
usfldc handle - e14.1298
System ID:
SFS0025619:00001


This item is only available as the following downloads:


Full Text

PAGE 1

The Construct Validation of an Instrument Based on Students’ University Choice and their Perceptions of Professor Effectiv eness and Academic Reputation at the University of Los Andes by Josefa Maria Montilla A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Measurement & Research College of Education University of South Florida Major Professor: Jeffrey Kromrey, Ph.D. Bruce Hall, Ed.D. John Ferron, Ph.D. Madhabi Chatterji, Ph.D. Date of Approval: December 3, 2004 Keywords: Enrollment motives; teaching evaluation; university image; validity; construct validity Copyright 2005, Josefa Maria Montilla

PAGE 2

Acknowledgements I would like to acknowledge those whom have helped me in achieving this amazing goal. My deepest appreciation goe s out to those people who have been supportive and encouraging throughout my studies. I would like to give a special thank you to my major professor Dr. Jeffrey Kromrey. He is a source and meaning of ha rd work, determination, and persistence. Thank you for all of the continuous dedicati on, patience, toleran ce, encouragement, mentoring, and guidance and for the setting st raight for both my pr ogram and dissertation research. Your professional and personal intelligence and understanding will always be of value to me. It is with amazing fortune that I was blessed with the supportive and guiding relationship with my major professor Dr. Kromrey. A sincere appreciation is extended to my other committee members: Dr. Bruce Hall, Dr. John Ferron, and Dr. Madhabi Chat terji, whose command of quantitative research techniques, and their prudent a nd timely advice changes have gratefully enhanced this research. Thank you fo r your participant a nd supportive guide. Beyond my committee, I will never forget the supportive and patience help of Lisa and Allyson, my esteemed friends. Thank you so much. To the outside reviewers of this re search, my husband, daughters, mother and brother, who gave me love and strength through all of the highlights and difficulties beside of this doctoral program.

PAGE 3

Dedication This dissertation is dedicated to thos e whom have loved, encouraged, supported, and guided me through the extensive and inte nsive process of writing this document. To my daughters: Diliana and Loreana for your love, support, and encouragement in the pursuit of this doctoral degree.

PAGE 4

i Table of Contents List of Tables................................................................................................................. ....iii List of Figures................................................................................................................ ......v Abstract....................................................................................................................... .......vi Chapter One Introduction....................................................................................................1 Statement of the Problem............................................................................................8 Purposes of the Study................................................................................................10 Research Questions...................................................................................................10 Significance of the Study..........................................................................................11 Assumptions of the Study.........................................................................................13 Definition of Terms...................................................................................................13 Delimitations of the Study........................................................................................15 Chapter Two Review of Related Literature.......................................................................17 General Overview of Student Ratings......................................................................18 Student’s University Choice.....................................................................................19 Summary..........................................................................................................30 Professor Effectiveness.............................................................................................31 Summary..........................................................................................................39 University Academic Reputation..............................................................................39 Summary..........................................................................................................45 Construct Validation: General Overview..................................................................45 Theoretical Rational for Ge nder and Campus Differences.......................................53 Summary..........................................................................................................56 Chapter Three Methods......................................................................................................58 Research Questions...................................................................................................58 Target Population......................................................................................................59 Sample Design..........................................................................................................60 Data Collection.........................................................................................................65 Instrumentation.........................................................................................................66 Validity............................................................................................................69 Pilot Study........................................................................................................72 Reliability.........................................................................................................75 Data Analysis............................................................................................................79

PAGE 5

ii Chapter Four Results Analysis...........................................................................................94 Descriptive Statistics.................................................................................................95 Reliability................................................................................................................104 Research Question #1: Reliability.................................................................104 Confirmatory Factor Analysis.................................................................................106 Confirmatory Factor Analysis by Domains.....................................................................109 Research Question #2: Five-First-Order Factors...........................................109 Specification, Identification, and Estimation........................................109 Assessment of Fit..................................................................................113 Research Question #3: Four-First-Order Factors...........................................121 Specification, Identification, and Estimation........................................121 Assessment of Fit..................................................................................124 Research Question #4 Three-First-Order Factors..........................................129 Specification, Identification, and Estimation.......................................130 Assessment of Fit..................................................................................132 Multivariate Analysis of Variance..........................................................................137 Research Question #5: Differences Across Gender.......................................140 Research Question #6: Differences Across University Campuses................142 Summary...............................................................................................150 Chapter Five Conclusions and Recommendations..........................................................152 Research Questions.................................................................................................152 Summary of Methods..............................................................................................153 Conclusions and Recommendations.......................................................................155 Recommendations for Future Research..................................................................163 References..................................................................................................................... ...166 Appendices..................................................................................................................... ..175 Appendix A: Programs of Study.............................................................................176 Appendix B: IRB Suggestions................................................................................179 Appendix C: Student Enrollments at the University of Los Andes........................181 Appendix D: Letter of the Secretar y of the Universidad de Los Andes.................182 Appendix E: English Version of Survey Instrument..............................................183 Appendix F: Spanish Version of Survey Instrument..............................................187 About the Author...................................................................................................End Page

PAGE 6

iii List of Tables Table 1 Number of Students by Colle ge, University Branch Campus, Course, and Semester.......................................................................................62 Table 2 Items Description for Univ ersity Choice Process, Professor Effectiveness, and University Academic Reputation.......................................75 Table 3 Internal Consistency Reliability by Domains and Factors...............................77 Table 4 Students’ Dem ographic Information................................................................97 Table 5 Items Means and Standard Deviations by Domains........................................99 Table 6 Ratings of Importance Factors by Domain....................................................102 Table 7 Internal Consistency Reliability by Domains and Factors Across Domains.............................................................................................104 Table 8 Skewness and Kurtosis Coefficients by Domains.........................................107 Table 9 Factor Loading, t-Values, St andard Errors and Error Variance Estimates Related to Students’ Decisions to Select the ULA........................110 Table 10 Interfactor Co rrelation, Standard Errors and t-Values for University Choice Process.............................................................................112 Table 11 Goodness of Fit Indices for the Model in University Choice Process...........115 Table 12 Goodness of Fit Indices for the Modified Model in University Choice Process...............................................................................................118 Table 13 Factor Loading, t-Values, St andard Error, and Error Variance Estimates in Students’ Percepti ons of Professor Effectiveness.....................122 Table 14 Goodness of Fit Indices for th e Model in Professo r Effectiveness................126 Table 15 Factor Loading, t-Values, St andard Error, and Error Variance Estimates in Students’ Perceptions of Academic Reputation........................130

PAGE 7

iv Table 16 Goodness of Fit Indices for the Model in Academic Reputation...................134 Table 17 Multivariate Skewness and Kurtosis by Campus and Domains....................139 Table 18 Multivariate Analyses of Variance of University Choice Process, Professor Effectiveness, and University Academic Reputation....................141 Table 19 Factor Mean and Standard Deviations Across Domains by Gender..............143 Table 20 Analyses of Variance of Univ ersity Academic Reputation by Campus........145 Table 21 Factor Mean and Standard Deviations Across Domains by Campus............146 Table 22 Tukey Multiple Compar ison Testfor Factors by Domains............................148

PAGE 8

v List of Figures Figure 1. Five-First-Order Confirmatory Factor Analysis Model of Students University Choice Process, with Different Indicators per Factor....................82 Figure 2. Four-First-Order Confirmatory Factor Analysis Model of Students Perceptions of Professor Effectiv eness, with Different Indicators per Factor.........................................................................................................84 Figure 3. Three-First-Order Confirmatory Factor Analysis Model of Students University Academic Reputat ion, with Different Indicators per Factor.........................................................................................................85 Figure 4. Estimates Data for Five-First-Order Factor Model Related to Students’ Decisions of University Choice Process........................................114 Figure 5. Estimates Data for Four-Fir st-Order Factor Model Related to Students’ Perceptions of Professor Effectiveness..........................................124 Figure 6. Estimates Data for Four-Fir st-Order Factor Model Related to Students’ Perceptions of Un iversity Academic Reputation...........................132

PAGE 9

vi The Construct Validation of an Instrument Based on Students’ University Choice and Their Perceptions of Professor Eff ectiveness and Academic Reputation at The University of Los Andes Josefa Maria Montilla ABSTRACT The purpose of this study was to examine the construct validation of an instrument based on students’ university choice and thei r perceptions of profe ssor effectiveness and academic reputation at the University of Lo s Andes (ULA). Moreover, a comparative analysis was carried out to determine how the selected factors that influence the students’ decisions and perceptions differ according to student demographic factors such as: gender and university campus. This instrument was developed with ite ms based on the three domains formulated: university choice process, professor effectiv eness, and university academic reputation. To determine the instrument’s appropriatene ss to measure the students’ decisions in university choice process and their percep tions about professor effectiveness and university academic reputation at the ULA, this research examined the reliability of scores by domains and factors across domains. The participants were un dergraduate students who were registered in the second semester of 2002 and enrolled in the different courses by college w ithin the ULA’s main

PAGE 10

vii campus, which consists of ten colleges thr oughout the city of Meri da, and within the other two university branch campuses in Tach ira and Trujillo. For purposes of this research, a stratified probability sample was used to select the participants. The data show that the instrument de signed has adequate in ternal consistency reliability estimates (all the domains exceeded .70). The confirmatory factor analysis shows that the overall fit indices revealed valu es at or close to the acceptable range .90, even when the model has statistically signifi cant chi-square and demonstrates significant problems with some of the standardized resi duals, which indicates that the fit of the model could possibly be significantly impr oved. The modified model revealed a relatively small improvement in the overall goodness of fit. Th ese results provide supportive evidence of construct validity. Finally, the multivariate analyses of vari ance using gender and university campus as the predictor variables revealed a nonsignifican t gender effect and a significant university campus effect, respectively. The Tukey multiple comparison test used to determine university campus differences across the domai ns showed approximately similar results, although they are separate and distinguishabl e. ULA-Merida established the highest mean scores when they are compared on the factors that influence their decisions in university choice process and their percep tions about professor effectiveness and university academic reputation, and the cam pus 1 (NURR-Trujillo) show the smaller mean scores.

PAGE 11

1 Chapter One Introduction The mission of a university is carried out for the transmission of knowledge through teaching, scientific research, and the study of regional and national problems and the development of alternative solutions One of the fundamental objectives of the university is to promote and stimulate the de velopment of academic activities in order to improve academic excellence. To assure such excellence, the university must establish and maintain higher standards for its student s and professors. Therefore, high quality teaching and research must head the list of priorities. Traditionally students and professors have been considered the central constituency in the Venezuelan higher educati on system. University students have been given a tremendous amount of attention, since their academic quality is important to the character of their institution and is often seen as an organizational resource, a measure of institutional quality, and as a source of institutional change. Consequently, student ratings are considered essential to the succe ss of a university, given that they contribute to and help describe the culture and stat us of the higher education institutions. Student decisions on university choice a nd student perceptio ns of professor effectiveness and university academic reput ation are important concerns for higher education institutions. When students finish high school, they are faced with the decision to enter a university. If they decide to attend a univ ersity, the next decision is about which university to choose. In the university choice research there have been three basic approaches to the study of university choi ce decision-making influences: 1) social

PAGE 12

2 psychological studies, which consider the impact of academic program, campus social climate, cost, location, and influences of othe rs in students’ choices ; student’s assessment of their fit with their chosen college; a nd the cognitive stages of college choice; 2) economic studies, which are based on the idea that a student maximizes a utility in their university choice and view th e university choice as an investment decision; 3) sociological status attainment studies, which analyze the impa ct of the individual’s social status on the development of aspirations for educational attainments (McDonough, 1997). The relative importance of these factors is determined by the characteristics of the specific university and the students. Thus, students’ decisions to enroll at a university should be based on these factors that li nk to their characteristics and needs. Professor effectiveness and academic reput ation are two of the most decisive factors that are dramatically increasing in importance for enrollment decisions (Delaney, 1998; Trusheim, Crouse, & Middaugh, 1990). Because these factors dominate the university choice literature, it is of special interest to l ook at students’ perceptions and what factors most influence professor effec tiveness and university academic reputation. Until recently no study related to student decisions to enroll at a university and their perception about academic reputation and professor effectiveness had been carried out in a Venezuelan university. Consequently, in this univers ity little is known about the actual enrollment motives of students, as well as the pers pective of students in relation to academic reputation and professor effectiveness. Some of the reas ons for the lack of research about these concerns are the enrollme nts that always have been growing and the lack of interest of the univer sity researchers to examine the theories about these important concerns to the higher education institutions. Therefore, the university administrators

PAGE 13

3 were also not particularly worried about th e specific influences on students’ university choice and their perceptions about professo r effectiveness, and university academic reputation. Given the growing trend in enrollment rates at Venezuelan universities and particularly at the University of Los Andes (ULA), and because in Venezuelan universities the investiga tion of these concerns is still in its infancy, there is a need for greater understanding of why st udents choose to attend a unive rsity, what variables have a strong impact on student’s de cision to enroll at that university, and what professor effectiveness and university’s academic reputat ion means from the student’s perspective at the University of Los Andes. Conseque ntly, the development of an instrument to measure students’ decisions of university c hoice and their perceptions of professor effectiveness and academic reputation is an important concern because it identifies strengths and weakness that guide the decisions related to the university goals and policies. In addition, considering th e importance that gender differences and university campus as demographic characteristics have had in students’ beha vior, this study also examined whether the students’ decisions of university choice process and their perceptions about professor effectiveness and university academic reputation are equally shared by gender and campus. Validity Generally “validity refers to the appropr iateness, meaningful ness, and usefulness of the specific inferences made from test scores” (American Educational Research Association, American Psyc hological Association, & Na tional Council on Measurement

PAGE 14

4 in Education, 1985, p. 8). Cronbach (1971) al so described validity as the process by which a test developer or test user collects evidence to s upport the types of inferences that are to be drawn from test scores. Validity also refers to the degree to which evidence and theory support the inte rpretations of test scores entailed by proposed uses of tests. ... The process of validation involves accumula ting evidence to provide a sound scientific basis for the proposed score interpretations ( The Standards for Educational and Psychological Testing, 1999). The 1985 and 1999 Standards for Educational and Psychological Testing, written by the Joint Committee on Educational and Psychological Test (AERA, APA, and NCME) recognize three different ways to gather evidence about the valid ity of test scores inference: content related evid ence, criterion related evidence and construct evidence of validity. Content validity refers to the de gree to which the scor es yielded by a test adequately represent the conceptual domain that these scores propose to measure; criterion validity refers to th e extent to which the test sc ores on a measuring instrument are related to an independent external crite rion (relevant, reliable) believed to measure directly the behavior or charac teristic in question; and cons truct validity refers to the extent to which a particular te st can be shown to assess the construct that it purports to measure. However, given that in this study the instrument is designed to measure the students’ decisions to enroll at a university and their perceptions about professor effectiveness and university academic reputa tion, the validity estimation is focused basically toward content and construct validity. Content va lidity evidence is usually gathered and examined carefully and critica lly by expert judges to determine if the

PAGE 15

5 content and objectives measured by the test are representative of those that constitute the content domain. Construct validity study invo lves several steps: formulating hypotheses based on the theoretical underpinnings of the construct; designing the study to allow for a test of the hypothesis; gathering and analyzi ng the data; and determin e if the results most likely support the formulated hypothesis or not (Crocker & Algina, 1986). There is no integrated approach us ed to gather evidence for the construct interpretations of a test. Some of the most common approaches used to establishing the construct validity of score in terpretations are: the logi cal method, the correlational method, and the experimental method. The main aspects of the logical approach include asking if the elements the test measures are those that structure the construct and checking the items to determine if they seem appropriate for assessing the elements in the construct. One aspect of the correlationa l approach to gather ing construct related evidence includes correlations between a m easure of the construct measure and other designed measures. When the correlation is high, one assumes evidence of construct validity. Another aspect of the corre lational approach is the f actor analysis, which is a statistical procedure for studyi ng the intercorrelation among a se t of test scores with the purpose of determine the number of factors or constructs required to account for the intercorrelations, and the per centage of variance accounted fo r by the factors. Results from factor analysis studies contribute to demonstrate evidence for the construct validity of an instrument. As stated earlier, in Venezuelan higher e ducation institutions, there is a lack of adequate instruments that permit to measur e students’ decisions of university choice

PAGE 16

6 process and their perceptions about profe ssor effectiveness and university academic reputation, therefore theory about these topics is needs to be used in order to construct instruments. The University of Los Andes The Venezuelan university system has constituted a fundame ntal factor in government plans for development. One of the primary constitutional precepts of Venezuelan Educational Law is the provision th at public education is free at all levels and that the Venezuelan State has the obliga tion to provide it. The education levels established free by the Venezuelan Educational Law are: a) primary, which includes the initial education (maternal a nd preschool) and elementary school (from first to sixth grade); b) secondary, which involves the middl e school (from seventh to ninth) and high school (from tenth to eleventh); and supe rior, which includes the technological and college education. The main sources of f unds for Venezuelan uni versities are obtained from the government. It is one of the reasons that explain why the institutions of higher education have been controlled and stri ctly supervised by th e central government according to uniform, nationwide standards. Higher education in Venezuela has b een marked by shifting patterns of enrollments and resource allocations due to the effect of a vari ety of larger socioeconomic and political forces. One result of these conditions is intensified competition for university students. These conditions ha ve made it difficult for higher education institutions to manage both the quantity and quality of their student populations. Student enrollment in Venezuelan higher education in stitutions has increas ed continuously over time, 66 % from 441,734 in 1987 to 773,294 st udents in 1997 (National Council of

PAGE 17

7 Universities, OPSU, 1998). The University of Los Andes has presented a growth in the number of student enrollments of 39.1 % from 24,359 in 1996 to 33,874 in 2002 (OCRE, 2002). With respect to the admission’s policy of the university, it is important to point out that due to the higher education in Venezu ela, in its majority, is free and the demand exceeds a couple of times the supply capacity. The ULA has established a restrictive admission’s policy under the following modalities : a national test (OPSU) and an internal test (PINA) of university admission (24.5 % and 57.5 %, re spectively); an d a special admission (18.0 %), which includes academic, artistic and sports excellence; high performance, and union agreement (gro wn children of university staff). The University of Los Andes (ULA) is a public, autonomous, and national institution with international tr anscendence, that take up the first position in research and the second position in number of students a nd academic reputation, within the group the higher education institutions in Venezuela The goals of this university have been directed to incorporate the institution in the global context of our country under the conditions of permanent and reciprocal coope ration. The supreme authority of the ULA reside in its university counc il, which is to exercise the government functions in accordance with its respective at tributions established in th e university law, article 26 (Republic of Venezuela Congress, 1970). The university council is composed of the rector, two vice rectors, one secretary, the deans of the faculties (colleges), five representatives from the teaching staff and th ree from the students, one from the alumni of the university and one delegate fr om the Ministry of Education.

PAGE 18

8 The University of Los Andes consists of three campuses: a main campus, which is located in the City of Merida, an ur ban area with a population of 686,709, and two branch campuses, which are located in the cities of Tachira with a population of 944,259 and Trujillo with a populati on of 562,762 (Presidency of the Venezuela republic, OCEI, 1995). The ULA’s total physical structur e is over 410,000 square meters. The ULA consists of over 3,500 professors serving a student population of 33,874, approximately. This University (ULA) has an academic structure that is constituted of ten colleges located in the main campus and two university branch campuses (Tachira and Trujillo), which are responsible to the uni versity council, operati ng in turn through two bodies, the faculty assembly and the faculty council. Th e colleges are composed of schools (30), which are subdivided into departments (122), de pending on different disciplines. Aside from the academic productivi ty, the university cons ists of important research institutes (13), cent ers (42), laboratories (17), a nd research groups (198), which make the ULA among the most important institutions of higher education in the country. ULA’s teaching and research are carried out in the following areas: Basic Sciences, Engineering, Architect ure, Agricultural and Envi ronmental Sciences, Health Sciences, Education, Humanities, Social Sciences, and Literature and Arts. The ULA offers a total of 51 degrees at the undergradua te level and 140 at th e graduate level (61 master degrees, 60 specialist degr ees, and 19 doctorate degrees). Statement of the Problem Usually information about student ratings is considered import ant to the success of a higher education institution, since they contributed to examine the strengths and

PAGE 19

9 weakness of the university policies. In the Venezuelan universities, the development of instruments to measure students’ decisions of university choice and their perceptions of professor effectiveness and university academic reputation have often been methodologically weak, given that these institu tions do not carry out a true and concrete policy of institutional evaluation. Conse quently, the lack of knowledge of these interested concerns by the unive rsity authorities or administ rators might lead to their misallocating resources when they are at tempting to improve their academic quality. Students’ decisions of university choi ce and their perceptions of professor effectiveness and university academic reputa tion are important concerns that help to understand different aspects of the students’ ro le within higher education institutions. So, research on student and professo r populations should be of grea t interest to the University of Los Andes, and therefore substantial attent ion must be given to studies that address factors influencing students’ de cisions to select a university and their perceptions about professor effectiveness and university academic reputation. A second concern of this study is the util ity of the information from the student ratings about these concerns for multiple polic y development, such as: course refinement, program assessment, faculty evaluation, and institutional evaluation, which allow to orient the decisions by university authorit ies on the institutional mission and policy that permit differentiate the institution across the higher education system. Additionally, potential gender and univers ity campus differences associated with students’ decisions in univer sity choice process and their perceptions about professor effectiveness and university academic reputa tion are important concerns in research.

PAGE 20

10 These results should be used in supporting un iversity decision maki ng, in order to attend and improve those university concerns that should show the lowest student ratings. Purposes of the Study The purpose of this research was to gath er construct validation evidence for an instrument designed to measur e students’ university choice process and thei r perceptions about professor effectiveness and university acad emic reputation at th e University of Los Andes. Additionally, a comparative analys is was carried out to determine how the selected factors that influence the studen ts’ university choice process and their perceptions of professor e ffectiveness and university academic reputation differ according to student gender and university campus. Research Questions Six research questions examined data collection and anal ysis on students’. decisions and perceptions in university choi ce process, and professor effectiveness and university academic reputation, respectively. 1. Are the student’s decisions of university choice process, and student’s perceptions of professor effectiveness and university a cademic reputation reliable within their respective factors at the Un iversity of Los Andes? 2. How well does the hypothesized measurem ent model involving five-first-order factors fit the observed da ta based on students’ de cisions to enroll at the University of Los Andes?

PAGE 21

11 3. How well does the hypothesized measurem ent model involving four-first-order factors fit the observed data based on the students’ pe rceptions about professor’s effectiveness at the University of Los Andes? 4. How well does the hypothesized measuremen t model involving three-first-order factors fit the observed da ta based on students’ per ceptions of university’s academic reputation at the University of Los Andes? 5. What are the differences across gender in perceived importance of the selected factors that influence the students’ deci sions about university choice process, and their perceptions of profe ssor’s effectiveness and university’s academic reputation at the University of Los Andes? 6. What are the differences across univers ity campuses in perceived importance of the selected factors that influence the st udents’ decisions abou t university choice process, and their perceptions of prof essor’s effectivene ss and university’s academic reputation at the University of Los Andes? In order to answer these research ques tions a 65-item survey instrument was developed, which solicits demographic inform ation and information related to students’ decisions to select the ULA and students’ pe rceptions about profe ssor effectiveness and university academic reputation. The survey was administered to students who were registered in the second semester of 2002. Significance of the Study Previously was stated that until recently no study related to st udent decisions to enroll at a university and their percep tion concerning professor effectiveness and

PAGE 22

12 academic reputation had been carried out in the Venezuelan university. Therefore, one contribution of this study was to provide an understanding of why students choose to attend the University of Los Andes and what factors have a strong impact on students’ decisions. Consequently, these results wi ll provide direction in improving student recruitment and university policies that can be used in overall educational planning decisions. In addition, the review of this research has suggested that professor effectiveness and university’s academic reputation have b een two of the most important factors in deciding to enroll at a university. Conse quently this study will contribute by examining these previously established concerns for maki ng decisions of the university authorities. The results related to students’ perceptions of professor effectiveness should be of great importance to the academic advocate co mmittee, who are responsible for evaluating faculty with respect to the execution of thei r dedication time; to the teaching faculty, by providing a feedback system on student’s percep tions of their teaching ability; and to the students, who seek information about th eir professor selection and courses. In the same way, the findings related to students’ perceptions of university academic reputation might be successfully favor able to the institution in order to keep campus appearance from a point of view of values and aesthetic appeal, prestige associated to alumni and professor qualit y, and prestige related to researches and publications. Obviously the University of Los Andes must continuously be concerned with understanding of why students c hoose to attend this university and what factors have a strong impact on students’ decisions, and dete rmining the professor’s effectiveness and

PAGE 23

13 its reputation or prestige from the studen t perceptions and through appropriate means work, in order to reinforce the results of its findings. Consequently one contribution of this study will be to provide to university authorities with a theoretically based instrument on these fundamental concerns. Consequently, this investigation should be useful for policy-making purposes (planning, rationally establishing priorities in al locating resources among the many disciplines of the university) and should make a significant contribution to the University of Los Andes. This research represents a first step in developing evaluation programs to improve learning and teaching through a student-to-professor feedback system. Assumptions of the Study For the purpose of this study the following assumptions were made: 1. The students included in the study were cons idered representative of all students at the University of Los Andes. 2. The responses of the students were considered as honest and sincere of their decisions and perceptions related to the University of Los Andes. Definition of Terms For the purpose of this study, the following definitions were used: University choice University choice is a process based on organizational theories of decision making to highlight the importance of divers ity of organizational contexts and status culture background on student decision-making (McDonough, 1997).

PAGE 24

14 Professor effectiveness There are numerous and different con ceptions existing that are related to professor effectiveness. It can be defined as an index of success or as an index of effectiveness for that professor. Professor effectiveness is a characteristic of those professors that meet the stakeholders’ needs. An effective professor is one who provides an atmosphere, which will foster desirable grow th in students, measured in terms of the objectives of education (Erickson, 1954). University reputation There are also different conceptions rela ted to reputation, for example, Sevier (1994) defines universit y reputation as a set of attitude s or beliefs that a person or audience holds about an institution. Re putation and image are sometimes used interchangeably in the literature. Image is an interpretation, a set of inferences, and reactions. It is a symbol because it is not the ob ject itself, but refers to and stands for it. The reputation includes its meanings-beliefs-attitu des, and feelings that have come to be attached to it. These meanings are learne d by component experiences people have with the product and these compone nts are particular symbol s (Sidney 1978, in Huddleston and Karr, 1982). Although there is not a clear differen ce between reputation and image, it is assumed that they are practically admitted as denominations of a same quality. In The Living Webster Encyclopedic Dictionary of the English Language (1974), reputation is defined as a good or honorable name, as an opinion of character generally held. The reputation is a distinct ion attributed to someone or something. However, image is the

PAGE 25

15 composite public impression of a person, due to its known procedures, philosophy, and values. Validity Generally “validity refers to the appropr iateness, meaningful ness, and usefulness of the specific inferences made from test scores” (American Psychological Association, American Educational Research Associat ion, and National Council on Measurement in Education, 1985, p. 8). Validity also is de scribed as the process by which a test developer or test user collect s evidence to support the types of inferences that are to be drawn from test scores Cronbach (1971). Construct Validity Construct validity refers to the extent to which a particular test can be shown to assess the construct that it pur ports to measure. It is a process that involves a group of methods for assessing the degree to which the instrument measur es the theoretical construct (Cronbach and Meehl, 1955). Delimitations of the Study The extent of this study is designe d within the following restrictions: 1. The students of this study are found at the University of Los Andes in three distinct campuses: Main campus ULAMerida (75.4); The “Rafael Rangel” University Branch Campus: “NURR-Truji llo” (13.2); and The T achira University Branch Campus: “NUTULA-Tachira” (11.4). 2. The study was restricted to University of Los Andes (Merida-Venezuela), a public, autonomous, and national institution with international transcendence, that

PAGE 26

16 take up the first position in research and the second position en number of students and academic reputation, with in the group the higher education institutions in Venezuela

PAGE 27

17 Chapter Two Review of Related Literature The review of literature presented in th is study provides a theoretical and practical rational for the construct validation for an instrument designed to measure students’ university choice process and their percep tions about professor effectiveness and university academic reputation. This review is divided into five secti ons: the first section is offered a general overview of student ratings. The second sect ion is associated with student’s university choice process; it presents an overview of the theories and models related to student university choice, and the set of variables that have been found to be consistently influential; the third section is related to professor effectiveness, it is examined over the criteria of professor effectiveness and eval uation, as well empirical studies related to students’ perceptions of professor effectivenes s; the fourth section related to university academic reputation is examined over theori es and criteria of university image or reputation, university quality and methods th at may serve as conceptual bases for the academic reputation of an institution; and fi nally, the last section is associated to construct validation, which incl udes a general overview as well as a review on the most common methods used to gather eviden ce for the construct validity of score interpretations. In addition, empirical studi es based on these concer ns are presented. As stated earlier, for the reason that the research in Venezuelan higher education on why students select a particul ar university and what fact ors have had a strong impact on their decision, as well as the student perc eptions concerning pr ofessor effectiveness

PAGE 28

18 and academic reputation, is still in its infancy, reviews of the literature presented in this study are directly related to researches on th ese concerns in American higher education institutions. General Overview of Student Ratings During the last decades stude nt ratings have been gi ven a tremendous amount of attention in educational literatures, since th ey have been viewed by the manner in which their decisions and perceptions about educationa l concerns contribute to and help explain the success or status of educational system s of higher education. In addition, students perceptions have been considered very impor tant in any investiga tion, since the students are in the institution almost every day and they be acquainted with what is going on. Student ratings related to educationa l area (teaching evaluation, professor effectiveness, university imag e or reputation, university choice process, etc.) have been probably the most systematically studied of all forms of personnel evaluation, and one of the best in terms of being supported by empi rical research. Many researcher have noted that student ratings are importa nt concerns in the higher education system, for example: Clark, 1970 comments that “students are important to the character of their institution” (p.253) and besides “the student body becomes a major force in defining the institution” (ibid); Astin (1985) argues that the student and their perceived academic quality are often seen as an organizational resource and as a measure of institutional quality; McKeachie, 1997 (p. 1224) stated that student will continue to be those most affected by teaching. Therefore, student ratings will continue to be useful.

PAGE 29

19 The validity of student rati ngs has been systematically evaluated and usually supported in many literatures during the last years (Feldman, 1989; Marsh, 1987; Marsh & Bailey, 1991; Dey & Hurtado, 1995). Thes e reviews of research indicate great evidence supporting the validity of student ratings. Conversely, although student rating have been received a great deal of attention, they also have been criticized, since they have been viewed by many researchers as individual attitudes, which may be defined as the importance an individual attaches to a specific attribute of a college or university and the belief that a specific institution possesses that attribute (T rushein, Crouse, & Middaugh, 1990). Consequently, it indicates that these ratings on educational areas di ffer among students due to their different attitudes/perceptions. Students’ University Choice Researchers of higher education have overtly expressed their opinions, and presented theories and models in numerous pr ofessional literatures on the issue of student university/college choice process. University/college choice is often defined as a process based on organizational theories of decisi on making to highlight the importance of diversity of organizational contexts and st atus culture backgro und on student decisionmaking, which provide insight into how a nd why a university/ college context can influence student behavior (McDonough, 1997). Murphy (1981) charact erizes university choice as a process that can be viewed fr om the consumer buying roles to guide the strategic decisions in university/ college choice. He considers that the different roles that different individuals assume in the decision pr ocess can be identified as: user role (e. g.,

PAGE 30

20 students), influencer role (e. g., family, friends, high school counselors, and other relatives), and decider role (e. g., this role is a crucial one and it should be determined to what extend the decision is a joint one between parent and the student, parent or an individual one made by the stud ent, parent or some other pe rson). Thus, the application of the buying center roles to th e choice of university or colleg e represents an appropriate practice of this theoretical s uggestion. In additio n, university/college choice is a process that has been situated in the social, cultura l, and organizational context, as well as the marketing perspective. However, despite the fact that little is know about th e actual enrollment motives of students, findings from thes e studies have documented the influences of demographic, status, parental, fixed institutional characteri stics and perceptual factors on students’ university/college choice deci sion process. Demographic variables include the student’s characteristics such as gender and age. Status variables include for example, the effect of family’s socioeconomic status, residence status and student’s ability. Parental variables include influences of parents, friends, a nd other family. The students are strongly influenced by the comments and recommendations of their friends and family, since their comments shape the students’ expectations of wh at a particular institution is like. Fixed institutional characteristics include loca tion, costs, scholarship programs, campus environment, and diversity of programs offeri ngs. Perceptual variab les include student’s perception of institutional quality, such as: professor, programs, teaching, research, and overall university/college. Other factors also have been found to be consistently influential in the university choice process such as: the university’s size, reputation, prestige, selectivity, phy sical facilities, guidance counselors, availability of financial aid,

PAGE 31

21 preparation for a good job, and liberal arts or ientation (Hamrick & Hossler, 1996; Harper & Hill, 1989; Maguire & Lay, 1981; Trusheim, Crouse, & Middaugh, 1990). However, the relative importance of these factors on students’ university/co llege choice process depends on individual attitudes; therefore, it indicates that the university/college choice decision process differs among persons (stude nts) due to their di fferent attitudes. University /college choice process also can be characterized into three basic approaches: Social Psychological, Economic, and Sociological. Social psychological studies which examine the impact of academic program, campus social climate, cost, location, and influence of others on students’ choices; students’ as sessment of their fit with their chosen college; and the cognitive stages of college choice. Economic studies which view college choice as an investment decision and assumes that students maximize perceived cost-benefits in their college c hoices; have perfect information; and are engaged in a process of rational choi ce. Sociological status attainment studies, which analyze the impact of the individual’s social status on the developmen t of aspirations for educational attainment and measure inequali ties in college access McDonough (1997). Many early and recent researchers ha ve focused on topics surrounding the student’s decision to enroll at a higher educa tion institution and their issues have been directly related. Early resear chers analyzed some factors th at prospective students use to evaluate and choose a university or a co llege. These studies tended to be purely descriptive, and were focusing on verbal repor ts or student explan ations of university choice process. For example, Greenshields (1957) designed a study to obtain from the students, in a free-response situation without the guidance of any suggestion, what in their opinions were the factors, which determ ined their college decision. The reasons

PAGE 32

22 most frequently stated were: preparati on for a good job and training for a specified vocation, which represented over a third of all reasons for students. Desire for a higher education institution ranked next in freque ncy. Ranking fourth was social education, such as: training for a more satisfactory soci al life, how to get al ong with others, and a wish to grow in social competence. Thes e variables contain 80 percent of all collegegoing reasons given by the students. He conc luded that it is worthy of notice that many students are thinking of a highe r education institution as a good place to be to improve their social maturity, learn how to be bette r citizens, develop th eir personalities, and satisfy their intellectual curiosity by lear ning more. The sociological hypothesis (a person is influenced in his attitudes a nd motivations by family, friends, and other relatives) was borne out of the results of this study. The parents make up the most important group influencing students, and teach er, principal, and counselor make up the second important influence. Similarly, Holland (1958) also examined st udent explanations of college choice and their relation to college popularity, coll ege productivity, and sex differences. The student explanations of colle ge choice were classified by sex. His research found a moderate similarity between the explanati ons of college choice reported by senior boys and senior girls. The findings showed that the actual choice of the institution was strongly influenced by such factors as in stitutional status, size location, religious affiliation, liberal arts orientation, closeness to home, and influences of friends. Other explanations of choice, reporte d in lesser percentage s, related to factors as the research reputation, coeducational status costs, and physical faciliti es. The trends common to both groups imply that the choice of a “mor e popular” institution reflected a greater

PAGE 33

23 concern with academic status, and the choice of a “less popular” institution revealed a greater concern with lower student socioeconomic status. These results also revealed that greater proportions of bright students with scientific goals attended institutions with high indices of productivity and the converse. Recent researchers also have focused on id entifying criteria using by student in selecting a higher educati on institution to attend. Fo r example, Murphy (1981) investigated the student “buying process” and states that uni versity choice is a process that can be viewed from the consumer buying roles to guide the st rategic decisions in university/ college choice. Hi s study on consumer buying roles in college choice showed that approximately 50 percent of both gr oups (parents’ and st udents’ perceptions) indicated that parents initiated the idea of going to college however, the majority of both groups indicated that the final decision on which college to attend was made by the student (decider). He came to the conclusion that the factors that relate to the marketing of college and university were academic reput ation, which was perceived to be the first most important factor, followed by price relate d issues, location of campus, closeness to home, size of campus, and parental opinions. Students seem to prefer higher quality colleges, but they would just as soon prefer to attend them for as low a net price as possible, additionally, financial aid influences their college choice behavior. Harper and Hill (1989) carried out a survey of a sample of students to determine factors, which the students perceived as having an influence upon their decisions to attend a particular universit y. Based upon the stated objec tives, the authors found that students majoring in agriculture are somewh at older than typical college students (average age: 23.2 years), and that few agricu lture majors intended to become agricultural

PAGE 34

24 producers. In addition, they found that the location of the univers ity was one of the factors of greatest influence fo r students. The next factors that student ranked as highly important in their college choice were closen ess to home, the influences of vocational agriculture teachers, friends and peers, and pa rents. Additional factors included the cost of tuition and program quality. The factors that had less influence for students were media recruitment campaigns, high school recr uitment visits, and university athletic programs. A study carried out by Blinn College (1994) revealed that the top five factors influencing students to attend Blinn College were: facilities (library, laboratories, computers, and recreational facilities), whic h were very highly rated as an influence factor by students on the campuses (67%); facu lty reputation was also an important factor in this process (66%). In a similar manner, Blinn’s academic reputation, size of institution and classes (63%), and costs (61%) were identified as being very influential in their decisions to attend Bli nn College. Additionally, althou gh counselors are still not a major influencing factor, the current surv ey findings showed an increment in the influencing role of high school counselors and teachers in their decision to attend this college. Other authors as Cleave-Hogg, McLean, and Cappe (1994) also examined what factors affecting applicants’ acceptance or de cline of offers to enroll in a particular institution. They surveyed tw o groups of students over a peri od of four consecutive years (1988-1991): group A had been accept ed at the medical school but had declined the offer; group B had been accepted and subsequently enrolled at the medical school. They found that the main factors influe ncing their decision to apply by group A were: reputation of

PAGE 35

25 the medical school, which was the most fre quently mentioned factor (84%), location, size: greater chance of acceptance, family, program, and research orientation; however, they declined the offer basically by the size of the class because they prefer smaller scale. The main factors influencing the decision of the group B were principally academic reputation (89%) and locati on since they lived within commuting distance of the university (69%). Delaney (1998) examined the relationship between parental income and student’s college choice process and identified factor s influencing the enrollment decisions of students from different income levels (highe r income and lower income categories). Both bivariate and multivariate statistical te chniques showed stat istically significant differences in academic, social, lifestyle, and financial aspects of the college. Students from higher income families attributed mo re importance to the college’s surrounding, such as the neighborhood and ge ographic location of the univers ity. In contrast, students from lower income families identified opportuniti es for internships as very important to their college choice; they also attribute somewhat more importance to the academic program available to them and the costs of attendance. Additionally, students from higher income families who perceived the college as challenging also rated their college of choice positively on academic reputation, qual ity of the faculty, major of interest, and perception of academic challenge. In contrast, students from lower income families who perceived the college as fun, comfortable, and friendly, also rated the college positively on surrounding, social life, extracurric ular activities, and cost. Other researchers have introduced seve ral models to the increased understanding of university/ college choice process. Although these models vary, they share a common

PAGE 36

26 nucleus of stages, since the university choice pr ocess involves several stages that take the students from wishing to atte nd it to actual enroll ment. For example, Chapman (1979) developed a model, which expresses that the probability that a student chooses a university/college is assumed to depend upon a ma trix of attributes or characteristics of the colleges in the student’s choice set (e. g., university/college quali ty, university/college size, etc.), a matrix of attributes that relate the student to the university/college in his/her choice set (e. g., financial aid, distance from th e home to the campus, etc.), and a vector of demographic and socioeconomic characterist ics associated with the student (such as sex, age, Scholastic Aptitude Test scores, etc.). Empirical results for estimating Chapman’s model, using a factor analysis proc edure led to the extract ion of six factors, which accounted for about 58% of the variance in the original 46 college raw variables. These factors were interpreted as: quality/affl uence, size/graduate orientation, masculine/ technical orientation, ruralness, fine arts orientations and liberalness. Chapman (1981) developed a conceptual model of student college choice, with the purpose to assist college administrators who are responsible for setting recruitment policy to identify the pressures and influen ces they need to consider in developing institutional recruiting policy, a nd to aid continued research in the area of student college choice. The results showed that student coll ege choice is influenced by a set of student characteristics with a series of external infl uences. Student characte ristics include factors such as socio-economic status, level of e ducational aspirations and expectations, and aptitude (high school achievement and pe rformance). External influences were categorized as significant pe rsons (e.g., family, friends, other students, counselors, teachers and college admission officers), the fixed college character istics (e.g., location,

PAGE 37

27 size, academic reputation, costs, availability of desired programs, and financial aid), and the institution’s communication efforts (e.g., mailings, brochures, and advertisements). Dembowski (1980) used a maximum likeli hood technique to illust rate a university specific methodology for predicting the probabi lity of a student entering a particular institution This probability is assumed to de pend on a vector of st udent characteristics (student’s income level, place of residen ce, sex, SAT scores, type of high school attended, rank in high school class, and an indi cation of his/her specia l interest), a vector of university/college admissions process comp onents that student expended (institution visits, interviews, talk with faculty memb ers, campus tours, and an open campus program), and a vector of the average scores of the characteristics of the other college choices of students (total enrollments, fres hman enrollments, number of faculty, number and type of majors, percentage of students that receive financial aid, and tuition for each college) and an error term. These results demonstrated that the admission process components of the university were influentia l in the student’s co llege choice decision process. Other influential variables were wh ether the student lived in New York State, and the student’s verbal SAT score. Jackson (1982), and Litten (1982) deve loped similar models that describe university choice as a developmental process. They have suggested that the student university choice process is divi ded into three phases: from an initial step of establishing a predisposition toward higher education to th e final step of selecting an institution to attend. Jackson’s three-phase model (1982) begi ns with a preference phase, which is an attitude toward university/colle ge enrollment that reflects sociological processes. The

PAGE 38

28 next stage is that of exclus ion (students consider their options), followed by a stage of evaluation (students evaluate their choice set and select an institution according to their judgments). The important variables in this model of student unive rsity/college choice are university/college and job attributes, co sts, family background, academic experience, and location. Litten’s three-phase model (1982) shar es some similarities with Jackson and Chapman’s model. The first stage of his mode l begins with the desi re to attend a higher education institution, followed by the decision to attend. The second stage includes the investigation of potential inst itutions of higher education. The final phase includes the application for admission followed by the actual admission and finally by the enrollment. The selected segmentation variab les included in this model are: racial groups, the sexes, ability groups, parents’ edu cation, and geographic location. Other studies have also focused on those models that could help assess the effects of university choice on student’s enrollm ent decision, for example the linear compensatory model developed by Cook and Zallocco (1983). This particular model “holds that an individual’s overall attitude toward a univers ity is a composite of his/her attitudes toward the many attr ibutes that a university po ssesses” (p. 200), and has two major components: importance values (the importance of individual attaches to the attribute) and beliefs (the i ndividual beliefs as to the ex tent attribute is offered by university). The implication of this model is that neither component can be ignored; thus the attitude of an individual toward a university is found by multiplying the importance value attached to the attribute times the belief that the university possesses the attribute. The set of variables that repr esent these attributes are a cademic reputation, specialized

PAGE 39

29 program of study, size of city, closeness to home, costs, university regulations, close faculty-student association, phys ical facilities, social acti vities, admission standards, financial aid, family influence, high school c ounselors, intercollegiat e athletic programs, intramural athletic programs, facilities, and college attendance plans of high school friends. In a later study based on the work of both Jackson (1982) and Litten (1982), Hossler and Gallagher (1987) also proposed a three-stage developmental model in which students move toward an increased understand ing of their educational options as they seek a postsecondary educational experience. This model show s at each phase influential factors such as individual a nd organizational factors interact to produce outcomes. It specifies those stages as predisposition (dev elopmental phase in which students decide whether to continue their education beyond high school), search (students search for general information about institutions), and c hoice (the students deci de which institution they will actually attend). In this model a number of variables have been found to be consistently influential: parents, univer sity’s size, location, academic program, reputation, prestige, student’s peers, friends and guidance co unselor, and availability of financial aid. Another study conducted by Trushei n, Crouse and Middaugh (1990) also extended the linear compensatory model developed by Cook and Zallocco in 1983 by investigating the importance of applicants’ a ttitudes about institutions, and by controlling for demographic and ability factors that may aff ect attitudes. Thus, the procedure used is based on a multi-attitude model that defines the attitude of an individual as the importance the individual attaches to a specifi c attribute of the inst itution, and the belief

PAGE 40

30 that it possesses that attribute. This model stat es that a student’s ove rall attitude about a particular university is a product of how important a specific item is and how much the student thinks that the univ ersity offers the item. The findings demonstrated that the overal l attitude measure had a moderate but significant relationship to college selection (.27), which indicates that applicants who differ by one standard deviation on the attitude scale are 27% more likely to enroll at the university. Similarly, attitude score about the university differed significantly between applicants who enrolled (X = 8.5) and applicants who did not enroll (X= 7.8; t = 10.4). A stepwise proce dure was employed to identify th e most important attributes that predict enrollment. The results revealed that 9 of the 18 attributes were statistically significant predictors of the enrollment decision (p .05). So, quality of academics, quality of programs, proximity to home, athl etic facilities, and the university’s general reputation were among the most important attrib utes that predict enrollment decisions. Summary In summary, there have been substantia l researches that have shown several factors to be consistently influential in the university/college choice process. Results of these researches have suggested that in stitutional characteristics are the main determinants of university choice for mo st students selecting a higher education institution. Thus, a greater number of student s rated their university selection positively on characteristics of academic quality as academ ic reputation, professor effectiveness and program quality. In the same way, financial constraints cause variability about their relative importance. Most re searchers documented that cost is a factor of secondary

PAGE 41

31 importance (Haper & Hill, 1989; Trushein, Crouse, & Middaugh, 1990); however, other researchers disagree, finding th at cost is an important fact or in determining university choice (Delaney, 1998; Wajeeh & Micceri, 1997). Further, personal considerations (parental influences, size a nd location of the university) often finalize the decisions. Professor Effectiveness Professor effectiveness may be judged in terms of how he/she adjusts his own unique pattern of behavior to the unique physic al setting and behavior al patterns of those with whom he/she has cont act (Erickson, 1954). The National Commission on Teacher Education and Professional Standards (1955) of the National Education Association, has been defined a competent teacher as an in telligent, socially adequate, personally desirable, and profession ally able individual. The evaluation of professor effectiveness ha s been of great interest basically to: 1) administrators who are responsible for c ounseling faculty members, as well as for evaluating them with respect to retention, permanent stat us, and promotion; 2) the teaching faculty themselves who wishes a feedback system on their teaching ability, seeking to improve learning; a nd 3) the students who seek information about courses and professors. The educational system has used su bjective and high inference evaluation systems, which are primarily based on pres age variables (Teacher Evaluation Project, 1984-1985). Consequently, professional judgmen ts and expert opinions have generated many points of view and specific techni ques for professor evaluation, such as: department chair evaluation, dean evaluation, sy stematic student ratings, informal student

PAGE 42

32 opinions, classroom visits, enrollment in co urses they might each, committee evaluation, and self-evaluation or report (Bor ich & Fenton, 1977; and Butler, 1978). Although the first source of information as an assessment tool is the chair evaluation, in the last decades student ratings of professo r effectiveness have been probably the most thoroughly st udied of all forms of personne l evaluation, and one of the best in terms of being supported by empirica l research (Seldin, 1989; and Marsh, 1984). Consequently, the instruments developed to measure student ratings of professor effectiveness are often based on student’s perceptions of the instruction received, therefore, it is assumed that their perceptio ns are reflections of instructional quality (Dunkin, 1986). However, many researches question the validity of the students’ instructional ratings, for example, Frey, ( 1974) argues that student s’ perceptions are a product of their own personalities, as well as of the teacher’s behavior. Therefore, the impression that a professor creates depends not only on his own actions but also on the behavior and viewpoints of hi s spectators. Marsh (1984) also documented that professor evaluation instruments do not contain items deri ved from a logical analysis of professor effectiveness. In Venezuelan higher education, usually professor evaluation is carried out through the respective academic units (dep artment and academic advocate committees), which have a pre-determined pl an or tendency to improve the institution according to the guidelines established by the law and statute of higher education (R epublic of Venezuela Congress, 1970) and/or by the respective institution through the principles and strategies to the academic transformation, which are ba sed on the academic-curricular principles to the transformation and modernization in Ve nezuelan higher education (ULA, 1999b).

PAGE 43

33 In the specific case of the university of Los Andes the academic units have not evaluated the professor’s beha vior in the classroom, the professors submit an annual detailed report to the academic unit for its consideration on the following activities: academic (name and number of courses a ttended, number of students, kinds of evaluations, percentage of the program given), research, admini strative, and extension, so that it guarantees the academic improvement of the university. Therefore, a system of student instructional ratings coul d be a first step to develop a system of student feedback on professor behavior and course evaluation at the ULA, a sy stem of feedback that will improve teaching and learning at the institut ion. Consequently, Venezuelan universities must emphasize a wider range of factors in the search for more accurate and in-depth evaluations of faculty perfor mance and academic reputation. The development of the rating form is ge nerally supported on a literature review, on instruments used and considered successful at other institutions, and on the personal opinion and approaches toward the teaching process of their professors. Since no standard nor universally accepted agreement crite ria of professor effectiveness exist, here there is obviously a fundamental problem in th e detection of a satisfactory criterion that should be considered as an ar bitrary standard that serves to evaluate an effective professor. Frequently, some of these criteria ar e based on research, philosophic principles and logic, and principally on opinion, often founded on cas ual observation and intuition (Anderson, 1954). Most of the criteria have been validated by some accepted systems of values; for example, Cook (1847, in Ande rson 1954) states that “the value of measurement depends on the extent to whic h the relationships established are crucial

PAGE 44

34 from the social point of view” (p. 42). Thus, the final vali dation of criteria of professor effectiveness rests with its agreement with the composite judgments of the individuals who are worried about the problem. Several studies have been conducted to estimate the criteria in determining professor effectiveness, for example Pitteng er (cited in Anderson 1954) states that logically there are three bases for estimating success or effect iveness of professors: 1) by the results produced, 2) by the processes em ployed in teaching, and 3) by the equipment that the professor possesses for teaching. Johnson (1955) considers that teacher effectiveness may be evaluated under the follo wing forms: 1) considering an evaluation instrument for the analysis of teacher effec tiveness in a concrete teaching situation, under three main approaches: evaluation of qualitie s assumed to function in the teaching act; assessment of teaching activity; and evaluati on of student progress; 2) to utilize instruments to measure the at titudes particularly significan t to teaching effectiveness (personality tests, academic records, intelligen t tests, and numerous rating scales); and 3) to establish the relationship between professor personality, as expressed by the selected measuring instruments, and professor effectiv eness, as expressed by over behavior with the propose of predicting professor success. In the same way, Mitzel (1960) supposes that there are three classes of criteria that provide a basis for development of vari ous dimensions of professor evaluation: 1) predictive measures, which describe what the professor brings to the classroom such as education, experience, personal ity attitudes; 2) process measures, which describe actual events in the classroom, e. g., teaching beha viors, class organization, student/ professor

PAGE 45

35 interaction; and 3) product measures, which describe the changes that occur in the student, such as achievement, attitude, and behavior. Similarly, Marsh (1992) examines students’ evaluation of teaching effectiveness as a multidimensional construct and emphasizes the students’ evaluation of educational quality instrument developed by Marsh (1987). In this study he evaluated longitudinal data derived from an archive of responses to nearly one million students’ evaluation of educational quality instruments that have been collected ove r a 13-years period of time. Based on a factor analysis, this results revealed a set of nine defined factors that provide measures of distinct components of teach ing and instructor effectiveness: 1) Learning/values, which include variables as course challenging/ stimulating, learned something valuable, increased subject intere st, learned/understood, s ubject matter, overall course rating; 2) Enthusiasm: enthusiastic about teaching, dynamic and energetic, enhanced presentations with humor, teaching style held your interest, overall instructor rating; 3) Organization: inst ructor explanations clear, course materials prepared and clear, objectives stated and pursued, lectures fac ilitated note taking; 4) Group interaction: encouraged class discussions, students shar ed ideas/knowledge, encouraged questions and answers, encouraged expression of id eas; 5) Individual rapport: friendly towards students, welcomed seeking help/advice, inte rested in individual students, accessible to individual students; 6) Breadth of co verage: contrasted implications, gave background of ideas/concepts, gave different points of view, discu ssed current developments; 7) Examinations/Grading: examination feedback valuable, evaluation methods valuables, tested emphasized course content; 8) Assi gnments: reading/texts valuables, added to

PAGE 46

36 course understanding, and 9) Workload/Difficu lty: course difficulty, course workload, course pace, and hours/week outside of class. Marsh (1982, 1984, 1987) examining reviews on students’ evaluations of teaching effectiveness have concluded that student ra tings about this topic are considered as multidimensional, reliable and stable cons tructs; principally are a function of the instructor who teaches a course rather than of the course that is taught; students’ evaluation are relatively valid against a variety of indicato rs of effective teaching and relatively unaffected by a vari ety of variables hypothesized as potential biases to the ratings; which seen to be useful by faculty as feedback about their teaching, by students for use in course selection, by administrators for use in personal decisions, and by researchers. Additionally, Marsh (1987) consid ered that students’ evaluation of teaching effectiveness have been one of the most systematically studie d forms of personnel evaluation, as well as one of the mo re supported by empirical research. Several research designs used to st udy student evaluati on of teaching and professor effectiveness (Bashki, 1976; Feld man, 1976; Marsh & Overall, 1980; Lytton & Gadzella, 1991; Young & Shaw, 1999) have receiv ed a great deal of attention and have been thoroughly analyzed and generally suppor ted in this literature review. They describe investigations that seek identifying characteristics, factors, traits, and classroom behaviors of effective or successful professors by rating instruction. For example, Bashki (1976) using an instrument to measure stude nt evaluation of facu lty at a particular university found that the criteria used by students in their ratings of instructors had much more to do with course objectives and cons umer satisfaction than with entertainment value. He discovered that the attributes as preparedness, clarit y, and stimulation of

PAGE 47

37 students’ intellectual curiosity were the highe st factors rated by students in describing their best instructor. Anothe r characteristic highly rated was warmth toward students. Besides, results of this study revealed that th ere was also some evidence that feedback in the form of student rating may im prove the teacher’s performance. Similarly, Feldman (1976) reviewed empi rical studies related to professor effectiveness and classified an array of char acteristics into a small number of categories or dimensions that specify the attitude and be havior that describe an ideal professor and good teaching. He found that stimulation of in terest and clarity of presentation were the two most highly related dimensions of good teaching and that the more effective professors generally were seen as very knowledgeable about the subject matter, were organized and prepared for class, and dem onstrated enthusiasm. Other less important characteristics, according to Feldman, were related to classroom management. Additionally, he found that interpersonal traits such as friendliness, helpfulness, and openness to opinions of others’ were considered by students to be important traits of good professors but not as important as the other characteristics. Marsh and Overall (1980) examined students’ evaluation of instructional effectiveness with respect to five dimens ions of the learning/teaching environment: instructor’s skill, course characteristics, structure, value, and instructor-student interaction. The findings of th is study showed that professo r effectiveness was strongly influenced by adequately outlined course objectives and the instructor involving the students in discussions. In contrast, the factors that s howed a lesser influence were: purpose of class assignments made clear and you learned something of value. They also found that these results provided strong suppor t for the assumed stab ility of students’

PAGE 48

38 evaluations of their courses and instructors a nd suggest that this st ability does not vary systematically with course level or content. Additionally, they found that the large and statistically significant correlations obtain ed between end-of-term and retrospective ratings indicted that after a period of time for reflection, students do not change their initial evaluative judgments, at least in a relative sense. In similar manner Lytton and Gadze lla (1991) analyzed responses to an instrument designed to measure students’ percep tions of an ideal professor. This study shows that the questionnaire is a statistically reliable instrument. The results revealed a great deal of similarity and consistency among the four campuses that participated in this research. From the analysis of the results th e authors shown that the three most important attributes of an ideal profe ssor were knowledge of subject matter, interest in subject matter, and presentation of material in a fl exible manner. They also found students feel that a professor who writes book/ articles, participates in community life, and participates in research are activities of least important to them. Similarly in a most recent study Young and Shaw (1999) examined the student’s perceptions about professor effectiveness and su bmitted the data to a variety of statistical analyses to describe and pr oduce a model of professor eff ectiveness. These results showed that effective communications, a comf ortable learning atmosphere, concern for student learning, student motiva tion, and course organizatio n were found to be highly related to the criterion measur e of professor effectiveness. However, it was not expected that the value of the course would emerge so strongly as a predictor of professor effectiveness in the analysis. Additionally a discriminant analysis showed the variables that best differentiate between effective and ineffective professors, so professors whom

PAGE 49

39 students rated as 7, 8, or 9 were characterized as the effectiv e professors, and those whom students rated as 1, 2, or 3 on the global item were characterize d as ineffective professors. Finally, five items were found to differenti ate significantly betw een the two groups of professors: value of the course, motiva ting students to do their best, effective communication, course organization, and resp ect for the students. The literature on professor evaluation has shown a relationship between specific background, behavioral or personality characteristic s and professor quality. Summary The findings related to students’ percepti on about professor effectiveness suggest that the importance attached to any character istic varies. Such variance results from different attitudes of the student s, as well as the specific char acteristics of the institution. Thus, these researchers have shown that prep aredness, clarity, stimulating of students’ intellectual curiosity, knowledge of subject matter, instructor’s enthusiasm, organization and preparation for class, interest in subject matter, presentation of material in a flexible manner, warmth toward students, effec tive communications, a comfortable learning atmosphere, concern for student learning, and student motivation were the highest factors rated by students as influenc ing professor effectiveness. University Academic Reputation Students’ perceptions about university academic reputation are examined over theories and criteria of unive rsity’s reputationimage, sin ce these have been practically admitted as denominations for a same quality. However, when they are analyzed we

PAGE 50

40 could find some apparent paradoxes between them. As an example, an institution may be seen as the most important among a set of other institutions af ter its excellence has diminished; similarly, another institution may retain its second class in reputation or image long after it has achieved a first class position. The image is what one person to another communicates. Because of this co mmunication over a long period of time, a reputation is created about a particular university. The two terms overlap whereby reputation begins inside th e image that is presented. Reputation can be described as an opini on of character gene rally held, it is a distinction or specific credit attributed to so meone or something, derived from previously established attributes, achievements that de rived from past success. The perceptions about the reputation may accurately reflect the object that is viewed or they may not. They may be formed as individuals gain information about a co llege or university through human senses, attitudes, values, me dia sources, interpersonal exchanges, and direct experience; therefore, university repu tation represents how people perceive an institution but does not necessarily reflect th e true nature of the institution. Jacoby and Alson (1985) stated that the perceived institutional reputa tion or image is subjective reality rather than objective reality that dete rmines most human behaviors. Therefore, the goal for university administrators and planne rs may be to understand how the students perceive the reputation of their institutions. The institution’s reputation is one of thei r most precious and powerful marketing tools. It has a tremendous and often underappr eciated effect on university/college choice. Repeatedly when researchers ask thousands of students why they selected a specific university, they generally offer four reasons: reputation or im age (the principal), location,

PAGE 51

41 cost, and the availability of a specific major. Most of these students make their decisions based on their perception of the in stitution’s reputa tion (Sevier, 1994). According to Sevier (1994) reputation can be perceived in two directions: vertical and horizontal. A reputation’s vertical context is when the people meet one negative or positive element of an institution or college that they are inclined to project to the entire institution. In contrast, a re putation’s horizontal situation is one of comparison; people often compare one institution on a particular di mension to another. Reputation is best understood and improved in a context that incl udes the competition. Sevier also shows some characteristics of strong and weak reput ation or image. Strong reputation indicates: high morale, high retention, lower cost to r ecruit a student, strong institutional vision, strong academic core and clearly defined curriculum, low faculty and administrative turnover, few job-related grievances and lo w absenteeism, able to present a strong, coherent message, and high local/community s upport. Frequently th e characteristics of weak reputation are the opposite of those of strong reputation: higher co sts to recruit a student, poor retention, no sense of direction, weak academic core and unfocused or dated curriculum, poor morale, high faculty and administrative turnover, and vandalism. Faculty, research, students, and enviro nment combined generally define the university/college qualit y. “The most prestigious U.S. universities tell us through their publications that, to be like them, a university as piring to join the elite should: 1) recruit a well-credentialed faculty, a faculty with a worl dwide (or at least a na tionwide) reputation for research productivity, 2) recruit the very be st, i.e., brightest stude nts, and 3) provide a learning environment second to none (Smith and Baxter, 1992).

PAGE 52

42 Huddleston and Karr (1982) measured a multi-dimensional concept of image by having subjects rate the univer sity/college on a series of semantic differential scales attached by bi-polar traits that may serve as the conceptual basis for the image of an institution. Thus, this method of assessing pe rceptions was modified to include pairs of descriptive phrases rather than adjectives. For example, faculty reputation, enrollment size, campus activities, academic environment, present two polar phrases that were used to describe these attributes. This technique represents only one example for diagnosing a university’s reputation, however, regardless of the method chosen, a university/college must continually be concerned with determ ining its reputation or image from target markets and through appropriate means work to reinforce or alter the results of its findings. A relatively small number of studies ha ve examined the factors that influence student’s perception concerni ng a university’s academic reputation. For example, Struckman-Johnson and Kinsley (1985) describe d how the administration of a particular university assessed institutional image usi ng the profile technique. A questionnaire instrument was used for measuring reputation or image, which was administered to 2,500 high school seniors, 1,400 undergraduate stude nts at this university, and 3,500 alumni who had graduated from this university. The questionnaire was only returned by 23 percent of the seniors (557), 25 percent of th e students (425) and 26 percent of the alumni (907). The findings of this study showed several positive outcomes. These results revealed that subject groups were very cons istent in their evalua tion of the university’s reputation or image on the following dimension: competitiveness admission policy, academic reputation, enrollment size, number of student activities, attractiveness of

PAGE 53

43 campus, competitiveness of academic environment, cost of tuition, personal atmosphere, attractiveness of city and neighborhood, distan ce from home, graduate and professional preparation, conservative/liberal so cial life, qualities of athlet ic facilities, and preparation for a job after graduation. Statistical techniques reveal ed that the groups had assi gned significantly different ratings. The authors also found that senior s had more favorable attitudes toward the University than the other two groups. Seni ors viewed the University as having more superior academic reputation, a larger numbe r of student activiti es, being more well known, more attractive city and neighborhood, providing a stronger graduate school preparation, and better prepar ation for a job. In contra st, the students viewed the university and the city and neighborhood as le ss attractive, the campus environment as less personal, and the cost of tuition as more expensive. The alumni, however, gave stronger ratings to the personal atmosphere, cam pus attractiveness, and city attractiveness than did the current university students. In conclusion, they found the University of South Dakota enjoys a positive overall imag e or reputation among a potential student population and among groups that have attended the institution. Wanat and Bowles (1989) have also doc umented students’ perceptions of a university’s academic reputation. They obser ved the process of a university’s academic reputation for academically talented students, who judged academic reputation in terms of the reputation of professors, research opportu nities, challenge of course work, prestige, and recognition of the school’s name. These students also preferred the institution that provided them with the greates t personal attention during re cruitment, and finally, they

PAGE 54

44 considered cost and financial packages as s econdary considerations to the institution’s academic reputation. Similarly, Matthews and Hadley (1993) ex amined how the students perceive the quality of an institution and how these per ceptions affect their application decisions. They developed an instrument, Student Perception of Institutional Quality, to measure aspects of institutional quality. This st udy found a significant relationship between students’ perceptions of instit utional quality and their applic ation sets for matriculation. Consequently, these results showed that th ere is evidence that students make their selection of university /college based on perceived inst itutional quality. Matthews and Hadley found that for each of the selected universities, 40% or mo re of the respondents reported no awareness of the following quality indicators: faculty spend as much time teaching as they do on their research, faculty spend time with student outside of class, faculty publish a great deal of research, many speakers and performers from off-campus, and high starting salaries for its gradua tes in fields that interest them. In a more recent study, Wajech and Mi cceri (1997) examined the factors influencing student’s perceptions about univers ity academic reputation. The focus of this study was to identify differences in stude nt ratings between metropolitan students, freshmen and overall, and traditional university freshmen on a set of factors considered to influence academic reputation. They examin ed a student’s perception of what most influences a university’s academic reputati on and demonstrated that cutting edge technology and widespread use of educa tional technology were the two top ranked factors influencing these perceptions of uni versity’s academic reputation. Quality of library and high-published ratings in reports were the third and fourth highest factors

PAGE 55

45 having either a high or moderate influence. The next factors having either a high or moderate influence were quality and quantit y of research and high admission standards. Summary The results linked to students’ percepti ons of university’s academic reputation have suggested that reputation is best unders tood and improved in a context that includes the competition. Other findings have revealed that reputation of professors, research opportunities, challenge of course work, pres tige, recognition of the school’s name, and educational technologies were the factors of greatest influence on the reputation or image of an institution. Construct Validation: General Overview The general concept of validity was traditi onally defined as "the degree to which a test measures what it claims, or purports, to be measuring" (Brown, 1996, p. 231). The 1985 Standards for Educational and Psychol ogical Testing define validity as the “appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores”. Validation also is considered as a process of gathering evidence that an instrument measures the construct what it is designed to meas ure (Nunnally, 1978). Cronbach (1971) describe validati on as “the process by which a te st developer or test user collects evidence to support the types of inferences that are to be drawn from test scores”. Validity also refers to the degree to which evidence and theory suppor t the interpretations of test scores entailed by pr oposed uses of tests. ... Th e process of validation involves

PAGE 56

46 accumulating evidence to provide a sound scientific basis for the proposed score interpretations ( The Standards for Educational an d Psychological Testing, 1999). Although validity is a unitary concept, there are different types of evidence that can be gathered to support the inferences be ing made from the scores of a measurement instrument. According to the joint committee of the American Educational Research Association (AERA), American Psychologica l Association (APA), and National Council on Measurement in Education (NCME) (1985, 1999) validity is a process that concerns three types of evidence of demonstrating the validity of test scor e inference: content related evidence, criterion related evidence and construct related evidence of validity. Content validity refers to the degree to which the scores yielded by a test adequately represent the conceptual domain th at these scores purport to measure, in other words, content validity refers to the extent to which the sample of items on a test is representative of the conceptual domain. Content-related validity evidence is not expressed in numerical form; it refers to the representativene ss that items on the instrument reflect the entire domain. Evidence of content validity is generally gathered by obtained expert judgment on domain representativeness, therefore, it involves a careful and critical examination of the items to determine if the content measured by the instrument is representative of the construct domain. To obtain an external evaluation of cont ent validity, the res earcher should ask a number of experts to examine the test conten t methodically and evaluate its relevancy to the particular universe, therefore, if they have the same opinion about domain representativeness, then the te st can be supposed to have content validity. (Crocker & Algina, 1986).

PAGE 57

47 Criterion related evidence refers to the extent to which the test scores on a measuring instrument are related to an indepe ndent external criterion (relevant, reliable) believed to measure directly the behavior or characteristic in que stion. There are two designs for obtaining criterion related validity: predictive and concurrent. Both designs are concerned with the empiri cal relationship between test scores and a criterion, but the difference is made on the basis of the time when the criterion data are colleted. Predictive criterion related evidence refers to the degree to which test scores predict a criterion measure that will be made at some point in the future, while concurrent criterion refers to the relationship between test scores and a criterion measure available at the same time. The emphasis in criterion related evid ence is on the criterion and the measurement procedures used to obtain criteri on scores (Crocker & Algina, 1986). Construct related evidence focuses on the te st scores as a measure of a construct, therefore, to understand the trad itional definition of construct validity, it is first necessary to understand what a construct is. A construc t, or psychological cons truct as it is also called, refers to something that is not itself di rectly measurable but rather must be inferred from their observable effects on be havior. The 1985 Standards for Educational and Psychological Testing define a construct as “a theoretical construction about the nature of human behavior” (p. 9). Construct validity has traditionally been defined as the experimental demonstration that a test is measuring th e construct it claims to be measuring. Construct validity is a process that i nvolves a group of methods for assessing the degree to which the instrument measures the theoretical cons truct (Cronbach & Meeh l, 1955). Construct

PAGE 58

48 validity would be involved wh en the attribute or quality could not be operationally defined. The concept of construct validity is very well accepted in educational measurement circles, howeve r, all three types of valid ity discussed above: content validity, criterion-related validity, and constr uct validity are now taken to be different facets of a single unified form of construct validity. This unified view of construct validity is considered a new development by many of the validity theorists (e.g., Angoff, 1988; Cronbach, 1988; Messick, 1980, 1981). Messick (1989) has argued that evid ence of construct validity is the most important type of evidence to seek concerning a measure’s validity because the validity of a measure concerns what the test scores mean. The impetus for construct validation came from personality theory and the researchers’ need fo r a method of validating the instruments used in theory development. Ne ither content nor criterion related evidence directly focuses on the constr uct being measured by a test. The objective in gathering construct evidence is to determine what psyc hological construct is being measured by a test and how well it is being measured. The general steps in a process to gath ering construct validity evidence include: formulating a hypothesis based on the theo retical underpinnings of the construct; designing a measurement instrument includi ng items that represent the specific and concrete demonstrations of the construct; gathering a nd analyzing the data; and determining if the re sults most likely suppor t the hypothesis or not (Crocker & Algina, 1986).

PAGE 59

49 There is no single method used to gather evidence for the constr uct interpretations of a test. Some of the most common pro cedures used to esta blishing the construct validity of score interp retations are: the logical method, the experimental method, and the correlational method. The main aspects of th e logical approach in clude asking if the elements the test measures are those that st ructure the construct and checking the items to determine if they seem appropriate for assessing the elements in the construct. The experimental methods are appropriate if the hypothesis involves a causal relationship. In experiments involving observ ational measurement, instrumentation effects are particularly expected. It ma y be hypothesized that test scores would change when certain types of experimental treatments are established. Correlational methods include the most wi dely used approaches to construct validation, such as: correlations between a m easure of the construct measure and other designed; multi-trait multi-method studi es; and factor analysis studies. One aspect of the correlati onal approach to gathering construct related evidence includes correlations between a measure of the construct measure and other designed measures. It is one of the simplest methods to establish evidence of construct validity. Correlation between scores of the construct and scores on an established test is considered to be a valid measure of the constr uct, for example if the correlation is high, it is assumes that the test is measuring the sa me construct as the established test, and one assumes evidence of construct validity. Another aspect of the corre lational approach to gather evidence of construct validity is the multi-trait multi-method matri x, developed by Campbell and Fiske (1959). This method examines patters of intercorrelati ons between different traits using different

PAGE 60

50 measurement methods. For example, the re liability coefficients examine correlations between measures of the same construct using the same (similar) methods; the validity coefficients examine correlations between meas ures of the same construct using different measurement methods (convergent validity) and correlations between measures of different construct using the same/different measurement methods (discriminant validity) (Crocker & Algina. 1986). Messick (1989) also discusse s the use of convergence of indicators of the construct by seeking out the other measures as valid indicators of the same construct and he as well points to the ne ed for evidence that the construct could be empirically distinguished from other construc ts (at least represent some aspect of the construct measures) by identif ying measures with which the construct should not be significantly correlated. An additional aspect of the correlational approach is the factor analysis, which is a statistical procedure for studyi ng the intercorrelation among a se t of test scores with the purpose of determine the number of factors or constructs required to account for the intercorrelations, and the percentage of variance acc ounted for by the factors. Consequently this method provides an empirical basis for reducing all these variables to a few factors by combining variables that are moderately or highly correlated with each other. Results from factor analysis studi es contribute to demonstrate evidence for the construct validity of an instrument. In general factor analysis is useful in determining th e minimum number of factors that account for the variance in the data pr ovided by an instrument However, studies using this method can be described in terms of an exploratory or confirmatory factor analysis. During the development of the inst rument exploratory factor analysis may be

PAGE 61

51 utilized to extract the number and nature of the factors including the measure, in order to determining what characteristics are being meas ured. Thus, the results of this analysis could help in the revision of the instrument itself, as well as the re vision of the necessary theory. In other way, the confirmatory factor analysis might be used as the method of choice in a construct validity st udy if the investigator states a hypothesis about the nature of the factors and/or about the numeric values of some of the parameters of the factor analysis. Results from this analysis contribu te to gather evidence to demonstrate support for the construct validity of an instrument. Studies of construct validation with refe rence to factor analysis have used exploratory factor analysis as well as confirmatory factor analysis as the method to examine the data. For example Crocker and Al gina (1986) illustrate d the application of factor analysis to an explor atory construct validation study involving a battery of tests. The purpose of this study was to determine the number of common factors required to account for a pattern of correlations among all pairs of tests in a set of tests, the nature of the common factors that account for the test intercorrelations, and the proportion of variance associated with common factors variance. Some studies of construct validation had provided supportive evidence involving exploratory factor analysis as the method of data an alysis (Maguire & Lay, 1981; Rickman & Green, 1993). For example Ri ckman and Green (1993) evaluating an instrument identified thirty-three items that could influence the university choice process. Exploratory factor analysis wa s used as the procedure to examine the factor structure. The results suggest that individuals use specific criteria when making the college selection decision. The findings revealed f our factors had statistically significant

PAGE 62

52 difference above the 0.05 alpha level, how ever, academic excellence, individual preference, and secondary information were found to be the most significant factors in their college decision process. Only one study was found on students’ pe rception about university reputation or image, which examined the application of fact or analysis. In this study Maguire and Lay (1981) using factor analysis with oblique rota tion of factors as the specific technique of data analysis found that academics, reputation, at hletics, social/special relations, cost and size/quality were the most influential fact ors that can accurately summarize the overall students’ perceptions fo r measuring reputation. Other studies were found that examined th e application of confirmatory factor analysis, which performed within the linear st ructural relations (LISREL) method (Marsh & Hocevar, 1985; Marsh, 1987, 1991, 1992; Mars h & Bailey, 1991). For example Marsh (1987), based on students evaluation and f aculty self-evaluation, summarize a study on students’ evaluation of teaching effectiven ess in higher education, which emphasized construct validity approach, and lead to the development of the students’ evaluation of educational quality (SEEQ) instrument. This study was designed to measure nine evaluation factors: learning values, instru ctor enthusiasm, or ganization, individual rapport, group interaction, breadth of covera ge, examinations and grading, assignments and reading, and workload difficulty; which have been supported by more than thirty exploratory factor analysis (Marsh, 1983, 1984). Marsh (1987) also reported consistent identification of these factor s on the SEEQ, and noted that the systematic approach employed in the development of the instrument and the similarity of the factors that they measure supports their construct validity. Fact or analysis has provide d a clear support for

PAGE 63

53 the factor structure it was designed to measure, demons trating that the students’ evaluation measure distinct compone nts of teaching effectiveness. Theoretical Rational for Gende r and Campus Differences Considering the importance that gender differences and university campus as demographic characteristics have had in the st udent’s behavior, this study also examined whether the students’ decisions of university choice proce ss and their perceptions about professor effectiven ess and university academic reputation are equally shared by gender and campus. The presence of gender differences in va riability in the stude nts’ behavior has been described and debated in the educatio nal and psychological literature for many years. Several published studies have dem onstrated the prevalence and stability of a gender difference in students’ behavior (cognitive abilities an d achievement) (BenShakhan & Sinai, 1991; Bolger & Kellaghan, 1990; Eccles & Blum enfeld, 1985; Feingol, 1992, 1994; Hedges & Friedman, 1993; Johnson 1987; Schibeci & Riley, 1986). The practical effect of gender differences in vari ability has been discussed with respect to selection of a mayor field of study in higher education (Chronicle of Higher education, 1996). In a recent study Boggs (1995) identifyi ng gender bias in teaching evaluations, stated that communication rese arch might provide some valu able information regarding evidence of gender bias affecting student ra tings of their college professors’ teaching effectiveness. This paper discusses evidence that students’ biases, including gender bias, may affect their evaluation of professors. The research is also presented and discussed

PAGE 64

54 regarding the influence of professors’ and students’ ge nder on classroom communication processes. Further research may provide insight into possible connections between communication patterns, gende r and student ratings. Gender differences in variability in ability and achievement have been generalized in different countries. For example, Feingold (1994) conducted a cross-cultural quantitative review of gender differences in variability in verbal, mathematical, and special abilities and concluded that in some c ountries, the males’ test scores showed more variability than females and the reverse was also true in other countries. In a most recent study James, Baldwin, a nd Melnnis (1999) examined the factors influencing the choices of prospective underg raduate students at different Australian universities. They conducted five su bgroup analyses by gender, by location, by socioeconomic background, by field of study a nd by category of pref erred university and used factor analysis techniques to assist in the reporting of the data. The analysis looking for variations in applicants’ responses according to gender revealed no statistically significant gender difference, when the student s are compared on the factors influencing the university choice process. On the other hand, campus differences in students’ perceptions about university concerns have also been debated in the e ducational and psychology literature. Several researchers have revealed the occurrence of campus differences in student decisions about university choice process, and their pe rceptions related to the quality of its professors and university prestige. For example, James, Baldwin, and Melnni s (1999) studied factors influencing the choice of prospective unde rgraduates in a randomly selected sample of 3,194

PAGE 65

55 undergraduate students at different Australia n universities. This study considers five subgroup analyses that were conducted by gender, by location, by socioeconomic background, by field of study and by category of pr eferred university. They classified the universities into four categories: research intensive universities, metropolitan universities, universities of technology, and re gional universities, moreover, they used factor analysis techniques in the analysis seeking distincti ons in student’s responses consistent with those subgroups. The analyses according to students’ gender and socioeconomic background revealed no statistically si gnificant differences among the group, respectively; however, showed some differences between the highe r and lower socioeconomic group, such as, the higher socioeconomic students are more influenced than the lower socioeconomic group by the prestige of the university and th e social and cultural life on campus. The analyses related to students’ location, field of study, and university chosen revealed a clear statistically signi ficant difference among the groups, respectively. The analysis revealed a statistically significant difference among the four categories of universities. St udents to research and technol ogy universities are the most similar in the consideration that have infl uenced their university choice decision. The metropolitan universities are the least well diffe rentiated in the thinking of prospective students. In another study done by Hayden (2000) on th e factors that influence the college choice process for African and American student s at different instit utions, she developed a 60-items survey that asked respondents to ra te the extent of influence of those four factors: academic, social, personal, and financia l on university choice process. The target

PAGE 66

56 sample included 180 students at Predomin antly White Institutions (PWI) and 180 at Historically Black Institutions (HBI). Factor analysis wa s conducted to create subscales of the items for each scale, and t-tests were performed to compare mean scores between groups. Results revealed no significant differences in mean score between group and any of the subscales. However, important diffe rences between groups were identified when the ranking of mean scores were examined. It is important to expose that the institutions had similar institutional missions; PWI and HBI were in reasonable proximity; PWI chosen was a large public research, land-grant institution while HBI was a small public comprehensive-land-grant institution in the ex tent to which academic, social, personal, and financial issues affected the universit y selection process of students who attended PWI and HBI. In a study done by Pike (2003) on a comp arison of United States (U.S) News rankings and the National Survey of Student Engagement (NSSE), this researcher compares the NSSE benchmark scores of 14 pub lic research universities with those same institutions ranking by U.S News and World Report This finding underscores the importance of taking into consideration the characteristics of stude nt population when comparing institutions. Results of this study re vealed statistically significant differences in mean NSSE benchmark for 14 public research universities. Summary In summary, there is a clear discrepa ncy among the findings associated with gender and university campus differences in students’ decisions and perceptions about

PAGE 67

57 university concerns. Although there have been researchers that have shown the prevalence and stability of a gender differen ce in students’ behavi or, teaching evaluation, and university choice process (Ben-Sha khan & Sinai, 1991; Boggs, 1995; Bolger & Kellaghan, 1990; Eccles & Blumenfeld, 1985; Feingol, 1992, 1994; Hedges & Friedman, 1993; Johnson 1987; Schibeci & Riley, 1986); other researchers have shown that there is no statistically significant gender difference wh en the students are compared on factors that influence their decisions about univ ersity choice process (James, Baldwin, and Melnnis, 1999). Similarly, while several researchers have revealed the occurrence of campus differences in student decisi ons about university choice proc ess, and their perceptions related to the quality of it s professors and university prestige (James, Baldwin, and Melnnis, 1999; Pike, 2003); other resear chers have showed no university campus difference in students decisions of university choice process (for example Hayden, 2000).

PAGE 68

58 Chapter Three Methods This chapter provides a description of the procedures used to address the research questions of concern in this study. The propos ed study involves both descriptive research and correlational research. The descriptive resear ch is a type of quantitative research that involves the description of educational phenomena It produces statistical information about aspects of education to policy makers, administrators, and educators. The correlational research may be classified as a descriptive research that allows analyzing how the variables affect a part icular pattern of behavior (G all, Borg & Gall, 1996). The purpose of this research was to gath er construct validation evidence for an instrument designed to measur e students’ university choice process and thei r perceptions about professor effectiveness and university acad emic reputation at th e University of Los Andes. Additionally, a comparative analys is was carried out to determine how the selected factors that influence the studen ts’ university choice process and their perceptions of professor e ffectiveness and university academic reputation differ according to student demographic factors such as gender and university campuses. Research Questions Six research questions examined data collection and anal ysis on students’ decisions and perceptions in university choice process, and professor effectiveness and university academic re putation, respectively. 1. Are the student’s decisions of university ch oice process, and student’s perceptions of professor effectiveness and university acad emic reputation reliable within their respective factors at the Un iversity of Los Andes?

PAGE 69

59 2. How well does the hypothesized measurement model involving five-first order factors fit the observed data based on student’s deci sion to enroll at the University of Los Andes? 3. How well does the hypothesized measurem ent model involving four-first order factors fit the observed data based on the student’s perceptions about professor effectiveness at the University of Los Andes? 4. How well does the hypothesized measurem ent model involving three-first order factors fit the observed data based on student’s perceptions of university academic reputation at the University of Los Andes? 5. What are the differences across gender in perceived importance of the selected factors that influence the students’ deci sions about university choice process, and their perceptions of professor effectiven ess and university academic reputation at the University of Los Andes? 6. What are the differences across university campuses in perceived importance of the selected factors that influence the stud ents’ decisions about university choice process, and their perceptions of professo r effectiveness and university academic reputation at the University of Los Andes? Target Population The target population for this study include s all students who were registered in the second semester of 2002 at the University of Los Andes. This research study considered the ten colleges (Architecture and Art, Science, Economic and Social Sciences, Forest and Environmental Sciences, Law and Political Sciences, Pharmacy,

PAGE 70

60 Humanities and Education, Engineering, Medici ne, and Dentistry) and the two-university branch campuses (The Tachira Universi ty Campus: “NUTULA” and The “Rafael Rangel” University Campus: “NURR”) of the University of Los Andes. Sample Design The selection of the sample is an impor tant concern in any research, thus, the methods used to determine how large the sa mple size should be and how it will be selected from the population of study are of great interest. The sampling frame of this research c onsisted of undergraduate students by college registered in the courses being o ffered in the second semester of 2002. For purposes of this research, a stratified probability sample was used to select the participants. This type of the sampling was selected to ensure that all colleges and university branch campuses at the ULA were included in the study. Thus, the different colleges and the two university branch campuses were used as a separate stratum. Then, a sample from the ULA main campus that cons ists of ten colleges th roughout the city of Merida and the other two university branch campuses in Tachira and Trujillo was selected randomly maintaining the popul ation proportion and e qual probability of selection. The sample for the study consisted of undergraduate students enrolled in the different courses by college within the UL A’s main campus, which consists of ten colleges throughout the city of Merida, and th e others two university branch campuses in Tachira and Trujillo.

PAGE 71

61 To ensure that the administration of the questionnaire was only given once to each student by college and university branch campus a random selection of courses by semester, college and university branch wa s used according to the program of study (See Appendix A). The semesters considered in th is study ranged from fift h to tenth semester, in order to ensure that the students selected by semester were adu lts (18 year olds and up). This suggestion was made by the IRB, since 17 year olds are considered a minor (See Appendix B). The number of students by college, university branch campus, course, and semester that were selected is shown in Ta ble 1. The courses were selected since they met the sample size requirements by each college and university branch campus. In some cases, an additional course was selected to ta ke only the number of students required to complete the sample size. This selection was made taking the students located in the first lines of the classroom. As a result, the comp lete data were collected from the twelve strata represented by the ten colleges at th e main campus and the two university branch campuses of the ULA (Tachira and Trujillo ), while maintaining the same population proportion (See Appendix C). There are many recommendations and research findings related to the sample size in applications of factor an alysis, some of which are dive rse and contradictory. Many researchers have suggested a wide variety of guidelines for estimating an

PAGE 72

62 Table 1. Number of Students by College, Universi ty Branch Campus, Course, and Semester Courses Number of Students Semester Architecture and Art 52 Graphic Techniques 30 6 Graphic Designs II 22 8 Dentistry 18 Dental Administration 18 8 Economic and Social Sciences 151 Marketing Principles 32 6 General System Theory 35 8 Operative Research 38 9 Econometrics 24 10 Economic Analysis of Projects 22 10 Engineering 115 Metallurgy I 30 5 Thermodynamics II 20 6 Design of Industrial Plants 24 7 Quality Control 25 8 Computational Systems 16 9 Forest and Environmental Sciences 39 Operation and Conservation 25 7 Project formulation 14 8 Humanities and Education 94 Algebra I 37 5 Philosophy and Theory of Education 19 6 Quantitative and Educational Research 26 7-8 Teaching Training V 12 9-10 Law and Political Sciences 114 Civil Law III 41 5 Administrative Law I 30 6 Labour Law 30 8 Procedural and Criminal Law 23 10

PAGE 73

63 Table 1 (continued). Number of Students by College, Universi ty Branch Campus, Course and Semester Courses Number of Students Semester Medicine 99 Physiopathology 35 6 Medicine I 35 7 Pediatrics I 29 8 Pharmacy 37 General Toxicology 23 7 Biopharmacy 14 8 Sciences 35 Mathematics 40 25 5 Organic Analysis 10 8 The Tachira University Campus: NUTULA 114 Mathematics III 23 5 Modern Physics 28 6 Industrial and Environment Chemistry 35 8 Seminar of Education Theory 28 9-10 Rafael Rangel University Campus: NURR 132 Matrix algebra 39 5 Finances I 30 7-8 General Statistic 34 7-8 Teaching Training IV 15 8 Thesis Project 14 9 Total 1000 Source: OCRE. Central Office of Student Registration. ULA, 2002. adequate sample size in factor analysis. Th ese guidelines typically involve determining the sample size in terms of the num ber of measured variables being analyzed, ranging from 5 to 25 subjects per variable. Theref ore, the use of the number of measured

PAGE 74

64 variables as a procedure to estimate sample size differ considerably according to the researcher. For example, Gorsuch (1983) a nd Hatcher (1998) suggested a minimum ratio of 5 participants per measured variable and that the sample size never be less than 100 participants; Nunnally (1978) recommended that the minimum ratio should be 10 to 1; in contrast, Cliff and Hamburger (1967) suggested a minimum ra tio of 20 participants per measured variable to ensure stable estimates; consequently, this procedure suggests that more measured variables require larger sample sizes. In one early study, Browne (1968) examined the quality of solutions produced by different factor analysis me thods. The author found that results obtained from larger simple sizes revealed greater stability a nd more accurate recovery of the population parameters estimates. Similarly, MacCa llum and Tucker (1991) and MacCallum, Widaman, Zhang, and Hong (1999) examined the influence of sample size in factor analysis, focusing on how sample size influenc es parameters estimates and model fit. They found that larger sample size improve th e factor solutions and consequently provide more precise and accurate results. A second important concern in sample de sign is related to the selection of the sample size. An efficient sample size is estimated depending on th e nature of the study under consideration. Considering th at this research us ed an exploratory f actor analysis to gather construct validation evidence for an instrument designed to measure students’ university choice process and their percep tions about professor effectiveness and university academic reputation at the University of Los Andes, the estimation of the efficient sample size was based on this specific statistical technique.

PAGE 75

65 Therefore, taking into account the recomm endations to determine a sample size that ensures an adequate stability in factor analysis, the minimal number of subjects in the sample should be 25 times the number of variab les being analyzed, which for this study it indicates a minimally adequate sample size of 700 participants. However, considering that larger sample sizes are required in confirmatory factor analysis, and that a certain number of students can be expected to leav e at least one question blank, which will not provide usable data for the factor analysis, the researcher considered that an adequate sample size should be 1,000 students, with the expectative of obtaini ng results that could be adequately stable and congruent with parameters estimates. Data Collection The data for this study were obtained by surveying students who were registered in the second semester of 2002, at the Universi ty of Los Andes. Fo r the administration of the questionnaire two types of permission were solicited: the first was obtained from the Secretary of the University of Los Andes to conduct the study (See Appendix D). The second was a verbal permission, which was obtai ned from each professor of the selected courses to go into the classes to admi nister the survey to the students. The researcher administered the survey personally in order to collect the data directly from the classrooms; this procedure was used to increase the chance of obtaining high response rates for the survey questionnaire. In some cases when the researcher was not able to administer the survey, graduate students and professors administered the questionnaire, under the same cond itions used by the researcher.

PAGE 76

66 Student response time to complete the instrument was approximately twenty-five minutes. Responses were received from a ll the students selected, which indicate a response rate of 100 percent, besides th ese responses revealed a non-significant percentage of missing values of 1.5 %, on de mographic information (parent’s educational level). The questionnaire includes basically : no student identifica tion; eight questions related to student demographic information; an explication of th e purpose of the study, the instructions to fill out the attached questions related to each topic: university choice process, professor effectiveness, and univers ity academic reputation; and finally, a thankyou to students for their part icipation in this study. Instrumentation The survey instrument used in this study was a self-administered paper–andpencil questionnaire, which includes closed-e nded questions, used to gather detailed information about the student’s characteri stics and on the three different research concerns: student’s universi ty choice process and thei r perceptions of professor effectiveness and university academic reputa tion (See appendix E). The instrument was grouped into four sections: The first section gathers data about eight (8) items on student demographic information: gender, age, geographic stat e, college/school, se mester/year of study, admission type, parents’ educational leve l, and monthly family income. These variables were included with the purpose to examine if the answers relate with each topic of this study varied acr oss certain student’s demogra phic characteristics such as gender and university campuses.

PAGE 77

67 The second section of the instrument was developed to identify the factors that influence the student’s university choice pr ocess. The university choice is a decision influenced by a number of factors that link directly to their char acteristics and needs. This section identified twenty-eight items, whic h measure the student’s decisions of university choice. Of the 28 items: a) five are related to the infl uence of quality and reputation factors: academic reputation of th e university; quality of the professors, quality of the programs, quality of the t eaching, and the value of a degree from this university; b) nine are associated to facili ties proportionate by the ULA: admission requirements and policies, library facilitie s and collections, res earch and computer facilities, availability of university residences, availabi lity of university transportation, availability of university dini ng hall, scholarship received, availability of part-time work and good possibilities of j ob; c) four are related to personal and vocational influences: interest in a speci fic academic program, parents’ influence, other family influences, and friends’ infl uences, which were developed because in some cases the students are persuaded by the comments and advice of their friends and family; d) two are associated to soci al influences, which bring out information about university athletic programs and campus social environment; and e) five are designed to measure the influence of practic al considerations such as: size of the university, size of the college/s chool, diversity of program offering, length of time to earn degree, and geographic location of the university. The third section was developed to bring out information about students’ perception of professor effectiveness. The student ratings of the prof essor effectiveness in this section are explained by variables associated with professor’s classroom behavior in a

PAGE 78

68 general form. This section contained 22 items, which measure the students’ perceptions of professor eff ectiveness. Of the 22 questions: a) four items were developed to measure the breadth of know ledge such as: prepares for classes, demonstrates knowledge of subject matter, in terprets abstract ideas and theories, and defines of class objectives clearly; b) six items to measure the learning values: stresses important material, supports ideas with examples, comparisons, and facts, includes out-of-text materials in lect ures, and uses of varied lecturing strategies/technology to enhance learning; c) four questions were developed to measure student centered: receptivity to st udent’s ideas and questions, attentiveness to student’s needs and concerns, disposed to help student, and regard for student’s opinion; d) two items to measure the group in teraction: encourage students to think for themselves and encourage open communicat ion; e) two questions to measure the instructor’s enthusiasm and behavior: en thusiastic for teaching and self-controlled and patient; and f) the remaining four items were developed to measure other considerations such as use of class-time efficiently, evaluation/assessment methods appropriate, clarity in presenta tions and explanations, and fl exibility course structure. The fourth section of the instrument deals with students’ percep tions of university academic reputation. The student ratings in this section are related to the beliefs, ideas and impressions that the student has of the University of Los Andes, since the student develops his own image on the basis of some interaction that he has maintained with the university or some commentaries relate d to past experiences of the academic reputation. This section incl udes fifteen items: a) three items were designed to measure the image and prestige of the unive rsity: professors’ quali ty, alumni’s quality

PAGE 79

69 and recognition of the univ ersity’s name; b) seven que stions were developed to measure the research quality: quantity and quality of research, quality of research centers, institutes, and laboratories, qual ity of published resear ch, and quality of library; c) three questions to measure the so cio-cultural factors: social environment, cultural activities, and successful athletic programs and d) the remaining two items were developed to measure other consid erations such as: use of educational technology and admission policies. Students answered the questions related to their decisions to select the university of Los Andes by rating the perceived importance of each item on a category rating scale that was arranged in the following order: 1 = extremely low importance, 2 = low importance, 3 = moderate importance, 4 = high importance, and 5 = extremely high importance. Similarly, the students answered the questions related to their perceptions about professor effectiveness and university academic reputation by rating each item on a five-category rating scale, which was arranged in the following order: 1 = poor, 2 = fair, 3 = good, 4 = very good, and 5 = excellent. Validity Validation is considered a process of gathering evidence that an instrument measures what it claims, or purports, to measure (Nunnally, 1978). The American Psychological Association, American Educatio nal Research Association, and National Council on Measurement in Education, (1985), de fined validity as the “appropriateness, meaningfulness, and usefulness of the specific infe rences made from test scores.” In this study, evidence of content and c onstruct validity is offered.

PAGE 80

70 Content related validity evid ence is not expressed in num erical form; it refers to the representativeness that items on the instrume nt reflect the entire domain. Evidence of content validity is generally gathered and examined carefu lly and critically by expert judges to determine if the content and objectiv es measured by the test are representative of those that constitute the content domain. On the other hand, construct validity is a process that involves a group of methods for assessing the degr ee to which the instrument measures the theoretical construct (Cronbach & Meehl, 1955). In the developed of the instrument an external evaluation was concerned with the selection of items from a universe in which th e investigator is interested. Thus, a large pool of items was drawn from related literature and the criteria of the researcher about the factors that influence the university choice process (Chapm an, 1979; Cleave-Hogg, McLean, & Cappe, 1994; McDonough, 1997), stud ent’s perceptions about professor effectiveness (Feldman, 1976; Marsh, 1984), and student’s perceptions of university academic reputation (Struckman-Johnson & Ki nsley, 1985; Wajech & Micceri, 1997). In order to examine the evidence of cont ent validity, the instrument was initially reviewed by a group of graduate students in a Survey Research Methods course. They examined the instrument to assist in the development of the test items (wording, grammar, and other technical fl aws). Following this peer re view, two expert professors from the department of measurement and resear ch at University of South Florida judged whether the test items cover the content th at the test purports to measure and then determined how well that content domain is sampled by the test items. Based on the professional judgments of the reviewers, some changes were made in the following areas: a) demographic information : in item # 2, include a minimal range of age among 18-21

PAGE 81

71 years; in item # 7, include “other” as a cate gory and specify it; and to add an item that includes the semester or year cursed by the student, and b) scale : change the levels of importance in the domain students’ decisions to select the ULA, by using a category rating scale: 1 = extremely low importan ce, 2 = low importance, 3 = moderate importance, 4 = high importance, and 5 = extremely high importance. Finally a content validation of the Spanish version of the instrument was realized (See Appendix F). This review was conducted for a small group of expert professors from the departments of Pedagogy (2 profe ssors) and Modern Languages (2 professors) at the “Rafael Rangel” University Campus: “NURR” in Trujillo. They reviewed the instrument to assist in the validation of the Spanish version. Thus, based on the professional judgments of the reviewers, some changes were made in the following areas: a) in demographic information change the question home place by home state, and b) in some items related to the students’ decisions to select the ULA the following changes: in item 2 change teaching quality by quality of the professors, in item9 length of the schooling by length of time to degree, and in items 18 and 19 change availability of student residences and tran sportation by availability of university residences and university transportation, respectively. C onsequently, this systematic evaluation constitutes evidence for the content validity of the instrument. Finally, to assess construct va lidity, the instrument was pilot tested on one of the university campuses of the University of Los Andes, the Rafael Rangel University Nucleus (NURR), which has special characteri stics such as: a) offers a diversity of programs of study, that are connected to different colleges such as: business administration, public accounting, agricultu ral engineering, education, as well as

PAGE 82

72 technical degrees in agriculture and livest ock, b) the second campus with the highest number of students, and c) is located outside the central campus, in Trujillo State. Pilot Study In the pilot study attempts were made to se lect students from all the different field of studies, thus, the sample was based on a non-probability sampling; specifically a convenience sample of 223 stude nts who were registered in the first semester of 2000. Exploratory factor analysis with oblique rotation was explored as the method of data analysis. Therefore, thr ee exploratory factor analyses were performed, one for each of the three principal domains that integrate the survey instrument: students’ university choice process, and student’s perceptions of professor effectiven ess and university academic reputation. The exploratory factor anal ysis approach fundamental ly involves the following steps: 1) The selection of the variables to be included in the analys is and the development of a correlation matrix, which shows the correla tion between every pair of variables to be analyzed; 2) Initial extraction of the factor s, where the number of factors extracted will be equal to the number of variables being an alyzed; and each factor will account for a maximum amount of variance and will be uncorre lated with all of th e factors at the time they are extracted; 3) to determine the num ber of meaningful fact ors to retain according to several criteria (Kaiser criterion, the scre e test, proportion of variance accounted for, interpretability criteria); and 4) the rotation to a final solution, this is a simplification process designed to find simple and interpreta ble factors through rotation to a terminal solution, orthogonal and oblique rotations as va rimax and promax, respectively; 5) finally

PAGE 83

73 the researcher names the resu lting factors, which involves identifying the variables that load significantly on a factor and deriving a na me that would apply to all variables (Kim & Mueller, 1978 and Hatcher, 1998). Responses to the survey instrument in the pilot test were subjected to exploratory factor analysis with oblique ro tation, in order to determine th e pattern of intercorrelations among the items. In order to do the interpreta tion of a onefactor solution, the maximum likelihood procedure was used as the method of factor extraction. This method provides a chi-square statistic that perm its to test the null hypothesis that retaining one factor is sufficient versus the alternative hypothesis that more factors should be retained. Results of this analysis show that th e obtained values of chi-square for the different tests (related to university choice, professor effectiveness, and academic reputation) were fairly large ( 2 = 1,335.64, df = 350; 2 = 921,02, df = 209; and 2 = 850.37, df = 90, respectively) and the p values for the obtain ed chi-square tests were significant at p < .0001. These findings sugge st that additional factors are needed, therefore, for the final decisi on; the criterion of the propor tion of variance accounted for was used as the procedure to se lect the number of f actor to retain. F actors were extracted based on the proportion of varian ce explained for the data set, at least 90 percent of the common variance. Thus, the factor analysis revealed: 1. A five-factor solution accounted for 93 % of the common variance in the instrument’s items related to students’ d ecision to select the ULA. The factors 15 loaded 7, 3, 5, 6, and 4 items, respectivel y. These factors can be labeled as facility/support, influential, academic resources, environment/prestige, and quality/ reputation.

PAGE 84

74 2. A four-factor solution accounted for 98 % of the common variance in the instrument’s items of students’ perceptions about professor effectiveness. These factors 1-4 loaded 6, 8, 4, and 3 items, respectively, which can be named as interested/student cente red, content and pedagogica l breadth of knowledge, behavior/receptive, and f acilitation of learning. 3. Finally a three-factor solution accounte d for 99.6% of the common variance in the domain students’ perceptions about university academic reputation. The factors 1to3 loaded on 6, 4 and 4 items, respectivel y. These factors can be labeled as technological/socio-cu ltural, research quality, and prestige. The factor pattern and factor structure resulting from a promax rotation, for each of the domains contained in the survey inst rument are summarized in appendix G. The results from the rotated factor pattern revelade d a vast majority of the items loaded from excellent (.86) to good (.50) and on the appr opriate factor. However, we can see a problem with some items, which have a mean ingful loading on more than one factor. These results demonstrated that the change s made to the research instrument only were the following: In the section related to students’ university choi ces, three items have a meaningful loading on more than one fact or: item 7, which referred to diversity of programs offering, item 10 referred to admissi on requirements and policies, and item 17, which referred to cost of tuition. Similarl y, in the sections on st udents’ perception about professor effectiveness and university’s academic reputation, two items loaded inappropriately, one in each se ction: item 2 referred to enco uraging students to think for themselves in the section about professor eff ectiveness, and item 3 referred to quantity of research produced per year in the secti on related to university academic reputation

PAGE 85

75 Table 2 summarizes the items by domains taking into consideration the results from the rotated factor pattern. Based on thes e results, the changes made to the original instrument were the following: a) the de letion of three items (items 7, 10, and 17) associated to students’ decisi ons in university choice pro cess, which reduced the number of items from twenty eight to twenty five; and b) the deletion of the items 2 and 3 in students’ perceptions about professor effec tiveness and university academic reputation, which reduced the number of items from twenty two to twenty one and from fifteen to fourteen, respectively. Reliability Reliability is usually defined in practice in terms of the internal consistency of the scores that are obtained on the measured variables. According to Hatcher (1998) reliability is defined as the percent of vari ance in an observed variable that is accounted for by true scores on the underlying constructs, and internal consistenc y is the extent to which the individual items that constitute a test correlate with one another or with the Table 2. Items Description for University Choice Process, Professor Effectiveness and University Academic Reputation University’s Choice Process Item1. Academic reputation of the university. Item2. Quality of the professors. Item3. Quality of the programs. Item4. Quality of the teaching. Item5. Size of the university. Item6. Size of the college/school. Item7. Interest in a specific program. Item8. Length of time to degree. Item9. Value of a degree from this university. Item10. University’s geographic location. Item11. Closeness to home.

PAGE 86

76 Table 2 (continued). Item12. Library facilities and collections. Item13. Research and computer facilities. Item14. Use of technologies. Item15. Availability of university dining hall. Item16. Availability of university residences. Item17. Availability of university transportation. Item18. Scholarship received. Item19. University athletic programs. Item20. Campus social environment. Item21. Availability of part-time work. Item22. Good possibilities of job. Item23. Parent’s influence. Item24. Other family influences. Item25. Friend’s influences. Professor Effectiveness Item1. Preparation for class. Item2. Breadth of knowledge of subject matter. Item3. Interpretation abstra ct ideas and theories clearly. Item4. Stress important material. Item5. Support ideas with ex amples, comparisons, and facts. Item6. Inclusion of out-oftext materials in lectures. Item7. Receptiveness to student’s ideas and questions. Item8. Self-controlled and patient. Item9. Use class time efficiently. Item10. Enthusiastic for teaching. Item11. Attentiveness to st udent’s needs and concerns. Item12. Willing to help students. Item13. Concerned about fair evaluation of students. Item14. Definition of class objectives clearly. Item15. Use varied lecturing st rategies to enhance learning. Item16. Use appropriate evaluation/assessment methods. Item17. Clarity in presen tations and explications. Item18. Use flexible course structure. Item19. Regard students’ opinion. Item20. Encourage open communication. Item21. Overall professors’ assessment. University Academic Reputation Item1. Professors’ quality. Item2. Alumni’s quality. Item3. Quality of research centers. Item4. Quality of research institutes. Item5. Quality of research laboratories. Item6. Quality of libraries. Item7. Quality of published research. Item8. Use of educational technology.

PAGE 87

77 Table 2 (continued). Item9. Admission policies. Item10. Social environment. Item11. Cultural activities. Item12. Athletic programs. Item13. Recognition of the university’s name. Item14. Overall academic reputation of the ULA. test total. Cronbach alpha coefficient is one of the most widely used indices of reliability. Consequently, it was used in the pilot st udy to determine the internal consistency reliability of the scale, which was determined on scale items by domains and factors across the domains. Cronbach alpha internal consistency reli abilities on the thr ee domains under study (students’ university choices, and students’ perception about profe ssor effectiveness and university academic reputati on) and by factor across the domains are summarized in Table 3. As indicated this Table, the result s by domains revealed relatively little error, and strong internal reliability coefficien ts (from .87 to .94), which all exceed the minimum value of .70 suggested by Nunnally (1978) Internal Table 3. Internal Consistency Reliabi lity by Domains and Factors __________________________________________________________________ By Domain Indices __________________________________________________________________ Domain 1: Students’ decisions to select the ULA…………….. .87 Domain 2: Students’ perceptions about professor’s effectiveness .94 Domain 3: Students’ perceptions about university academic reputation. …………………………………………. .90

PAGE 88

78 By Factors Students’ decision to select the ULA Factor 1: facility/ support ……………………………… .81 Factor 2: influential ……………………………………. .89 Factor 3: academic resources. …………………………. .72 Factor 4: prestige ...…………………………………….. .61 Factor 5: quality/reputation ……………………………. .72 Students’ perception of professor effectiveness Factor 1: interested/student centered …………………. .90 Factor 2: content and pedagogical knowledge ………… .89 Factor 3: receptive/behavior …………………………… .83 Factor 4: facilitation of learning………………………... .82 Students’ perception of uni versity academic reputation Factor 1: technologi cal/socio-cultural ………………… .87 Factor 2: research quality ……………………………… .87 Factor 3: prestige ……………………………………… .85 Note: n = 223 for all items. consistency reliability coefficients related to student’s decisions to select the ULA were very good, since that the greater part of them exceeded the minimum value of .70, except the coefficient associated with factor 4, whic h is considered relatively low, however, it should be improved by dropping from the scale those items that demonstrated poor itemtotal correlation or revealed meaningful load ing on more than one factor. Specifically, the results from the rotated fact or pattern revealed that the item 7, (diversity of programs offering), item 10 (admission requirements a nd policies), and item 17 (cost of tuition) have a meaningful loading on more than one factor; which should give explanation for this outcome. Internal consistency reliability coefficien ts by factor across the domain related to students’ perception of professor effectiveness are more than adequate; all values of alpha exceeded .82. Similarly, the results of Cronbach alpha internal consistency reliability in

PAGE 89

79 all subscales related to stude nts’ perception of university academic reputation have a coefficient of at least .85 which is mo re than adequate for instrument use. The pilot study results involving exploratory factor analysis as the method of data analysis permitted to determine the final factor structure, which provide supportive evidence of using confirmatory factor anal ysis as evidence of construct validation. Data Analysis In this section the statistical treatmen t of the data will be described, which is divided into seven sections: Software : Data collected were analyzed us ing one of the more commonly used statistical software packag es: Statistical Analysis Software (SAS) version 8.1, specifically the SAS System’s CORR -ALPHA, and CALIS procedures. Descriptive Statistics : These procedures were used to determine the items means and standard deviations, to pr ovide descriptive information about the three concerns in this study: student’s decision to enroll at the University of Los Andes, and their perceptions about professor e ffectiveness and university’s academic reputation. Besides this, demographic data were summarized and analyzed. Reliability : Scale reliability was assessed by cal culating Cronbach alpha internal consistency, which were obtained for the thr ee domains considered in this study, and by factors resulting of the factor analysis solution by domains. A great majority of these coefficients indicated that the scale relia bility was more than adequate. C onfirmatory Factor Analysis : In section four confirmato ry factor analysis (CFA) will be presented: To address the research questions two to four, three separate

PAGE 90

80 confirmatory factor analyses were perfor med to evaluate the hypothesized models underlying: a) the twenty-five items as observe d variables of students’ decisions to enroll at the University of Los Ande s; b) the twenty-one items of students’ perceptions about professor effectiveness; and c) the fourt een items of students’ perceptions about university’s academic reputation. All the an alyses were conducted using the SAS System’s CALIS procedure. In order to perform confirmatory fact or analysis, items means, variability, skewness, kurtosis, correlations among the items for each scale on the three domains in this study were performed to evaluate the conf irmatory factor analysis assumptions; since CFA is very sensitive basically to violations of normality and lack of variability on items. However, the statistical test used with proc CALIS assume that the observed variables have a multivariate normal distribution. Besides this, Anderson and Gerbing, (1988) and Joreskog and Sorbon, (1989) have been argue d that maximum likelihood procedures appear to be reasonably robust against m oderate violations of this assumption. The hypothesized model represents a typica l covariance structure represented for a structural model, which defines the pattern of relations among the unobserved constructs or factors and a measurement m odel that defines relations between observed variables and unobserved hypothetical construc ts or factors. In this case, the measurement model in conjunction with the structural model enables a comprehensive confirmatory assessment of construct validity. Specification, Identification and Estimation of the confirmatory factor model by domain : The confirmatory factor analysis model, which specifies the posited relations of the observed variables to the underlying cons tructs, with the constructs allowed to

PAGE 91

81 intercorrelate freely ( s), may be written in matrix form as: X = + where X is a vector of q observed variables; is a vector of n u nderlying factors (n
PAGE 92

82 Figure 1. Five-First Order Confirmatory Factor Analysis Model on Students University Choice Process, with Different Indicators per Factor. 3 3 2 2 15 15 1 1 4 4 10 10 9 9 8 8 7 7 6 6 5 5 22 22 14 14 1 3 1 3 12 12 11 11 25 25 24 24 2 3 2 3 21 21 20 20 18 18 17 17 16 16 X 15 X 2 X 16 X 18 X 17 X 20 X 21 X 23 X 24 X 9 X 10 X 7 X 25 X 4 X 3 X 12 X 6 X 11 X 8 X 13 X 5 X 22 X 14 F1 Facilit y /Su pp or t F2 Influential F3 Academic Resources F4 Environment/ Prestige X 1 F5 Quality/Reputation 19 19 X 19

PAGE 93

83 observed variables represent th e factor loadings, which indi cate the magnitude of the expected change in the observe d variable for a oneunit change in the factor, these are represented by the symbol Each of the items by factor is hypothesized to load only on the factor that it is intended to measure. The symbol represents the measurement errors of the X observed variables, which were not hypothesized to be correlated, and therefore should be estimated. In this confirmatory factor model the info rmation available are the elements of the covariance matrix for the observed variables: [p (p + 1) / 2], where p is the number of observed variables, th erefore the information available are [25(25 + 1) / 2] = 325 data points; the number of parameters to be estimated would be th e twenty five factor loading, plus the ten factor correlations, plus the tw enty five measurement error variances, for a total of 60 parameters. Similarly, the Figures 2 and 3 present the four and three first-order confirmatory factor models related to students’ perceptio ns of professor effec tiveness and university academic reputation, respectively. Figure 2 sh ows twenty-one observed variables (21 squares); F1 to F4 factors (4 ovals): Interested/Student Centered, Content/Pedagogical Knowledge, Behavior/Receptive, and Facilitation of learni ng, respectively, which are hypothesized to be correlated; twenty-one f actor loadings represented by the symbol ; and twenty-one measurement errors ( s). Figure 3 presents f ourteen observed variables (14 squares); F1 to F3 factors (3 ovals): Technology/Socio-cultural, Research Quality, and Prestige/ Quality, respectively, which are al so hypothesized to be correlated; fourteen factor loadings repr esented by the symbol ; and fourteen measurement errors ( s), one for each of the observed variables.

PAGE 94

84 Figure 2. Four-First Order Confirmatory Factor Analysis Model on Students Perceptions of Professor Effectiven ess, with Different Indicators per Factor. 5 5 8 8 1 6 1 6 X 11 X 13 X 12 X 18 X 19 X 20 X 1 X 2 X 5 X 4 X 9 X 3 X 15 X 8 X 14 X 10 X 16 X 7 X 21 X 17 X 6 11 11 12 12 1 3 1 3 1 8 1 8 1 9 1 9 2 0 2 0 1 1 2 2 3 3 14 14 15 15 1 7 1 7 21 21 7 7 9 9 1 0 1 0 4 4 6 6 F4 Facilitation of Learning F3 Behavior/Receptive F2 Content/Pedagog. Knowledge F1 Interested/Student Centered

PAGE 95

85 Figure 3. Three-First -Order Confirmato ry Factor Analysis Model on Students Perceptions of Academic Reputation, with Different Indicators per Factor. In these confirmatory factor models (Figure 2 and Figure 3) the information available are 231 and 105 data points, respec tively; the number of parameters to be estimated would be the twenty one factor load ings, plus the six fact or correlations, plus the twenty one measurement error variances, for a total of 48 parameters for the model two; and fourteen factor loadings, plus the three factor correlations, plus the fourteen measurement error variances, for a total of 31 parameters for the model three Since the 5 5 9 9 6 6 8 8 11 11 12 12 3 3 4 4 7 7 X6 X95 X8 X10 X11 F2 Research Quality X12 X3 X4 X5 F3 Prestige/Quality X1 X7 X2 X14 X13 F1 Technology/ Socio-Cultural 1 1 2 2 13 13 14 14 10 10

PAGE 96

86 data points are greater than the number of pa rameters to be estimated, the five-factor model, the four-factor model and the three-f actor model are identified and can be solved and in fact testable statistically. In confirmatory factor an alysis the identification of the model is a necessary condition to obtain correct estim ates of the parameter values. Identification refers to whether the parameters of the model can be uniquely determined, therefore, models with more information that unknown parameters are simply identified models and can be solved uniquely and tested statistically. In addition, the confirmatory factor model also should be identified if he has at least three items for each f actor, and if the variances of the factors are set equal to one. Maximum Likelihood Estimation (MLE) method was used to estimate all parameters in the first-order factor model, wh ich is based on the covariance matrix of the observed variables. MLE is one of the standa rd methods of estimating free parameters in confirmatory factor analysis. The main purpos e in estimating the factor model is to find estimates of the parameters that reprodu ce the sample matrix of variances and covariances of the observed variab les as closely as possible. A natural concern in the estimation methods is the sample size needed to obtain meaningful parameter estimates. This method assumes that when the sample size gets larger, the MLE is approximately unbiased a nd normally distributed. Consequently: the expected value of the sample estimates get closer to the true popul ation parameter; the variance of the sampling distribution of the MLE estimators becomes as small as possible with any estimator; and the sampling distribut ion of the estimator becomes normal. In the same way, Browne (1984) establishe d that the maximum likelihood parameter

PAGE 97

87 estimates in at least moderately sized samp les, also appear to be robust against a moderate violation of multivariate normalit y. Similarly, Tanaka, (1984) and Anderson and Gerbing, (1984) also argued that large sa mple sizes are needed in order to obtain correct parameter estimates, wi th small standard errors. Assessment of Fit Goodness-of-fit indices c oncern determining how well a model fits the data. In CFA th e assessment of model’s fit is not a simple process, this is because there is no established criterion or de finitive way to assess how well the specific model accounted for the data using some of the goodness-of-fit indices. Therefore, it is necessary to examine multiple fit criteria, although controversy still exists over the most appropriate indices to eval uate the model’s fit. Many indices have been developed (Ben tler, 1998; Bollen, 1986; Bollen, 1989; Bollen, 1990; Hu & Bentler, 1998; and Ma rsh, Balla, & McDonal d, 1988) to provide somewhat different information and for the purpose that general goodness-of-fit indices evaluate only certain aspects of a model. Therefore they must be used sensibly in connection with other methods for the evaluation of a m odel. Consequently, it is necessary to examine multiple fit criteria, thus in this study, an appropriate assessment of a model’s fit involves evaluation of the ove rall fit of the model and these that are concerning with the indivi dual parameter estimates. Traditionally to test the overall model’s fit, the chi-square statistic ( 2) derived from maximum likelihood has been used, whic h provides a test of the null hypothesis that the model fits the data. This statistic asse sses the magnitude of di screpancy between the sample and population covariance matrices of th e observed variables. A general form of the chi-square statistic ( 2) can be written as ( 2)ML = tr ( -1 S – I) – log | -1 S|, where tr

PAGE 98

88 is the trace of a matrix; is the population covariance matrix; S is the sample covariance matrix or unbiased estimator of a population covariance matrix; and ML is the maximum likelihood. In this study, chi-square was determ ined using the SAS System’s CALIS procedure and the maximum likelihood estimati on method. A smaller rather than larger chi-square value is indicativ e of a good fit, however, the 2 statistic is almost always statistically significant even when other goodness of fit indices reveal a relatively good fit to data. Consequently, it is one of the reasons for which Joreskog, (1969) recommended that the chi-square statistic be used more as a general goodness of f it index rather than a statistical test. However, given the known sensitivity of 2 to larger samples sizes and departures from multivariate normality, this study used se veral practical indices of fit, such as: Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Bentler’s Comparative Fit Index (BCFI), Bentler and B onett’s Normed Fit Index (BBNFI), Bentler and Bonett’s Non-normed Fit Index (BBNNFI), Bollen Normed Fit Index (BNFI), and Bollen Non-normed Fit Index (BNNFI); which have been proposed to evaluate the overall model’s fit for the analys is of covariance structures. These indices generally quantify the extent to which the variation and covariation in the data are accounted for by a model. Hu and Bentler (1998) adopted a distinction among those indices: absolute versus increm ental indices. An incremental fit index directly assesses how well an a priori model reproduces the sa mple data; in contrast, an incremental fit index measures the proportio nate improvement in fit by comparing a target model with a more restrictive model (a baseline model in which all the observed

PAGE 99

89 variables are allowed to have variances but are uncorrelated w ith each other). If any of those indices assume a value of .90, it can be approximately interpreted as being able to explain 90 percent of the covariati on among the measured variables. A general form of the absolute fit indices can be written as: a) the Joreskog and Sorbom, 1984, goodness of fit index: GFIML = 1 – [tr ( -1 S – I)2 / tr ( -1 S) 2], where tr, and S are defined as in 2; they stated that GFI is a m easure of the relative amount of variances and covariances jo intly accounted for by the model. Although no reference model is used to assess the amount of increm ent in model fit, an implicit or explicit comparison may be made to a saturated m odel that exactly reproduces the observed covariance matrix; and b) th e Joreskog and Sorbom, 1984, goodne ss of fit index adjusted for degrees of freedom: AGFI = 1 – [p (p + 1) / 2 df] (1 – GFI), where p is the number of observed variables; and df is the degrees of freedom for the model (Hu & Bentler, 1998). The Joreskog and Sorbom’s, 1984, Goodness of Fit Index (GFI), and the Adjusted (for degree of freedom) Goodness of Fit Index (AGFI) are two absolute fit indices that are analogous to R2 in multiple regression, by compari ng the goodness of fit using mean squares instead of total sum of squares. These indices are characterized according to the following properties: they should be between zero and one, although theoretically they can become negative; they are independent of sample size; and are relatively robust against violations of normality (Anderson & Gerbing, 1984). These indices are generally recommended since they are independent of sample size. Bentler’s Comparative Fit Index (BCFI), Bentler and Bonett’s Normed Fit Index (BBNFI), Bentler and Bonett’s Non-norme d Fit Index (BBNNFI), Bollen Normed Fit

PAGE 100

90 Index (BNFI), and Bollen Non-normed Fit Index (BNNFI) have been classified as incremental fit indices. A common form of the incremental fit indi ces can be written as: a) the Bentler’s comparative fit index: CFI = 1 – max [(TT – dfT), 0] / max [(TT – dfT), (TB – dfB), 0], where TT is the T statistic for the target model; dfT is the degrees of freedom for the target model, TB is the T statistic for the baseline model; and dfB is the degrees of freedom for the baseline model. The T is usually called the chi-square statisti cs, and the T = (N – 1)Fmin has an asymptotic (large sample) chisquare distribution, where F represents a discrepancy function F = F [S, ] that indicates the discrepancy between S and evaluated at an estimator and is minimized to yield Fmin; b) the normed fit indices: Bentler and Bonett (1980), nor med fit index, BBNFI = [(TB – TT) / TB], and Bollen (1986), normed fit index, BNFI = [(TB /dfB ) – (TT / dfT ) ] / (TB /dfB ), where TB, TT, and their degree of freedom are defined as in CFI; and c) the nonnorme d fit indice s: Bentler and Bonett (1980), non-normed fit index, BBNNFI = [(TB /dfB ) – (TT / dfT ) ] / [(TB /dfB ) 1], and Bollen (1989), nonnormed fit index, BNNFI = [(TB – TT) / (TB dfB)], where TB, TT, and their degree of freedom are defined as in CFI. These incremental indices range from zero to one, and the values of these indices also are based on the assumption that they ar e independent of sample size, except the Bentler and Bonett’s Non-normed Fit Index, wh ich is dependent on sample size, it is inversely related to sample size (Joreskog & Sorbom, 1981). Additionally, this study also used as m easure of goodness of fit the standardized residuals; which have been suggested to quant ify the extent to which the variation and covariation in the data are account ed for by the model (Bollen, 1986).

PAGE 101

91 In the evaluation of each model’s fit the following criteria were considered to indicate a reasonably good fit : a) the p value associated with the chi-square test should exceed .05 (the closer to 1.00, the better) (Hatch er, 1998); b) for the alternative fit indices values of .90 or greater (Bentle r & Bonett, 1980); the root mean square residual should be zero or close to zero (however, a liberal criterion of large residuals is a value of .10); standardized residuals whose absolute values do not exceed 2.0; and the t statistic values greater than 1.96 in absolute values are statistically significant (Hatcher, 1998). Multivariate Analysis of Variance (MANOVA) To address the research questions five and six multivariate analyses of vari ance were conducted to compare the means of the estimated factor scores acros s gender and university campuses. Multivariate analyses of variance with the groups gender and university campuses as the independent variables were designed to test simultaneously differences among the groups (gender and campuses) on multiple factors as dependent variables (i.e., factors influencing students’ decisi ons to enroll at the ULA, and their perceptions about professor’s effectiveness and university’s acad emic reputation). In consequence, an overall test of significance in MANOVA test the null hypothesis that the mean vectors of the groups are equal, which indicates that the groups are equal on all the dependent variables. MANOVA identifies a subset of dependent variables contributing to the difference among groups, and compares the mean s of different groups with respect to a set of different measures. The multivariate analysis of variance involves basically three assumptions: a) independence of the observations; b) multivariate normality on the dependent variables in

PAGE 102

92 each population; and c) equality of the populatio n covariance matrices (referred to as the homogeneity of the covariance matrices). Although the previous assumptions are re quirements for MANOVA, it is unlikely that all of the assumptions will be met exactly, therefore, violation of some assumptions do not necessarily invalidate the results. MANOVA is not robust to violation of independence of the observations, but may be robust to violations of the last two assumptions: multivariate normality on the dependent variables and homogeneity of the covariance matrices (Stevens, 1996). For checking normality assumptions Skewness (b1P) and Kurtosis (b2p) coefficients were used, since they are consid ered more powerful in detecting departures from normality (Stevens, 1996). For substa ntiating homogeneity of the covariances matrices the Box M test was used. The Box te st uses the generalized variances, that is the determinants of the within covariance matrices. Therefore, in this study a practical assessment of these assumptions was interpreted. In order to test the multivariate null hypot hesis, this study used the most widely known test statistic in MANOVA: Wilk’s lambda statistic. This test statistic tests whether there are differences between the means of id entified groups of subjects on a combination of dependent variables. Wilk’s lambda ( ) ranges from zero to one. Notice, also that Wilk’s lambda is an inverse criterion, the smaller the value of lambda the more evidence for treatment effect (Stevens, 1996). MANOVA produces a single F statistic that pe rmits to test the null hypothesis. If the overall test of significance in MANOVA using gender and university campus as the predictor variables is statisti cally significant, which consists of two and three groups,

PAGE 103

93 respectively, it is necessary to analyze th e univariate analyses of variance and to determine which combinations of groups differ significantly from each other. In consequence, this study used the Tukey mu ltiple comparison procedure to each of the dependent variables, in order to identify the specific differences.

PAGE 104

94 Chapter Four Results The purpose of this research was to examine the construct validation of an instrument to measure students’ decisions to enroll at the University of Los Andes and their perceptions about professor effectiv eness and university academic reputation. Additionally, a comparative analysis was car ried out to determine how the university selection process and the perceptions of e ffectiveness and reputation differ according to student demographic factors. The present chapter presents the results of the data analysis related to demographic variables associated with the students object of study and on the following research questions formulated in this study: 1. Are the student’s decisions of university choice process, and student’s perceptions of professor effectiveness and university a cademic reputation reliable within their respective factors at the Un iversity of Los Andes? 2. How well does the hypothesized measurem ent model involving five-first order factors fit the observed data based on student’s decision to enroll at the University of Los Andes? 3. How well does the hypothesized measurem ent model involving four-first order factors fit the observed data based on the student’s perceptions about professor effectiveness at the University of Los Andes?

PAGE 105

95 4. How well does the hypothesized measurem ent model involving three-first order factors fit the observed data based on student’s perceptions of university academic reputation at the University of Los Andes? 5. What are the differences across gender in perceived importance of the selected factors that influence the students’ deci sions about university choice process, and their perceptions of profe ssor’s effectiveness and university’s academic reputation at the University of Los Andes? 6. What are the differences across univers ity campuses in perceived importance of the selected factors that influence the st udents’ decisions abou t university choice process, and their perceptions of prof essor’s effectivene ss and university’s academic reputation at the University of Los Andes? The results of this study include the fo llowing sections: descriptive statistics, reliability, confirmatory factor analysis (CFA), and multivariate analysis of variance (MANOVA) in relation to the su rvey instrument (paper and pencil questionnaire). Descriptive Statistics Prior to addressing the res earch questions, a summary of characteristics associated with student demographic information were analyzed. Additionally, item means and standard deviations, and percei ved importance of factors rela ted to the three domains in this study: students’ decisions to enroll at the ULA, and thei r perceptions about professor effectiveness and university academic reputa tion, were summarized and analyzed using descriptive statistics.

PAGE 106

96 A summary of the percentages of the students’ demographic information is presented in Table 4. In a ddition, the data on gender, age, geographic region, college, admission’s policy, parents’ educational level, family income, and semester of study were obtained from each of the one thous and students registered in the courses being offered in the second semester of 2002; of which 40.8 % of the respondents were females, while 59.2 % were males. The major percentage of the students (44.8 %) had a range of age between 22 and 25 years; the great percentage of students (15.1 %) are enrolled within the College of Economic and Social Scie nces (FACESMerida), 13.2 % in the NURR university branch (Trujillo), 11.5 in the Co llege of Engineering (Merida), 11.4 in the NUTULA university branch (Tachira), 11.4 % in the College of Humanities and Education (Merida), 9.9 % inside the College of Medicine (Merida), and the remaining percentage under 9.4 ar e within of the Colleges of Ar chitecture, Forest, Pharmacy, Sciences, and Dentistry, all th em located in Merida. Other data include the parent’s educationa l level and student’s family income: the major mother’s educational level of the students was se condary (44.8 %), while the father’s educational level was superior (39.5 %) ; with respect to students’ family income, the major percentage of students (31.6 %) was assigned to third social stratum representing from 600,000 to 899,000 bolivars by month (from $ 312.5 to $ 468.75, because each dollar is equivalent to 1,920 boliv ars); and the last data are related to the semester of study; the major pe rcentage of students selected in the sample (28.0 %) were enrolled in the eighth semester of study. From the summary of characteristics a ssociated with students’ demographic information could be obtained some important inferences related to gender, parents’

PAGE 107

97 Table 4. Students’ Demographic Information ________________________________________________________________________ Characteristic Percentage Characteristic Percentage Gender Admission’s Policy Female 40.8 OPSU 24.5 Male 59.2 PINA 57.5 Special Admission 18.0 Age 18 21 27.9 Mother’s Educational Level 22 – 25 44.8 Primary 13.5 26 – 29 16.6 Secondary 44.8 30 – 33 6.2 Superior 29.7 > 33 4.5 Other 10.9 Did not respond 1.1 Geographic Region Los Andes 62.7 Father’s Educational Level Oriental 12.1 Primary 10.0 Central 7.7 Secondary 34.1 Occidental 5.6 Superior 39.5 Centro-Occidental 3.7 Other 12.9 Los Llanos 6.7 Did not respond 3.5 Guayana 1.5 Family Income (Thousand) Faculty (College) < 300 4.2 NURR 13.2 300 – 599 21.4 NUTULA 11.4 600 – 899 31.6 Education 9.4 900 – 1199 27.3 Engineering 11.5 > 1199 15.5 Sciences 3.5 FACES 15.1 Semester Forest 3.9 Fifth 15.9 Pharmacy 3.7 Sixth 14.7 Medicine 9.9 Seventh 14.5 Dentistry 1.8 Eight 28.0 Laws 11.4 Ninth 13.3 Architecture 5.2 Tenth 13.6 Note: n = 1000.

PAGE 108

98 educational level, geographic region and ad mission’s policy. The comparative analysis of the data about gender and parents’ educat ional level suggests an important inference about these two variables: Approximately 40% of the fathers, but only 30% of the mothers have university level education. Si milarly, 60% of the current students are males; these results suggest that the university is contributing to preserve the professionalism of male gender. The relation by geographic region among the students is approximately for every 3 students, 2 belong to the Andes Region. Because the University of Los Andes is located in the three states that conform this region, most of the students that beginning higher education, select the ULA as a first option. Further, th e prestige of this university extends to the whole country, which explai ns why students from all the Venezuelan regions also choose this university. The item means and standard deviations fo r the three domains considered in this study are presented in Table 5. All item mean s and standard deviations were calculated based upon a five-point rati ng scale, where 1 = extremely low importance and 5= extremely high importance, in students’ decisi ons to select the ULA, and where 1 = poor and 5 = excellent, in students’ perceptions about professor effec tiveness and university academic reputation. As indicated in Table 5, the items I1, I 2, I4, I3, and I9, “academic reputation of the university, quality of the pr ofessors, quality of the teachin g, quality of the programs, and value of a degree from this university”, had the highest mean ratings of importance (having high to extremely high importance, mean > 4.0) by the students’ decisions to

PAGE 109

99 select the ULA; other items I7, I5, I20, and I 6, “Interest in a specific program, size of the university, campus social environment, and size of the college/school”, also had high mean rating of importance by the students (mean > 3.65); whereas items I24 and Table 5. Item Means and Standard Deviations by Domain __________________________________________________________ Item Number Item Description Mean S.D. Domain 1: Student University Choice : I1 Academic reputation of the university 4.43 0.57 I2 Quality of the professors 4.23 0.53 I4 Quality of the teaching 4.13 0.61 I3 Quality of the programs 4.05 0.64 I9 Value of a degree from this university 4.01 0.80 I7 Interest in a specific program 3.80 0.99 I5 Size of the university 3.76 1.03 I20 Campus social environment 3.69 0.95 I6 Size of the college/school 3.67 1.00 I12 Library facilities and collections 3.65 1.09 I14 Use of technologies 3.63 1.02 I18 Scholarship received 3.59 1.15 I10 University’s geographic location 3.56 1.06 I13 Research and computer facilities 3.52 1.12 I15 Availability of university dining hall 3.47 1.19 I17 Availability of university transportation 3.43 1.21 I22 Good possibilities of job 3.43 1.13 I8 Length of time to degree 3.41 1.06 I11 Closeness to home 3.40 1.16 I21 Availability of part-time work 3.38 1.07 I23 Parent’s influence 3.08 1.47 I19 University athletic programs 3.07 1.05 I16 Availability of university residences 3.04 1.34 I24 Other family influences 2.59 1.39 I25 Friend’s influences 2.41 1.37 Domain 2: Professor Effectiveness: I2 Breadth of knowledge of subject matter 3.90 0.61 I14 Definition of class objectives clearly 3.87 0.69

PAGE 110

100I21 Overall professor’s assessment 3.85 0.61 I1 Preparation for class 3.86 0.70 I17 Clarity in presentations and explications 3.83 0.62 I3 Interpretation abstract ideas and theories clearly 3.59 0.73 I5 Support ideas with examples, comparisons, and facts 3.51 0.76 I4 Stress important material 3.50 0.76 I15 Use varied lecturing strategies to enhance learning 3.49 0.71 I6 Inclusion of out-of-text materials in lectures 3.48 0.74 I9 Use class time efficiently 3.47 0.79 I19 Regard students’ opinion 3.47 0.72 I10 Enthusiastic for teaching 3.43 0.72 I11 attentiveness to student’s needs and concerns 3.43 0.77 I20 Encourage open communication 3.43 0.71 I8 Self-controlled and patient 3.43 0.75 I7 Receptiveness to student’s ideas and questions 3.42 0.77 I16 Use appropriate evaluation/assessment methods 3.40 0.74 I12 Willing to help students 3.38 0.85 I13 Concerned about fair evaluation of students 3.37 0.75 I18 Use flexible course structure 3.30 0.68 Domain 3: University Academic Reputation: I13 Recognition of the university’s name 4.22 0.75 I14 Overall academic reputation of theULA 4.22 0.70 I3 Quality of research centers 3.95 0.71 I4 Quality of research institutes 3.95 0.72 I5 Quality of research laboratories 3.92 0.71 I7 Quality of published research 3.91 0.77 I1 Professors’ quality 3.84 0.56 I2 Alumni’s quality 3.73 0.74 I9 Admission policies 2.63 0.89 I10 Social environment 3.60 0.92 I6 Quality of libraries 3.53 0.87 I8 Use of educational technology 3.32 0.89 I12 Athletic programs 3.18 1.01 I11 Cultural activities 3.12 0.91 Note: n = 1000 for all items. I25, “other family members’ influences and friends’ influences”, had the lowest mean rating of importance by the stud ents (mean: 2.54 and 2.41, respectively). Also of interest is the fact that the high-rated items show ed less variability (mean = 4.43, S.D. = 0.57) than the low-rated items (mean = 2.41, S.D. = 1.37).

PAGE 111

101 The high-rated and low-rated items by the students’ perceptions about professor effectiveness and university academic reput ation also showed th e same pattern of variability as those of the stude nts’ decisions to select the ULA. The highest rated items by students’ perception of profe ssor effectiveness were I2, I14, I21, I1, and I7 (mean > 3.82), “breadth of knowledge of subject matter, definition of classes objectives clearly, encourage open communication, preparation fo r class, and receptiveness to students’ ideas and questions”, while the lowest ra ted items were I13 and I18 (means: 3.37 and 3.30, respectively), “Concerned about fair evalua tion of students and flexibility in course planning”. The S.D. for the highest-rated items was 0.61, whereas the S.D. for the lowest-rating items was 0.85. The items I13, I14, I3, I4, I5, and I7, “recognition of the university’s name, overall academic reputation of theULA, quality of research centers, quality of research institutes, quality of research laboratories, and quality of published research”, had the highest mean rating (mean >3.91) by student s’ perceptions about university academic reputation; whereas the lowest rated items were I12 and I11 (mean: 3.18 and 3.12, correspondingly), “Athletic programs, and cultural activities’. The high-rated items also showed less variability than the low-rated items. Table 6 shows the average ratings, the rank order of the average, and the standard deviations for each factor related to univers ity choice process, professor effectiveness, and university academic reputation. As i ndicated in this Table, the factor 5 “Quality/Reputation” was rated as substantially higher factor influencing university choice process (mean = 4.21, S.D.= .43), as comp ared to the other factors. Factor 4 “Environment/Prestige” and factor 3 “ Academic resources” were rated as the second and

PAGE 112

102 third highest factors in its influence about university choice (mean = 3.70, S.D. = .49; and mean = 3.53, S.D = .59, respectively), while th e factor 2 “Influential” was the factor suggesting very little influence in decidi ng upon the University of Los Andes. These results reveal that the students seen to prefer higher quality university. Table 6. Ratings of Importance Factors by Domain _________________________________________________________ Factor Number Factor Description Mean Rank S.D. Domain 1: University Choice Process F5 Quality/Reputation 4.21 1 0.43 F4 Environment/Prestige 3.70 2 0.49 F3 Academic Resources 3.53 3 0.59 F1 Facility/Support 3.38 4 0.80 F2 Influential 2.70 5 1.14 Domain 2: Professor Effectiveness F2 Content/Pedagogical Knowledge 3.67 1 0.51 F1 Interested/Student Centered 3.31 2 0.63 F4 Facilitation of Learning 3.31 3 0.72 F3 Behavior/Receptive 3.30 4 0.63 Domain 3: University Academic Reputation F3 Prestige/Quality 4.06 1 0.52 F2 Research Quality 3.91 2 0.68 F1 Technology/Socio-Cultural 3.23 3 0.65 Note: n = 1000. As indicated in Table 6, the factors that characterized professor effectiveness are interested/student centere d, content/pedagogical knowledge behavior/receptive, and facilitation of learning. These outputs s how that factor 2 “Content/Pedagogical

PAGE 113

103 Knowledge” was the most highly rated dimens ion of effectiveness (mean = 3.67, S.D. = .51), following for the factors 1 “Interested/St udent Centered” and factor 4 “ Facilitation of Learning” (mean = 3.31), which show that professor effectivene ss was seen as very concerned about students’ needs and helping students’ learning. The factor 3 “Behavior /Receptive” was considered by students to be important (mean = 3.30, S.D. = .63), but not as important as other factors. The factors that characterized university academic reputation are also presented in Table 6. As indicated in this table, factor 3 ”Prestige/Quality” wa s the most highly rated factor related to university academic reputa tion (mean = 4.06, S.D. = .52), this indicating that the university (ULA) is seen as havi ng a high level of reputation, given by its recognition of the university’ name, professors and alumni. The second most important factor was ”Research Quality” (mean = 3.91, S. D. = .68), this result also shows that the reputation of this university is seen as impor tant by the quality of its research centers, institutes, and laboratories, and the published research. Factor 1 “Technology/ SocioCultural” was the lowest rated factor, which is seen by its technology, social environment, and cultural and athletic program s. These findings show also that the highrated factors showed less variabil ity than the low-rated factors. Reliability Research Question # 1 In order to answer the research ques tion number one: Are the student’s decisions of university choice process, and student’s perceptions of professor effectiveness and university academic reputation re liable within their respective factors at the University of

PAGE 114

104 Los Andes? Cronbach alpha internal consistenc y reliability was determined for the three domains considered in this study and by factor across the three domains. A summary of the internal consistency reliability by do main and factor across the domain is provided in Table 7. As indicated in the Table, the reliability estimates for the three domains under study revealed adequa te reliability, all exceeded the value .70 (minimum value suggested by Nunna lly, 1978): 75, .92, and .91, respectively. Table 7. Internal Consistency Reliability by Domain and Factor across Domain By Domain Indices Domain 1: Students’ decisions to select the ULA..……………. .75 Domain 2: Students’ perceptions about professor’s effectiveness .92 Domain 3: Students’ perceptions about university academic reputation. …………………………………………. .91 By Factor Students’ decision to select the ULA Factor 1: facility/ support ……………………………… .83 Factor 2: influential. …………………………………. .74 Factor 3: academic resources. …………………………. .41 Factor 4: prestige ...…………………………………….. .41 Factor 5: quality/reputation. …………………………. .71 Students’ perception of professor effectiveness Factor 1: interested/student centered. …………………. .83 Factor 2: content and pedagogical knowledge ………… .86 Factor 3: receptive/behavior …………………………… .71 Factor 4: facilitation of learning………………………... .70 Students’ perception of unive rsity academic reputation Factor 1: technologi cal/socio-cultural ………………… .80 Factor 2: research quality ……………………………… .92 Factor 3: prestige ……………………………………… .81 _________________________________________________________ Note: n = 1000 for all items.

PAGE 115

105 Table 7 also provides a summar y of the reliability estimates by factor across the three domains. These results also suggest adequate reliability, although some of them are more than adequate, the reliability estimates fo r the factors 3 and 4, academic resources and prestige, respectively, related to students’ deci sions to select the ULA, are not acceptable. These low reliabilities are due to some items of the factors demonstr ating poor item-total correlation, which is evidence that these it ems are not measuring the same factor. Specifically factor 3 presents a problem with item 11 (closeness to home), which demonstrated a low item-total correlation (.067) ; in addition it pres ents insignificants r2. Factor 4 also presents a problem with ite m 8 (length of time to degree) and item 10 (university’s geographic locat ion), which shows an item-to tal correlation strongly low (.05 and .06, respectively); both ite ms also present insignificant r2. These results suggest that items should be dropped from the instrume nt, in order to improve the reliability of the scale. Once realized the changes suggested by the outputs related to internal consistency reliability on the initial scale, the reliabi lity values for the modified scale related to students’ decisions to sel ect the ULA, revealed a re latively slight increase of approximately three percent (.77), when it is co mpared to the reliability coefficient of the initial scale (.75). The reliabil ity of factors three (academic resources) and four (prestige) shown a much better increase; factor three of approximately seven percent (.44) and factor four of approximately twenty seven percent (.52), when they are compared with the early factor’s reliability (.41 a nd .41, respectively) These findings show that the performed changes resulted in higher reliability values; therefore, these modifications (removing items I8, I10, and I 11) increased estimated scale reliability.

PAGE 116

106 Although future research should utilize this redu ced set of items, subsequent analyses in the current study were conducted with th e full set of the instrument items. Confirmatory Factor Analysis In order to answer research questions two to four three separate confirmatory factor analyses were performed to eval uate the hypothesized measurement models underlying the students’ decisi ons to select the ULA a nd their perceptions about professor effectiveness and university academ ic reputation. The use of confirmatory factor analysis assumes that a number of requirements have been met concerning the nature of the data as well as the confirmatory factor model. So, it is necessary that some important assumptions associated with this an alysis (e.g., normally distributed data, lack of variability in items, absence of mu lticolineality) be inspected and satisfied. Skewness and kurtosis coefficients across the three domains are summarized in Table 8. As indicated in the Table, skewne ss and kurtosis were computed for the twenty five (25), twenty-one (21), and fourteen ( 14) variables (items) related to students’ decisions to select the ULA, and students’ perceptions about profe ssor effectiveness and university academic reputati on, respectively. Th ese results show that skewness and kurtosis of the univariate dist ributions have a slight departure from normality. Each measured variable showed slightly nonnorma lity with skewness values less than .75 and kurtosis values less than 1.2. It is important to underline that slight nonnormality and large sample size lead to robust standard errors that pr ovide generally accurate para meter estimates, since with

PAGE 117

107 small and moderate sample size it is difficu lt to tell whether the nonnormality is evident or actual (Chou, Bentler, & Satorra, 1991). As indicated in Table 5, each measured variable (items) had acceptable variability, which constitutes an important and necessary condition in the confirmatory factor analysis. The assumption of multicollinearity was evaluated by examining the correlation matrix. Multicollinearity results wh en two or more variables correlate highly with each other (above .80). The study of the correlation matrix showed that the measured variables are free of multicollinearity, since they demonstrated low correlations with one another (less than .67) Table 8. Skewness and Kurtosis Coefficients by Domain Student University Choice Professor Effec tiveness Academic Reputation _________________________________________________________________ Item Skewness Kurtosis Item Skewness Kurtosis Item Skewness Kurtosis _________________________________________________________________ I1 -0.39 -0.77 I1 -0.02 -0.12 I1 -0.28 0.01 I2 -0.15 -0.20 I2 -0.29 0.57 I2 -0.10 -0.32 I3 -0.04 -0.59 I3 -0.20 -0.21 I3 -0.30 0.03 I4 -0.8 -0.42 I4 -0/13 -0.37 I4 -0.29 -0.14 I5 -0.86 0.47 I5 -0.01 -0.35 I5 -0.25 -0.15 I6 -0.64 0.01 I6 0.07 -0.31 I6 -0.49 0.27 I7 -0.78 0.34 I7 0.05 -0.36 I7 -0.30 -0.35 I8 -0.51 -0.26 I8 0.14 -0.28 I8 -0.24 -0.26 I9 -0.48 -0.28 I9 0.07 -0.44 I9 -0.14 -0.73 I10 -0.60 -0.12 I10 -0.01 -0.29 I10 -0.28 -0.26 I11 -0.52 -0.54 I11 0.13 -0.34 I11 -0.17 -0.18 I12 -0.74 -0.06 I12 -0.11 0.07 I12 -0.28 -0.38 I13 -0.64 -0.27 I13 0.12 -0.30 I13 -0.72 0.17 I14 -0.72 -0.21 I14 -0.44 0.43 I14 -0.66 0.38

PAGE 118

108 I15 -0.37 -0.76 I15 -0.23 -0.25 I16 -0.08 -1.11 I16 -0.03 -0.35 I17 -0.47 -0.69 I17 -0.27 0.32 I18 -0.58 -0.45 I18 0.06 -0.20 I19 -0.32 -0.48 I19 -0.03 -0.29 I20 -0.49 0.07 I20 0.13 -0.20 I21 -0.28 -0.53 I21 0.09 -0.41 I22 -0.57 -0.38 I23 -0.20 -1.16 I24 0.27 -1.18 I25 0.44 -1.10 ___________________________________________________________________ Notes: n = 1000 for all items. Item description is presented Table 5. Confirmatory Factor Analysis by Domains Research Question # 2: How well does the hypothesized measurem ent model involving five-first order factors fit the observed data ba sed on students’ decisions to select the University of Los Andes? In order to answer research question tw o, a confirmatory factor analysis was performed to evaluate the hypothesized m easurement model underlying the students’ decisions to select the ULA. Specification, Identification, and Estimati on of the Five-First-Order Factor Model. As was presented previously, the m easurement model proposed to measure the university choice process consis ts of twenty-five measured variables and five factors, which are assumed as: facility/support (it has se ven observed variables), Influential (three observed variables), Academic Reso urces (five observed variables),

PAGE 119

109 Environment/Prestige (six observed variables), and Quality/Reputation (four observed variables). Notice that each of the measured variables for each factor was predicted to load on only the factor it was proposed to meas ure; the five factor s were all hypothesized to correlate with one another; there are no covariances between any of the measured variables and the standard errors were not hypothesized to be corre lated. The residual term was also created by measured variable; and the factor variance s were set to one in order to assume identification of confirmatory factor model. Furthermore, since the data points [25( 25 + 1) / 2 = 325] are greater than the number of parameters to be estimated (twent y five factor loading, plus the ten factor correlations, plus the twenty five measur ement error variances, for a total of 60 parameters), the five-factor model is identifie d and it can be solved and in fact testable statistically. In addition, the c onfirmatory factor model also should be identified if it has at least three items for each factor, which is satisfied in this case. After identification has been established, estimation of the confirmatory factor model can proceed. All analyses in this study were conducted usi ng the SAS system’s CALIS procedure, which used the maximu m likelihood method of parameter estimation in the model. Factor loadings, t-values, standard errors, and error variances are presented in Table 9. Factor loadings indi cate the unique contributions th at each factor makes to the variance of the observed variable s. A high factor loading is considered when it is equal or greater than .40 (Hatcher, 1998) which also means that the variable is measuring that factor. The t-values present th e test of the null hypothesis that the factor loading is equal

PAGE 120

110 to zero in the population. The t-va lues greater than 1.960, 2.576, and 3.291 are significant at p < .05; at p < .01, and at p < .001, respectively (Hatcher, 1998, p. 295). The obtained t-values for all f actor loading coefficients dem onstrated to be statistically significant at p < .001, indicating that th ey were meaningful coefficients, with the exception of the factor loading related to I11 and I10 that illustrate to be statistically significant at p < .05, and the factor loading to I8 that showed to be statistically significant at p < .10. The factor loading coefficients associated with these items (I11, I10, and I8) showed values substantially low (between .06 and .010). However, from these results, one can conclude that greater pa rt of the factor loadi ngs were significant. The fourth column in the Table contains the standard error for loading of each measured variable on its intended factor. The error values range from 0.025 to 0.043, Table 9. Factor Loading, t-Value, Standard Error and Error Variance Estimates Related to Students’ Decisions to Select the ULA. _________________________________________________________ Item-Factor Loading ( ) t-Value Stand. Error Error Variance ___________________________________________________________ Factor 1: Facility/Support Item15 0.73 25.42 0.029 .47 Item16 0.81 29.27 0.028 .35 Item17 0.79 28.54 0.028 .37 Item18 0.73 25.30 0.029 .47 Item19 0.52 16.42 0.031 .73 Item20 0.30 8.82 0.033 .91 Item21 0.55 17.63 0.031 .70 Factor 2: Influential Item23 0.59 17.68 0.033 .60 Item24 0.83 24.15 0.025 .42 Item25 0.70 20.72 0.033 .50 Factor 3: Academic resources Item11 0.10 2.26 0.041 .99 Item12 0.30 6.80 0.041 .92 Item13 0.52 12.40 0.042 .73

PAGE 121

111 Item14 0.60 13.43 0.043 .66 Item22 0.24 5.81 0.041 .94 Factor 4: Environment/Prestige Item05 0.72 19.31 0.037 .48 Item06 0.67 18.21 0.037 .55 Item07 0.20 4.99 0.037 .96 Item08 0.06 1.58 0.038 .99 Item09 0.25 6.90 0.037 .93 Item10 0.08 2.03 0.038 .99 Factor 5: Quality/Reputation Item01 0.62 18.40 0.033 .62 Item02 0.60 18.67 0.034 .68 Item03 0.65 19.50 0.033 .58 Item04 0.63 18.91 0.033 .60 Notes: n = 1000 for all items. Item description is presented Table 5. which showed no problematic values (such as 0.0003) for acceptable errors, therefore, these results presented reasonable values for all measured variables. The covariances estimated between every pa ir of factors are summarized in Table 10. The covariances were estimated for every possible pair of fact ors since all latent variables are normally allowed to covary in a confirmatory factor analysis. The estimated covariances of the factors de monstrated reasonable values (ranges from .16 to .63), except the covariances between the factor two and five, two and three and factor two and four, which showed insignificant values (-.08, .06 and .01, respectively), and the correlations between these pairs of factors were near zero. The standardized factor loadings were at least moderately large (from .30 to .83), except the standardized coefficients to the items 8, 10, and 11, which are under .10. Similarly, the r-square for the measured variab les revealed moderately large coefficients

PAGE 122

112 (from .30 to .70), with the ex ception of the r-square relate d to the items 8. 10, and 11, which are under .10. Figure 4 contains a diagram of the parameter estimates for the first model related to students’ decisions to enro ll at the University of Los Andes, and it does not provide new information with respect to the data. As shown in Figure 4, the factors are indicated within the ovals and the twenty -five measured variables within the rectangles; straight lines pointing to each indicator with the load ing associated with the variable denote the effects of the latent factors; the effect of m easurement error is marked with a straight line to the indicator variable; the covarian ces are denoted with curved paths. Assessment of Fit in the FiveFirst-Order Factor Model. Assessment of fit involves conducting hypothesis te sts to assess the statisti cal significance of individual parameters and overall fit of the model to the da ta set. The criteria used to assess fit of the model were: examination of the values of individual parameter estimates and their Table 10. Interfactor Correlation, Standard Erro rs, and t-Values for University Choice Process ________________________________________________________ Pair of Factor Estimated Standard Error t-Value ________________________________________________________ CF1F2 0.25 0.04 6.79 CF1F3 0.41 0.04 9.36 CF1F4 0.16 0.04 3.97 CF1F5 0.27 0.04 7.23 CF2F3 0.06 0.05 1.21 CF2F4 0.01 0.04 0.16 CF2F5 -0.08 0.04 -1.75 CF3F4 0.41 0.05 8.14 CF3F5 0.51 0.05 10.76 CF4F5 0.63 0.03 17.61

PAGE 123

113 standard errors to test the statistical signi ficance; evaluation of th e overall fit of the model, such as: evaluation of the overall chisquare in terms of statistical significance, evaluation of the alternative indices of goodness of fit: Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Be ntler’s Comparative Fit Index (BCFI), Bentler and Bonett’s Normed Fit Index (BBN FI), Bentler and Bonett’s Non-normed Fit Index (BBNNFI), Bollen Normed Fit Index (BNFI), and Bollen Non-normed Fit Index (BNNFI); and examination of the normalized residuals in order to determine the similarity between the elements of the original and predicted matrices; and finally an examination of the model modification indice s to determine which specific modification might best improve the fit if th e a priori model is inadequate. Individual parameter values were analyzed to test statistical significance. A great part of these results obtained demonstrat ed to be statistical ly significant at p < .001. The factor loadings I8, I10, and I11 showed values substantially low (.06, .08, and .10, respectively). From these results, one can conclude that greater part of the factor loadings were significant. A summary of the data used to assessme nt the overall goodness of fit of the fivefirst-order factor model related to students’ de cisions to enroll at the University of Los Andes is presented in Table 11. Estimation of the model revealed a significant chisquare. The chi-square value of 990.86 with 265 degree of freedom is significant with a probability of .0001, indicating that the model do es not provide an adequate fit to the data. This significant value, however, was e xpected and it may be for the reason that the chi-square value is in part due to the large sample size used in the study, rather than to

PAGE 124

114 .73 .47 .35 s .81 .37 .79 .73 .47 .73 .52 .30 .91 .16 .41 .25 .55 .70 .71 .60 .71 .42 .71 .50 .27 .01 .06 .10 .99 .30 .92 .52 .73 .60 .66 .24 .94 .51 .41 -.08 .72 .48 .67 .55 .20 .96 .06 .99 .25 .93 .63 .08 .99 .62 .62 .60 68 .65 .58 .63 .60 F1 Facility/Support X15 X21 X16 X18 X17 X19 X20 F2 Influential X21 X23 X25 X24 X9 X10 X7 X25 F3 Academic Resources F4 Environment/ Prestige F5 Quality/Reputation X22 X22 X12 X6 X11 X8 X13 X5 X22 X14 Figure 4. Estimates Data for Five-First-Order Factor Model Related to Students’ Decisions of University Choice Process

PAGE 125

115 misspecification of the model, since the mode l was identified according to the criteria used to this purpose. However, some of the alternative fit i ndices revealed a relatively good fit even when the ( 2) test suggests rejection of the model. For example, the goodness of fit index (GFI), the adjusted goodness of fit index (AGF I), the Bentler’s comparative fit index, and Bollen non-normed index of .923, .905, .863, and .864 respectively, are at or close to the acceptable criterion of .90, used by many resear chers as an indication of a good fit to the data, indicating that these indices have an acceptable fit of the five-first-order factor model related to students’ decisions to enro ll at the University of Los Andes. Moreover, although the altern ative indices of the ove rall fit: Goodness of Fit Index (GFI), Adjusted Goodness of Fit Inde x (AGFI), Bentler’s Comparative Fit Index (BCFI), Bentler and Bonett’s Non-normed Fit Index (BBNNFI), and Bollen Non-normed Fit Index (BNNFI), demonstrated values th at exceed or are n ear the criterion of.90, indicating an adequately fit to the da ta, other indicators as: the significant Table 11. Goodness of Fit Indices for the Mode l in University Choice Process ______________________________________________________ Indices Value Chi-square ( 2) 990.86 Degree of freedom (df) 265 p -value > chi-square < .0001 Goodness of Fit Index (GFI) 0.923 Adjusted Goodness of Fit Index (AGFI) 0.905 Bentler’s Comparative Fit Index (BCFI) 0.863 Bentler and Bonett’s Normed F it Index (BBNFI) 0.823 Bentler and Bonett’s Non-normed Fit Index (BBNNFI) 0.850 Bollen Normed Fit Index (BNFI) 0.800 Bollen Non-normed Fit Index (BNNFI) 0.864

PAGE 126

116 chi-square test, some factor loadings (I8, I10, and I11) and the remaining alternative indices: Bentler and Bonett’s Normed Fit Index (BBNFI) and Bollen Normed Fit Index (BNFI) with values less than .90, indicate that the model does not provide an adequate fit to the data. Consequently, these results reveal that the fit of the model to the data could possibly be significantly improved, consid ering the outputs of these indicators. Another measure of overall fit is the examination of the normalized residual matrix. Thus, considering that the norm alized residuals over 2.00 are generally considered large and therefore problematic (Hatcher, 19980), the average standardized residual showed a moderate abso lute value of 1.61; however, some of the elements of this matrix revealed absolute values that ex ceed 2.00, which indicates that there are some problems with the theoretical model formulated. Consequently, given that some overall fit indices showed values less than .90, and the model had statistically significant chi-s quare, and demonstrates significant problems with some of the standardized residuals and with some of the factor loading estimates, it was considered important to examine the m odification of the model with the propose of formulating a posteriori model that w ould fit the data more adequately. This is carried out by making some modifications in the initial model that will result in improvement in overall model fit. In practice, a number of modifications should be carried out to determine how the m odel should be changed, however, several considerations may be supposed, in order to avoid nongeneralizable models: a) use large samples (n = 800 – 1200), since small samp les model modification leads to poor outcomes; b) make few modifications: only the first few changes have a reasonable possibility of leading to a relatively large improvement in fit, and then the model will

PAGE 127

117 generalize; each successive change resulting smaller improvements; and c) only perform those modifications that can be justified in light of existing theory or prior research (Joreskog and Sorbom, 1989, in Hatcher, 1998). Taking account the last considerations, the modification of the initial model should be carried out using in overall goodness of fit: a) the findings of the statistical significance of the individual pa rameters; and b) the indices that may be useful in suggesting possible model modifications, such as the Wald and Lagrange Multiplier tests that are available in the SAS System’s CALIS procedure. These indices identify parameters that should possibly be dropped and added, respectively, from the model; and estimate the decrease in the chi-square value that would result if a given parameter were to be added/dropped to the model. Realized the changes suggested by the si gnificance of the individual parameter (drop I8, I10, and I11), the results related to the five-first-order modified model, in students’ decisions of university choice, s howed a chi-square for the revised model of 865.34, with 200 degrees of freedom, this chi-square value is still stat istically significant (p < .0001). These results show a moderate descend of approximately thirteen percent from that observed with the initial meas urement model, where chi-square was 990.86, with 265 degrees of freedom. By doing these changes the model’s ch i-square decreased by 125.52, while the degree of freedom decreased by only 66. Th is modification model shown a relatively moderate decrease in chi-square when it is compared to the changes in degrees of freedom, consequently this transformation show ed that dropping thes e variables from the model improvement the model’s fit.

PAGE 128

118 A summary of the data used to assess the overall goodness of fit of the five-firstorder modified model related to students’ de cisions to enroll at the University of Los Andes is presented in Table 12. As indicated this Table, some of the alternative indices for assessing the overall goodness of fit are no t only acceptable, they are also somewhat higher than those observed with the initial model. These findings reveal that the modified measurement model provides significa nt factor loadings (all are statistically significant at p < .001); and shows an accepta ble fit to the data, indicated by the Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Bentler’s Comparative Fit Index (BCFI), Bentler and Bonett’s Non-normed Fit Index (BBNNFI), and Bollen Non-normed Fit Index (BNNFI), whose values are .923, .902, .873, .853, and .873, respectively. Therefore, these results pr ovide support for the modified model, since the reliability of the factors al so performed more adequately. The Wald test suggests that the covarian ces between factor two (F2) and factor four (F4) and between factor two (F2) and factor three (F 3), and the factor loading Table 12. Goodness of Fit Indices for the Modified ______________________________________________________ Model in University Choice Process Indices Value Chi-square ( 2) 865.34 Degree of freedom (df) 200 p -value > chi-square < .0001 Goodness of Fit Index (GFI) 0.923 Adjusted Goodness of Fit Index (AGFI) 0.902 Bentler’s Comparative Fit Index (BCFI) 0.873 Bentler and Bonett’s Normed F it Index (BBNFI) 0.841 Bentler and Bonett’s Non-normed Fit Index (BBNNFI) 0.853 Bollen Normed Fit Index (BNFI) 0.820 Bollen Non-normed Fit Index (BNNFI) 0.873

PAGE 129

119 estimate for I8 (length of time to degree) may need to be dropped or substantially modified from the model. However, in this case only the factor lo ading could be dropped since it was not doing a good job in the meas urement of the factor to which it was assigned (F4: environment/prestige); and the factor covariances will not be dropped from the model simply because were nonsignificant, du e to in confirmatory factor analysis all factors are normally allowed to covary in the analysis. Realized the change suggested by the Wald test (drop I8), the results of the fivefirst-order modified model, related to uni versity choice process, demonstrated a chisquare of 966.49 with 242 degree of freedom significant at p < .0001. These results show a small descend of approximately 2.5 per cent as compared with the chi-square of initial measurement model. Moreover, the alternative indices for assessing the overall goodness of fit were smaller than those observe d in the initial mode l; consequently, this modification can not be justified since it does not provide a acceptable fit to the data. On the other hand, the Lagrange Multiplier test suggests that th e greatest decrease in the overall chi-square value would occur if variables I1 (academic reputation of the university) was allowed to load on factor 4 “Environment/Pres tige”; variable I19 (university athletic programs) on the factor 2 “Influential”; and variables I9 (value of a degree from this university) was allowed to load on factor 5 “Quality/ Reputation”. However, these changes should be carefully c onsidered as tentative, if they should be theoretically justified. One of the suggestions of the Lagrange Mu ltiplier test is the assignation of the indicator variable I1 (acade mic reputation of the university ) to factor 4 “Environment/ Prestige”, in order to estimate the reduc tion in model chi-square. This suggestion

PAGE 130

120 indicates the assigning of one indicator variab le to two factor simu ltaneously (I1 to F5, and I1 to F4); in this particular case, is pr eferable to drop this indicator variable rather than assign it to factor 5 a nd factor 4 at the same time, since in the confirmatory measurement model, all of the indicator vari ables are unifactorial (each indicator loads on only one factor), moreover, the variable I1 showed a large and statistically significant loading ( = .62, t = 18.40) for the factor 5 “Qual ity/Reputation”. Therefore, the best alternative is to drop this variable from the analysis. Taking account the considerations early, the results related to the five-first-order modified model, in students ’ decisions of university choi ce, showed a chi-square of 900.4, with 242 degrees of freedom, this chi-squa re value is still statistically significant (p < .0001). This value represen ts a decreased in chi-square (9.1 %), as compared with the initial measurement model, where chi-square was 990.86, with 265 degrees of freedom. The alternative indices for assessi ng the overall goodness of fit revealed values larger than the observed in the initial model (G oodness of Fit Index (GFI) =. 93, Adjusted Goodness of Fit Index (AGFI) = .91, Bentle r’s Comparative Fit Index (BCFI) = .87, Bentler and Bonett’s Normed Fit Index (BBNFI) = .83, Bentler and Bonett’s Nonnormed Fit Index (BBNNFI) = 0.85, Bollen Norm ed Fit Index (BNFI) = .81, and Bollen Non-normed Fit Index (BNNFI) = 0.87, therefore, these outc omes suggest that these changes are justified, since they provid e an improvement in model fit. Research Question # 3: How well does the hypothesized measurem ent model involving four-first order factors fit the observed data ba sed on students’ perceptions ab out professor effectiveness?

PAGE 131

121 In order to answer the research question th ree, a confirmatory factor analysis was performed to evaluate the hypothesized m easurement model underlying the students’ perceptions about professor effectiveness. Specification, Identification, and Estimati on of the Four-First-Order Factor Model. As was established earlier, the measur ement model proposed to measure the professor effectiveness consists of twenty-one measured variables and four factors, which are assumed as: Interested/Student Centered (it has six observed variables), Content/ Pedagogical Knowledge (eight observed vari ables), Behavior/Receptive (four observed variables), and Facilita tion of learning (three observed va riables). Notice that each of the measured variables for each factor was pr edicted to load on only the factor it was proposed to measure; the four factors were al l hypothesized to correla te with one another; there are no covariances between any of the measured variables and the standard errors were not hypothesized to be correlated. Noti ce also that a residual term was created by measured variable; and the factor variances were set to one in order to assume identification of confirma tory factor model. Additionally, since the data points [21(21 + 1) / 2 = 231] are greater than the number of parameters to be estimated (twe nty one factor loading, plus the six factor correlations, plus the twenty one measur ement error variances, for a total of 48 parameters), the four-factor model is identified and it can be testable statistically. Besides, this confirmatory fact or model also should be identi fied if it has at least three items for each factor, which is satisfied in this case. Factor loadings, t-values, a nd standard errors for the f our-first-order factor model related to students’ percepti ons about professor effectiven ess are summarized in Table

PAGE 132

122 13. These results show that the obtained t-va lues for all factor loading coefficients related to the four-first order factor model demonstrated to be statistically significant at p < .001, and all factor loading exceeded .45 in absolute magnitude, indicating that they were meaningful coefficients. As a result, one can conclude that a ll factor loading were statistically significant. The fourth column in the Table contains the standard error for loading of each measured variable on its in tended factor. The erro r values range from 0.027 to 0.033, which showed no problematic values (such as 0.0003) for acceptable errors, thus these results presented reasona ble values for all measured variables. In the four-first-order model related to professor effectiven ess, all four factors were hypothesized to be correlated, consequently the covariances were estimated for the six pair of factors: CF1F2 = 0.83, CF1F3 = 0.77, CF1F4 = 0.63, CF2F3 = 0.70, CF2F4 = Table 13. Factor Loading, t-Values, Standard Error and Error Variance Estimates in Students’ Perceptions of Professor Effectiveness. ____________________________________________________________ Item-Factor Loading ( ) t-Value Stand. Error Error Variance ____________________________________________________________ Factor 1:Interested/Student Centered Item11 0.61 19.81 0.030 .64 Item12 0.72 25.04 0.029 .48 Item13 0.72 25.15 0.029 .48 Item18 0.67 22.60 0.030 .55 Item19 0.70 23.49 0.029 .53 Item20 0.66 22.48 0.030 .56 Factor 2: Content/Pedagogical Knowledge Item01 0.70 24.42 0.029 .51 Item02 0.66 22.78 0.029 .56 Item03 0.61 20.25 0.030 .63 Item14 0.76 27.28 0.028 .43 Item15 0.50 16.03 0.031 .75 Item16 0.55 18.10 0.030 .70 Item17 0.77 27.83 0.028 .41 Item21 0.83 31.04 0.027 .32

PAGE 133

123 Factor 3: Behavior/Receptive Item07 0.68 21.67 0.030 .54 Item08 0.60 18.77 0.032 .64 Item09 0.73 23.61 0.031 .48 Item10 0.46 13.83 0.033 .79 Factor 4: Facilitation of Learning Item04 0.62 19.06 0.033 .61 Item05 0.72 22.40 0.032 .49 Item06 0.63 19.39 0.032 .60 Notes: n = 1000 for all items. Item description is presented Table 5. 0.74, and CF3F4 = 0.69. We can observe that th ese estimated covariances demonstrated reasonable values, and all they were statisti cally significant at p < .001; and the standard deviation of these estimat ed range from .02 to .03. Figure 5 contains a diagram of the estimates data for the four-first-order factor model related to students’ perceptions about professor effectiveness. These data were described in the section related to specification, identi fication, and estimation of the four-first-order factor model. Assessment of Fit in the Four -First-Order Factor Model. Assessment of model fit involves conducting hypothesis te sts to assess the statisti cal significance of individual parameters and overall fit of the model to th e data set. The procedures for determining whether the four-first-order f actor model fits the data were the following: examination of the values of individual parameter estimates and their standard errors to test the statistical

PAGE 134

124 .61 .64 .72 .48 .72 .48 .67 .55 .70 .53 .66 .56 .83 .70 .51 .66 .56 .77 .61 .63 .76 .43 .50 .75 .55 .70 .63 .70 .77 .41 .83 .32 .74 .68 .54 .60 .64 .73 .48 .46 .79 .69 .62 .61 .72 .49 .63 .60 F1 Interested/Student Centered X11 X13 X12 X18 X19 F2 Content/ Pedagog. Knowledge X20 X1 X2 X5 X4 X9 X3 F3 Behavior/Receptive F4 Facilitation of Learning X15 X8 X14 X10 X16 X7 X21 X17 X6 Figure 5. Estimates Data for Fou r -First-Order Factor Model Related to Students’ Perceptions of Professor Effectivenes s.

PAGE 135

125 significance; evaluation of the ove rall fit of the model, such as: evaluation of the overall chi-square in terms of statistical significan ce; evaluation of the alternative indices of goodness of fit: Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Bentler’s Comparative Fit Index (BCFI), Bentler and Bonett’s Normed Fit Index (BBNFI), Bentler and Bonett’s Non-norme d Fit Index (BBNNFI), Bollen Normed Fit Index (BNFI), and Bollen Non-normed Fit Index (BNNFI); and examination of the normalized residuals in order to determine the similarity between the elements of the original and predicted matrices; and finally an examination of the modification model to determine which specific modification might best improve the fit if the a priori model is inadequate. Individual parameter values were analyzed to test statistical significance. The tvalues obtained demonstrated to be statistically significant at p < .001. From these results, one can conclude that all the fact or loadings were large and statistically significant. A summary of the data used to assessme nt the overall goodness of fit of the fourfirst-order factor model related to students’ perceptions about profe ssor effectiveness are presented in Table 14. In this case, estima tion of the model also revealed a significant chi-square. The chi-square value of 1080.4 with 183 degree of freedom is significant with a probability of .0001, indi cating that the mode l does not provide an adequately fit to the data. This significant value also wa s expected for the same reason that, the chisquare value is in part due to the large sample size used, ra ther than to a specification of the model.

PAGE 136

126 All the alternative fit indices used to assessment the overall goodness of fit revealed a relatively good fit even when the ( 2) test suggests rejecti on of the model. As indicated in this Table, all the values of th ese indices are at or close to the acceptable criterion of .90, used by many researchers as an indication of a good fit to the data, indicating that these indices have an acceptabl e fit of the four-first-order factor model related to students’ perceptions a bout professor effectiveness. Another measure of overall fit is the examination of the normalized residual matrix. Thus, considering that the norm alized residuals over 2.00 are generally considered large and therefore problematic (Hatcher, 1998), the av erage standardized residual showed a moderate abso lute value of 1.77; however, some of the elements of this matrix revealed absolutes values that exceed 2.00, which indicates that there are some problems with the theoretical model formulate d. These residuals showed that the fourfirst-order model underpredicted the strength of the relationship between the following pairs of variables: I12 and I11, I2 and I1, I16 and I18, and I21 and I20, since the predicted covariance was much smal ler than the actual covariance. Consequently, given that the model had statistically significant chi-square, and demonstrates significant problems with some of the standardized residuals, it was Table 14. Goodness of Fit Indices for the M odel in Professo r Effectiveness ______________________________________________________ Indices Value Chi-square ( 2) 1080.40 Degree of freedom (df) 183 p -value > chi-square < .0001 Goodness of Fit Index (GFI) 0.902 Adjusted Goodness of Fit Index (AGFI) 0.880

PAGE 137

127 Bentler’s Comparative Fit Index (BCFI) 0.900 Bentler and Bonett’s Normed F it Index (BBNFI) 0.880 Bentler and Bonett’s Non-normed Fit Index (BBNNFI) 0.882 Bollen Normed Fit Index (BNFI) 0.860 Bollen Non-normed Fit Index (BNNFI) 0.900 considered important to examine the m odification of the model with the propose formulating an a posteriori model that would fit the data more adequately. This is carried out using indices that may be useful in suggesting possible model modifications, such as the Wald and Lagrange Multiplier tests, whic h estimate the decrease in the chi-square value that would result if a given parameter were to be added/dropped to the model. The Wald test suggests that there are not parameters that could be dropped from the model in order to decrease the chi-square va lue, since that the ttests for all of the factor loadings were statistically significant, indicating that all measured variables showed doing a excellent job of measuring th e factors to which they were assigned. On the other hand, the Lagrange Multiplier test suggests some changes in order to reduce the chi-square if a new factor loading or covariance were added to the model. From the 10 largest Lagrange Mu ltipliers we can observe that the greatest decrease in the overall chi-square value would occur if vari ables I3 (interpretation abstract ideas and theories clearly) was allowed to load on factor four “Facilita tion of Learning”, and variables I1 (preparation for class) and I2 (breadth of know ledge of subject matter) were allowed to load on factor one “Interested/Student Centered”. We can also observe that these results are very consistent with the patt ern of large residuals an alyzed earlier, which showed that this model underpredicts some of these relationships. However, these

PAGE 138

128 changes should be carefully considered as tentative, if they should be theoretically justified. One of the suggestions of the Lagrange Mu ltiplier test is the assignation of the indicator variable I3 (interpret ation abstract ideas and theori es clearly) to factor four “Facilitation of Learning”, in order to estimate the larger reduction in model chi-square. As established early, is preferable to drop this indicator variable rather than assign it to factor four, since in the conf irmatory measurement model, all of the indicator variables are unifactorial (each indicator loads on only one factor), moreover, the variable Item3 showed a large and statistically significant loading ( = .61, t = 19.81) for the factor one “Interested/Student Centered”. Then, the best alternative is to drop this variable from the analysis. The results related to the four-first-order modified model, in students’ perceptions about professor effectiveness, showed a chi-square of 927.7, with 164 degrees of freedom, this chi-square value is still stat istically significant (p < .0001). This value represents a moderate decrease in chi-square (14.1 %), as compared with the initial measurement model, where chi-square wa s 1080.4, with 183 degrees of freedom. The alternative indices for assessing th e overall goodness of fit revealed values larger than those observed in the initial model (Goodness of Fit Index (GFI) =. 91, Adjusted Goodness of Fit Index (AGFI) = .89, Bentler’s Comparative Fit Index (BCFI) = .905, Bentler and Bonett’s Normed Fit Inde x (BBNFI) = .89, Bentler and Bonett’s Nonnormed Fit Index (BBNNFI) = 0.89, Bollen Norm ed Fit Index (BNFI) = .87, and Bollen Non-normed Fit Index (BNNFI) = 0.905.

PAGE 139

129 The average standardized residual shows a smaller value (1.68) as compared with the initial model (1.77), moreover, all the f actor loading are large and statistically significant at p < .001. Therefore, these resu lts suggest that these changes are justified, since they provide an improvement in model fit. Research Question # 4: How well does the hypothesized measurem ent model involving three-first order factors fit the observed data based on students’ perceptions about university academic reputation? In order to answer the research question four, a confirmatory factor analysis was carried out to assess the hypothesized m easurement model underlying the students’ perceptions about university’s academic reputation. Specification, Identification, and Estimation of the three-first-order factor model. As was recognized earlier, the measurement model proposed to measure the university’s academic reputation consists of fourteen meas ured variables and three factors, which are assumed as: Technology/Socio-cultural, (it has six observed variables), Research Quality (four observed variables), and Prestige/Quality (four observed variables). Observe that each of the measured variables for each factor was expected to load on only the factor it was proposed to measure; the three factors were all hypothesized to correlate with one another; there are no covariances between any of the measured variables and the standard errors were not hypothesized to be correlate d. Notice also that a residual term was created by measured variable; a nd the factor variances were se t to one in order to assume identification of confirma tory factor model.

PAGE 140

130 Additionally, since the data points [14(14 + 1) / 2 = 105] are greater than the number of parameters to be estimated (four teen factor loading, plus the three factor correlations, plus the fourteen measurement e rror variances, for a total of 31 parameters), the three-factor model is identified and it can be solved and in fact tested statistically. In addition, the confirmatory factor model also s hould be identified if it has at least three items for each factor, which is satisfied in this case. A summary of factor loadi ngs, t-values, standard erro rs, and error variances are presented in Table 15. These results indicate that the obtained t-values for all factor loading coefficients demonstrated to be statistically significant at p < .001, and all factor loading exceeded .50 in absolute magnitude indicating that they were meaningful coefficients, which also means that the measur ed variables are measuring the factor it was proposed to measure. From these results, one can conclude that all the factor loadings were statistically significant. The fourth colu mn in the Table contains the standard error for loading of each measured variable on its in tended factor. The error values range from 0.024 to 0.032, which showed reasonable va lues for all measured variables. In the three-first-order model related to university academic reputation, all three factors were hypothesized to be correlated, c onsequently the covariances were estimated for the three pairs of factors: CF1F2 = 0.75, CF1F3 = 0.72, and CF2F3 = Table 15. Factor Loading, t-Values, Standard Error and Error Variance Estimates in Students’ Perceptions of Ac ademic Reputation Item-Factor Loading ( ) t-Value Stand. Error Error Variance Factor 1: Technology/Socio-Cultural Item06 0.73 25.41 0.03 .34 Item08 0.81 29.27 0.03 .39

PAGE 141

131 Item09 0.79 28.54 0.03 .60 Item10 0.73 25.30 0.03 .51 Item11 0.52 16.42 0.03 .56 Item12 0.30 8.82 0.03 .72 Factor 2: Research Quality Item03 0.55 17.63 0.03 .06 Item04 0.71 28.38 0.03 .05 Item05 0.71 28.38 0.03 .09 Item07 0.71 21.30 0.03 .32 Factor 3: Prestige/Quality Item01 0.10 2.26 0.04 .24 Item02 0.30 6.80 0.04 .27 Item13 0.52 12.40 0.04 .30 Item14 0.60 13.43 0.04 .19 Notes: n = 1000 for all items. Item description is presented Table 5. 0.69. We can observe that these estimated c ovariances demonstrated reasonable values, and all they were statistically significant at p < .001; and th e standard deviation of these estimated are .02. Figure 6 contains a diagram of the estimates data for the three-first-order factor model related to students’ perceptions about university academic reputation, and it does not provide new information with respect to the data. The data were described in the section related to specifica tion, identification, and estimatio n of the three-first-order factor model. As shown in Figure 6, the factors Technology/Socio-cultural, Research Quality, and Prestige/Quality are indicated wi thin the ovals and the fourteen indicator variables within the rectangles Straight lines pointing to each indicator variable with loading associated with the indi cator denotes the effects of the latent factors. The effect of measurement error is marked with a straight line to th e indicator and the covariance

PAGE 142

132 factors are denoted with the cu rved arrows connecti ng the ovals, which indicate that all three factors were hypothesi zed to be correlated. .73 .34 s .81 .39 .79 .60 .73 .51 .74 .52 .56 .30 .72 .55 .06 .71 .05 .71 .71 .09 .71 .32 .69 .10 .24 .30 .27 .52 .30 .60 .19 Figure 6. Estimates Data for Three-Fi rst Order Factor Model Related to Students’ Perceptions of Univ ersity Academic Reputation. Assessment of fit in the threefirst-order factor model. As established earlier, the criteria used to assess fit of the model we re examination of the values of individual F1 Technology/ Socio-Cultural X6 X95 X8 X10 X11 F2 Research Quality X12 X23 X24 X25 F3 Prestige/Quality X1 X11 X2 X14 X13

PAGE 143

133 parameter estimates and their standard errors to test the statistical significance; evaluation of the overall fit of the model, such as eval uation of the overall ch i-square in terms of statistical significance, evaluation of the alternative indices of goodness of fit (Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Bentler’s Comparative Fit Index (BCFI), Bentler and Bonett’s Normed Fit Index (BBNFI), Bentler and Bonett’s Non-normed Fit Index (BBNNFI), Bollen Normed Fit Index (BNFI), and Bollen Non-normed Fit Index (BNNFI)), and examination of the normalized residuals in order to determine the similarity between the elements of the original and predicted matrices; and finally an examination of the modifica tion model to determine which specific modification might best improve the fit if the a priori model is inadequate. Individual parameter values were analyzed to test statistical significance and all demonstrated to be statistically significant at p < .001, therefore, thes e results show that all the factor loading are large and statistically significant. The data used to assessment the overa ll goodness of fit of th e three-first-order factor model related to stude nts’ perceptions about univers ity’s academic reputation are presented in Table 16. The chi-square va lue of 882.21 with 74 degree of freedom is significant with a proba bility of .0001, indicating that the model does not provide an adequate fit to the data. This significant valu e, was also expected due to the chisquare value is in part due to the large sample size used in this study, rather than to a misspecification of the model, since the model was identified according to the criteria

PAGE 144

134 Table 16. Goodness of Fit Indices for the Model in Academic Reputation ______________________________________________________ Indices Value Chi-square ( 2) 882.21 Degree of freedom (df) 74 p -value > chi-square < .0001 Goodness of Fit Index (GFI) 0.890 Adjusted Goodness of Fit Index (AGFI) 0.840 Bentler’s Comparative Fit Index (BCFI) 0.902 Bentler and Bonett’s Normed F it Index (BBNFI) 0.894 Bentler and Bonett’s Non-normed Fit Index (BBNNFI) 0.880 Bollen Normed Fit Index (BNFI) 0.869 Bollen Non-normed Fit Index (BNNFI) 0.902 used to this purpose. As shown in Table 16, the alternative fit indices revealed a relatively good fit even when the ( 2) test suggests rejection of the model; all alternative fit indices are at or close to the acceptable criterion of .90, indicating that these indices have an acceptable fit of the three-first-order factor model related to students’ perceptions about university academic reputation, except the Adjusted Goodness of Fit Index (AGFI), which shown a value less than .85. Another measure of overall fit is the examination of the normalized residual matrix. The average standardized residua l showed a high absolute value of 2.90; indicating that several of the elements of this matrix rev ealed absolutes values that exceed 2.00, which indicate that there are some problems with the theoretical model formulated. Consequently, these residuals showed that the three-first-order model underpredicted the strength of the relationship between the fo llowing pairs of variables:

PAGE 145

135 I14 and I13, I12 and I13, I2 and I1, and I7 and I6, since the predicted covariance was much smaller than th e actual covariance. Consequently, given that the majority of overall fit indices showed values at or close to the acceptable range and the model ha d statistically significant chi-square, and demonstrated significant problem with some of the standardized residuals it was also considered important to examine the modi fication of the model with the propose of formulating a posteriori model th at would fit the data more ad equately. This is carry out using indices that may be useful in sugges ting possible model modification, such as the Wald and Lagrange Multipli er tests, which are modifi cation indices that identify parameters that should possibly be droppe d and added, respectively, and estimate the decrease in the chi-square value that woul d result if a given parameter were to be added/dropped to the model. The Wald test recommends that there ar e not parameters that could be dropped from the model in order to decrease the chi-squa re value, since that the ttests for all of the factor loadings were statistically signifi cant, indicating that all measured variables demonstrated an excellent job of measuring the factors to which they were assigned. On the other hand, the Lagrange Multiplier test suggests some changes in order to estimates the reduction in model chi-square. From the 10 largest Lagrange Multiplier indices we can observe that the largest va lue of this index is 103.7, which was for the variable I6 (quality of libr aries): factor F2 “Research Quality” relationship, indicating that the greatest decrease in the overall chisquare value would o ccur if the Item6 was allowed to load on factor two. These results ar e very consistent with the pattern of large residuals analyzed earlier, which showed th at this model underpre dicts some of these

PAGE 146

136 relationships. However, these changes should be carefully considered as tentative, if they should be theoretically justified. One of the suggestions of the Lagrange Mu ltiplier test is the assignation of the indicator variable I6 (quality of libraries) to factor 2 “Research Quality”, in order to estimate the reduction in model chi-square. As established in the earl y analyses, in this particular case, is also preferable to drop this indicator variable rath er than assign it to factor 2 and factor 1 at the same time, since in the confir matory measurement model, all of the indicator variables are unifactorial and this variable I6 showed a large and statistically significant loading ( = .73, t = 25.41) for the factor one “Research Quality”. Therefore, these results suggest that this item may need to be dropped or substantially modified. Realized the changes suggested, the resu lts related to the three-first-order modified model, in students ’ perception about university academic reputation, showed a moderate decreased in chi-square (18.1 %), when this chi-square ( 2 = 722.2, df = 62) is compared with the chi-square in initial measurement model ( 2 = 882.21, with 74 degrees of freedom). The alternative indices for assessing th e overall goodness of fit revealed values larger than the observed in the initia l model (Goodness of F it Index (GFI) =. 901, Adjusted Goodness of Fit Index (AGFI) = .86, Bentler’s Comparative Fit Index (BCFI) = .912, Bentler and Bonett’s Normed Fit Inde x (BBNFI) = .905, Bentler and Bonett’s Nonnormed Fit Index (BBNNFI) = .90, Bollen Norm ed Fit Index (BNFI) = .88, and Bollen Non-normed Fit Index (BNNFI) = .912.

PAGE 147

137 The average standardized residual shows a smaller value (2.79) as compared with the initial model (2.90), moreover, all the f actor loading are large and statistically significant al p < .001. Therefore, these outcomes suggest that these changes are justified, since they provide an improve ment in overall goodness of fit. Multivariate Analysis of Variance Multivariate analysis of variance is a tec hnique that has the advantage of testing whether there are any differences between the groups on multiple criteri on variables, with a single probability associated with the test. However, there are three assumptions (the same assumptions for ANOVA) that have to be met when conducting a multivariate analysis of variance. These assumptions are a) independence of the observations; b) multivariate normality on the dependent variable s in each population; and c) equality of the population covariance matrices (referred to as the homogeneity of the covariance matrices). As established early, although th e previous assumptions are requirements for MANOVA, it is unlikely that all of the assumptions will be met exactly, therefore, violation of some assumptions do not necessar ily invalidate the resu lts. MANOVA is not robust to violation of independence of the obs ervations, but may be r obust to violations of multivariate normality on the dependent variables and homogeneity of the covariance matrices (Stevens, 1996). Independence of the observations requires that the dependent measures for each respondent be totally uncorre lated with the responses from other respondent in the sample. In this study, the obs ervations are independent, sinc e the survey instrument was individually administrated in the classrooms, and only one time

PAGE 148

138 Multivariate normality of the dependent variables assumption is a much more rigorous than it in ANOVA, therefore, norma lity on the univariate variables does not guarantee multivariate normality. For check ing multivariate normality were used the skewness (B1P) and kurtosis (B2P) coefficien ts. The skewness coefficient determines whether the matrix is symmetric or asymme tric, and B1P indicates the average cubed element in this matrix. The skewness and kurtosis coefficients by domains and campus are presented in Table 17. As indicated in this table, these coefficients show a positive skewness (extending toward posit ive values) indicating that the matrix is asymmetric. The skewness coefficients for the ULA campus indicate that these va riable distributions have a slight departure from normality, wh ile the skewness coefficients for NURR campus and NUTULA campus show a larger departure from normality, except the NURR campus in university academic reputati on that also demonstrated a slight departure from normality. On the other hand, kurtosis is a measure of whether the data are peaked or flat relative to a normal distribution. The kurtosi s coefficients revealed positive kurtosis, which means that it is relatively peaked (Lepto kurtosis) rather than flat (Platykurtosis). Although deviation from multiv ariate normality has only a small effect on type I error (it is the probability of incorrectly rejecting a tr ue null hypothesis), nonnormality (Platykurtosis) may reduce the relative statistic al power (it is the pr obability of rejecting the null hypothesis when it is false) of the MANOVA test statistic. Homogeneity of covariance matrices is a very restrictive assumption (given that two matrices are equal only if all correspondi ng elements are equal). The Box M test was used for determining whether the covariances ma trices are equal. This test is very

PAGE 149

139 sensitive to nonnormality, for example one may re ject the null hypothesis with this test and conclude that the covariance matrices ar e different when actuality the rejection may Table 17. Multivariate Skewness and Kurt osis by Campus and Domains ___________________________________________ Campus Skewness Kurtosis University Choice Process Campus 1: NURR 1.93 32.77 Campus 2: NUTULA 2.10 31.05 Campus 3: ULA 1.00 34.54 Professor Effectiveness Campus 1: NURR 1.94 28.09 Campus 2: NUTULA 2.39 29.36 Campus 3: ULA 0.39 29.08 University Academic Reputation Campus 1: NURR 0.83 16.16 Campus 2: NUTULA 3.78 19.32 Campus 3: ULA 0.57 16.56 Note: n=132 (NURR), n=114 (NUTULA), and n=754 (ULA) have been due to nonnormality in the underlying populations. The results of this analysis reveal ed significant Box tests across the domains university choice process, prof essor effectiveness and unive rsity academic reputation, since the chi-square tests ( 2 = 44.96 with 30 degree of freedom; 2 = 51.67 with 20 degree of freedom; and 2 = 75.35 with 10 degree of freedom, respectively) are significant at p-values of .039, .0001, and .0001, resp ectively. These resu lts indicate that

PAGE 150

140 the covariance matrices are unequal. Howe ver, from the within covariance matrix information, we can observe that the MANO VA test by domains would be considered conservative, given that the larger generalized variances were with the larger group sizes. Research Question # 5. What are the differences across gender in perceived importance of the selected factors that influence the students’ decisions about university choi ce process, and their perceptions of professor e ffectiveness and university a cademic reputation at the University of Los Andes? In order to answer research question fi ve, Multivariate Analysis of Variance (MANOVA) using gender as the predictor vari able, and the mean of the student ratings by factor as the criterion variab les were performed. Results of this analysis revealed a nonsignificant multivariate gender effect. These analyses were conducted to test simultaneously differences between the gende r groups on multiple factors as dependent variables; in consequence, the overall test of significance in M ANOVA addresses the null hypothesis that the means vectors by gender ar e equal on the criteri on variables in the population. Summaries of the multivariate analyses of variances of university choice process, professor effectiveness, and university acad emic reputation by gender and university campuses are presented in Table 18. As s hown in this table, the results of the multivariate analysis of variance related to students’ decisions in university choice process produced a Wilks’ Lambda statistic va lue of .998 (large value, close to 1), this value indicates a relatively weak relationship between the multiple factors and gender

PAGE 151

141 taken as a group. However, the multivariate F statistic (based on Wilks’ Lambda) that tests the significance of this relationship is approximately 0.39 with 5 and 994 degree of freedom and a p-value of .8484, revealed a nonsignificant multivariate gender effect. Table 18. Multivariate Analyses of Vari ance of University Choice Proce ss, Professor Effectiveness, and University Academic Reputation Across gender and campus _______________________________________________________________________ Domain Statistic Value F Ratio DF F Prob. (Gender) University Choice Wilks’ Lambda 0.998 0.39 5; 994 .8584 Professor Effectiveness Wilk s’ Lambda 0.992 1.94 4; 995 .1011 Academic Reputation Wilks’ Lambda 0.995 1.61 3; 996 .1856 (University Campus) University Choice Wilks’ Lambda 0.920 8.44 10; 1986 < .0001 Professor Effectiveness Wilks’ Lambda 0.890 14.59 8; 1988 < .0001 Academic Reputation Wilks’ Lambda 0.810 37.25 6; 1990 < .0001 Note: Wilks’Lambda is a multivariate measure of association, used when there are multiple criterion variables, which range from 0 to 1, va lues near zero indicate a strong relationship and near to one weak relationship; DF is degree of freedom. Similarly, the multivariate analyses of variance related to students’ perceptions of professor effectiveness and uni versity academic reputation, yi eld a large Wilks’ Lambda of .992 and .995, respectively, indicating a re latively weak relationship between the predictor variable (gender) and the criter ion variables (factors). Likewise, the multivariate F statistic of 1.99 (with 4 and 995 degree of freedom and a p-value of .1011) and 1.61 (with 3 and 996 degree of freedom a nd a p-value of .1856), respectively, shown

PAGE 152

142 a nonsignificant multivariate gender effect. In other words, there is no difference between the male and female students when they are compared simultaneously on the factors that influence their de cisions about university choice process and their perceptions about professor effectiveness and university academic reputation. The hypothesis test results can be corroborat ed with the data presented in Table 19, which shows the factor means and standard deviations across domain by gender. As indicated in this table, the factor mean s by gender show an insignificant difference; moreover, we can observe that the highest f actor mean showed less variability than the lowest factor mean, as demonstrated for the respective standard deviation; for example, the male mean for the factor 5 “Quality/Reputation” is 4.22 (highest) with a standard deviation of 0.43, and the male m ean for the factor 2 “Influentia l” is 2.69 (lowest) with a standard deviation of 1.14. Research Question # 6 What are the differences across univers ity campuses in perceived importance of the selected factors that influence the student s’ decisions about unive rsity choice process, and their perceptions of professor effectiven ess and university academic reputation at the University of Los Andes? In order to answer the research question six, Multivariate Analysis of Variance (MANOVA) using university campuses as the pr edictor variable a nd the mean of the student ratings by factor as the criterion variables were performed. Results of this analysis revealed a significant multivariate university campus effect. These analyses

PAGE 153

143 were conducted to test simultaneously differences between the campus groups on multiples factors; in consequence, the overall test of significance in MANOVA Table 19. Factor Means and Standard Deviations across Domain by Gender ____________________________________________________________ Female: n = 408 Male: n = 592 Factor Description Mean S.D Mean S.D. University Choice Process F1: Facility/Support 3.36 0.75 3.39 0.84 F2: Influential 2.70 1.16 2.69 1.14 F3: Academic Resources 3.51 0.61 3.54 0.49 F4: Environment/Prestige 3.70 0.49 3.70 0.41 F5: Quality/Reputation 4.20 0.43 4.22 0.43 Professor Effectiveness F1: Interested/Student Cent. 3.36 0.62 3.27 0.63 F2: Content/Pedag. Knowledge 3.70 0.49 3 .65 0.53 F3: Behavior/receptive 3.31 0.65 3.29 0.62 F4: Facilitation of Learning 3.31 0.74 3.31 0.72 University Academic Reputation F1: Technology/Socio-Cult. 3.28 0.62 3.20 0.66 F2: Research Quality 3.95 0.66 3.90 0.69 F3: Prestige Quality 4.10 0.50 4.04 0.54 addresses the null hypothesis that the means vectors by uni versity campuses are equal on the criterion variable s in the population.

PAGE 154

144 As indicated in Table 18, the results of the multivariate analyses of variance related to students’ decisions in university choice process and students’ perceptions of professor effectiveness and university acad emic reputation, produced Wilks’ Lambda values closer to 1 (.92, .89, and .81, respectivel y) indicating a relativ ely weak relationship between the multiple factors and the university campus taken as a group. The multivariate F statistics by domain (based on Wilks’ Lambda) that tests the significance of this relationship yielded values of 8.44, 14.59, an 37.25, respectively, all them with p-values less than .0001, indicating that the null hypothese s are rejected, it means that the university campus is significantly different with respect to at least one of the factors that influenci ng university choice process, professor effectiveness and university academic reputation. Consequent ly, since the multivariate F statistics by domains were statistically significant, it is n ecessary to interpret the univariate analyses of variance (ANOVAs) and then, the results of the Tukey multiple comparison tests, in order to determine which pairs of means ar e significantly different from one another. Table 20 shown a summary of univariat e analyses of variances using the factors that influencing the students’ decisions in univers ity choice process and their perception about professor effectiveness, and university academic reputation as the dependent variables and university campus as the independent variable As indicated in this Table, the results of the univariate anal yses of variance (uni versity choice process) using the factor1: facility/support, fact or3: academic resources, and factor4; environment/prestige as the criterion vari ables, produced F ratios of 27.43 (p < .0001), 3.39 (p < .0341), and 9.32 (p < .0001), respecti vely; these results revealed a significant

PAGE 155

145 university campus effect, which indicate that there are significan t differences among the students by university campus when they are compared on those factors that influence their decisions about university choice pr ocess. The univariate ANOVAs using the factor2 (influential) and factor5 (quality/reputation) as the dependent variables revealed Table 20. Analysis of Variance of University Choi ce Process, Professor Effectiveness, and University Academic Reputation by Campus _______________________________________________________________ Factor Description DF SS MS F Ratio F Prob. (University Choice Process) F1: Facility/Support 2 33.76 16.87 27.43 < .0001 F2: Influential 2 3.63 1.82 1.93 .2502 F3: Academic Resources 2 2.40 1.20 3.39 .0341 F4: Environment/Prestige 2 4.43 2.22 9.32 < .0001 F5: Quality/Reputation 2 0.30 1.59 27.43 .2041 (Professor Effectiveness) F1: Interested/Student Cent. 2 21.46 10.73 28.79 < .0001 F2: Content/Pedag. Knowled. 2 18.67 9.34 38.23 < .0001 F3: Behavior/receptive 2 8.47 4.24 10.86 < .0001 F4: Facilitation of Learning 2 27.93 13.97 28.11 < .0001 (University Academic Reputation) F1: Technology/Socio-Cult. 2 41.70 20.85 54.88 < .0001 F2: Research Quality 2 72.34 36.17 92.67 < .0001

PAGE 156

146 F3: Prestige Quality 2 6.52 3.26 12.09 < .0001 Note: DF is degrees of freedom; SS is Sum of Square; MS is Mean Square. a nonsignificant university campus effect. In the same way, this analysis is corrobor ating with the statistics showed in Table 21. We can observe in this Table that some factors mean across domain by campus demonstrated significant differences. In addition, the highest factor means Table 21. Factor Mean and Standard Deviation across Domain by Campus _______________________________________________________________________ Campus1: n = 132 Campus2: n = 114 Campus3: n = 754 Factor Description Mean S.D Mean S.D. Mean S.D. University Choice Process F1: Facility/Support 3.73 0.76 2.99 0.87 3.38 0.77 F2: Influential 2.81 1.29 2.56 1.06 2.70 1.13 F3: Academic Resources 3.53 0.59 3.39 0.72 3.55 0.57 F4: Environment/Prestige 3.58 0.55 3.59 0.50 3.74 0.47 F5: Quality/Reputation 4.27 0.43 4.18 0.44 4.20 0.43 Professor Effectiveness F1: Interested/Student Cent. 2.97 0.63 3.19 0.64 3.39 0.60 F2: Content/Pedag. Knowledge 3.35 0.51 3.57 0.57 3.74 0.48 F3: Behavior/receptive 3.14 0.61 3.13 0.64 3.35 0.63 F4: Facilitation of Learning 2.88 0.72 3.34 0.77 3.38 0.69 University Academic Reputation F1: Technology/Socio-Cult. 2.72 0.70 3.16 0.69 3.33 0.59 F2: Research Quality 3.27 0.72 3.76 0.73 4.05 0.59 F3: Prestige Quality 3.89 0.59 3.95 0.58 4.10 0.49

PAGE 157

147 Note: Campus1 is The “Rafael Rangel” University Campus – Trujillo (NURR) Campus2 is The Tachira University Campus – Tachira (NUTULA) Campus3 is University of Los Andes – Merida, main campus (ULA) showed less variability than the lowest fact or mean, as demonstrated for the standard deviations by factor; for example the factor 5 related to university choice showed a mean = 4.27 with S.D. = 0.43 (campus1), and the factor 2, in the same domain, revealed a mean = 2.56 with S.D = 1.06 (campus2). Consequently, once that F statistic has id entified there is a significant overall difference, the Tukey multiple comparison test was used to evaluate the factor 1, 3, and 4 in university choice process, in order to ex amine all group comparisons and to determine the specific differences among the campuses. A summary of the Tukey multiple compar ison tests for factors by domains is presented in Table 22. As indica ted in this table, the Tukey te st used in university choice process to determine university campus diffe rences, at the 0.05 level of significance, revealed the following significant differences : about factor1: facility/ support as dependent variable, we can observe that the main difference was established by the university campus 1 (NURRTrujillo) wh en it compared wi th campus 2 (NUTULATachira), which indicates that the students ’ decisions at this campus gave more importance to the factor 1 than the other tw o campuses, moreover, it also indicates that the campus 2 gave less weight to this fact or, like influencing their decisions about selecting the ULA. Relating to factor 3: ac ademic resources, we can distinguish only one significant difference among the campuses, be tween campus 2 (NUTULA-Tachira) and campus 3 (ULA-Merida), where the campus 3 co nfers more importance to this factor in

PAGE 158

148 the selection of this university. The other f actor that established a significant difference among the campus was factor 4: environment/p restige, we can observe two significant differences, between campus 1 and campus 3, and between campus 2 and campus 3, these results reveal that the higher difference wa s established by the campus 3 and the smaller by the campus 1, it indicated that the stude nts at the campus 3 conferred a major importance to this factor (4) as influencing their decision about sel ecting this university. As shown in Table 20, the univariate ANOVAs related to professor effectiveness and university academic reputation usi ng factor as dependent variables Table 22. Tukey Multiple Comparison Test for Factors by Domains _______________________________________________________________________ Factor Description Mean Differences by Campus (Significant) University Choice Process F1: Facility/Support 1 – 2 = 0.74 1 – 3 = 0.36 2 – 3 = -0.39 F2: Influential There is not significant differences F3: Academic Resources 2 – 3 = -0.16 F4: Environment/Prestige 1 – 3 = -0.16 2 – 3 = -0.15 F5: Quality/Reputation There is not significant differences Professor Effectiveness F1: Interested/Student Cent. 1 – 2 = -0.22 1 – 3 = -0.42 2 – 3 = -0.20 F2: Content/Pedag. Knowledge 1 – 2 = 0.22 1 – 3 = -0.39 2 – 3 = -0.17 F3: Behavior/receptive 1 – 3 = -0.21 2 – 3 = -0.22 F4: Facilitation of Learning 1 – 2 = -0.45 1 – 3 = -0.50 University Academic Reputation F1: Technology/Socio-Cult. 1 – 2 = -0.44 1 – 3 = -0.61 2 – 3 = -0.17 F2: Research Quality 1 – 2 = -0.50 1 – 3 = -0.79 2 – 3 = -0.29 F3: Prestige Quality 1 – 3 = -0.21 2 – 3 = -0.16

PAGE 159

149 Note: 1 = Campus1 is The “Rafael Rangel” University Campus – Trujillo (NURR) 2 = Campus2 is The Tach ira University Campus – Tachira (NUTULA) 3 = Campus3 is University of Los Andes – Merida, main campus (ULA) (factor1: interested/student centered, factor2: content/pe dagogical knowledge, factor3: behavior/receptive, factor4: facilitation of learning, and factor 1: technology/sociocultural, factor2: research qua lity, factor3: prestige/qual ity, respectively), revealed a significant university campus effect, indica ting that there are si gnificant differences among the students by campus when they are co mpared on the factors that influence their perceptions about professor effectiven ess and university academic reputation, respectively. The Tukey multiple comparison test used in professor effectiveness and university academic reputation to determine university campus differences, shown approximately similar results at the 0.05 leve l of significance. However, the university campus 3 (ULAMerida) established the higher mean scores among the students when they are compared on the factors than influencing their per ceptions about professor effectiveness and university academic reputation, following the campus 2 (NUTULA-Tachira) and the campus 1 (NURR-Trujillo). It also indicates that the campus 1 (NURR-Trujillo) showed the smaller mean scores. In those analyses, it is important to point out that alt hough these results are superficially similar, they are separate and statistically distinguishable. Basically, these findings should be explai ned by the heterogeneity respect to the origin of the ULA-Merida university campus st udents (they come from all the regions of the country), given that the mass of the st udents in the other university campuses

PAGE 160

150 (NUTULA-Tachira and NURR-Trujillo) come from their respective states. At this point, is important to indicate that Venezuela’s Los Andes are located in the central-west portion of the country and incl ude the states of Merida, Tachira, and Trujillo. The campuses in this region will be mentioned in the order in which they have previously been presented: a) Campus 1 “NU RR-Trujillo” located in the city of TrujilloTrujillo (the smallest population of the Andes region), Trujillo state has fewer higher education institutions of minor prestige; most of the students in this state attend the ULA university branch; b) campus 2 “NUTULA-Tachira” located in the city of San CristobalTachira (the most populated state of the thre e that forms the Andes region); and c) ULAMerida main campus located in Merida-M erida, this campus be composed of approximately eighty percent of the teaching st aff and the seventy percent of the students. Summary The instrument designed to measure st udents’ decisions to enroll at the University of Los Andes and their percep tions about professo r effectiveness and university academic reputation has adequate in ternal consistency reliability estimates (all the domains exceeded .70), however, although some of the internal c onsistency reliability estimates by factor across the domains are more than adequate, the reliability for factor 3 and 4 in university choice process are not acceptable (at least .41). Regarding the confirmatory factor analysis the overall fit indices revealed values at or close to the acceptable range .90, even when the model has statistically significant chi-square, which indicates that the fit of the model could possibly be significantly improved. So, considering the model modifica tion indices we can observe a relatively

PAGE 161

151 small improvement in the overall goodness of fit. These resu lts provide supportive evidence of construct validity. Finally, the multivariate analyses of va riance using gender and university campus as the predictor variables revealed a nons ignificant gender effect and a significant university campus effect, respectively. Th e Tukey multiple comparison test used to determine university campus differences show n approximately similar results, although they are separate and distinguishable. It is important to point out that the university campus 3: ULA-Merida established the highest mean scores when they are compared on the factors that influence their decisions in university choice process and their perceptions about professor effectiveness and univers ity academic reputation, and the campus 1 (NURR-Trujillo) showed the smaller mean scores.

PAGE 162

152 Chapter Five Conclusions and Recommendations The purpose of this research was to examine the construct validation of an instrument to measure student’s decisions to enroll at the University of Los Andes and their perceptions about professor effectiv eness and university academic reputation. Additionally, a comparative analysis was car ried out to determine how the university selection process and the perceptions of e ffectiveness and reputation differ according to student demographic factors. The present chapter presents the six form ulated research questions, a summary of the methods used in this study, the conclusi ons based on the results obtained and the recommendations associated with each research question formul ated in this research, and finally the recommendations for further research. Research Questions The following research questions we re formulated in this study: 1. Are the student’s decisions of university ch oice process, and student’s perceptions of professor effectiveness and university acad emic reputation reliable within their respective factors at the Un iversity of Los Andes? 2. How well does the hypothesized measurement model involving five-first order factors fit the observed data based on student’s deci sion to enroll at the University of Los Andes?

PAGE 163

153 3. How well does the hypothesized measurem ent model involving four-first order factors fit the observed data based on the student’s perceptions about professor effectiveness at the University of Los Andes? 4. How well does the hypothesized measuremen t model involving three-first order factors fit the observed data based on stude nt’s perceptions of university academic reputation at the University of Los Andes? 5. What are the differences across gender in perceived importance of the selected factors that influence the students’ decisi ons about university c hoice process, and their perceptions of professo r’s effectiveness and university’s academic reputation at the University of Los Andes? 6. What are the differences across univers ity campuses in perceived importance of the selected factors that influence the students ’ decisions about univer sity choice process, and their perceptions of professor’s effectiveness and university’s academic reputation at the University of Los Andes? Summary of Methods The survey instrument used in this st udy was a self-administered paper-and-pencil questionnaire that was admini stered to students who were registered in the second semester of 2002 at University of Los Andes. This survey was developed based on the selection of items from a universe in which the investigator is intended and on the main theoretical concepts derived from related literature about university choice process, professor effectiveness, and university academic reputation. Af ter subjecting it to content review by two expert professors from the De partment of Measurement and Research at

PAGE 164

154 University of South Florida, the instru ment was piloted on 223 students who were registered in the first seme ster of 2000 on one of the uni versity branch campus at University of Los Andes (NURR-Trujillo). Based on results from the pilot study, the instrument was revised from 28 to 25 items in students’ decisions to select the ULA and from 22 to 21 items and from 15 to 14 items in students’ perceptions about prof essor effectiveness and university academic reputation, respectively. Responses to the survey instrument in the pilot test were subjected to exploratory factor analysis with oblique ro tation, in order to determine th e number of factors to retain. This analysis revealed a five-factor solution related to students’ decisions to select the ULA, a four-factor solution associated to students’ perceptions about professor effectiveness, and a three-factor solution re lated to students’ perception on university academic reputation. Item means, standard deviations, and normality for the survey data were determined and the constructs (domains) we re analyzed for relia bility using Cronbach Alpha coefficient. The reliability estimate s for the three domains revealed adequate reliability, and the re liability estimates by factor across the domains also suggest acceptable reliability, except the estimates for the factors academic resources (3) and prestige (4) related to students’ decisions to select the ULA. To address the research questions two to four, three separate confirmatory factor analyses were performed to evaluate the hypothesized models underl ying the twenty-five items associated to university choice, th e twenty-one items related to professor effectiveness, and the fourteen items asso ciated to university academic reputation.

PAGE 165

155 Multivariate skewness and kurtosis coefficients were performed to evaluate multivariate normality and homogeneity of covariances matrices, respectively. The confirmatory factor analysis findings showed a significant chi-square at p < .0001. Overall goodness of fit indices were used for determining how well the models fit the data, these indices revealed a relativel y good fit even when the chi-square test suggests rejection of the model. These results demonstrated that the fit of the model to the data could be significant improved, cons idering the outputs of these indicators. Finally to address research questions five and six, multivariate analyses of variance were conducted to compare the mean s of the estimated factor scores across gender and university campus. These findi ngs showed a nonsignificant multivariate gender effect and a significant multivariate university campus effect. The Tukey multiple comparison test was used to identify th e specific differences among the campuses. Conclusions and Recommendations Based on the stated resear ch questions and data anal ysis presented, the following conclusions and recommendations may be made. However, it is necessary indicate that the instrument is still in a state of developm ent and that caution should be exercised when making policy recommendations based on th ese scores, pending further validation evidence. 1. Are the student’s decisions of university choice process, and student’s perceptions of professor effectiveness and university a cademic reputation reliable within their respective factors at the Un iversity of Los Andes? Cronbach alpha coefficient was used to de termine internal consistency reliability

PAGE 166

156 for the three domains considered in this study and for the factor across these domains. The findings indicate that the instrument based on the studen ts’ decisions to select the ULA and their perceptions on professor e ffectiveness and university academic reputation has adequate internal consistency reliability with valu es that exceeded .75 across all the domains considered. The internal consistency reliability by factor across the three domains also revealed adequately reliability except the estim ates for factor 3 and 4 (academic resources and prestige) related to stude nts’ decisions to select th e ULA, which demonstrated inadequate estimates (.41). Recommendations : The findings of this analysis sugge st that the item 11 “closeness to home” (factor 3) and the items 8 ‘length of time to degree” and item 10 “university’s geographic location” (factor 4) should be dropped from the instrument, in order to improve the internal consistency reliability of the scale related to university choice process. These changes should be performed, since they reveal ed, in the modified model, higher reliability values. 2. How well does the hypothesized measurem ent model involving five-first order factors fit the observed data based on student’s decision to enroll at the University of Los Andes? Confirmatory factor analysis was perform ed to evaluate the hypothesized model underlying the student’s decision to enroll at the University of Los Andes. The five first-order factor model consists of twenty-f ive measured variables, and each of these variables was allowed to load on only the factor it was proposed to measure. The factors

PAGE 167

157 were hypothesized to correlate with one anothe r and the factor varian ces were set to one in order to assume identificati on of confirmatory factor model. All analyses were carried out using the SAS system’s CALIS proce dure, which used the maximum likelihood method of parameter estimation in the model. All factor loadings demonstrated to be m eaningful coefficients, except the factor loadings associated with the items 8, 10 and 11, which showed nonsignificant coefficients. Some of the alternative i ndices (GFI, AGFI, BCFI, and BNNFI) revealed a relatively good fit even when the chi-square test suggests rejection of the model fit, however, the remainder altern ative indices (BBNFI, BBNNFI, and BNFI) indicate an inadequate fit. Further, the analysis of the normalized residuals also indicated that there are some problems with the model formulated. So, these results reveal that the five-firstorder model could possibly be significantly improved. The findings related to significance of th e parameter estimates and modification indices revealed some changes that should be carefully considered if they should be theoretically justified. The significance of th e parameter estimates, equal to reliability estimates, showed that the items 8, 10, and 11 should be dropped from the scale. The Wald test reveals that the f actor loading I8 should be dr opped from the model, and the Lagrange Multiplier test suggests that the great est decrease in the ove rall chi-square value would occur if variables I1 (academic reputa tion of the university) was allowed to load on factor 4 “Environment/Prestig e”; variable I19 (university athletic programs) on factor 2 “Influential”; and variables I9 (value of a degree from this university) was allowed to load on factor 5 “Quality/ Reputation”.

PAGE 168

158 Recommendation : The findings of the confirmatory factor analysis involving the fivefirst-order model related to st udent’s decisions to select the University of Los Andes suggest that this model can be modified fo r future research. Considering the changes suggested in the analysis, this study recomme nds that the only change that was justified was to drop from the scale the measured variables 8 (length of time to degree), 10 (university’s geographic locat ion), and 11 (closeness to home), since it improved the model’s fit. 3. How well does the hypothesized measurem ent model involving four-first order factors fit the observed data based on the student’s pe rceptions about professor’s effectiveness at the University of Los Andes? Confirmatory factor analysis was perform ed to evaluate the hypothesized model underlying the student’s percepti ons about professor effectiven ess at the University of Los Andes. The measurement model proposed to measure the prof essor effectiveness consists of twenty-one measured variables and four factors, each of them were also hypothesized under the same conditions esta blished earlier. A ll factor loading coefficients related to the four-first order f actor model demonstrated to be statistically significant at p < .001, indicating that th ey were meaningful coefficients. All the alternative fit indices used to assessment th e overall goodness of fit revealed a good fit to the data even when the estimation of the model revealed a significant chi-square. Moreover, examination of the normalized re sidual matrix showed also some problems with the theoretical model, therefore, it was considered as elements important to examine

PAGE 169

159 the modification of the model, in order to fo rmulate an a posteriori model that would fit the data more adequately. The Wald test suggests that there are not parameters that could be dropped from the model, since that all factor loadings we re statistically significant, and the Lagrange Multiplier test recommends some changes in order to estimate the reduction in model chisquare: that the measured variables variable I3 (interpretation abstra ct ideas and theories clearly) should be allowed to load on the f actor 4 (facilitatio n of learning), the measured variables I1 (preparation fo r class) and I2 (breadth of knowledge of subject matter) should be allowed to load on factor 1 (int erested/student centere d), and variable I11 (attentiveness to students’ needs and concer ns) should be allowed to load on factor 2 (content/pedagogical kn owledge). However, these changes should be carefully considered as tentative, if they should be theoretically justified. Recommendation : The findings of the confirmatory factor analysis involving the fourfirst-order model related to student’s per ception about professor effectiveness suggest that this model can be modified for future re search, in order to improve the model’s fit. However, considering the analysis of the modification indices, this study recommends that the only change that was justified was to drop from the scale the measured variable I3 (interpretation abstract ideas and theories clearly), because it m odification provided an improvement in model’s fit. 4. How well does the hypothesized measurem ent model involving three-first-order factors fit the observed data based on student’s perceptions of university academic reputation at the University of Los Andes?

PAGE 170

160 Confirmatory factor analysis was perform ed to evaluate the hypothesized model underlying the student’s per ceptions about university a cademic reputation. The measurement model proposed to measure the un iversity academic reputation consists of fourteen measured variables a nd three factors, each of them were also hypothesized under the same conditions established ear lier. All factor loading coefficients related to the fourfirst order factor model demonstrated to be statistically significant at p < .001, indicating that they were meaningful coefficients. Th e entire alternative fit indices, except the Adjusted Goodness of Fit Index (AGFI), (which shown a value less than .90), revealed a relatively good fit even when the ( 2) test suggests rejection of the model. The average standardized residual indicates that several of the elements of this matrix revealed large absolutes values, which indicat e that there are some problem s with the theoretical model formulated. Consequently, given that the majority of overall fit indices showed values at or close to the acceptable range and the model ha d statistically significant chi-square, and demonstrates significant problems with so me of the standardized residuals it was considered important to examine the modi fication of the model with the propose of formulate a posteriori model that would fit th e data more adequately. The Wald test recommends that there are not parameters that could be dropped from the model and the Lagrange Multiplier test suggests that the meas ured variable I6 ( quality of libraries) should be allowed to load on the factor two (research quality). Further, this change should be carefully considered as tentativ e, if it should be th eoretically justified. Recommendation : The findings of the confirmatory factor analysis involving the threefirst-order model related to students per ception about university academic reputation

PAGE 171

161 suggests that this model can be modified fo r future research. Considering the changes suggested in the results anal ysis, this study recommends that the only change that was justified was to drop from the scale the measured variable I6 (quality of libraries), given that this change provided an improvement in model’s fit. 5. What are the differences across gender in perceived importance of the selected factors that influence the students’ decisi ons about university c hoice process, and their perceptions of profe ssor’s effectiveness and university’s academic reputation at the University of Los Andes? Multivariate Analysis of Variance using gender as the predictor variable and the means of the student ratings by factor as th e criterion variables was conducted to test the gender effect. The results of the Wilks’ Lambda statisti c by domains shown a relatively weak relationship between the predictor variable (gender) and the criterion variables (factors), and the overall test of significance in multivariate analyses of variance revealed a nonsignificant multivariate gender effect, therefore, the findings of this analysis reveal that there is no difference between the male and female students when they are compared simultaneously on the factors that influenc ing their decisions a bout university choice process and their perceptions about profe ssor effectiveness and university academic reputation. These results are consistent with th e James, Baldwin, and Melnnis, (1999) findings. They also found that there is no significant differe nce across gender in perceived importance of the fact ors that influence the student s’ decisions about university choice process at different universities.

PAGE 172

162 6. What are the differences across univers ity campuses in perceived importance of the selected factors that influence the st udents’ decisions abou t university choice process, and their perceptions of prof essor’s effectivene ss and university’s academic reputation at the University of Los Andes? Multivariate Analyses of Variance using university campus as the predictor variable and the mean of the student ratings by factor as the criterion variables was conducted to test the university campus effect The results of the multivariate analyses of variance produced large Wilks’ Lambda indicating a relativel y weak relationship between the multiple factors and the university campus taken as a group, and the overall test of significance in multivariate analysis of variance by domains, conducted to test simultaneously differences between the campus groups on multiples factors, indicates that the null hypotheses are rejected, it means th at the university campus is significantly different with respect to at least one of th e factors that influencing university choice process, professor effectiveness and university academic reputation. Results of this study are consistent with the conclusions revealed for James, Baldwin, and Melnnis, (1999) and Hayden (2000) studies. They also found that university campus has a significant differe nce, when the students’ decisions and perceptions are compared on factors that influence university choice process and university academic reputation. The Tukey multiple comparison test used to determine university campus differences shown approximately similar results. However, the university campus ULAMerida established the main differences am ong the students when they are compares on the factors than influencing their decisi ons in university choi ce process and their

PAGE 173

163 perceptions about professor effectiveness and university academic reputation, following the campus 2 (NUTULA-Tachira) and the cam pus 1 (NURR-Trujillo). These findings should be explained by the he terogeneity respect to the origin of the ULA-Merida university campus students, given that they come from all the regions of the country. Based on the analyses of the results, this study could have several implications for the University of Los Andes, since it identi fies strengths and w eakness that guide the decisions related to university goals and policies. One imp lication could be the results’ interpretation for the decision makers; thes e findings could be us ed by the university’s authorities in the definition of academic polic ies such as permanent professor formation, professor evaluation, student enrollments, and re search’s stimulation; in order to keep university appearance from a point of view of values and prestige associated to professors, students, alumni, researches, and publications. Another implication could be the instrument’s utilization for a conti nuous assessment. The instrument could periodically be administrated possibly one tim e a year, in order to assess the university cultural evolution, given that whereas a requirement is satis fied, other become priority. Thus for example, in a university advanced culture with the basic requirements satisfied, the university athletic program s hould assume a priority position. Recommendations for Future Research This research has offered a construct validation of an instrument based on student’s university choice pr ocess and their perceptions of professor effectiveness and university academic reputation, as described pr eviously. Beside, it is hoped that the development of this instrument to orie nt the making decisions of the university

PAGE 174

164 authorities on the institutional goals and po licies that permit differentiate the institution across the higher education system. Because of the recommendations of this study are basically directed to the instrument’s validation, the sugge stions for future research sh ould be concentrated in first instance on the instrument and in second in stance to the polic y implementations. Consequently the following suggestions should be made as recommendations for future research: a) Additional investigations on demographic differences can be undertaken (e.g., age differences, socio-economic status differe nces, differences in academic rank), and to acknowledge the possibility that diffe rences across colleges (campuses) may have masked gender differences in the ove rall analysis made in this study, since that the students traditionally have pres ented variability in noticeable attributes such as age, race, gender, family struct ure, family income and home/university environment. Thus, understanding of thes e differences should be a challenge to the university system to find instructive st rategies that will meet the needs of all university students. b) In applications of factor analysis (exploratory and confirmatory) is widely understood that the use of large samples te nds to provide more precise and stable, or less variable estimates across repe ated sampling. Many authors (Ancher & Jennrich, 1976; Browne, 1968; Cudeck & O’Dell, 1994; MacCallum & Tucker, 1991) have presented and evaluated how random sampling influences parameter estimates and model fit, and they found th at solutions obtained from large sample showed greater stability in parameter estimat es and model fit. Therefore, a further

PAGE 175

165 research should be conducted with an e xploratory factor analysis based on this larger sample, followed by another conf irmatory study, in order to obtaining solutions that are adequately more stable and congruent with population factors. Consequently, these results involve gath ering construct validation evidence for the developed instrument. c) Considering that factor analysis does not appear to provide a criterion as to how many factors to accept, this study sugge sts for a further research, that the interpretation and a comparison of f it with a one-factor model should be undertaken before making a final decision respect to the assessment of model’s fit. Thus, these findings can be used as evidence that more factors should be retained and yield stable results.

PAGE 176

166 References American Educational Research Association, American Psychological Association, and Nationa l Council on Measurement in Education. (1985). The Standards for Educational and Psychological Testing Washington, DC: American Psychological Association. American Educational Research Association, American Psychological Association, and Nationa l Council on Measurement in Education. (1999). The Standards for Educational and Psychological Testing Washington, DC: American Psychological Association. Anderson, H. M. (1954). A study of certain criteria of teaching effectiveness. Journal of Experime ntal Education, 23 (1), 41-71. Anderson, J. C., & Gerbing, D. W. (1984) The effect of sampling error on convergence, improper solutions, and goodnessof-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155-173. Anderson, J. C. & Gerbing, D. W. (1988) Structural equation modeling in practice: A review and reco mmended two-step approach. Psychological Bulletin, 103 (3), 411-423. Angoff, W. H. (1988). Validity: An evol ving concept. In H. Wainer & H.I. Braun (Eds.), Test validity (pp. 19-32). Hillsdale, New Jersey: Lawrence Erlbaum. Archer, J. O., & Jennrich, R. I. (1976). A look, by simulation, at the validity of some asymptotic distribution results for rotated loadings. Psychometrika, 41, 537-541. Astin, A.W. (1985). Achieving educational excellence. San Francisco: JosseyBass. Babski, Carl. (1976). Does students’ expectation of teacher affect students’ evaluation of teachers? (ERIC Document Repro duction Service No. ED 118206). Ben-Shakhar, G., & Sinai, Y. (1991). Gende r differences in multiple-choice tests: The role of differential guessing tendencies. Journal of Educational Measurement, 28, 23-35. Blinn College. (1994). Results of entering student survey, 1993 fall and 1994 Spring semesters Research Study Report. Brenham, TX. Office of Inst itutional Research and Effectiveness.

PAGE 177

167 Boggs, C. (1995). Identifying gender bias in te aching evaluations: Insights offered by communication research Paper presented at the 45th Annual Meeting of the international Communication A ssociation, Albuquerque, NM. Bolger, N., & Kellaghan, T. (1990). Method of measurement and gender differences in scholastic achievement. Journal of Educational Measurement, 27, 165174. Bollen, K. A. (1986). Sample size and Be ntler and Bonett’s n onnormed fit index. Psychometrika, 51 (3), 375-377. Bollen, K. A. (1989). Structural equations with latent variables New York: John Wiley & Sons. Bollen, K. A. (1990). Overall fit in c ovariance structure models: Two types of sample size effects. Psychological Bulletin, 107 (2), 256-259. Borish, G. D., & Fenton, K. S. (1977). The appraisal of teaching: Concepts and process Massachusetts: Addison-Wesley Publishing Company. Brown, J. D. (1996). Testing in languag e programs. Upper Saddle River, NJ: Prentice Hall Regents. Browne, M.W. (1968). A comparison of factor analytic techniques. Psychometrika, 33 267-334. Browne, M.W. (1984). Asymptotical distri bution-free methods for the analysis of covariance structures. British Journal of Mathematic al and Statistical Psychology, 37 62-83. Butler, G. D. (1978). Teacher evaluation: The relati onship of self-concept to perceived effectiveness University Microfilms International. Campbell, D. T., & Fiske, D. W. (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological bulletin, 56 81-105. Central Office of Student Registration (OCRE). (2002). Student Statistics. University of Los Andes. Merida, Venezuela. Chapman, R. G. (1979). Pricing policy and the college choice process. Research in Higher Education, 10 (1), 37-57. Chapman, D. W. (1981). A model of student college choice. Journal of Higher Education, 52 (5), 490-505.

PAGE 178

168 Chou, C. P., Bentler, P. M., & Satorra A. ( 1991). Scaled test st atistics and robust standard errors for non-normal data in cova riance structure analysis: A Monte Carlo study. British Journal of Mathematic al and Statistical Psychology, 44 347-357. Clark, B.R. (1970). The distinctive college Chicago: Aldine. Cleave-Hogg, B., McLean, A.G., & Cappe L. E. (1994). Factors affecting applicants’ acceptance or decline of o ffers to enroll in a medical school. College and University, 69 (2), 104-109. Cliff, N., & Hamburger, C. D. (1967). The study of sampling errors in factor analysis by means of artificial experiments. Psychological Bulletin, 68 430-445. Cook, R. W., & Zollocco, R. L. (1983). Predicting university preference and attendance: Applied marketing in higher administration. Research in Higher Education, 19 (2), 197-211. Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory University of Florida. Harcourt Brace Jovanovich College Publishers. Cronbach, L. J. (1971). Test valida tion. In R. L. Thorndike (Ed.), Educational Measurement, 2nd Ed (pp. 443-507). Washington: American Council on Education. Cronbach, L. J. (1988). Five perspectives on the validity argument In H. Wainer & H.I. Braun (Eds.), Test validity (pp. 3-17). Hillsdale, New Jersey: Lawrence Erlbaum. Cronbach, L. J. & Meehl, P.E. (1955). Cons truct validity in psychological tests. Psychological Bulletin 52, 281-302. Cudeck, R. & O’Dell, L., L. (1994). Appli cations of standard error estimates in unrestricted factor analysis: Significance test s for factor loadings and correlations. Psychological Bulletin, 115, 475-487. Delaney, A. M. (1998). Parental income and students’ college choice process: Research findings to guide recruitment strategies Paper presented at the Annual Forum of the Association for Institutional Resear ch. (ERIC Document Re production Service No. ED 422 810). Dembowski, F. L. (1980). A model for predicting student college choice. College and University, 55 (2), 103-112. Dey, E. L., & Hurtado, S. (1995). College impact, student impact: A reconsideration of the role of stude nts within American higher education. Higher Education, 30, 207-223.

PAGE 179

169 Dunkin, M. J. (1986). Research on teaching in higher education. Handbook of Research on Teaching NY: MacMillan Publishing Company. Eccles, J. S., & Blumenfeld, P. (1985 ). Classroom experiences and student gender: Are there differences and do they matte r? In L. C. Wilkinson & C. B. Marrett (Eds.), Gender influences in classroom interaction (pp. 79-114). New York: Academic Press. Erickson, H. E. (1954). A factor ial study of teaching ability. Journal of Experimental Education, 23 (1), 1-39. Feingold, A. (1992). Sex differe nces in variability in in tellectual abilities: A new look at an old controversy. Review of Educational Research, 62, 61-84. Feingold, A. (1994). Gender differences in variability in intell ectual abilities: a cross-cultural perspective. Sex Roles, 30 81-90. Feldman, K. A. (1976). The superior colle ge teacher from the students’ view. Research in Higher Education, 5, 243-288. Feldman, K. A. (1989). Instructional eff ectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education, 30, 113-135. Fink, G. (1997). Survey of college plans of Maryland high ability students Annapolis, MD: Maryland State Higher E ducation Commission. (ERIC Document Reproduction Service No. ED 408 922). Frey, P. (1974). The ongoing debate : Student evaluation of teaching. Change, 6 (1). Gall, M., Borg, W., & Gall, J. (1996). Educational Research: An introduction (6th ed.). Longman Publishers USA. Gorsuch, R. (1983). Factor Analysis (2nd. Ed.). Hillsdale NJ: Erlbaum. Greenshields, M. J. (1957). The college -going decision: High school seniors give their reasons. College and University, 32 (2), 208-217. Hamrick, F., & Hossler, D. (1996). Dive rse information gathering methods in the postsecondary decision-making process. The Review of Higher Education 19 (2), 179-198.

PAGE 180

170 Harper, J., & Hill, G. (1989). Factors infl uencing college of agriculture students’ decision to enroll at the Un iversity of Nevada-Reno. College Student Journal, 23 (1), 912. Hatcher, L. (1998). A step-by-step approach to usi ng the SAS system for factor analysis and structural equation modeling Cary, NC: SAS Institute Inc., 588 pp. Hayden, M. (2000). Factors that influence the colle ge choice process for African American students Unpublished thesis, Faculty of the Virginia Polytechnic Institute and State University. Virginia, USA. Hedges, L., & Friedman, L. (1993). Ge nder differences in variability in intellectual abilities : A reanalysis of Feingold’s results. Review of Educational Research, 63, 94-105. Holland, J. L. (1958). Student explanations of college choice and their relation to college popularity, college produc tivity, and sex differences. College and University, 33 (3), 313-320. Hossler, D., & Gallagher, K. S. (1987). Studying student college choice: A threephase model and the implications for policy makers. College and University, 62 (3), 207-221. Hu, L., & Bentler, P. M. (1998). Fit indi ces in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3 (4), 424-453. Huddleston, T., & Karr, M. B. (1982) Assessing college choice. College and University, 57 (4), 364-370. Jackson, G. A. (1982). Public efficiency and private choice in higher education. Educational Evaluation and Policy Analysis, 4 (2), 237-247. Jacoby, J., & Olson, J. C. (1985). Perceived quality. Lexington, MA: Heath and Company. James, R., Baldwin, G., and Melnnis, C. (1999). Which University? The factors influencing the choices of prospective undergraduate The evaluations and investigations programmed of the Department of Education, Training, and Youth Affairs. Australia. Johnson, G. B. (1955). An evaluation instrument for the analysis of teacher effectiveness. Journal of Experimental Education, 23, 331-344. Johnson, S. (1987). Gender differences in scie nce: Parallels in in terest, experience and performance. International Journal of Science Education, 9 467-481.

PAGE 181

171 Joreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34 (2), 183-202. Joreskog, K.G., & Sorbom, D. (1989). LISREL 7: A guide to the program and applications, 2nd edition Chicago: SPSS Inc. Kim, J. O., & Muller, C. W. (1978). Factor analysis: Statistical methods and practical issues Beverly Hills, CA: Sage. Litten, L. H. (1982). Different strokes in the application pool: Some refinements in a model of student college choice. Journal of Higher Education, 53 (4), 383-402. Lytton, C., & Gadzella, B. (1991). Analysis of an instrument on a quality professor Paper presented at the Annual Meeting of the Texas Psychological Association. (ERIC Document Reproduction Service No. ED 353 279). MacCallum, R.C., & Tucker, L.R. (1991). Representing sources of error in the common factor model: Implicati ons for theory and practice. Psychological Bulletin, 109 502-511. MacCallum, R.C., Widaman, K., Zhang, S ., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4 (1), 84-99. Maguire, J., & Lay, R. (1981). Modeling th e college choice process: image and decision. College and University, 56 (2), 123-139. Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students’ evaluati on of university teaching. British Journal of Educational Psychology, 52, 77-95. Marsh, H. W. (1983). Multidimensional ratings of teaching effectiveness by students from different academic departments and their relation to student/course/instructor characteristics. Journal of Educational Psychology, 75 150166. Marsh, H. W. (1984). Students’ ev aluation of university teaching: Dimensionality, reliability, validit y, potential biases, and utility. Journal of Educational Psychology, 76 (5), 707-754. Marsh, H. W. (1987). Stude nts’ evaluation of univers ity teaching: Research findings, methodological issues, and di rections for further research. International Journal of Educational Research, 11 (3), 253-388.

PAGE 182

172 Marsh, H. W. (1992). A longitudinal perspective of students’ evaluation of university teaching: rati ngs of the same teachers over a 13-year period. Paper presented at the Annual meeting of the American E ducational Research Association. (ERIC Document Reproduction Se rvice No. ED 353 282). Marsh, H. W., & Bailey, M. (1991). Multidimensional students’ evaluation of teaching effectiveness: A Profile Analysis. Reports: Research/Technical (143). (ERIC Document Reproduction Serv ice No. ED 350 310). Marsh, H.W., Balla, J.R., McDonald, R.P. (1988). Goodness-of-fit index in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103 (3), 391-410. Marsh, H. W., & Hocevar, D. (1985). Th e application of confirmatory factor analysis to the study of self-c oncept: First and higher order factor structures and their invariance across age groups. Psychological Bulletin, 97 562-582. Marsh, H. W., & Overall, J. U. (1980). Students’ evaluations of instruction: A longitudinal study of their stability. Journal of Educati onal Psychology, 72 (3), 321-325. Mathews, M. B. & Hadley, T.D. (1993). The relationship between an institution’s perceived quality and stude nts’ application sets. College and University, 68 (1), 22-30. McDonough, P. (1997). Choosing colleges: How social class and schools structure opportunity State University of New York press. McKeachie, W. J. (1997). Student ratings. American Psychologist, 52, 1218-1225. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35 1012-1027. Messick, S. (1981). Evidence and ethi cs in the evaluations of tests. Educational Research, 10 (9), 9-20. Messick, S. (1989). Valid ity. In R. Linn (ed.). Educational Measurement (pp.13-103). New York: Macmillan. Mitzel, H. E. (1960). Teacher effectiveness Encyclopedia of Educational Research (3rd. ed.). New York, N.Y.: Macmillan. Murphy, P. E. (1981). Consumer buying ro les in college choice: Parents’ and students’ perceptions. College and University, 56, 140-150.

PAGE 183

173 National Commission of Teacher Education and professional Standards. (1955). “ Defining Teacher Competences ”. National Education Asso ciation, Washington, D.C. National Council of Venezuela. (1998). Statistics of the university sector Planning Office of University Sect or. OPSU. Caracas, Venezuela. Nunnally, J. (1978). Psychometric Theory (2nd. Ed.) New York: McGraw-Hill. Pike, G. (2003). Measuring Quality: A comparis on of U.S News rankings and NSSE benchmarks. Paper presented at the annual meeting of the Association for Institutional Research. Tampa, FL. Presidency of the Venezuela Republic. (1995) OCEI: Central Office of Statistics and Information Technologi cal. Caracas, Venezuela. Republic of Venezuela Congress. (1970). University Law Caracas, Venezuela. Rickman, C., & Green, G. (1993). Market segmentation differences using factors of the college selection process. College and University, (1), 32-37. Schibeci, R. A., & Riley, J. P. (1986) Influence of students’ background and perceptions on science a ttitudes and achievement. Journal of Research in Science Teaching, 23, 177-187. Seneca, J. T., & Taussing, M. K. (1987). The effects of tuition and financial aid on the enrollment decision at a state university. Research in Higher Education, 26 (4). Seldin, P. (1989). How colleges evaluate professors (ERIC Document Reproduction Service No. ED 305 872). Sevier, R. A. (1994). Image is everyt hing: Strategies for measuring, changing, and maintaining your institution’s image. College and University, 69 (2), 60-75. Smith, E. R., & Baxter, B. (1992). Quality education: An inquiry into its definition and a framework for its measurement. College and University, 67 (2), 107112. Stevens, J. (1996). Applied Multivariate Statisti cs for the Social Sciences Hillsdale, N.J.: Lawrence Erlbaum Associates. Struckman-Johnson, C., & Kinsley, S. ( 1985). Assessment and comparison of college image among high school seniors, college students, and alumni. College and University, 60 (4), 316-327.

PAGE 184

174 Tanaka, J.C. (1984). Some result on th e estimation of covariance structure models. Dissertation Abstracts International, 45 924B. Teacher Evaluation Project: Report for 1984-1985. (ERIC Document Reproduction Service No. Ed 266 120). The English Language Institute of America. (1973-1974). The Living Webster Encyclopedic Dictionary of the English Language Chicago. Trusheim, D., Crouse, J., & Middaugh, M. (1990). College applicants’ attitudes and enrollment decisions. Research in Higher Education, 31 (3), 295-305. University of Los Andes. (1999a). La Universidad de Los Andes……1785-2000. Merida, Venezuela. University of Los Andes. (1999b). Vicer rectorado Acadmico. Boletin No. 6 Merida, Venezuela. University of Los Andes. (2000). Proposiciones para el Vicerrectorado Acadmico. Merida, Venezuela. Wajeeh, E. M., & Miccerri, T. (1997). Factors influencing students college choice at traditional and metropolitan university Paper presented at the Annual Forum of the Association for institu tional research. (ERIC Docu ment Reproduction Service No. ED 416 727). Wanat, C., & Bowles, B. (1989). College choice and recruitment of Wisconsin’s all state academic scholars Paper Presented at the Annual meeting of the American Educational Research Associ ation. San Francisco, CA. (ERIC Document Reproduction Service No. ED 305 873). Young, S., & Shaw, D. G. (1999). Profile s of effective college and university teachers. Journal of Higher Education, 70 (6), 670-687.

PAGE 185

175 Appendices

PAGE 186

176 Appendix A: Programs of Study

PAGE 187

177 Appendix A: (Continued)

PAGE 188

178 Appendix A: (Continued)

PAGE 189

179 Appendix B: IRB Suggestions

PAGE 190

180 Appendix B: (Continued)

PAGE 191

181 Appendix C: Student Enrollments at the University of Los Andes.1 (Academic year, 2002) Faculty/School # Students % Sample Size2 Architecture and Art 1764 5.2 52 Dentistry 602 1.8 18 Economic and Social Sciences 5119 15.1 151 Engineering 3908 11.5 115 Forest and Environmental Sciences 1320 3.9 39 Humanities and Education 3181 9.4 94 Law and Political Sciences 3868 11.4 114 The Tachira University campus 3881 11.4 114 The “Rafael Rangel” Campus 4456 13.2 132 Medicine 3343 9.9 99 Pharmacy 1251 3.7 37 Sciences 1181 3.5 35 TOTAL 33,874 100% 1000 1Adapted from Statistics, Stude nts’ Enrollment. Computation Department: OCRE, ULA. 2Sample size = 1000 students.

PAGE 192

182 Appendix D: Letter of the Secretar y of the University of Los Andes

PAGE 193

183 Appendix E: English Version of Survey Instrument

PAGE 194

184 Appendix E: (Continued)

PAGE 195

185 Appendix E: (Continued)

PAGE 196

186 Appendix E: (Continued)

PAGE 197

187 Appendix F: Spanish Version of Survey Instrument

PAGE 198

188 Appendix F: (Continued)

PAGE 199

189 Appendix F: (Continued)

PAGE 200

190 Appendix F: (Continued)

PAGE 201

0 About The Author Josefa Maria Montilla received her econom ist’s degree in the University of Los Andes (ULA), Merida, Venezuela, in 1978. Th is same year she was admitted as an ordinary professor at the ULA. Subsequen tly, she obtained a study grant from the ULA to realize studies in the Central University of Venezuela, Caracas, Venezuela and in 1990 received a Master degree in agricultural economy. She has 26 years of experience in the field of higher education. Her occupation evolved from classroom teaching in Economy and Statistics courses in undergraduate program to Statistics courses in graduate pr ograms at the ULA. As both, professor and administrator, she developed and implemented several university programs, and has held several administrative positions in the Univer sity of Los Andes in Trujillo, Venezuela: Principal member of the University’s supe rior council, chair of the department of economic and administrative sciences, chair of the statistics area, chair of the economic theory area, and chair of the planning office. At the time of this doctoral progra m, she had a study grant from the ULA to realize studies at USF and finally she went b ack to continue her university tasks at the ULA.


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001709542
003 fts
005 20060614112224.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 060519s2004 flua sbm s000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001298
035
(OCoLC)69019189
SFE0001298
040
FHM
c FHM
049
FHMM
090
LB3051 (Online)
1 100
Montilla, Josefa Maria.
4 245
The construct validation of an instrument based on students university choice and their perceptions of professor effectiveness and academic reputation at the University of Los Andes
h [electronic resource] /
by Josefa Maria Montilla.
260
[Tampa, Fla.] :
b University of South Florida,
2004.
502
Thesis (Ph.D.)--University of South Florida, 2004.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 201 pages.
Includes vita.
520
ABSTRACT: The purpose of this study was to examine the construct validation of an instrument based on students university choice and their perceptions of professor effectiveness and academic reputation at the University of Los Andes (ULA). Moreover, a comparative analysis was carried out to determine how the selected factors that influence the students decisions and perceptions differ according to student demographic factors such as: gender and university campus. This instrument was developed with items based on the three domains formulated: university choice process, professor effectiveness, and university academic reputation. To determine the instruments appropriateness to measure the students decisions in university choice process and their perceptions about professor effectiveness and university academic reputation at the ULA, this research examined the reliability of scores by domains and factors across domains.The participants were undergraduate students who were registered in the second semester of 2002 and enrolled in the different courses by college within the ULAs main campus, which consists of ten colleges throughout the city of Merida, and within the other two university branch campuses in Tachira and Trujillo. For purposes of this research, a stratified probability sample was used to select the participants. The data show that the instrument designed has adequate internal consistency reliability estimates (all the domains exceeded .70). The confirmatory factor analysis shows that the overall fit indices revealed values at or close to the acceptable range .90, even when the model has statistically significant chi-square and demonstrates significant problems with some of the standardized residuals, which indicates that the fit of the model could possibly be significantly improved.The modified model revealed a relatively small improvement in the overall goodness of fit. These results provide supportive evidence of construct validity. Finally, the multivariate analyses of variance using gender and university campus as the predictor variables revealed a nonsignificant gender effect and a significant university campus effect, respectively. The Tukey multiple comparison test used to determine university campus differences across the domains showed approximately similar results, although they are separate and distinguishable. ULA-Merida established the highest mean scores when they are compared on the factors that influence their decisions in university choice process and their perceptions about professor effectiveness and university academic reputation, and the campus 1 (NURR-Trujillo) show the smaller mean scores.
590
Adviser: Jeffrey Kromrey, Ph.D.
653
Enrollment motives.
Teaching evaluation.
University image.
Validity.
Construct validity.
0 690
Dissertations, Academic
z USF
x Measurement and Evaluation
Doctoral.
773
t USF Electronic Theses and Dissertations.
856
u http://digital.lib.usf.edu/?e14.1298