|USFDC Home||| RSS|
This item is only available as the following downloads:
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c19979999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00088
Educational policy analysis archives.
n Vol. 5, no. 19 (September 23, 1997).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c September 23, 1997
Comprehensive study of factors impacting perceived quality in school organizations : findings from resesarch on quality assessment in Iowa school districts / William K. Poston.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 5issue 19series Year mods:caption 19971997Month September9Day 2323mods:originInfo mods:dateIssued iso8601 1997-09-23
1 of 43 Education Policy Analysis Archives Volume 5 Number 19September 23, 1997ISSN 1068-2341A peer-reviewed scholarly electronic journal. Editor: Gene V Glass Glass@ASU.EDU. College of Education Arizona State University,Tempe AZ 85287-2411 Copyright 1997, the EDUCATION POLICY ANALYSIS ARCHIVES.Permission is hereby granted to copy any arti cle provided that EDUCATION POLICY ANALYSIS ARCHIVES is credited and copies are not sold.Comprehensive Study of Factors Impacting Perceived Quality in School Organizations: Findings from research on quality assessment in Iow a school districts William K. Poston Jr. Associate Professor Department of Professional Studies Iowa State University Abstract This paper presents the findings of stu dies conducted at Iowa State University of public schools in Iowa in the area of perceived quality as sessment. Demographic characteristics of the respondents on the Perceived Quality Assessment Ins trument from forty-four school districts were described by position, home annual income, gender, age, level of education, and years experience in current or a similar job. The research project undertaken incorpo rated several studies of quality improvement characteristics of public schools. The project res ulted in a compendium of coordinated research aimed at learning more about the relationships and effects of quality improvement efforts with other factors of school district operations. The individual study components (doctor al dissertations) focused on the following issues: Assessment of quality improvement climate in commun ity colleges (Bax, 1994). Teachers' perceptions of training programs and thei r relationships to total district perceptions of quality management (Johnson, 1995). Performance-based pay of chief executive officers a nd effects upon quality improvement
2 of 43processes in school organizations (Behounek, 1996).Financial characteristics of school organizations a nd relationships to quality management factors (Kirchoff, 1996). This series of studies conducted over a period beginning in 1993 and culminating in 1996, was designed to assess the perceptions of school di strict stakeholders about the quality of their scho ol district in terms of the seven quality dimensions o f the Baldrige Award criteria. The instrument, in two versions, focused on the following Baldrige Awa rd areas: 1. Leadership 2. Information and Analysis 3. Strategic Quality Planning 4. Human Resource Development and Management 5. Management of Process Quality 6. Quality and Operational Results 7. Client Focus and Satisfaction Two other instruments were developed fo r use as a part of the -the Staff Development and the Executive Compensation Questionnaire was develo ped and used in the study. Financial information was obtained from government records. A determination was also sought to esta blish if the PQAI differentiated in terms of quali ty between high ranking and low ranking school distric ts. Inferential statistics established not only a significant difference between the high and low gro ups' quality effectiveness index, but there were also significant differences between the groups in each of the seven dimensions or sub-areas of the PQAI instrument. A significant positive relationship was found between the perceived quality of district st aff development and the perceived quality effectiveness index of the districts. Also, differences betwee n performance-related and situational-related (nonper formance) factors were evidenced between board presidents and superintendents and a weak inverse r elationship was found between performance-based compensation support and the perc eived quality of the systems. Significant differences were found between the high and low QEI groups in two areas -revenues per pupil in the leadership sub-area, and transportation cost per pu pil in the information and analysis category. No correlation was found between the sample schools fi nancial characteristics and their any PQAI rating area with one exception -transportation cost per pupil and information and analysis. Introduction Fragmentation is the process by which we take thing s apart and study the parts as a means to understanding the whole. This is an appropriate mo del for mechanical systems, but for organizations, it is the interaction among the part s that makes things happen (Graff, 1995). The quality improvement emphasis of the private sector has been permeating gradually into the public schools for over a decade. The direction school organizations have been heading has been modified during this period in attempts to improve the quality of schooling, the value of educational programs, and the accomplishments of learners. Ma ny changes were effected in schools in America during the aforementioned period, and support for s ustained efforts to reform education has come from federal, state, and local levels of governmen t (Lezotte, 1994). Once focused exclusively on manufacturing industries, quality improvement strat egies have been developed and implemented in many service organizations, including school system s. Quality improvement has become nearly a standard fixture of organizational development and staff training in schools, and the importance given to various principles of quality improvement has spawned considerable interest in the suitability of such strategies for school systems. Suddenly, policy officials are asking whether quality improv ement strategies make sense for school organizations, in which terms such as "product" or "customer" have imprecise definition.
3 of 43 School organizations' focus on quality improvement strategies is an outgrowth of several policy efforts in the United States and other educa tionally progressive countries. These policy effor ts indicate that everyone must become "champions of ch ange" (Rowe, 1994). Old leadership ideas won't work anymore, and new leadership principles m ust recognize a number of factors: Organizational hierarchies don't work Vision has power Shared values create alignment Successful management incorporates coaching, visi on, and facilitation Feedback on performance is crucial for improvemen t over time In order to foster organizational chang e, educational institutions have gathered in ideas from systems theory, psychology, management theory, huma n-resource and organizational development, statistical process control, and human synergy. Al l of these ideas, in many guises and combinations, aim to remake organizations so they become more foc used, disciplined, quick-footed, humane and competitive (Marchese, 1993). Many of these effort s are aimed at reducing the level of criticism toward schools and traditional types of organizatio nal operations (Bradley, 1993) Regardless of motivation, school system s have undertaken a number of initiatives to improv e classroom instruction, academic units, staff attitu des, and financial costs (Teeter and Lozier, 1993). A plethora of school improvements and reforms have been designed and implemented with little examination of the net results or effect. Often, s chool reforms and improvements are focused on a subordinate part of the system rather than on syste m-wide change (U.S. General Accounting Office, 1993). Specific effects of quality improvement eff orts have not been adequately anchored in research findings to determine their suitability and usefuln ess, and the need is widely apparent for identifyin g how and under what circumstances quality is manifes ted in school systems This medley of studies examines relatio nships and differences among groups, factors, and conditions associated with quality improvement in s chools. Specifically, this paper summarizes the problems, procedures, findings, and conclusions of four doctoral studies conducted under the supervision of Dr. William K. Poston Jr., Associate Professor of Professional Studies, Iowa State University. The four studies addressed relationshi ps and inferences in the areas of job satisfaction and work climate, staff development and training, c ompensation formats and incentives, and financial characteristics. The doctoral students involved in these four study areas included Dr. Rashid M. Bax, Malaysia Ministry of Education, Dr. Pamela D. Johns on, West Des Moines (Iowa) Community School District, Dr. Thomas E. Behounek, Milo (Minn esota) Public School District, and Dr. Joseph E. Kirchoff, Manchester (Iowa) Community School Dis trict. A number of additional studies are currently underway in the areas of strategic planni ng, facilities and productivity, and student achievement by several other graduate students at I owa State University. Definitions of Terms Quality improvement is characterized by specific terminology. Terms utilized in this stud y are defined below to provide clarity and understand ing of their use in these studies: Executive Compensation: The sum paid to the chief executive officer of an educational institution and the underlying rationale for its de termination, including results of performance or situational circumstances.Job Satisfaction: One's perceived emotional state expressed as a po int on a scale representing the degree of positivity or pleasure r esulting from the appraisal of one's job or one's experience (Locke, 1976).Malcolm Baldrige Award: A national award presented by the United States G overnment to businesses, companies, or organizations for thei r demonstration of excellence and
4 of 43exemplification of quality principles (Schenkat, 19 93). It is similar to the Deming Award in Japan at least in intent.Organizational Commitment: Relative strength of an individual's identificati on with, and involvement in, a particular organization. Concept ually, it can be characterized by at least three factors: (1) a strong belief in, and ac ceptance of, the organization's goals and values; (2) a willingness to exert considerable eff ort on behalf of the organization, and (3) a strong desire to maintain membership in the o rganization (Porter, Steers, Mowday, and Boulian, 1974, p. 604).Perceived Quality Assessment Instrument: A questionnaire developed collaboratively by William K. Poston Jr. and Rashid M. Bax at Iowa Sta te University designed to elicit perceptions of organizational stakeholders on seven dimensions of quality improvement modeled after the Baldrige Award criteria.Perceived Quality Effectiveness Index: A ratio or score produced by dividing the stakeholders' perceptions of the current status res ponse score by the desired status response score, expressed as a decimal. Ratios les s than one (1) indicate need for improvement; ratios greater than one (1) indicate e xceeding expectations. Quality Improvement (synonyms: quality management, continuous quality improvement, or total quality management): a customer-focused st rategic and systematic approach to continuous performance improvement (in an organizat ion) (Vincoli, 1991, p. 28). Staff Development or Professional Development: Activities of school organizations that seek to prepare employees for improved performance in their present or future duties and responsibilities.Staff Development Questionnaire: An instrument developed for the purpose of measuring the perceived quality of staff developmen t activities in an educational organization (Johnson, 1995).Systems Theory: A practice or intention to view and work with org anizations as an interdependent set of components including purpose, people, methods, environment, materials and other factors that influence the orga nization's functioning and activities and that affect and interrelate with all other factors. Purposes and Rationale for the Study There were three main purposes for cond ucting this series of studies. The first purpose o f these studies was to determine if the Perceived Qua lity Assessment Instrument (commonly referred to as the PQAI) designed and developed at Iowa State U niversity was effective in measuring factors commonly associated with excellence, efficacy, and effectiveness in school organizations. The second purpose was to identify the extent of any re lationships and the nature of any associations among variables measured by the PQAI and other meas ures used as a part of the studies. Finally, the purpose was to find whethe r or not operations and management actions in schoo ls have an impact or influence upon the perceived leve l of quality within the system. Understandings obtained from these data should help school systems seeking improvement of quality to see more clearly what gaps exist among present and desired o rganizational conditions and what part the studied variables play in a systems approach for school org anizations. The studies also attempted to demonstrate how management decisions might close th e gap between the "current" and "ideal" status of factors as measured on the PQAI.
5 of 43 Research Questions for the Study Questions relevant to the studies were formulated and postulated in four areas -general perceptions of quality management, staff developmen t, executive compensation, and financial factors. The questions addressed are listed below 1. Perceptions of Quality Management Assessment: What are the perceived current and ideal levels of quality management and the quality effectiveness index (ratio) between the two levels in the selected districts? What components of the perceived quality assessment instrument have the greatest impact on the overall quality rating of the selecte d districts? What value can be ascribed to the Perceived Quality Assessment Instrument in reliability and validity in measuring system quality? 2. Staff Development Questions: What are perceptions of quality of staff developmen t programs in the selected districts? Is there a relationship between the perceived quali ty of district staff development and the perceived level of quality management in the select ed districts? Is there a relationship between ratings of perceive d levels of staff development programs and the seven dimensions of district quality? How do districts rated highest and lowest in percei ved quality of staff development differ or compare on the perceived quality effectiveness i ndex? 3. Executive Compensation Questions What factors are used and which are preferred in th e compensation of the chief executive officer of school districts by superintendents and governing board presidents and how do these two groups differ in choice of factors determ ining compensation? How do methods used and preferred in determining co mpensation for the chief executive relate to levels of quality in the system as measur ed by the Perceived Quality Effectiveness Index? 4. Financial Factors Questions To what extent are there differences between distri cts with perceived high and low quality on the Perceived Quality Assessment Instrum ent in terms of financial characteristics? Is there a relationship between the financial sta bility or soundness of school districts and their perceived quality effectiveness index or sub-scales of the Perceived Quality Assessment Instrument? Hypotheses and Assumptions of the Studies All hypotheses were constructed and ana lyzed in null form for statistical analyses. Several assumptions supported the studi es, including the following:
6 of 43The population of the study was representative of t he districts in the state of Iowa, and respondents were representative of the total popula tion of the systems involved in the studies. Respondents understood the content and direction of instrumentation. Subjects voluntarily participated in the studies by completing instrumentation. The measured perceptions accurately reflected actua l levels of organizational quality. Respondents were knowledgeable about their school s ystem, and responded accurately and honestly to all instrumentation. Rationale for Quality Improvement in School Organiz ations Continuous quality improvement, also in accurately called total quality management, has bee n gathering momentum in the United States and elsewhe re. The systems thinking approach was led by W. Edwards Deming, a Sioux City, Iowa native, befor e his death in 1993, and it has continued strongly since. The "new philosophy" has affected businesses, industries, government, and educational institutions (Brown, 1992). Literature on quality improvement has mushroomed in the past several years with application for school orga nizations. Deming often wrote and spoke of "continuous improvement" with the goal of quality, so the term "continuous quality improvement" is frequently used in discussions about school organiz ation improvement, reform, or transformation (Johnson, 1995). Education is in need of dramatic change according to recent critics. Transformation has been called for in government, industry, and education i n the United States, and such change must "be a change of state, metamorphosis, not mere patchwork on the present system of management (Deming, 1990). Specific approaches for improvement are less than clear and distinct. According to Minnesota's Lieutenant Governor, Joanell Drystad, f ragmented, individual attempts at system repair have not succeeded, and the only answer is fundamen tal, systematic transformation. In an address to over 2000 teachers, administrators, students, polic y-makers, and business partners from 26 of the United States, Finland, and Canada, the call was ma de to "accelerate local transformation efforts in order to meet the national goals by the year 2000 b y deploying quality quickly. By sharing and working together through the total quality systems approach, we can improve our nation's schools, provide better learning options to our students, an d ensure a world-class work force" (Drystad, 1994). A study group of forty-four school dist ricts was identified as the population for the rese arch project. The Perceived Quality Assessment Instrume nt was administered to the group of school districts. The criteria used in development of the specific items on the instrument (see Appendix for sample of the instrument) were drawn from research literature and documented successful institutional practice. The criteria reflect a num ber of factors that focus upon the development of educational excellence. Continuous improvement is an organizati onal behavior grounded in several important characteristics -clear goals, mission, and organi zational expectations, defined direction for the design and delivery of superior teaching and learni ng, continuous focus on results and the use of feedback in decision-making, equitable and consiste nt connections among all organizational components for all clients and stakeholders, and ef ficient and effective use of resources (Frase, English, and Poston, 1995). These factors form a f oundation for the assessment of organizational functioning, and comprise a comprehensive assessmen t strategy. The characteristics of such a strategy should include the following (US Dept. Com merce, 1995): Clear ties between what is assessed and the school system's objectives, particularly in what clients are to obtain (learning, services, etc.) or gain from the organization. A focus on improvement -built upon a definition o f student performance, faculty and staff capabilities, and program performance. Assessment as "embedded and ongoing" that is curric ulum-based, criterion-referenced, and aimed at fostering improved understanding and accom plishment of goals and requirements.
7 of 43Clear guidelines as to how assessment results will be used. An ongoing method to evaluate the evaluation proces s to improve the connection between goals and client success. For effective school improvement to occ ur, significant changes will have to be made in how school systems function. Many facets of school ope rations will have to be modified, but serious commitment calls for an understanding and implement ation of quality management. The past success of quality management in business and industry rais e the possibility of applying its principles to education (Teigland, 1994). It is important that t he highest levels of leadership provide direction f or quality improvement to occur (Bax, 1994; Walton, 19 86). School administrators must seriously consider quality improvement as one option for brin ging about much needed change and improvement in school organizations (Teigland, 1995 ). The Quality Improvement Studies Despite its recency in use as an organi zational development tool, quality improvement in schools does not have a long history of research in its efficacy and impact upon commonly held measures of school organizational characteristics. At Iowa State University, instrumentation was developed for use in measurement of the status of p erceived quality within a school organization. The assessment tool, the Perceived Quality Assessme nt Instrument (PQAI) was structured to determine the perceived level of quality of a schoo l system based upon selected Baldrige Award criteria.Baldrige Award Criteria This group of impact studies involved a n adaptation of the Malcolm Baldrige Award criteria The Baldrige Award, begun in the United States, was designed to recognize corporations for excellence in achieving quality. The Malcolm Baldr ige National Quality Award (Baldrige Award) was established in 1987 through legislation (P.L. 1 00-107). The purposes of the Baldrige Award were threefold: To promote awareness of the importance of quality i mprovement to the national economy; To recognize organizations which have made substant ial improvements in products, services, and overall competitive performance; and To foster sharing of best practices information amo ng U.S. organizations. Eligibility for the Baldrige Award was initially open only to for-profit organizations. However, recent developments indicate that eligibil ity might be extended in the future to not-for-profit organizations. The Baldrige Award Program strategy con sists of two parts: (1) conceptual and (2) institutional. The conceptual part of the strategy involves the creation of consensus criteria which project clear values, set high standards, focus on key requirements for organizational excellence, and create means for assessing progress relative to the se requirements. The institutional part of the strategy involves use of the Criteria as a basis fo r consistent communications within and among organizations of all types. Such communications sti mulate broad involvement and cooperation, and afford a meaningful and consistent basis for sharin g information. An important part of the communications is the sharing of information by Bal drige Award recipients (U.S. Dept. of Commerce, 1995). Through the Baldrige Award, rigorous cr iteria were created to evaluate applicants for the Award. The Baldrige Award Criteria, based upon a se t of core values and concepts, focus on key requirements for organizational excellence. These r equirements are incorporated in a seven-part Criteria framework. Accompanying this framework is a set of Scoring Guidelines which permit
8 of 43evaluation of performance relative to the detailed Criteria. The evaluation leads to a feedback report--a summary of strengths and areas for improv ement. All Baldrige Award applicants receive a feedback report. The Baldrige Award Criteria and Scoring Guidelines have led to a number of key developments: Creation of a means for self-assessment; Replication of an award system by hundreds of organ izations, including states, cities, companies, and not-for-profit organizations; and Creation of training programs. Throughout the life of the Baldrige Awa rd Program, the principal uses of the Criteria have been for such other purposes. To date, more than on e million copies of the Criteria have been disseminated, and a like number of copies have been duplicated by others. This compares with a total of 546 applicants for the Baldrige Award. Since the Baldrige Award was establishe d in 1987, there have been 22 Award recipients (1988-1994). Award recipients have demonstrated a w ide range of improvements and achievements, including product and service quality, productivity growth, customer satisfaction, reduced operating costs, and improved responsiveness. Also, Award re cipients are among the Nation's leaders in investment in developing the skills of the work for ce. Working Toward an Education Category Si nce the inception of the Baldrige Award in 1987, some educators have been involved in the Program th rough their service on the Award's Board of Examiners. In addition, the Award recipients have sought to involve educators and educational organizations, locally and nationally. Also, some s tate and local award programs already include education categories. In parallel with these Award developments, many educators have launched quality improvement efforts. National initiatives such as Goals 2000 reflect a growing national consensus to strengthen education. As a result of these and related developments, interest has grown in establishing a Baldrige Award category for educa tion. In 1993, a decision was reached to launch Pilot activities in 1994 and 1995 to address the ma ny issues that arise in extending eligibility to education. Perceived Quality Assessment Instrumentation Parallel with the development of the Ba ldrige Award criteria, instrumentation was under development to apply principles of the system to ed ucation. In 1993, an attempt was undertaken to develop instrumentation that focused on the factors of quality improvement in connection with school system organizational performance. The Perceived Q uality Assessment Instrument was developed for use with school districts by William K. Poston Jr. and Rashid M. Bax at Iowa State University in 1993-94. The instrument was built along the line s of the Baldrige criteria framework, and included seven dimensions: leadership, information and anal ysis, strategic quality planning, human resource development and management, management of process q uality, quality and operational results, and client focus and satisfaction. Six to eight items were developed and included in each category. Statements were developed addressing the operations and functions of school districts, and respondents were requested to report their percepti on of the current situation of their system on a fi ve point Likert-type scale. The respondents were also requested to indicate their perception of the idea l or preferred situation on the same scale on each st atement. Responses included "strongly agree, agree, neutral, disagree, and strongly disagree."Calculating the Quality Effectiveness Index Unique to this instrument was a quality effectiveness index, which was a calculated ratio of the current situation and the ideal situation score s on each item, each of the seven dimensions, and f or
9 of 43 the total instrument. The formula for the index wa s expressed as follows:QEI = Rc / Ri QEI = Quality Effectiveness Index Rc = Rating of Current Situation Ri = Rating of Ideal or Preferred Situation Calculating the quality effectiveness i ndex in this manner permitted a basis for compariso n across school systems as to the level of quality co nsidering the ideal or favored status of the system The expressed ratio expressed a bi-directional asse ssment of quality, and indicated a diminution of quality as the ratio decreased from 1.0, and an ach ievement of expectations as the ratio increased above 1.0. In other words, if a school system had a quality effectiveness ratio of .50, it was only approximately 50% of the way toward the level of qu ality it perceived as desirable. A quality effectiveness ratio of 1.25 indicates surpassing th ose same expectations by approximately 25% Such an instrument in educational resea rch can be very valuable if carefully planned and developed (Borg and Gail, 1989). Self-assessment t ools like the PQAI have been described in the professional literature as tools for obtaining an a erial view of the territory to be explored (Neuroth Plastik, and Cleveland, 1992). By plotting a distr ict's current location on the map, educational leaders can determine what courses of action would be appropriate for improvement. Validation and reliability determinatio ns for the instrument are discussed in the findings section of this report. The Perceived Quality Asse ssment Instrument criteria are most explicit in the areas of organizational functioning pertaining to t eaching and learning. The focus on teaching and learning depends upon leadership and organizational expectations, information and analysis of assessment data for improvement, strategic planning for quality improvement and mission accomplishment, development and management of human resources congruent with participatory management, employment of process quality principle s and procedures, monitoring of quality and operational results, and a clear focus on clientele and their satisfaction. The point values and Likert-type scale represent an initial attempt to provide a basis for scoring participating school systems in terms of pr ogress in performance improvement. Methodology of the StudiesPopulation of the Study The study involved school districts in the State of Iowa, a mid-western state with an agricultural-industrial economic climate and a rura l-urban mix of communities. The specific population of the study was identified prior to the commencement of the study, so that questions could be postulated toward the respondents selected (Borg & Gall, 1989). A graduate seminar group of Ph.D. students and the major professor, common t o all the students, was formed for the purpose of framing and jointly planning the study. The semina r determined several aspects of the study, including the population selected, the variables me asured and scrutinized, and the activities pursuant to the completion of the research on perceived qual ity improvement. Forty-four of Iowa's 360 districts were identified and selected to participate in the stud y by the graduate seminar group. Participation was voluntar y on the part of each school organization, and efforts were made to include systems representative of different sizes and configurations. The districts selected were representative of the state as a whole in size and geographic distribution. Agreements to cooperate in the study were received from each district superintendent in order to increase the response rate. Instruments were deliv ered to the superintendents' offices for distributi on
10 of 43 to respondents. The superintendents were also aske d to have the system's official legal representative, commonly referred to as board secre taries, select the respondents randomly and distribute the surveys. A sample of personnel completed the stu dy instrumentation. The Perceived Quality Assessment Instrument (PQAI) was completed by the s uperintendent, all board members, two administrators, five teachers, three support person nel, and two high school students for a total of 72 0 possible respondents in the 44 districts. The other instruments were distributed in similar manner, with different personnel responding. Staff development information was elici ted by a questionnaire designed specifically for th at purpose, as was the information on compensation str uctures. Both the Staff Development Questionnaire and the Executive Compensation Instru ment were designed specifically for this study, and both were validated with standard review and de velopmental procedures. Financial information used in the study was obtained from official publis hed sources, provided by the Iowa State Department of Education.Research Design and Variables of the Study A survey design was used to answer the research questions. The first instrument (PQAI) consisted of two parts: Part I Demographic Inform ation, and Part II Rating of School System Quality Components. The other instruments were draw n from research literature. The dependent variables were the ratios (Quality Effectiveness In dex) between current and ideal perceptions of quality management in each district. The specific calculation involved a ratio of the current perceived quality improvement status divided by the perceived ideal quality improvement rating. The ratios also included the seven sub-scales: Leadersh ip, Information and Analysis, Strategic Quality Planning, Human Resource Development and Management Management of Process Quality, Quality and Operational Results, Client Focus and Satisfact ion. The independent variable of the study was the perception of quality of the districts' staff devel opment programs. The sub-scale areas and demographic variables of the PQAI are shown in Tabl e 1 below. Table 1. Sub-scale areas and demographic variables of the PQAI Quality ComponentsDemographic Information LeadershipPositionInformation and AnalysisHome Annual IncomeStrategic Quality PlanningGenderHuman Resource Development and Management Age Management of Process QualityLevel of EducationQuality and Operational ResultsYears Experience in Job Client Focus and Satisfaction Development of the Instruments and Data Source The Perceived Quality Assessment Instrument (PQAI). Use of the questionnaire in educational research can be very valuable if carefu lly planned and developed (Borg & Gall, 1989). In this study, three questionnaire surveys were develo ped for use in this study, and a fourth data-gathering technique was also used. The first survey instrument was based upon the Baldrige award criteria and labeled, "School System Perceive d Quality Assessment Instrument" (Poston &
11 of 43Bax, 1994, see Appendix A). The instrument was bas ed upon the seven dimensions of the Malcolm Baldrige Award areas: Leadership, Information and Analysis, Strategic Quality Planning, Human Resource Development and Management, Management of Process Quality, Quality and Operational Results, and Client Focus and Satisfaction. Six to eight items are included in each category with statements addressing the operations and policies of school districts. Respondents were aske d to judge their current situation and the desired or ideal situation in their school system for each ite m using a Likert scale (Borg & Gall, 1989) of five possible responses for each (Strongly Disagree, Dis agree, Neutral, Agree, Strongly Agree). Demographic items were also included (position, inc ome, gender, age, level of education, and years of experience in current or similar job) to aid in possible statistical comparisons and analyses of th e groups. The instrument was refined and improved fo llowing Bax's study (Bax, 1994). The Perceived Quality Assessment Instru ment (PQAI) was developed and based upon principles of sound informational and research prin ciples. Borg and Gall (1989) report that specific behaviors can be predicted from attitude measures a bout those behaviors, therefore a Likert-type scale of five possible responses for each item was again used: Almost Never, Occasionally, Don't Know, Frequently, Almost Always (Knudsen, 1993). T he third response, Don't Know, was included in the third position to encourage either a negativ e or positive answer from respondents. Questions posed in closed form aid in the efficient quantific ation and analysis of results (Borg & Gall, 1989). Some items were reversed from the others, so that n ot all statements would be written in similar (positive) format (Borg & Gall, 1989), and (2) so t hat respondents would not feel that every answer should be marked Almost Always to be "correct." Th e length of the items and the length of the questionnaire itself were kept as short as possible for ease in understanding and increased chance of the instruments being returned (Borg & Gall, 1989). In addition, several devices were used to encourage timely participation, including a self-ad dressed postage-paid mailer, phone calls, and personal contacts. The PQAI was validated by thirteen expe rts in quality improvement who were educational leaders, superintendents, university professors, or university officials with an understanding of qual ity processes. The panel is listed in the Appendix. E ach member of the panel was asked to assure that each item accurately reflects the concepts purporte d to be measured by the instruments, to evaluate the items for clarity and completeness, and to make suggestions for improvement. Based on the recommendations of the panels, the instruments were revised. The Perceived Quality Assessment Instrument was further pilot tested by a group of I owa State University doctoral students studying quality management in public schools. The PQAI ins trument took approximately ten to fifteen minutes to complete. Development of the Other Instruments and Data Sourc es Staff Development Instrumentation. The District Staff Developmental Questionnaire (Johnson, 1995) was developed by one of the members of the study group (Johnson, 1995) after reviewing staff development literature in the follo wing areas: (1) planning, (2) administrative support, (3) delivery, (4) follow-up, and (5) evalu ation. (Wood et al., 1982; Fullan, 1985; Joyce & Showers, 1988; Sparks and Loucks-Horsley, 1989; Sen ge, 1990; Rebore, 1991; Guskey, 1994; McBride et al., 1994; Sparks & Vaughn, 1994) Orlich 1983; Guskey & Sparks, 1991; Templeman & Peters, 1992; Showers, 1985; Wade, 1985; Bates & St achowski, 1991; Wood & Thompson, 1993). The instrument was validated by ten staff developme nt professionals recognized throughout the state and nation as very knowledgeable in this field. A Likert scale approach was used for the instrument. Executive Compensation Instrumentation. The Current and Preferred Methods of Awarding Superintendent Salary Increases Questionnaire (Beho unek, 1996) survey instrument was developed following a review of related literature and an int erview of ten superintendents and twelve board members about methods of awarding compensation to s uperintendents. Twelve criteria were incorporated in the instrument, with five of the cr iteria based upon the superintendent's performance
12 of 43and seven of the criteria nonperformance related. The instrument was validated by a panel of nine practicing administrators and researchers, and fiel d tested with twelve board members. A Likert-type scale was used for the instrument. Financial Information Instrumentation and Data Sour ces. The source of financial information for the selected Iowa school districts was obtained from the official financial audits for each system from the 1992-93 school year, the most recent data available (Kirchoff, 1996). The source of the audits used in the study was the Stat e Auditor's Office, a division of government in Iowa. The audits were used as the source of studen t enrollment, fund balances, revenues and expenditures. Human Subjects Committee Approval The Iowa State University Committee on the Use of Human Subjects in Research reviewed all research projects as a part of this study and concl uded that the rights and welfare of the human subjects were adequately protected, that risks were outweighed by the potential benefits and expected value of the knowledge sought, that confidentiality of data was assured, and that informed consent was obtained by appropriate procedures. All instruments employed the use of ano nymity for respondents, as there was no request for names of respondents. However, the questionnaires w ere coded to identify which school system the respondents represented. This was necessary for fo llow-up and data analysis. The completion of the questionnaires was voluntary and constituted consen t to participate in the research project. All questionnaires were kept secure throughout the dura tion of the study and filed for safekeeping after the study was concluded.Data Collection Procedures Study instrumentation was personally de livered to each superintendent or his/her designee in each district in late February and early March of 1 994. The accompanying directions for completing the surveys were self-explanatory. Instructions we re printed on the fronts and backs of the questionnaires to mail the completed instruments di rectly to Iowa State University by dropping them in the U.S. mail. A postal permit was printed on t he back of each. To increase the rate of return, phone calls were made to participating superintende nts in April asking them to encourage those who had not yet responded to do so. As a result, 471 of the 720 PQAI survey s were received for a return rate of 65.4%. Statistical Analysis of The Data Perceived Quality Assessment Instrument. After the surveys were returned, the responses were entered into a Microsoft Excel spreadsheet pro gram. Only one of the PQAI instruments was discarded because it was damaged in the mail. Four of the districts were removed from the study because an insufficient number of PQAI surveys were returned (one survey and was returned from two districts and three surveys were returned from two others). As a result, 462 surveys representing forty districts were used for data analysis. The it ems were ranked on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). Means for each re spondent and each district for each question were computed. On the Perceived Quality Assessment Inst rument, means for each respondent and district for the current and ideal situations were calculate d as well as the ratio (quality effectiveness index ) of the current divided by the ideal. Descriptive statistics were computed on the demographic variables of the PQAI. Preliminar y analyses using the one-way analysis of variance (AN OVA) were conducted to compare the responses of demographic groups. Factor analysis was conduct ed on the PQAI to determine reliability. The ANOVA and Scheffe' post-hoc method were used to det ermine differences between the four groups. Other instruments. Spearman rho and Pearson product-moment correlatio n coefficients were
13 of 43computed to test for a relationship between respons es on the PQAI and the staff development and compensation instruments and between financial fact ors and the PQAI. Stepwise multiple regression was used to find a relationship between staff devel opment and the seven PQAI dimensions. The differences between high and low districts on the c ompensation and staff development instruments and financial factors were determined using the t t est and analysis of variance techniques. The Cronbach alpha reliability coeffici ent was also calculated for the Staff Development Questionnaire and the Executive Compensation Instru ment to determine the internal consistency of the total instruments. Findings of the StudiesIntroduction The findings of the study are organized into the following sections: (1) General Characteristics of the Sample, (2) Reliability Anal ysis of the Instruments, (3) Quality Management Perceptions, (4) Staff Development Perceptions, (5) Results of Hypotheses Tested, (6) Evaluation of the Perceived Quality Assessment Instrument, and (7 ) Summary. General Characteristics of the Sample The primary purpose of this section is to describe the participants in this study who comp leted the Perceived Quality Assessment Instrument (PQAI) with respect to the following demographic variables: (a) position, (b) home annual income, (c ) gender, (d) age, (e) level of education, and (f) years experience in current/similar job. The descri ptive information is presented in Table 2 which follows. Position Of the 458 respondents who filled in this questi on, 132 (28.8%) were teachers, 76 support staff (16.6%), 68 administrators (14.8%) 30 superintendents (6.6%), 108 board members (23.6%), and 44 students (9.6%). Four people did not complete this category.Home Annual Income Of the 358 people answering this question, 24 re ported earnings of less than $10,000 (6.7%), 77 from $10,000 to $29 ,999 (21.4%), 147 from $30,000 to 49,999 (40.9%), and 110 reported earnings of $50,00 0 or more (30.9%). There were 104 respondents who did not fill in this information.Gender The number of male and female participants repor ting their gender was fairly even. Male respondents numbered 215 (52.6%) while f emales totaled 194 (47.4%). The number who did not fill in this item was 53.Age The age of the respondents was divided into five categories: (1) under 18, (2) 18-29, (3) 30-55, (4) 56-70, and (5) over 70. The l argest group was the 3055 year category (73.8%). The second largest group was the 56-70 year group (11.9%). The smallest group was the over 70 category with only o ne person. Forty-two people did not report their age group.Level of Education The educational level of the respondents was cla ssified into four categories: (1) less than a B.A. degree, (2) B.A. d egree, (3) Master's degree, and (4) Doctorate degree. The largest group (151 people) re presented those who had less than a B.A. degree, (34.6%). Those having a B.A. degree we re the second largest group with 142 people (32.5%). The third largest group, those having a Master's degree, were very close with 129 people ((29.5%). Fifteen respondents (3.4%) reported having a Doctorate
14 of 43 degree. Twenty-five respondents did not report thei r level of education. Years of Experience The years of experience of the respondents were divided into four categories: (1) under 5 years, (2) 5-10 years, (3) 11-25 years, and (4) 25 years or more. The findings revealed that 191 (44.3%) of the parti cipants had 11-25 years of experience, 96 (22.3%) had 5-10 years of experience, 88 (20.4%) had under five years, and 56 (13%) had 25 or more years of experience. Thirty-one resp ondents did not complete this item. Table 2. Demographic information of respondents on the PQAI Category FrequencyPercent Position Teacher Support Staff Administrator Superintendent Board Member Other Total 132 766830 108 44 458 28.816.614.8 6.6 23.6 9.6 100.0 Income < $10,000 $10,000 $29,999 $30,000 $49,999 $50,000 + Total 2477 147110358 6.7 21.440.930.9 100.0 Gender Male Female Total 215194409 52.647.4 100.0 Age < 18 18 29 30 55 56 70 70 + Total 2336 310 50 1 420 5.58.6 73.811.9 0.2 100.0 Education < Baccalaureate Baccalaureate Master's Degree Doctorate Total 151142129 15 437 34.632.529.5 3.4 100.0 Experience < 5 years 5 10 years 11 -25 years 26 + years Total 8896 191 56 431 20.422.344.313.0 100.0 One-way analysis of variance and t-test procedures were used to determine differences in t he quality effectiveness index among different demogra phic groups. It was found that there were significant differences in responses by position, g ender and education. The Scheff method was used to analyze these differences and established that t here were significant differences in the position category between the responses of teachers and admi nistrators and between teachers and board
15 of 43 members, both at the .05 level. A description of re spondents by position follows. Analysis by position In order to gain a better understanding of the respondents on the PQAI and their differing characteristics, further analysis was conducted on the position category by generating cross-tabulations with the other variables--income, gender, age, education and experience. The results are shown in Table 3 that follows. Position with income As shown in the following Table the majority o f teachers (50) reported incomes in the $30,000 to $49,999 category with the second highest number (44) being at the over $50,000 level. Support staff reported 29 p eople in the $30,000 to $49,999 and 27 at the $10,000 to $29,999 level. For administrators, the two highest cat egories were over $50,000 (39) and $30,000 to $49,9 99 (28). Only one administrator reported making less t han $29,999. All of the superintendents except two (27) reported incomes over $50,000, while the r emaining two reported earnings in the $30,000 to $49,999 category. The majority of board members (51 ) were in the over $50,000 category, with 31 at the $30,000 to $49,999 level, and 16 at the $10,000 to $29,999 level. Of those in the other category (students), 23 reported earnings under $10,000 with the remainder (16) dispersed throughout the other three levels. Table 3. Demographic information of PQAI responden ts by position CategoryTeacherSupport Staff Admin.Supt.BoardOtherTotalPercent Income (35 missing) < $10,000 $10,000 $29,999 $30,000 $49,000 $50,000 + Total 0 305044 124 1 27291269 01 283968 002 2729 0 16315198 23 376 39 24 77 147 179 427 5.6 18.034.441.9 100.0 Gender (53 missing) Male Female Total 3982 121 214869 501060 22 0 22 653095 182442 215194409 52.647.4 100.0 Age (42 missing) < 18 18 29 30 55 56 70 70 + Total 1 18 93 10 0 122 14 56 90 70 01 54 80 63 01 22 60 29 00 8014 1 95 2112 530 41 2336 310 50 1 420 5.58.6 73.811.9 .2 100.0 Education (25 missing) < Baccalaureate Baccalaureate Masters Degree Doctorate Total 1 95 00 126 5710 00 71 1300 68 0299 29 5532 66 403 37 000 40 151142 1515 437 34.632.5 3.43.4 100.0
16 of 43 Experience (31 missing) < 5 years 5 10 years 11 25 years 26 + years Total 12306717 126 132133 6 73 181724 9 68 48 15 2 29 17204818 103 24 044 32 8896 191 56 431 20.422.344.313.0 100.0 Position with gender There were over twice as many females as males i n the teacher (82 and 39) and support staff (48 and 21) categories, a s shown on Table 3. However, male administrators (50) outnu mbered females (10) 5 to 1, while there were no female superintendents and 22 male superintendents who reported their gender. There were more than twice as many male board members (65) as femal es (30). In the other category, 18 were males and 24 were females. Position with age The majority of the teachers (93) were in the 30 -55 years age group, with 18 falling in the 18-29 category, 10 in the 56-70 d ivision, and one reporting being under 18. Most of the support staff (56) reported being 30-55, with 9 in the 56-70 age group, 4 in the 30-55 range, and one under 18. Administrators, too, reported the maj ority (54) in the 30-55 group, with 8 in the 56-70 category, and one in the 18-29 age group. Twenty-tw o superintendents reported being in the 30-55 age range, with six in the 56-70 category, and only one in the 18-29 age group. No board members reported being under 30, while the vast majority (8 0) were in the 30-55 range, 14 in the 56-70 category, and 1 over 70. In the other group, 21 rep orted being under 18, 12 said they were 18-29, 5 filled in the 30-55 group, and 3 marked the 5670 ag e group. Position with education Most of the teachers (95) reported having B.A. d egrees, with 30 having Master's degrees, and one reporting less tha n a B.A. The majority of support staff (57) had less than a B.A. degree, with 10 having a B.A., and 4 having a master's degree. Most of the administrators (62) had a master's degree, with onl y 3 listing a B.A., and one filling in the less tha n B.A. category. Nearly two-thirds of the administrat ors (18) had master's degrees, while nine had doctorates, and two had completed their B.A.'s. Of the board members reporting, over half (55) had less than a B.A., while 32 had a B.A., 10 had a mas ter's degree, and six held doctorates. In the other group, 37 reported having less than a B.A., while t hree filled in the master's degree response. It wou ld appear that there is an inverse relationship betwee n the education and income levels of the board members. While the majority of board members did no t have a college degree, over half of them reported incomes of more than $50,000. Position with experience The results from Table 2 show that the largest n umber of respondents in every group except other had 11-25 y ears experience in the current or similar job. The second highest level in every group except administ rators and other was the 5-10 year category. The remainder of respondents were distributed fairly ev enly between the other two categories. Analysis of other demographics The ANOVA procedure revealed no signifi cant differences on the quality effectiveness index in income levels, age, or experience. However, diff erences were found for education levels. Using Scheff, significant differences were found between those with less than a B.A. and those with a B.A. degree. No other significant differences were found between the other education levels. Although the t test indicated a signifi cant difference in the quality effectiveness index between males and females, when t tests were conduc ted for gender by position, no significant differences were found in any position level for ge nder. Thus the difference was only present when all levels were considered as a whole.
17 of 43Reliability Analysis of the Instruments The SPSS program was utilized to determ ine the reliability of the Perceived Quality Assessment Instrument. Analyses were conducted for the current and ideal sections and for the overall scale (total instrument). The alpha reliab ility coefficients are reported in the Table which follows. The alpha coefficients ranged from .68 to .85 for the current section of the instrument, with an overall reliability of .96. The alpha coefficie nts were somewhat different for the ideal section, ranging from .61 to .89, with an overall reliabilit y of .94, indicating a high positive correlation am ong all items. Table 4 below presents the information and reliability coefficients for the PQAI Instrumen t. Table 4. Reliability analysis of current and idea l sections of the PQAI Reliability PQAI DimensionItem NumbersCurrentNIdealN Leadership1 6.84454.82451Information and Analysis7 12.85443.82430Strategic Quality Planning13 18.84451.87445H.R. Development & Mgmt. 19 25.83451.61441 Mgmt. of process Quality26 31.79440.85437Quality & Operational Results 32 37.68454.75448 Client Focus & Satisfaction38 45.85453.89444 Overall1 45.96 .94 The Cronbach alpha reliability coeffic ient was also calculated for the Staff Development Questionnaire to determine the internal consistency of the total instrument. Estimates of internal consistency are based on the average correlation am ong items within a test or instrument. The reliability coefficient for all thirty items was .9 5, again a high positive correlation. This level o f reliability was determined to be more than sufficie nt for the purposes of this research. On the Compensation Assessment Instrume nt, the Cronbach alpha reliability coefficient was also calculated to determine an estimate of the int ernal consistency of the instrument. Reliability coefficients were calculated for the two sets of qu estions -one for current practices and the other for ideal (or preferred) practices in compensating supe rintendents, or school system chief executive officers. Table 5 below contains the reliability c oefficients for the four sub-areas and the total instrument (Behounek, 1996). Table 5. Cronbach Alpha reliability coefficients for sub-areas of the Compensation Assessment VariableAlphaNumber of Items Current Practice Non-performance related items.757 Performance related items.835Ideal (Preferred) Practice Non-performance related items.877
18 of 43 Performance related items.675 Total of all items.6724 The level of reliability of the Compens ation Assessment Instrument was determined to be adequate, albeit modest, for the purposes of this r esearch (Nunnally and Durham, 1975). Quality Management Perceptions The study sought to determine the perce ived current and ideal levels of quality management in each participating district and the Quality Effe ctiveness Index ratio between the two assessments of perception. Means were first calculated for the responses for each item for each district. The means of all of the current and ideal responses for each district were then figured. Current means ranged from 2.80 (district HH) to 3.92 (district MM ). Ideal means ranged from 3.77 (LL) to 4.56 (P and R). Finally, the ratio between the current and ideal means for each district were determined. The quality effectiveness index ranged from .6532 (dist rict HH) to .9506 (district A). Four districts were removed from the study because of their low return (N for districts F and V = 1 survey each; N for districts I and M = 3 surveys each). It was felt th at such a small sample from those districts would n ot give a reliable representation of the perceptions o f the entire district. The presentation in Table 6 below shows the distribution of means for current a nd ideal situations and the ratios for the remainin g 40 districts.
19 of 43Table 6. Current and ideal means and quality effec tiveness index for districts on the Perceived Quality Assessment Instrument District CodeRank Current Situation Mean Ideal Situation Mean Quality Effectiveness Index A13.603.79.9506 AA233.104.00.7747 B103.524.18.8408 BB333.444.66.7371 C73.704.32.8572 CC313.324.53.7322 D3220.127.116.1119 DD153.394.23.8005 E303.084.15.7419 EE322.974.01.7399 FF43.664.15.8859 G213.444.37.7865 GG163.304.12.8001 H143.244.04.8020 HH402.814.30.6532 II218.104.22.16800 J133.454.26.8083 JJ113.564.31.8250 K193.334.19.7943 KK382.974.30.6899 L273.264.32.7554 LL53.503.95.8843 MM63.924.46.8784 N263.434.50.7619 NN223.404.42.7703 O33.774.23.8909 OO23.714.08.9107 P392.994.56.6545 PP83.644.29.8501 Q322.214.171.12457 QQ183.584.47.7989 R293.424.58.7483 RR203.564.50.7889 S253.234.23.7628 T243.264.25.7669 U93.464.09.8480 W343.274.44.7369 X173.444.31.7990Y373.384.28.7097 Z123.654.44.8234
20 of 43Staff Development Perceptions and PQAI To determine the perceptions of teacher s in the participating school districts about their systems staff development programs, means for each question for each district were calculated. Then the mean of all the responses for each district was determined. These means ranged from 1.963 to 3.789. The same four districts (F, I, M, and V) we re removed from the list for comparisons with the group used with the PQAI. Ranks of districts on th e Staff Development Questionnaire were compared with the districts' ranks on the PQAI, as shown in Table 7 below.
21 of 43Table 7. Comparison of ranks on the Staff Developm ent Questionnaire and the Perceived Quality Assessment Instrument District CodeSDQ RankPQAI RankSub-Group A111K2191 PP381 J4131 JJ5111 LL651 QQ7181 L8271 C971 II10281 O1132Q12262 OO1322DD14152 H15142 B16102 RR17202 P18392 R19292 MM2062 N21263 BB22263 Z23163 GG24163NN25223AA26233 W27343 U2893D29363 T30243 X31174 S32254 Y33374 EE34324 E35304 CC36314 G37214 FF3844 HH39404KK40384 The forty districts, listed above, were divided into four groups for further analysis. Th e sub-groups were comprised of groups of ten district s each, according to their rank on the Staff Development Questionnaire. A one-way analysis of v ariance was calculated to determine differences
22 of 43in means of the four sub-groups. The results of th e ANOVA are shown below in Table 8 below. Table 8. One-way analysis of variance of group mea ns on the Staff Development Questionnaire SourcedfSSMSF Fcv Between groups37.532.51129.534.51Within groups36.70.02 Total398.22 p= <.0001 The four groups appeared to differ in terms of their group means on the Staff Development Questionnaire, as illustrated in the Table (Table 9 ) below. Table 9. Sub-group means and standard deviations o n the Staff Development Questionnaire GroupNMeanSt.Dev. 1103.36.21292102.93.07203102.65.08594102.17.1399 A one way analysis of variance was the n employed to determine differences in means between and among the four sub-groups. Significant differences were found among the groups, and a Scheff method was used to determine which sub-grou ps differed from one another. All groups were found to differ from each other significantly, as shown in Table 10 which follows: Table 10. Comparison of group means on the Staff D evelopment Questionnaire Sub-Group ComparisonMean DifferenceScheffe F test 1 vs. 2.436616.4*1 vs. 3.715544.05*1 vs. 41.1944122.76*2 vs. 3.27896.69*2 vs. 4.757849.42*3 vs. 4.478919.74* *Significant a t .01 level
23 of 43 PQAI and SDQ Relationships The relationship between the PQAI measu rement of perceived quality and the Staff Development Questionnaire was explored. The compos ite district ratings of staff development programs were correlated with the Quality Effective ness Index computations. A Spearman rho test was used to calculate correlations between the two indices. A correlation of .50131 was found, and it was significant at the .01 level of confidence. Th is relationship was determined to be moderate, but significant. A second exploration was conducted to d etermine if there were any relationships between ratings of perceived levels of staff development qu ality and the seven dimensions of district quality (current) measured by the PQAI in the participating districts. To accomplish this purpose, means of th e current perceptions in each of the seven quality dimensions of the Perceived Quality Assessment Inst rument for the ten highest ranking districts on the Staff Development Questionnaire were computed. The mean of each district on the Staff Development Questionnaire was compared with the mea n of each quality dimension for these districts. Stepwise multiple regression was used t o determine the relationship of the measured qualit y of staff development to the seven quality dimension s. The seven variables were entered one at a time and a significance test was conducted to determine the contribution of each (Hinkle, Wiersma, & Jurs, 1988). The stepwise solution was terminated w hen the remaining variables did not make a statistically significant contribution to the regre ssion. It was found that three of the seven di mensions--Client Focus and Satisfaction, Quality an d Operational Results, and Management of Process Qual ity--were good predictors of levels of quality staff development. The adjusted R2 (squared) was .9 56 indicating that 96 percent of the variance on the Staff Development Questionnaire was explained b y these three dimensions. The remaining four dimensions -Leadership, Information and Analysis, Strategic Quality Planning, and Human Resource Development and Management -did not pred ict levels of district staff development in any significant degree. Correlation matrices were also construc ted in order to show the interrelationships between all the sub-scales on the Perceived Quality Assessment Instrument. Table 11 shows the correlation coefficients for each of the seven sub-scales. As a rule of thumb, correlation coefficients between 00 and 30 show little if any correlation; 30 to 50, a low correlation; 50 to 70, a moderate correlation; 70 to 90, a high correlation; and .90 to 1.00, a very high correlation (Hinkle, Wiersma, & Jurs, 1988). Using these guidelines, the matrix show s that the majority of the correlations are moderat e positive correlations. None of the relationships ar e below 50, with the lowest being between Leadership and Quality and Operational Results (.52 ). Five of the relationships were in the h igh positive category: Human Resource Development and Management and Strategic Quality Planning (.77) Human Resource Development and Management of Process Quality (.73), Management of Process Quality and Quality and Operational Results (.71), Client Focus and Satisfaction and Ma nagement of Process Quality (.72), and Client Focus and Satisfaction and Quality and Operational Results (.74). These relationships show that all seven dimensions are related to each other and meas ure the same concept -district quality management. Table 11. Correlation matrix for the seven dimensio ns of the PQAI PQAI Sub-scale Leadership Infor. & Analysis Strategic Planning H.R. Dev. & Mgt. Process Quality Quality Results Client Focus Leadership 1.00 Information .631.00
24 of 43Planning .61.691.00 H.R. Mgt. .62.67.771.00 Process .126.96.36.1991.00 Results .188.8.131.52.711.00 Focus .184.108.40.206.72.741.00 Still another exploration was conducted into the area of determining whether or not distri cts ranked highest or lowest in perceived staff develop ment quality differed significantly when measured by the perceived quality effectiveness index. Table 12 shows these highest and lowest districts, their ranks on both instruments and the ir mean scores for each. Table 12. Mean scores and ranks for the highest an d lowest ten districts on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument Table 12. Mean scores and ranks for the highest an d lowest ten districts on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument DistrictRank on SDQGroupSDQ MeanPQAI MeanRank on PQ AI A113.789.95061K213.567.794319 PP314.442.85018 J413.427.808313 JJ513.408.825011 LL613.278.88435 CD713.278.798918 L813.273.755427 C913.113.85727 II1013.067.750028 X3142.408.799017 S3242.310.762825 Y3342/280.709737 EE3442.211.739932 E3542.203.741930 CC3642.167.732231 G3742.150.786521 P3842.042.88594 HH3941.994.653240KK4041.963.689938 Using the unpaired t test on the means of the two groups on both instruments, significant differences were found for the highest and lowest g roups on the Perceived Quality Assessment Instrument. A correlation coefficient (Pearson pro duct-moment) was then calculated among groups compared on both instruments. A moderate positive c orrelation (.564) was found for the top ten districts which was significant at the .05 level. T he correlation between the bottom ten districts on both instruments (.264) was not significant. Howeve r, when district FF was removed which ranked
25 of 43high (number 4) on the Perceived Quality Assessment Instrument, the correlation rose to .710, a high positive correlation. It was determined that the h igher a system demonstrated perceived quality in staff development, the higher its perceived quality effectiveness index. Analysis of Executive Compensation Perceptions and PQAI The compensation of executives and lead ers has often been thought to be instrumental in influencing the level of performance on the job. In short, some believe that money can motivate (Poston and Frase, 1992). Deming, long considered the "father of quality management," however, felt that the theory of psychology eroded any confi dence that money could motivate individuals on the job (Deming, 1986). His notion was that money and employee compensation had little to do with the quality of an organization. To test that theory in part, this study undertook to examine the perceptions of school boa rd members and superintendents about the effect or inf luence of salary compensation upon the quality of an organization. Perceptions from board members an d superintendents were elicited with the instrument earlier described. The following Table demonstrates the tabulated results of the perceptual assessment.
26 of 43Table 13. Aggregate responses of superintendents a nd board presidents concerning practices and preferences for salary increases Board PresidentsSuperintendents ItemBelief statement N MeanSt.Dev. N MeanSt.Dev. CURRENT: Unrelated to Performance 1Length of service to system442.67.94442.68.932Increased formal education442.75.97442.27.953Cost of living adjustment442.861.00442.481.094Growth in revenues443.461.15443.501.075Settlement of teacher contract433.561.22443.801.176Regional colleagues status443.39.99443.481.077Student enrollment change441.861.02442.161.16 Total 442.941.17442.911.25 CURRENT: Related to Performance 8Performance on job443.961.16443.73.859Achievement of set goals443.611.13443.11.99 10School Board evaluation443.771.05443.61.9911Student achievement442.771.14442.021.0212Test score results442.051.12441.73.90 Total 443.231.32442.841.25 IDEAL: Unrelated to Performance 1Length of service to system442.551.02442.971.052Increased formal education443.41.79443.66.783Cost of living adjustment443.41.79443.411.004Growth in revenues443.021.00443.48.955Settlement of teacher contract443.461.07443.211.106Regional colleagues status443.071.13444.18.627Student enrollment chnage442.46.79442.59.87 Total 443.071.04443.361.03 IDEAL: Related to Performance 8Performance on job444.36.72444.59.509Achievement of set goals444.18.79444.32.86 10School Board evaluation444.11.75444.43.5011Student achievement443.21.85442.841.0812Test score results442.55.82442.02.88 Total 443.681.05443.641.29 For the purposes of this study, the str ength of agreement indicated by each level of respo nse was categorized as low if less than 2.25, moderate if between 2.26 and 3.75, and high if more than 3.76. These designations were selected arbitrarily for purposes of comparison only. The results in Table 13 represent superintendent and board preside nt perceptions on how increases for superintendents are awarded in current practice and how they should be awarded given conditions of ideal practice. Board presidents and superintendents ag reed that increases in enrollment have little to do with compensation, but levels of compensation awarded pu rsuant to teacher contract deliberations have a
27 of 43high impact upon the level of superintendent's comp ensation. In the area of performance-related factors, the highest factor of influence to both bo ard presidents and superintendents was the superintendent's performance on the job. The lowes t relationship to compensation was perceived to be with standardized test results. Ironically, thi s measure of organizational effectiveness had low agreement as to its importance in determining compe nsation. When considering the ideal situation, b oard presidents and superintendents agreed that on-the-job performance, achievement of district goa ls, and board evaluation of the superintendent should have the greatest level of influence on the superintendent's compensation. Ideally, as in the current status, student test performance was percei ved of low importance. In non-performance rated ideal conditions, superintendents favored most stro ngly the use of "benchmarking" salaries of peers or colleagues in the region for use in determining the salary of the superintendent. Changes in enrollment fared no better in perceptions of ideal factors than with current conditions, as it was rat ed as the lowest in agreement for both groups. To more adequately compare board presid ents' and superintendents' perceptions about compensation, the mean scores of both groups were s ubjected to analysis. Unpaired, two-tailed t-tests were used to determine if significant diffe rences existed between the two groups on the four areas of the study. Responses from superintendents and board presidents were also compared between performance and non-performance factors and current practice and ideal situation. In these cases, paired, two-tailed t-tests were utilized. S ignificance was set at the .05 level of confidence for the purpose of this analysis. Table 14. t-Test results of group means for determ ining superintendent compensation GroupNSt. Dev.Mean t P Current: Unrelated to Performance Board Presidents44.502.93.18.857 Superintendents44.712.91Current: Related to Performance Board Presidents44.903.232.2.030 Superintendents44.762.84Ideal: Unrelated to Performance Board Presidents44.493.07-2.70.008 Superintendents44.513.36Ideal: Related to Performance Board Presidents44.4220.127.116.110 Superintendents44.503.64 In the area of perceptions of board pre sidents and superintendents considering the importa nce of non-performance related job factors, the analysi s revealed that there was no significant difference between the two groups. On the other hand, when lo oking at how they two group means compare on performance-related job factors, there was a signif icant difference between board presidents and superintendents in their perceptions. Board presid ents felt more strongly about using performance in the determination of compensation for the chief exe cutive. Two more tests of significant differenc es were employed, dealing with perceptions in the i deal or preferred situation. When looking at the import ance of non-performance ideal factors, board presidents placed less credence on these factors th an did superintendents. The difference was
28 of 43significant, indicating that superintendents placed more importance on non-performance job factors in determining superintendent compensation than did board members in an ideal situation. In the ideal situation using performanc e-related compensation factors, superintendents and board members were not significantly in disagreemen t. Both groups rated the use of performance factors in determining the level of job compensatio n in the "high-moderate" range. To compare performance with non-perform ance factors across groups, a series of analyses employing t-tests were conducted. Superintendents' and board presidents' perceptions on performance against nonperformance factors in curre nt practice and performance against nonperformance factors in ideal practice were analy zed. Comparisons of superintendent and board president perceptions of current practices against ideal practices across performance and non-performance areas were also examined. Results of these analyses are displayed in Table 15 which follows. Table 15. T-Test results between groups on perform ance and non-performance factors in determining superintendent compensation GroupNSt. Dev.Mean t P Current: Superintendents Non-performance factors44.718.104.22.16836 Performance factors44.762.84Current: Superintendents Non-performance factors44.503.36-2.46.0180 Performance factors44.513.64Ideal: Board Presidents Non-performance factors44.582.93-2.22.0317 Performance factors44.903.23Ideal: Board Presidents Non-performance factors44.493.07-6.05.0001 Performance factors44.463.68 Given these results, it was determined that superintendents didn't differ in rating the re lative importance of current practice performance-based co mpensation when compared to non-performance based compensation factors for superintendents. Ho wever, in the ideal situation, superintendents did rate performance-based compensation factors higher than non-performance based factors with a higher aggregate response. Tests of significance were also applied to aggregate responses of board presidents. Board presidents differed somewhat by placing more import ance on performance-based factors than non-performance based factors in both the current s ituation and in the ideal situation. The ideal difference was dramatic and strong in favor of usin g performance-based factors given the opportunity in an ideal or preferred situation. Other tests involved comparing aggregat e means for superintendents and board presidents between current practice and ideal situations in no n-performance and performance-based areas. The results of these analyses indicated significant dif ferences between current practice and ideal situati ons on non-performance and performance factors for supe rintendents, but not for board presidents (Behounek, 1996). Another analytical exercise was underta ken to determine if the perceptions about performance-based compensation had any relationship to the perceived quality assessment instrument data. Superintendents perceptions on performance-b ased compensation were categorized in three
29 of 43groups low support, moderate support, and high su pport relative to strength. These groups then were analyzed with ANOVA to determine if there is a difference related to the superintendents' perceptions. Low support was defined as a mean sco re of less than 3.0, moderate included means between 3.1 and 4.25, and high included mean scores greater than 4.26. The categories were selected for purposes of comparison, and a confidence level of .05 was used. The results of the analysis are presented in the following Table. Table 16. ANOVA summary table comparing Quality Ef fectiveness Index rating of school systems by perceived support of performance-based c ompensation by superintendents SourcedfSS X2F ratio FP Between groups 22.214.171.124.303 Within groups 413.43.08 Total 433.64 Group NX St. Dev. Low support for performance compensation63.42.22Moderate support for performance compensation353.35 .30 High support for performance compensation33.10.34 Total 443.34.29 Statistically, there is no significant difference in the rated quality of the school syste ms based upon the level of the superintendent's support for performance-based compensation. The results, given full awareness of the small sample size, do d isplay some interesting configurations, and a trend is somewhat evident. The group of superintendents with the highest mean response indicating high support for performance-based compensation, served in districts with the lowest mean quality index (3.10). The group of superintendents with the lowe st mean response indicating low support for performance-based compensation, served in districts with the highest mean quality effectiveness index (3.42). As school systems are categorized by their superintendent's support of performance-based compensation, the aggregate mean scores of the quality of the districts increase as the support decreases. To test the significance of this percei ved relationship, a correlation (Pearson product-mo ment) was calculated to determine if the strength of the superintendents' support of performance-based compensation was related to the ratings of perceive d system quality reflected in the aggregate quality effectiveness index ratio. The results of the anal ysis are shown in Table 17 below. Table 17. Correlation of school system quality eff ectiveness index by level of superintendents' support for performance-b ased compensation N Covariance R R2 44-.03-.18.03 The results in Table 17 reflect a sligh t negative correlation between these two variables. A negative correlation of -.18 is determined to be "l ittle if any" (Hinkle, 1988). Weak as it is, the relationship between perceived organizational quali ty and support for performance-based compensation places imperfect credibility on Deming 's theory that performance pay is detrimental to an organization (Deming, 1986).
30 of 43 Analysis of Financial Factors and PQAI The impact of financial soundness upon organizational quality has been demonstrated in a number of ways in the private sector, but has not b een demonstrated unequivocally in public education (Kirchoff, 1996). Despite a paucity of r esearch demonstrating the influence of financial wealth upon educational institutional quality, ther e has been a belief that financial stability of a system may contribute to collaboration efficacy and to constancy of purpose (Schmoker and Wilson, 1993). Several financial characteristics of ed ucational institutions were identified and assemble d and considered in relationship to the quality effective ness index generated by the Perceived Quality Assessment Instrument. The factors included the fi nancial solvency ratio (FSR), revenues per pupil (R/P), expenditures per pupil E/P), undesignated un reserved fund balance per pupil (UUFB/P), unspent balance per pupil (UB/P), and the transport ation cost per pupil (TC/P). These six factors were examined in two ways, first with a correlation al matrix, using Pearson product-moment techniques, and then with the t-test (two-tailed) b etween the high and low financial factors compared to the PQAI score. In other words, the most financ ially sound districts and the least financially sou nd districts were compared as to the level of PQAI sco re, or perceived quality effectiveness index. To obtain the highest and lowest catego ries, the sample group of school systems was divide d in to three equal groups of fourteen school distric ts each (this analysis involved the use of 42 of th e 44 school systems) based upon the districts perceiv ed quality effectiveness index. The index is the ratio of the perceived current mean score on the PQ AI to the perceived ideal mean score on the PQAI for the school district. The means and standard de viations for the sample group, the high PQAI group, and the low PQAI group are shown in the Tabl e below. Table 18. Statistical data for the PQAI sample, hi gh, and low groups Group N MeanSt. Dev. Sample Total PQAI42.796.080High PQAI Group14.885.044Low PQAI Group14.716.047 Correlations were calculated for each o f the financial characteristics between the high qu ality effectiveness index group and the low quality effec tiveness index group. The results of those calculations are illustrated in Table 19 below. Table 19. Correlations between financial characteristics and PQAI Financial characteristics PQAI FactorFSRR/PE/PUUFB/PUB/PTC/P Sample Group Total-0.0110.1030.113-0.0090.0990.226Leadership-0.0830.1440.189-0.077-0.0280.115Inf. and analysis-0.0370.0950.095-0.0310.1110.326*Strategic planning0.017-0.002-0.0070.0060.1240.079HRD and mgmt.-0.1040.2400.269-0.0600.1930.238
31 of 43Mgmt. of process quality-0.0910.0690.056-0.0920.086 0.177 Quality and op. results0.0240.0970.1060.0290.0610.2 16 Client focus and satisfaction0.0120.1200.1280.1120. 1120.259 p <.01 A statistically significant relationsh ip was shown between the transportation cost per pu pil and the PQAI of the sample in the information and a nalysis category of the perceived quality assessment instrument, however, the variance (r2) w as calculated to be only 0.106. This means that only 10.6% of the variance in the transportation co st per pupil was attributable to the PQAI and that almost 90% of the variance was attributable to othe r factors. T -tests were also calculated for each of the financi al factors comparing the high Quality Effectiveness Index group with the low Quality Effe ctiveness Index group to determine if the differences between the groups on any financial fac tors were significant. The results of this analysi s is shown in the table below. Table 20. T tests between the high and low PQAI groups and fin ancial characteristics GroupPQAIFSRR/PE/PUUFB/PUB/PTC/P Sample Group Total9.706*-0.4410.3520.379-0.4400.411 1.016 Leadership8.878*-0.0922.336*0.7600.0630.763-0.014Inf. and analysis10.665*-0.5510.3070.171-0.4860.185 2.250* Strategic planning7.976*-0.262-0.363-0.527-0.3091.0 440.000 HRD and mgmt.6.136*-0.2161.7581.581-0.0420.3781.493Mgmt. of process quality10.367*-0.3530.2210.269-0.5 130.5680.572 Quality and op. results8.146*0.2180.9011.0950.319-0 .1301.751 Client focus and satisfaction11.159*0.6611.0771.124 0.7460.8971.728 p <.0001 Significant differences were found bet ween the high and low group PQAI means, for the revenues per pupil in the leadership category, and the transportation cost per pupil in the informatio n and analysis category. One of the significant t-te st results was in the area of leadership and revenu es per pupil. Another was in the area of transportati on costs per pupil and the information and analysis sub-scale. While statistical significance was foun d between two of the financial characteristics, forty-six differences were not found to be signific ant. Analysis of the Perceived Quality Assessment Instru ment The study also sought to determine if t he PQAI items on the perception of quality scales o f current and ideal status would align consistently w ith the seven a-priori determined dimensions based on the PQAI (Baldrige) criteria. A factor analysis was used based on a varimax rotation technique. The results of the analysis revealed three possible factors with Eigenvalues greater than 1.0 on both the current and ideal scales. The three factors accounted for 42.3% a nd 43.3% of the total variance on the current and i deal scales respectively. The distribution of the number of items from each PQAI category among the empirical factors suggested is shown in Table 21. For example, the six a-priori items for the Leadership category (current responses ) were all c ontained in factor 3; however the eight items for
32 of 43the Customer Focus and Satisfaction were distribute d among factors 2, 3, 6, and 7. Table 21. Comparison of the current and ideal a-pri ori PQAI dimensions with empirical factors ScaleItems Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 Factor 7 Factor 8 Eigenvalue 16.281.501.27.99.79.73.59.49 CurrentLeadership1-66Information & Analysis 7-1215 Strategic Quality Planning 13-1835 H.R. Devel. & Mgt. 19-2552 Mgt. of Process Quality 26-3123 Quality & Op. Results 32-375 Customer Focus38-453113 Eigenvalue 16.331.9126.96.36.199.56 IdealLeadership1-66Information & Analysis 7-12141 Strategic Quality Planning 13-186 H.R. Devel. & Mgt. 19-2516 Mgt. of Process Quality 26-3151 Quality & Op. Results 32-375 1 Customer Focus38-457 The results, as shown in the Table abo ve, suggest that the seven original a-priori dimens ions may be reduced to three factors on both scales. Ho wever, the majority of the items in the Information and Analysis dimension load on factor 4 on the curr ent scale and factor 5 on the ideal scale and the majority of items in the Human Resource Development and Management load on factor 4 in the ideal scale, indicating the importance of these additiona l factors. On the ideal scale, three dimensions (M anagement of Process Quality, Quality and Operational Results, and Customer Focus and Satisfa ction) seem to load on one factor (factor 1) which suggests that respondents were not able to di fferentiate among these three dimensions as originally conceptualized or that the three were pe rceived as measuring the same thing. The factor analysis results show the items grouped differently than on the original instrument showing that the instrument may be measuring different factors than originally conceived and the items may need to
33 of 43be regrouped. However, when combined with the resul ts of the correlation matrix, it appears that, overall, the instrument is measuring one underlying concept on both scales. Summary and Conclusions This paper presents the findings of stu dies of public schools in Iowa in the area of perce ived quality assessment conducted at Iowa State Universi ty. Demographic characteristics of the respondents on the Perceived Quality Assessment Ins trument from forty-four school districts were described by position, home annual income, gender, age, level of education, and years experience in current or a similar job. The research project undertaken incorpo rated several studies of quality improvement characteristics of public schools. The project res ulted in a compendium of coordinated research aimed at learning more about the relationships and effects of quality improvement efforts with other factors of school district operations. Several dissertations resulted from the studies, including writings by Rashid M. Bax, Pame la D. Johnson, Thomas J. Behounek, and Joseph E. Kirch off, over a three-year period. All of the dissertations were supervised by William K. Poston Jr. at Iowa State University during the period 1993-1995. Each of the study components (parts of the total study) gathered specific data relevant to the issues of the individual study component. The individual study components (doctoral dissertations) focused on the following issues: Assessment of quality improvement climate in commun ity colleges (Bax, 1994). Teachers' perceptions of training programs and thei r relationships to total district perceptions of quality management (Johnson, 1995). Performance-based pay of chief executive officers a nd effects upon quality improvement processes in school organizations (Behounek, 1996). Financial characteristics of school organizations a nd relationships to quality management factors (Kirchoff, 1996). Continuous quality improvement is a rec ent phenomenon in educational institutions, but it has been popular in business and industry since the lat e 1970's in the United States. The management philosophy inherent in continuous quality improveme nt has been largely based upon the ideas of W. Edwards Deming. The incorporation of Deming's idea s into educational organizational thinking has been gradual and focused on application of the qual ity concepts into everyday practice. Given the goals of public education, questions arise as to wh ether or not the use of continuous quality improvement principles have relevance in the public school organizational environment. Moreover, it is even more important to know what relationship s and effects result from the application of qualit y improvement strategies in the public school setting This series of studies conducted over a period beginning in 1993 and culminating in 1996, was designed to assess the perceptions of school di strict stakeholders about the quality of their scho ol district in terms of the seven quality dimensions o f the Baldrige Award criteria. The Baldrige criter ia were used in the development of an instrument that was employed in evaluating the levels of quality in public school organizations. This version of th e Baldrige instrumentation was developed and modified for use in the studies (Poston and Bax, 19 94). The instrument, in two versions, was administered to 44 independent local school systems and to 15 community college in the state of Iowa. and focused on the following Baldrige Award a reas: 1. Leadership 2. Information and Analysis 3. Strategic Quality Planning 4. Human Resource Development and Management 5. Management of Process Quality
34 of 436. Quality and Operational Results 7. Client Focus and Satisfaction The criteria for the instrument were mo dified to fit the public school environment, and th e instrument contained 45 items (see Appendix). Vali dity and reliability of the instrument was established using statistical inquiry, panel review and research validation procedures. The instrument was found to be both valid and reliable for use in educational organizations and institutions. Respondents were instructed to indic ate the current state of affairs on each item of quality, as well as the ideal, or preferred, state of affairs for each item. Two other instruments were developed fo r use as a part of the studies. Both instruments w ere developed based upon information from professional literature, and verified by a panel of reviewers. In the study component by Pamela Johnson, the Staff Development Questionnaire was developed and employed, and the Executive Compensation Questionna ire was developed and used in the study component managed by Thomas Behounek. Joseph Kirch off's study employed the use of publicly-documented data available from state gover nment agencies. Study Procedures In the comprehensive studies, it was im portant to adequately select a population of school districts reflective of the State of Iowa. The pop ulation consisted of forty-four (44) school distric ts selected, invited, and willing to participate in th e studies voluntarily. The districts represented m ore than ten percent (10%) of the districts in the Stat e of Iowa, and they were generally reflective of th e state in terms of size, economic conditions, and ge ographic distribution. The chief executive officer s of each school system were requested to assign rand om distribution of the instrumentation to the stakeholders in their districts to a third party, a nd respondents were asked to complete all instrumen ts, following the directions provided. Those eligible for completing the Perce ived Quality Assessment Instrument were the superintendents, all board members, two members of the administrative staff, three members of the support staff, five teachers, and two high school s tudents. The comprised a total of 18-20 persons from each school district, depending upon the size of the board (five to seven members). All responses to all instruments were anonymous and vol untary. The total possible number of questionnaires to be returned amounted to 720 each, of which 471 instruments were received. This amounted to a return rate of approximately 65 perce nt. Four districts were eliminated from the final analysis of data due to insufficient returns from t hose districts. The Staff Development Questionnaire was planned to be completed by six teachers in each of the forty-four districts, and they were not allowed to be the same teachers used in the PQAI instrumentation response. Of a possible 264 instru ments, 196 were returned for a response rate of approximately 80 percent. The Executive Compensation Questionnair e was administered to all superintendents and board presidents of the 44 school districts. Remar kably, all 44 superintendents and all 44 board presidents responded by completing and returning th e questionnaire. The financial information used in the s tudies was obtained from officially-filed audit rep orts from each school district from the offices of the I owa State Department of Education. All data were gathered by June, 1994, a nd all responses were entered into a spreadsheet program for analysis and computation of means for c urrent and ideal status. The quality effectiveness index was also computed for each district on each i tem, sub-area, and for the total instrument. District means were also computed for all instrumen t data from the Staff Development Questionnaires and the Executive Compensation Quest ionnaire. Districts were then rank-ordered from high to low, and different groupings were used for data analysis and comparisons. Findings
35 of 43 Of those responding on the PQAI, superi ntendents (30 each) comprised 6.6% of the sample group, teachers (132) comprised 28.8%, support staf f (76) comprised 16.6%,, board members (108) made up 23.6%, and students (44) comprised 9.6%. D emographic data were also gathered on each of the responding individuals, including occupational positions, income, gender, age, levels of education, and work experience of individual respon dents. Differences were noted between male and female respondents, which was significant overall, but not in any of the separate sub-areas of the ins trument when broken down by gender. Teachers were noted to differ from board members, and differ ences between administrators and teachers were also noted on the PQAI. Respondents with a baccala ureate degree also differed from those without the degree. No significant differences were noted in the demographic categories of age, income levels, or job experience.PQAI Findings The perceived current level of quality and the ideal level of quality were calculated, and the means for the current status ranging from 2.81 to 3 .92. Ideal means ranged from 3.79 to 4.66. Quality effectiveness indices were calculated, and ranged from .6532 to .9506. Two districts had a QEI above .90, and three had a QEI below .70. The majority (28 or 70%) of districts had a QEI above .75 indicating that those districts perceive that they are achieving at least 75% of their quali ty management goals. Interrelationships among the seven subareas of the PQAI were also found using a matrix of relationships, with moderate to high correlations a mong the factors. A correlation of .77 was found between human resource development and management a nd strategic quality planning, .74 between client focus and satisfaction and quality and opera tional results, .73 for human resource development and management and management of process quality, a nd .71 between management of process quality and operational results. A determination was also sought to esta blish if the PQAI differentiated in terms of qualit y between high ranking and low ranking school distric ts. Inferential statistics were used to assess the difference and established a significant difference between the ten highest rated and the ten lowest rated groups. There was not only a significant dif ference between the high and low groups' quality effectiveness index, but there were also significan t differences between the groups in each of the seven dimensions or sub-areas of the PQAI instrumen t. Finally, the Perceived Quality Assessment Instrument was evaluated utilizing factor-analysis techniques, and results indicated that some items may need to be regrouped into three categories or s ub-areas empirically developed but undefined. Staff Development Findings A significant positive relationship was found between the perceived quality of district st aff development and the perceived quality effectiveness index of the districts. District means on the Staff Development Questionnaire were used to catego rize the districts into four equally-sized groups according to their ranks, and there were significan t differences between all four groups in their rankings of staff development practices. In other words, significant differences were found between the districts ranked highest and lowest in perceive d staff development quality on the quality effectiveness index. The top ten districts demonst rated a modest correlation of .564 between the means of the QEI and the SDQ, indicating that distr icts which have higher quality staff development programs also have higher quality overall district management as measured by the Quality Effectiveness Index slightly more than half of the time. Significant relationships were also fou nd between ratings of perceived levels of district staff development and current ratings on three of the sev en dimensions of the Perceived Quality Assessment Instrument. Client focus and satisfacti on, quality and operational results, and management of process quality were found to correla te positively and have predictive value with perceptions of quality in staff development. The o ther four areas -leadership, information and analysis, strategic quality planning, and human res ource development and management -showed limited correlative or predictive value.
36 of 43Executive Compensation Findings Board presidents and superintendents sh owed agreement that the student enrollment levels should have little effect in determining compensati on levels for the superintendent, and also agreed that standardized testing should not be used for su ch purposes. That teacher union increases should be considered in any compensation increases for sup erintendents was also an area of agreement, as was the area calling for use of job description fac tors in compensation decisions. The differences between performance-rel ated and situational-related (nonperformance) factors were evidenced between board presidents and superintendents with the former more supportive of using performance-based criteria than the latter. However in an ideal setting, superintendents were more inclined to consider perf ormance-based criteria more important than currently. Performance-based compensation initiati ves would find little or no support for such practices from these studies. Ironically, the use of feedback and assessment information for improvement efforts appears to be contradicted to a degree in the perceptions of key organizational leaders -superintendents and board presidents. A very weak finding was that there may be some credence to W. Edwards Deming's proscription of performance-based (merit) pay, in t hat a weak inverse relationship was found between performance-based criteria compensation support and the perceived quality of the systems. Deming's notion that performance-based pay might have a dele terious effect upon organizational quality was confirmed in this study, albeit tenuously.Financial Factors Findings Five selected financial characteristics and the financial solvency ratio were studied in relationship to perceived quality of the schools in the study. The five factors included revenues per pupil, expenditures per pupil, undesignated, unrese rved fund balance per pupil, unspent balance per pupil, and transportation cost per pupil. Means of the financial characteristics for the study population were calculated, and statistical analyse s were completed. Significant differences, using t-tests, were found between the high and low QEI groups in two areas -revenues per pupil in the leadership sub-a rea, and transportation cost per pupil in the information and analysis category. The other 46 po ssibilities were not found to be significant. The perceived leadership quality increases slightly as the amount of revenue per pupil increases, and vice-versa for transportation cost and perceived qu ality in information and analysis. No correlation was found between the sample schools financial char acteristics and their any PQAI rating area with one exception -transportation cost per pupil and information and analysis. Conclusions and Recommendations The comprehensive studies of quality im provement perceptions and other selected factors in Iowa schools revealed several encouraging results a nd disappointments simultaneously. The Perceived Quality Assessment Instrument appeared to function relatively well as evidenced by its significant discrimination between high and low rat ed systems. It also discriminated between high and low perceived quality school districts on the s ub-areas of the instrument. Reliability and validi ty of the instrumentation was well within tolerable li mits, and the ease of administration of the instrument contributed to a reasonable return of in struments. The instrument actually seems to be mea suring three empirically-determined factors, but th ese remain undefined pending further research. The str ength of the instrument's utility lies in its capability to accurately measure a school district' s quality improvement status against a definable, solid standard, but currently, the instrument is in need of further refinement. The quality effectiveness index or ratio provides a convenient, easily understood criterion for comparing district s against the fixed standard, and when used, it does reveal to the organization its level of quality attainment against its own definition of ideal quality. The efficacy of staff development progr ams in relationship to organizational quality was supported in the results of these studies, although the relationship between staff development and
37 of 43overall quality was not as strong as might be expec ted. However, effective staff development calls for resources, time, and strong leadership, and tea chers' ratings may not be aware of the demands upon those things which require choices and limitat ions. An encouraging finding was the relation ship of staff development quality and the perceived organizational quality levels of the PQAI. It woul d stand to reason that districts that make strong commitments to staff development would also make co mmitments to other areas of quality improvement as well. In addition, training of staf f should have some beneficial effect upon effective practice and organizational functioning. Excellenc e in training reasonably should be expected to produce excellence in organizational behavior. Individual analyses of all items in the seven dimensions of the PQAI show the existence of moderate to strong relationships among the group, i ndicating that they are dealing with factors that are connected. The means of the ten districts rank ing highest and lowest in staff development quality were compared with their means on the Quality Effec tiveness Index, and the top ten were significantly different than the lowest ten, sugges ting that although quality staff development is one of the factors important in the implementation of q uality management. Another important outcome of these stud ies was found in both the executive compensation and financial factors components -that some thing s are not related to quality. The lack of strong confirmation that performance-based pay is related to or contributory to some level of difference is a n important finding in light of continuing efforts to tie job compensation to individual performance. Such a reward-penalty approach was not supported in these studies, but was not refuted all together. The notion of performance-based or merit pay system s will no doubt continue to be popular in certain political arenas, but it did not gain vindication i n these studies. Many educators have sought to establish a clear, straight line relationship between financ ial wealth of schools and the level of quality within t he system. Again, little support was brought about in these studies for such an inclination. Although a couple of relationships were found between financial characteristics and perceived quality in these studies, but the absence of such verification encourages researchers and educational practitioner s to seek contributory factors in other directions. Recommendations Based on the findings and conclusions o f these studies, a number of recommendations emerge. The Perceived Quality Assessment Instrumen t appears to have considerable merit as a tool in determining the levels of quality in school syst ems, and as a tool, it might be effectively used as a team informational device in planning and developin g improvement efforts. The seven dimensions provide a framework for improvement actions at all levels in school organizations. There should be no hesitation to proceed with continued development research, and refinement of the perceived quality assessment process. Greater use of the instrument in resear ch activities could fashion improvement in the instrument's quality, given feedback and revision o f items and organization over time. The three empirically-derived measurement areas provide an op portunity for other developmental activity. In addition, there is considerable room for use of the instrumentation in other studies an d research efforts. The PQAI and similar tools could focus on the improvement of practices and programs in schools by making results-oriented info rmation available. The common understanding of quality is yet to be defined in educational orga nizations, and any instrument like the PQAI should be considered for use in improving school performan ce, planning, training, and institutional assessment. Without better understandings of these and other factors, improved productivity in school organizations rests upon a better grasp of q uality than currently exists. Quality improvement work should also in volve the use of growth-monitoring strategies. Using the PQAI as a "baseline" establishing device after further refinement and development, would enable districts to measure current status, establi sh directions for improvement, and monitor progress toward clearly defined ends and goals. Trend data, establishment of performance levels, and benchmarking information can all be derived from us e of the PQAI, and more effective evaluation of
38 of 43system progress becomes attainable.Closing Comments The purpose of these studies centered o n the determination of whether or not the Perceived Quality Assessment was effective in measuring facto rs commonly associated with excellence, efficacy, and effectiveness in school organizations A second purpose was to identify the extent of any relationships and the nature of any association s among variables measured by the PQAI and other measures, and the final purpose was to find whethe r or not operations and management actions in schools have an impact or influence upon the percei ved level of quality within the system. Given the findings and outcomes of thes e studies, the studies' purposes have been achieved However, that is only part of the voyage. Organiza tional quality is a continual process, not a destination, and enduring perseverance toward findi ng new and better ways to improve overall school effectiveness, use of resources and capabilities, a nd the overall development and well-being of clientele is essential for human progress. The jou rney must go on. References Bates, M. and Stachowski, E. (1991). Planning comp rehensive staff development programs. San Francisco, CA: Association for Supervision and Curr iculum Development. Bax, R. (1994). An assessment of the quality impr ovement climate as perceived by community college leadership in Iowa. Doctoral dissertation, Iowa State University, Ames, Iowa. Bax, R. M. (1994). Relationships between quality r atings, job satisfaction, and organizational commitment in school districts implementing total q uality management. (Unpublished paper, Iowa State University, Ames, Iowa, 2 May 1994).Behounek, T. J. (1996). A study of the perceived quality factors and methods of awarding salary increases for superintendents in selected school districts. Doctoral dissertation, Iowa State University, Ames, Iowa.Borg, W.R. and Gail, M.D. (1989) Educational resear ch: an introduction. New York: Longman. Bradley, L.H. (1993). Total quality management for schools. Lancaster, Pennsylvania: Technomic Publishing Co. Brown, E.H. (1992). On Deming an d school quality: a conversation with Enid Brown (in Brandt, R., 1992). Educational Leadersh ip, 50(3), 28-31. Deming, W.E. (1986). Out of the crisis. Cambridge MA: Massachusetts Institute of Technology Center for Advanced Engineering Study.Deming, W.E. (1990). Quoted in Walton, M. Deming Management At Work. New York, NY: Putnam. P. 11.Drystad, J. (1994) Quoted in Rubach, L. Transform ing U.S. education with the quality movement. Quality Progress. 27(9), 93-95.Frase, L.E., English, F.W., and Poston, W.K. (1995 ). The curriculum management audit: improving school quality. Lancaster, PA: Technomic Publis hers. Fullan, M.G. (1991). The new meaning of education al change. New York: Teachers College Press.
39 of 43Graff, L. (1995). Committing to growth through a l earning organization. (Paper published on the Horizon "Listserv" online newsletter, 15 November 1995). Horizon@gibbs.oit.unc.edu. Guskey, T.R. and Sparks, D. (1991). What to consid er when evaluating staff development. Educational Leadership, 47(3), 73-76.Guskey, T.R. (1994). The most significant advances in the field of staff development over the last twenty-five years. Journal of Staff Development, 15(4), 5-6. Hinkle, D.E., Wiersma, W., and Jurs, S.G. (1988). Applied statistics for the behavioral sciences. (2n d Ed.) Boston: Houghton Mifflin.Johnson, P. D. (1995). An examination of teachers' perceptions of district staff development programs and their relationship to perceptions of district quality management. Doctoral dissertatio n, Iowa State University, Ames, Iowa.Joyce, B. & Showers, B. (1988). Student achievemen t through staff development. New York: Longman.Kirchoff, J. E. (1996). A study of quality improve ment and its relationship to financial characterist ics of selected Iowa public school districts. (Docto ral dissertation, Iowa State University, Ames. Iowa 1996).Knudson, B. (1993). Continuous improvement profil e. Minneapolis, MN: Positive Directions, Inc. Lezotte, L. (1994). The nexus of instructional lea dership and effective schools. The School Administrator, 51 (6), 20-23.Locke, E.A. (1976). The nature and cause of job sa tisfaction. (in Dunnette, M.D., ed.). Handbook of Industrial and Organizational Psychology. Chicag o, Illinois: Rand McNally. Marchese, T. (1993). A time for ideas. Change, Ma y-June, 1993, P. 11. McBride, R., Reed, J., and Dollar, J. (1994). Teac her attitudes toward staff development: a symbolic relationship at best. Journal of Staff Developme nt, 15(2), 36-41. Nunnally and Durham, 1975 (see Behounek)Orlich, D.C. (1983). Some consideration for effect ive inservice education. The Clearing House, 56(5), 197-201.Porter, L., Steers, R. Mowday, R., and Boulian, P. (1974). Organizational commitment, job satisfaction, and turnover among psychiatric tech nicians. Journal of Applied Psychology, 59, 603-609.Poston, W.K., Jr. and Frase, L.E. (1991). Local teacher alternative compensation programs: rolling boulders up the mountain of reform. The Phi Del ta KAPPAN, 17(4) 317-320. Poston, W. K. Jr. (1995). Making Governance Work: Total Quality Education for School Boards. Newbury Park, California: Corwin Press. Poston, W K. Jr. (1995). Studies of the aspects of the use of the Malcolm Baldrige national quality award cr iteria in the assessment of educational organizations. (Paper presented to the National Conference on Education at the American Association of School Administrators, New Orleans, Louisiana, February, 1995). Poston, W.K. Jr. and Bax, Rashid, PQAI
40 of 43Rebore, R. (1991). Personnel administration in educ ation: a management approach. Englewood Cliffs, NJ: Prentice Hall.Rowe, J. (1994). Outmoded leadership ideas won't work in (the) 21st Century. (Article in the Des Moines Register, 27 June 1994, Page 8-B).Schenkat, R. (1993). Deming's quality: our last bu t best hope. Educational Leadership, 51(1), 64-65. Schenkat, R. (1993). Quality Connections: Transfor ming Schools Through Total quality Management. Alexandria, Virginia: American Assoc iation of School Administrators. Schmoker, M. and Wilson, R.B. (1993). Transformin g school through total quality education. The Phi Delta KAPPAN, 74(5), 389-395.Senge, P. (1990). The fifth discipline: the art an d practice of the learning organization. New York: Doubleday.Showers, B. (1985). Teachers coaching teachers. Educational Leadership, 42(7), 43-48. Sparks, D. and Loucks-Horsley, S. (1989). Five mod els of staff development for teachers. Journal of Staff Development, 10(4), 40-57.Sparks, D. and Vaughn, S. (1994). What every schoo l board members should know about staff development. Journal of Staff Development, 15(2), 20-22. Teeter, D.J. and Lozier, G.G. (1993). Six foundatio ns for total quality management. In D.J. Teeter and G.G. Tozier, (eds.). Pursuit of quality in h igher education: case studies in total quality management. New Directions for Institutional Rese arch, No. 78, pp. 5-11. San Francisco, California: Jossey-Bass Publishers.Teigland, M.D. (1995). A study of the beliefs for total quality management comparing superintendents, board members, and classroom tea chers in Iowa schools. Doctoral dissertation, Iowa State University, 1993.Templeman, T.P. and Peters, J. (1992). Teaching re search inservice model. Monmouth, OR: Western Oregon State College.U. S. Department of Commerce. (1995). The Malcolm Baldrige National Quality Award: Education Pilot Criteria, 1995. National Institute of Stan dards and Technology, Washington, DC. U.S. General Accounting Office. (1993). System-wi de education reform: Federal leadership district-level efforts. Report to Congressional R equesters. U.S.G.A.O.: Washington, DC. Vincoli, J. (1991). Total quality management and the safety and health professional. Professional Safety, 6, 27-32.Wade, R.K. (1985). What makes a difference in in-s ervice teacher education? A meta-analysis of research. Educational Leadership, 42(4), 48-54.Wood, F., Killian, J., McQuarrie, F., and Thompson, S. (1993). How to organize a school based staff development program. Alexandria, VA: Associatio n for Supervision and Curriculum Development. Appendix:
41 of 43 Sample Instruments Perceived Quality Assessment InstrumentStaff Development QuestionnaireExecutive Compensation QuestionnaireNote: Copies of instruments may be obtained from: William K. Poston Jr. Department of Professional Studies N229 Lagomarcino Hall Iowa State University Ames, Iowa 50011About the Author William K. Poston Jr., Ed.D.Associate Professor, Educational Administration Department of Professional Studies N225 Lagomarcino Hall Ames, Iowa 50021 Phone: (515) 294-9468 Fax: (515) 294-4942 Email: firstname.lastname@example.org Iowa State University Dr. Poston is currently Associate Profess or of Educational Administration in the Department of Professional Studies at Iowa State University in Am es, Iowa. Bill is a former math and physical science teacher and an experienced administrator, w ith 25 years of experience in school administration including 15 years as a superintende nt in the Flowing Wells (Tucson) and Kyrene (Tempe-Phoenix) Schools in Arizona and in the Billi ngs Public Schools, in Montana. Bill earned his B. A. at the University of Northern Iowa, and his E d. S. and Ed. D. at Arizona State University. He has many distinctive professional achievements, inc luding service as the youngest-elected international president of Phi Delta Kappa, and sel ection as an Outstanding Young Leader in American Education in 1980. Bill has authored many articles and has recently published four books: Making Schools Work: Practical Management of School Operations (C orwin Press, 1992), Effective School Board Governance (Phi Delta Kappa International, 1994), Making Governance Work: Total Quality Education for School Boards, (Corwin Press, 1994), The Curriculum Audit: Improving School Quality (Co-author, Technomic Publishing, 1995). Dr. Poston teaches courses in contempor ary management, strategic planning, school finance, and educational leadership at Iowa State University He is also the executive director of the Iowa Superintendents' Academy and the Iowa School Busine ss Management Academy. He continues to provide extensive service to schools in the areas o f evaluation, curriculum management auditing, performance-based budgeting, and continual quality improvement. Bill has been married for 35 years to his w ife, Marcia, and they have two daughters, both of whom are teachers in the West Des Moines Community Schools.
42 of 43 Copyright 1997 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is http://olam.ed.asu.edu/epaa General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, Glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-2411. (602-965-26 92)EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Andrew Coulson email@example.com Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov firstname.lastname@example.org Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Marshall University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Richard M. Jaeger University of North Carolina--Greensboro Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Dewayne Matthews Western Interstate Commission for Higher Education William McInerney Purdue University Mary P. McKeown Arizona Board of Regents Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton email@example.com Hugh G. Petrie SUNY Buffalo Richard C. Richardson Arizona State University Anthony G. Rud Jr. Purdue University Dennis Sayers University of California at Davis Jay D. Scribner University of Texas at Austin Michael Scriven firstname.lastname@example.org Robert E. Stake University of Illinois--UC Robert Stonehill U.S. Department of Education Robert T. Stout Arizona State University
43 of 43