Information pathways : how organizational culture influences the utilization of outcome measures in the Texas Children's Mental Health Plan

Citation
Information pathways : how organizational culture influences the utilization of outcome measures in the Texas Children's Mental Health Plan

Material Information

Title:
Information pathways : how organizational culture influences the utilization of outcome measures in the Texas Children's Mental Health Plan
Creator:
Hodges, Sharon
Place of Publication:
Tampa, Florida
Publisher:
University of South Florida
Publication Date:
Language:
English
Physical Description:
ix, 212 leaves : ill. ; 29 cm.

Subjects

Subjects / Keywords:
Corporate culture ( lcsh )
Child mental health services -- Texas ( lcsh )
Evaluation research (Social action programs) ( lcsh )
Dissertations, Academic -- Applied Anthropology -- Doctoral -- USF ( FTS )

Notes

General Note:
Includes vita. Thesis (Ph. D.)--University of South Florida, 1997. Includes bibliographical references (leaves 190-201).

Record Information

Source Institution:
University of South Florida
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
024648910 ( ALEPH )
39175580 ( OCLC )
F51-00202 ( USFLDC DOI )
f51.202 ( USFLDC Handle )

Postcard Information

Format:
Book

Downloads

This item is only available as the following downloads:


Full Text

PAGE 1

INFORMATION PATHWAYS : HOW ORGANIZATIONAL CULTURE INFLUENCES THE UTILIZATION OF OUTCOME MEASURES IN THE TEXAS CHILDREN S MENTAL HEALTH PLAN by SHARON HODGES A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Anthropology University of South Florida December 1997 Major Professor: Susan Greenbaum Ph. D

PAGE 2

Graduate School University of South Florida Tampa Florida CERTIFICATE OF APPROVAL Ph.D Dissertation This is to certify that the Ph. D Dissertation of SHARON HODGES with a major in Anthropology has been approved by the Examining Committee on November 5 1997 as satisfactory for the dissertation requirement for the Doctor of Philosophy degree Examining Committee : Major Professor : Susan Greenbaum, Ph D Member : Alvin Wolfe, Ph D Member: Christine Ho, Ph.D Member : Robert Friedman Ph .D. Member : Robert Anderson Ph. D

PAGE 3

DEDICATION With love for Barry Laurel Amelia and Beba

PAGE 4

ACKNOWLEDGMENTS This dissertation would not have been possible without the generous support and the many opportunities provided by the Department of Child and Family Studies at the Louis de Ia Parte Florida Mental Health Institute, University of South Florida. The inspiration for this dissertation came from Dr. Mario Hernandez, whose good mentoring and warm friendship allowed me to grow professionally and opened my eyes to the many opportunities for applying organizational anthropology in human service settings For her advice, guidance and well-placed sense ofhumor, beginning in my earliest days as a Ph.D student and continuing through the dissertation process I wish to thank my major professor Dr. Susan Greenbaum I would also like to acknowledge the support of my committee members : Dr. Alvin Wolfe and Dr. Christine Ho from the Anthropology Department Dr. Robert Anderson Dean ofthe College ofBusiness and Dr. Robert Friedman who generously provided dual feedback in his roles as a committee member and my department chair at the Institute This research project could not have been completed without the seemingly inexhaustible efforts of the research team at the Institute : Teresa Nesman an applied anthropology Ph. D student turned field researcher transcriber data analyst and copy editor, and Dr. Joko Sengova whose encouragement and thoughtful approach to this work so often served as a guide to both my analysis and writing.

PAGE 5

Information Pathways offered our research team many opportunities for making rewarding professional connections The many individuals at the four local mental health authorities participating in this research who gave generously of their time and their thinking deserve a hearty thank you In addition, we benefitted greatly from the commitment made to this project by our colleagues at the Texas Department of Mental Health and Mental Retardation Andres Guariguata Dr. Marcia Toprac, and Dr. Tracy Levins Perhaps most of all, the enthusiasm and intellectual curiosity of Dr. Lawrence Rouse was valued and is much missed On a more personal note it was the encouragement and support of my family that made my even embarking on a Ph D. possible and sustained me throughout the process Although behind the scenes they were the real team for this work.

PAGE 6

TABLE OF CONTENTS LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . 1v LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . v1 LIST OF ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . vu ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vll CHAPTER ONE: INTRODUCTION . . . . . . . . . . . . . . . . . . . 1 Statement of Purpose . . . . . . . . . . . . . . . . . . . . . . 1 Defining the Researc h Problem . . . . . . . . . . . . . . . . . . . 3 Guiding Research Questions . . . . . . . . . . . . . . . . . . . . 5 The Research Setting . . . . . . . . . . . . . . . . . . . . . . 6 Summary ofFindings ............................................. 7 CHAPTER TWO : LITERATURE REVIEW ................................. 9 Overview .................... ................................ 9 Systems of Care in Children s Mental Health ....... ................... 10 Outcome Accountability in Systems of Care . . . . . . . . . . 12 Contributions of Applied Anthropology to Mental Health . . . . . . . . 15 Organizational Anthropology . . . . . . . . . . . . . . . . . . . 16 Critical Issues in Organizational Anthropology . . . . . . . . . 19 Perspect i ves on Organizational Culture . . . . . . . . . . . . 23 Knowledge Utilization . . . . . . . . . . . . . . . . . . . . . 31 The Results ofKnowl edge Utilization .......................... 33 Converting Knowledge to Action . . . . . . . . . . . . . . 34 Collaborative Approaches to Research Utilization ................. 38 The Application of Anthropological Theory in Information Pathways . . . . 40 Utilization as a Cultural Domain .................................... 43

PAGE 7

CHAPTER THREE: STATE PERSPECTIVES ON OUTCOME INFORMATION UTILIZATION ....................................... 46 Establishing the Texas Children s Mental Health Plan .................... 46 The TCl\1HP as an Interagency Ini tiative . . . . . . . . . . . . . . 48 The TCl\1HP Evaluation . . . . . . . . . . . . . . . . . . . . 50 The Evaluation Feedback Loop .................................... 55 State-Level Perceptions ofLocai-Level Information Utilization ............ 56 CHAPTER FOUR: METHODOLOGY ................................... 60 Backgr ound . . . . . . . . . . . . . . . . . . . . . . . . . 6 0 The Research T earn . . . . . . . . . . . . . . . . . . . . . . 61 Data Collection Methods . . . . . . . . . . . . . . . . . . . . 63 Local Mental Hea lth Authority Site Selection . . . . . . . . . . . . . 64 The Data Collection Process ...................................... 70 Data Collection Issues ..................................... 73 Data Reduction and Analysis ..................................... 75 Data Reduction .......................................... 7 6 D ata Analysis . . . . . . . . . . . . . . . . . . . . . . 77 Ethical Con s iderations . . . . . . . . . . . . . . . . . . . . . 78 CHAPTER FIVE : RESULTS : LOCAL PERSPECTIVES ON OUTCOME INFORMATION UTILIZATION .................................. 85 Background on Local Mental Health Authorities . . . . . . . . . . . 85 Sit e Q Results . . . . . . . . . . . . . . . . . . . . . . . . 88 Sit e Introduction . . . . . . . . . . . . . . . . . . . . 88 Aspects of Organizational Culture Influencing Information Utilization at S ite Q .............................................. 91 How Outcome Information Impacts Decision Making at Site Q . . . 97 Conditions that Support or Impede the Use of Outcome Information 102 Summary of Site Q Findings . . . . . . . . . . . . . . . . 104 Site R Results . . . . . . . . . . . . . . . . . . . . . . . . 105 Site Introduction . . . . . . . . . . . . . . . . . . . . 105 Aspects of Organizational Culture Influencing Information Utilization at Site R .... ........................................ 109 Summary of Site R Findings . . . . . . . . . . . . . . . . 119 Site S Results . . . . . . . . . . . . . . . . . . . . . . . . 121 Site Introduction . . . . . . . . . . . . . . . . . . . . 121 Aspects of Organizational Culture Influencin g Information Util i zation at SiteS ............................................. 124 H o w Outcome Information Affect s Decision Making at SiteS ...... 127 Conditions that Support or Impede the Use of Outcome I nformation 131 Summary of Site S Findings . . . . . . . . . . . . . . . . 132 11

PAGE 8

Site T Results ................................................. 133 Site Introduction ........................................ 133 Aspects of Organizational Culture Influencing Information Utilization at Site T ............................................. 138 How Outcome Information Impacts Decision Making at Site T . . . 141 Conditions that Support or Impede the Use of Outcome Information 143 Summary of Site T Findings . . . . . . . . . . . . . . . . 146 Summary ofLocal-Level Results ...... ........... ............... 147 CHAPTER SIX: ANALYSIS AND DISCUSSION .............. ........... 149 Building a Model of Utilization ................................... 149 Linking Information to Knowledge in the TCMHP . . . . . . . . . . 157 High Use Sites ...................... .... ................ 158 Low Use Sites .......................................... 164 Linking Knowledge to Action : Cross-Site Analysis . . . . . . . . . . 169 High Use Sites . . . . . . . . . . . . . . . . . . . . . 170 Low Use Sites . . . . . . . . . . . . . . . . . . . . . 172 Shared Characteristics ofHigh and Low Use Sites .......... ........... 173 Lessons Learned : Maximizing the Utility of Outcome Information ......... 180 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 184 REFERENCES 190 APPENDICES ..................................................... 202 APPENDIX 1. GLOSSARY OF TERMS IN CHILDREN'S MENTAL HEALTH ........................................ 203 APPENDIX 2 CORE VALVES AND GUIDING PRINCIPLES FOR THE SYSTEM OF CARE ..... ......................... 205 APPENDIX 3 EXECUTIVE SUMMARY OF THE SYSTEM ACCOUNT ABILITY STUDY ........................ 207 APPENDIX 4 LETTER INVITING PARTICIPATION IN INFORMATION PATHWAYS ..................................... 208 APPENDIX 5 INFORMATION PATHWAYS SEMI-STRUCTURED INTER VIEW GUIDE FOR LOCAL-LEVEL INTER VIEWS .................................. 209 APPENDIX 6. CODE SHEET FOR INFORMATION PATHWAYS TRANSCRIPTS . . . . . . . . . . . . . . . . . 211 VITA . . . . . . . . . . . . . . . . . . . . . . . . . . . . end page lll

PAGE 9

LIST OF TABLES Table 1 : Performance Contract Measures for FY 1997 ....................... 53 Table 2 : Field Researchers for Local Mental Health Authority Site Visit s .......... 62 Table 3 : Information Pathways Data Collection Process ....................... 64 Table 4 : Information Pathways Site Selection Matrix ......................... 6 5 Table 5: Local Mental Health Authority Interviews for Information Pathways ....... 72 Table 6 : Site Findings Worksheet ........................................ 7 9 Table 7 : Information Pathways Stakeholder Needs . ........................ 82 Table 8 : Summary ofLocal-Level Results . . . . . . . . . . . . . . . . 148 Table 9 : Factors Affecting the Degree ofintersection between the TCMHP Evaluation Design and Local Culture at Site Q ............... 161 Table 10: Factors Affecting the Degree ofintersection between the TCMHP Evaluation Design and Local Culture at Site T ...... ___ _ _ D64 Table 11: Factors Affecting the Degree oflntersection between the TCMHP Evaluation Design and Local Culture at Site R ... ____ .... D6$ Table 12: Factors Affecting the Degree ofintersection between the TCMHP Evaluation Design and Local Culture at SiteS ___ .. _ ... __ n1({)) Table 13: Interview Questions Relating to Guiding Research Questions _________ 21 D I V

PAGE 10

LIST OF FIGURES Figure 1 : Domains of Data Analysis for Information Pathways . . . . . . . . . 63 Figure 2 : Linking Information to Action . . . . . . . . . . . . . . . . . 156 v

PAGE 11

LIST OF ACRONYMS CASSP Child and Adolescent Service System Program CBCL Child Behavior Check List CMHC Community Mental Health Center CMHS Center for Mental Health Services CMT Community Management Team FMHI Louis de Ia Parte Florida Mental Health Institute FY Fiscal Year MHAT-Mental Health Association ofTexas PL Public Law SMT-State Management Team TCMHP-Texas Children's Mental Health Plan Texas MHMR-Texas Department ofMental Health and Mental Retardation TQM-Total Quality Management Vl

PAGE 12

INFORMATION PATHWAYS : HOW ORGANIZATIONAL CULTURE INFLUENCES THE UTILIZATION OF OUTCOME MEASURES IN THE TEXAS CHILDREN S MENTAL HEALTH PLAN by SHARON HODGES An Abstract Of a dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Anthropology University of South Florida December 1997 Major Professor : Susan Greenbaum Ph. D Vll

PAGE 13

The purpose of this study was to identify aspects of organizational culture which influence the utilization of measurable outcome information in the planning and delivery of children's mental health services The study examined factors that both support and impede the use of outcome information in local-level decisions affecting the delivery of children's mental health services The goal of the research was to assist human service systems in developing strategies to use outcome information more effectively The study drew upon anthropological understanding of organizations to compare and contrast the organizational cultures at four local mental health authorities participating in the Texas Children's Mental Health Plan (TC.MHP) For the purpose of this study organizational culture was regarded as constructed through interaction at all levels of organization This approach directed this research toward an investigation of interactions among individuals at state and local levels of the TCMHP in order to understand how cultural processes influence the utilization of outcome information at the local level. In addition to interviews with state-level staff and a review of state and local documents, data collection included forty-one interviews with key stakeholders at the local level. The findings of this study support the conclusion that patterns of outcome information utilization are deeply rooted in the organizational cultures of the local mental health authorities Increasing the utility of outcome information for local service-related decisions means increasing the degree of interface between the structure of the outcome evaluation and the organization culture in which the evaluation is occurring In order to maximize the utility of outcome information human service systems must carefully consider both the structural domains oftheir outcome evaluation and the interface Vlll

PAGE 14

between this evaluation structure and the o r ganizational culture where the evaluation is being implemented Focusing on evaluation structure without adequate consideration of the c ultural processes that will influence its implementation reduces the potential for the result s of the outcome evaluation to provide crucial feedback on the effectiveness of s e rvice delivery strategies Abstract Approved : ____________________ Major Professor : Susan Greenbaum Ph D Profe s sor Department of Anthropology Date Approved : ______________ IX

PAGE 15

CHAPTER ONE INTRODUCTION Statement ofPurpose 1 This dissertation investigates how organizational culture influences the utilization of outcome information in the plann i ng and delivery of children's mental health services. Across the United States there are efforts underway to establish mechanisms that can track the outcomes of services provided for children with severe emotional disturbance Outcomes in children s mental health can be defined as the result or impact of services on the children and their families who receive these services Similarly outcome accountability can be defined as the responsibility of a service system for accomplishing publicly articulated and accurately tracked goals regarding the services they provide (Hernandez and Hodges 1996 : 7) The outcomes monitored by children's service systems can include tracking trends in the behavioral and emotional functioning of children, trends in school behavior or juvenile arrest rates which provide information about children s social functioning, or trends in the restrictiveness of services children are placed into, which can provide information about whether appropriate efforts are being made to help children remain safely at home or in their communities while they receive treatment. The recent focus on outcomes in children's mental health throughout the United States is in direct contrast with the traditional approach to planning human services in

PAGE 16

which services have been conceived and funded on the basis of expressed need or the problems they intend to address (Burchard and Schaefer 1992). In addition, funding agencies have tended to judge the effectiveness of public programs based on how much they cost, regardless of whether they achieve stated goals (Gore and Brown 1993) Consequently it has been difficult to know if or when human service programs have successfully met the needs of the people they intend to serve 2 Both within and outside of mental health, interest in outcome accountability is growing and outcomes are beginning to play a prominent role in the design and management of service delivery systems (Young et al1994; Hernandez and Hodges 1996) This move toward building systems of outcome accountability is leading to discussion about the information strategies that must be created in order to move systems from service-related decisions that focus on compliance with rules and regulations to decision making based on practical data that informs how services are delivered (Burchard and Schaefer 1992; Cohen and Ooms 1993; Usher 1993a,b; Weiss and Jacobs 1988) The purpose of this dissertation, titled Information Pathways, is to identify aspects of organizational culture which influence the utilization of measurable outcome information in the local-level planning and delivery of children's mental health services The goal of the research is to assist service systems in developing strategies to use outcome information more effectively in service planning and delivery The study is focused on the Texas Children s Mental Health Plan (TCl\1HP), Texas' first statewide program for delivering mental health services to children with severe emotional disturbance This study will examine factors that both support and impede the use of

PAGE 17

3 outcome information in decisions affecting the delivery of children s mental health services at the local level in Texas These factors will include understandings of outcome evaluation and outcome information utility, perceptions of outcome information use and conditions that support and impede the use of outcome information in decision making Established in 1990 through the Texas Department ofMental Health and Mental Retardation (Texas MHMR), the TCMHP began developing a system of outcome information from its inception (Rouse, MacCabe and Toprac 1995) The outcome information system has now been expanded to include the entire state through 45 local mental health authorities from which mental health services are provided for children and families This study differs from existing efforts to identify priority outcomes in children s mental health or to recognize the best practices for establishing systems of outcome accountability, because it focuses on issues of outcome information utilization at the local level where the day-to-day work of providing mental health services for children and families occurs. It is hoped that this study will build understanding of the conditions which both support and impede the use of outcome information by local mental health authorities as well as offer strategies for increasing the positive impact of outcome information on decisions affecting service planning and delivery Defining the Research Problem The central focus of this study is to compare and contrast the organizational cultures in four community mental health authorities participating in the TCMHP They will be compared with one another as well as with the outcome information system as it is

PAGE 18

formally defined by the TCMHP Anthropological concepts of culture have in common the belief that culture is shared and systematic although specific definitions of culture suggest different ways oflooking for culture in organizations For the purpose of this research culture is seen as both a process and product of organization Culture is regarded as emergent from social organization and as constructed through interaction at all levels of organization This approach to culture directs this research toward an i nvestigation of the interactions among individuals at all levels of the TCMHP in order to understand how cultural processes influence the utilization of outcome information 4 The formal system of outcome information for the TCMHP was designed to provide equal access to outcome information for all 45 community mental health authorities in Texas (Te x as Children's Mental Health Plan 1995) In addition, the TCMHP has placed a special emphasis on local decision making in implementing the outcome information system in local mental health authorities (Rouse, MacCabe, and Toprac 1995) Interviews with state-level participants in the TCMHP outcome evaluation indicated that, even with an explicit effort to design an outcome evaluation that wquld foster local-level decision making there exists variation in the use ofthis information for decision making at the local level. Informants at the state level in the TCMHP expressed the view that some local mental health authorities in the Texas system were better able to use outcome information in their local service planning and delivery decisions The identification of differential information use as an issue in the TCMHP prompted this investigation of factors that both promote and impede utilization at the local level.

PAGE 19

Definitions of what outcome information is, if outcome information is used, and perceptions of why outcome information is or is not used will be explored with regard to similarities and differences within the individual community mental health authorities, among the four sites and in comparison with state-level understandings of outcome information utilization An important result of this study will be an analysis of how the principles of outcome accountability that have been developed at the state level in the TCMHP are operationalized in the context of local-level decision making regarding 5 service planning and delivery It is hoped that this analysis will prove useful in establishing strategies that child-serving organizations can employ to better ensure the incorporation of outcome information into their decision making processes Guiding Research Questions This study was designed to investigate aspects of the outcome information system at the local level that emerge from the implicit and observable interactional relationships related to the information system This study is grounded in the assumption that children and youth with severe emotional disturbance are best served by integrated systems of care in which there is coordination of services from multiple agencies and outcomes or results of the services delivered are tracked and shared across the various service components This suggests the necessity of tracking outcomes that cross agency boundaries so that outcomes in mental health such as the improved behavioral and emotional functioning of a child or adolescent are seen as related to improvements in other aspects of the child's life such as improved school attendance and behavior or reduced juvenile justice involvement.

PAGE 20

This study will work from the hypothesis that the effective utilization of outcome measures by a child-serving organization necessarily involves three integrated processes : 1) collecting data; 2) analyzing and synthesizing data into information ; and 3) disseminating this information to stakeholders by transforming data into knowledge that contributes to decision making The following guiding research questions were developed for the purpose of investigating the conditions under which information is and is not used in making decisions affecting service planning and delivery: 1 What aspects of organizational culture influence the utilization of outcome information among local mental health authorities in the Texas Children's Mental Health Plan ? 2 How does outcome information impact decision making within high-use and low use local mental health authorities? 3 What conditions support or impede the use of outcome information in service planning and service delivery? The Research Setting Information Pathways was conducted through the System Accountability Project for Children's Mental Health at the Louis de Ia Parte Florida Mental Health Institute 6 (FMHI) and was supported by a grant from the Annie E Casey Foundation Inc FMHI is Florida's primary research and training facility for public mental health services FMHI has four departments by which it addresses the issues of a broad range of populations through basic and applied research training and consultation, and multi-disciplinary demonstration projects involving professionals from psychology public health social work, education, and anthropology These departments are : the Department of Community Mental Health, the Department of Mental Health Law and Policy the

PAGE 21

7 Department of Aging and Mental Health and the Department of Child and Family Studies This research was conducted in the Department of Child and Family Studies for which Robert Friedman Ph.D serves as chair. The System Accountability Project for Children's Mental Health is focused on promoting the use of measurable outcomes by mental health providers. Researchers with the System Accountability Project are investigating how information about outcomes impacts service planning and delivery The goal of the System Accountability Project is to promote the best practices currently available that track measurable outcomes as a means to improve service delivery As an anthropologist, I have assumed several roles in this project. Initially, my role could be described as an evaluator and policy researcher in the study of how outcome information may contribute to organizational goals and objectives I hope the conclusions drawn from this study will be useful in informing policy decisions regarding the utilization of outcome measures in mental health service delivery systems for children The role I find most interesting however is that of change agent. It is my hope that the results of this study will prove useful in informing the implementation of actual systems of outcome tracking In the role of change agent I would like to help child-serving organizations develop strategies which better ensure the incorporation of outcome information into their decision processes Summary ofFindings The findings of this study support the position that patterns of information utilization are deeply rooted in the interpretive processes of the organizational cultures at

PAGE 22

8 the local mental health authorities under consideration The data collected in this study demonstrated that each of the local mental health authorities participating in this study took action and made service-related decisions that were based on shared meaning and acquired knowledge of the utility of outcome information within their local contexts The low-use sites acquired knowledge resulting in modes of utilization that only minimally related TCMHP outcome information to service planning and delivery while the high-use sites acquired knowledge that resulted in modes of utilization that directly related TCMHP outcome information to their service planning and delivery decisions This dissertation consists of six chapters The present chapter provides an introduction to the research and a brief background on the project. Chapter Two presents a review of the literature that explores the pertinent issues of organizational anthropology, systems of care and knowledge utilization relating to the utilization of outcome information in service planning and delivery for children's mental health. Chapter Three presents the results of the state-level data collection In Chapter Four, the methodology used in this project is discussed including data collection data reduction and analysis and ethical considerations Chapter Five presents the results of the local-level data collection. The dissertation concludes with a discussion of these results in Chapter Six comparing the characteristics of highand low-use sites

PAGE 23

CHAPTER TWO LITERATURE REVIEW Overview 9 In describing the nature of applied work in anthropology, Chambers (1985 : 9) wrote that applied anthropology involves a conscious decision to enter a setting where interests that are strictly disciplinary play alongside concerns of a more diffuse and public nature Similarly Baba (1995 : 58) observed that since its beginnings applied anthropology has been directed at solving problems defined outside of the profession The focus of applied anthropology on issues and concerns outside of the discipline is reflected in this dissertation research and the literature review that informs this work. The issues questions and concerns raised in this study are grounded in the field of children s mental health It is therefore, important to have a basic understanding of the service systems that care for children and adolescents with severe emotional disturbance and their families This literature review begins with a discussion of systems of care in children s mental health Applied anthropologists generally seek to understand some aspect of human behavior in a group context Information Pathways is informed by anthropological knowledge of organizational culture and so the literature review provides a discussion of critical issues in organizational anthropology and perspectives on organizational culture

PAGE 24

10 Because this study is concerned with how information is or is not used in decision making, literature on knowledge utilization is also reviewed Finally, an overview of anthropological theory as it is applied in the context of this study is provided. Systems of Care in Children's Mental Health Outcomes, out-of-home placements wraparound, systems of care : this is the language of services for children and youth with severe emotional disturbance In addition to the clinical language of serving children with severe emotional disturbance, research and evaluation in children's mental health includes concepts and terminology such as these that guide the planning and delivery of those services to children and their families Learning the language of children's mental health services is a necessary beginning to developing an understanding of the issues that shape children s mental health services in the United States A glossary of basic terms is provided in Appendix l. A central concept in planning and administering services for severely emotionally disturbed children is that of a System of C are. Almost three decades ago The Joint Commission on the Mental Health of Children ( 1969) reported that millions of children were not receiving adequate mental health services Knitzer (1982) documented that the majority of children with severe emotional disturbance received no services at all and that many of those who did received inappropriate and often very restrictive services. Both of these studies served to increase the attention paid to the needs of children with emotional disturbance Stroul and Friedman ( 1986) reported that while there was a great deal of discussion about systems of care for children s mental health beginning with the Joint

PAGE 25

11 Commission findings, the concept lacked clarity In 1984 the National Institute ofMental Health created the Child and Adolescent Service System Program (CASSP) which undertook a project to define the concept of "syst em of care more clearly This project resulted in the publication of A System of Care for Severely Disturbed Children & Youth (Strout and Friedman 1986) When this monograph was written, there was little guidance in how to build systems that would allow children to receive services while remaining in their home or community. Few communities offered comprehensive, fully integrated, community-based services Today many communities have established systems of care and many more are in the process of developing systems of care for children with severe emotional disturbance (DeCarolis 1994). A system of care is defined as "a comprehensive spectrum of mental health and other necessary serv ices which are organized into a coordinated network to meet the multiple and changing needs of children and adolescents with severe emotional disturbances and their families (Strout and Friedman 1986 rev : xx). System of care represents both a philo so phy about the way services should be delivered and a conceptual framework that provides direction to policy makers program administrators, and direct service workers. The core values of the system of care philosophy are that services should be child-centered w ith the needs of the child and family dictatin g the types and mix of services provided and community-based with the locus of services as well as management and decision making responsibility resting at the community level (Strout and Friedman 1986: vii) As a conceptual framework the system of care principles can guide the operational philosophies that dictate the organizational structure and service

PAGE 26

components from community to community Core values and guiding principles for a system of care are presented in Appendix 2 Outcome Accountability in Systems of Care The emphasis on outcome accountability in mental health can be traced to the Community Mental Health Centers (C:MHC) Amendments in 1975 (P.L. 94-63) This federal legislation required community mental health centers to evaluate the efficacy of their own services through a self-evaluation process that included citizen review and participation (Flaherty and Windle 1981 ) The systematic collection of information on client outcomes was intended to improve the functioning of CMHCs by fostering better decision making and improved program operations (Elmore 1978 ; Flaherty and Windle 1981 ; U.S. General Accounting Office 1976 ; Wortman 1975) The focus on outcome accountability in CMHCs failed, however to receive sufficient attention from mental health researchers and service providers As a result the collection of outcome information intended to improve program operations was not widely implemented 12 It is helpful to consider definitions of outcomes and outcome accountability within the conceptual framework of systems of care Accountability refers to whether systems are actually accomplishing what they set out to do and whether the children and families the system serves are better off as a result of the services they provide Because the systems of care philosophy encompasses service systems that extend beyond what might traditionally be considered mental health systems of care often include educational child welfare and juvenile justice service systems as well. Corning from outside of the field of mental health I initially understood references to outcomes and outcome accountability as

PAGE 27

13 referring only to the clinical status of individuals being served. Clinical functioning is indeed an important type of measurable outcome. Rosenblatt and Attkisson (1993) however suggest a typology of rele v ant client outcomes based on earlier work by Ciarlo (1982) Hargreaves and Shumway (1989) and Ware (1989, 1986) that is broader than clinical measures The Rosenblatt and Attkisson model of outcomes includes three domains : 1) outcomes ; 2) sources of information ; and 3) social context Their typology identifies five outcome categories : clinical status functional status life satisfaction and fulfillment welfare and safety and satisfaction with services One ofthe advantages ofthe Rosenblatt and Attkisson model is its incorporation of multiple outcome categories into the typology The sources of outcome information include input from the client family members, members ofthe client's social network clinical prac titioners, and others These outcome categories and sources of outcome informat i on are then interpreted through four social conte x ts of an individual s experience : the personal life of an individual the famil y life of an individual the work and school life of an ind i vidual and the individual's life within the c ommun i ty FMHI considers this conceptual framework for outcome measurement to be the most comprehensive to date. Thi s model is instructive because it recognizes that severe emotional disturbance i mpacts a wide range of social activities including family life school and work, and life within the communit y It also recognizes a wide range of stakeholder i nterest in outcomes by its inclusion of several sources of outcome information In addition it is important for outcome measures to be straightforward, practical and easily understood by stakeholders ins i de and outs ide of the mental health system It is to be

PAGE 28

14 e x pected that many outside of the field of mental health will have a limited understanding of clinical measures However a child's ability to live safely at home remain in school or employed and have some measure of satisfaction in life are obvious indications of stability as well as outcomes t h at are easily understood by stakeholders external to menta l health systems Although it did not specifically encompass the concept of outcome accountability A Sys t e m of C ar e for Sever e ly Disturb e d C hildren & Youth (Stroul and Friedman 1986) more clearly defined the concept of system of care and provided guidance on how to build systems that would allow children to receive services while remaining at home and in their communities The shift in interest toward results-based accountability in children s mental health has raised hopes that service systems will respond more flexibly to those they serve that public trust in the ability of human service institutions to accomplish their intended purposes will be restored and that communities will be better able to plan their support of children and families (Schorr Farrow Hornbeck et al. 1994) The increased focus on outcomes both complements and expands the systems of care concept by helping policy makers and administrators establish strategies to know whether human service programs have successfully met the needs of the people they are serving Outcome information can help determine if services provided for children and families are, in fact child-centered family-focused and community-based as the core values and guiding principles ofthe system of care philosophy indicate are so important.

PAGE 29

15 Contributions of Applied Anthropology to Mental Health A review of the mental health literature suggests that the anthropological perspective can and does contribute valuable insight into a broad range of issues in the field of mental health Based on a review of this literature, three areas of mental health research to which anthropology can make significant contributions can be identified These three areas are : I Research that explores the relationship between mental health and culture Examples of this would include research into cultural factors that influence treatment-seeking behavior (e. g Fosu 1995), the relationship between culture and symptoms (e g Kirkmayer Young and Robbins 1994), and the cultural influences on how mental illness is classified and categorized (e g Jenkins 1993 Gaines 1992 Fabrega 1989) 2 Research that considers mental illness in the context of larger social issues such as the study of the homeless mentally ill (e g Koegel 1992) the study of mental illness in the context of family and the community (e g Boutte 1992) and research into the relationship between social support and mental well-being (e g Jacobson 1987) 3 Research that investigates the organizations and systems that provide mental health services Examples of this would include research into the effective integration of mental health care into primary health services (e g Fosu 1992), the deinstitutionalization of mental health care (e g Scheper-Hughes and Lovell 1986) and the implementation of mental health programs and services (e. g Richeport 1984) While each of these are areas of investigation in which anthropology can make valuable contributions my own interest and emphasis rests with the investigation ofthe organizations and systems which provide mental health services to children and their families A review ofthe proceedings from the 7th Annual Research Conference s ponsored by the Research and Training Center for Children's Mental Health (Liberton

PAGE 30

16 Kutash and Friedman 1995) indicates that research on systems of care for children s mental health is a multi-disciplinary field which currently includes parents psychologists, medical doctors, social workers educators criminologists public administrators sociologists and anthropologists For example at the most recent conference held in February 1997 anthropologists reported on their research and evaluation work in children s mental health. The papers presented included topics such as the use of case study methodologies for evaluating systems of care (Gutierrez-Mayka 1997) and child protection reform (Gomez 1997), issues of cultural competence in children's mental health (Uzzell and Whiteford 1997), and the use of ethnographic data in defeating stereotypes of inner-city African American communities (Greenbaum 1997) The planning and delivery of children's mental health involves multiple and comple x levels of public and private organizations which provide services to children and their families In such a multi-disciplinary field it is important for an anthropologist to understand and be able to communicate to others the value of anthropology s contributions to the study of systems of care for children s mental health With regard to research that investigates the organizations and systems that provide mental health services one way anthropology can contribute to a better understanding of how to build more effective systems of care for children with severe emotional disturbance is by drawing f r om a long heritage of study of complex organizations Organizational Anthropology The anthropological literature on the study of complex organizations identifies several terms used to reference the same or similar anthropological in v olvements Gamst

PAGE 31

17 refers to industrial ethnology (the comparative study of cultural differences and similarities) in his discussion of the ethnological study of modem industry in the United States ( 1977). Holtzberg and Giovannini (1981) refer to industrial anthropology Baba s (1986) monograph speaks ofboth business and industrial anthropology Focusing on the anthr o pology of work The Anthropology of Work Review published a special edition devoted to organizational culture (Sachs 1989) Jordan (1994) refers to practicing anthropology in corporate America It can be difficult to distinguish among corporate anthropology business anthropology, industrial anthropology and the anthropology of work The term organizational anthropology as opposed to the anthropology of work, business industry or corporations seems most appropriate to the research of this dissertation This term is used to refer to the anthropological study and analysis of organizations in the public private for-profit and non-profit domains Organizational anthropology is both broader and narrower than the concept ofbusiness anthropology : broader in that it extends beyond the private sector by including organizations in the public domain and narrower because it does not include the area of marketing and consumer behavior traditionally included under the domain of business anthropology (see Gardner 1978) In addition to the inclusion ofthe public domain in organizational anthropology the concept of organizational anthropology differs substantially from Baba' s ( 1986) distinction between industrial and business anthropology in its purposeful incorporation of both research and practice This distinction accurately reflects the practice of

PAGE 32

18 organizational anthropology as it is represented in the current literature which demonstrates the combination of research and practice in most projects1 Several authors suggest domains of study within the field of business-industrialorganizational anthropology Gardner (1978) suggests three areas: consumer behavior external corporate relations, and internal organizational issues for the practice of business anthropology. Baba (1986) builds upon Gardner's model, adding entrepreneurship product design research and intercultural training to the domain ofbusiness practice Schwartzman (1993) discusses three categories of organizational ethnography : anthropology of work studies organizational culture studies and the analysis of organizing processes and their relation to larger systems Schwartzman's categories most closely mirror those of organizational anthropology, largely because of her broader organizational focus which does not restrict the domain to private sector business pursuits I suggest that the following domains of research and practice may be considered as included in organizational anthropology : 1) internal organizational issues ; 2) external organizational work which could encompass the social impact of decisions a particular organization might make; 3) the anthropology ofwork, both in specific occupations and across industries; and 4) cross-cultural studies These four domains allow for the distinction between an internal and external organizational focus and add cross-cultural analysis to Schwartzman s breakdown 1 See Hamada and Jordan 1990 for detailed cases which describe the collection and analysis of data and the reporting of research as a part of practicing organizational anthropology

PAGE 33

19 Critical Issues in Organizational Anthropology In describing industrial ethnology Gamst ( 1977 : 1-8) identified three elements that distinguished the anthropological study of work from that of other disciplines They are : I) culture as a holistic master concept; 2) cross-cultural perspective ; and 3) an ernie approach These are critical elements of anthropological inquiry in general, but as anthropologists we must take care in how these guidelines are applied in orgarrizational analysis For instance, if we use culture as the holistic master concept through which we develop our explanations of human behavior in orgarrizations it is important to be explicit in our definition of culture. The NAP A bulletin about consulting on organizational culture edited by Jordan (1994a) presents three cases of anthropological consulting in organizations each of which explicitly states the concept of culture employed in the analysis In addition to the clarity with which the individual cases are presented, Jordan argues for the importance of carefully developing the approach definition and method in any work an orgarrizational anthropologist might do (1994b) In contrast the NAPA bulletin about anthropology and management consulting by Giovanrrirri and Rosansky ( 1990) provides guidance for anthropologists interested in working as management consultants Following a discussion ofwhat management is, types of management consultants types of consulting firms and the consulting process the publication offers eight brief cases as e x amples of management consulting by anthropologists. These cases focus heavily on the methods employed by anthropologists as they worked as consultants including projects such as developing and managing trairring programs evaluating a famil y

PAGE 34

20 medicine program implementing quality improvement processes improving performance of middleand lower-level managers, and uncovering rites of passage for new consultants However, none of the case descriptions hinted at the concept of culture that informed the work that was being done This is not to say that the anthropologists involv e d in the con s ulting did not define culture for the purposes of their projects but rather that a publication aimed at instructing anthropologists about management consulting focu s ed heavily on method and mirumized theory The issue of clarifying the c ulture concept was presented belatedly as a caveat after the cases were described (see Giovannini and Rosansky 1 990 : 28 ) If comparative cross-cultural analysis is implicit in the anthropological approach to orgaruzational studies and we use our interpretive skill to solve problems, then it is incumbent upon those of us studying organizations in a complex society to analyze informal as well as formal and local as well as centralized cultures of the organizations we study Gregory (1983) argues that the small-society metaphor from traditional ethnography is often inappropriate for organizations which are large heterogeneous changing and require only part-time commitment from their members "People participate in many groups and acquire culture in all e x periences (e g ., as family members, residents, citizens, employees). As affiliations change o ver time, culture from past experiences is carried into new ones. Thus, people as culture bearers link groups simultaneousl y through joint membership and sequentiall y over their careers (Gregory 1983 :365). Her research on technical professional s in the Silic o n V alle y illustrates the application of multip l e n a tive-view paradigms as a m et h o d o f e x ploring multiple organizational perspectives_

PAGE 35

21 In a related issue Jordan ( 1994b) acknowledges that many anthropologists consider the use of the culture construct in an organizational setting to be an inappropriate use of the concept of culture believing that an organization cannot have a culture in the anthropological sense, and certainly not multiple potentially conflicting subcultures within She concedes that in the most traditional sense culture results from primary enculturation beginning at birth but argues that a secondary enculturation begins with employment r e sulting in recognizable patterns of behav ior i n an organization (1994 : 9). Reflecting Gregory s comments Jordan includes voluntary associations gender groupings or a variety of other structures as potentially influential in secondary enculturation Similarly a discussion oflanguage and enculturation refers to secondary enculturation as extended acquisition noting that adults are influenced significantly by social institutions such as the i r workplace (Saville-Troike 1989 : 246) The ernie approach to research makes the anthropologist better able to go beyond the surface or obvious explanations of behavior in an organization in an effort to understand the logic and world-view of his/her informants In organizational work, it is critical to distingui s h the tendency of informants to describe culture as it should be rather than as it is. Briody (19 89 : 5) found that managers and non-managerial employees both had a tendenc y to describe the culture of their work environment in terms of specific ideal characteristics a lcind of optimal or ideal culture that they perceived as slipping when n e w job tasks changed their routines Briody s work illustrates that particularly in an environment in which the term culture carries multiple non-anthropological definitions w e must be aware of the difference s betw e en optimal and operative cultures while at the

PAGE 36

same time recognizing that an informant's view of culture optimal or otherwise, represents part of his or her world-view 22 Another issue critical to organizational anthropology is how organizational culture is defined in relation to societal culture The current literature in organizational anthropology is frequently so focused on methodology and on the analysis of a single organization that it neglects the issue of how organizational culture is related to larger societal culture Notable exceptions to this are Giovannini and Rosansky (1990), Jordan (1994b) and Maccoby (1994:267) Giovannini and Rosansky (1990) imply the existence of a relationship between societal culture and organizational culture when they suggest that anthropology can help management consultants and their clients respond successfully to global environmental changes such as increased globalization diversification of the workforce and technological innovation Jordan (1994b :6) deals with this relationship more directly when she describes organizational culture as "a part culture in one or more larger culture groups that might include regional cultures within one nation, more than one national culture and even a world culture of international business ." She describes organizational culture as nestled within larger cultures, influenced by cultures which might cut across both the organizational and the larger culture, as well as sub-cultures which might exist within the organizational context. This interrelated web of cultures is instructive to an understanding of how an organization may have a culture which fits into a larger context Maccoby ( 1994 : 267) argues that organizational culture cannot stand alone and that anthropology s holistic approach can help managers and administrators realize the symbiotic connection to a larger culture ." MaGcoby emphasizes that considering

PAGE 37

23 organizational culture over time allows the analysis of the influences of the larger culture of which the organization is a part Although much of the literature in organizational anthropology focuses on case studies which detail the inside view of particular organizations, it is important to remain cognizant of the webs of cultural meaning that exist in our broader culture and how the organizations we study fit into the larger societal culture It is this connection that ultimately gives meaning to our analysis How we define culture, use our interpretive skills to solve problems and employ an ernie approach are critical issues for all applied anthropologists but particularly important for organizational anthropologists Resistance within our discipline to application of the culture concept in an organizational context and the skepticism, bordering on thinly veiled disdain, with which some of our colleagues regard anthropological work in large bureaucratic and often powerful organizations2 means that it is especially important for us to protect the integrity of the work we do by guarding against the reduction and simplification of anthropological theory and method as we apply it. Perspectiv e s on Organizational Culture One of the basic debates that exists with respect to organizational culture is culture's relationship to an organization Srnircich (1983) draws the basic distinction between defining culture as a critical organizational variable and defining culture as a metaphor for conceptualizing an organization (this distinction is also referenced by other 2 See Ruano 1995 who worries that applied anthropology will be the MacScience ofthe nineties

PAGE 38

24 organizational theorists e g Morgan 1986 and Reichers and Schneider 1990) These are fundamentally different ways of viewing culture because they make fundamentally different assumptions about what organizational culture is and, as a result, they generate fundamentally different emphases in research and application. The conceptualization of culture as an organizational variable reflects metaphors of the organization as machine and organism Smircich (1983) maintains that defining culture as an independent variable brought into the organization by its individual members reflects both classical management theory in which organizations are social instruments for task accomplishment and the functionalism of Malinowski in which culture is an instrument serving human biological and psychological needs (1983 : 342) She offers the popularity of Ouchi s ( 1981) Theory Z, describing Japanese management practices as a reflection of the le vel of interest in defining culture from this perspective Smircich (1983 : 344) contrasts this view of culture as an independent variable with a view of culture as something an organization produces a dependent variable This approach suggests that organizations are culture-producing phenomena and is reflected in the popular works by Deal and Kennedy (1982) and Peters and Waterman (1982) as well as more academic work by Schein (1985) and Kilmann (1984) Smircich (1983 : 342) offers c ontin g ency theory as an e xample of organizational theory which maintains the perspective of organizations as adaptive organisms existing by process of exchange with the environment and culture as something that organizations produce. Within anthropology Smircich offers that a similar perspective is found in Radcliffe-Browne s s tructural functionalism, which maintained that culture functions as an adaptive regulatory

PAGE 39

25 mechanism uniting individuals into social structure This approach to culture is reflective of the metaphor of organization as an organism in which culture reflected in shared values and beliefs within the organization is considered a dimension that contributes to system balance. The implication is that changing the environment can affect a change in the organizational culture, making it stronger or weaker, or more or less supportive of organizational effectiveness and goals Regardless of the differences between the machine and organic model, according to Smircich's analysis these two approaches are grouped together as viewing culture as a variable Although anthropological theorists Radcliffe-Brown and Malinowski had considerable theoretical disagreements, many present-day organizational theoreticians consider them more alike than different (Sanday 1979 Srnircich 1983, Allaire and Firsirotu 1984, Morgan 1986) Both thought of cultures as wholes and used the term function in considering the social effect of customs and institutions (Sanday 1979) Both saw organizations as focused on sociocultural qualities that developed within the organization (Srnircich 1983) The view of culture as variable seems to be the most significant element of their analysis and one which has ramifications in how organizational culture theory is applied in modem organizations Srnircich (1983) suggests that the alternative to defining culture as an organizational variable is a position that views culture as a root metaphor for conceptualizing organization From an anthropological perspective, Srnircich includes Goodenough's ethnoscience (culture is a system of shared cognitions) as a category of organizational cognition Geertz' s symbolic anthropology (culture is a system of shared

PAGE 40

26 symbols and meanings, and symbolic actions need to be interpreted in order to be understood) as a category of organizational symbolism, and Levi-Strauss' struct uralism (culture is a projection of the mind's universal uncon scious infrastructure) in a category she calls unconscious processes and organization (1983:342) Each ofthese corresponds with a type of organizational theory: cognitive organization theory as an example of organizational cognition symbolic organization theory as organizational symbolism and transformational organizational theory as unconscious processes and organization respectively Allaire and Firsirotu (1984) also offer an analysis ofthe intersection of organizational and anthropological theory, with a s lightly different emphasis The major division they make in analyzing anthropological and organizational theory is between viewing culture as : I) a sociocultural system a component of the social system, manifested in the behavior and products of behavior ; or 2) an ideation system in which cultural and social realms are distinct but interrelated (Allaire and Firsirotu 1984 : 196). Malinowski and Radcliffe-Brown are among the theorists identified as defining culture as a sociocultural system as are anthropologists ofthe historical-diffusionist school (Boas, Benedict Kluckholn, and Kroeber) and the ecological-adaptationists (White, Service, Rappoport and Harris), who were absent from the Smircich analysis. The cognitive symbolic and structuralist schools are included among the theorists who define culture as a system of ideation, as is Wallace representing the mutual equivalence school (also absent from the Smircich analysis)

PAGE 41

27 This distinction between culture as a system of ideation and culture as a sociocultural system can be troublesome The idea of culture as distinct and separate from the social system can be difficult to grasp and may seem subtle at best An article by Keesing (1974) offers clarification regarding the issue of separate or integrated social and cultural systems comparing and contrasting the anthropological theorists who consider cultural and social realms and those that consider sociocultural systems as one Even more enlightening is Meek's ( 1988) article about the conceptual distinction between culture and social structure Meek argues that although both an organization's structure and culture are socially created the distinction between social structure and culture is crucial. Meek agrees with Smircich ( 1983) that approaches to organizational theory can be divided into two camps : those who believe that culture is a variable within an organization and those who believe that culture is something an organization is. The first group of theorists thinks of culture either as something imported into an organization or created by management to serve a function of social cohesion. The second group approaches culture as ideational and believes that culture is negotiated implicitly and emer ges from social interaction In this latter view, interpretation of organizational culture must be embedded in the contextual richness of the social life of organizational m e mbers Given this distinction, Meek finds it crucial to further distinguish between a view of culture as inclusive of or separate from social structure Interestingly this division occurs along similar lines as the division between culture as va riable in an organization and culture as root metaphor for organization Theorists who define culture as ideational also conceive of it as separate from the social structure

PAGE 42

28 (Allaire and Firsirotu 1984). Theorists who adhere to a basically structural-functionalist theoretical paradigm view of culture as an organizational variable, believe it to be integrated into the social system and as s ume that social and structural components are fully integrated and synchronized with the ideational components of the organization Meek (1988 :464) sees this approach as problematic for several reasons : I if organizational culture and social structure are integrated then it is problematic to take into account shifts in social structure which may occur without corresponding shifts in member norms ; 2 if culture is considered inclusive it denies the possibility of conflict between the ideas ideologies and values of organizational members and the organization's structure (formal and informal) "; 3 if the emphasis is on integration it is problematic to deal with conflict between individual and group interest and the power, authority and control structured in the organization Srnircich (19 83) calls this distinction the difference between culture as something an organization has and culture as something an organization is This distinction in our definitions of culture leads us toward different types of organizational analysis and different understandings of how interpretation of culture might be valuable in an organizational context. The most common approach to organizational culture focuses on managing controlling or fixing the culture of an organization Titles to that effect abound : Beyond the Quick Fix (Kilmann 1 9 84) Managing the New Organization (Limerick and Cunnington 19 9 3) Gaining Control ofthe Corporate Culture (Kilmann Saxon Serpa 1 9 85) Culture Traits Strength and Organizational Performance : Moving Beyond Strong Culture (Saffold 1 9 88) Efficient Cultures : Exploring the Relationship between Culture

PAGE 43

29 and Organizational Performance (Wilkins and Ouchi 1983), Managing Culture : The Invisible Barrier to Strategic Change (Lorsch 1986) These writings are representative of the notion that organizational culture is something that can and should be managed for the purpose of increasing organizational effectiveness Schein ( 1985 : 2) writes that "the only thing of real importance managers do is to create and manage culture In this view culture is a top-down function in organizations Schein (1985 : 2) continues We must recognize the centrality of this culture management function in the leadership concept. This perspective emanates jointly from the idea that culture is a variable something an organization has and that culture is integrated into the social structure making it the domain of the organization s leaders and managers This strongly reflects the explicit, goal-oriented purposive perspective of the rational paradigm (Walck 1989). Morey and Morey (1994) tell us that managers largely believe that organizational culture can be imposed from the top downward and that managers tend to think of culture only in terms of the middle and upper levels of an organization. Although some (e g Walck 1989, Schein 1985) suggest that this approach to organizational culture is not inherently anthropological it seems representative of a particular genre of anthropological theory the applied anthropology of the Hawthorne studies that is not as widely accepted within anthropology as it once was. In contrast stands the perspective of culture as implicit interactional and unconsciously generated Both approaches consider culture shared but this view of culture as ideational denies the kind of consciously created cultural causality suggested by the rational paradigm and postulates instead that culture is a negotiated process rather

PAGE 44

30 than one created by organizational authority or universal rules Similarly Daft and Weick (1984) argue that interpretation is a critical element in human organization and that people in organizations try to interpret what they have done define what they have learned and determine what they should do next According to their interpretation model of organizations decision making is part of the information and interpretation processes in organizations Daft and Weick define the learning process in organizations as involving scanning the environment for data giving meaning to that data and learning which involves taking action (1984 : 286) Although they argue that the process of organizational interpretation is more than what occurs at an individual level in an organization, Daft and Weick s model defines organizational interpretation as the domain of managers rather than as existing throughout the organization They assume that it i s managers who constitute the interpretation system rather than interpretation being a process that occurs at all levels of organization If, however we view culture as both process and product and as emerging from social interaction at all levels of the organization their proposition that interpretation is the process through which information is given meaning and actions are chosen becomes quite useful in organizational analysis. Smircich ( 1983 : 348) describes this concept of culture developed in anthropology as culture as root metaphor for organization, an epistemological device in much the same way as the organismic metaphor serves as a basis for the development of the s y stems theory perspective on organizations ." Several theoris t s reference Smircich s definition of culture as metaphor (e g Morgan 1986 Barley et.al 1988 Meek 1988) Srnircich s analysis of the intersection of anthropological culture theory with traditional

PAGE 45

31 organizational theory is informative but her point that culture is metaphor for organization is troublesome Culture may be ideational and emergent from social interaction, but culture is not a device and surely is more than a metaphor for organization. If culture is considered a metaphor for organization, then culture is simply illu s trative of organization. If culture is a process and product of organization, then it is part ofthe organization's reality Walck (1989 :3) speaks directly to this point : Culture is more than a metaphor : it is a social proce ss and product that demands attention in o rganizational contexts. Knowledge Utilization If culture is viewed as described above as a negotiated process and a product that is emergen t from interactions throughout an organization, then how information is g iven meaning in organizations becomes an important point of emphasis in the analysis of utilization patterns Issues of knowledge utilization in public policy are not unique to outcome evaluation in children s mental health and in fact s pan the range of research that informs policy issues. Underutilization of evaluation research is frequently cited as a problem central to policy research (Patton et al. 1 977; Weiss 1977 ; Scheinfeld 1987 ; Schensul 1987 ; Whiteford 1987 ; van Willigen Rylko-Bauer McElroy 1 989) Before examining what makes knowledge useful and how research can be structured to promote utilization it is necessary to consider definitions for and understandings of the concept of knowledge utilization Knowledge utilization is often described as relating directly to decision making and utili zatio n defined as basing specific deci s ions on specific information The knowledge utilization literature refers to the direct

PAGE 46

32 or specific use of information for decision-making purposes as instrumental use (Rich 1977 ; Weiss 1977 ; Pelz 1978 ; Leviton and Hughes 1981) Although utilization can be thought of as affecting decision making, a number of researchers have broadened the definition of utilization beyond that of having an immediate and tangible impact on decision making For example Patton et al. (1977) distinguish between the direct impact of evaluation research on concrete program decisions and a more indirect impact which serves to reduce uncertainty for decision makers Rich ( 1977 : 200) offers a broad definition of use as information entering into the policy-making process, but distinguishes between instrumental use which involves specific ways in which information is used for decision making, and conceptual use, which involves influencing the policy maker without any specific use being documentable Pelz (1978) drawing from Knorr (1977) and Weiss ( 1977) makes finer dist i nctions in the concept of use by adding the mode of symbolic use which refers to using information in the legitimization of previous decisions and actions to conceptual and instrumental modes ofutilization Leviton and Hughes (1981) confine their definition of the use of evaluation results to the application of this research in program and policy decisions rather than by academicians or by the press In addition, their analysis adds the category of persuasive use to instrumental and conceptual use as a way of describing the use of evaluation results to convince others to support or defend a political position The move toward outcome accountability within human services systems is steeped in language of systems change and the need to take action in ways that would support this change. This emphasis on change comes in the form of demands for greater

PAGE 47

33 system responsiveness accomplishments and progress There is an explicit link made in the outcome literature between outcomes and action which is described by the relationship between the availability of outcome information and a system s ability to be more responsive to those it serves (Schorr Farrow, Hornbeck, et al. 1994); human service administrators ability to know what they are accomplishing (Burchard and Schaefer 1992) ; and agencies ability to be more accountable for progress on specific outcomes for the children and families they serve (Nelson 1993) Outcome accountability is associated with improving the ability of human service organizations to accomplish their intended purposes. Although there is no reason to believe that conceptual, symbolic or persuasive modes of utilization are not at play in this utilization of outcome information, the emphasis placed on linking outcome evaluation to systems change suggests outcome evaluations are most frequentl y viewed in the context of the decision-based or instrumental mode of utilization. The R e sults of Knowledg e Utili zatio n The decision orientation of outcome evaluation means that understanding issues of knowledge utilization in this context necessitates consideration of what the expected results of utilization are Schensul (1987) stipulates that utilization involves the uses of knowledge for a particular purpose or to defined ends. In this sense, utilization can be linked to change in the form of interventions which seek to improve client outcomes or improve program effectiveness. It is limiting however, to link knowledge utilization inextricably to change as Glazer Abelson and Garrison (1983) do in their analysis because it does not leave open the possibility of using knowledge in decisions which

PAGE 48

34 support stability rather than change. Van Willigen {1984) suggests that knowledge can be utilized for purposes of producing either change or stability in cultural systems, making it possible for the defined ends of knowledge utilization to be stability as well as change This view of utilization as producing either stability or change is discussed by Hernandez and Hodges ( 1996), who argue that outcome information can suggest the need to modify service planning and delivery through mid-course corrections as well as support the maintenance of systems as they are currently operating In their analysis, decisions made on the basis of outcome information involve taking action on what has been learned, but this action can take the form of decisions to change or not change aspects of service planning and service delivery Hernandez and Hodges believe that this decision-making process is best understood as a continual process of working toward improved results rather than an end result and that utilizing the results of outcome evaluation should be embedded in day-to-day management rather than seen as a one-time process. This approach to outcome evaluation suggests the use of outcome information in the context of self-evaluation so that outcome information provides an organization with opportunities for learning and self-correction Usher (1993b) provides a comprehensive description of how an organization's self-improvement strategy should be developed, suggesting that this type of internal evaluation strategy will enable a service system to continually improve its service-delivery processes C onverting Knowledg e to Action If it is accepted that the action resulting from knowledge utilization in outcome accountability is inclusive of both decisions to change and not change aspects of service

PAGE 49

35 planning and delivery then the real issue of utilization seems to be how to move from the research domain into the action domain which contributes to more effective program operations The need to understand "the means by which knowledge is converted into action" was a critical issue identified by Rylko-Bauer van Willigen, and McElroy (1989 : 1) in their discussion of strategies for increasing the utilization of research focused on solving practical problems. The literature identifies linkages between knowledge and action as critical to utilization. Writing about how knowledge can be used or applied to solve some practical problem, van Willigen (1984) suggests that linkages between research policy, and action exist in which research produces information information informs policy formation and policy acts as a guide for consistent action Schensul (1987) writes about knowledge brokers or systems linkers" who bridge between research and program development. In writing about how decisions are made in organizations Daft and Weick describe organ i zational interpretation as "a process through which information is given meaning and actions are chosen (1984:294) In this respect interpretation might be considered a process of linkage between information and action Weick and Browning (1986) discuss the differences between argumentation and narration in organizational communication defining argumentation as based on the rational perspective which presupposes that decision making is clear-cut, inferential, and implies a hierarchical system in which some are more qualified to judge than others. In contrast the authors describe narration in organizational communication as based on interpretation and understanding which presupposes all people have narrative capacity and the ability to judge and implies a less

PAGE 50

36 stratified organization than the process of argumentation The differences between narration and argumentation have important implications in understanding the process of linking research and program development because the two paradigms suggest differences in how information guides action Weick and Browning suggest that narration has power in organizations because it ties together complex experiences and "preserves their interactions in a compact summary that can be reconstructed starting from any one of its parts (1986 : 250) and that the process of narration in organizations may create as well as carry culture They argue that although an increase in data bases and information may create an increase in argumentative rationality, an increase in shared experience may increase the incidence of narrative rationality They conclude that information must be ti e d together with good reasons and narration so that it can be processed and remembered This contrast of argumentation and narration in organizational communication is paralleled by the contrast between regulative and generative approaches to planning (Uzzell 1990) Regulative planning is defined as the implementation of formal plans which rely on coercive or institutional power for implementation Generative planning by contrast relies less on top-down argumentative styles of communication and is instead based on less formal operational information generated at all levels of organization Similar to the narrative rationality described by Weick and Browning (1986) generati v e planning implies an ongoing process of interpretation and communication that serves to guide decisions by linking information and action Also relating to the process of l i nking research and program development Caplan ( 1977 ) argues that problems of utilization are largely non-technical and that bridging the

PAGE 51

37 perspectives of researchers and policy makers is the single most important step in increasing research utilization Caplan's (1977) discussion of the need to bridge varying perspectives recognizes that the perspectives of researchers and policy makers may differ. More recently Grasso and Epstein (1993) write that one of the significant challenges to integrating information technology in human services is recognizing that there is conflict regarding information utilization at different levels of an organization. Using Patti's (1983) identification offour levels of hierarchy in human service organizations, they argue that information utilization must successfully mediate the conflict between different organizational levels in order to support strong performance at each level. Specifically Grasso and Epstein ( 1983 :24-27) describe these conflicts as : 1 Conflict at the direct service level between client demands for more service and accountability demands for more efficiency. 2 Conflict experienced at the level of supervisors of direct service workers resulting from their being held responsible for outcomes they do not have direct control over 3. Conflict at the level of program management resulting from the need to preserve professional autonomy among service providers and the responsibility to report and control results of service provision 4 Conflict at the executive level in human service organizations where concerns rest more with general operations than with day-to-day operations creating conflict between the need to defend the good work of their organization and still deal with less-than-desirable program outcomes Grasso and Epstein point out that with regard to the use of information in human service agencies, these conflicts often translate into a reluctance on the part of direct service providers to use quantitative measures of performance because these measures are perceived as unable to adequately reflect the difficulty and complexity of their work and

PAGE 52

38 what they accomplish. This is in contrast with the demand at administrative levels to be able to clearly describe the aggregated results of program performance Grasso and Epstein argue for outcome information systems that integrate the clinical, supervisory and managerial levels of human service organizations and conclude that for outcome information to support high-quality performance in human service organizations accountability for results must be partnered with providing information that helps improve performance at all levels of organization, thereby promoting staff learning and professional growth at each level of organization In this respect, a commitment to providing information that helps improve performance across the organization can be considered a link between research results and program development. Collaborative Approaches to Research Utilizati o n While Grasso and Epstein discuss information utilization in human service organi z ation, the principles of knowledge utilization enumerated in the anthropological literature address issues of research utilization more broadly and stress collaborative approaches to research as a link or bridge between research and action that is critical to increasing utilization of research results (Schensul 1987; Scheinfeld 1987 ; Whiteford 1987 ; van Willigen, Rylko-Bauer and McElroy 1989) For example Rylko-Bauer van Willigen and McElroy identify the need for research findings to be more relevant to potential information users as one of the most commonly noted factors in discussions of how to improve research utilization and argue that collaboration with the various stakeholders of research is an important factor in making research more relevant In

PAGE 53

addition to collaboration, the authors identify seven factors as important strategies for increasing utilization : 1 Advocacy for research findings researchers taking on the role of advocates for their research findings and recommendations; 2. Agency factors-a focus on understanding the nature of the policy makers' organization by whom how, when and why organizational decisions are made ; 39 3 Communication factors the need to consider how the communication media report language identification of relevant decision makers, and phrasing of results in action terms might affect results ; 4. Community and political factorsa need to understand the potential impact that relative power bases, such as the relationship between the client requesting research and the community or group affected by the research might have on utilization of the results; 5 Ethical issues understanding that policy research is value laden and awareness of the potential for both intended and unintended consequences of research utilization; 6 Research process factorsunderstanding how flexibility in methods credibility of the process and findings quality, usually defined in terms of methodological rigor and applicability or the degree to which the research informs issues people feel they can do something about relate to utilization of results ; 7 Time factors recognition of how timeliness of results is important, particularly in short-term instrumental use. Whiteford (1987) identifies similar criteria for research utility noting that in her model none have priority over the other These concepts include six points of agreement or understanding affecting the conduct of the research reached through collaboration between the researcher and the client:

PAGE 54

40 I Relevance that the research relates to the concerns of potential users ; 2 Credibility that the researcher and user agree on the types of information needed the process by which the information should be gathered the types of measurement that would support conclusions ; 3 Process an understanding of how and to whom research findings will be disseminated ; 4 Access an understanding of who should receive the research findings ; 5 Constraints that recommendations based on the research findings are formulated with an understanding of the kinds of changes that are possible ; 6 Perspectivethat recommendations based on the research findings address issues of immediate and long-term application Based on the literature understanding the means by which knowledge is conv erted into action and learning more about bridging research and program development are integral to providing information that helps improve performance across the organizati o n From this perspective a better understanding of the process of transforming knowledge to action can be considered an appropriate focus for the study of utilization of outcome information The Application of Anthropological Theory in Information Pathways How anthropologists investigate the cultural processes of organizations and where we look for culture in organizations is dependent on how we apply anthropological theory about culture in the context of our work Because there is no univer s al definition of culture within anthropology and no single ethnographic technique for investigating culture there is, similarly no s i ngle anthropological approach (Jordan 1994 ) Although anthropologists might agree that culture can be viewed as shared and systematic one

PAGE 55

41 anthropologist may look for culture in material artifacts another in organizational ritual yet another through the use of domain an a lys i s Wright (1994) belie ves that although anthropology takes many approaches to culture any anthropological view of organizational culture would include such considerations as its systematic nature its permeation of all aspects of organization its persisting over time, and its shared quality (although in anthropology being shared would not require consensus throughout the organ i zation) In general terms I believe organizational culture can be thought of as a web of interwoven and hierarchical culture groups that is a part culture in one or more larger cultures which might include regional or national cultures (see discussion in Jordan 19 9 4 : 5) Organizational culture does not exist in isolation ofthe world around it and must be considered as embedded in a cluster of lar g er cultures The definition of culture applied in this research rests upon thre e assump t ions about the nature of cultural processes in organizations The first assumpti o n is that c ultur e is both a process and a product of organization From this we can understand that culture both impacts and is impacted by organization Second the assumption is made that culture is emergent from social interaction which suggests that culture is implicit and that m e aning is constructed through interaction Finally culture is a negotiated process that occurs at all levels of organizati o n This su g gests that while cultural processes in org anizations might be influenced by managers, they cannot be easil y controlled or dir e cted in the interest o f management or any other subculture within the org anization Put differently this implies that culture cannot be created by authority or rule

PAGE 56

42 Information Pathways is focused on issues of knowledge utilization The concept of culture that seemed most useful in this context is one drawn from Spradley (I 979) who defines culture as acquired knowledge people use to give order to their world, to interpret their experiences and to generate social behavior. The first step in understanding how this definition of culture has been applied in this research is to understand the meaning of knowledge This research draws on Schensul's (1987 : 4) definition of knowledge as what is known to be true or meaningful by two or more people as tested through time and as consisting ofbeliefs and experience commonly shared by groups If we assume, however that cultural meaning is constructed through interaction then the emphasis in Spradley's definition of culture is more appropriately shifted from a focus on the knowled g e itself to consideration of the verbs in the definition : using knowledge to int e rpr e t experience and gen e rate social behavior This shift in emphasis directs the attention of this study on the processes of interaction : written and oral communication, description and explanation of experiences and responses to experience observed behaviors Geertz argues that in understanding culture attention should be paid to the flow of behavior and social action because this is how cultural forms find articulation ( 1973 : 17) This focus allows Information Pathways to concentrate attention on the flow of outcome information within the TCMHP with the aim of understanding this process from multiple perspectives. Geertz ( 1973) suggests that the process of analysis should be an attempt to understand or interpret the significance of events and experiences from the perspectives of the different individuals involved He describes culture as webs of significance and

PAGE 57

43 describes the analysis of culture as an interpretive search for meaning in those webs (1973 : 5) The definition of culture applied in this research is therefore expanded from Spradley's (1979) definition of culture as acquired knowledge to culture as the process of using acquired knowledge to give order to the world interpret experiences, and generate social behavior In this way culture remains ideational, but it is also focused on the actions of the individuals involved rather than the knowledge in their heads This study strives to understand factors which influence the utilization of information in local-level serv ice planning and delivery from the multiple perspectives ofthose involved in the process and to develop strategies for more effective use of outcome information with the awareness that cultural processes occur at all levels of organization Utilization as a Cultural Domain The thre e guiding research questions for this research were designed to examine the differential use of outcome information at the local level in the Texas Children's Mental Health Plan (TCMHP) The literature on research utilization while offering important insight on how to improve utility is somewhat unclear about what exactly is being used -data information, or knowledge These terms are often used without making distinctions am o ng them For example in an article on increas i ng research utilization, Scheinfeld (1987) shifts interchangeably between the terms data and knowledge Writing about improving the utilization of research results Whiteford ( 1987) uses both data and information to offer insight into the relationship between researchers and research users Rylko-Bauer van Willigen and McElroy (1989) focus their discussion of research utility

PAGE 58

44 on knowledge and suggest that in order to improve utilization it is necessary to better understand how users move from knowledge to action For the purposes of understanding the different patterns of outcome utilization observed at the local level in the TCMHP, it was useful to begin by distinguishing between the terms data information, and knowledge, so that specific linkages necessary for optimal utilization could be more fully explored For this study, outcome data can be understood as the domains of measurement that define the outcomes of interest to the child-serving organization : who is served ; what services are provided; and how change is measured In the TCMHP, participating local mental health authorities submit data to the state offices of research and evaluation at an indi v idual or case level and the data are aggregated to a regional and state level. An example of this would be the reporting of demographic information about the child, what services the child is receiving, and measurements of change in the child's functioning as measured by changes in school behavior, juvenile justice involvement or behavioraVemotional functioning Outcome information can be defined as the analysis and s y nthesis of the data on outcome results into a useable form. This process can include the format and presentation of data the language of presentation and the level at which the data is aggregated For example in the TCMHP, th e state research and evaluation staff receive outcome data from l oca l mental health authorities aggregate this data and return information on outcome results to local mental health authorities in the specific form format and l anguage of monthly and quarterly reports as well as periodic special requests for outcome information

PAGE 59

45 Knowledge in this study draws its definition from Schensul (1987 :4) who defined knowledge as what is known to be true or meaningful by two or more people" and as consisting of beliefs and experiences shared by groups ." Knowledge can be thought of as ideas inferred from information and connotes awareness, understanding and interpretation of information provided by th e outcome data In the TCMHP lo cal-level knowledge of outcomes or what is known to be true and meaningful about outcomes, exerts a potent influence on how outcome information is used The acquired knowledge of outcomes however, is not simply the outcome information that has been provided to the sites in the reports from the state As knowledge is acquired the outcome information is interpreted and given meaning in substantiall y different ways at each of the local mental health authorities investigated for this study Interpretation this process by which information is given meaning can thus be viewed as the primary link between information and knowledge This link between information and knowledge places the understanding of differential outcome utilization firmly in the cultural domain of each local mental health auth o rity participating in this study

PAGE 60

46 CHAPTER THREE STATE PERSPECTIVES ON OUTCOME INFORMATION UTILIZATION Establishing the Texas Children's Mental Health Plan The Texas Children s Mental Health Plan (TCMHP) was established in 1992 as Texas first state-wide, state -funded effort to develop and implement a community-based system of public mental health services for children and adolescents with severe emotional disturbance and their families (Rouse 1996). State-level informants emphasized that the goal of the TCMHP has been to establish a continuum of services as needed in each community based on a systems of care model. TCMHP services have been organized into three components : 1) core mental health services 2) services to children referred from the juvenile justice system, and 3) early intervention and prevention services State-level respondents stressed that an essential feature of the TCMHP is a list of outcomes to be measured for each of the components of the TCMHP In the late 1980s a number of state and local-level efforts served as the catalyst for the establishment of the TCMHP, but informants at the state level generally credited the work ofthe Mental Health Association ofTexas (MHAT) with providing the foundation for the TCMHP by documenting the lack of children's mental health services in Texas Kids Do Count: How Texas Serves Children and Adolescents with Severe Emotional Disturbances (Menta l Health Association of Texas 1989) specifically documented the

PAGE 61

47 problems that members of local communities were having in accessing children's mental health services as well as the money that major child-serving agencies in the state were spending on mental health services for children and families. This report confirmed that the largest amount of public money was going to provide in-patient psychiatric care and residential treatment purchased primarily through the private sector for children under the care of child service agencies such as juvenile justice child protective services, and education Prevention and early intervention and moderate community-based interventions and treatment were largely unavailable In addition, state-level respondents concur that MHAT's subsequent efforts to advocate for the establishment of collaborative and interagency children s mental health services with state agencies and the state legislature was instrumental in the establislunent of the TCMHP State-level sources described the development of the TCMHP as occurring in three stages Initially the Texas Legislature provided a line item for children's mental health services in 1990 by funding five demonstration programs. In 1992, the Texas Legislature appropriated monies to the Texas Department ofMental Health and Mental Retardation (Texas MHMR) for the implementation of the TCMHP in 16 sites At the same time, 29 sites were given funds for system development and were asked to submit a needs assessment for their communities during FY 1992 In 1994 the Texas Legislature provided funding to Texas MHMR for the purpose of funding 45 community service sites across the state effectively initiating a statewide system of services. The state initiated services through Community Mental Health Centers (CMHCs) in areas of the state where these centers were established For geographic areas with no established CMHC, the state

PAGE 62

48 contracted with the Community Service Divisions of state mental hospitals for the implementation of the TCMHP in order to provide coverage throughout the state. Both C:MHC and the state-linked local mental health authorities met identical requirements for participation in the TCMHP and were funded at the same rate Texas MHMR distributes funding on a contract basis with all 45 local mental health authorities across the state at an annual rate of$1,489 per child Informants at the state level pointed out that a significant aspect of development of the TCMHP was the placement of the funds with Texas MHMR. When the initial request for TCMHP funding was made to the Texas Legislature, the monies were to be divided among the budgets of eight collaborating state child-serving agencies. There was, however concern that this fragmented approach would prevent a true interagency provision of community-based services The legislature placed all the money for the TCMHP in the Texas MHMR. budget to prevent this from happening, effectively making MHMR. the banker for these funds The TCMHP as an Interagency Initiative State-level informants representing child-serving agencies both inside and outside of mental health indicate that the TCMHP was conceived as an interagency initiative which emphasizes collaboration on the part of all child-serving agencies in the state Children with severe emotional disturbance often require services from multiple agencies Although ch ildren with severe emotional disturbance require mental health services, these services are thought to be most effective when they are delivered in the context of the larger and multi-dimensional needs of children and families. This requires a

PAGE 63

49 comprehensive and coordinated network of services provided through multiple agencies For example an adolescent with a mental health diagnosis of attention deficit hyperactivity disorder may also be involved in the juvenile justice system, need placement in special education programs at school and/or family preservation services to build family stability at a time of crisis The TCMHP is structured, therefore, to underscore the provision of mental health services for children as an inter-agency collaborative effort. In reporting on a process evaluation ofthe TCMHP completed in 1994 Ellmer Lein and Hormuth (1995) characterize the interagency focus ofthe TCMHP as a purposeful response to the established pattern of removing children with severe emotional disturbance from their families which results in over-reliance on expensive residential services State-level respondents stress that the focus on interagency collaboration was an effort to address the fragmented and categorical patterns of service delivery common to human service delivery systems According to state-level sources an essential feature ofthe TC.MHP is the participation at the state and local levels of child-serving agencies, advocates, and consumers i n management teams with the express purpose of making collaborative decisions regarding TCMHP activities At the state level, nine agencies form the State Management Team (SMT) of the TCMHP These state agencies are the Texas Commission on Alcohol and Drug Abuse Texas Department ofHealth, Texas Department of Human Services, Texas Department ofMental Health and Mental Retardation, Texas Education Agency Texas Juvenile Probation Commission, Texas Rehabilitation

PAGE 64

50 Commission Texas Youth Commission, and the Interagency Council on Early Childhood Intervention. This structure is mirrored at the local level where local mental health authorities receiving TCMHP funding were required to form organizations composed of local representatives of the same state agencies This process resulted in the formation of Community Management Teams (CMTs) designed to guide the TCMHP at the local level (Rouse 1996) Both the 16 service implementation sites and 29 system development sites were required to establish CMTs in FY 1992 The TCMHP Evaluation The evaluation of the TCMHP is described by informants at the state level as a continuous evaluation system in that it is designed to support an accountable service delivery system through an evaluation-stakeholder feedback loop which provides evaluation information on a timely and regular basis for the purpose of program adjustment and improvement (Rouse 1996) State-level respondents emphasized that the evaluation was developed in order to provide children's mental health administrators at the state and local levels with information about children s mental health services and to demonstrate accountability to the consumers and funding sources of TC:tvfHP services The evaluation of the TCMHP was developed and is guided by an interagency committee consisting of an array of stakeholders State-level respondents indicate that six questions guided this committee in directing evaluation activities : 1) what are the characteristics ofthe children served ? 2) what services did they receive? 3) what are the effects of services on child functioning ? 4) which services work best for which children ?

PAGE 65

5) to what extent are the consumers satisfied with services? and 6) what are costs of the services? 51 An essential feature of the evaluation is a list of outcomes to be measured for each of the components of the TC:MHP: core services, juvenile justice referrals and early intervention and prevention services Rather than providing funds for the provision of services the TCMHP provides funds for the achievement of certain outcomes as a result of the delivery of child and adolescent services For example, rather than funding a local mental health agency to provide counseling or family preservation services the state funds the achievement of improved behavioral and emotional functioning of child and adolescent clients as measured by improved Child Behavior Check List (CBCL) scores Similarly, rather than funding services to children and adolescents with a history of arrest or a history of difficult behavior in school, the TCMHP funds the avoidance of re-arrest for clients with a history of arrest or the improvement of school behavior for students with a history of difficult school behavior. The theory behind the TC:MHP evaluation is that providing local mental health authorities with timely and regular information about the results of their service delivery would allow them to improve results through incremental adjustments in service delivery strategies The achievement of outcomes or results of service delivery were written into the TCMHP from its very beginning to assure the stakeholders, including interagency partners and the state legislature that the effectiveness of services was bein g measured and that decisions about the TCMHP were being assisted by evaluation data (Rouse 1996) Texas MHMR s unique position as the banker of the TCMHP funds originally targeted for other state child-serving agencies placed this agency

PAGE 66

52 in the position of needing to demonstrate on a regular basis and at both the state and local levels that these funds are being used for the intended purposes. State-level TCl\1HP staff within mental health also cite a commitment to the principles of Total Quality Management (TQM) as a guiding work philosophy and integral to the establishment of a continuous evaluation for the TCl\1HP (Rouse 1996) TCl\1HP staff stressed that one of the hallmarks of TQM is the incremental measurement of work activities and the use of this information in modifying work processes to increase productivity and effectiveness State-level respondents describe the collection and dissemination of outcome evaluation information to Texas MHMR. staff and local-level program directors as a crucial component in providing feedback to local managers so that service delivery can be modified for the purpose of providing more effective services Based on experience with the 16 TCl\1HP sites established in 1992, the evaluation design and measures were modified and streamlined in an effort to reduce the burden of data collection on the service providers and to administer only those measures which were considered key indicators This revision was informed by a telephone survey to local program directors asking their thoughts about the meaningfulness difficulty of completion, and utility of the evaluation measures Although outcomes were written into the evaluation of the TCMHP from its inception emphasis has recently been placed on the local sites' achievement of certain level of outcome results State-level TCl\1HP staff described this shift as consistent with managed care approaches to providing mental health services in that the contracts with local mental health authorities focus on purchasing the outcomes of services rather than

PAGE 67

53 buying the delivery of actual units of service Beginning in FY 1996, the Texas MHMR contracts with local mental health authorities included specific performance results for each site. Reports on each site's progress toward these contract goals were included in the evaluation throughout the year, but no sanctions were applied for local mental health authorities that fell short of these goals and were, therefore, out of compliance with their contract. Beginning in FY 1997, however, local mental health authorities are being asked to repay funds at a per-child rate if the contract outcomes are not achieved. The targeted performance outcomes contracted for Fiscal Year 1997 are listed on the table below (Texas MHMR 1996). It is the relative utility ofthis outcome information to each ofthe sites participating in this study that is the focus of Information Pathways. Table 1 : Performance Contract Measures for FY 1997 Outcome Measure Contract Target Number of Children Served in Target Population Specific targets set for each l oca l mental health authority Number of Children Served in Fir st Time Offender Specific ta r gets set for each l ocal mental health authority. Program Parent Satisfaction 80% parents satisfied Child Satisfaction 70% children satisfied Coll a teral (other child serving agencies) Provider 65% co llateral providers satisfied Satisfaction Schoo l Behavior 60% children with improved school behavior Rearrests 75% o f children with history of arre sts avoiding rearrest Behavioral-Emotional Fu n ctioning Improved behavior-emotional functioning as measur ed by a 6.5 point reduction between average beginning and ending CBCL scores A review of Texas MHMR documents indicates that the contract targets for each ofthe outcomes shown above was established through a study ofbaseline data collected

PAGE 68

54 though the TCMHP (Texas MHMR 1995) Baseline figures were defined by Texas MHMR as the figures that represent the entire group of children about whom a given measure has been submitted within a given time frame For each of the contract targets that was set, data were analyzed from children registered for TCMHP services at the 16 initial service sites from September 1, 1992, through August 31, 1994 and from aU 45 local mental health authorities beginning September 1, 1994 In the case of each outcome, the state target was set slightly below the performance established through the analysis of baseline figures For instance for the Parent Satisfaction target baseline data was taken from four reports in which the average percentage of parent satisfaction ranged from 84 3 percent to 88 percent. The contracted target for Parent Satisfaction was set at 80 percent. Similarly for the Improved School Behavior target baseline data was taken from three reports in which improved school behavior was reported in between 61.3 percent and 62 6 percent of students. The state contract target for Improved School Behavior was set at 60% Interviews with state-level TCMHP staff underscored the fact that although a more traditional one-time summative evaluation was considered as a way of evaluating the success of the five pilot sites, the need to provide stakeholders with regular and timely information shifted the emphasis of the TCMHP evaluation toward creating a continuous evaluation system that would serve as a permanent part of the service delivery process State-level TCMHP respondents describe the basic evaluation design for the TC.MHP as a pre-treatment post-treatment and follow-up measurement of consumer demographics, history, and outcomes (Rouse 1996) In addition satisfaction with services data are

PAGE 69

55 gathered on a cross section of consumers at a given point in time (Rouse 1996) A multimethod, multi-rater approach to measurement is employed to collect information from servi c e providers children, parents, and collateral providers using i nterviews checklists and rating scales ( Rouse 1996) The data for the evaluation is collected primarily by service providers at preand post-treatment. The evaluation is staffed at the state level by a team of professionals who work for the Texas MlllviR Research and Evaluation office State-level TCMHP respondents identify several methodological limitations of the evaluation including the fact that there are no control groups and random assignments to treatment because treatment plans are based on need In addition evaluation measures such as the ending CBCL scores are not always administered according to plan due to the practical constraints of providing services Finally the reliabil i ty and validity of some of the evaluation measures such as the satisfaction measures are unknown. The Evaluation Feedback Loop State-level informants indicated that the process of implementation at the initial 16 TCMHP sites increased the awareness that stakeholders at the state and local level needed a basic set of relevant and timely information to assist with managing the TCMHP This was largely due to the growing emphasis on quality management and managed care operations at Texas :MHMR, both ofwhich emphasize the involvement ofvarious stakeholders and the use of data in assisting managers with decision making As a result a participatory evaluation approach was used as a way to get stakeholders involved as user s of the evaluation products while creating a dialogue between evaluators at the state level and state and local-level stakeholders (Rouse and Toprac in press) This approach is

PAGE 70

56 characterized by what state-level TCMHP staff describe as the evaluator-feedback loop Stakeholders of the TCMHP have been identified by state-level informants as including state-level mental health staff, as well as local program directors and service providers, state and local community management teams, state legislators and their staffs The stakeholder feedback loop is described as a process by which stakeholders raise questions about TC.MHP programs evaluators collect and analyze data, evaluators collaborate with stakeholders in using the results and stakeholders make decisions based on conclusions drawn from the evaluation results Currently, the TCMHP evaluation includes the communication of evaluation results through : 1) a monthly Children's Mental Health Services Report and Contract Outcomes Report available on-line to each local site from a statewide data base, which provides a summary of key indicators in the areas of numbers served, demographics and outcome measures for each mental health authority 2) a Quarterly Service Report provided in print format which presents a comparison of all 45 local site results across the contract outcome measures, and 3) special reports produced at the request ofTCMHP managers and other stakeholders These publications are seen by state-level TCMHP staff as the essential tools for facilitating the evaluation-stakeholder feedback loop State-Level Perceptions ofLocai-Level Information Utilization The state-level description of the feedback loop reveals much about state-level perceptions of outcome information utilization in general and local-level utilization specifically Throughout the interview and document data, the focus on using evaluation data in decision making is stressed State-level TCMHP staff stress the stakeholder

PAGE 71

57 feedba c k loop as the process by which questions are raised and managers subsequently make program decisions One state-level informant commented that, "If evaluation data is not used in decision making, then the evaluation has not reached its true goal." State level research and evaluation specialists for the TCMHP report that it is their belief that the TCMHP evaluation has helped local program directors define the services they are offering In general the state-level perspective on the usefulness of the TCMHP evaluation at the local level can be summarized as follows : 1) it is useful in ensuring that children and families receiving services across the state are those targeted as the priority population by the TCMHP, 2) it is useful as a way of ensuring that local mental health authorities are accountable to their community and the funding agency for the services they provide, 3) it is useful in reinforcing interagency relationships at the local level and ensuring that services are provided through interagency and collaborative processes and 4) it is useful in increasing both access to and the effectiveness of services provided through the TCMHP The TCMHP was created to serve children and families that are most in need and the target population guidelines were developed to ensure that those most in need were the recipients of the TCMHP services. The evaluation information provided through the evaluation process tracks the number of children in the priority population who have received services during a given period of time The inclusion of a specific number of children within the priority population in the performance contract of each local mental

PAGE 72

58 health authority serves to ensure that the resources of the TCl\1HP are, in fact, being used to serve the intended children State-level TCl\1HP respondents stressed that the TCl\1HP and its system of continuous evaluation was intended to create "an accountable service delivery system." In conversation state-level TCl\1HP staff frequently referred to the importance of the evaluation being relevant to stakeholders so that stakeholders in the community, as well as at the state level, might be assured that decisions relating to service planning and delivery were grounded in data State-level staff indicate that the TCl\1HP emphasizes an interagency collaborative effort on the part of all of the child-serving agencies in the state The structure of the TCMHP at the local level, particularly the required establishment of a Community Management Team (CMT), served to reinforce the delivery of children's mental health services as a collaborative effort The continuous evaluation system of the TCl\1HP was conceived as one method of reinforcing the role of the CMT as the local overseer of the plan through the provision of feedback on the results of programs and services. Both the target population contract goal and the reports providing local mental health authorities with a summary of key indicators in the areas of numbers served, demographics and outcome measures for each mental health authority are viewed by state level TCMHP staff as providing local mental health authorities with information useful in evaluating both access to and effectiveness of local services State-level informants expressed the beliefthat when used most effectively, the evaluation reports can be used to make incremental adjustments and changes to local-level services

PAGE 73

59 Finally the evaluation is seen by state-level respondents as a vehicle for collaboration with local sites in providing community-based services. State level TCMHP staff expressed the belief that state level evaluators must collaborate with the stakeholders in using the results of the TCMHP evaluation and that project directors and service providers need education to enable them to read and interpret data more proficiently from clinical and administrative viewpoints

PAGE 74

CHAPTER FOUR :METHODOLOGY Background 60 Information Pathwa y s was part of the System Accountability Study a 5-year research project currently being conducted through the Research and Training Center for Children's Mental Health at FMHI. The purpose ofthe System Accountability Study is to investigate the impact that utilizing measurable outcome data has on service delivery and planning During the first two years ofthis project which was begun in 1994, case studies were conducted on three service systems with established outcome information systems The Texas Children's Mental Health Plan (TCMHP) served as one of these service systems and as a result of its participation interviews were conducted and documents reviewed at both the state and local levels regarding the best practices and challenges of the outcome accountability system The results of the System Accountability Study will be presented in a monograph titled Building Outcome Accountability in Children s Mental Health : Experiences from the Field (Hernandez and Hodges 1997 in press) An executive summary of the System Accountability Study is shown in Appendix 3 The guiding research questions for this research emerged from the data collected in the System Accountability Study Specifically it was observed by informants at the state level that although the outcome information system operated statewide through 45 local

PAGE 75

61 mental health authorities some of those sites used outcome information to a greater or lesser degree than others in decis i ons affecting service planning and delivery. It was this perceived differential use of outcome information in decisions affecting service planning and delivery that prompted the investigation of how organizational culture influences the use of outcome information at the local level in the TCMHP The guiding research questions developed for thi s study were : 1 What aspects of organizational culture influence the utilization of outcome information among local mental health authorities in the Texas Children s Mental Health Plan ? 2 How does outcome information impact decision making within high-use and low use local mental health authorities ? 3 What conditions support or impede the use of outcome information in service planning and service delivery ? The TCMHP was a particularly good service system in which to study issues of outcome utilization because each of the 45 local mental health authorities received outcome information in the same format and on a timely and regular basis This meant that the sites chosen for study in this research could be considered comparable in terms of their access to outcome information The Research Team The field research team for Information Pathways consisted of Sharon Hodges coprincip a l investigator for this study research coordinator for the System Accountability Study and doctoral candidate in applied anthropology at USF Dr. Mario Hernandez a clinical psychologist principal investigator for this study and principal investigator for the System Ac c ountability Study and Teresa Nesman Ph D student in applied anthropology

PAGE 76

62 at USF and graduate assistant for this project. Eah of the four field visits to local mental health authorities in the TC.MHP involved two field researchers as shown on the table below Table 2: Field Researchers for Local Mental Health Authority Site Visits Information Pathways Site Q Sharon Hodges Mario Hernandez Information Pathways Site R Sharon Hodges Teresa Nesman Information Pathways Site S Sharon Hodges Mario Hernandez Information Pathways Site T Sharon Hodges Teresa Nesman The research team responsible for data reduction and analysis included Dr. Joko Sengova a linguist and faculty member in the Departm e nt of Child and Family Studies at FMHI and John Staversky MA student in applied anthropology at USF and graduate assist a nt for this project as well as the three researchers who served as field research staff Miles and Huberman (1994) argue that in qualitative research validity and reliability rest prim a rily with the skills of the researchers They suggest a multi-disciplinary approach to assur e a broad focus and balance to a qualitative study Each of the five members ofthe research team brought different skills to the project and the broad base of these skills enhanced the research process.

PAGE 77

63 Data Collection Methods Data for this research were collected in the form of on-site and telephone interviews, direct observation, and document review Three domains of data were identified and triangulated in this study These domains are described in the figure below Figure 1 : Domains of Data Analysis for Information Pathways lnfonnation Pathways Data Collection : s tate and l oca l level semis tructured interviews (on si te and by teleph one) related directly to how th e out come infonnation system is actualized at the l ocal le ve l among high and low infonnation users begun May 199 6 and completed in April 1997 Document review : ongoing from May '95 for the purpose of uncovering the written history th e outcome infonnation system, the fonnal o rganizational structure, best practices a nd challenges System Accountability Study Data Collection : open -end ed interviews initiated in August 1995 with state and local level infonnants (on si te and b y telephone) for description of the outcome infonnation system and the fonnal organizational structure. Semi-structured t e lephone interviews for data on best practices and challenges co nducted through 1995-97 A summary of the data collection process at both the state and local levels of the TCMHP is shown on the table below The semi-structured interviews for this research began with state-level participants in the TCMHP by telephone in May 1996 These interviews served the purpose of exploring state-level perspectives on the definitions of outcome information use, high and low outcome information, and how outcome information is or isn't used in decision making regarding service planning and delivery at the local level. These data were also used to systematically obtain information about the 45 local-level sites and to identify explicit criteria for the site selection of four local mental health authorities in this study

PAGE 78

64 Table 3: Information Pathways Data Collection Process Research Task Information Data Collection Method Define formal structure of outcome Formal structure ofthe outcome Document review information sys tem as articulated at information system State and local level key informant the state level Formal st ructure of the TCMHP interviews How outcome information i s or isn't Direct observation used in local-level decision making regarding s ervice planning and delivery Elicit sta te-level definitions of l ocal-level How informant defines high and low State-level key informant interviews information utilization information users, what informant think s it means to be a high and lo w u se r, how informant knows if local mental health authority is a high and low u se r what makes local s ites high and low users Local-level si te se l ection: se l ect two high Investigate high and low use Document review information use si te s and two low characteristics : l evel of state Semi-structured local-level informant u s e sites as defined by state-level funding specia l s tudies requ ests interview s participants in the TCMHP urban/ruraVsuburban Elicit local-le ve l definitions oflocal-level How local-level key informants define Local-leve l key informant interviews high and low information information utilization do th ey see Direct observation u ser/ non u se r at the sites se lected themselves as high/low u sers, w h at for participation do informants think it means to be a h igh/low user how do they distinguish themselves as being either a high or low user Describe information sys tem as How l ocal-leve l systems defin e the Local -level key informant interviews actualized at the l oca l le ve l outcome information system how Direct observation outcome information is or isn't used in local-level decision making regarding service planning and delivery Local Mental Health Authority Site Selection The selection of local mental health authorities to participate in this study involved focused sampling, the inclusion of cases based on clearly specified rather than random sampling (see Silverman Ricci, and Gunter 1990) Initial discussions about the selection of the local-level sites to participate in this research were conducted with Lawrence Rouse, Ph D ., the state-level program specialist in research and evaluation who was

PAGE 79

65 responsible for analyzing the outcome data and generating the reports for the participants in the TCMHP. The goals of the site selection were described t o Dr. R o use as the identification of between two and four local mental health authorities which were perceived by state-level participants in the TCMHP as relative high and low information users Dr. Rouse was most interested in comparing sites which had been in the TCMHP since its inception with sites which were relative newcomers to the TCMHP. Setting this criterion against the issue of relative use and non-use it sug g ested the s election of four sites : a high u ser that is established in the TCMHP a low user that is established i n the TCMHP, a high user that is a relatively recent participant in the TCMHP, and a low user that is a relatively recent participant. This relationship is shown o n the ma t rix below The length of time local mental health authorities have been part of the TCMHP could be easil y and clearly determined by the fiscal year in whi c h they began to participate The determination of the s tate level perspective of a local authority's status as a high or low information user was however necessarily sub j e c tive and a n important Table 4 : Information Pathways Site Selection Matrix Site Selection for High Information User Low Information U ser Information Pathways Early Part i cipan t i n T CMHP I site whic h is both an e arly I site whic h is an early particip ant in participant in the TCMHP and i s the TC MHP and is i d entifie d b y i d entifie d b y s t a t e-level participa n ts a s sta t e-leve l p articipant s a s a l ow a high info nnat ion u s er. info nn atio n use r Recen t Participant in TCMHP I sit e w hich i s a recen t p articipa n t in I site which is both a recent the TCMHP and identified b y the part i cipant in the TCMHP and state as a high i nfonn a tion us er. identifie d by state l evel participants as a low i nfonnati on user.

PAGE 80

66 aspect of this study because it provided data for comparison with local-level perspectives on information utilization Among the research tasks identified for this study was the investigation of state-level definitions of l ocall evel information use Because the research design involved clarifying any similarities and differences between state and local-level perceptions oflnformation use, it was important to select local-level participants in a way that established and highlighted the state perspective The research design included key state-level informants being interviewed for the purpose of understanding : 1) how the informant defines high and low local-leve l information use; 2) what the informant thinks it means to be a high or low user; 3) how the informant knows if a local mental health authority is a high or low user ; and 4) what makes a local authority a high or low user. Similar questions were posed at the local level for the purpose of comparison with one another and the state perspective It should be noted that at no time was the assumption made that consensus would or would not exist among state-level informants regarding the definition and description of what constitutes use and non-use of outcome information at the local level. In fact the research team looked upon the identification of the level of differentiation or consensus among state-level perceptions of information utilization as part ofthe findings ofthis study In early discussions regarding the operationalization of high and low information users Dr. Rouse persisted in wanting FMHI to provide him with the criteria for making this determination This request was consistently turned back to him by asking him to define the differentiation between relative high and low information use from his

PAGE 81

67 perspective. Site selection discussions began in May 1996 and it was not until July 1996 that Dr. Rouse seemed to understand the importance of clarifying his own distinctions between what he perceived to be relative high and low outcome information users. In a sort of 'ah-ha' moment, he seemed to realize the value of examining his own definition and comparing it with others at the state level, and in a matter of moments he listed his criteria for use and non-use Dr. Rouse Identified three elements as crucial in determining whether a local mental health authority was a high or low information user : 1) project director feedback whether the local project director called the state-level TC:MHP research and evaluation staff with feedback on the evaluation reports with further questions or to say that the information had been useful in working with their Community Management Team (CMT), 2) rates of data submission whether the site is submitting data regularly to the state and 3) results of the data-whether the data submitted indicates the local mental health authority is meeting its contracted target outcomes. This informant then evaluated each of the 45 local mental health authorities in the TCMHP on the basis ofthese criteria to determine which of the local mental health authorities would be considered high information users and which would be considered low users. The rating process is described below. 1. Data Submission Each site was categorized by trend in submitting local evaluation data to the state with regard to three types of data : a) termination of assessment form b) beginning CBCL score and c) ending CBCL score Each center was given a check ( ./) for increasing the percentage of the required forms from the second quarter to the third quarter ofFY 1996 a zero (0) for staying the same or a minus (-) for decreasing the percentage required

PAGE 82

2 Data Results Each site was categorized by the trend in relation to two types of data: a) the percentage of children served toward the first-time offender targets Each center was given a check ( .1') for increasing the percentage of children served from the second quarter to the third quarter of FY 1996 a zero (0) for staying the same, or a minus ( ) for decreasing the percentage served 68 3 Contact with TCMHP Research and Evaluation Staff Each site was categorized by trend in relation to contacts with Research and Evaluation about evaluation information (i.e. requesting reports, checking figures asking questions, etc ) Each center was given a check ( .!) for frequently contacting Research and Evaluation, a zero (0) for occasionally contacting Research and Evaluation or a minus() for never contacting Research and evaluation The informant totaled the marks for each center Minuses canceled out checks and zeros did not count in the total. Scores for the 45 sites ranged from -2 to +8 Centers with four or more checks were rated as high users of the outcome information provided as part ofthe continuous evaluation ofthe TCMHP This process resulted in identifYing 24 local mental health authorities as high users of outcome information and 21 as low users of outcome information. The informant was quick to qualify this process as necessarily simplistic in providing for only two categories of use, high and low, noting that this would probably be more accurately portrayed as a range of utilization behaviors However the informant felt this process adequately accomplished the goal of sorting the local mental health authorities on the basis of differential use The next step in site selection was to ask him to identify others at the state level who help shape the TCMHP and for whom this distinction between high users and low users would be important. Dr. Rouse identified a state-level program specialist in children s mental health whom he believed should be involved in this process. He then volunteered to work with this individual (without divulging his own criteria) in articulating

PAGE 83

69 criteria for separating the 45 local mental health authorities into relative high and low user categories on this basis This informant's method of identifying patterns of differential information utilization focused exclusively on the results of data with regard to performance contract rather than including other criteria such as rates of data-submission and contacts with TC.MHP research and evaluation staff These criteria were : I) the percentage ofpriority-population served, 2) collateral provider satisfaction, 3) parent satisfaction, 4) child and adolescent satisfaction 5) behavioral-emotional functioning, and 6) school functioning However the results ofthis sorting process also resulted in identifying 24 local mental health authorities as high information users and 21 as low information users. Although these were not identical in their characterization of specific local mental health authorities as high or low information users, the two state-level informants were in agreement on 70 percent of the local mental health authorities. The final site selection recommendations for participation in this study was made by state-level TCMHP staff, and the participating local mental health authorities were selected from the 70 percent of the sites on which the two informants were in agreement. In light of the detailed analysis of what constitutes a highand low-use site it is interesting to note that the same explanation for differences in outcome information utilization was offered separately and on numerous occasions by four of the state-level TCMHP staff Differential outcome information utilization was explained and accounted for by these informants as resulting from observed differences in leadership style They each expressed however considerable interest in what might be learned from this study

PAGE 84

70 Three individuals at the state-level were subsequently, involved in the selection of specific sites to participate in this study The list of high and low use sites were reviewed in October 1996 by Texas rvtfllvfR Director of Children's Services the Director of Research and Evaluation for Texas MHMR, and the TCMHP State Project Director, and it was agreed that these were indeed, representative of high and low use sites After considerable discussion of which sites might be the most willing to participate and which might be the most open and forthright in their opinions about the outcome information system, the state-level informants identified and agreed upon four sites as appropriate for participation one for each of the positions described in the matrix illustrated in Table 4 A key informant at each local site was identified by the state le v el participants at the same October 1996 meeting in Austin These key informants were sent a letter from the Texas MHMR Director of Children s S e rvices and the Texas MHMR Director of R esea rch and Evaluation which described the research and invited them to participate. This letter is shown in Appendix 4 The letter was followed by a telephone call from Sharon Hodges to determine their interest in participation All four sites agreed to participate with no apparent hesitation The Data Collection Process Beginning in December 1996 and continuing through April 1997 direct observation and semi-structured interviews were conducted with key informants at the four local mental health authorities for the purpose of exploring local definitions of the outcome information system, outcome information use high and low outcome information users and how outcome information is or isn't used in decision making regarding service

PAGE 85

71 planning and delivery Using the three guiding research questions a semi-structured interview guide including a set of 15 questions was designed for the local-level data collection State-level participants provided input into these questions at a state-level site visit in October 1996. The final interview guide was drafted at Fl\1Hl and sent to the Director of Research and Evaluation at Texas MHMR for further input. The interview questions were accepted as drafted and are shown in Appendix 5 During this initial contact with the FMlll team, each of the key local-level informants was asked to identify individuals on site who would be important to talk with about issues of information utilization Each of the sites identified similar local le vel informants : managers and administrators at the local mental health authority, direct service providers and members of the local Community Management Team Each site's key informant agreed to schedule interviews prior to the research team's site visit. During the lo cal-level site visits respondents were asked, as part of the interview process to identify others who might have pertinent information of these topics (as suggested by Silverman, Ricci and Gunter 1990) This meant the process of data collection remained necessarily flexible toward the inclusion of additional informants who were not scheduled prior to the arrangement of site visits When time permitted, multiple interviewers were used for the same informant. The interview strategy included taping of all interviews, extensive field notes and written descriptions ofthe interviews prepared by each interviewer Every effort was made to leave the process open to interviews targeted at verification of information about issues which were discovere d to be complicated controversial, or pivotal to understanding aspects of organizational cu lture that influence the utilization of

PAGE 86

72 outcome information (see Silverman, Ricci and Gunter 1990). This meant that interview schedules changed and interviews were added as each field visit progressed A total of 41 interviews were conducted with l ocal-level. Informants included managers and administrators at the local mental health authority, direct service providers, and members of the local Community Management Team A table illustrating the breakdown ofthese interviews is shown below Table 5 : Local Mental Health Authority Interviews for Information Pathways Information Pathways Site Q 3 Managers and administrators 3 Direct service supervisors and providers 4 CMT members Information Pathways Site R 4 Managers and administrators 4 Direct service supervisors and providers 2 CMT members Information Pathways Site S 4 Managers and administrators 4 Direct service supervisors and providers 2 CMT member Information Pathways Site T 4 Managers and administrators 3 Direct service supervisors and providers 4 CMT members Total Individuals Interviewed for Information 41 Total Pathways The local-level interviews formed the core of the data collected for this study However the total data base included state-level interviews a review of state and locallevel documents and direct observation In combination the triangulation of the interview data from the System Accountability Study the document review and state and local-level interviews was designed to provide the basis for a cultural description of the formal and

PAGE 87

73 informal organizations at the local level of the TC.MHP as they relate to utilization of the outcome information system for local-level service planning and delivery Data C ollection Issues The research design for this research drew heavily from qualitative research methodologies qualitative being defined as focusing on data in the form of words through observation, interviews or documents (Miles and Huberman 1994) The traditional methodology of anthropology would be participant observation, requiring extended time on site in Texas Finan and van Willigen (1991) argue that because oftime constraints imposed on current anthropological research participant observation is not as readily applied Time was a constraint on this project although the constraint was not so much the total time that could be devoted to the project but rather the amount of time that could be spent at Texas J\.1HMR and the local TC.MHP sites In circumstances such as these Finan and van Willigen (1991 : 7) propose a stepwise approach which allows the anthropologist to design a research agenda in several related components to be carried out in a logical sequence ." This allows the researcher to step back from the research periodically and reflect upon interim fieldwork results and consult colleagues on specific issues," thus increasing the quality of social knowledge (Finan and van Willigen 1991:8) In triangulating data from several sources and over an extended period of time an effort was made to apply this stepwise approach in developing the research methodology Another result of the limitations on field time at the local sites was the decision not to pursue the collection of ethnographic data on the communities surrounding each site This was largely due to the fact that the catchment areas of each local site represented

PAGE 88

74 large and diverse geographic areas and multiple communities. Although this would have been valuab le information time did not permit the collection of extensive ethnographic data on these many and varied communities. An effort was made to choose sites with enough similarities to allow comparability among them It is a l so important to note that the site selection process restricted the potential sites to rural settings rather than strictly urban or a combination of rural and urban sites. Two of the sites suggested as potential participants by state-level staff early in the site selection process were rural sites. In addition, one of these sites had participated in the System Accountability Study allowing researchers to expand their knowledge of outcome accountability at this site. In an effort to maximize the comparability of participating sites, it was decided to select from rural rather than urban sites for this research The FMHI research team worked with Texas MHMR for approximately two years although the time spent on site in Texas was limited to one three-day site visit to each local mental health authority and three site visits to the state offices of Texas MHMR in Austin. These lim ited on-site contacts were augmented with telephone contact throughout the process Given the restrictions on time which could be spent on site, the primary methods of data collection included (taped) semi-structured interviews, document review and direct observation The "steps" in the research process for Information Pathway s involved a continuous reevaluation of the data collected to date, with particular attention paid t o how in-person interviews and observations could be strengthened with telephone interviews and document review

PAGE 89

75 In addition to the concern for the limited time spent on site, there was also an awareness of the possibility that because state-level informants had been involved in the selection of l ocal-level sites, that these sites would participate reluctantly or unwillingly or that they would not approach the interview process in an open and straightforward manner. The key informants at each local-level site seemed to accept the invitation to participate without hesitation Since the range of descriptors elicited for the outcome information system during the interviews ranged from "a crock" and "a necessary evil to an integral management tool ," it does not appear that local-level informants were reluctant to serve as candid participants in the interview process Data Reduction and Analysis Mintzberg (1979 : 584) describes inductive research based on qualitative methods as the process of trackin g down of patterns [and] consistencies One searches through a phenomenon looking for order, following one lead to another. But the process itself isn't neat ." One of the criticisms rendered against qualitative data is that methods of analysis are not well formulated (Miles 197 9 : 590) lea ving qualitative researchers open to the criticism that their work is neither valid nor reliable to scientific or policymaking audiences Aunger ( 1995 : 97) calls this a crisis of confidence in ethnography and poses the question of how we know whether ethnographic statements are anything more that personal impression These difficulties r es ult in part from the enormous amount of information generated through qualitative methodolo g ies One of the significant challenges of qualitative methods is the mass of data they produce Quantitative data appear more

PAGE 90

76 concrete, and so the confidence of analysis seems to rest more neatly there Dauber (1995 : 75) writing about how ethnographers deal with information overload suggests that it is important to look at the tools ethnographers use--files, charts, and records"as the most durable source of ethnographic authority ." Dauber was not so much offering a solution to the information overload, as observing how the process of analysis impacts the results Aunger (1995), also writing about analysis, argues that ethnography must make specific the method by which analysis is made, and that these must meet the criteria of 1) involving the reader closely in the ethnographic experience ; 2) acknowledging the context of the data collection in time and space; and 3) following methods of analysis which look for patterns not predetermined by the investigator This kind ofbalance was an explicit goal for the data reduction and analysis in this research Data Reduction A research team of five members described earlier in this chapter, was responsible for the reduction analysis and triangulation of the data collected using the methods described above. Data reduction began in the field, where debriefings were held each evening for the purpose of reviewing critical aspects of the interview and other data gathered that day The interview summary asked field researchers to review their notes for clarity and completeness, as well as to review the guiding research questions and record their immediate impressions, ideas and conclusions These details were discussed during debriefing sessions. In addition, regular meetings were held at FMlll for the purpose of discussing major themes across sites, newly emerging data needs, and trends in the data as the data were collected

PAGE 91

77 The data for this research took the form of field notes resulting from direct observation notes resulting from in-field and telephone interviews audio recordings of all interviews (with the exception of one incident of equipment failure and one incident of equipment unavailability), and the documents and notes on documents reviewed at both the state and local levels ofthe TCMHP All taped interviews were transcribed verbatim from the audio tapes Notes on untaped interviews were also transcribed State and local-level interviews, documents and notes on observations were coded according to codes representing the questions of the semi structured interview proto c ol used in local-level interviews Each interview transcription was reviewed and coded by 2 researchers A list of codes and how they relate to the guiding research questions is presented in Appendix 6 Data Analysis The three guiding research questions for this study also provided the foundation for data analysis in their use as summative questions The analysis involved reading and rereading the coded data in order to uncover patterns consistencies and inconsistencies within and across the local-level sites as well as in comparison to state-level responses After the coding was complete a team meeting was held for each site and the state-level data during which the patterns emerging from the data, as well as similarities and differences in team member perspectives were discussed A work sheet shown on Table 6, was developed to aid the process of ana l ysis at each site This work sheet prompted the team to consider the interview data for each of the research questions from the perspectives of informants who worked w i thin mental health at each site as well as

PAGE 92

78 from the perspectives of informants who worked outside of mental health as part of the sites's Community Management Team (CMT) These CMT members, also referred to as collateral providers of children's services represent the nine child-serving agencies in Texas that have collaborated at the state level in the TCMHP. This process of summarizing the data allowed informants' comments pertinent to each of the guiding research questions to be considered by the team making it possible for the review of the data by question across respondents and by respondent across questions In this way the process of analysis was structured to capture the cultural processes that grew out of people's experience with the outcome information system and the TCMHP Silverman et al ( 1990 : 70) suggest several strategies for enhancing the reliability and validity of qualitative data in the evaluation of health care programs These include offering a draft report for review by selected participants as well as research staff for the purpose of identifying factual errors and unfounded or unreasonable generalizations and hypotheses ." Sites were asked to verify factual data for their organization, including their service array staffing and budget information. State-level TCMHP staffwere sent a copy of the state-level and site specific results for review purposes at this time Ethical Considerations One of the complications of maintaining an ethical climate of anthropological practice is the fact that our established codes of ethics are sometimes in conflict with one another--if not in spirit certainly in their detail. For example the American Anthropological Association's guidelines (Section 3a) specify "Anthropologists should

PAGE 93

79 Table 6: Site Findings Worksheet General Research Questions Interview Topics Summary of Responses Summary of Responses from Informants within from Informants outside of Mental Health Mental Health (CMT Members) I What aspects of Understanding and organizationa l c ultur e pe r ce ption s of outcome influen ce the utilization of eva lu ation a nd its outcome information at t h e usefu lnes s site ? Perception of information u se View of community mental health 2 How does inform atio n Outcome information impact decision making at the considered usefuVnot site? usefu l How information s upp o rt s decision making How outcome information does no t s upport decision making 3. What cond iti o n s s upport Information flow of imp ede the use of ou tcome info rm a tion in service planning and delivery at th e What would make s ite ? informati o n more useful undertake no secret research or any research whose results cannot be freely deri ved and publicly reported The Society for Applied Anthropology's guidelines (S ection 3) refer to the "legitimate proprietary interests of ou r sponsors The National Association for the Practice of Anthropology's guide lines (Sect ion 3) also recognize some proprietary interest on the part of clients or employers and urge the anthropologist to clarify "the extent to which the information developed during our activities can be made available to the public" at the outset. Each ofthese presents a different perspective on the subject ofthe

PAGE 94

80 researcher's freedom to publish the results of the research I think this is evidence that the dialogue on ethics within anthropology will continue to develop for the foreseeable future One approach to creating a healthy ethical climate for applied work is to identify the stakeholders in the research and consider stakeholder needs or interests from the perspective of each stakeholder I believe this process is useful in anticipating any conflict that might arise in the conduct of the research and offers a perspective from which to review ethical issues as the research progresses It should be noted that the stakeholder needs shown in the table below represent my anticipation of stakeholder issues rather than any specifically expressed stakeholder need or interest. State-level participants in this study were either part ofResearch and Evaluation for Texas l\.1HMR or closely associated with the research staff Because the state-level participants were part of a government agency they were accustomed to working in the public eye In addition the state-level participants were accustomed to reporting the findings of their own research at conferences and in journals in the field of children's mental health This research orientation provided an environment which was both understanding of and open to the nature of the inquiry in this research With regard to the needs oflocal-level participants in this study, one ofthe strategies of the research design was to protect the identities of these participants so that they might feel free and open to comment on the outcome information system of the TCMHP regardless of whether their observations and experiences were positive or critical. This seemed particularly important because the research design specifically involved the participation ofTexas MHMR state-level managers and administrators in the

PAGE 95

81 designation of sites as high and low users as well as the selection of local-level sites for participation When the local-level sites agreed to participate, they did so with the knowledge that statelevel participants would be aware of exactly which sites were involved in the study. After the initial site selection process was complete, it was decided that the identities of the participating local mental health authorities would not be divulged in any reports or debriefings resulting from this research Additionally all local-level participants were advised that they would be provided individual anonymity for their participation within the study with the assurance that when direct interview quotes were used for reporting purposes, they would be identified only in terms of the relative position they held within their organization. For instance, a respondent might be identified as a direct service provider a CMT member, or a manager / administrator, but never by name or specific location In addition, participants were told that the names of local sites would be identified by pseudonym in any reports and an individual site's participation would not be divulged to the other participants Although the l oca l-le ve l interviews were recorded on audio tape, individual participants were asked permission before the taping commenced No one declined to be tape recorded However one participant asked that the tape recorder be turned off for short portions of the interview This seemed to occur when this informant mentioned local political figures by name although the researchers could not identify these political figures as having any direct relationship with the outcome information system or this research

PAGE 96

82 Table 7: Informati on P athways Stakeholder Needs Stakeholders Stakeholder Needs State-level participants ment a l he a lth agencies Information to assis t them in their work and improve voluntarily participating in th e s tudy service effect iven ess. Concern that the y will be represented bad l y or inaccurate l y in research reports Local-level menta l health agencies voluntarily Information to assist them in their work and improve participating in the study. service effectiveness.* Concern that they will b e represented badl y or inaccurately in research reports Co ncern that potential comments crit i cal of TCMHP would be held against th e m at the statel evel. Children and Famil ies Services that better meet their needs Funding Agency Impro ved serv i ce effectiveness Concern that research th ey hav e funded will be well received and respected. To be considered a cataly st for systems refo rm FMHI T he term s of the grant be fulfilled (inc lud ing the provision of technical assistance to si t es wishing to improve their utilization of o utcome measures) The publication of paper s and presentation of research fmdings at professional conferences. Concern that th e research results be well received and respected Team m e mber s The publication of paper s and presentation of research fmdings at professional conference s Concern that the research resu l ts be well rec eive d and respected Me Fulfilling my degree requirements Establishing myself professionally as a practicing anthropologist. P res umably, a commitment to this project would mean the se stake h o lder need s are a l so goa l s for FMHI, the work t eam, and the m yself. Written informed consent forms were not used for this study Upon initial phone contact with t he key inform a nt at each local site the goa ls purposes, and expected reporting of the results were discussed and all key informants were reminded that their participation was vo lunt ary before being asked if they wanted to participate As an

PAGE 97

83 additional guarantee to each individual participant's informed and consensual participation in this research field researchers were given specific instructions for introducing the interview which included : 1 Providing the informant with a brief overview of the work being done in the field of system accountability at FMIIT 2 Discussing the goals and purpose oflnformation Pathwa y s as well as the resulting monograph 3 Noting that Information Pathways was supported in part by funding from the Annie E Casey Foundation, Inc. 4 Noting that although the Texas :MHMR endorsed Information Pathways this study was not part of the state evaluation 5 Discussing the confidentiality and the voluntary nature of participation in this study and determining the willingness to participate 6 Noting that sites would not be identified in any reports resulting from this study and that individuals would be identified only by their position (e. g CMT member direct service provider, etc ) It should be noted that one of the underlying strategies to protecting the identities of specific sites was to protect the identities of individual participants at the local level and thus their ability to comment in an unrestrained manner on their observations and experiences with outcomes and the TCMHP If there was any concern that the identification of local sites would compromise the identities of individual participants it was never mentioned as a concern by either the sites key informants or other site participants The overriding attitude seemed to be one of acceptance that the informants were working in a public domain and therefore willing to make commentary

PAGE 98

84 This approach was readily accepted and seemed to satisfy all interview participants No one reque sted exclusion from the interviews However, once the research team was on site, individuals at each of the local sites asked the identity of the other three participating sites When it was explained once again that the individual site identities were not being divulged the response was perplexed and a different perspective emerged Public agencies are accustomed to working under public scrutiny; the key informant s at each site, as well as numerou s individual participants expressed an inability to understand why the site identities should be camouflaged under these circumstances The research team maintained its stance to the bemusement and frustration of the local sites I have little doubt that the identities of each of the parti cipating sites have been discuss ed with one another as well as with other local mental health authorities in the TCl\.1HP and that the site identities for this study are part of the informal "pub lic record ."

PAGE 99

CHAPTER FIVE RESULTS: LOCAL PERSPECTIVES ON OUTCOME INFORMATION UTILIZATION Background on Local Mental Health Authorities 85 Four local mental health authorities were selected to participate in this study The site selection process and criteria for selecting these sites is described in detail in Chapter Four. In all reports and discussion resulting from this research the actual names of these sites will be protected by the use of pseudonyms The four sites will be identified as Site Q Site R, Site S and Site T. The four sites that participated in this research have in common the fact that their catchment areas or the geographic areas they are designated to provide services for include predominantly rural counties in which few if any private mental health services are available This means that in many cases the services of the TCMHP are the only mental health services available to children and families in the local communities being served The TCMHP services are relatively new to all of these communities having been planned and implemented since FY 1992 Although state-level staff refer to FY 1994 as the year that all 45 sites in the TCMHP were fully funded ," there is ready acknowledgment at the state level that the local mental health authorities are not funded adequately to meet the demand that exists

PAGE 100

86 for children's mental health services Due to the limited resources available, none of the sites involved in this study expressed the belief that their mental health authority is able to meet the demand for children's mental health services in their catchment area The geographical distribution of services for each of the sites in this study will be discussed in the reporting of the results for each site later in this report but it is important to note that all four sites emphasized the increasing and sometimes overwhelming demand for their services experienced since the inception of the TCMHP Consequently, each of the local mental health authorities in this study has had to make decisions regarding how to prioritize the location of services in their catchment areas given the limited availability of resources and the demands of their large predominantly rural catchment areas The transition from providing outcome information for the purpose of informing service planning and delivery decisions regarding service effectiveness to providing outcome information as evidence of contract compliance or non-compliance began in FY 19 9 6 with the creation of performance contracts for each local mental health authority participating in the TCMHP This transition is moving toward completion in FY 1997 (which began October 1996) when TCMHP contracts are imposing financial sanctions on local mental health authorities that do not fulfill the terms of their contracts Regardless of whether a site was in or out of compliance with their contract at the time of the interviews the staff at each center emphasized the pressure they felt as a result of the requirement that they fulfill the terms of their TCMHP contract in order to receive funding In addition to the pressure to meet their TCMHP contract obligations all of the sites participating in this study identified an additional stressor which although external to

PAGE 101

87 the administration of the TCMHP diverted their attention from service delivery issues This stressor involved recent changes in rules governing Medicaid billing that have a strong impact on local mental health funding streams because of what services Medicaid will or will not agree to reimburse Participants described in varying detail the difficulties experienced from the reduction in Medicaid reimbursements on traditionally reimbursable services and commented further on the stress of not being able to predict adequately how future changes might affect their funding streams and service delivery At the time of the interviews participants believed that anticipated changes in the administrative guidelines of the TCMHP resulting from the adoption of a system in which children will be assigned services based upon guidelines that identify appropriate services on the basis of a child s level of need would also serve as a significant stressor for local mental health authorities. This procedure had not yet been adopted by the TCMHP during the interview period but its anticipated implementation contributes to what local mental health authorities describe as the stresses of operating in an environment of change Also adding to the environment of change was the death of Dr. Lawrence Rouse in August 1996 Dr. Rouse had been instrumental in developing and implementing the TCMHP evaluation and three of the local mental health authorities noted his loss as contributing to feelings of stress and uncertainty in the TCMHP and its evaluation The following sections will present the data for each of the four sites The three guiding research questions for this study will be addressed in the discussion of the findings for each site The results will be presented in the alphabetical order of the site's pseudonym's beginning with Site Q and concluding with Site T.

PAGE 102

Site Q Results Site Introduction 88 Site Q has the largest catchment area of the four research sites in this project, a fact visually reinforced for the researchers by the stark beauty of the wide expanses of open space greeting their arrival to this site. A winter visit to Site Q brought with it wind, cold snow, and more wind, somehow making the landscape outside of the small city which anchors this local mental health authority appear to sweep even more distantly toward the horizon The mental health authority's main offices are located in a multi-story freestanding office building easily visible from the major east-west thoroughfare in this area Entering the children's services waiting room for this mental health center, the researchers were greeted warmly by a receptionist who was simultaneously answering the phones and puzzling over a computer software installation. Posters decorated the walls and an assortment of children's toys were strewn across the carpeted floor A few adults sat in chairs as a child flipped absentmindedly through a book. Although Site Q staff are placed in multiple locations in Site Q' s catchment area, this office serves as the administrative center for the full-time staff of28 and three part timers Because much of Site Q's catchment area is sparsely populated ranch country, children's services under the TCMHP have been concentrated in the five counties where the demand for services is considered largest and the logistics of providing services in geographically distant rural areas are not so challenging The staff, however, remain

PAGE 103

cognizant and somewhat frustrated over their inability to serve their entire catchment area, even minimally under current funding constraints. 89 The children s services director at Site Q described their array of children's services as including assessment, case coordination, family education, in-home intervention, crisis stabilization, parent training, medication-related services, skills training, respite counseling day treatment, and mentoring services Prior to TCMHP funding, Site Q was limited in the number of children it could serve with the full-time equivalent of two and one-half children's workers for the entire catchment area Once funded by the TCMHP, Site Q began offering expanded children s services in 1992 Site Q estimates that 61.7% ofits children's services budget is funded through the TCMHP Site Q is considered a relatively long-term participant in the TCMHP, having been a part of both the Plan and its evaluation since 1992 Informants at the state level of the TCMHP believed Site Q should be considered a high user of outcome information This assessment was based on the site selection criteria of data submission, data results and contact with the Texas MHMR. Research and Evaluation staff described in Chapter 3 According to these criteria, Site Q had increased the percentage of its priority population served between the second and third quarters ofFY 1996, as well as the percentage of juvenile first-time offenders being served In addition, the site had increased the percentage of termination forms submitted to the Texas MHMR Research and Evaluation staff as well as improved its rate of submission of ending CBCL data Site Q was also noted as having increased contact with state-level Research and Evaluation staff For

PAGE 104

these reasons, this site was considered a relatively high user of outcome information by state TC:MHP staff 90 Site Q informants within mental health as well as the collateral providers of children's services outside of mental health who serve on the Community Management Team (CMT) described themselves enthusiastically as TCMHP outcome information users Among informants within mental health the interview data provided evidence of both the awareness and utilization of outcome information at all levels of organization In addition CMT members demonstrated themselves to be both adequately informed about the results of the outcome evaluation and involved in the taken as a result of this evaluation information being available. A licenced therapist Site Q's children's services director acknowledged his belief that as a result of the focus on developing a broad base of children s services in his catchment area some therapeutic approaches are simply not reasonable service alternatives, "We're not going to get into long term psychotherapy with these kids ... Our goal is to get a base level of services in all of our counties Although there was acknowledgment that long-term psychotherapy would not adapt easily to the TCMHP informants both within and outside of mental health at Site Q voiced support ofbuilding a broad range of children's mental health services through the collaboration of interagency partners in the TCMHP Ten people were interviewed formally at Site Q. These included the children s services director the mental health director, the quality assurance manager the CMT chair two CMT members representing juvenile justice services, a CMT public school

PAGE 105

representative the case management supervisor a registered nurse and the family and communit y education coordinator. Asp ec ts of Organizational C ulture Influ e ncing Information U tilization at Sit e Q 91 This study assumes that culture is emergent from social interaction, suggesting that culture is implicit that meaning is constructed through interaction and that culture is a negotiated process that occurs at all levels of organization This necessarily focuses the attention of this research on how knowledge is used to interpret experience and generate social beh av ior at each site through building an understanding of the experiences and actions of indi v iduals as they relate to one another throughout the organization The organizational relationships observed by and described to researchers visiting Site Q are charact e rized b y a high degree of partnership between the local mental health authority and state-level administrators and e v aluation staff In additi o n well-de v eloped partnerships between the local mental health staff and the staff of collateral child-serving agencies represented on the CMT were evident. The local mental health authority s partner s hip with state-le v el program and evaluation staffbecame evident during interviews with the local mental health staff, and it also became apparent that this familiarity with state-level staff e xt ended to CMT members as well Both the mental health authority staff and CMT members referred to state-le v el staff by their first names and made numerous references to the positive value of contact with state-level TCMHP staff At the local level the partnership between the mental hea l th staff and the collatera l children s services pro v iders who serve on the CMT was evident in the use of the per s onal

PAGE 106

92 pronouns we and ours by CMT members when referring to children s services offered at Site Q and outcomes achieved. For example in describing the importance of having outcome information, one CMT member said "We need to be an accountable agency effectively illustrating, through the use of the inclusive plural form a strong level of partnership and shared responsibility among child-serving agencies at this site CMT members described their interagency collaboration as being established before the implementation of the TCMHP and emphasize how some of the individuals who remain active on the CMT were involved in developing the initial proposal to the state for TCMHP funding Site Q's mental health director expressed the view that the TCMHP has fostered further collaboration among child-serving agencies through the CMT, noting as an example that without the CMT structure the mental health authority's executive director (who serves on the CMT) would have no vehicle for direct communication with other children s services providers. While always acknowledging that there is more to be done and noting their belief that children in their catchment area remain under served, the CMT members interviewed expressed pride in the children's mental health services offered at Site Q Among mental health staff, interview data indicated that outcome information is commonly understood as "a management tool that is useful for assessing and enhancing program effectiveness ." The program director's references to the monthly and quarterly outcome reports as "a report card of the effectiveness of mental health services in the community and an accountability tool provided evidence of this perception of outcome information as a tool of effectiveness within mental health The opinion was repeatedly

PAGE 107

expressed that although the data reporting requirements can be burdensome, the information produced in the TCMHP was considered well worth the effort 93 The children's services director, who has been a part of children's services at this site since the inception of the TCMHP is able to download data for his mental health authority's children's programs on a monthly basis from the state's computer network and receives statewide comparative data on contract performance from the state research and evaluation staff on a quarterly basis. The children s services director serves as the main distribution point for children's outcome information dissemination at the local mental health authority and is responsible for making this information available to the agency's executive director, mental health director, supervisory and direct service staff, parent advisory group, and CMT members The children's director reports distributing outcome information to the direct service staff at least quarterly and sometimes more often With case managers and therapists this information flow is primarily oral and communicated at regular staff meetings This strategy of oral communication is described b y the children s services director as an effort not to overwhelm the staff with paper We get so much paper here that it is incredible. So I generally try to verbally tell this stuff And they all know they have access to this stuff [the written reports]. We have a folder in there [pointing to a filing cabinet] with all this stuff in it." The children's services director described the typical process of getting feedback from the staff on the outcome results as a brainstorming session "For example, we're not getting enough [ending] CBCLs in And

PAGE 108

94 as someone who is not really that involved in direct care I can come up with solutions all day on how to improve that but until they tell me a realistic way of doing that .... The children s director finds the outcome reports particularly useful in providing positive reinforcement for direct service staff and demonstrating to the direct service staff in a concrete way that they are making a difference in the work they do as well as encouraging people to do a better job, "It helps them reaffirm why we' re doing what we're doing You know, their rewards are very few And that s one way of[saying], 'You are making a difference and where you might not see it right now, overall things are going well.' Commenting on the mental health authority s failure to serve the contracted number of children during the past fiscal year, the children's services director reports communicating challenges as well as successes with the direct service staff, "We didn t make that and so we have to devise ways that we're going to make that [contract target] next year Also expressing the utility of outcome information as a management tool the case management supervisor commented that doing her job without access to outcome information would be something like going to Sears and buying a barbeque grill and taking it home and putting it together and not having the instructions I mean you could probably [put it together] eventually, but it might look kind of funny when you get it done ." Although time constraints prevented direct service staff from being interviewed at Site Q results of the interview with the case management supervisor suggested that the language of outcomes filters down to the client level at Site Q where case planning includes setting outcome goals for children and families

PAGE 109

95 Outcome information is communicated to the CMT by the children s services director both in written and oral formats Approximately one week before their monthly meetings the director mails copies of the monthly report as well as the quarterly summary when it becomes available to CMT members This gives them several days to review and digest the material before attending the CMT meeting. At the meetings, the outcome information is reported orally, usually with the aid of overhead transparencies Interview data indicate that the CMT members interviewed also view outcome information as a tool of management. For example one CMT member described the role ofthe CMT at Site Q as one of monitoring the utilization of services against what services are available and what dollars are being spent. CMT members also demonstrated an understanding of the TCMHP as b eing specifically desi gne d to serve children and adolescents experiencing severe emotional, behavior and mental disturbance and the resulting target population definition that guides who is eligible for TCMHP. Although the need to serve more children was discussed by CMT members along with the strong desire to provide services beyond those currently in c luded in the TCMHP the restriction of services to children who fit the target population definition was understood and accepted as a component of the TCMHP Within the mental health authority the children s services director provides outcome information indirectly to the mental health center's board of dire ctors through written and oral reports of the results from the center s executive and mental health directors Like the children s services director the mental health dire ctor expressed some sense ofburden with the data-reportin g requirements but the mental health director at

PAGE 110

Site Q described the outcome measures as providing a useful tool of management particularly in decisions regarding what kinds of services best meet the needs of specific populations of children and adolescents 96 The link to the mental health authority's board of directors in the information feedback loop is reported to be less significant than the link to the CMT. The children s services director described the relatively stronger influence ofthe CMT on children's programs as resulting from the structure of [the TCMHP]" and went on to describe the role ofthe CMT in children's services I think ofthe CMT as the children's services board of directors .... Structunilly my understanding is that if the CMT made a decision that the board of directors didn t go with ... the CMT is going to prevail. Although the children's services director reports this has caused some problems in other centers he said that their center's board of directors has taken the position that the CMT is more involved with children s programs and better understands the needs of the community. Therefore no conflict between the center's board of directors and the CMT was reported. Data from both within and outside of mental health suggest that the Site Q mental health agency is viewed increasingly more positively by other child-serving agencies as a result of the availability of outcome information The children s services director believes that outcome information has improved mental h ea lth s reputation as a team player in the community because it provides evidence to collateral providers of mental health s willingness to evaluate the effectiveness of their service planning and delivery decisions CMT member interviews mirrored this perspective noting the mental health authority s willingness to bring both good and bad outcomes into disc ussion at CMT meetings

PAGE 111

97 Among examples of mental health being viewed more positively by the CMT as a result of the availability of outcome information provided, one CMT member commented that her supervisors listened more carefully to her reports of children s mental health services because of their perception that the mental health authority had the data to support its conclusions. How Outcome Information Impacts Decision Making at Site Q In addition to descriptions of the general utility of outcome information at Site Q informants provided examples of specific ways outcome information has had an impact on decision making regarding service planning and delivery. Because the outcome reports help the mental health authority monitor demand for services by primary diagnosis as well as trends in service demand, informants within and outside of mental health described outcome information as being useful in informing staffing decisions such as what type of staff might best serve the caseload. With the aid of periodic supplemental reports provided upon request from the state offices, Site Q is able to determine service need geographically as well. This allows them to make optimal geographic placement of staff CMT members commented that basic demographic indicators such as gender, age ethnicity and economic status of children served in addition to information about outcome results have been useful in their discussions of how to best serve children in their caseload For example the need to better serve minority groups at Site Q surfaced as a direct result of having the information provided by the TCMHP evaluation reports. The children's service director confirms that he relies on the CMT to both raise critical service issues and help forge solutions to service delivery problems "They will often look at me and say,

PAGE 112

98 'That's a problem ; we need to work on this.' The CMT has raised the issue of providing services to minority communities and the director commented, "We do not do very well reaching the minority population in this community ... We are not doing a very good job reaching out to those areas Currently struggling with this issue and not yet finding adequate solutions, the children's services director reported that the CMT has formed task forces to explore the issue and is making an effort to open access through Hispanic organizations in the community ." The children s services director provided another example of using outcome information to inform decisions about how to allocate staff positions by service type such as day treatment services and outpatient counseling services In this case the state research and evaluation staff was asked to do a collective study between the outcomes of outpatient counseling and in-home intervention services to see which seemed to produce better outcomes. Using client satisfaction data and data on ending CBCL scores, no significant differences between the results of these services was determined Although no shift in service emphasis was indicated at that time the children's services director would look to a similar reporting of data in the future to inform a decision about how to use resources where they are most needed The most frequently cited application of outcome information in service planning and delivery decisions at Site Q was the choice to develop day treatment services Day treatment is Texas most intensive form of non-residential services available and the decision to commit to the establishment of a day treatment program meant choosing not to expand the already established in-home counseling program or develop other new

PAGE 113

99 programs under consideration. Information about the effectiveness of day treatment services across the state was critical to the CMT decision to develop day treatment services at Site Q As a school-based day treatment program the establishment of a day treatment program required the commitment of local TCMHP funds as well as facility and staff support from the local school district. When asked what this decision process would have been like without the availability of service effectiveness data, a school district representative to the CMT commented "A lot of things would have been done on perception Either that or we would have been scrambling around trying to find the data ourselves This CMT member further described how the service effectiveness data on day treatment provided a better understanding of the population for which day treatment services would likely be most effective in serving. According to a CMT member the data showed day treatment would have the most effect for students who are less aggressive and need "more social skills and lots and lots of consistency rather than highly aggressive or disturbed students with a long history of problems at school. According to this CMT member that feedback significantly shaped thinking about who the day treatment program could most effectively serve. As often as the informants at Site Q advocated for the increased availability and use of outcome information in service planning and delivery decisions individuals both within and outside of mental health offered examples of how outcome information fails to support decision making. For example the director commented that there was some staff turnover early in the TCMHP among staff who did not feel that the effectiveness of the ir work with children and families could be meaningfully and adequately measured through

PAGE 114

100 the TCMHP evaluation The strategy used by the children's services director to counter these objections to outcome data as meaningless is to coach and encourage the staff to reconsider the impact that outcome information can have at the local level I just have to tell [my staff] over and over that not only do we want to meet our [contract] requirements but we want to have a good center and we might as well use this information to help us constructively Another issue raised regarding how outcome information can act as an impediment to decision making at Site Q was raised by CMT members who expressed some concern that the emphasis on contract compliance could have a negative effect on the quality of services delivered With some degree of sarcasm, one CMT member described the contract goals set by Texas l\.1HMR as "a little bit unrealistic and expressed the belief that performance pressure and emphasis on meeting contract targets created by the contract requirements could serve to reduce the quality of service deli v ery by focusing attention on quantity served rather than quality of services provided. In fact, Site Q did not meet its contract target for the number of children served durin g FY 96, and the m e ntal health staff described themselves as feeling disappoint ed in their results and feeling pressured to bring themselves into compliance. Nonetheless the children's services director expressed the belief that Site Q is providing quality services and, because of partnerships with collateral providers it is reaching children and families that "wo uld not t ypical ly walk into a mental health center." The co ntract measures while imposing performance pressure are described by the children's services director as "a reality Discussing how he has dealt with a direct service staff member who is not serving enough children, the children's

PAGE 115

101 director expressed a certain level of acceptance of the contract targets, "Sometimes you just have to say, 'George, you need to serve this many more kids before the end of the year .' Really you can't negotiate that kind of thing because the number is what the number is, and you have to meet it." Describing his approach as one of coaching direct service staff regarding improved performance, he continued, "We will help our staff out clinically as much as possible, but the fact is that we still have to serve that number of kids That's a reality ." A significant frustration for mental health staff at Site Q is that the state's data system and the site's in-house data system are rarely reconciled with one another without a considerable effort The in-house data system, developed by another mental health authority and used by a number of mental health authorities throughout the state, was developed for the adult system and was in place prior to the implementation of the TCMHP Because of the differences in the target population definitions for children and adults, the inhouse data system often incorrectly applies the adult criteria for target population and categorically rejects children who do in fact meet the state's criteria for children eligible for services The in-house system provides enough important information however that it cannot be easily overlooked For instance, the in-house system maintains financial records, including Medicaid records, and does client billing. In addition although the state's system provides global information at the catchment and county levels such as primary diagnosis information for children in service, the in-house system provides case-level data that is useful on a day-to-day basis By comparison, the children's services director at Site Q says that he uses information from the in-house

PAGE 116

102 system on a more regular basis, but commented that both information systems serve important purposes, I also want to get across that we couldn't survive without the CARE [the state information system] function ... because ... it gives us information on the quality of things an overview of the quality of the outcomes An anticipated upgrade of the in-house software is expected to reduce the conflict and confusion between the two information systems. Conditions that Support or Impede the Use of Outcome Information Site Q demonstrated many qualities that support the use of outcome information in local service planning and delivery decisions Descriptions of information flow at Site Q offered by informants both within and outside of mental health suggest a multi-directional flow of information about and discussion regarding outcomes results throughout the organization Data show that outcomes are discussed at all levels of the organization within the mental health authority and that administrative as well as direct service staff provide feedback to the children s services director. CMT members discuss outcome information both at formal CMT meetings and informally among themselves between meetings There is also evidence that CMT members take reports of TCMHP program results back to their home agencies and that this information is held in some value in those agencies Regarding improvements that could be made in the outcome information system fears were expressed that the outcome data may lack integrity across the state Commenting that the integrity of the data often rests on the integrity of the individuals filling out the forms the children's service director at Site Q noted that it would be easy to

PAGE 117

103 take liberties with the accuracy of the data input into the system Although informants at Site Q expressed a great deal of trust in the data generated from their mental health authority, there was considerable suspicion about the accuracy of the data generated by other mental health authorities This served to significantly reduce the degree to which the comparison data across the state was considered For example, sample sizes across the state make it difficult to understand the true meaning of the data and difficult to compare one site to another. The children s services director at Site Q iUustrated this point by referring to the improved school behavior measures for the third quarter ofFY 96 The state contracted target for this measure was 60% improvement and rates of return of this measure have been much lower than expected across the state. Referencing the Quarterly Service Report he compared one local mental health authority which showed 57% improvement based on 143 surveys returned and was therefore, considered out of compliance with its contract. Another local mental health authority which showed 67% improvement based on only three surveys returned was however, considered in compliance with its contract. The children's services director commented that this irony is the source of frustration for staff and the CMT who share a strong commitment to excellence and want to be among the best local mental health authorities in the state. The implication in this conversation was that it would be easy to influence results favorably on some measures The children's director pointed out that it would be easy to influence the measure of children avoiding rearrest by leaving that data item off of the appropriate form While strongly denouncing this behavior, improved CBCL scores were also used by the director as an example of this potential for influencing results I could

PAGE 118

104 send out one survey that I know is going to be positive and I'm going to meet my target ." The director reiterated the integrity of his own site s outcome results laughingly commenting that if their results were being altered they d be in complete compliance with performance contract measures Summary of Site Q Findings The findings for Site Q can be summarized with respect to the guiding research questions for this study The organizational culture at Site Q is marked by strong partnerships both between the local mental health authority and state-level TC:MHP staff and between the mental health authority and the collateral children's providers serving on the CMT This partnership forms a foundation for dialogue regarding the outcome information and provides a sense of shared responsibility and decision making in which CMT members take a hands-on approach to managing children s mental health services Informants within and outside of mental health consider themselves outcome information users and provided numerous examples of how this information has informed service planning and delivery decisions Supplemental reports are periodically requested from the state research and evaluation staff to enhance the standard outcome reports Although outcome information is considered an important source of information regarding service effectiveness several weaknesses in the TCMHP outcome evaluation were identified and some frustration with the increasing demands of the Plan were voiced In general Site Q s approach to outcome information utilization might be taken from the Nike advertising slogan Just Do It! Using outcome information to become a

PAGE 119

more accountable agency is difficult and frustrating work, but well worth the effort so they just do it. Site R Results Site Introduction 105 Site R is approximately a one-and-a-half-hour drive from the nearest commercial airport With clear blue skies and a few minutes to spare before the arranged appointment time, the research team hoped to explore on foot the small town that is home to the local mental health authority's main offices However, the beautiful day belied winter temperatures that were in the single digits The team opted for a short drive through the town instead which revealed a small main street of tum-of-the century buildings a few local eateries and a Wal-mart store on the edge oftown. Although one edge of Site R s geographic boundary is within an hour of a large city, the environment of this service area is distinctly rural and the environs are marked by numerous small towns connected by two-lane roads that roll past open fields The mental health authority's main offices are located in a strip shopping center that includes a drug store and a discount store Mistakenly entering neighboring offices which housed another social service agency the research team was re directed next door to the mental health center With no external sign to mark the office, it was difficult still, to know if this was the correct location Approaching a receptionist behind a glass service window the research team was directed down a long narrow hallway to the children's services director's office

PAGE 120

106 Site R serves a nine-county area that was, until recently, under the domain of a state mental hospital. For geographic areas with no established community mental health center the TCMHP contracted with the state facilities to provide children s services under the Plan in order to provide coverage throughout the state. Site R, like the other state hospital facilities in Texas is in the process of transition from a state-operated facility to an independent community mental health center. Site R is considered a relatively new participant in the TCMHP, having received funding for the provision of services beginning in FY 1994 when the TCMHP was expanded to include 45 local mental health authorities across the state. Site R s Community Management Team (CMT), however was established in 1992 as part ofthe initial process of service development in this site An interview with the children s services director indicated that services offered through the TCMHP at Site R include assessment case management counseling parent skills training crisis stabilization medication education and management psychiatric evaluation, in-home counseling and family education The children's services director at Site R estimates that 82.4% of their children s services budget is funded through the TCMHP. Site R has grown to include nine full-time and two half-time employees Staff turnover, however at both administrative and direct service levels was reported to have been a significant issue for Site R. The children s services director pointed out that vacancies had been hard to fill and that it is difficult to meet contract requirements for serving a targeted number of children and families without a complete staff One member of the mental health staff commented on the unfairness of pay differentials between long-

PAGE 121

107 time state employees and the children s workers who were in relatively newly created positions. This pay differential may have had an impact on the children s services director's ability to fill vacancies In her IS-month tenure at Site R, the children's services director felt an important accomplishment was filling staff vacancies and reducing staff turnover. Administratively, in addition to the children's services director being somewhat new in her position, the deputy executive director, her immediate supervisor had been at Site Rand in his position for only a few weeks at the time of this interview State-level staff involved in recommending sites for this research agreed that Site R should be considered a relatively low user of outcome information This assessment was based on the site selection criteria for data submission, data results and contact with the Texas MHMR Research and Evaluation staff described in Chapter 3. According to these criteria, Site R had increased the percentage of the priority population being served between the second and third quarters ofFY 1996, as well as the percentage of first-time offenders being served In addition, the site had increased the percentage of termination forms submitted to the Texas MHMR Research and Evaluation Staff However submission ofbeginning and ending CBCL data, as well as contact with the research and evaluation staff had not increased Therefore this site was evaluated as a relatively low user of outcome information by state-level informants Informants from within mental health at Site R as well as the collateral providers of children's services outside of mental health who were interviewed as part ofthe Community Management Team (CMT) described the site as a low user of the outcome information provided by the TC:MHP evaluation Among informants within mental health

PAGE 122

108 the interview data provided little evidence of outcome information being used to inform service planning and delivery decisions. When asked directly about using outcome information in service planning and delivery, the children's services director responded, "Well, it wouldn't be the first thing I think of ... but I'm just an administrator I guess it's available to me ." Direct service supervisors and providers described their use of outcome information as limited to discussion, and described outcomes as difficult to apply to issues of practical problem solving and oflittle assistance in suggesting solutions, We discuss the outcomes that come down to us .... and we try to brainstorm how to utilize that information ." This direct service supervisor continued, So some ofthe reports we get are not helpful because they're more fiustrating than helpful, because we don't know what to do with it." CMT members while expressing interest in the results of the TCMHP evaluation, described their utilization of outcome information as restricted by the limited amount of information provided to them by the local mental health authority Although the children's mental health staff at Site R expressed frustration with the pressures associated with serving their contracted number of clients and the restrictions imposed by the TCMHP's definition of who is eligible for children s service, there were no explicit references indicating support for or opposition to the TCMHP strategy of creating a base of children's mental health services at the local level through an interagency collaboration among nine child-serving agencies References to a particular philosophical or therapeutic approach to the delivery of services at Site R were brought up by individual service providers as expressions oftheir personal expectation of serv i ce delivery, but did not seem to indicate or respond to a specific or unified approach to the delivery of services

PAGE 123

109 at Site Rasa whole For example an individual therapist voiced concern about the emphasis on short-term services in terms of personal experience I always thought that my work would be pure, based on client need and what I could provide What I find is that I'm told by a program director or supervisor that we need to hurry up and run kids through so we can get more kids in to meet our contract numbers Ten people were interviewed at Site R. These included the children s services director, both the executive director and the deputy executive director the CRCG coordinator, two regional supervisors with therapeutic responsibility one child and adolescent counselor a quality assurance manager, and both the outgoing and incoming CMT chairs. Aspects of Organizational C ultur e Influencing Informat ion Utilization at Site R As mentioned earlier, this study proposes that culture be viewed as emergent from social interaction suggesting that culture is implicit that meanin g is constructed through interaction and that culture is a negotiated process that occurs at all levels of organization. As a result the attention of this research is focused on how knowledge is used to interpret experience and generate social behavior at each of the participating sites through building an understanding of the experiences and actions of individuals as they relate to one another throu g hout th e organization The organizational relationships observed by and described to rese a rchers visiting Site Rare characterized by a certain de gree of distance or s eparation between Site R s children's mental health staff and the state-level TCMHP administrators and evaluation staff as well as weak relationships betw ee n th e children's mental health staff a nd other

PAGE 124

110 local providers of children's services serving on the CMT. The relationship with the state level TC:MHP staff is described in "us / them" terms by both the administrative-level and direct service children s staff at Site R, providing little indication of partnership or collaboration with the state regarding the implementation of local mental health services or the TCMHP outcome evaluation Lacking the first-name familiarity with state-level TCMHP staff that was observed at the other three sites in this study mental health informants at Site R often referred to individuals at the state level singularly as Austin, a reference to somewhat distant power and hierarchy in the TCMHP The data provided numerous examples of the us / them relationship between TCMHP state and local staff For example, a direct service provider commented, "But sometimes when those reports come, there is an air of cynicism [on the part of Site R direct service staff] saying What do they expect from us now?"' In this example outcome measures at Site R seemed to reinforce the distinction between us and them ." The children's services director at Site R also described the relationship of the CMT with the local mental health authority in us / them terms, indicating little local-level collaboration among child-serving agencies in the planning and delivery of children's mental health services. In discussing how information from the TCMHP evaluation is disseminated to CMT members, the children s services director indicated that it is given to members during meetings Exactly what evaluation results and in what format the evaluation information is disseminated to the CMT remains somewhat unclear The children s services director indicating that minimal use is made of outcome information at CMT meetings commented, Well I pass out bits to them but then we might talk about it

PAGE 125

at the meeting Usually we don't talk about it that much becau s e they [CMT memb e rs] don't understand what it means ." Ill Both the local mental health authority staff and CMT members describe the relationship between mental health and the CMT as somewhat strained as a result of what a CMT member referred to as a "history of bicker i ng and c onflict between the mental health authority and other local child-serving agencies CMT members commented that the role and responsibility ofthe CMT with regard to children s programs remains unclear, although there was the feeling that these roles are slowly being clarified CMT members interviewed for this research were only vaguely aware of the TCMHP performance contracts and the potential for financial sanctioning of mental health authorities that do not fulfill their contract. They were also unsure what outcome and evaluation reports were the subject of the research team s interviews After researcher s described some of the outcome reports, a CMT member commented that this kind of information is what they should have been looking at and e v aluating," but added that we didn t do this kind of results analysis ." CMT members explain the lack of communication between the CMT and the local mental health authority as resulting from considerable staff turnover at the mental health authority and a history of struggle between the agenc y and the CMT. CMT members felt that the local mental health authorit y uses the CMT onl y minimally and did not place them in a position of using results One CMT member added that recently they have been able to establish a clearer role for the CMT in planning and program development. CMT informants stressed that the CMT has pushed the mental health authority to change its programs and services but expres s ed frustration at the

PAGE 126

112 mental health authority's reluctance to change the traditional way of offering services on an outpatient counseling basis One example of friction between mental health staff and the CMT was described by informants inside mental health as an ongoing conflict with CMT members regarding where the mental health authority provides services There has always been a lot of conflict about where the child and adolescent staff was supposed to do their work ... [The CMT] did not want office-based child and adolescent specialists so they thought they should be in the homes and the schools ." The CMT members also noted the location of servi c es as a point of conflict indicating that the mental health authority had more traditional ways of operating but that the CMT wanted them "to move toward work in the home ." The shift away from traditional in-office service provision i s gradually occurring with the establishment of more in-home services as well as a day treatment program based in the school system The mental health staff often expressed the view that the CMT demanded more of them than the y were able to do According to mental health staff, another point of friction between the mental health staff and the CMT was to whom TCMHP services should be available Interview data indicated that CMT members did not fully understand the TCMHP target population definition which describes the program as serving children most in need and specifies which children are eligible for TCMHP services One CMT member described a funding problem that arose because funds were only supposed to be used on the priority population and we didn t know what a priority population was ." This type of misunderstanding strongly sugge s ts an incomplete understanding on the part of CMT members of the goals of the TCMHP and could

PAGE 127

113 contribute to the feeling expressed by the CMT members interviewed that the mental health authority was not as responsive as it needed to be to the needs of children and families in the community It also drew a ttention to the feeling expressed by mental health staff members that the CMT places unrealistic demands on the mental health staff Staff turnover at the mental health authority seems to have contributed to the strained relations between the CMT and the mental health authority One CMT member cited the high staff turnover w i thin the mental health authority as a partial explanation for the mental health authority being slow getting on the move to change services." Turnover was noted as a considerable problem for the CMT membership as well. Community Management Team members interviewed for this research described both a high level of turnover among CMT members including the chair's positions and an overall low le vel of active participation on the par.t of those who remained members In the beginning one CMT member commented the CMT was losing members serving in the position of chair left and right. Although the TCMHP was designed to be a collaborative effort involving nine child-serving agencies the absence of strong partnerships with state-level TCMHP staff and local-level c ollateral providers of children s services seems to leave the local mental health staff at Site R feeling as if they shoulder the entire burden of carrying out the TCMHP The mental health staff expressed the view that "we" have too much to do and that it would be impossible to adequately meet the needs of everyone who needs care. It is important to note however that these references to we were inclusive only of mental he a lth staff and did not express the view that this responsibility belonged to multiple child-

PAGE 128

114 serving agencies. The isolation ofthe children's mental health staff from support at the state level and collaboration with interagency partners through the CMT at the local level seems to have direct impact on Site R's approach to utilizing the outcome information generated by the TCMHP evaluation Direct service and direct service supervisory staff at Site R demonstrated good knowledge ofthe TCMHP outcome information. However, one respondent, who could name and describe each report accurately emphasized that the reports are of little use, "Truthfully these reports that I get each month, we talk about them in the staff meeting and then I shred them after the meeting because they have no bearing on what I do with my caseload." The interview data also indicated that attempts to use outcome information within mental health at Site R are accompanied by anxiety from the pressure to serve greater numbers of children and families than they believe they are able to and fiustration at trying to understand how to use it. For example, comments from both supervisory and direct service staff at Site R demonstrated the stress associated with trying to understand how the outcome information relates to service planning and delivery and how it might be used to inform programmatic decisions, "It is just that Austin has made [CBCL scores] one of our performance contracts and we're in a bind, we're in a catch ... there's a lot of discussion and a lot of various trial and error things that go on as we try to work on things and try to work things out." A direct service provider commented on the impact of outcome information on services, "I think it's affected the bottom line as far as we need more people (clients)-we need numbers. I don t see that it has altered the way we do services."

PAGE 129

115 Direct service staff members express a sincere commitment to serving children and families and do not believe that the outcome reports adequately reflect the results of their hard work. At both the direct service and supervisory levels, there is a strong sentiment that TCMHP outcome information does not adequately reflect the complexities of the work that is being done One mental health staff member expressed frustration at community and state expectations for improvements in children who are very seriously ill. Comparing the typical TCMHP client with the less serious cases seen in private practice, a supervisor commented, "But when we get the kids here, these are kids who have had years and years of mental health problems When making decisions at the state [and] the community, [they] want to see the same quick results [as with a less serious case] they want to see the perfect child, they still want all of those things ." Frustration with the school-related outcomes was also a point of discussion. The children s services director commented that she did not understand what improved behavior meant A supervisor commented that often the schools don't understand that we don t have a magic wand that we can wave on that kid and that kid will go back and fit into the classroom and just be a cheerleader just a nice kid ... So if they don t see that kind of miraculous healing they are not satisfied ." The frustration and stress associated with the outcome reports seems to manifest itself in a compliance-oriented approach to utilization a desire to meet the contract requirements coupled with the belief that the outcome information provided by the TCMHP is not reflecti v e ofthe efforts of the mental health staff This compliance oriented approach is reflected in th e children s services director's description of how Site

PAGE 130

116 R has recently used outcome information in a service-related decision intended to improve contract compliance When asked directly if she thought outcomes had an impact on decision making, the children's services director commented, "Well, I do in fact . and went on to describe how outcome information was used in Site R's decision to offer group therapy in the schools The compliance orientation was evident as she described this decision, "We [began offering group counseling in the schools] because we wanted to get our numbers .... and we thought surely we could more easily reach the numbers by working in the schools ." The decision to offer school-based services in order to increase the numbers of children served stands in contrast to a decision to offer services in the schools because that might be a more effective strategy for serving families and children This compliance-oriented description of how outcome information is used at Site R suggests that the outcome information provided by the TCWIP evaluation offers the local mental health authority little insight into the effectiveness of services being offered at Site R The children's services director s description of how outcome information is used at Site R tended to be more passive than active as it related to service planning and delivery For example the director summarized Site R's use of the outcome information that "comes down from Austin" by saying "It gives me a lot to think about, mainly some things to be concerned with ." One of the regional supervisors spoke of how outcome reports are shared with the therapeutic staff in the form of discussion at staff meetings Although the interview data indicate that the children s services staff do not actively use outcomes as a tool of management or a measure of service effectiveness the mental health staff revealed that the

PAGE 131

117 client and collateral satisfaction data are informative and that service providers benefit from the positive feedback received through these client satisfaction measures Some direct service staff also mentioned that they like to see how Site R compares with other centers across the state. References to how outcomes impact decision making at the local level at Site R are often made in terms of how outcomes could or would be useful if utilization was optimal rather than restricted as it is currently perceived For example, a CMT informant described outcome utilization by saying that outcomes "haven t been very valuable in the past," but added that the local mental health authority is at a point now where the y (outcomes) should be a guiding force CMT members suggested that one reason outcomes have not been used in decision making is that the focus of the mental health staff was necessarily on establishing basic services because there were so few services actually available Similarly, the mental health authority s deputy executive director indicated that in an ideal setting outcome information should provide an understanding ofbest practices and what programs are successful in achieving positive outcomes These comments point to both an interest in and a basic understanding of the possibility of using outcome information to assess service effectiveness. One ofthe supervisory-level mental health staff commented that "In Texas it's only a couple ofyears old that they've really begun to focus in this area and it's improved as we ve gone along However, the TC:MHP reports as they are currently perceived by the mental health staff, are not generally viewed as providing useful feedback on service effectiveness

PAGE 132

118 There was also frustration expressed by individuals at administrative, managerial, and direct service levels within mental health that there are few opportunities for programmatic action based on the results of the outcome evaluation Direct service providers said that they saw few alternatives for altering services as a result of what they learn from the outcome information they receive One provider stated, "When the funding comes out to us it is specific as to what it should be used for, and we can't change that." At administrative levels within mental health, a similar frustration is expressed over the fact that the outcome-based environment currently part of the TCMHP offers no rewards for achieving positive or successful outcomes In addition to frustration with the potential for financial sanctioning of mental health authorities that do not achieve contract compliance the deputy executive director at Site R mentioned that there seemed to be no facility for rewarding success in meeting contract goals and also expressed fear that good contract performance might be punished rather than rewarded "It's just as likely the deputy executive director continued, that instead of [being rewarded] for achieving good results that .... your funding could be cut because you've already showed that you can serve this many people before The deputy executive director explained that Site R is still very much in the process of changing its organizational focus to become more outcome oriented "We're constantly educating staff and families, providing information about how our system works and what role they [outcomes] have." In addition the deputy executive director said, "We're trying to get our footing on what s the best way to organize ourselves to outcomes.

PAGE 133

119 Informants within mental health felt that because the outcome results were reported in an aggregated form for their entire catchment area these outcome results were too general to be of use One suggestion for making outcome information more useful that was heard repeatedly during the interview process was the need for results to be reported on a county-specific basis It was felt that this would make the information more reflective of specific communities and therefore more useful in service planning and delivery Summary of Site R Findings The findings for Site R can be summarized with respect to the guiding research questions The organizational culture at Site R is marked by an isolation from state TC:MHP staff and local-level partners in the TCMHP that results in a compliance orientation to outcome information utilization On the surface the pattern of informa t ion flow described at Site R appears to be similar to the optimal pattern described by state level informants Local-level informants indicate that Site R's children's services director receives the reports of the TCMHP evaluation on the monthly and quarterly schedule described by state-level informants The director distributes them to the Executive and Deputy Executive Directors of the mental health authority the quality assurance manager the children s direct service staff and supervisors and to members ofthe CMT However the quality of the outcome information flow and the resulting outcome discussions within Site R seems to be limited as a result of the isolation of the local mental health authori ty from both the state level TCMHP staff and the local-le v el collateral providers serving on the CMT

PAGE 134

120 Neither mental health staff nor CMT members describe themselves as outcome information users Menta l health informants examples of outcome information utilization were characterized by the anxiety and frustration associated with the pressure of meeting contract performance requirements. The CMT members interviewed for this research demonstrated little exposure to or experience with outcome information The flow of outcome information at Site R, both within and outside of mental health are reported to be primarily in a written format with intermittent discussion of the outcome results Interviews indicate that the mental health staff receives outcome information reports in writing in their boxes and sometimes this information is discussed at staff meetings One direct service worker reported shredding the outcome reports after a staff meeting because they lacked relevance to direct service work Interview data also indicate that CMT memb e rs were not fully aware of what outcome information was reported to Site R through the TCMHP evaluation a result it seems, of what was described as a long-estab l ished conflict betw e en the mental health authority and CMT members and expre s sed fears that the CMT would not fully understand the meanin g of the data In general Site R s compliance-oriented approach to outcome utili z ation might be characteri z ed b y the phrase Do the rig ht thing The c hildren s mental health staff demon s trated s trong dedication to providin g quality s ervices to children and families ; the y w ant to be successful in their work but did not find that outcome information inform e d their decision s on how best to pro v ide services As a result their approa c h to the outcome evaluation is to try to comply with what is required of them to do what is rig ht.

PAGE 135

Site S Results Site Introduction 121 The highway across the rural landscape toward Site S is punctuated by exit signs for small towns that could hardly be seen from the road Although the drive seemed to take longer than the actual three hours from the airport the small city that anchors Site S seemed to materialize rather suddenly. After the research team was welcomed with coffee and a bit ofteasing about the team s apparent trepidation at the impending ice storm the visit to SiteS began with a regularly scheduled meeting of the c hildren s mental health staff The mental health authority s main children's offices are located in a two-story free-standing building on a comer adjacent to additional children s offices in a refurbished house. Entering the building with the clinical director for children s services, the research team was greeted by the re c eptionist from an office behind a window in the waiting area While staff gathered for a regularly scheduled meeting in an upstairs conference room, the waiting area was not busy Although Site S has a TCMHP catchment area that includes six counties service delivery strategies ha v e been concentrated on providing services in a single count y where the demand for services was considered greatest. The children's clinical director at SiteS indicated that their children's services include psychotherapy (individual group play family marital and school-based) family preservation case management psychiatric medication and nursing services crisis intervention psychosocial training caregiver

PAGE 136

122 support and education and mental health screenings in court Prior to TCMHP funding the children's project director reported that SiteS had one children's worker Currently Site S counts staff at 19 people. Because of this significant growth, Site S' s program director believes the TC:MHP has "absolutely" had a positive impact on children's services The children's project director believes that this increased ability to provide access to services has had a positive impact on the community's view of children s mental health The children s clinical director at SiteS estimates that approximately 54% of its children's budget is funded through the TCMHP. SiteS is considered a relatively long-term participant in the TCMHP, having been a part of both the Plan and its evaluation since 1992 Informants at the state level of the TCMHP believed Site S should be considered a low user of outcome information This assessment was based on the site selection criteria of data submission, data results, and contact with the Texas MHMR Research and Evaluation staff described in Chapter 4 According to these criteria Site S had increased the percentage of children served in the priority population as well as the percentage of children served in their First Time Offender program between the second and third quarters of 199 6 However rates of data submission were lower for the termination assessment forms and the beginning CBCL scores and there was no improvement in the of ending CBCL scores In addition Site S was reported as having little or no contact with state-level research and evaluation staff For these reasons this site was considered a relatively low user of outcome information by state level TCMHP staff

PAGE 137

123 Site S informants on the mental health staff as well as the collateral providers of children s services outside of mental health who serve on the Community Management Team (CMT) described themselves as non-users ofthe outcome information provided as part of the TCMHP evaluation Among mental health staff interviewed, staff at administrative levels in the organization demonstrated thorough knowledge of the measures and the contract with the state but described the outcome reports as having little bearing on their service planning and delivery decisions Direct service staff described themselves as having little knowledge of the TCMHP e v aluation explaining that the clinical director protects them from this sort of thing allowing them to focus their attention on clinical issues CMT members interviewed also described themselves as low users of outcome information A juvenile justice representative on the CMT expressed considerable satisfaction with mental health services at Site S but described the TCMHP and its evaluation as confusing and overly regulative Therapy is described by the SiteS project director as the major part ofthe service we provide ." This is seen by the children s mental health st aff at Site S as a fundamental conflict with the goals and structure of the TCMHP because as the project director continued, The state is really not crazy about therapy ... They want short term crisis intervention and we want more longer-term [therapy] ." The strong commitment to therapy at Site S was and continues to be a potent force in de v elopment of services as the TCMHP is implemented at this site Ten people were interviewed at SiteS. These included the children's clinical director who is responsible for staff supervision and program development the TCMHP

PAGE 138

project director whose role was to organize the CMT originally the program director who is responsible for the budget and TCMHP contract compliance the mental health authority s executive director two staff with both supervisory and therapeutic responsibility one therapist one truanc y case worker and two CMT members A s p e cts of Organizational C ultur e Influencing Information Utilization at Site S 124 Based on the assumptions of study that culture is implicit and emergent from social interaction and that meaning is constructed though interact i on culture is seen as a negotiated process that occurs at all levels of organization The attention of this research is therefore focused on how knowledge is used to interpret experience and generate social behavior at each site by developing an understanding of the experiences and actions of individuals a s they relate to one another throughout the organization The organizational relationships observed by and described to researchers visiting Site S are ch a racterized by a high degree of independence on the part of the local mental health authority toward state level TCMHP administrators and on the part of the mental health authority toward the staff of collateral child-serving agencies represented on the CMT. The mental health authority s desire for a relationship autonomous from state-level TCMHP staff became evident early in the data collection process The us/them language used to describe relationships with the state had neither frustration nor anxiety associated with it. In fact state-level staff are described with respect as "good, good people who are very helpful and who "have worked very hard to establish the TCMHP." Local ch i ldren s mental health administrators hold the efforts of the state staff and their successful advocacy for children s mental health with the state legislature in high esteem

PAGE 139

125 Independence from the s tate on issues of therapeutic concern is however a strong value which was evident throughout interviews with Site S s administrative as well as direct service staff Interview data indicated that the desire for independence at Site S is directly related to a th e rapeutic approach to services that departs from the TCMHP service delivery model in significant ways Staff at Site S describe their services as therapeutically driven and the conceptualization of their treatment model as "dev elopmental psychoanalytic and focused on object relations ." Although the clinical director described the interventions resulting from this approach as not classical ps y choanal ysis in a strict sense ," therapeutic services were described as play therapy, more child-centered and non-directive ." A therapi s t with staff supervisory responsib i lit y described Site S s approach to therapy in these terms For us, therapy is seen as a process Other centers are more cognitively based ... We are therapeutic in our approach. We work from a psycho-dynamic frame of reference ... Our approach considers kids' relationships with the world and environment they grew up in." The priority at SiteS clearly rests in their clinical work. The childr e n s project director commented I think we go about doing clinically what we think needs to be done and the outcomes are usually there When asked if the CMT has backed the strong therapeutic approach to service development, the children s project director said They take the position [that] when they have a kid who n e eds something they 'll usually ask me for therapy ." It should be noted that this description ofCMT involvement is at the level ofthe individual child a perspective or position encouraged b y the site s therapeutic

PAGE 140

126 approach This i s a marked difference from a systems of care approach which, although advocating that services should meet the individual needs of children, is more focused on building an int erage ncy array of services at a community level. The priority SiteS places on therapy can also be observed in their Family Preservation Program, typically a short-term intervention designed to build family stability The children's project director described family preservation as "initially presented [by the state] as a real crisis kind ofthing. She continued, "We've retooled it a little differently .. . a lo t of what we do is try to stabilize families through family preservation in order to get them into therapy ." The significance of this site's commitment to a longterm therapeutic treatment approach is that this emphasis when used to define services at Site S translates i nto longer-term services designed to center on the child's long term well being and life satisfaction rather than on shorter duration stabilization and crisis intervention services designed to improve the child s more immediate day-to-day functioning and family stability. This difference in approach has strong repercussion s on how the TCMHP outcome evaluation is perceived and utilized by the mental health staff at Site S because of the TCMHP evaluation measures are functionally based ( e .g. measuring improvements in school behavior or reductions in juvenile justice involvement) and even at the local level, they provide a global look at how children who are in the system are functioning The outcomes that were des c ribed as v aluable by mental health staff at Site S are largely qualitative assessments of how individual chi ldren are progressing clinically as a result of

PAGE 141

127 therapy As SiteS's children s program director put it, the TCl\1HP outcome evaluation ')ust isn't a great fit with the approach to services at Site S. The director continued, You might could make it fit but it just wasn t designed for what we do." There is, however no compromising on completing the paperwork required to meet the contract requirements The children s clinical director described the outcome reporting requirements a necessary evil," but went on to say, but we don t slip on the necessary part at all. It's absolutely necessary You can't work here unless you can do this stuff and get this stuff filled out and turned in and copied and get it out. Staying in compliance with the TCl\1HP contract is part of the strategy of independence from the state-level TCMHP at SiteS. How Outcome Information Affects Decision Making at SiteS As a result the very basic contrast in service philosophy between the local level and the TCl\1HP the mental health staff at Site S described patterns of outcome information utilization as minimal and themselves as non-users The clinical director described the outcome evaluation's utility as a tool for funding at the legislati ve level but quickly added We haven t used any of the data locally to decide what programs to start to decide how to treat anyone or to really even look at how any groups of children are doing ." The project director at Site S echoes a similar sentiment I have never used it as a decision making tool. It's been more of just information and that's it. I'm never going to use it that way ." SiteS's children's program director described the outcome information as designed more for the purpose of feedback at the state or legislative level rather than at an individual program level.

PAGE 142

128 For tracking the clinical status of clients, SiteS relies much more heavily on its inhouse information which was described as originating with clinical issues in mind than the state system associated with the TCMHP evaluation The children's project director described the two systems as in conflict, almost always containing discrepancies in the data She continued, "If we ever want to know something we use our own data ." The children s clinical director described a significant part of his job as "shepherding data from the real world into this [the state] monitoring system .... because we cannot function without it. We would not get funded ." As an example he discussed the importance of efforts to submit CBCL data to the state system Although the measure is not seen as useful for clinical purposes, he added, "It' s a pretty good instrument as far as the reliability and everything But it's imperative that we know so much more about any g iven case than the CBCL could ever tell u s .... the early development of the child, and of the parents patterns of relationships over time .... that the CBCL doesn t really add anything ." Site S s therapy-driven approach to service planning and delivery impacts the local mental health authority s relationship with collateral providers of children s services resulting in a relationship which can also be described as largely autonomous or independent. SiteS' s CMT meets quarterly Similar to the relationship with state-level TC.MHP staff, as soc iations with CMT members do not appear strained The children s proje c t director describes the r e lationship of the CMT to children s mental health at SiteS as not real hands-on .... They have their opinions but they re not real controlling and are not in there trying to change things They re kind ofbusy doing their own age ncies

PAGE 143

129 things One CMT member described the CMT function as primarily to approve how money is spent. This same member described issues at CMT meetings as being introduced by either the clinical or project director and then discussed by the CMT membership rather than issues generated by the CMT membership. Because there is little evidence of CMT members taking action independently of the mental health staff, the interview data do not support the idea of this CMT serving a TCMHP oversight and partnership function at Site S One CMT member described the CMT as "MHMR (mental health)-driven and CMT decisions are generally described by CMT members as modifications of MHMR recommendations. If outcome information has not actively informed service planning and delivery decisions at Site S the impact of contract performance requirements is felt nonetheless The children s project director describes outcomes as changing how they do their jobs probably a little bit. This change was described as emanating from a desire to stay in contract compliance At a regularly occurrin g quarterly staff meeting for the purpose of taking stock, performance on outcome measures was discussed for the first time but the discussion focused on what was needed to stay in compliance Interviews with the children s mental health staff indicate that pressure to meet performance contract requirements has encouraged building a stron ger relationship with the public schools an emphasis in effort that would not as described by informants have been as pronounced without contract measures requiring certain levels of collateral provider satisfaction and improvements in children s school behavior The mental health staff at SiteS also say

PAGE 144

that the CMT structure has helped build collaboration they do not believe would have otherwise occurred 130 This sentiment is echoed by one CMT member who said, "At meetings we share input ; the CMT has made us work together better .... Years ago we didn't work with other agencies, but it is coming together now This same CMT member also commented however that it is hard to understand MHMR (the mental health authority) and that the meaning of the reports and measures needs more explanation Another CMT member interviewed also felt that the information presented to the CMT by the mental health staff needs more explanation This informant added that "MHMR shields [the CMT] from knowing about contract requirements and how they are doing. This CMT member did however, provide an example of the CMT's direct influence over the mental health services in which services were modified to include programs in the schools at the CMT' s request. The description of the actual information flow at Site S was vague in the interview data although this is thought to be more a function of the outcome information's relative lack of importance at Site S than a result of a desire on the part of informants to obscure the researchers understanding ofthe outcome information flow For example, the interview data indicate that outcome information is distributed to the executive director the direct service staff, and the CMT. However the pattern ofthat flow is unclear and not of particular concern to Site S s administrative staff. One illustration of this is the children s project dire ctor's description ofhow outcome information is distributed to the CMT. The CMT was described as receiving outcome information when we have

PAGE 145

I 3 I meetings, but we only meet a couple oftimes a year .... unless they get it directly [from the state], which they might." Conditions that Support or Impede the Use of Outcome Information SiteS does not consider itselfto be an outcome information user, and, although mental health staff were able to describe a number of factors that impede their use of outcome information, these were all thought to derive from the contrast between a therapeutic and systems of care perspective According to informants at Site S, the change that would make outcome information more useful would be to "bridge" between clinical information Site S informants find useful at the client level and the more global information Site S informants believe is necessary at the state or legislative level. This difference in information needs is what the clinical director at Site S called the "client level and the mega level." Site S defined those client level or clinical information needs very differently from the quantitative summaries currently reported through the TCMHP evaluation. The information currently provided to Site S through the TCMHP evaluation is described by informants within mental health at Site S as satisfYing the more global or "mega" information needs useful at the policy or legislative level, but not meeting the local clinical level needs Clinical information needs were described as more qualitative, the kind of individually client-focused information that is generally discussed during the clinical supervision of direct service staff This duality of needs was a theme running through interviews with the children s programs administrative staff at SiteS.

PAGE 146

132 Summary of Site S Findings The findings at SiteS can be summarized based upon the framework of the guiding research questions for this study The organizational culture as SiteS is marked by autonomous relationships with state-level TClMHP staff and the local-level collateral providers serving on the CMT. The deep commitment to clinical therapy at Site S focuses staff energies on therapy and minimizes the need for discussion of outcome results within the mental health authority and with the CMT. Outcome information does not appear to impact service planning and delivery decisions at Site S, although complying with the performance contract requirements is considered important both because it secures funding and because it reduces the interference from state staff and collateral providers in the therapeutic process Site S 's focus on developing services from a therapeutic perspective left little room for discussion of other conditions thought to support or impede the use of outcome information in service planning and delivery In general Site S' s strong commitment to a therapeutic approach to service delivery resulted in a pattern of outcome information utilization that might be described as "Doing it our way ." SiteS wants to protect and preserve their approach to services and will make use of outcome measures as a strategy of minimizing the influence of relationships they believe have the potential to infringe on their ability to adequately serve children therapeutically

PAGE 147

Site T Results Site Introduction 133 Site T's main administrative offices are located in an aging state facility Although the office acts as the administrative hub for this local mental health authority it became clear almost immediately that the real action at Site T takes place in the satellite centers serving the six counties in its catchment area The six counties stretch across a major transportation corridor and the site is situated near a small city that is not part of its catchment area The county nearest this city is considered suburban but the remaining counties are rural and dotted across their open spaces by small towns Site Twas, until recently under the domain of a state mental hospital. For geographic areas such as Site T with no established community mental health center the TCMHP contracted with the state facilities to provide children's services under the Plan in order to provide coverage throughout the state Site T like the other state-operated facilities in Texas is in the process oftransition to an independent community mental health center. Site Tis considered a relatively new participant in the TCMHP, having received funding for the provision of services beginning in FY 1994 when the TCMHP was expanded to include 45 local mental health authorities across the state Site T established its CMT structure in 1992, however, as part of the initial process of service development in this site Because Site T includes a wide geographic area and also includes communities that the mental health authority considers substantially different from one another administrators have chosen to operate with two CMTs guiding their children's services in

PAGE 148

134 an effort to make the TCMHP more representative of and responsive to the needs of the communities they serve The children's services director discussed this decision "It [the catchment area] was pretty diverse, just as far as the type of population served the areas the interests, and the needs of the community .... We just weren't sure how we were going to satisfy everyone's needs with one Community Management Team ." Both of the CMTs hold monthly meetings Site Tis unique among the sites participating in this study in that children's services were established prior to the implementation ofthe TCMHP In 1984-85 funds were appropriated from the mental health authority s general revenue dollars to establish positions for three full-time children's workers When the TCMHP system development funds were made available to Site T in FY 1992, followed by funding for the pro v ision of services for the TCMHP in FY 1994 the mental health authority considered this an opportunity to add to the basic counseling services they were already providing The children s services director explained We sta rted off with two pots of money : [the money from general revenue source] .... and then we got the apportionment from the Legislature for the Texas Children's Mental Health Plan. Because Site Twas already providing services and already producing outcomes (even if these were not formally tracked through an outcome evaluation), the site found itself in a position of early success with the TCMHP evaluation and was thus more easily able to be proactive in its approach to developing TCMHP s ervices. The mental health director described their accomplishments to date by saying We have good outcomes and people are satisfied with our services we' re doing OK.

PAGE 149

135 Children s services are provided at Site T through a center serving two counties This office space is donated by the schools Children s services are also available at a center serving Site T' s remaining four counties that is housed in facilities donated by the county judicial system. Family preservation offices are located in two additional offices. The goal expressed by administrative staff was to provide a uniform distribution of services throughout their entire catchment area The children s services director acknowledged that some mental health authorities have chosen to concentrate services in limited parts of their catchment area in order to develop a broader base of services, but that Site T' s commitment was to building accessible services in their rural counties Services offered at Site T include outpatient counseling (individual, group, family) Family Preservation, assessment psychiatric evaluation and medication-related services, screening and crisis intervention service coordination with schools and other child-serving agencies, and aftercare mental health services following discharge from psychiatric facilities The staff at Site T has grown to include the children's services director the project director a CRCG coordinator, ten child and youth specialists based in the centers (including those working in services for first time offenders) one supervisor at each center and seven family preservation specialists Site T estimates that approximately 25% of its children s services budget is funded through the TCMHP Informants at the state level of the TCMHP believed Site T should be cons i dered a high user of outcome information. This assessment was based on the site selection criteria of data submission, data results and contact with the Texas MHMR Research and Evaluation staff described in Chapter 4 According to these criteria, Site Thad increased

PAGE 150

136 the percentage of children served in the priority population as well as increased the percentage of children served in their First Time Offender program between the second and third quarters ofFY 1996 Site Twas also noted as having increased contact with the state-level research and evaluation staff In addition, Site T had increased the percentage of termination assessment forms submitted to TCMHP and increased the percentage of ending CBCL scores submitted to TCMHP For these reasons Site Twas considered a relatively high user of outcome information by the state TCMHP staff. Site T informants within mental health as well as collateral providers of children s services outside of mental health who serve on the Community Management Team (CMT) described themselves as TCMHP outcome information users. Among informants within mental health the interview data provided evidence ofboth awareness and utilization of outcome information at all levels of the organization. The approach to outcomes monitoring at Site Tis shifting, however and the tenor ofthe discussion is becoming more serious The children's services director explains "Outcomes have really only recently become very important to us They have always been important but now they are of utmost importance because funding is tied into those outcomes. CMT members demonstrated an increasing awareness of the outcome measures as they relate to TCMHP performance contract requirements. This is a result of a specific strategy on the part of the mental health authority to invol v e the CMT more directly in the planning and management of children s mental health services. Site T s philosophical approach to buildin g children s services might be described as one of seeking balance between shortand lon g er-term treatment alternatives The

PAGE 151

137 children's services director described how the satellite centers, which provide counseling services, work cooperatively with their family preservation services, which provide shorter-term crisis intervention "When the family needs more intensive intervention, they'll most likely go through family preservation, and when they become more or less stabilized they'll be likely referred back to the center or maybe closed. Even with regard to counseling services, the emphasis does appear to be more behaviorally focused concentrating on changing children's behaviors by helping them link their actions to consequences rather than placing an emphasis on a longer-term less directive model of therapy Direct service providers described the increasing emphasis on short-term interventions in these terms, The idea is to get families in and as clearly as possible define short-term brief goals, work on those and move the family on . . so we can serve as many families as possible ." This strategy was seen by direct service providers as a necessity because of increasing demand As one ofthe direct service staff put it "because what we're seeing is that we're increasingly having to struggle to serve everybody [who wants services] ." The direct service staff supports the idea of building community-based services as an alternative to more expensive residential treatments and believes this strategy reduces the need for more expensive interventions, With family preservation and other mental health center services in place we did not have to place our children out of the home .... I don t think the state has any concept of how many kids we're actually keeping out of in-patient services or residential services ." Eleven people were interviewed at Site T. These included the children's services director the children s program director the mental health services director, the clinical

PAGE 152

138 director serving both adult and youth programs, one satellite center director, a child and youth specialist a family preservation supervisor, the CRCG coordinator, and four CMT members In addition the research team attended a CRCG training and development semmar. Aspects of Organizational Culture Influencing Information Utilization at Site T This study assumes that culture is emergent from social interaction, suggesting that culture is implicit meaning it is constructed through interaction, and that culture is a negotiated process that occurs at all levels of organization For the purpose of this study, it is important to focus the attention of this research on how knowledge is used to interpret experience and generate social behavior at each site through an understanding of the experiences and actions of individuals as they relate to one another throughout the organization The organizational relationships observed by and described to researchers visiting Site T are characterized by open communication with the state TCMHP staff This relationship is described by Site T mental health staff as a reciprocal relationship Mental health staff at Site T characterize the direction of communication and the flow of information in their relationship with TCMHP state-level staff as going both directions and clearly appreciate the give and take of this arrangement. Evidence of relationships among the mental health staff at Site T reveals a success oriented approach to service planning and delivery and indicates that there exist both flexibility and openness in discussion regarding TCMHP outcomes at all levels of the organization The children's services director commented I think one ofthe things having outcomes has done for me is increase the communication I have with people out in

PAGE 153

139 the field about what they re doing ." Observing the children's services director interacting with direct service staff provided insight into a conversational style of discussion in which there is an easy flow of information back and forth A similar style of discourse, in which ideas and feedback flowed easily among communicators, was observed among administrative staff The data also indicated that Site T has built willing partnerships among collateral providers in the communities they serve The children's services director described mental health as one of nine agencies" in the TCMHP demonstrating an understanding ofwho the key stakeholders were in the local network of organizations and how the Plan was intended to work. This collaboration is evident in the support Site T receives from both local school and court systems in the form of the provision of space for satellite centers that is donated rent-free Although the "we/us" language often used by CMT members to describe mental health services indicated their collaboration in the TCMHP, interview data indicated the CMT viewed mental health as taking the lead on service planning and delivery According to mental health staff at Site T this view is accurate to a large extent because the TCMHP funds represent less than half the children s budget at Site T The CMT chair described the CMT role as one of an executive board "active in the beginning getting the ball rolling .... and once it rolled, it's like people are doing their jobs a good job. He later added, "I think we re in awfully good shape as a board From the perspective ofthe mental health authority administrators interviewed service delivery and the CMT relationships are 'in good shape' now Site T met all but one of its TCMHP performance contract requirements in FY 1996 and unlike sites that

PAGE 154

140 are struggling to meet their targeted number of children served Site T expects to exceed by far that contract requirement in the corning year The mental health staff, however has had the growing realization that the funding base for children s mental health services is no longer expanding and that funding they have always relied on may not continue to be available As a result the ment a l health staff is placing significant energies into the transformation of their two CMTs from executive boards providing overall guidance to more active boards that have a full understanding of the financial status of children s mental health services and the ability to provide hands-on management for community mental health services This process includes explaining the importance of the outcome measures to the CMT which the children s services director described to the Team, "We're trying to get better, be leaner and do a bett e r job, and this [the outcome reports] helps us because we know the areas we' re deficient in." This strategy of fostering change in the CMT' s role is a direct result of the impact of funding issues such as new Medicaid billing rules which are beginning to restrict the flow of funds at Site T and are forcing changes in how they provide services in an environment of increasing demand for services The mental health director believes that the demand for services will only grow Once the door [availability of services] was open, or cracked a little wider the flood came The need for children s services is so great and monies are not really following The potential financial sanctions of the TCMHP performance contracts have added to the feeling that the financial resources the site has drawn upon are becoming increasingly restricted The mental health authority wants the CMT to understand these pressures and take a more active part in guiding the TCMHP at

PAGE 155

141 Site T through these changes The children's services director described the CMT response to learning about the importance of outcomes and the performance contrac t I think that they are tending to be less critical of mental health as a result of that because I don' t know that they were aware that we were under that much pressure as far as outcomes. How Out c om e Info rmation Impa c t s Decision Makin g at S ite T In spite of the increasing funding pressures felt at Site T, the outcome evaluation of the TCMHP is viewed as a marker of success The site views the scope of its services as going beyond the present scope of the TCMHP contract and the site exhibits a certain compliance savvy, having few issues resulting from difficulty in paperwork flow or accuracy Because ofthe relative success experienced at Site T outcomes have not been used as a tool for correcti v e action as was observed at Site Q but hav e instead encouraged a future orientation The performance contract requirements have however helped Site T administrators make a link between cost and service effectiveness. Family preservation services which were added to the Site T service array as a direct result of the TCMHP funding, were cited during the interview process as an example of this increased awareness of the link between cost and service effectiveness Changes in Medicaid rules have made it difficult for Site T to be full y reimbursed for the time of the family preservation staff spend providing intensive in-home services As a result, revenues are reduced and the cost of such intensive services continues to be high At the same time staff at Site T have found that because [famil y preservation] is so expensi v e it is not very much in line with the state' s new idea of gi v ing us target numbers

PAGE 156

142 of people we have to serve." The mental health director explained, "The performance contract made us pay closer attention to what we were doing out there in the field. We all had this perception that family preservation was a darn good service but they [the service providers] had to have low case loads We had never really entertained the idea of how the service was affecting our budget." This has resulted in a shift in how services are provided in order to increase the number of clients served The children's services director explained, "We've been encouraging the family preservation staff, when it is at all possible -and it is possible -to have families come to the centers Get some groups started go into the schools see kids in schools. As a result, they doubled the number of people who they were seeing, and they re still giving quality services." The direct service providers acknowledged that there has been a move to provide family preservation services in an office-based setting when possible and expressed satisfaction with the quality and intensity of these services Direct service staff and supervisors at Site T discussed how outcomes have focused greater attention on relationships with school personnel. The contract measure for improved school behavior was the one on which Site T had performed below standard This has prompted the staff to increase their contact with the schools. Although they do not have feedback yet on whether this strategy has been successful the staff reiterated the importance of the feedback from the outcome report influencing their interactions with the community in what they believe is a positive way In addition to interactions with the community the direct service staff expressed excitement at the opportunity to score CBCLs locally and have immediate feedback on the results (something viewed as a burden

PAGE 157

imposed by Austin at some of the other sites visited) Even while acknowledging perceived weaknesses in this instrument direct service providers commented on the usefulness of having immediate feedback on a child s level offunctioning 143 The direct service staff agreed that perhaps the single most important factor in helping them understand the importance of the outcome evaluation to the work they are doing was having Larry Rouse, from the state research and evaluation staff, conduct a six hour training on the TCMHP evaluation system for the child and youth staff at one of the quarterly clinical staff meetings One of the direct service providers interviewed commented that he spoke "in incredible detail about why this is important what kinds of things we are looking at, and how this has an impact on funding and on the contract we have with central office ." The direct service staff believed this training "put into perspective why the outcome evaluation was important. The children's services director described this connection as Job security Because they d be meeting the contract, they'd continue to get money from central office to fund their jobs." C onditions that Support or Impede th e Use o f Outcome Information The formal information flow within the mental health authority at Site T was described as "kind of from the administration down but the informal flow observed during the site visit was considerably more interactional than that description would allow. Feedback and exchange of information was observed to be a continual process moving up, down, and across levels of the organization The communication of outcome information to the CMT was observed to be more formal in the reporting of the data to the Team, but the actual process of discussion during the CMT meeting had an air of ease and familiarity

PAGE 158

144 in which suggestions and observations were offered freely between mental health staff and CMT members In general, the flow of outcome information at Site T could be said to be open and unrestricted. The specific factor most often identified as needed for improving the outcome evaluation reports was reporting the outcome information at a county level. This refrain was heard from administrators direct service providers, and the CMT members all of whom felt outcomes would be more useful in a less aggregated form This strategy, it was believed would help focus service planning and delivery decisions more directly toward particular community needs The need for more specific data extended to the measures of satisfaction The mental health staff believed that access to the commentary on these reports would provide better qualitative insight into what service delivery strategies were perceived by the community as successful and which were not. Although the mental health staff realized that the anonymity ofthe respondents must be protected, they found the quantitative data difficult to use or apply without additional descriptive or explanatory detail. In general these suggestions for ways the utility of the outcome evaluation might be enhanced reflect the opinion that the TCMHP evaluation would be improved by the addition of more qualitative data CMT members expressed a strong preference for qualitative data, saying they are more interested in the quality of services provided at Site T rather than quantification of the results of those services provided in the outcome reports Direct service providers at Site T believed that qualitative or descriptive data "would be more helpful than just a graph [referring to the bar graphs on the quarterly

PAGE 159

145 outcome reports]. The direct service pro v iders at Site T were clearly asking for additional outcome information rather than suggesting the current reports be replaced by another format. One emphasized "I mean, personally I'm real competitive so I like to know where we' re at compared to everybody else, so I don't want to give that piece up But I'm greedy and I want the written [qualitative] piece too. Because you know that's just easier for me to understand ." A member of the direct service staff provided an example of the need for further discussion of outcome results I thought it [an outcome report] was saying one thing and [the children s services director] had to explain to me that no it's saying something else And that might have been less likely to occur ifi could have seen something [providing more explanation] in writing. Administrative staff also comm e nted on the is s ue of outcome data sample siz es varyin g sig nificantly from one local mental health authorit y to another "It's frustrating," s aid the mental health director adding that he does not understand how a l o cal mental h e alth authority can be considered in compliance with their performance contracts with very low rates of data return The children s services director continued this discussion referring to their 71 percent satisfa c tion rating from collateral providers I have a feeling that s pretty accurate for us .... we had 155 people respond ." However she questioned whether a center's 100 percent satisfaction rate was truly belie v able on the basis of only ten returns As this discussion continued Site T s administration acknowled g ed that the local mental health authorities across the state would ha v e considerable v ariation in the numbers of children in service, a factor affectin g the rate of data return Still th ey

PAGE 160

146 suggested that the number of children in service be reported in addition to the data return rates so that the soundness of the sample size could be determined Summary of Site T Findings The findings for Site T can be summarized with respect to the guiding research questions for this study The organizational culture at Site T can be characterized as one focused on maintaining success Site T is actively involved in strategies to create transition in its CMT relationships which would build a stronger sense of shared responsibility in decision making for the TCMHP Informants within mental health consider themselves to be outcome information users and provided examples of how their awareness of outcomes is affecting service planning and delivery strategies There is a growing awareness of how outcomes are informing the link between costs and service effectiveness. Although informants within the CMT describe themselves as information users, it is clear they are being urged by the local mental health staff to build capacity in a more strategic use of outcome reports In spite of the general sense that outcome information is important at Site T several weaknesses in the TCMHP evaluation were identified and suggestions for improving the utility of the evaluation were made by informants at Site T. In general Site T s approach to outcome information utilization might be characterized by the phrase "We do it better! Outcomes have provided positive feedback of success at Site T but the undercurrent is clearly one of developing strategies to maintain their success

PAGE 161

147 Summary of Local-Level Results The results of local-level data collection for this study highlight significant differences in the cultural processes in evidence at the participating local mental health authorities Regardless of whether a site was considered a high or low user of outcome information three sets of relationships were identified as key to the processes of organizational culture that affect outcome information utilization These key relationships were : 1) the mental health staff ; 2) the Community Management Team ; and 3) the state level TCMHP staff Local organizational culture is viewed as an interactional and negotiated process and can be thought of as arising through interactions among these key participants in the evaluation Interviews provided data regarding relationships and interactions within each of the three categories of key participants noted above as well as interactions which extend to other key participants TCMHP participants For example site interviews provide insight into relationships within mental health at administrative and direct service levels In addition the interviews provide data regarding relationships of the mental health staff with the Community Management Team and state-level TCMHP staff The analysis of interactions among each site s key participants in the TCMHP forms the basis of an understanding of how outcome information is given meaning within a local context and how understandings of outcome information specific to each site develop. Using the format ofthe site selection matrix which structured a comparison of the sites identified as high and low-users of outcome information in terms of whether they were early or recent participant s in the TCMHP we can consider an overview of the four

PAGE 162

148 participating sites regarding the cultural processes observed during the site visits These are described on the table below. Table 8: Summary ofLocai-Level Results Sites Selected for High Information User Low Information User Information Pathways Early Participant in Site Q : SiteS: TCMHP Evidence of interactions which Evidence of interaction s which indicate strong partnerships in key indicate independent relationships relationships and form the foundation with key participants and provide of a dialog regarding outcomes and autonomy in the implementation of a create a sense of shared responsibility particular treatment philosophy in for chi ldren's mental health children s mental health. Recent Participant in Site I: SiteR: TCMHP Evidence of interactions which Evidence of interactions which indicate the strengthening of indicate isolated re l ationships with partner s hips in key relationships and key participants and contribute to building an environment of s hared fragmented respo n s ibility for responsibility for children s mental children's mental health among local health. child serving agencies. The results summarized above are discussed in the following chapter with respect to how each site links outcome information to service related decision making.

PAGE 163

CHAPTER SIX ANALYSIS AND DISCUSSION Building a Model of Utilization 149 The analysis of the data from this study suggested the need for a model that would allow for exploration of both how outcome information is interpreted at the local level and how knowledge about the utility of outcome information in local level service planning and delivery is created A model of outcome information utilization is presented in Figure 2 This figure serves as a heuristic device for understanding variation in the patterns of outcome information utilization The top portion ofFigure 2 provides an illustration ofthe interpretive process by which outcome information is given meaning and local understanding of the utility of this outcome information is developed. Six structural domains or defining characteristics of an outcome evaluation have been identified : 1) a clearly articulated vision/mission ; 2) a regulative function ; 3) a generative function ; 4) data credibility ; 5) data access ; and 6) training These structural domains although not immune to change can be thought of as fixed at any given point in time because they outline the primary assumptions about an outcome evaluation at the governing level and, thus, serve to define the basic structure of the evaluation A general discussion ofthe six domains is presented below and a more

PAGE 164

150 detailed discussion of how these domains relate to issues of organizational culture and utilization within the TCMHP is presented in the following section The first of these domains, a clearly articulated vision and mission, serves to tie the outcome evaluation to the goals ofthe programs or services being evaluated The relevance of the outcome evaluation to decision making at the local level, or what information the evaluation is tracking and why this information is being tracked, 1s clarified through the articulation of the vision/mission The explanation of the vision/mission should include the following : 1. Theory of change -a description of the theory of change being tested by the outcome evaluation which specifies who the target population for services is what services are to be delivered, and what the expected results of those services are; 2 Service delivery strategyan explanation of how the services to be delivered fit the theory of change; 3. Therapeutic approach-in children s mental health, a discussion ofthe therapeutic approach or approaches most suitable to the theory of change Within the TCMHP the stated vision/mission includes a theory of change which asserts that children and youth with severe emotional disturbance are best served through an integrated array of community-based services The TCMHP theory of change suggests a service delivery strategies include building a core of community-based services managed by an interagency team of children s services providers The results-oriented goals that establish a minimum requirement for the numbers of children with serious emotional disturbance to be served in each catchment area and establish interagency outcomes such as a reduction in rearrests rates and improved school behavior illustrate how expected

PAGE 165

results can be tied to a theory of change While no particular therapeutic approach is specified for the TCMHP, the emphasis on opening access to broad-based community mental health services for children suggests a strategy of short-term intervention rather than longer-term therapeutically focused treatment. 151 The second structural domain is a regulative function for the outcome evaluation The regulative function is a formal governing function and can be described as operating in a context-free environment in that it provides uniform regulation across organizations at the local level and is removed from the context and complexities of these individual organizational cultures in its implementation The regulative function in outcome evaluation serves to set minimum performance standards for participating organizations, provide continuity of services, and provide comparability across local service delivery sites In the TCMHP the regulative function of the outcome evaluation includes performance contract requirements to provide certain basic or core services and to achieve certain contracted outcomes as a result of those services The TC.MHP regulative function also requires the establishment of an interagency community management team to oversee the local administration of the TCMHP The third domain the generative function serves the purpose of fostering a context-rich environment for outcome-based decision making The generative function promotes an ongoing process of interpretation and communication of outcome information The generative function can be described as fostering a context-rich environment in that it reflects the needs experiences, and complexities at the local level through the provision of local-level flexibility in the development of service planning and

PAGE 166

152 delivery strategies For example, the generative function in an outcome evaluation would provide the opportunity to respond to local or community needs by providing decision alternatives that allow services to be tailored to local issues and concerns The generative function in an outcome evaluation might also include the provision of rewards and incentives for improved performance on outcome objectives as well as the opportunity to make service-related decisions based on community and organizational strengths and what has been learned about best practices through the outcome evaluation The stated goal of the TCMHP outcome evaluation has been to allow local tailoring of programs based on feedback from the outcome evaluation. However the ratio of funding to the core service requirements the absence of an incentive structure such as a system of financial reward for local organizations that are successful in meeting performance contract goals, and the implementation of punitive financial sanctions that are imposed when contract requirements are not met all serve to significantly limit the generative aspects of the TCMHP outcome evaluation The fourth domain involves the credibility of the outcome data This function includes perceived validity of the data, whether the data are seen as representing what has actually occurred and data integrity whether the data is considered trustworthy or believable by stakeholders of the evaluation Interview data indicate that local sites generally trust the integrity of the outcome data reported on their own site However, inconsistencies in how data is reported across the state have contributed to a lack confidence that results reported across the state are trustworthy The measures used in the evaluation are generally considered valid although there is some local-level concern

PAGE 167

that terms such as "satisfaction" and improved school behavior" are not uniformly understood by the key stakeholders of the TCMHP at the local level. 153 The fifth structural domain of outcome evaluation presented in this model is the issue of access to the reported outcome results This would include consideration of the timeliness and regularity of outcome information, the format and language in which the outcome information is presented, the quantity and complexity of the data being reported, the method of distribution of the outcome results and the availability of specialized reports when a more detailed analysis of outcome data is needed The outcome evaluation of the TCMHP is structured to provide timely and regular reporting of the data The evaluation also limits the quantity of data reported through the evaluation to a pre determined number of measures and seeks to simplify complicated data by summarizing the results in a graph format and in user-friendly language Outcome information can be accessed at the local level through the state-wide computer network. In addition, specialized reporting is available upon request. The sixth structural domain of outcome evaluation presented in this model is training The model suggests two aspects oftraining are important: 1) building an understanding of how to effectively participate in the outcome evaluation and 2) an ongoing effort to develop an understanding of the goals of the evaluation as they relate to the vision/mission ofthe program being evaluated The TCMHP provides a detailed "how to" handbook about the TCMHP outcome evaluation that is straightforward and has been regularly updated There do not appear however to be any formal procedures in place

PAGE 168

for ongoing training regarding the vision/mission of the TCMHP and the role of the outcome evaluation in attaining TCMHP goals 154 Local organizational culture is also illustrated in the top portion of Figure 2. Local culture is defined as arising from the human reactions and interactions among key participants in the evaluation at the local level. The interface between the structural domains of the evaluation design and the local organizational culture are illustrated in Figure 2 as the intersection of the two circles As a result of the interface between the local culture and the domains of the evaluation design local knowledge of outcome utilit y, or what is known to be true and meaningful about the utility of outcome information develops According to the model greater overlap between the local culture and the structural domains of the outcome evaluation yields increased utilization of outcome information at the local level. Because the meaning of outcome information must be interpreted at the local level before it can be included or excluded from local level service planning and delivery processes and because of the potentially different ways the local organization s culture interfaces with the domains ofthe e v aluation desi g n the local interpretation of outcome utility is potentially different for each local organi z ation The lower portion ofFigure 2 illustrates the link between acquired knowledge of outcome information and the actions or decisions taken by local mental health authorities on the basis of what has been learned about the utility of outcomes Differentials in the interpretation of outcome information yield potentially different modes of utilization These modes of utilization represent how outcome information is related to service

PAGE 169

155 planning and delivery decisions at the local level. It should be stressed that regardless of the degree to which outcome information is determined to be useful, local decisions relating to service planning and delivery are made Therefore, differing modes of utilization illustrate that decisions made at the local level can and do involve the use of outcome information to varying degrees

PAGE 170

Figure 2: Linking Information to Action Linking Information to Action A Model for Outcome Information Utilization in the TCMHP Structura l Domains of Outcome Evaluation Vision I Mission Regulative Function Generative Function Data Credibility Access Training Outcome Acquiring Knowledge Thr o ugh the Interpretation of Outcome Information Local Organizational Culture Arising from i ntera ctions among key participants at the local level Mental Health State TCMHP Staff Community Management Team Mode of Utilization Relating outcome information to service planning and delivery Action Taken on the basis of what is learned in the form of change or no change 156

PAGE 171

157 Linking Information to Knowledge in the TCMHP The link between outcome information and acquired knowledge about the utiljty of this infurmation involves the interaction between the structural domain of the TCMHP and the cu lture of the local mental health authority The culture of the local health authority is thought to arise from interactions among key participants in the 1'CMHP at the local level. The interview and observational data suggest that three TCMHP participants are integral to the local culture at each site : I) the staff of the local mental health authority ; 2) the members ofthe CMT, who represent the collateral pro viders of the children's services for the site ; and 3) the state TCMHP staff The data 1nd1cate that the nature of these relationships at each site results in distinct and different local cultural processes and that these distinct local organizational cultures interact differently with the TCMHP evaluation design at each site The relationship between the local culture and the TCMHP evaluation design d:i:ffer.s at each site in the degree of shared knowledge between the two. Greater overlap between the evaluation design and the local culture indicates greater sirrUlarity in local and state v iews of the TCMHP and its evaluation and greater similarity in state and local interpretation of experiences with the TCMHP and its evaluation This increased overlap y ields higher utilization of outcome information in service planning and delivery decisions The data for tltis s tudy provide insight into this relat io nship at both the highand lowuse sites and are disc ussed below The discussio n below relates each site s local culture to the e l ements of t h e TCMHP e valuation de s ign and identifies factors that both

PAGE 172

increase and decrease the degree of intersection between the local culture and the evaluation design High U s e Sites 158 Sites Q and T have both been identified as high users of outcome information in decisions that affect local service planning and delivery A comparative analysis however reveals important differences in local culture at each site which lead to different bases of acquired knowledge about the utility of outcome information Several characteristics of Site Q s organizational culture can be identified as contributing to patterns of high utilization of outcome information The local culture at Site Q is characterized by a strong partnership with state-level TCMHP staff and long standing partnerships with collateral providers both of which foster robust discussion and multi-directional communication about outcomes throughout the organization As a result a shared understanding of the vision and mission of the TCMHP has been firmly established and a local commitment to providing the kind of community-based interagency services articulated by the TCMHP is well developed In addition, the cross-agency outcomes tracked through the TCMHP evaluation are considered a useful way to monitor the results of local service delivery This serves to increase the intersection between the local culture and the structural domains of the TCMHP. Even though the expectations set by the performance contract are considered a challenge and somewhat unrealistic the multi-directional communication and partnership that exist at Site Q have contributed to a shared belief that tracking outcome information is a viable way to build accountability in human service systems As a result the

PAGE 173

regulative function of the TCMHP evaluation is accepted, even when its limitations are acknowledged 159 Receipt of TCMHP evaluation information at Site Q initiates a generative process of interpretation and the reciprocal communication of the meaning of outcome results as they relate to local service delivery strategies. The well-established partnerships at Site Q contribute to a sense of cooperation and shared responsibility among the mental health staff and the collateral child-serving agencies that provide some degree of local flexibility and action alternatives in service-related decision making Although the TC:MHP structure tends to limit generative processes the degree of partnership characteristic of Site Q s organizational culture provides service-related decision alternatives that would otherwise not exist without the combined resources of multiple agencies responsible for the local implementation of the TCMHP. The intersection of the local organizational culture with the TCMHP structural domain of data credibility serves to both increase and decrease the degree of outcome information utilization at Site Q Although Site Q's own outcome information is considered credible concerns about data reporting practices across the state serve to reduce the credibility of state-wide data and thus the comparability of outcome data across the state. In addition the frequent need to reconcile the reports generated through the locally managed and state-managed data bases calls the credibility of the data into question Another fa c t o r increasing the intersection between the local culture and the TCMl-IP evaluation domains i s the accessability of feedback Site Q regularly requests a

PAGE 174

160 county-by-county aggregation of outcome results from the state TCMHP staff to provide a more detailed analysis for decision making and has also made requests for supplemental information in the form of special reports In addition communication patterns within Site Q ensure ready access to outcome information and feedback on that information throughout the local organization. Finally the local organizational culture s interface with the training domain of the TCMHP has contributed to increased utilization Although no need for formal training in TCMHP processes was indicated by the interview data there was evidence of an on-going process of informal training throughout Site Q about how outcome information can be used and about the history of the TCMHP in the form of regular communication throughout the local organization Established partnerships with state-level TCMHP staff have contributed to an environment in which the local organization has an open dialog with the state In addition the dialog about outcomes extends to the direct service level where the discussion of outcome results is a regular part of the clinical supervision of direct service staff as well as into the community where CMT members report relating the importance of the TCMHP and the outcome evaluation to the staffs of their collateral agenctes Considering these points of i ntersection between the structural elements in the TCMHP evaluation design and Site Q s culture of partnership a high degree of overlap is evident as is summarized on the table below

PAGE 175

161 Table 9: Factors Affecting the Degree oflntersection between the TCMHP Evaluation Design and Local Culture at Site Q Factors Reducing Intersection Factors Increasing Intersection CMT finds contract expectations Embraces system of care approach: mental he a lth considered somew h a t unrealistic one of nine agencies in the TCMHP Mental he a lth finds contract expectatio n s Finds TCMHP cross-agency outcomes usefu l although cont ract a challenge targets are a challenge State-wide TCMHP data l ack c redibility Perceives system as providing some l oca l flexibility and action Mus t reconcile state and l ocal data bases alternatives Find s environment of change reduces believes l ocal TCMHP data is credible local flexibility and action alternatives believes outcome information is timely and regular finds feedback from state very accessible and periodically reque sts county -level aggregation of outcome re su lts Like Site Q severa l elements of Site T's organizational culture can be identified as contributing to patterns of high utilization of outcome information The local culture at Site T is characterized by efforts to maintain their current s u ccess This local mental health authority ha s developed a strong partnership with the state and has estab l ished a willing, if not fully developed partnership with collatera l providers on their CMTs Site T demon st rates flexible and open discussion a bout outcomes at all levels of the organization. Although a concerted effort is being made to build b e tter understanding of outcomes and contract performance requirements among CMT members the menta l health staff demonstrate a good understanding of how outcomes have and can be used within the organization. Co ntributing to the increased interface between the local organizational culture at Site T and the structura l e l ements of the TCMHP evaluation design is the commitment that exists at the local l eve l to building a community-based array of interagency services to serve children with severe emotiona l disturbance This provides the basis of a vision and mis s ion for children's mental health services that is shared at both the s tate and local

PAGE 176

162 levels Although Site Q is attempting to build the capacity of the CMT structures to participate effectively in decision making based on outcome information, there exists a commitment within the local mental health authority to the idea ofbuilding accountability in human service agencies through the monitoring of outcome information Cross-agency outcomes are considered a valid way of assessing the success of service delivery strategies Due to S i te T' s early success with achieving performance contract expectations, the contract targets set by the state TC:MHP staff are considered reasonable These factors contribute to a basic acceptance of the TCMHP s regulative function even while acknowledging limitations of the evaluation, such as the perception by mental health staff and CMT members that the evaluation results are overly quantitative The receipt of outcome information from the TCMHP evaluation begins a generative process of interpretation and problem solving regarding service delivery at Site T. Although the CMTs at Site T have not been actively involved in decisions based on outcome information, a pattern of open-ended and reciprocal dialog about service delivery and the well-established partnerships among interagency providers of children's services have created a sense of cooperation and shared responsibility that seems to provide a sense oflocal flexibility in decisions relating to service delivery Both mental health staff and the CMT commented that the utility of outcome information would be greatly increased however, with the addition of qualitative explanations of the outcome results Issues of data credibility at Site T both support and impede the use of outcome information Local results reported through the TCMH.P are believed to be credible by the

PAGE 177

163 local mental health staff However concerns about data reporting practices across the state raise concerns about the credibility of data for purposes of statewide comparisons Although one characteristic of Site T' s organizational culture is an established partnership and ongoing dialog with the state TCMHP staff, a perceived lack of access to a county-level reporting of outcome results limits its utility in service related decision making In fact, a county-level breakdown of outcome information is available from the state upon request but mental health authorities unaware of this option cannot make use ofthis additional information Within Site T communication pattern s which provide for ready access to outcome information and feedback on that information is a cultural characteristic which contributes to the increased overlap between the local culture and the structural domains of the TCMHP outcome evaluation. Finally with regard to the training domain the well-established partnership with state-leve l TCMHP staff led Site T to request a formal training session from the state on both the purpose and the functioning of the TCMHP outcome evaluation In addition the ongoing dialo g within mental health re ga rding the meanin g of outcome information in the day -to-day work of the staff provides on ongoing reinforcement of the formal trainin g received from the state Considering the intersection between the TCMHP evaluation design and Site T' s cu ltu re of maintainin g success through strong partnerships and ope n and reciprocal communication a h ig h de gree of overlap is evident as is s ummari zed on the table below

PAGE 178

164 Table 10 : Factors Affecting the Degree oflntersection between the TCMHP Evaluation Design and Loca l Culture at Site T Factors Reducing Intersection Factors Increasing Intersection CMT perce ives o u tcome reports as B uildin g sys tem of care approach and id ea that mental healt h is overly quantitative one of nine agencies in the TCMHP among CMT members Staff woul d like more qu a l itative data and Finds TCMHP cross agency outcomes useful, particul arly in narrative e x planati on of re s ult s l inking costs to service effectiveness Finds contract targets Staff questions credibility of sta te-wide reasonable TCMHPdata Perceives system as providing some l oca l flexibi l ity and action Needs better feedback from state alternatives particularly in form of countyl evel 0 Believes local TCMHP data is credible reporting of outcome re sults believes outcome information is timely and regular Mental h ea lth finds envir o nment of Formal training by state TCMHP staff contributed to increased change re duce s l oca l flex ibili ty and action ca p ac ity to use outcome information at the l oca l leve l alternatives Low Use Sites Sites R and S have both b een identified as low users of o utcome information in decisions that affect local service planning and delivery Although both are low use sites a comparative analysis reveals important differ ences in local culture which lead to diffe rent bases of acq uired knowledge at each site The relationship between the local culture and the TCMHP evaluation design differs at these sites in the de gree of shared knowledge between the evaluation design and the local cu ltur-e. Less overlap between the evaluation design and the lo ca l culture indicates less similarity bet ween l oca l and state views of the TCMHP and its evaluation and less similarity in state and local interpretations of experiences with the TCMHP and its evaluation This decr eased overlap yields lower utilization of outcome information in service planning and delivery decision The data for this study provide insight into this relationship at the low us e sites and are discussed below

PAGE 179

165 The local culture at Site R is characterized by isolation from state-level TCMHP staff and weak partnerships with collateral providers both of which generate strong feelings of frustration associated with the TCMHP evaluation and efforts to make use of outcome information Communication at Site R is restricted and tends to be uni directional, flowing from administration to the direct service staff and CMT members with limited discussion The Site R cultural characteristics of isolation from the state and underdeveloped partnerships at the local level have contributed limited interface between the vision/mission of the TCMHP at the state and local levels The data at Site R indicate that the vision of mental health as one of nine agencies carrying out the TCMHP at the local level and the mission of the TCMHP as building a core of community-based interagency services for children with severe emotional disturbance is not widely held at the local level. As a result it is difficult to relate the results of cross-agency outcomes to decisions affecting service planning and delivery at the local level. Site R data indicate a hig h degree of staff turnover within mental health both at administrative and direct service levels In addition, there has been a high degree of turnover among CMT members This turnover seems to have resulted in faint memory of the inception of the TCMHP and its vision/mission The individuals interviewed at Site R did not speak clearly about theory of change that underlies the TCMHP, the resulting service delivery strategies or the therapeutic perspectives that might best allow them to carry out the stated mission of the TCMHP. Although these conditions tend to reduce the interface between and local organizational culture and the domains of the TCMHP evaluation desi g n two factors

PAGE 180

166 serve to increase this intersection Recent stability in the mental health staff and the enthusiasm of the new CMT chair provide opportunities for developing a clearer and more widely held local understanding of how the TCMHP evaluation can be related to service delivery strategies In addition, the mental health staff expressed the belief that information about best practices and what works well would aid them in service planning and delivery Generally, the performance contract was regarded as overly regulative and as being imposed with little understanding by the state of the difficulties and challenges of delivering children's mental health services at the local level. The mental health staff also perceived no local flexibility the TCMHP In fact Site R is more reliant on TCMHP funding than the other sites participating in this study so that the ratio of funding to the state's core service requirements might serve as a more significant constraint on generative processes at Site R In addition the underdeveloped local partnerships among child-serving agencies create a situation in which the mental health authority bears the burden of planning and delivering services almost entirely and this effectively reduces the flexibility and generative processes that might be possible through true interagency collaboration In the absence of strong interagency ties at the local level and a strong partnership with the state mental health staff are frustrated with the requirements of the TCMHP and express the belief that the only way to comply with performance contract demands is to release children from services prematurely in order to bring additional children into the system

PAGE 181

167 Because outcome information was not widely used at Site R, the issue of data credibility did not arise in interviews at this site In addition, access issues such as the timeliness and regularity of outcome reports seemed irrelevant because information is not used in service-related decisions. However the need for a reporting of outcome information at the county level was noted as a change that would make outcome more relevant and possibly more useful. Staff at Site R were not aware that county-level data could be requested from the state TCMHP staff Although Site R seemed to operate in isolation from the state TCMHP staff, the local mental health staff expressed an open and willing attitude toward learning about how to use outcomes in service planning and delivery and Site R administration described their efforts as being increasingly directed toward and outcome focus Although references to learning more about using outcome information were generally restricted to their need to build an understanding of how to effectively participate in the outcome evaluation training efforts directed developing an understanding of how the evaluation relates to the vision/mission of the TCMHP could effectively the interface between the local organizational culture and the structural domains of the TCMHP evaluation Considering the intersection between the TCMHP evaluation design and Site R's culture of isolation a low degree of overlap is evident as is summarized on the table below

PAGE 182

168 Table 11: Factors Affecting the Degree oflntersection between the TCl\.1HP Evaluation Design and Local Culture at Site R Factors Reducing Intersection Factors Increasing Intersection Weak interage n cy partnerships prevent m ental health from Recent stabi lity in mental health staff and working as one among nine agenc ie s in the TCMHP enthusiasm of new CMT chair provide Cross-agency o u tcomes hard to relate t o service delivery opportunities for development Staff perceive no loca l flexibility in TCMHP Staff believes information about best Staff views outco m es as directives from Austin that do not practice s and what works woul d aid them adequately consi der the demands on their time. Staff express in serv ice planning and delivery frustration at the difficulty they experience in trying to use Staff open to learning more about how to outcomes in developing service s trategi es use outcomes and administration Staff perceptions are that availab l e action alternat ives requir e describes orga nizati o n as changing them t o re l ease children from treatment prematurely in order to toward an outcome focu s bring additiona l children into sys tem Timeliness and regula rity of outcome information is irrel evant because information does n o t assist in service -related deci sions Need better access to feedback from sta te p artic ularly in form of county-level reportin g of o ut come result s Moving the discussion to Site S the data indicate that the culture at Site S is characterized by independence Relationships with state-level TCl\.1HP staff, while amiab l e, provide little support for service planning or delivery at Site S. Relationships with CMT members are purposely autonomous allowing a high degree of independence in the delivery of children s mental health services Site S' s strong commitment to a therapeutically-driven service delivery strategy is in direct conflict with the TCl\.1HP's strategy of building multi-agency systems of care This places SiteS' s theory of change which focuses on psychoanalysis and is more childcentered and non-directive in contrast with the vision/mission ofthe TCl\.1HP ofbuilding an integrated array of community-based mental health services for children with severe emotional disturbance As a result the cross-agency outcomes provided through the TCMHP outcome evaluation have little meaning for a child-focused therapeutic strategy

PAGE 183

169 of service delivery This serves to reduce the interface between the local organizational culture and the TCMHP evaluation design Although this fundamental difference in theories of change prevents virtually any overlap between the outcome evaluation and the local organizational culture at Site S efforts to preserve the autonomy of their therapeutic approach mean that mental health staff at SiteS take the regulatory function of the TCMHP evaluation quite seriously. Site S 's primary strategy for preserving their autonomy and securing funding is through careful efforts to comply with their performance contract. Similarly s y stem flexibility and decision alternatives are directed at maintaining independence and further developing the therapeutically-driven service delivery strategies Although inconsistencies in reporting outcome data across the state call the trustworthiness of the statewide data into question credibility of TCMHP data was not raised as a significant issue at Site S because these data are not used in service related decisions Site S relies instead on their local data base for any service-related feedback. Issues of access to TCMHP outcome information were not raised during interviews at Site S Considering the intersection between the TCMHP evaluation design and Site S' s culture of independence very little overlap is evident as is summarized on Table 12 Linking Knowledge to Action : Cross-Site Analysis If culture is defined as the process of using acquired knowledge to g ive order to the world interpret experiences and generate social behavior then the different social

PAGE 184

170 Tab l e 12 : Factors Affecting the Degree oflntersection between the TC:MHP Eva l uation Design and Loca l Cu lture at Site S Factors Reducing Intersection Factors Increasing Intersection Strong commitment to ther ape utic approach to service delivery Preserve a utonomy of therapeutic Cross-agency outcomes h av e no meaning in child-focused therapy approach through contract System fle xi bility and action alternatives directed at maintaining compl iance indep e nd ence Contract compliance sec ure s funding Site relies on internal dat a base f o r feedback rather than sta te syste m Timelines s, regularity and accessability of feedback do not rel ate to s ervice planning and delivery dec i sio n s behaviors generated as a result of each site's knowledge regarding the utility ofTC:MHP o u tcome information for service p l anning and delivery decisions can be said to invo lve varying modes of outcome utilization Each of the four sites investigated in this study re l ate outcome information to service p l anning and delivery decisions in distinctly different ways It should be noted that t h e information utilization strategies at all of the sites invo l ved some degree of contract comp l iance as a result of the financia l sanctions associated with not fulfilling the contract agreements Even considering the emphasis on con trac t comp l iance the sites demo n st r ate d different strateg i es for out come uti l ization High Use Sites Site Q' s dominant cu l tura l characteristic of part n ership and Site T' s domi n ant cultural characteristic of maintaining success both support behaviors of outcome uti l izat i on in the sites' service planning and delivery decisions. However the c u ltural characteristics at each site result in considerab l y different modes of utili z ation or ways of re l ating outcome information to service strategies Site Q's partnerships, fo r example, result in a mode of utilization that can best be described as problem solving Site Q has both the willingness and abi l ity to use outcome

PAGE 185

171 information in service planning and delivery, and informants from within mental health as well as collateral providers of children's services offered clear examples of how outcome information has influenced serv ice planning and delivery decisions For Site Q, outcome information serves as a report card, providing necessary feedback to current service related activities. The TCMHP outcome reports are used to gauge the effectiveness of services and to inform the development of alternate service strategies Site Q is proactive in its mode of utilization, requesting additional information from state-level TCMHP staff as needed Outcome reports at Site Q offer opportunities for adjustment and mid-course correction as well as positive feedback on successes When placed on a time continuum of past present and future Site Q's mode of utilization places the emphasis of their energies in the present using outcome information as immediate feedback to solve present day problems The efforts to maintain success at Site T have resulted in a mode of utilization that can best be described as increasing awareness Site T has the willingness to use outcome information and is developing the site's capacity for its use through the strategy of building the CMTs' understanding ofthe importance of outcome information To date outcome information has been used as evidence ofthe site s success in delivering services but increasingly Site T is using outcome information as a way to link cost and service effectiveness. Site T is becoming more proactive in its mode of utilization and is beginning to consider how additional information from state-level TCMHP staff might be useful. Outcome reports at Site T have up to this point offered positive feedback on successes but mental health staff at Site T believe the environment in which they provide

PAGE 186

172 services is changing significantly When placed on a time continuum of past, present and future, Site T' s mode of utilization places the emphasis of their energies in the future, using outcome information as a tool for dealing with issues and stresses they see looming on the horizon Low Use Sites Site R' s dominant cultural characteristic of isolation and Site S' s dominant cultural characteristic of independence both support low outcome utilization behaviors in the sites service planning and delivery decisions. However the cultural characteristics at each site result in considerably different modes of utilization, demonstrating that low utilization can manifest itself in noticeably different ways of relating outcome information to service planning and delivery Site R s isolation from state-level staff and local partnerships with child-serving agencies results in a mode of utilization that can best be characterized as compliance oriented Site R' s mental health staff want to make outcomes work and to do a good job serving children and families The staff demonstrates good knowledge of TCMHP outcome reports and expresses a willingness to put outcome information to use, but express an inability to do so Mental health staff express a willingnes s to follow the rules of the TCMHP evaluation, but are unclear about how the outcome results relate to the work they do each day The pressure of contract compliance and the belief that these outcome requirements have been handed down from the TCMHP at the state level have obscured the utility of outcome information in Site s R s efforts to develop service strategies For Site R, outcome information serves as a reminder of what they haven t

PAGE 187

173 been able to accomplish, despite their hard work. When considering outcome utilization in relation to the time perspectives of past, present and future Site R's mode of utilization places the emphasis of their energies in the past with feedback both reinforcing their frustration at trying to meet performance contract requirements and seeming to lack the ability to assist in solving related service delivery issues Site S' s desire for independence from state-level TCMHP staff and local providers of children's services results in a mode of utilization that can best be described as figurative Because of Site S' s deep commitment to therapeutically-driv en services the TCMHP outcome information simply does not relate to the work they do. Site S has the ability to use outcome information in service planning and delivery decisions but lacks the willingness to do so M e ntal health staff at Site S express the strong conviction that their therapeutically-driven service strategy is the best long-term solution for children and families Submitting the TCMHP outcome data and evaluating the outcome information is considered a necessary chore at Site S but one that is taken seriously because outcomes rated as by the TCMHP outcome evaluation provide them with funding and their much desired autonomy An attempt to place SiteS's mode of utilization on a time continuum of past present, and future was unsuccessful because of the minimal connection Site S makes between the outcome information and their service delivery strategies Shared Characteristics of High and Low Use Sites In spite of the different modes of utilization present at each of the sites several characteristics surfaced that were common to highand low-use local mental health authorities The high-use sites share a value for a systems of care approach to service

PAGE 188

174 delivery Even while acknowledging the limitations inherent in the TCMHP approach, which emphasizes building a base ofbasic services and opening access to as many children and families as possible, the high outcome sites continue to build strong inter-agency partnerships at the local level and find that the cross-agency outcomes that are part of the TCMHP evaluation inform the work they are doing. Generally, the high-use sites view outcomes as part of their job and an enhancement for what they do The low-use sites do not show the same support for the TCMHP approach to inter-agency service delivery, although their reasons for this differ significantly from one another. In the case of Site S an interagency base of services and cross-agency outcomes do not provide useful feedback to their therapeutically-oriented approach to service delivery. The TCMHP outcomes are a moot point from this perspective. Unlike Site T, the staff at Site R do not convey a commitment to a particular approach to service delivery. Site R mental health staff do, however, express frustration that the CMT does not understand their work and the CMT communicated strong feelings that the mental health staff at Site R were unwilling to move beyond their traditional, office-based counseling approach to service delivery. While this conflict does not indicate the strong commitment to an alternative approach to service delivery such as the one identified at Site T it does suggest less understanding of or commitment to the cross-agency system of care strategy for developing children's mental health services that is implicit in the TCMHP design. The two high-use sites demonstrated a strong sense of partnership at both the state and local levels Although Site T is in the process of moving its CMT toward more

PAGE 189

175 involvement with outcomes and developing service delivery strategies both sites offered examples of working partnerships that contributed significantly to the local-level services Site S s local-level partnerships were congenial but kept purposefully at arms length. This did not appear to be because of any friction between collateral agencies and the local mental health authority, but rather out of a convict i on that mental health has its job to do, and the collateral agencies are quite involved in their own work. Mental health staff at Site R, on the other hand, express frustration at being held accountable for cross agency outcomes such as improved school behavior and reduced juvenile rearrest rates Indeed from the perspective of a site with weak inter-agency partnerships cross-agency outcomes such as these may seem unrelated to their work and of limited usefulness in solving service delivery problems that are associated primarily with the mental health system. The multi-system outcomes ofthe TCMHP were designed to evaluate service delivery systems in which mental health is a partner among nine child-serving agencies The frustration expressed at Site R may be an understandable result of trying to accomplish these outcomes without the support of interagency partners at the local level. Another characteristic of high use sites is the tendency to view the TCMHP as an ongoing process rather than an end result. For Site Q receiving outcome reports was the beginning of a pro c ess of solution-focused discussion both within the mental health authority and amon g interagenc y partners Each report whether monthly or quarterly was cons idered part of a c y cle of evaluation adjustment reevaluation and more adjustment and the mi x of services continues to be developed and adapted to the changing need s of th e co mmunit y Sit e T e xhibited a commitment to using outcome information as

PAGE 190

176 an ongomg process The TCMHP outcome information is assisting their efforts to thrive in an environment of increasing change by helping them link costs with issues of service effectiveness Both low use sites tended to view the TCMHP evaluation as an end result rather than an ongoing process At Site R, the mental health staff was interested in learning what worked but the implication was that this information would be used to implement that specific change rather than for the purpose of making ongoing adjustments in response to regularly received feedback Site S' s interest in evaluation was for the specific purpose or end result of securing funding and therapeutic autonomy from state-level TCMHP staff and local level collateral providers Another observation common to the high-use sites was the permeation of outcomes at all levels of organization Discussion of the outcome information was detailed and involved at all levels of Site Q s organization CMT members at Site Q typically reviewed outcome results prior to CMT meetings in preparation for discussion and sometimes brought pertinent issues to pers onnel within their collateral agencies prior to CMT meeting so that their feedback to the CMT would be representative of their agency s position Similarly discussion of outcome results at Site T moved easily from dir ect service through top administrative staff at Site T. Administrators typically sought feedback from direct service staff on outcome results and were increasingly bringing this discussion to their CMT. By contrast, evidence of outcome discussion throughout the organizations was not present at the low-use sites Site R s direct service staff, while well-informed on the

PAGE 191

177 content and focus of outcome information reported that discussion of outcome information was not an important element of staff meetings CMT members at this site were not fully aware of the outcome reports or performance contract requirements At Site S direct service providers were not actively involved in any discussion of outcome information and one described the direct service staff as protected from outcome reports Similarly the CMT entertained only minimal discussion of outcomes reports at quarterly meetings. Perhaps the strongest thread linking the high use sites was the dominance of narrative forms of communication in their use of outcome information to inform service planning and service delivery According to Weick and Browning ( 1986) narrative patterns of communication in organizations are based on interpretation and building understanding Narrative forms of communication assume that all people within an organization have narrative capacity, or the ability to tell stories and share experiences that relate their experience to the world around them Narration focuses on communication as a reciprocal process allowing the sender and receiver to shift roles during discourse Because communication becomes an iterative process in its narrative form, narration takes the form of exchange rather than unidirectional communication At high use sites narrative forms of communication were observed in several ways For example, Site Q s communication of outcome information took multiple forms and were communicated throu g h multiple media: it was distributed in written form brou g ht into discussion at meetings and illustrated with overheard transparencies More importantly however there was an easy and multi-directional sharing of ideas about

PAGE 192

178 outcome information, what it means and how it might be used in the service planning and delivery decisions at this site that allowed communication about outcome information to be both a reciprocal and iterative process At Site T, communication patterns were also observed to be multi-directional, and conversations about outcome information were a flexible and constructive process during which ideas were easily conveyed and meanings were easily shared Even among CMT members at Site T who were in the process of learning more about outcome information, the flow of ideas advice, and recommended solutions to issues identified through the outcome information was easy and unrestricted. In general when outcome information was communicated at the high use sites it was used as a vehicle to generate questions both impl i ed and direct about its meaning This strategy of using outcome information to generate questions opened discussion and fostered interpretive processes. What does this information tell us, what do we understand it to mean, how should we respond to it? The answers to these questions was not assumed in advance The meaning ofthe outcome information was not presupposed and discussion was used as a tool for building understanding and coming to shared meaning about the significance of the information and how this information could be used The narrative style of communication used at high use sites implied to participants that there was the potential for change and adaptation as a result of the outcome-related discussion The communication style as it relates to using outcome information in service planning and delivery at the low-use sites was more likely to be linear and unidirectional

PAGE 193

179 following the mode of communication described in the literature as argumentation Weick and Browning ( 1986) describe argument as a form of discourse which is logical and rule driven In argumentation decision making tends to be clear-cut and hierarchical. It should be noted that there was simply less communication about outcome information at the low use sites. Communication regarding outcome information tended to be unidirectional at the low-use sites focused on getting information from the sender to the receiver The communication flow did not demonstrate the shift of roles between sender and receiver characteristic of narrative styles of communication When outcome information was infused into discussion at the low use sites it was absent of the implied or direct questioning of its meaning that was evident at the high use sites. At low use sites, the meaning of outcome information was inferred before it was included in discussion For example at Site R, where potential users had difficulty applying the outcome information to their day-to-day work, the discussion of outcome information was not generally accompanied by questions about its meanin g The outcome information instead tended to reinforce shared understandings of frustration with the difficulty of meeting the performance contract requirements and overwhelming demands for serv ice delivery At SiteS, outcome information was a ss umed not to have relevance for their work and therefore did not rai se questions about how services might be changed or adapted in res ponse to it. The hiera rc hical structure regarding outcome information was particularl y e v ident at SiteS in that outcome information was the domain of the children's clinical director and th e direct service staff were "pro tected from it.

PAGE 194

180 In general, when outcome information wa s communicated at the low-use sites it was not accompanied by questions either implied or direct as a tool for opening discussion and building meaning Instead the meaning of outcome information and the absence of its utility or app l ication was assumed in advance of the transfer of this information from sender to receiver. The transfer of outcome information at low-use sites implied to participants that there was no potential for change and adaptation as a result of the receipt ofthe information Lessons Learned : Maximizing the Utility of Outcome Information Increasing the utility of outcome information for service-related decisions means increasing the degree of interface between the structure of the outcome evaluation and the organizational culture in which the evaluation is occurring In order to maximize the utility of outcome information human service systems must carefully consider both the structural domains of their outcome evaluation and the interface between this evaluation structure and the organizational culture in which the evaluation is being implemented When establishing a system of outcome accountability, a great deal of energy is appropriately invested in planning the structural aspects of the outcome evaluation However, focusing on evaluation structure without adequate consideration of the cultural processes that will influence its implementation restricts the potential for the results of the outcome evaluat i on to provide crucial feedback on the effectiveness of service-delivery strategies. Regulativ e planning in organizations is defined as the implementation of formal plan s whi c h typi c all y re l y o n ins titutional or coercive power (Uzell 1990) Regulative

PAGE 195

181 planning tends to extend top down within an organization and seeks to make the organization conform to a set of ideal standards or forms Regulative planning is characteristically rigid and is context free in its application. The structure of an outcome evaluation serves an i mportant and necessary regulative function within a human service system by providing uniformity continuity, and comparability across local service-delivery organizations. Local or program-level decision making, however, occurs in the complex and often unique environments of d i rect service delivery and requires generative processes of interpretation and communication in order to successfully bridge the complexities of local needs and allow services to be tailored to local issues and concerns Generati v e planning in organizations which relies less on top down communication implies an ongoing process of interpretation throughout the organization. The interpretive processes inherent in generative planning promotes a context-rich environment which allows more flexibility in adjusting to changing conditions The most important lesson learned as a result of this research is the need to balance the regulati v e and generative aspects of an outcome evaluation. Data from the two high-use sites participating in this study indicate that these sites have established generative processes which contribute significantly to the increased interface between the structure of the outcome evaluation and the organizational culture in which the evaluation is occurring In order to increase the utilization of outcome information throughout the 45 participating local mental health authorities it is necessary to build generative processes throughout the system.

PAGE 196

182 Six strategies for increasing the utility of outcome information at the local level are presented below All are based on building the capacity of the local-level generative processes that promote the interpretation of outcome information in a local context and, thus enhance its utility in decision making In general, increasing the capacity for generative processes can be accomplished by developing the qualitative aspects of the outcome reports and building the narrative capacity of evaluation participants to discuss and interpret the meaning of outcome information in their local context The anthropological literature on knowledge utilization stresses increased collaboration (Schensul 1987 ; Scheinfeld 1987; Whiteford, 1987; van Willigen, Rylko Bauer and McElroy 1989) The second lesson learned from this study is that successful collaboration rests upon a foundation of interpretive processes in organizations through which meaning is constructed. In keeping with the TC:MHP goal of creating a participatory evaluation the recommendations below are all based on building local-level collaboration among the three participants in the TCMHP evaluation that are integral to the local culture at each site : 1) the staff of the local mental health authority ; 2) the members ofthe Community Management Team; and 3) the state TCMHP staff These efforts to build collaboration are fundamentally focused on interpretive processes at the local level and ways to improve communication about the TCMHP evaluation through the development of narrative patterns of communication at both the state and local levels of the TCMHP Because the TCMHP evaluation was conceived as a participatory process, the strategies are not expected to change the direction or purpose of the TCMHP evaluation but rather to serve as a guide to future action.

PAGE 197

183 The recommended strategies for increased util i ty of the TCMHP evaluation at the local level and are presented below These strategies are purposefully general in scope and are intended to frame a debriefing process that the research staff from this study will conduct with TCMHP staff During a facilitated process in which the results of this study will be discussed it is hoped the staff will generate more specific ideas based on the recommended strategies for improving local outcome information utilization. 1. Boost the Relevance of Outcomes Repeatedly articulate how the outcome evaluation and its results relate to local needs and concerns for service planning and delivery 2 Advocate for OutcomesUse training sessions, outcome reports publications, and informal communications to advocate for the benefits of the outcome evaluation by providing specific examples of how outcome information can be used to develop and adjust local service-delivery strategies 3 Provide for Fle x ibilit y Make sure local users have access to more detailed or specialized analyses of the outcome data so that outcome information can be tailored to local needs when necessary 4 Address ConflictOutcome reporting and performance requirements can result in user-level stress Be aware of perceived constraints and weaknesses in the outcome evaluation and be forthright and proactive in addressing these issues at the local level. 5. Bridge ComplexityHelp local mental health administrators use outcome information to bridge the comple x ities of their local environment by designing outcome reports that balance quantitative and qualitative information 6 Build Credibility -Clearly distinguish between the purposes of outcome evaluation for service-related decision making and research for the purpose of scientific generalizability and provide outcome infonnation that is valid reliable and plausible in the context of day-to-day decisions

PAGE 198

184 Conclusions The analysis of the data collected in this study advances the position that patterns of outcome information utilization are deeply rooted in the interpretive processes of the organizational culture at the local mental health authorities under consideration Further the study demonstrates that the acquisition of knowledge regarding TC.l\.1HP outcome information and the continuously developing understanding of how outcome information relates to local-level service planning and delivery decisions are functions of the interface between the TC.l\.1HP evaluation design and the local organizational culture. Each of the local mental health authorities participating in this study took action and made decisions regarding their service-delivery strategies that were based on shared meanings or acquired knowledge of the utility of outcome information For the low-use sites the acquired knowledge resulted in modes of utilization that only minimally related TC.l\.1HP outcome information to local service planning and delivery For the high-use sites the acquired knowledge resulted in modes of utilization that directly related TC.l\.1HP outcome information to their service planning and delivery decisions Despite significant differences between them the high-use sites shared the characteristics ofvaluing the TCMHP multi-agency approach to service delivery, strong partnerships with state and local partners viewing the TCMHP evaluation as an ongoing process viewing outcome information as an enhancement to their jobs, and demonstrating strong patterns of narrative communication which integrated outcome information into all levels of their organizations Conversely sites that showed lower level of use of outcome information were less committed to the TCMHP s multi-agency approach to service

PAGE 199

delivery, established fewer partnersh i ps with other state and local agencies viewed evaluation as an end result did not believe that outcome information contributes significantly to their ability to serve children and families and used linear and unidirectional arguments in discourse about outcomes 185 This study drew upon anthropological understandings of organizations in order to compare and contrast the organizational cultures at four local mental health authorities participating in the TCMHP. For the purpose ofthis research culture was seen as ideational and as constructed through interaction at all levels of organizat ion. In this way organization culture was seen as both process and product of organization This approach to culture directed this research toward the investigation of interactions among individuals at the state and local levels of the TCMHP in order to understand how cultural processes influence the utilization of outcome information at the local level in the TCMHP The analytical advantage to this theoretical orientation in defining the concept of culture i n organizations is that it considers social structure and culture as two separate domains This allowed researchers to consider the cultures of local mental health author i ties separately from the structure ofthe TC:MHP evaluation and the linkage between outcome information and local-level decision making as a function of the interface between the evaluation structure and local culture A number oftheorists (e.g Keesing 1974 ; Smircich 1983 ; Allaire and Firsirotu 1984) argue for the necessity of a conceptual distinction between social structure and culture in organizations rather than assuming that social and structural components of an organization are fully integrated with the organization s ideational components Meek s

PAGE 200

186 (1988:464) arguments in favor of separating the structural and ideational components of organizations were particularly useful in considering issues of outcome information utilization in the TCMHP. Among these arguments, Meek points out that if culture and social structure are integrated, then it is problematic to account for shifts in social structure which occur without corresponding shifts in social norms The data for this study provided numerous examples of how incongruous shifts in structure and local culture contribute to lower incidence of outcome information utilization For example the establishment of the TCMHP with its commitment to developing interagency service delivery systems, belied the strong commitment to a different therapeutic perspective taken by SiteS. This represented a shift in structure that was not followed by a corresponding shift in cultural processes at the local level. Failure to distinguish conceptually between the evaluation structure and local cultural processes would have made it impossible to consider the impact of Site S s therapeutic perspective on its relation s hips with local child-servin g ag encies and the state TCMHP. Meek (1988) also argues that ifthe emphasis in organizational analysis is on the integration of sociocultural elements in organizations, then it is problematic to deal with conflict between individual and group interests and the power authority and control structured into the organization This distinction is useful in understanding the conflict that resulted particularly at the low-use sites between the local mental health authorities and the state-level power and authority that is structured into the TCMHP throu g h the implementation of performance-based contracts Although the uneven power relationship between the state and local levels ofthe T CMHP existed at each of the sites in this study,

PAGE 201

187 at high-use sites it was mitigated by a strong sense of partnership with state-level TC:MHP staff which resulted in modes of increased outcome information utilization At low-use sites however the demands of performance-based contracts represented conflict between local-level organiz a tional members and the organization's structure and contributed to modes of decreased information utilization It is hoped that this study contributes to the understanding of cultural processes in organizations in a number of ways. In the field of management it is hoped that this work advances the understanding of organizat i onal culture as an implicit interactional process occurring at all levels of organization This position stands in opposition to an understanding of culture as a variable that is integrated into organizational structure and therefore the domain of managers who exercise more direct control over structure Viewing organizational culture as interactional allows for the focus on more subjective realities in decision-making processes rather than on the creation of order through managerial uniformity and consistency. Within the field of children's mental health this study is among the first that examines issues of outcome information utilization In an environment in which human services are struggling to integrate outcome accountability into service-delivery systems it is hoped that this study contributes to shifting the emphasis of such efforts from the structural aspects of outcome evaluation to a consideration of the cultural processes that will influence the implementation of outcome information systems This study demonstrates that emphasizing cultural processes when planning outcome evaluation

PAGE 202

188 would improve the potential for the results of the outcome evaluation to provide crucial feedback on the effectiveness of service-delivery strategies Finally from an anthropological perspective, it is hoped that this study will contribute to the growing body of work that draws upon anthropological concepts of organization to build understanding of organizational processes. Both through the operationalization of the culture concept and the analysis of the data, this study offers explanation of how the interaction of structure and culture impact outcome utilization In particular the analysis of the data in this study demonstrate the utility of anthropology in moving from macro to micro levels of organization Morey and Morey (1994 : 21) argue that anthropology is particularly useful in bridging between the macroand micro-level explanations of organizational functioning In fact state-level efforts to improve local-level utilization were generally focused on changes in structural aspects of the evaluation (e g data-reporting requirements and service-delivery guidelines) The only explanation offered by state-level informants for the different utilization patterns observed at the local level was differences in leadership style at each site It is interesting that state-level strategies for improving utilization shifted immediately from the emphasis on macro-level concerns about structure to the micro level or individual concerns about leadership The analysis of cultural processes at the four sites in this study allowed the identification of numerous organizational factors involving the interaction of key stakeholders at the local level. These factors serve to bridge the micro and macro aspects

PAGE 203

189 of organization and suggest intermediate strategies for impacting outcome information utilization The recommended strategies more effectively reflect the layered or embedded quality of organizational processes and focus efforts to improve outcome utilization on the interactional and negotiated processes through which local-level meaning about the relative utility of outcome information is created It is hoped that the results of this study and the recommended strategies for improving outcome information in the TCMHP demonstrate the positive contribution and practical application of anthropology in organizational settings

PAGE 204

REFERENCES Allaire Yvan and Mihaela E Firsirotu 1984 Theories of Organizational Culture Organization Studies 5(3) : 193-226 Aunger Robert 1995 On Ethnography : Storytelling or Science Current Anthropology 36(1) : 97130 Baba, Marietta L. 1986 Business and Industrial Anthropology : An Overview NAP A Bulletin 2 American Anthropological Association 1995 Letters : Using vs Applying Anthropology, reply to a letter to the editor. Practicing Anthropology 17(1-2) : 58-59 Barley, Stephen Gordon Meyer and Debra Gash 190 1988 Cultures of Culture : Academics Practitioners and the Pragmatics of Normative Control. Adm i nistrative Science Quarterly 33:24 -60 Boutte Marie 1992 Discredited Inheritance : Machado-Joseph Disease and Family Dynamics in the Azores Human Organization 51 (2) : 144-150 Briody Elizabeth 1989 Organizational Culture: From Concept to Applications Anthropology of Work Review 1 0( 4): 4-10 Burchard J.D and M Schaefer 1992 Improving Accountability in a Service Delivery System in Children's Mental Health Clinical Psychology Review 12: 867-882 Cap lan Nathan 1 9 77 A Minimal Set of Conditions Necessary for the Utilization of Social Science Knowledge in Policy Formulation at the National Level. In Using Social Research in Public Policy Making Carol Weiss ed Pp. 183-197 Lexington, MA : Lexington Books D C Health and Company

PAGE 205

Chambers Erve 1985 Applied Anthropology : A Practical Guide Prospect Heights, IL: Waveland Press Ciarlo James 1982 Accountability Revisited : The Arrival of Client Outcome Evaluation Evaluation and Program Planning 5 : 31-36. Cohen, E. and T. Ooms 191 1993 Data Integration and Evaluation : Essential Components ofFamily-Centered Systems Reform Washington D C.:The American Association for Marriage and Family Therapy Research and Education Foundation Daft Richard and Karl Weick 1984 Toward a Model of Organizations as Interpretation Systems Academy of Management Review 9(2) : 284-295. Dauber Kenneth 1995 Bureaucratizing the Ethnographer s Magic Current Anthropology 36(1):7595. Deal Terrence and Allen Kennedy 1982 Corporate Cultures : The Rites and Rituals of Corporate Life Reading MA: AddisonWessley Publishing Company, Inc DeCarolis Gary 1994 Foreword In A System of Care for Children & Youth with Severe Emotional Disturbances 1994 revised edition Beth Stroul and Robert Friedman Washington DC: Georgetown University Child Development Center Ellmer Rosem a ry Laura Lein and Pamela Hormuth 1995 coordinated Services for children's Mental Health : A Process Evaluation Journal ofMental Health Administration 22(4):346-357 Elmore R. 1978 Organizational Model of Social Program Implementation. Public Policy 26 : 185-228. Fabrega Horacio Jr. 1989 On the Significance of an Anthropological Approach to Schizophrenia Psychiatry 52(1): 45-65

PAGE 206

192 Finan Timothy and John van Willigen 1991 The Pursuit of Social Knowledge : Methodology and the Practice of Anthropology In Soundings : Rapid and Reliable Research Methods for Practicing Anthropologists John van Willigen and Timot h y Finan, eds Pp. 1-10 NAPA Bulletin 10, American Anthropological Association. Flaherty, E.W. and C. Windle 1981 Mandated Evaluation in Community Mental Health Centers : Framework for a New Policy Evaluation Review 5 : 620-638 Fosu Gabriel B 1992 Perceptions of Mental Disorders in the Context of Social Change : Correlates and Implications for Socio-Medical Behavior. Central Issues in Anthropology, 10 : 103-116 1995 Women s Orientation toward Help-Seeking for Mental Disorders Social Science and Medicine 40(8) : 1029 1040 Gaines Atwood 1992 From DSMI to III-R; Voices of Self Mastery and the Other : A Cultural Constructivist Reading of U S Psychiatric Classification. Social Science and Medicine 35(1) : 3-24 Gamst Frederick C. 1977 An Integrating View of the Underlying Premises of an Industrial Ethnology in the United States and Canada Anthropologica l Quarterly 50 : 1-8 Garnder Burleigh B. 1978 Doing Business with Management. In Applied Anthropology in America E l izabeth M. Eddy and William L. Partridge eds Pp. 245-260 New York: Col umbia University Press Giovannini Maureen and Lynne Rosansky 1990 Anthropology and Management Consulting: Forging a New Alliance NAP A Bulletin 9 Washington D C. : American Anthropological Association Glazer Edward Harold Abelson and Kathalee Garrison 1983 Putting Knowledge to Use : Facilitating the Diffusion ofKnowledge and the Implementation of Planned Change San Francisco : Jossey-Bass Publishers. Geertz Clifford 1973 The Interpretation of Cultures Basic Books A Division ofHarperCollins

PAGE 207

193 Gomez Angela 1997 Using a Qualitative Measure to Assess a Child Protection Reform Paper presented at the 1Oth Annual Research Conference for Children's Mental Health, Louis de Ia Parte Florida Mental Health Institute University of South Florida Greenbaum, Susan 1997 Cultural competency in the Casey Foundation Project. Paper presented at the lOth Annual Research Conference for Children's Mental Health, Louis de Ia Parte Florida Mental Health Institute University of South Florida Gore A. And R Brown 1993 The National Information Infrastructure : Agenda for Action. Washington DC: The White House Grasso Anthony and Irwin Epstein 1993 Theoretical Requirements for Successful Integration oflnformation Technology in Human Service Agencies In Information Systems in Child Youth and Family Agencies Anthony Grasso and Irwin Epstein eds Pp. 1732 New York : Haworth Press Inc Gregory Kathleen L. 1983 NativeView Paradigms : Multiple Cultures and Culture Conflicts in Organizations Administrative Science Quarterly 28 : 359-376 Gutierre zMayka Marcela 1997 Theoretical and Methodological Foundations ofthe Family Experience Study Paper presented at the lOth Annual Res earch Conference for Children's Mental Health, Louis de Ia Parte Florida Mental Health Institute University of South Florida Hamada Tomoko and Ann Jordan eds. 1 990 Cross Cultural Management and Organizational Culture Studies in Third World Societies number forty-two : Williamsburg VA: Department of Anthropology College ofWilliam and Mary. H a r g reaves, W A. and M Shumway 1989 Effectiveness ofMental Health Services for the Severely Mentally Ill. In the Future of Mental Health Services Research C.A. Taube D Mechanic and A. Hohmann, eds DHHS Publication No (ADM)89-1600. pp 253-284 Washington DC: U.S Government Printing Office

PAGE 208

194 Hernandez, Mario and Sharon Hodges 1996 The Ecology of Outcomes Tampa FL: University of South Florida, Louis de Ia Parte Florida Mental Health Institute Department of Child and Family Studies The System Accountability Project for Children s Mental Health 1997 Building Systems of Outcome Accountability : Experiences from the Field Workshop presented at the lOth Annual Research Conference for Children's Mental Health Louis de Ia Parte Florida Mental Health Institute University of South Florida Holtzberg CarolS. and Maureen Giovannini 1981 Anthropology and Industry : Reappraisal and New Directions Annual Review of Anthropology 10 : 317-360 Jacobson David E 1987 The Cultural Context of Social Support and Support Networks Medical Anthropology Quarterly (new series) 1(1) : 42-67 Jenkins Richard 1993 Incompetence and Learning Difficulties : Anthropological Perspectives. Anthropology Today 9(3) : 1 6 -20 Joint Commi s sion on the Mental Health of Children 1969 Crisis in Child Mental Health : Challenge for the Seventies. New York : Harper and Row. Jordan, Ann T. 1994a Practicing Anthropology in Corporate America : Consulting on Organizational Culture Ann T Jordan ed NAPA Bulletin #14 American Anthropological Association 19 9 4b Organizational Culture : The Anthropolo g ical Approach In Practicing Anthropology in Corporate America : Consulting on Organizational Culture Pp. 3-16 Ann T. Jordan ed NAPA Bulletin #14 American Anthropological Asso c iation Keesing Roger 1974 Theories of Culture Annual Review of Anthropology 3 : 73-97 Kilmann Ralph 1984 Beyond the Quick Fix : Managing Five Tracks to Organizational Success San Francisco : Jossey-Bass Publishers Kilmann Ralph Mary Saxton and Roy Serpa 1985 Gaining Control of the Corporate Culture San Francisco : Jossey-Bass Inc

PAGE 209

195 Kirkma yer, Laurence, Allan Young and James Robbins 19 9 4 Symptom Attribution in Cultural Perspective. Canadian Journal ofPsychiatry 39(10 ): 584-5 9 5 Knitzer Jan e 1982 Unclaimed Children : The Failure ofPublic Responsibility to Children and Adolescents in Need of Mental Health Services Washington D C. : The Children's Defense Fund Knorr, Karen D 1977 Policymakers Use of Social Science Knowledge : Symbolic or Instrumental ? In Using Social Research in Public Poli c y Making Carol Weis s ed. Pp. 165-182 Lexington MA: Lexington Books, D C Health and Company Koegel Paul 1 9 92 Through a D i fferent Lens : An Anthropological Perspective on the Homeless Mentally Ill. Culture Medicine and Psychiatry 16(1) : 1-2 2. Leviton, Laura C. and Edward Hughes 1981 Research on th e Utili z ation ofEvaluations : A Review and Synthesis Evaluation Review 5 ( 4):525-548. Liberton Cindy Krista Kutash and Robert Friedman 1995 A System of Care for Children's Mental Health : Expanding the Research Base. Proceedings from the 7th Annual Research Conference (February 28 to March 2 1 99 4) .Tampa, FL: University of South Florida Florida Mental Health Institute Research and Trai ning Center for Children s Mental Health Limerick David and Bert Cunnington 19 9 3 Managing th e New Organization : A Blueprint for Networks and Strategic Alliances San Francisco : Jossey-Bass Inc Lorsch, Jay ed 1986 Managin g Culture: the Invisible Barrier to Strategic Change California Management Review 28(2) :9 5-109 Maccoby, Michael 1994 The Corporation as P a rt-Culture In Anthropological Perspecti v es in Organi z ational Culture Tomoko Hamada and Willis Sibley eds Pp 2 67-2 78 Lanham N e w York, London: University Press of America

PAGE 210

Meek Lynn 1988 Organizational Culture : Origins and Weaknesses. Organization Studies 9(4):453 473 Mental Health Association ofTexas 1989 Do Kids Count? How Texas Serves Children and Adolescents with Severe Emotional Disturbance Second ed Miles, Matthew 1979 Qualitative Data as an Attractive Nuisance : The Problem of Analysis Administrative Science Quarterly 24 : 590-601 Miles, Matthew and A. Michael Huberman 1994 Qualitative Data Analysis second edition Thousand Oaks CA: Sage Publications Mintzberg Henry 1979 An Emerging Strategy of"Direct" Research. Administrative Science Quarterly. 28 :582 -589 Morey Nancy and Robert Morey 1994 Organizational Culture : The Management Approach. In Practicing Anthropology in Corporate America : Consulting on Organizational Culture Pp. 17-36 Ann Jordan, ed. NAPA Bulletin #14 American Anthropological Association. Morgan Gareth 1 986 Images of Organization. Beverly Hills : Sage. Nelson Douglas 196 1 993 Introducti o n In Kids Count Data Book. Annie E Casey Foundation and the Center for the Study of Social Policy Washington DC: Center for the Study of Social Policy Ouchi William 1981 Theory Z : How American Business Can Meet the Japanese Challenge Reading MA : Addison-Wessley Publishing Company Patti R 1983 Social Welfare Administration Englewood Cliffs N .J.: Prentice Hall Inc

PAGE 211

Patton Michael Q Patricia Smith Grimes and Kathryn Guthrie et al. 1977 In Search oflmpact: An Analysis ofthe Utilization ofFederal Health Evaluation Research In Using Social Research in Public Policy Making Carol Weiss, ed Pp. 141-163 Lexington, MA: Lexington Books D C. Health and Company Pelz Donald C. 197 1978 Some Expanded Perspectives on Use of Social Science in Public Policy. In Major Social Issues : A Multidisciplinary View J. Milton Yinger and Stephen J. Cutler eds Pp. 346-357. New York: The Free Press A Division of Macmillan Publishing Co., Inc Peters Thomas and Robert Waterman Jr 1982 In Search ofExcellence: Lessons from America's Best Run Companies New York : Harper and Row Publishers Reichers, Arnon and Benjamin Schneider 1990 Climate and Culture : An Evolution of Constructs In Organizational Climate and Culture, Pp. S-49 Benjamin Schneider, ed. San Francisco : Jessey Bass Publisher Rich Robert F 1977 Uses of Social Science Information by Federal Bureaucrats : Knowledge for Action Versus Knowledge for Understanding In Using Social Research in Public Policy Making Carol Weiss, ed Pp. 199-211 Lexington, MA: Lexington Books D C Health and Company Richeport Madeline 1984 Strategies and Outcomes oflntroducing a Mental Health Plan in Brazil. Social Science and Medicine 19(3): 261-271. Rosenblatt Abram and C. C Attkisson 1993 Assessing Outcomes for Sufferers of Severe Mental Disorder : A Conceptual Framework and Review Evaluation and Program Planning 16: 347-363 Rouse Lawrence W 1996 The Development of a Continuous Evaluation system for the Texas Children's Mental Health Plan: Building an Evaluation-Stakeholder Feedback Loop Presentation at the 9th Annual Research Conference for Children's Mental Health, Louis de La Parte Florida Mental Health Institute University of South Florida

PAGE 212

198 Rouse Lawrence, Nancy MacCabe, and Marcia Toprac 1995 The Development of a StateWide Continuous Evaluation system for the Texas Children's Mental Health Plan : A Total Quality Management Approach Presentation at the 8th Annual Research Conference for Children's Mental Health Louis de la Parte Florida Mental Health Institute, University of South Florida. Rouse Lawrence and Marcia Toprac 1997 The Development of a State-wide Continuous Evaluation System for the Texas Children's Mental Health Plan Journal of Mental Health Administration in press Ruano Carlos 1995 Letters : Using vs Applying Anthropology Practicing Anthropology 17(12) : 57-58 RylkoBauer Barbara John van Willigen, and Ann McElroy 1989 Strategies for Increasing the Use of Anthropological Research in the Policy Process : A Cross-Disciplinary Analysis In Making Our Research Useful : Case Studies in the Utilization of Anthropological Knowledge John van Willigen Barbara Rylko-Bauer and Ann McElroy, eds. Pp 1-26 Boulder : Westview Press Sachs Patricia ed. 1989 Anthropological Approaches to Organizational Culture Anthropology of Work Review special edition 10(3) Saffold III GuyS. 1988 Culture Traits Strength and Organizational Performance : Moving Beyond Strong Culture Academy ofManagement Review 13(4) : 546-558 Sanday Peggy Reeve s 1 979 The Ethnographic Paradigm(s) Administrative Science Quarterly 24: 527-538 SavilleTroike, Muriel 1989 The Ethnography of Communication an Introduction 2nd ed. Oxford UK : Blackwell Schein Edgar 1985 Organizational Culture and Leadership San Francisco : Jossey-Bass Publishers Scheinfeld Daniel R. 1987 A Collaborative Approach to Research Utilization Practicing Anthropolo gy 9(1) : 2-3

PAGE 213

Schensul, Jean J. 1987 Knowledge Utilization : An Anthropological Perspective Practicing Anthropology 9(1) : 4-5 Scheper Hughes Nancy and Anne Lovell 199 1986 Breaking the Circuit of Social Control : Lessons in Public Psychiatry from Italy and Granco Basagla Social Science and Medicine 23(2) : 159-178 Schorr Lisbeth, F Farrow and D Hornbeck et al. 1994 The Case for Shifting to Results-Based Accountability In Making a Difference : Moving to Outcome-Based Accountability for Comprehensive Service Reforms. N Young S Gardner S Coley L. Schorr, and C Bruner, eds Pp 1328 National Center for Service Integration Resource Brief?. Falls Church, VA: National Center for Service Integration Schwartzman Helen B 1993 Ethnography in Organizations Qualitati ve Research Methods Series volume 27 Newbury Park : Sage Publications. Silverman Myrna Edmund Ricci and Margaret Gunter 1 990 Strategies for Increasing the Rigor of Qualitative Methods in Evaluation of Health Care Pro g rams Evaluation Review 14(1) : 57-74 Srnircich Linda 1983 Concepts of Culture and Organizational Analysis Administrative Science Quarterly 28 : 339-358 Spradley James 1 979 The Ethnographic Interview Fort Worth : Harcourt Brace Jovanovich College Publishers Stroul Beth and Robert Friedman 1986 A System of Care for Severely Emotionally Disturbed Children and Youth Washington DC: Georgetown University Child Development Center. 1994 A System of Care for Children & Youth for Children with Severe Emotional Disturbance revised edition Washin g ton DC : Georgetown University Child Development Center. Texas Children's Mental Health Plan 1995 The TXMHMR. Children's Mental Health Service s Evaluation System Research and Special Projects Texas Department ofMental Health and Mental R eta rdation

PAGE 214

200 TexasMHMR 1995 Description of Contract Outcome Variables under Consideration Evaluation Report No. 23. Research and Evaluation July 5 1995 199 6 Children s Mental Health Services Evaluation Handbook. Research and Evaluation May 1996 U S General Accounting Office 1976 Evaluation and Analysis to Support Decision-Making PAD-76-9 Washington D .C. September Usher Lynn 1993a Balancing Stakeholder Interests in Evaluations oflnnovative Programs to Serve Families and Children Paper presented at the annual meeting of the Association for Policy Analysis and Management. 1993b Building Capacity for Self Evaluation in Family and Children s Services Reform Efforts Paper presented at the annual meeting of the American Evaluation Association Uzzell Douglas 1 99 0 Dissonance ofFormal and Informal Planning Styles or Can Planners Do Brocolage ? City and Society 4(2) : 114-130 U z zell Douglas and Linda Whiteford 1 9 97 Deconstructing cultural competency : Constructivist Views of the Data. Paper presented at the 1Oth Annual Research Conference for Children s Mental Health Louis de la Parte Florida Mental Health Institute University of South Florida van Willi g en John Barbara Rylko-Bauer, and Ann McElroy eds 1 9 89 Making Our Research Useful : Case Studies in the Util i zation of Anthropological Knowledge Boulder : Westview Press Walck Christa 1 9 8 9 Why Organizational Culture Cannot Survive the Rational Paradigm Anthropology of Work Review 10(4) : 1-4 Ware J. E 1986 The Assessment of Health Status In Applications of Social Science to Clinical Medi c ine and Health Policy L H Aiken and D Mechanic eds Pp. 204-228 New Brunswick NJ : Rutgers University Press 1 9 8 9 Measuring Health and Functional Status in Mental Health Services Research In The Future of Mental Health Services Research. C.A. Taub e D Mechanic and A. Hohmann eds Pp. 289 302 DlffiS Publication No. (ADM)89-1 6 00 Washington DC : U.S Government Printing Office

PAGE 215

201 Weick Karl and Larry Browning 1986 Argument and Narration in Organizational Communication In Yearly Review of Management ofthe Journal ofManagement 12(2) : 243 259 Weiss, Carol H ed 1977 Using Social Research in Public Policy Making Weiss Heather and Francine Jacobs 1988 Evaluation Family Programs New York : Aldine de Gruyter Whiteford Linda M 1987 Staying Out ofthe Bottom Drawer Practicing Anthropology 9(1) : 9-11. Wilkins, Alan and William Ouchi 1983 Efficient Culture: Exploring the Relationship between Culture and Organizational Performance Administrative Science Quarterly 28:468-481 Wortman P 1975 Evaluation Research : A Psychological Perspective American Psychologist 30 : 562-575 Wright Susan 1994 Culture in Anthropology and Organizational Studies In Anthropology of Organizations. Susan Wright, ed Pp 1-31. Routledge : London Young Nancy, et.al. 1994 Making a Difference : Moving to Outcome-Based Accountability for Comprehensive Service Reforms Falls Church VA : National Center for Service Integrat i on.

PAGE 216

202 APPENDICES

PAGE 217

203 APPENDIX 1 GLOSSARY OF TERMS IN CIDLDREN S :MENTAL HEALTH Glossary ofTerms in Children's Mental Health CASSPthe Child and Adolescent Service System Program, a federal program supporting the development of interagency efforts to improve the systems under which the most troubled children and youth receive services Day treatment the most intensive of non-residential services which can occur for longer periods of time, providing an integrated set of educational, counseling and family interventions Interagency collaboration the combined coordinated and cooperative efforts of multiple child-serving agencies to provide services that meet the specific needs of children and their families. Normalized Care providing services in the least restrictive most clinically appropriate setting possible This would include providing services within the child s community and using acute residential treatments sparingly and only when clinically appropriate. NAS.MHPD the National Association of State Mental Health Program Directors Out-of-home placement services that require the child to live away from home and possibly outside of the community in order to receive services This would include residential placements in juvenile justice, child welfare, special education and mental health Outcome the impact or results of services provided for the children and their families Outcome accountability responsibility on the part of systems of care for accomplishing publicly articulated and accurately tracked goals regarding the services they provide children and their families. Severely emotionally disturbed children young people whose problems are so severe that they require the long-term intervention of mental health and other agencies S:MHRCY State Mental Health Representatives for Children and Youth a division of the National Association of State Mental Health Program Directors (Continued on next page)

PAGE 218

204 APPENDIX 1 (Continued) System of Care -a philosophy about the way services should be provided for children and families which is founded on the principle that care should be provided child-centered (driven by the needs ofthe child and family) and community-based (provided in less restrictive settings within or near a child's home community) Wraparound the delivery of coordinated interdisciplinary services provided with the input of the child and family and tailored to the strengths and needs of the individual child and family

PAGE 219

APPENDIX 2 CORE VALUES AND GUIDING PRINCIPLES FOR THE SYSTEM OF CARE Core Values and Guiding Principles for the System of Care Core Values for the System of Care 1. The system of care should be child centered, with the needs of the child and family dictating the types and mix of seiVices provided 205 2 The system of care should be community-based with the locus of seiVices as well as management and decision-making responsibility resting at the community level. Guiding Principles for the System of Care 1 Emotionally disturbed children should have access to a comprehensive array of seiVices that address the child s physical emotional social and educational needs. 2 Emotionally disturbed children should receive individualized seiVices in accordance with the unique needs and potentials of each child and guided by an individualized seiVice plan 3 Emotionally disturbed children should receive seiVices within the least restrict i ve most normative environment that is clinically appropriate. 4 The families and surrogate families of emotionally disturbed children should be full participants in all aspects of the planning and delivery of seiVices 5 Emotionally disturbed children should receive seiVices that are integrated with linkages between child-caring agencies and programs and mechanisms for planning developing and coordinating seiVices 6. Emotionally disturbed children should be provided with case management or similar mechanisms to ensure that multiple seiVices are delivered in a coordinated and therapeutic manner, and that they can move through the system of seiVices in accordance with their changing needs. 7 Early identification and inteiVention for children with emotional problems should be promoted by the system of care in order to enhance the likelihood of positive outcomes (Continued on next page)

PAGE 220

APPENDIX 2 (Continued) 8 Emotionally disturb e d children should be ensu r ed smooth transitions to the adult system as they reach maturity 9. The rights of emotionally disturbed children should be protected and effective advocacy efforts for emotionally disturbed children and youth should be promoted I 0 Emotionally disturbed children should receive services without regard to race religion national origin sex physical disability or other characteristics, and services should be sensitive and responsive to cultural differences and special needs 206

PAGE 221

APPENDIX 3. EXECUTIVE SUMMARY OF THE SYSTEM ACCOUNT ABILITY STUDY SYSTEM ACCOUNTABILITY STUDY: IMPROVING MULTI-AGENCY SERVICE DELIVERY SYSTEMS USING FUNCTIONAL OUTCOME MEASURES PROJECT OVERVIEW The System Acco untabilit y Study is a five-year study being conducted through the Research and Training Center for Children's Mental Health under the direction of Dr. Robert Friedman at the Florida Mental H ealt h Institute. During the first two years of this project, case studies of various sites with exemplary outcome information systems are being conducted. In the remaining years of the project, techn i cal assist ance will be lo wanting to improve their use oi information to enhan ce service delivery. The purpose of the System Accountability Stud y is to investigate the impact that utilizing measur able outcomes has o n service delivery and planning Specifically, the study will de scr ibe the utilization of outcome accountability on service delivery, on agencies associated with the systems under study, and on the level of involvement of families and other key stakeholders in service system p l anning. The System Accountability Study will identify, evaluate and describe service systems that use exemplary methods of measuring functional outcome s and will provide technical assistance to agencies wishing to establish or develop outcome driven information systems. The study will describe the exem plary sit e s across six areas: Procedures for selecting outcomes and the out comes themselves; Procedure s for knowing who is being served relative to who the system intend s to serv e ; Procedures for knowing the form, integrity and cost of services being p r ovi ded relative to what the sys tem intends to provide; P roc edures that allow results of service delivery to be appropriately int erpreted; Pr ocedures that p r ovide feedback to all key stakeholders; Pr oce dures that can affect service delivery in response to results and/or procedures to provide positive feedback when indicated. '. SYSTEM ACCOUNTABILITY STUDY GOALS The res ults o f the study will be presented in a monograph titled B u i l ding Out come Accountability in Children s Mental Health : Experiences from the Field Lessons learned from the work with e stab l ished systems will be used in subseq uent years of the System Accountability Study to provide co nsult ation to sites interested in developing outcome inf ormatio n systems. : SYSTEM ACCOUNTABILITY STUDY STAFF The Sy st
PAGE 222

APPENDIX 4 LETTER INVITING PARTICIPATION IN Thi FOR1\.1ATION PATHWAYS November 7, 1996 Dear : The Texas Department of Mental Health and Retardation bas t o partic i pate in an investigation of the usc.'UineS3 of outcome information in decisions affecting service planning and delivery of children's mental health services in Texas This study, called Information Pathways, is being conducted through the System Accountability Project for Children's Mental Health at Florida Mental Health Instirutc (FMHI) The purpose of Information Pathways is to identify the factors that both support aod impede the usc of outcome infonnation in decisions made at the local level. It is hoped that the lessons learned about why outcome information is both used and not used will inform the development ofbetter outcome information processes in TcXllS as well as other child-serving systems aaoss the country. Infonnation Pathways is funded by the Annie E. Casey Foundation and will be l;d by Sharon Hodges with the System Accountability Project for Children's Mental Health in the Department of Child and Family Studies at FMHJ. We were asJced to identify four local mental health authorities to participato in Information Pathways Each oi the four sites selected was identified as a local authority that would be willing and able to forthrightly share ideas and opinions about the relativ e use:fulneS3 of outcome infammtion as it relates to the local planning and service provision We hope the four sites., wben taken together, will repr=t a variety of opinions and C;(pc.ricnco .,.,;th tho evaluation of children's mental health services in T exu The data rollcction fOl' Infoonation Pathways will involve an initial telephone interview as well as a 2-3 day site visit by FMHI researchers during which they will conduct semi structured interviews about information utilization A more detailed summary of this proceduro is attached. Site visits would begin in Nov=ber and would be roncluded by January. We have recommended that your center be one sites to participate in lnfol'IIlZlion Pathways. Participation of individual sites in Information Pathways is voluntary Site as well as individuals participating in Information P3thways will be held in strict confidence by the researchers from FMHI. We bope you will strongly consider taking part in this study. Sincerely, Andres Guariguata Marcia Toprac, Ph D. Director, Childre.n's Sen-ices Research & Evaluation CC : Sharon Hodges, Florida Mental Health Institute 208

PAGE 223

APPENDIX 5 INFORMATION PATHWAYS SEMI-STRUCTURED INTERVIEW GUIDE FOR LOCAL-LEVEL INTERVIEWS 209 Information Pathways Semi-Structured Interview Guide for Local-Level Interviews 1 Preface the interview with our interest in learning about how information about outcomes is or is not used in making decisions about program planning and delivery. What kind of outcome information does the informant need in order to do his/her job as it relates to children's mental health services and the TCMHP? Where does that information come from? Regularity? Timeliness? 2. General discussion of what informant knows about the TCMHP outcome information system with the goal of learning how much the informant knows about the outcome information system and informant perceptions on what the information system is and whether it is useful. 3 What kind of outcome information/reports does the informant receive on a regular basis? Clarify if information described is part of TCMHP or other reports generated locally or at the state level. Also clarify form and format of information receivedprinted, verbal on-line. 4. How does informant get outcome information (from whom) and with whom does informant discuss outcome results? 5 How does informant define information use? What does it mean to be an information user? Does informant think of him/herself as an information user? 6 Who gets outcome information at the local level? In what format : written reports, meetings, other ways to learn about outcome information? 7 How does the informant describe the outcome information generated by the state office of research and evaluation with respect to its usefulness in decisions relating to program planning and service delivery? 7a. In what ways does informant find outcome information useful in decision making affecting service planning and delivery? What types of information? What factors make this information useful? (Look for concrete examples ) 7b In what ways does informant find outcome information does not support decision making affecting service planning and delivery? What types of information? What keeps information from being useful? (Look for concrete examples.) (Continued on next page)

PAGE 224

210 APPENDIX 5. (Continued) 8 Strengths of outcome information system : examples of how information is useful or makes a difference in service planning and delivery. 9 Shortcomings of outcome information system : examples of how information is not useful or does not support good decisions in service planning and delivery 10 What does informant think would make outcome information more useful ? 1 1 Does informant think the outcome information system has shifted the mind set or way people within the local mental health authority view community mental health ? 12. Does informant think the outcome information system has shifted the mind set or way people collateral agencies view community mental health ? 13. Does informant think the outcome information system has shifted the mind set or way people in the community view community mental health ? 14 Does informant think outcome information has changed the way he / she does their job? Does informant think it has changed the way they think of their job? 15. What would the informant like to add to this conversation that we haven t touched on in the interview?

PAGE 225

APPENDIX 6 CODE SHEET FOR INFORMATION PATHWAYS TRANSCRIPTS Code Sheet for Info Pathways Transcripts Cross-match of interview questio n s with guiding research questions : (lower case letter indicates indirect relationship with designated guiding research question) Table 13: Interview Questions Rel ating to Guiding Research Questions Rel ating Interview Guiding Research Questions Questions 211 1, 4, 5, 6 7, 8 12, A. What aspects of organizational culture influence the utilization of 13, 14 outcome information at local mental health authorities in the TxCMHP? 6, 7, 8 9, 10 B How does information impact decision making at the local level in the TxCMHP? 2 3, 9 IO, II C What conditions support or impede the use of outcome information in service planning and service delivery at these sites ? Code Interview Q u estion 1 A General discussion of what informant knows about the outcome information system with goal of l earning about informant perceptions on what the info system is and whether it is useful. 2 C What kind of outcome information does informant receive on a regular bas i s? Is this info part of TxCMHP? 3. C How does informant get outcome information and with whom does informant discuss outcome results ? 4 A Does informant think ofhim!herselfas information user ? How does informant define information use ? 5 A Who gets outcome information at the local l eve l ? In what format : written reports, meetings othe r ways to learn about outcome information? (Continued on next page)

PAGE 226

212 APPENDIX 6 (Continued) 6 A.b Would informant describe outcome information usefuVnot useful (need to consider scale for this) in decisions regarding program planning and service delivery? 7 .B.a In what ways does informant find outcome information useful in decision making affecting service planning and delivery? What types of information? What factors make this information useful ? 8 .B.a In what ways does informant find outcome information does not support decision making affecting service planning and delivery? What types of information? What keeps outcome information from being useful ? 9.B.c Strengths of outcome information system : examples ofhow information is useful or makes difference in service planning and delivery 10 B c Shortcomings of outcome information system : examples ofhow information is not useful or does not support good decisions in service planning and delivery 11. C What would make outcome information more useful? 12 A Has outcome information system shifted mind set or way people within the local mental health authority view community mental health ? 13. A Has outcome information system shifted mind set or way people in collateral agencies view community mental health? 14 A Has outcome information system shifted mind set or way people in community view community mental health?

PAGE 227

VITA Sharon Hodges received a Bachelor's Degree in Economics from the University of Florida in 1975 and an M.B .A. with a concentration in Management from the University of South Florida in 1981. She began teaching management in the College of Business at the University of South Florida in 1982 while she was a consultant at the Small Business Development Center at the University of South Florida and continued teaching until she entered the Ph D program in Applied Anthropology at the University of South Florida in 1993 While in the Ph D program, Ms. Hodges worked in research and evaluation in the Department of Child and Family Studies at the Louis de la Parte Florida Mental Health Institute at the University of South Florida She has made numerous presentations to national state and local audiences on outcome accountability in human service systems and has co-authored a monograph titled The Ecology of Outcomes as well as other articles and reports relating to issues of system accountability. Ms Hodges will complete her Ph D in December 1997


printinsert_linkshareget_appmore_horiz

Download Options

close
No images are available for this item.
Cite this item close

APA

Cras ut cursus ante, a fringilla nunc. Mauris lorem nunc, cursus sit amet enim ac, vehicula vestibulum mi. Mauris viverra nisl vel enim faucibus porta. Praesent sit amet ornare diam, non finibus nulla.

MLA

Cras efficitur magna et sapien varius, luctus ullamcorper dolor convallis. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Fusce sit amet justo ut erat laoreet congue sed a ante.

CHICAGO

Phasellus ornare in augue eu imperdiet. Donec malesuada sapien ante, at vehicula orci tempor molestie. Proin vitae urna elit. Pellentesque vitae nisi et diam euismod malesuada aliquet non erat.

WIKIPEDIA

Nunc fringilla dolor ut dictum placerat. Proin ac neque rutrum, consectetur ligula id, laoreet ligula. Nulla lorem massa, consectetur vitae consequat in, lobortis at dolor. Nunc sed leo odio.