xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001413315
007 cr mnu|||uuuuu
008 030703s2000 flu sbm s000|0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0000005
Zelski, Robert F.
Use of the Client Oriented Scale of Improvement as a clinical outcome measure in the Veterans Affairs National Hearing Aid Program
h [electronic resource] /
by Robert F. Zelski.
University of South Florida,
Professional research project (Au.D.)--University of South Florida, 2000.
System requirements: World Wide Web browser and PDF reader.
Includes bibliographical references.
Text (Electronic thesis) in PDF format.
Mode of access: World Wide Web.
Title from PDF of title page.
Document formatted into pages; contains 46 pages.
ABSTRACT: In the present health care environment, there is an increased demand for audiologists to measure the outcomes of hearing aid intervention. In addition to the more traditional objective outcome measures, many subjective outcome measures have been developed in the last 20 years. Two such subjective outcome measures are the Abbreviated Profile of Hearing Aid Benefit (APHAB) and the Hearing Handicap Inventory for the Elderly (HHIE). These instruments consist of a series of pre-selected questions that may or may not be applicable to an individual. An alternative to the pre-selected question format is an open format design that allows the person with a hearing loss to designate areas of concern to them. One subjective outcome measure that uses this format is the Client Oriented Scale of Improvement (COSI) developed by Dillon and his colleagues in Australia. The COSI has been validated and may be useful for oversight with multi-clinician or for multi-clinic systems. The purpose of this study was to address tthe potential of the COSI for such oversight. Specifically, the study examined the inter-observer agreement of the classification of individually identified situations into general categories. The study also re-examined the clinical utility of the COSI as an outcome measure in individual hearing aid fittings. The results demonstrated very good inter-observer agreement for the classification of individually identified situations. In addition, the study supported the usefulness as a clinical outcome measure that had been found by Dillon and his colleagues in Australia. These results indicate that the COSI has potential for oversight of the outcomes of hearing aid intervention in hearing aid delivery organizations.
Adviser: Abrams, Harvey B.
t USF Electronic Theses and Dissertations.
Use of the Client Oriented Scale of Improvement as a Clinical Outcome Measure in the Veterans Affairs National Hearing Aid Program Robert F. Zelski Professional Research Project submitted to the Faculty of the University of South Florida in partial fulfillment of the requirements for the degree of Doctor of Audiology Harvey B. Abrams, Chair Theresa Hnath-Chisolm Jennifer J. Lister December 6, 2000 Tampa, Florida Keywords: COSI, VANHAP, subjective outcome measures, hearing aids Copyright 2000, Robert F. Zelski
Robert F. Zelski 2 Use of the Client Oriented Scale of Improvement as a Clinical Outcome Measure in the Veterans Affairs National Hearing Aid Program Robert F. Zelski (ABSTRACT) In the present health care environment, there is an increased demand for audiologists to measure the outcomes of hearing aid intervention. In addition to the more traditional objective outcome measures, many subjective outcome measures have been developed in the last 20 years. Two such subjective outcome measures are the Abbreviated Profile of Hearing Aid Benefit (APHAB) and the Hearing Handicap Inventory for the Elderly (HHIE). These instruments consist of a series of pre-selected questions that may or may not be applicable to an individual. An alternative to the preselected question format is an open format design that allows the person with a hearing loss to designate areas of concern to them. One subjective outcome measure that uses this format is the Client Oriented Scale of Improvement (COSI) developed by Dillon and his colleagues in Australia. The COSI has been validated and may be useful for oversight with multi-clinician facilities or for multi-clinic systems. The purpose of this study was to address the potential of the COSI for such oversight. Specifically, the study examined the inter-observer agreement of the classification of individually identified situations into general categories. The study also re-examined the clinical utility of the COSI as an outcome measure in individual hearing aid fittings. The results demonstrate very good inter-observer agreement for the classification of individually identified situations. In addition, the study supported the usefulness as a clinical outcome measure that had been found by Dillon and his colleagues in Australia. These results indicate that the COSI has potential for oversight of the outcomes of hearing aid intervention in hearing aid delivery organizations.
Robert F. Zelski 3 ACKNOWLEDGEMENTS I wish to acknowledge and, more importantly, thank Harvey B. Abrams, Ph.D., Theresa Hnath-Chisolm, Ph.D., and Jennifer J. Lister, Ph.D., the members of my committee, for their help in the conduct of this study and in the preparation of this report. From the original concept for the project in May of 1999 through the development of the plan of the study and the collection of the data to the many rewrites of this document, Drs. Abrams and Chisolm have devoted many hours of work and thought to the success of the study. Dr. Lister graciously agreed to serve on this committee even though this study was in an area outside of her specific areas of interest. As expected her insights and suggestions brought a fresh perspective to the project and her careful readings of the document have led to many improvements. In addition to the efforts of these three persons, I wish to thank the audiologists at the three participating VA clinics (Bay Pines, Florida, Mountain Home, Tennessee, and Little Rock, Arkansas). In particular, I want to thank Ann Strouse from the Mountain Home, Tennessee facility. She went out of her way to be supportive and responsive to our needs. Finally, I want to thank Judy Reese and Rachel Ridge for being the independent observers. Without their thoughtful and time-consuming contributions, this project could not have been completed.
Robert F. Zelski 4 INTRODUCTION In the present health care environment there is an increasing demand for providers of rehabilitative services to demonstrate the efficacy of their efforts. That is, they are required to measure the outcomes of intervention. For hearing aid rehabilitation, this means demonstrating improved communication capabilities with hearing aid use. Traditionally, the outcomes of hearing aid intervention have been demonstrated using objective measures such as the functional gain, speech recognition testing, and realear responses. Discussions of these measures can be found in many audiology texts and research literature (e.g. Kuk, Harper, & Doubek, 1994; Mason & Popelka, 1986; Millen, 1975; Northern, 1992; Ringdahl & Lejohn, 1984; Studebaker, 1982; Tobin, Baquet, & Koslowski, 1997). In general, functional gain measures are used to demonstrate that individuals can detect less intense sounds when using amplification. Word recognition tests are used to demonstrate improvement in the ability to understand average intensity levels of speech. Real-ear measures are similar to functional gain measures, but are nonbehavioral. That is, they do not require the active participation of the patient. Real-ear measures are used to demonstrate that the hearing aid increases the sound pressure level (SPL) of sounds reaching the tympanic membrane. The real-ear measures are often used in conjunction with some prescriptive formula that is designed to predict successful hearing aid amplification values. While auditory communication is clearly dependent on the ability to detect and recognize important acoustic information in the speech spectrum, improvements assessed by these objective techniques do not always correspond to an individualÂ’s communication functioning in everyday life, i.e. to real-world outcomes (e.g., Beck, 1982; Dillon, James, & Ginis, 1997; Humes, Christensen, Bess, Hedley-Williams, 1997; Nilsson, Vesterager, Sibelle, Sieck, & Christensen, 1997; Schwartz, 1982; Surr, Cord, & Walden, 1997; Walden, 1982; Weinstein, 1997).
Robert F. Zelski 5 An alternative to the use of objective hearing aid benefit measurements is the use of self-report methodology. Over the last 20 years, several subjective, self-report tools have been developed or adapted for the purpose of assessing the real-world outcomes of hearing aid use. Typically, these instruments are composed of a series of predetermined questions that ask hearing aid users to assess their ability to hear and/or understand in various listening situations or they ask patients to relate some emotional or social reaction to the hearing loss. Either the clinician presents the questions in a face-to-face format or the hearing aid user completes the questionnaire without the participation of the clinician A review of many of these subjective outcome measures can be found in Huch (1999). One of the earliest subjective outcome measures that gained widespread use in for assessing hearing aid benefit was the Hearing Handicap Inventory for the Elderly (HHIE) developed by Ventry & Weinstein (1982). In its original form, the HHIE was designed to quantify the effects of hearing impairment on the ability of older persons to function in everyday life, but it was adopted for the assessment of hearing aid benefit very soon after development (Newman & Weinstein, 1988). The 25 questions in the HHIE include 13 that assess the emotional impacts of the hearing loss and 12 questions that assess the social impact. The patient answers each question with a response of Â“yes,Â” Â“sometimes,Â” or Â“no.Â” The audiologist scores the answers with a Â“4Â” when the patient responds Â“yesÂ”, a Â“2Â” when the patient responds Â“sometimes,Â” and a Â“0Â” when the patient responds Â“no.Â” The maximum score is 100 and the minimum score is 0. This numeric scoring allows for the handicap to be quantified, with a higher score corresponds to a more handicapping condition. The HHIE can be administered verbally or in a paper-and-pencil format and the administration time is approximately 10 minutes (Ventry & Weinstein, 1982). In the 1980s and 1990s several other subjective measurement instruments were developed to assess hearing aid benefit (e.g., Demorest & Erdman, 1986; Cox & Gilmore, 1990; Walden, Demorest, & Hepler, 1984, Cox & Alexander, 1995). One of the most common subjective benefit measures in use today is the Abbreviated Profile of Hearing
Robert F. Zelski 6 Aid Benefit (APHAB) developed by Cox & Alexander (1995). This measure calls for a patient to answer 24 predetermined questions during the initial visit. At the end of the hearing aid fitting process the same 24 questions are administered. The difference in the rating between the initial and the final visit is scored as the amount of benefit in each of four general categories. As an alternative, both the pre-fitting and post-fitting assessments can be completed at the end of the fitting process. The four categories are ease of communication (EC), listening in background noise (BN), listening in reverberant conditions (RV), and aversiveness of sounds (AV). The APHAB can be administered in a paper-and-pencil format or with the use of a computer. The computerized format allows for the production of charts that demonstrate the derived benefit in each of the four categories. Both the HHIE and the APHAB are well researched and psychometrically sound instruments. Recently, however, the use of predetermined items to assess hearing aid benefit has been questioned. Researchers in Australia (Dillon et al., 1991a & 1991b and Dillon, James, & Ginis, 1997) and in Great Britain (Gatehouse, 1994; Gatehouse, 1999) raised the issue that using predetermined questions might lack precision in assessing hearing aid benefit for a particular patient. These researchers note that this is because some questions may not be relevant for the individual. Inclusion of non-relevant items increases the administration time and may limit the amount of beneficial information derived from using the instrument. These concerns may account, in part, for the recent survey data of Martin (1998) who found only 22% of audiologic practices in the United States report use of standardized self-report instruments. In recent years two subjective measures have been introduced to address the issue of using predetermined items to subjectively assess hearing aid benefit. These are the Client Oriented Scale of Improvement (COSI), which was developed by Dillon and his associates in Australia (Dillon, et al. 1997) and the Glasgow Hearing Aid Benefit Profile (GHABP), developed by Gatehouse (1999). Both instruments utilize the open response format first suggested by Stephens (1980). In this approach no preset items exist. Rather, the patient nominates situations or circumstances that are important to him/her in
Robert F. Zelski 7 obtaining amplification. The benefit measures are then determined by how well these patient identified situations are met. The GHABP is actually a hybrid approach utilizing both pre-selected items and open response items. The GHABP, therefore, consists of two sections. In the first section there are four preset items that may or may not be applicable to a patient. One such question asks if the patient has difficulty when having a conversation with one person in a quiet environment. Another question asks if the patient has difficulty having a conversation in a group. The patient first identifies whether or not the question is applicable to his/her life and, if it is applicable, answers a series of six questions about that item. The six questions, which are the same for all four items, assess the degree of difficulty encountered with and without a hearing aid in that situation. The second section of the GHABP utilizes Stephens (1980) open response approach. The patient nominates up to four additional situations that he/she feels is appropriate in his/her life and completes the six-item questionnaire for each of the nominated situations. For both sections, the patients rate the degree of difficulty for each of the six questions. The degree of difficulty is rated with a descriptor ranging from Â“no difficultyÂ” to Â“cannot manage at allÂ” and with a numeric equivalent ranging from 1 for Â“no difficultyÂ” up to 5 for Â“cannot manage at all.Â” By using numeric equivalents the ratings can be statistically analyzed. Unlike the GHABP the COSI uses only the open response format. The COSI is described by its developers as a quick and simple procedure and in two research studies the developers found the COSI to be an effective method of assessing hearing aid benefit for patients in the Australian Hearing Services (Dillon, et al., 1997; Dillon, Birtles, & Lovegrove, 1999). To use the COSI, the patients identify up to five specific situations that they would like to have improved by wearing amplification. These situations can be listening situations or they can be emotional or social situations. This identification of specific situations is made prior to the hearing aid fitting and the clinician groups the patient identified situations into one of 16 standard categories. At the end of the
Robert F. Zelski 8 rehabilitation process each patient is asked two questions about each of the specific situations identified at the beginning of the process. The first question asks the patient to rate the degree of change in his/her hearing for that situation. The degree of change is noted by a series of five descriptors ranging from Â“worseÂ” to Â“much better.Â” The second question asks the patient to rate his/her final ability with the hearing aid for each of the identified situations. This rating ranges from Â“can hardly ever hearÂ” in that situation to Â“can almost always hearÂ” in that situation. Again there are five choices for the patient. These final ability ratings have percentage equivalents ranging from 10% for Â“hardly everÂ” to 95% for Â“almost always.Â” The degree of change rating does not have a numerical equivalent on the COSI form, but the developers noted that the five descriptors can be rank ordered on a scale of 1 to 5 for analysis purposes, with one corresponding to Â“worseÂ” and five corresponding to Â“much betterÂ” (Dillon, et al., 1997; Dillon, et al., 1999). In 1997, Dillon et al. compared the COSI to several other subjective measures of hearing aid benefit. The other instruments used in the study were the HHIE (Ventry and Weinstein, 1982), a modification of the Profile of Hearing Aid Benefit (Cox, Gilmore, & Alexander, 1991; Cox & Rivera, 1992); the Shortened Hearing Aid Performance Inventory for the Elderly (Dillon, 1994); and the Hearing Aid Users Questionnaire (Forster and Tomlin, 1988). From this study Dillon, et al. (1997), concluded that the COSI could be used validly and reliably in individual hearing aid fittings. In 1999, Dillon et al. utilized the COSI along with the HAUQ (Forster & Tomlin, 1988) to assess the outcomes of the Australian national hearing aid program. While the COSI measures client needs, changes in listening ability and final listening ability, the HAUQ assesses reported hearing aid use, benefit, problems and satisfaction. The results of the study, which were based on clinician questionnaires, confirmed previous work (Dillon et al., 1997) finding the COSI is useful in individualized hearing aid fittings. To use COSI for programmatic comparisons, the specific situations were grouped into the 16 standard categories and the data from these 16 categories was then compiled across
Robert F. Zelski 9 patient groups of interest. The results indicated that COSI data were useful for describing differences in hearing aid benefit across sub-groups of the total population served. From this the researchers suggested that COSI data could be used for comparing service quality and outcomes across different centers. Although the results of this study (Dillon, et al., 1999) support using the COSI as part of program evaluation, one issue that remains to be addressed is whether or not multiple clinicians would group a specific situation nominated by a patient into the same standard category. If the COSI data is to be used to assess service quality and outcomes for a multi-practitioner dispensing system, then it is important to ascertain if there is agreement and reliability of categorizations across clinicians. In the same study Dillon et al (1999) noted that although the use of the COSI in the Australian Hearing Service was voluntary, it was being administered to 40% of the patients in the system and that the overall opinion of the audiologists in the system was favorable toward using the COSI. While 40% may seem low, it appears to be substantially greater than the 22% rate reported for all subjective outcome measures in the United States (Martin, 1998). In the United States, administrators of the Department of VeteranÂ’s Affairs Hearing Aid Program (VANHAP) have expressed a need to implement subjective outcome measures as a routine part of its hearing aid dispensing protocol (personal communication with L. Beck, 1999). At present the VANHAP reports contain only numeric data such as the number of hearing aids dispensed and the number of hearing aids returned. The central office of the VANHAP obtains no other clinical outcome measures at this time (personal communication with L. Beck, 2000). Audiologists in the VANHAP dispensed over 130,000 hearing aids in 1998 at a cost exceeding $45,000,000. By fiscal year 2000 these numbers had grown to 187,500 and $66,500,000. Assessing the quality of the service provided is important in any
Robert F. Zelski 10 professional practice, and this is especially true with a system as large as the VANHAP. The use of subjective outcome measures can provide information as to how well the needs of the individual veteran are being met (i.e., they can assess the quality of the intervention). This information would be particularly useful for the central office of the VANHAP if the results from the many centers in the system can be compiled and used for systemic oversight. Many VANHAP clinics already use subjective outcome measures, but there is no standardization across the entire system. To use subjective outcome measures systemically, standardization is necessary. Several key elements must be considered before a particular instrument can be suggested for standard use across the dispensing clinics of the VANHAP. Among these are: 1) the time required to administer the instrument must be commensurate with the utility of the measure; 2) the information provided must be beneficial to the clinician in helping an individual patient; and 3), the information provided should allow for systemic oversight of a multi-clinic/multi-clinician system. The COSI appears to meet these requirements. Before the COSI can be implemented in the VANHAP, however, the agreement of the categorization process across clinicians must be determined and the assessment of the utility of the COSI by the audiologists within the VANHAP should be obtained. Also, it is important to demonstrate if the utility of the COSI that was evident in Australia is also evident in an evaluation conducted independently from the instrumentÂ’s developers. The specific purposes of this work were to: (1) Determine inter-rater agreement and reliability of classification of patient identified specific situations into the 16 standard categories; and, (2) Conduct an independent examination of the COSI as a fitting tool.
Robert F. Zelski 11 METHOD Participants To address the goals of this study, it was important for data to be collected from multiple VA clinics and from multiple clinicians. Attempts were made to achieve a balance among the size and complexity of audiology clinics in the VA system. Audiologists from three VANHAP centers (labeled Clinic A, Clinic B, and Clinic C in this report) were recruited for participation and one audiologist at each of these clinics was designated as the site coordinator for the investigation. A total of 50 COSI forms were to be completed at each of these sites. Within the three centers, a total of eight audiologists participated in the study. After a site had agreed to participate COSI forms were sent to the site coordinator for distribution among the participating audiologists. A copy of the COSI form is shown in Appendix A. Independent Observers To assess inter-observer agreement, two experienced audiologists were recruited. Both were experienced in hearing aid fitting and in the administration of the COSI. Instructions for COSI Administration In addition to receiving the COSI forms, the site coordinator at each of the three VANHAP sites was provided with written instructions for the administration of the COSI. These instructions were to be distributed to each audiologist participating in the investigation. No efforts were made to verify if the participating audiologists had read the instructions. The instructions were very similar to those used by the Australian Hearing Aid System (Dillon, et al., 1997). A copy of the instructions is shown in Appendix B. A key element of the instructions was to stress that the patient should be as specific as possible about the situation identified. For example, Â“wanting to hear better in a noisy environmentÂ” would not be sufficiently specific. On the other hand, Â“wanting to hear my wife better at the dinner tableÂ” would be better and Â“wanting hear my wife better
Robert F. Zelski 12 at the dinner table when there are more than four to five people at dinnerÂ” would be better still. COSI Procedure The standard COSI form shown in Appendix A was used in this investigation. Other than adding the COSI administration, participating clinics were instructed to make no other changes in their standard protocols for hearing aid selection, fitting, and followup. Also, the procedures for administering the COSI as a subjective outcome measure in a clinic followed the format recommended by Dillon et al. (1997). During the initial hearing aid selection patient visit, the patient was instructed to identify from one to five specific situations that were important to him/her and that he/she wished to have improved by wearing amplification. Once all situations were identified the patient was instructed to rank order them. Next, the audiologist recorded the appropriate standard category on the COSI form for each of the specific situations. A list of the 16 standard categories is on each COSI form and is shown in Table 1. For example, if the specific situation identified by the patient was Â“wanting to hear my wife better at the dinner table when there are more than four to five people at dinnerÂ” would be placed in category three, which is conversation with 1 or 2 in noise. Table 1. COSI Categories 1. Conversation with 1 or 2 in quiet 2. Conversation with 1 or 2 in noise 3. Conversation with group in quiet 4. Conversation with group in noise 5. Television/Radio @ normal volume 6. Familiar speaker on phone 7. Unfamiliar speaker on phone 8. Hearing phone ring from another room 9. Hear front door bell or knock 10. Hear traffic 11. Increased social contact 12. Feel embarrassed or stupid 13. Feeling left out 14. Feeling upset or angry 15. Church or meeting 16. Other
Robert F. Zelski 13 At the end of the hearing aid fitting process, the second part of COSI administration was to be completed during a follow-up visit. The participating clinics were allowed to complete this portion of the COSI within the normal clinical fitting protocol of the facility. For the purposes of this investigation, the second phase of COSI administration was not required. Once the COSI form was completed, a copy was sent to the investigators. The specific situations identified by the patients and the categorization of the situations served as the input data for the first part of this investigation. Clinician Questionnaire Once each audiologist had completed his/her allotted number of COSI forms, he/she was given a copy of the Clinician Questionnaire (Appendix C) to complete. The Clinician Questionnaire consisted of two parts. Part I consisted of five questions that required the clinician to rate several aspects of the COSI. A list of these questions is shown in Table 2. For all five questions, the clinician was asked to respond by circling a number on a slide scale that ranked from 0.0 to 4.0 in increments of 0.5 with 0.0 rating as the poorest opinion and 4.0 rating as the highest opinion. To facilitate selection, four descriptors were associated with the numeric ratings. For example, for question 5, the four descriptors of ease of use due to the open response format were Â“very easy to use,Â” Â“somewhat easy to use,Â” Â“not very easy to use,Â” and Â“very hard to use.Â” These descriptors were placed between the numeric ratings. For example, Â“very easy to useÂ” was positioned between 4.0 and 3.0, while Â“very hard to useÂ” was placed between 1.0 and 0.0. Part II of the Clinician Questionnaire was a series of informational questions. These were designed primarily to obtain background information about the participating audiologist that might facilitate the interpretation of the data obtained in Part I of the
Robert F. Zelski 14 Clinician Questionnaire. For example, one question was to determine if the clinician had ever used a subjective outcome questionnaire, and a second question asked, what measure(s). Another question asked what measures the clinician generally used to validate hearing aid fittings. Other questions covered demographic data such as years practicing audiology, the size and type of audiology clinic, the academic level of the audiologist, and the number of audiologists in the clinic. Table 2. Part I of Clinicians Questionnaire 1. Clinical Usefulness Rate the overall usefulness of the COSI in the hearing aid fitting process 2. Administration Time Asses the time required to administer the COSI relative to its clinical utility 3. Comparison As compared to other subjective outcome measures (such as the APHAB) the COSI is: 4. Effectiveness of data collected Rate the effectiveness of the data collected from the patients by using the COSI in refining and implementing the hearing aid selection and fitting process 5. COSI format The open response format makes the COSI: Data analysis To assess inter-observer agreement and reliability, the categorization data were first examined in a series of confusion matrices comparing (1) Observer 1 (originating audiologist) to Observer 2; (2) Observer 1 to Observer 3; and, (3) Observer 2 to Observer 3. Second, CohenÂ’s kappa (Bateman & Gottman, 1986) was used to assess the level of agreement across the three observers. This statistic was chosen because it takes into account agreement that may occur by chance alone. Finally, as recommended by Bateman & Gottman (1986), the data were used to calculate CronbachÂ’s alpha, a measure of generalizability. Then, to address the issue of clinical utility, the data obtained through the clinician questionnaire were examined using descriptive analysis.
Robert F. Zelski 15 RESULTS AND PRELIMINARY DISCUSSION Inter-observer agreement The first purpose of this study was to examine the inter-observer agreement of the COSI categorization procedure. 107 patients (37 from Clinic A, 50 from Clinic B, and 20 from Clinic C) at the three participating clinics identified a total of 313 specific situations. Overall, this is an average of 2.9 specific situations for each patient. By clinic, the average number of specificsituations was 2.6 for Clinic A, 3.1 situations for Clinic B, and 2.7 situations for Clinic C. It will be recalled that the instructions were for the audiologist to classify each of the identified situations into one of the 16 standard categories. This classification was then repeated by each of the two independent observers. In nine cases (six at the originating clinics and three by the independent observers) specific situations were placed in more than one category. As a result, these nine cases were omitted from the analysis leaving a total of 304 cases to be analyzed. Confusion matrices were constructed to assess the agreement across the observers as a function of the categories. The matrices are shown in Appendix D. Agreement between observers was very good. The overall agreement across all three observations is 78.6 %. The agreement between the original observer and each of the two independent observers and between the two independent raters were also very good (77.15 %, 77.5 %, and 80.1 %). On closer inspection the majority of the disagreements arose when distinguishing between two of the general categories. These two categories were: (1) for understanding Â“familiar speaker on the phoneÂ” (53.4 %) and (2) for understanding Â“unfamiliar speaker on the phoneÂ” (67.8 %). Also, the disagreements among the observers in these two categories were almost entirely due to placing the situation into the other of the two phone groups. For example, all nine of the disagreements for the category Â“unfamiliar speaker on the phoneÂ” categorized the situation as Â“familiar speaker on the phone.Â” The disagreements for Â“familiar speaker on the phoneÂ” were similar with
Robert F. Zelski 16 16 of the 20 disagreements placing the situation in Â“unfamiliar speaker on the phone.Â” If the two categories had been entered as a single Â“understanding speakers on the phoneÂ” category, the agreement levels across observers jump to 90.7 % for Â“familiar speakers on the phoneÂ” and to 100 % for Â“unfamiliar speakers on the phone.Â” When assessing levels of agreement between independent raters, there is agreement at some level simply due to chance. To fully assess the true agreement across raters, the agreement due to chance needs to be taken into account. CohenÂ’s kappa is designed to account for agreement due to chance. CohenÂ’s kappa is defined as: k = P o P c_ (1) 1 Pc where Po is the proportion of agreement actually observed and Pc is the proportion of agreement due to chance. To derive Po the actual number of agreements among the original scoring at the participating clinics and the independent raters are summed. This sum is then divided by the total number of comparisons made by the raters. Pc is obtained by summing the by chance agreement for each of the sixteen categories. To determine the chance probabilities for any of the sixteen categories, the first step is to determine the chance probability for each of the two raters in that category and then find the product of the two probabilities. All of these products are then added together to determine the total probability by chance (Bakeman & Gottman, 1986). An analysis of the data in this study revealed a CohenÂ’s kappa score of 0.753. According to Bakeman & Gottman (1986), a kappa score above 0.70 denotes Â“significantÂ” agreement. Fleiss (1981) reports that a kappa above 0.75 denotes Â“excellentÂ” agreement across observers. The final factor that must be considered when assessing agreement across independent observers is whether or not the agreement is reliable and generalizable. Bakeman and Gottman (1986) note that CronbachÂ’s alpha can be used to assess these
Robert F. Zelski 17 factors. CronbachÂ’s alpha is a coefficient of reliability across observers and is computed as: = MS c _-_MS o (2) MSc + MSo where MSc represents the pooled variance for the categories into which the specific situations were placed and MSo represents the variance for the observers. In their discussion of CronbachÂ’s alpha, Bakeman and Gottman (1986) noted that it was a novel reliability concept that equates reliability with two factors. First, it attempts to assess whether the instrument does the work it was intended to do and second, if multiple raters do the work, it assess whether or not they get the same results. They add that this implies that all we need to do to demonstrate generalizability and reliability of the measure is to show that the observers are essentially interchangeable. CronbachÂ’s alpha is a measure of how interchangeable the observers are. In this study, CronbachÂ’s alpha was .887. This score demonstrates a high degree of reliability for the categorization and indicates an ability to generalize the procedure to other observers. COSI as a clinical tool The second purpose of this study was to examine the usefulness of the COSI as a routine clinical tool. The data for this analysis was collected using the Clinician Questionnaire (Appendix C) that was completed by each clinician following completion of all of their allotted COSI forms. As noted in the methods section, in order for an outcome measure to be clinically acceptable it must meet several criteria. First, the measure must be of clinical assistance in the intervention process. Second, the clinician must find the measure to be relatively simple to use. Third, the data collection process must be simple. Fourth, the administration time for the measure must be commensurate with the usefulness of the data obtained. The Clinician Questionnaire was designed to assess these criteria. Eight clinicians from two clinics (Clinic A and Clinic B) submitted completed questionnaires. This number is not considered adequate for achieving a valid and/or reliable assessment of the questions asked in the study. The findings reported here are, therefore, merely preliminary. These results will be used as pilot data to more fully
Robert F. Zelski 18 assess the clinical utility of the COSI and to validate the Clinician Questionnaire for use in future studies. The results obtained from the questionnaire were analyzed with descriptive methods. The results of the descriptive analysis of answers to Part I of the questionnaire are shown in Table 3. Table 3. Clinician Questionnaire scores, means, and standard deviations Question 1 Usefulness Question 2 Time Question 3 Comparison Question 4 Effectiveness Question 5 Format Clinician 1 3.5 2.5 3.5 3.5 4.0 Clinician 2 3.5 3.5 2.5 3.5 4.0 Clinician 3 3.0 2.5 2.5 2.5 3.5 Clinician 4 2.0 2.0 2.0 2.0 2.0 Clinician 5 3.0 3.0 2.5 3.0 1.5 Clinician 6 3.0 3.5 2.0 3.0 2.5 Clinician 7 2.5 3.5 2.0 2.5 2.5 Clinician 8 2.5 4.0 3.0 1.5 2.5 Mean 2.9 3.1 2.8 2.7 2.9 Std. Dev 0.52 0.68 0.66 0.70 0.93 Question 1 asked the clinicians to rate the usefulness of the COSI in the hearing aid fitting process. The mean score of 2.9 on Question 1 suggests that the clinicians found the COSI to be between Â“very usefulÂ” and Â“somewhat usefulÂ” in the fitting process. The standard deviation was 0.52. This was the smallest variability among the eight questions and suggests that there is consistency across clinicians in their opinions of the utility of the COSI in the hearing aid fitting process. Clinicians 1 and 2 rated the usefulness of the COSI at 3.5, Clinicians 3, 5, and 6 rated the usefulness at 3.0, and Clinicians 7 and 8 rated the usefulness at 2.5. Only Clinician 4 rated the usefulness lower than Â“somewhat usefulÂ” and this rating was a neutral 2.0.
Robert F. Zelski 19 Question 2 asked the clinicians to assess the administration time required for using the COSI relative to its clinical utility. The average score of 3.1 was the highest rating obtained for any of the five questions. The standard deviation was 0.68 with a high score of 4.0 and a low score of 2.0. On closer inspection, it was observed that one Clinician 8 felt that the COSI was very efficient when assessing administration time (4.0). Clinicians 2, 6, and 7 also felt that the administration time was very good (3.5) and Clinician 5 rated the administration time at 3.0. Clinicians 1 and 3 rated the administration at 2.5, which equates to an average time. Again, only Clinician 4 rated the administration time lower, with a 2.0 score. Question 3 asked the clinicians to compare the COSI to other subjective outcome measures. The 2.8 average suggested that clinicians found the COSI to be Â“better than mostÂ” other subjective outcome measures. For this question only one Clinician 1 rated the COSI Â“the bestÂ” and only Clinician 8 rated the COSI between Â“the bestÂ” and Â“better than most.Â” Three (Clinicians 2, 3, and 5) rated the COSI at 2.5 (better than most) and three (Clinicians 4, 6, and 7) rated the COSI between Â“better than mostÂ” and Â“worse than mostÂ” (2.0) as compared to other subjective outcome measures. Question 4, which asked the clinicians to rate the effectiveness of the COSI data in refining and implementing the hearing aid selection and fitting process. On average the effectiveness was rated at 2.7. Clinicians 1 and 2 rated the effectiveness of the COSI data for refining and implementing the selection and fitting process at 3.5 (very effective) and two others (Clinicians 5 and 6) rated it at 3.0 (between Â“very effectiveÂ” and Â“somewhat effectiveÂ”). Clinicians 4, 6, and 7 had neutral ratings of 2.0 and Clinician 8 rated the COSI as Â“not very effectiveÂ” with a 1.5 score. The last question, Question 5, asked the clinicians to assess the open response format used in the COSI relative to the ease of use. The mean score of 2.9 indicated that the clinicians rate the open response format between Â“somewhat easy to useÂ” and Â“very easy to use.Â” However, the standard deviation (0.93) for Question 5 was the largest
Robert F. Zelski 20 among the five questions, which suggests a lack of consistency in clinician opinions of the open format. Clinicians 1 and 2 rated the open format at a very positive 4.0, but Clinician 4 rated the format at a neutral 2.0 and Clinician 8 rated the format at 1.5. For the other four clinicians, Clinician 3 rated the COSI at 3.5 and Clinicians 6, 7, and 8 rated the open response format of the COSI at 2.5. Overall, the analysis of the responses to the questionnaire suggests a generally favorable opinion of the COSI among the participating clinicians. The results also indicated that the ratings were not uniformly favorable across all clinicians nor were they uniformly favorable for all the questions asked. Overall, the most favorable rating was for the time of administration (Question 2) and the poorest rating was for the effectiveness of the data collected (Question 4). The standard deviations were somewhat large, ranging from a low of 0.52 to a high of 0.93. This variability was not unexpected given the small sample size. On closer inspection, several key elements stand out. First, clinicians 1 Â– 3 were from one clinic (Clinic A), while clinicians 4 Â– 8 were from the second clinic (Clinic B). For all questions except Question 2 the respondents at the Clinic A had a more favorable impression of the COSI than did the respondents from Clinic B. One possible reason for this difference may be that the clinicians at Clinic A had used the COSI as part of their hearing aid fitting protocol for some time. At Clinic B, the COSI had not been used prior to the conduct of this investigation. A second possible reason for the difference between the two clinics is that one respondent from Clinic B (Clinician 4) had a substantially poorer opinion of the COSI than did any other participant, thereby skewing the data for Clinic B. Clinician 4 rated each item on the questionnaire at 2.0. For the 35 responses obtained from the other seven participants, only four other ratings were at the level of 2.0 or below. It should be noted that Clinician 4 was the only participant in the investigation who reported had never
Robert F. Zelski 21 having used any subjective outcome measure to assess hearing aid performance prior to this study. A closer examination of individual responses reveals a relationship between the cliniciansÂ’ responses to the items on the questionnaire and their previous experience using subjective outcome measures. Clinician 5, for example, found the open format structure (Question 5) of the COSI very hard to use. This clinician reported using the APHAB as a standard fitting tool. Clinician 8 did not find the information obtained from the COSI very effective in refining and implementing the hearing aid selection and fitting process (Question 4). Clinician 8 reported experience using both the APHAB and HHIE, but also reported not using these or any other subjective outcome measure on a routine basis. Among the eight respondents to the clinician questionnaire, two other clinicians (both from Clinic B) reported that they not use any subjective outcome measure, while the other six reported that they do use one or more subjective outcome measures (all 3 clinicians at Clinic A and 2 or the 5 from Clinic B). Clinician 4, who rated the COSI at 2.0 for each of the 5 questions, reported never having used any subjective outcome measure prior to this investigation. Clinician 7, who reported not routinely using any subjective outcome measures, rated the COSI at 2.0 when comparing the COSI to other subjective outcome measures (Question 3). Clinician 6, who reported use of the APHAB as a routine outcome measure, also rated the COSI at 2.0 as compared to other outcome measures. SUMMARY AND CONCLUSIONS General Summary The results of the study are encouraging. The response pattern of the participating clinicians to using the COSI as a standard part of the fitting protocol is good. The clinicians found the COSI to be useful in the overall fitting process, they felt the administration time was commensurate with the clinical utility, and they felt that the COSI was somewhat helpful in the hearing aid selection and fitting process. Also, most participating clinicians rated the COSI better than most other subjective outcome
Robert F. Zelski 22 measures and they also reported that the open response format was somewhat easy to use. The combined ratings of overall usefulness and appropriate administration time are especially encouraging for the possibility of implementing the COSI on a system wide basis within the VANHAP. The very high degree of agreement across observers in the process of categorizing the specific situations identified by patients is also very encouraging. As a result it should be possible to use the categorization as a tool of systemic oversight. This allows the managers of the VANHAP to use the categorization portion of the COSI to help assess the efficacy of hearing aid fittings throughout its system. This capability can be used in several ways. First, the managers can assess how well patients with certain types of outcome desires are being served. For example, they could determine how well patients who identified specific situations that were categorized as listening to one or two persons in quiet as their most important goal in the amplification process were fit. The two followup measures in the COSI (degree of change and final ability [with hearing aid]) could be summarized across all patients who are sorted into that category and used to determine how much benefit these patients received and how well they felt they hear when listening to one or two persons in quiet environments. A second way in which the reliable categorization can be used is to assess the outcomes of fittings at individual clinics within the VANHAP. The managers of the system should be able to look at how well individual clinics do in fitting patients in each of the sixteen categories. If some clinics show a greater degree of change improvement or higher final ability (with hearing aid) scores than others, the data about type of hearing aid fittings made and/or the fitting protocol can be examined to determine why one clinic might do better than others for certain types of problems.
Robert F. Zelski 23 A third way to use this result is to help provide information to hearing aid researchers and manufacturers. By identifying those categories where patients do very well or do very poorly, research efforts can be focused on those areas in the greatest need of improvement. A fourth area of use is to allow for the assessment of different amplification strategies for different types of hearing problems. This use is closely related to the critical assessment across clinics. For example, the use of compression limiting or wide dynamic range compression could be compared according to the listening environments important to the patient. Problems Encountered In the conduct of this study some problems did arise. The first problem was that not all COSI forms were completed in full compliance with the instructions. The researchers chose to supply each clinic with written instructions for the use of the COSI. It was decided not to provide specific on site training beyond these written instructions. In most cases, the written instructions were adequate. In a few situations, the input from the participating clinics indicated that further instruction would be beneficial. For example, in six of the 313 specific situations identified by patients, the clinician placed the situation into multiple categories rather than one category as instructed. This problem should be very easy to remediate. A second example of a need for additional training was that some clinicians tended to describe the specific listening situation on the COSI form in words that approximated the words used in the sixteen general categories. This problem would probably be eliminated by simply reminding the clinicians that the intent of the initial COSI interview is to have the situation described in the patientÂ’s own words. Neither of these situations was widespread in the data collected and they were not felt to have degraded the validity of the study in any way. If the use of the COSI for systemic oversight is implemented within the VANHAP or any other hearing aid delivery
Robert F. Zelski 24 system, early inspection of the completed forms followed by periodic random review of COSI forms should allow for rapid repair of these minor difficulties. More meaningful problems were encountered in the pilot study of the assessment of the COSI as a clinical tool. Most importantly, the questionnaire did not include a means of assessing the overall rating of the COSI as a clinical tool. In addition, there was question as to the placement of the descriptors in Part I of the questionnaire relative to the numeric value. For example, in Question 1, Â“of no useÂ” was placed between the numeric ratings of Â“0.0Â” and Â“1.0.Â” The question arose as to whether this placement is appropriate or would it be more appropriate for Â“of no useÂ” to be equated with a numeric equivalent of Â“0.0.Â” None of these problems hampered the conduct of the present study. All, however, should be addressed prior to conducting a larger survey of clinician opinions of the COSI as routine part of clinical operations. Recommendations While the results of the study suggest that the COSI is a very user-friendly subjective outcome measure capable of being used to provide systemic oversight of the VANHAP, the study findings some shortcomings that may be overcome by implementing the following recommendations: 1) Reduce the number of standard categories. For example, combining the two telephone listening categories (categories 6 & 7) would increase the agreement across different observers. 2) Provide training beyond the written instructions. The two areas of problems encountered in the data collection are both readily amenable by additional training. The two problems encountered were 1) sorting the specific situations into more than 1 of the 16 standard categories and 2) using the terms of the standard categories in listing the specific situations. More extensive training on the clinical use of the COSI should resolve both of these problems.
Robert F. Zelski 25 3) Add a question to the Clinician Questionnaire to determine the cliniciansÂ’ overall opinion of the COSI. 4) Examine changing the scaling procedure used for the Clinician Questionnaire to create a more consistent descriptor among all of the questions 5) Examine the relationship between the descriptors and the numeric scaling in the Clinician Questionnaire. Final Comments In summary, this study shows that the COSI can be used reliably within the VANHAP or any other hearing aid delivery system to oversee the effectiveness of its hearing aid delivery process. In addition, this reliability can also be used to assist in conducting hearing aid research and in developing new amplification products and/or strategies to meet the listening needs of patients.
Robert F. Zelski 26 References American Speech-Language-Hearing Association (1998). Guidelines for hearing aid fitting for adults. American Journal of Audiology, 7 5-13. Beck, L. (1982). The VeteranÂ’s Administration Hearing Aid Program: Research needs in amplification. In Studebaker, G.A. & Bess, F.H. (Eds.). The Vanderbilt Hearing-Aid Report Upper Darby, Pa., Monographs in Contemporary Audiology. 156-160. Cox, R.M.& Gilmore, C. (1990). Development of the profile of hearing aid benefit. Journal of Speech and Hearing Research, 33 343-357. Cox, R.M., Gilmore, C., and Alexander, G.C. (1991). Comparison of two questionnaires for patient-assessed hearing aid benefit. Journal of the American Academy of Audiology, 2 134-145. Cox, R.M. and Rivera, I.M. (1992). Predictability and reliability of hearing aid benefit measured using the PHAB. Journal of the American Academy of Audiology, 3 242-254. Cox, R. M. and Alexander, G. C. (1995). The abbreviated profile of hearing aid benefit. Ear and Hearing, 16, 176-186. Demorest, M.E. & Erdman, S.A. (1987). Development of the communication profile for the hearing impaired. Journal of Speech and Hearing Disorders, 52 129-143. Dillon, H. (1994). Shortened Hearing Aid Performance Inventory for the Elderly (SHAPIE): a statistical approach. Australian Journal of Audiology, 16 37-48. Dillon, H., Koritschoner, E., Battaglia, J., Lovegrove, R., Ginis, J., Mavrias, G., Carnie, L., Ray, P., Forsythe, L., Towers, E., Goulias, H., and Macaskill, F. (1991a). Rehabilitation
Robert F. Zelski 27 effectiveness I: assessing the needs of clients entering a national hearing rehabilitation program. Australian Journal of Audiology, 13, 55-65. Dillon, H., Koritschoner, E., Battaglia, J., Lovegrove, R., Ginis, J., Mavrias, G., Carnie, L., Ray, P., Forsythe, L., Towers, E., Goulias, H., and Macaskill, F. (1991b). Rehabilitation effectiveness II: assessing the outcomes for clients of a national hearing rehabilitation program. Australian Journal of Audiology, 13, 68-82. Dillon, H., James, A., and Ginis, J. (1997). Client oriented scale of improvement (COSI) and its relationship to several other measures of benefit and satisfaction provided by hearing aids. Journal of the American Academy of Audiology, 8, 27-43. Dillon, H., Birtles, G., & Lovegrove, R. (1999). Measuring the outcomes of a national rehabilitation program: Normative data for the client oriented scale of improvement (COSI) and the hearing aid userÂ’s questionnaire (HAUQ). Journal of the American Academy of Audiology, 10, 67-79. Fleiss, J.L. (1981). Statistical Methods for Rates and Proportions New York, Wiley. Forster, S. and Tomlin, A. (1988, May). Hearing Aid Usage in Queensland. Paper presented at the Audiological Society of Australia Conference, Perth. Gatehouse, S. (1990). Determinants of self-reported disability in older subjects. Ear and Hearing, 11, 57S-65S. Gatehouse, S. (1994). Components and determinants of hearing aid benefit. Ear and Hearing 15, 30-49.
Robert F. Zelski 28 Gatehouse, S. (1999). Glasgow hearing aid benefit profile: Derivation and validation of a client-centered outcome measure for hearing aid services. Journal of the American Academy of Audiology, 10, 80-103. Huch, J. (1999). In documenting user benefit/satisfaction, there are many tools to choose among. The Hearing Journal, 52(4) 60-70. Humes, L.E., Christensen, L.A., Bess, F.H., & Hedley-Williams, A. (1997). A comparison of the benefit provided by well-fit linear hearing aids and instruments with automatic reductions of low-frequency gain. Journal of Speech, Hearing, Language Research, 40 666-685. Kuk, F.K., Harper, T., & Doubek, K. (1994). Preferred real-ear insertion gain on a commercial hearing aid at different speech and noise levels. Journal of the American Academy of Audiology, 5 99-109. Martin, F.N., Champlin, C.A., & Chambers, J.A. (1998). Seventh survey of audiometric practices in the United States. Journal of the American Academy of Audiology, 9(2) 95-104. Mason, D. & Popelka, G.R. (1986). Comparison of hearing aid gain using functional, coupler, and probe-tube measurements. Journal of Speech and Hearing Research, 29 218-226. Millin, J.P. (1975). Practical and Philosophical Considerations. In Pollack, M.C. (Ed.) Amplification for the Hearing-Impaired New York, Grune & Stratton, 115-144. Nilsson, P., Vesterager, V., Sibelle, P., Sieck, L., & Christensen (1997). A double-blind study of a non-linear hearing aid. Audiology, 36 325-338. Newman, C. W. & Weinstein, B. A. (1988). The hearing handicap inventory for the elderly as a measure of hearing aid benefit. Ear and Hearing, 9, 81-85.
Robert F. Zelski 29 Northern, J.L. (1992). Introduction to computerized probe-microphone real-ear measurements in hearing aid evaluation procedures. In Mueller, H.G., Hawkins, D.B., & Northern, J.L. (Eds.). Probe Microphone Measurements: Hearing Aid Selection and Assessment San Diego, Singular Publishing Group, Inc., 1-20. Ringdahl, A. & Leijon, A. (1984). The reliability of insertion gain measurements using probe microphones in the ear canal. Scandinavian Audiology, 13 173-178. Studebaker, G.A. (1982). Hearing aid selection: An overview. In Studebaker, G.A. & Bess, F.H. (Eds.). The Vanderbilt Hearing-Aid Report Upper Darby, Pa., Monographs in Contemporary Audiology. 147-155. Schow, R.L. & Gatehouse, S. (1990). Fundamental issues in self-assessment of hearing. Ear and Hearing, 11(5), Supplement 6S-16S. Stephens, S.D.G. (1980). Evaluating the problems of the hearing impaired. Audiology, 19 205-220. Surr, R.K., Cord, M.T., & Walden, B.E. (1997). Comparison of linear and K-amp circuits. Ear and Hearing, 18(2) 140-146. Tomlin, A., Thomas, K., James, A., & Dillon, H. (1995). The National Acoustic Laboratories Client Oriented Scale of Improvement (COSI). Audiology Circular, 1995/4 Chatsworth, NSW, Australian Hearing Services 1-8. Ventry, I. M. and Weinstein, B. A. (1982). The hearing handicap inventory for the elderly: A new tool. Ear and Hearing, 3, 128-134.
Robert F. Zelski 30 Tobin, H., Baquet, G.M., & Koslowski, J.A. (1997). Evaluation procedures. In Tobin, H. (Ed.) Practical Hearing Aid Selection and Fitting Baltimore, MD, Scientific and Technical Publications Section, Rehabilitation Research and Development Service, 95-103. Walden, B.E. (1982). Validating measures for hearing aid success. In Studebaker, G.A. & Bess, F.H. (Eds.). The Vanderbilt Hearing-Aid Report Upper Darby, Pa., Monographs in Contemporary Audiology. 188-192. Walden, B.E., Demorest, M., & Hepler, E. (1984). Self-report approach to assessing benefit derived from amplification. Journal of Speech and Hearing Research, 27 49-56. Weinstein, B.E. (1997). Outcome measures in the hearing aid fitting/selection process. Trends in Amplification, 2 (4)
Robert F. Zelski 31 Appendix A COSI Form
Robert F. Zelski 32 Appendix B Instructions for COSI Administration I. Introduction The COSI is a subjective hearing aid outcome measure that is administered by the audiologist in two phases. In the first phase the patient identifies listening situations that he/she would like to have improved with amplification. In the second phase, after the hearing aid(s) is/are fit, the change in hearing function for the identified listening situation is recorded. This change is noted descriptively among five choices ranging from Â“worseÂ” to Â“much better.Â” At this time the patient is also asked to note their final hearing ability. Once again the patient chooses among five options. For this selection the options range from Â“hardly everÂ” to Â“almost always.Â” The descriptive terms used to label the final hearing ability relate to a numerical equivalent that allows for quantifying the degree of benefit derived. The COSI also contains one other component. At the end of the first phase of COSI administration the audiologist categorizes each of the patient identified specific listening situations into one of sixteen general acoustic categories. II. COSI Administration (See COSI form in Appendix A). 1. Phase I: Identification of specific listening situations and categorization into general acoustic categories. a. Identification of specific listening situations. During the initial hearing aid selection visit, the audiologist will ask the patient to identify up to five specific listening situations in which he/she would like to hear better. The key word in this step is Â“specific.Â” The patient should be encouraged to be as specific as possible. For example, Â“wanting to hear better in a noisy environmentÂ” would not be sufficiently specific. Â“Wanting to hear better at the dinner tableÂ” much better, but even this should be further delineated. If hearing better at the dinner table is identified, the audiologist should clarify how many people typically are at the dinner table. Each of the identified listening situations should be recorded on the COSI form under the appropriate heading. If the patient identifies more than one specific listening situation the audiologist should ask the
Robert F. Zelski 33 patient to rank each situation in order of importance. The audiologist will note the importance in the appropriate box next to the description of the specific listening situation. The most important situation would be ranked as Â“1Â”, with the second most important ranked as Â“2Â”, and so on until each situation identified by the patient is so ordered. b. After the patient has identified from one to five specific listening situations, the audiologist should categorize each identified situation into one of the sixteen general acoustic categories listed on the COSI form. The category is noted in the box corresponding to the description of the specific listening situation. 2. Phase II. Assessment of improvement a. The audiologist should ask the patient to rate the degree of change in hearing ability for each specific listening situation identified in phase one. The patient choices are Â“worse,Â” Â“no difference,Â” Â“slightly better,Â” Â“better,Â” and Â“much better.Â” The audiologist will record the results in the appropriate box on the COSI form. b. Finally, the audiologist should ask the patient to rank their final ability to hear with the hearing aid(s) in each identified specific listening situation. The choices for this response are Â“hardly ever,Â” Â“occasionally,Â” Â“half the time,Â” Â“most of the time,Â” and Â“almost always.Â” If the patient prefers an numerical scale, each of these responses has a numerical equivalent on the COSI form. The audiologist also records these responses in the appropriate box on the COSI form. 3. Phase III. Submission of COSI form for this study a. The results obtained during this session should be used to assist in counseling the patient and/or in guiding the audiologist in making adjustments to the hearing aid fitting(s). Once the degree of change and final ability information are recorded the
Robert F. Zelski 34 clinical used of the COSI is complete. For the purposes of this study, one more action should be taken. Upon completion of the clinical use of the COSI, the audiologist should make a copy of the COSI form with the patient name and audiologist name obliterated. This unidentifiable copy should then be sent to the researchers at the following address: Robert F. Zelski, M.M.Sc. Dept. CSD 4202 East Fowler Avenue BEH 255 Tampa, FL. 33620-8150 b. If there are any questions, please contact Robert Zelski at (813) 974-9772 or at firstname.lastname@example.org
Robert F. Zelski 35 Appendix C COSI CLINICIANÂ’S QUESTIONNAIRE Part I (Please circle the number that most closely reflects your opinion) 1. CLINICAL USEFULNESS: Rate the overall usefulness of the COSI in the hearing aid fitting process. Very Somewhat Slightly Of No Useful Useful Useful Use ___________ ___________ ___________ ___________ 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 2. Administration Time: Assess the time required to administer the COSI relative to its clinical utility Very Little Average Too Much Excessive Time Time Time Time__ ___________ __________ ___________ ___________ 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 3. Comparison: As compared to other subjective outcome measures (such as the HHIE or APHAB) the COSI is: The Better than Worse than The Best Most Most Worst ___________ ___________ ___________ ___________ 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
Robert F. Zelski 36 COSI CLINICIANÂ’S QUESTIONNAIRE 4. Effectiveness of data collected: Rate the effectiveness of the data collected from the patients by using the COSI in refining and implementing the hearing aid selection and fitting process. Very Somewhat Not very Effective Effective Effective Useless ___________ ___________ ___________ ___________ 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 5. COSI format: The open response format makes the COSI: Very easy Somewhat easy Not very Very hard to use to use to use to use__ ___________ ____________ ___________ ___________ 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 COSI CLINICIANÂ’S QUESTIONNAIRE Part II 1. Have you ever used subjective hearing aid benefit outcome measures before? Yes No 2. What other subjective hearing aid benefit measures have you used? APHAB HHIE GHABP SHAPI Other___________ 3. Do you presently use any subjective hearing aid benefit measure(s) as a standard fitting tool? If so, which one(s)? Yes No
Robert F. Zelski 37 COSI CLINICIANÂ’S QUESTIONNAIRE Part II 4. a. Have you ever used the COSI before? Yes No b. If so, do you use it now? Yes No c. If you use it now, how regularly do you use it (prior to this study)? Routinely Frequently Occasionally Rarely 5. What other measures do you use routinely to validate the efficacy of hearing aid fittings? Functional Gain Real Ear Measures Word Recognition Speech in Noise Other______________ Other_________________ 6. How many audiologists dispense in this clinic? _____ 7. If there are multiple audiologists who dispense through this clinic, do all use the same dispensing protocol? Yes No 8. What type of VA clinic do you work in? Medical Center Hospital Full-Time Outpatient Clinic Part-Time Outpatient Clinic 9. How many years have you been practicing audiology? _________ 10. What your academic level (degree) Ph.D. Au.D. Other Doctorate Masters Comments:______________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
Robert F. Zelski 38 Appendix D Confusion Matrices D-1 Summary Confusion Matrix for all Observers 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 128 4 4 5 2 1 13 2 4 123 7 14 1 4 8 3 6 4 38 2 2 5 4 4 16 2 49 5 2 164 2 2 4 6 2 23 16 2 7 9 19 8 15 9 10 2 2 11 2 7 12 1 2 14 3 13 6 14 15 5 1 56 16 6 3 1 2 8 2 4 2 2 76
Robert F. Zelski 39 Appendix D Confusion Matrices D-2 Confusion Matrix for Observer 1 vs. Observer 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 45 2 1 1 3 2 1 41 3 4 3 3 3 2 13 1 4 2 7 1 15 5 55 2 2 6 9 1 7 7 4 8 5 9 10 2 11 2 12 1 1 4 2 13 2 14 15 1 20 16 3 1 1 1 2 2 2 23
Robert F. Zelski 40 Appendix D Confusion Matrices D-3 Confusion Matrix for Observer 1 vs. Observer 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 40 1 1 1 1 1 4 2 2 39 2 6 1 3 3 2 12 1 2 2 4 2 7 16 5 1 53 2 2 6 7 3 1 7 1 10 8 5 9 10 2 11 1 3 12 1 6 1 13 2 14 15 2 1 18 16 3 1 1 5 2 24
Robert F. Zelski 41 Appendix D Confusion Matrices D-4 Confusion Matrix for Observer 2 vs. Observer 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 43 1 2 3 1 6 2 1 43 2 4 4 2 3 1 2 12 1 2 4 2 1 18 5 1 56 6 2 7 12 1 7 1 5 8 5 9 10 11 2 12 4 13 2 14 15 2 18 16 1 1 2 2 29
Robert F. Zelski 42 Vita Robert F. Zelski RESUM ADDRESS; Robert F. Zelski, M.M.Sc. Department of Communication Sciences & Disorders University of South Florida 4202 East Fowler Avenue BEH255 Tampa, FL. 33620-8150 Work: (813)974-9772 Home: (813)558-8311 Fax: (813)974-0822 E-mail: email@example.com EDUCATION: -University of South Florida, Tampa, FL. 1999present Au.D. student -The University of Georgia, Athens, GA. 1993-1997 Ph.D. studies in Audiology -Emory University, Atlanta, GA. 1972-1974 M.M.Sc. Masters Degree program in Audiology -University of Pennsylvania, Philadelphia, PA. 1963-1967 B.S. Undergraduate studies at the Wharton School Of Economics Â– Major: Economics of Transportation and Public Utilities Honors: -DeanÂ’s List Â– University of Pennsylvania, 1966. -Scabbard & Blade Military Honor Society
Robert F. Zelski 43 ACCREDIDATIONS & LICENSES -Certificate of Clinical Competence Â– Audiology American Speech, Hearing, and Language Association -Fellow of the American Academy of Audiology -Board Certified Hearing Instrument Specialist International Hearing Aid Society -Certified training in implications & implementation of the American Disabilities Act Georgia Department of Human Resources -Certified training in cochlear implant evaluation & rehabilitation House Ear Institute -Licensed Audiologist Â– Florida MILITARY -ROTC: University of Pennsylvania -Commissioned: 2nd Lt., May 22, 1967 -Active Duty: December 1967 Â– June 1971 -Honorable Discharge with rank of Captain, June 5, 1971 -Duty Assignments: -Platoon Leader: Amored Cavalry Troop, W. Germany -Assistant Personnel Officer: Amored Cavalry Squadron, W. Germany -Base Defense Advisor: MACV, Ton Son Nhut Air Force Base, S. Vietnam -Headquarters Detachment Commander: Signal Corp Battalion, Texas -Finance Systems Officer (Computer Research), Texas -Honors: -CommandantÂ’s List (Honor Graduate): Armor Officers Basic Training, April 1968, Ft. Knox, KY. -Army Commendation Medal, May 1971, Ft. Hood, Texas
Robert F. Zelski 44 BUSINESS -August 1999: present: visiting clinical faculty, University of South Florida, Tampa, FL. -1997 Â– 1999: Clinical supervisor in Audiology, The University of Georgia, Athens, GA. -1975 Â– 1992: 50% owner. Vice-President & General Manager, Audiology and Hearing Aid Division, SEHAS, Inc., Atlanta, GA. (Division management duties, in addition to providing professional diagnostic & rehabilitative services, included management & oversight of audiology contracts with seven otolaryngologists; personnel management of 35 employees, to include nine audiologists and eight hearing instrument specialists. Corporate duties included marketing and financial management responsibilities). -1973 Â– 1975: Audiologist and Hearing Instrument Dispenser, SEHAS, Inc. Atlanta, GA. -1971 Â– 1973: ManufacturerÂ’s Sales Representative for Audiologic Equipment, SEHAS, Inc., Atlanta, GA. -Honors: -Audiologist of the Year, 1987: Audiologic Resource Association -Examiner for Georgia State Hearing Aid Licensure Board (1986 Â– 1992) -Beta site facility for Nicolet Aurora & Phoenix digital hearing aid system (only private practice facility selected) -Beta site facility for 3M Memory Mate programmable hearing aid system (only private practice facility selected) -Member: National Dealer Advisory Board Â– Gould batteries (1984 Â– 1989) -Member: National Dealer Advisory Board Â– Zenith Hearing Instrument Corporation (1974 Â– 1979). -Member: Professional Dispenser Advisory Board Â– Siemens Hearing Instrument Corporation (1983 & 1987) -Unofficial advisor to Georgia State Department of Health (to include ChildrenÂ’s
Robert F. Zelski 45 Medical Services & Vocational Rehabilitation Services) (1978 Â–1985). TEACHING EXPERIENCE -Undergraduate: -Introduction to Audiology Â– 1994, 1995 -Basic Principles of Audiometry Â– 1994 -Graduate: -Business and Practice Management Â– 1999, 2000 -Clinical Research Seminar (co-teach) Â– 1998, 1999 -Auditory Brainstem Response Laboratory Â– 1999 -Hearing Aid Laboratory Â– 1994 Â– 1996 -Principles of Amplification I Â– 1997 PUBLICATIONS & PRESENTATIONS -Five articles published in Hearing Instruments Magazine (1982 Â– 1990) -Two articles published in The Hearing Journal (1984 & 1991) -Invited speaker -Georgia Medical Association Â– 1990 -Academy of Dispensing Audiologists Â– 1989, 1992, 1996 -International Hearing Aid Society Â– 1988, 1989, 1992 -Georgia Speech, Hearing, & Language Association Â– 1990 -South Carolina Speech, Hearing & Language Association Â– 1985 -Georgia Hearing Aid Society Â– 1979, 1983, 1985, 1986, 1991 -Oklahoma Hearing Society Â– 1987 -Florida Academy of Audiology 2000 -Florida Silver Haired Legislature 2000 COMMUNITY SERVICE -Youth soccer coach (1977 Â– 1990), Decatur-Dekalb YMCA -Head soccer coach (1987 Â– 1989), Decatur-Dekalb YMCA -Adult soccer coach (1989 Â– 1992), Atlanta Area Amateur Soccer Association
Robert F. Zelski 46 -Executive Board Member (1988), Decatur-Dekalb YMCA -Druid Hills High School Soccer Boosters Club -Member, 1986 Â– 1992 -Treasurer, 1987 -President, 1988 & 1991 -Management Council for purchase & redevelopment of Venetion Community Pools of Druid Hills -Volunteer tutor for Athens Youth Tutorial Program: 1997 Â– 1999.