 |
This Web site is a component of the SAMHSA Health Information Network. |
|
Evidence-Based Practices:
Shaping Mental Health Services Toward Recovery
Co-Occurring Disorders:
Integrated Dual Disorders Treatment
IDDT Fidelity Assessor Checklist
Before the Fidelity Site Visit:
Review the sample cover sheet.
This sheet is useful for organizing your fidelity assessment, identifying where the specific assessment was completed, along with general descriptive information about the site. You may need to tailor this sheet to your specific needs (e.g., unique data sources, purposes for the fidelity assessment).
Create a timeline for the fidelity assessment.
Fidelity assessments require careful coordination of efforts and good communication, particularly if there are multiple assessors. For instance, the timeline might include a note to make reminder calls to the program site to confirm interview dates and times.
Establish a contact person at the program.
You should have one key person who arranges your visit and communicates beforehand the purpose and scope of your assessment. Typically this will be the IDDT program director or coordinator. Exercise common courtesy in scheduling well in advance, respecting the competing time demands on clinicians, etc.
Identify program staff with whom you will need to meet during your fidelity visit.
Work with the program contact person to arrange a schedule of interviews for the day of your visit with case managers, substance abuse specialists, rehabilitation services providers (i.e., vocational staff, relevant PHP staff), therapists, psychiatrist or medication prescriber, etc. Again, scheduling your fidelity visit well in advance will more likely enable you to meet with all necessary staff members.
Establish a shared understanding with the site being assessed.
It is essential that the fidelity assessment team communicates to the programs the goals of the fidelity assessment. Assessors should also inform program staff about who will see the report, whether the program site will receive this information, and exactly what information will be provided. The most successful fidelity assessments are those in which there is a shared goal among the assessors and the program site to understand how the program is progressing according to evidence based principles. If administrators or line staff fear that they will lose funding or look bad if they don't score well, then the accuracy of the data may be compromised.
Indicate what you will need from respondents during your fidelity visit.
In addition to the purpose of the assessment, briefly describe what information you will need, who you will need to speak with, and how long each interview or visit will take to complete. The site visit is likely to go the most smoothly if the contact person could, where available, assemble the following information prior to your site visit:
- A copy of agency brochure
- A copy of IDDT Program Mission Statement
- Roster of IDDT staff (roles, FTEs)
- A copy of the substance use screening instrument used by the agency
- A copy of the standardized DD assessment instrument used by the program
- Total number of clients served by the agency
- Number of active clients receiving DD services
- Number of clients served in the previous year
- Number of clients who dropped out of the program in the previous year
- Number of active clients receiving specific DD services (e.g., substance abuse counseling, DD group counseling, family interventions)
- Number of active clients receiving additional rehabilitation services from the agency
- Number of active clients who attend a self-help group in the community
- Weekly schedule for counseling services
- Clinician training curriculum and schedule
- List of process and/or outcome variables
- Quality assurance data
Inform that you will need to observe at least one team meeting (or supervision meeting) and at least one group or counseling session during your visit.
This is an important factor in determining when you should schedule your assessment visit to the program.
Alert your contact person that you will need to sample 10 charts.
It is preferable from a time efficiency standpoint that the charts be drawn beforehand, using a random selection procedure. Obviously, a program can falsify the system by hand picking charts and/or updating them right before the visit. If there is a shared understanding that the goal is to better understand how a program is implementing services, this is less likely to occur.
During Your Fidelity Site Visit:
Tailor terminology used in the interview to the site.
For example, if the site uses the term consumer for client, use that term. If case managers are referred to as clinicians, use that terminology. Every agency has specific job titles for particular staff roles. By adopting the local terminology, the assessor will improve communication.
During the interview, record all the important names and numbers (e.g., numbers of clinicians, active clients, clients served in the preceding year, etc.)
If discrepancies between sources occur, query the program leader to get a better sense of the program's performance in a particular area.
The most common discrepancy is likely to occur when the interview with program leader gives a more idealistic picture of the program's functioning than do the chart and observational data. For example, on Item 5 (Outreach), the clinicians may report that they often spend their time working in the community, while the chart review may show that client contact takes place largely in the office. To understand and resolve this discrepancy, the assessor may go back to the clinicians and say something like, "Our chart review shows client contact is office-based the majority of the time. Since you had reported you often provided outreach services in the community, we wanted your help to understand the difference."
Before you leave, check for missing data.
After Your Fidelity Site Visit:
- The same day of the site visit, both assessors should independently rate the fidelity scale. Within 24 hours the assessors should then compare their ratings and resolve any disagreements. Come up with a consensus rating for each item.
- Sometimes assessors have collected different data or have interpreted the response differently during the interview. Within a week of the fidelity assessment (ideally, the next day or two), the fidelity assessors should follow up with contact to the program leader to clarify any item for which there is a lack of consensus. This is also the time to follow up on any missing data.
- Tally the item scores and determine which level of implementation was achieved (see Score Sheet).
TOC | Cover Sheet | Score Sheet
|
|
