Evidence-Based Practices: Shaping Mental Health Services Toward Recovery

Family Psychoeducation

Using General Organizational Index for Evidence-Based Practices

Overview

The General Organizational Index (GOI) measures a set of general operating characteristics of an organization hypothesized to be related to its overall capacity to implement and sustain any evidence-based practice. The items on the GOI were derived from clinical experience, although the research literature also supports the importance of many of these factors. The 6/26/02 draft version of this index contains 10 broad principles regarding elements such as program philosophy, training, supervision, and program monitoring. In future drafts, several items regarding cultural competency will be added. Whereas the fidelity scales are specific to each EBP, the GOI refers to operating characteristics that should be very similar across the EBPs.

The GOI is intended to be a companion assessment tool used at the same time as the EBP fidelity scale is administered. When conducting fidelity site visits, the implementation monitors should include GOI interview items (as outlined in the General Organizational Index Protocol).

The same set of 10 items is used for all 5 evidence-based practices (EBPs). One item—G2—has two alternate forms, G2A and G2B. G2A, for family psychoeducation, illness management and recovery, and supported employment, refers to information provision. G2B, for assertive community treatment and integrated dual disorders treatment, refers to screening. With the exception of item G2A/B, the wording of all the items is the same for all EBPs. However, in administering this index, the implementation monitor should tailor the language to fit with the specific practice.

Why measure general organization characteristics?

The rationale for the use of the GOI is similar to the one given for fidelity scales (See “Using Fidelity Scales”). Clinical experience suggests that agencies that generally do an excellent job in implementing a practice have the GOI elements in place within the organization. Programs scoring high on the GOI are expected to be more effective in implementing an EBP and in achieving desired outcomes.

We also recommend that agencies implementing an EBP use the GOI as a self-assessment tool for monitoring programs over the course of their development (and even after they are fully established). Considerable experience by implementers has suggested that routine use of such indices provides an objective, structured way to give feedback about program development.

How is the GOI used?

The assessment philosophy for the GOI mirrors that for fidelity scales. The GOI contains simple-to-understand face-valid items that are rated on a 5-point response format, ranging from 1 equals no implementation to 5 equals full implementation, with intermediate numbers representing progressively greater degrees of implementation. The response alternatives are behaviorally anchored, that is, they identify concrete measurable elements of the practice. Our experience is that independent evaluators using multiple sources of information make the most valid ratings. Typical sources of information include interviews with staff, observation of team meetings, review of charts, and observation of interventions. Although we recommend outside raters, the GOI can also be used by program managers to conduct self-ratings. The validity of self-ratings (or any ratings, for that matter) depends on the knowledge of the person making the ratings, access to accurate information pertaining to the ratings, and the objectivity of the ratings. We encourage the use of self-ratings, with appropriate caveats regarding potential biases that can be introduced by raters who are invested in seeing a program “look good” or who do not fully understand the principles of the General Organizational Index. In addition to the scales developed for independent evaluators and program managers, companion fidelity measures intended for consumers and family members are under development for some EBPs.

Graphing GOI

We recommend that programs implementing an EBP graph their GOI over time. See the section on fidelity scales for a related example.

This document is part of an evidence-based practice implementation resource kit developed through a contract (no. 280-00-8049) from the Substance Abuse and Mental Health Services Administration's (SAMHSA) Center for Mental Health Services (CMHS) and a grant from The Robert Wood Johnson Foundation (RWJF), and support from the West Family Foundation. These materials are in draft form for use in a pilot study. No one may reproduce, reprint, or distribute this publication for a fee without specific authorization from SAMHSA.

Back to Family Psychoeducation