This Web site is a component of the SAMHSA Health Information Network

Enter Keywords Choose Website This Site Mental Health at HHS All Mental Health

Online Publications Order Publications National Library of Medicine National Academies Press Publications Homepage

printer friendly page e-mail this page bookmark this page shopping cart  current or new account

This Web site is a component of the SAMHSA Health Information Network.

1999 Annual Report to Congress on the Evaluation
of the Comprehensive Community Mental Health
Services for Children and Their Families Program

Home | Summary | Table of Contents | Figures | Tables | I | II | III | IV | V | VI | VII | VIII | IX | Appendix

II. SYSTEM-OF-CARE IMPLEMENTATION

INTRODUCTION

PURPOSE OF THE SYSTEM-OF-CARE ASSESSMENT

This component of the evaluation makes a unique contribution to the knowledge base about children's mental health service delivery systems. While there have been some attempts to understand and assess system development, these have been mostly focused on linkages among agencies within the system (Heflinger, 1996; Morrissey, 1992). Such studies have not captured features like family involvement and cultural competence, which are central to the system-of-care approach. Other efforts to assess systems along system-of-care principles have been limited to single site assessments, assessments of sites within a single State system, or were not conducted longitudinally (Dunham & Ringwood, 1999).

The system-of-care assessment has three primary goals. First, it provides a description of each CMHS-funded system to document how grantees have operationalized system-of-care principles. Second, it assesses each program's status annually in order to track system-of-care development over time. Finally, the system-of-care assessment examines the extent to which systems implement system-of-care principles.

DIVERSITY IN APPROACHES TO SYSTEM-OF-CARE IMPLEMENTATION

While all systems of care share the same goals of providing comprehensive, community-based, individualized, family-focused, and culturally competent services, specific strategies used to achieve these goals are based on each community's unique needs and resources. Community-specific environmental factors, including geography, sociopolitical structures, cultural influences, and economic resources, have significant effects on system development (The National Assembly of National Voluntary Health and Social Welfare Organizations, 1991).

Recognizing the variation in system-of-care approaches implemented by each grantee is critical to understanding how systems of care develop. The sites studied in this evaluation vary on several key characteristics that affect their implementation and operation:

• The catchment areas of the communities served by the grantees varied. Grants were awarded to ensure that systems of care operated in a variety of geographic configurations, including urban, small city or county, rural, and Native American settings. Some systems of care operated in small areas within larger communities.
• The management of the system varied. In many grantee sites, public agencies ran the system. In other sites, nonprofit agencies were responsible for implementing the grant.
• The history of approaches to serving children and the relationships among the core agencies in the communities in which systems of care operated varied. In some sites, the basic structure and service array existed for over a decade before the infusion of CMHS grant funds. In contrast, in other sites, very few public services existed before the grant to help children with serious emotional disorders and their families. Many sites built services on the foundation laid by earlier, related initiatives such as the Child and Adolescent Service System Program (CASSP). Further, specific legislative mandates have been shown to have major effects on system-of-care development (Lourie, 1994), and this was true for the grantee sites.
• The levels of pre-existing resources varied across the communities. Some communities had a strong base of existing services and resources to draw upon. In other communities a less well-developed foundation was available for building a system of care. Each of these situations created different benefits and challenges, and required building relationships among different partners.

These site-level differences provided an important context for understanding how sites focused their system development, how priorities were set, and how choices were made to pursue some aspects of system-of-care development over others. Taking into account these differences provides a context for assessing the relative value of any given initiative.

OVERVIEW OF THE PHASE I AND PHASE II ASSESSMENTS

This chapter presents findings from the Phase I and Phase II system-of-care assessments. The Phase I assessment was conducted among the 22 original grantees funded in 1993 and 1994. Site visits were conducted annually for 4 years corresponding with the second, third, fourth, and fifth years of grant funding (i.e., 1995, 1996, 1997, and 1998). The information reported for the Phase I grantees in this chapter was collected during the final site visit. The Phase II assessment was conducted among the nine grantees funded in 1997. The information presented for the Phase II assessment is based on the grantees' first site visits, which were conducted during the winter and spring of 1999, corresponding to the sites' second year of grant funding.

The Phase I and Phase II assessments entailed similar data collection approaches but were conducted with somewhat different instruments. The assessment and analysis approaches for each phase are described in more detail below. The assessment results differ in format and level of detail across the two phases because the Phase I assessment results are based on a content analysis of site visit reports, while the Phase II results are based on quantitative assessment data as well as a content analysis of reports. Further, because the information for the Phase I sites was based on the fourth site visit to each site, a greater depth of information about sites' implementation strategies was obtained.

PHASE I

Phase I system-of-care assessment data reported in this section were collected at 20 of the 22 grantees funded in 1993 and 1994.¹ Because some grantees included multiple sites, a total of 27 separate sites were assessed. Most of the information reported here was collected in the fifth year of funding, and this final year is the focus of the discussion.

¹The two missing grantees, East Baltimore and Stark County, are participating in the comparison study and were assessed using the revised Phase II tool. Also included in the comparison study was Santa Cruz, one of the California grantee sites. Findings from these three sites are reported in chapter IV with other comparison study data.

ASSESSMENT PROTOCOL AND DATA COLLECTION PROCEDURES

One task of the national evaluation was to assess the funded systems of care and document their implementation and development. The grant program provided minimum requirements, and the monograph by Stroul and Friedman (1986) offered guiding principles and broad expectations that no one "model" for implementing a system of care was prescribed. In addition, no standard assessment tool had yet been developed to measure the existence or extent of development of a system of care. To develop the assessment protocol and data collection tools, a review of the literature was conducted to identify the key features and functions of mental health services. Experts in the field of children's mental health services, including practitioners, researchers, and family members, were consulted throughout the development process. Finally, pilot tests were conducted to further refine the evaluation framework and assessment protocol.

Using traditional approaches to program evaluation, the protocol assessed two broad domains: infrastructure and service delivery. Infrastructure refers to organizational structures and processes that together create the framework for the local community system. Service delivery is the application of system-of-care principles to the direct provision of care to children and families. The program theory indicates that change is needed in both domains in order to build a comprehensive system of care. Each of these domains is comprised of attributes, defined as the key characteristics of systems of care (see Table 2 for a list of attributes). Each attribute can then be further subdivided into key elements specific to that attribute. These elements are identified in each attribute's definition. Definitions for each of the attributes were devised specifically for this assessment and represent an idealized version of the attribute. The extent to which the idealized form of each attribute was manifested in each community was expected to vary because of the flexibility grantees had in shaping their system of care to fit the unique needs and circumstances in their communities.

A variety of qualitative data collection tools was developed for the assessment, including semi-structured interviews, protocols for recording observations and for abstracting data from case records, and a standard review of specified site documents. For a given site, the bulk of the data were collected by two trained site visitors during a 3-day site visit. During the site visit, interviews were conducted with a wide variety of participants in the system of care, including the site director, other administrative staff, direct service staff, representatives of core child-serving agencies and family organizations, and caregivers of children served by the system.

Each respondent was asked his or her perspective on system functioning and operations in relation to domain attributes. For example, interview guides for case managers, agency administrators, families, site directors, and other respondents all contained questions regarding family involvement in the system of care to assess the degree to which the system of care embodied the "Family Focus" attribute. In addition, respondents were asked to describe efforts to sustain the system of care fiscally as Federal grant funding decreased, to identify system and program aspects they anticipated would last beyond grant funding as well as those that would be challenging to sustain, and to reflect on lessons learned in implementing their system.

ANALYSIS OF ASSESSMENT DATA

WITHIN SITE

To compile a complete picture of a given site, data were collected and analyzed from multiple sources, including respondent reports, observations of meetings and group processes, reviews of randomly selected case records, and reviews of documents. Findings were considered legitimate if they were reported across multiple categories of respondents (e.g., direct providers, administrators, families) or were reported across respondents within a single category of respondents and were also contained in documents or case records.

After each site visit, these findings were synthesized into a comprehensive report that was descriptive as well as evaluative. Each report followed a standardized outline that was based on the framework.

ACROSS SITES

In-depth analysis of site visit reports was performed to identify patterns and findings across sites for the fifth and last year of grant funding. Each attribute was divided into its key elements, and each site report was rated according to these elements. For example, to establish cross-site findings regarding case management, the following aspects were rated across all reports: agency auspices, type of staff responsible, levels of case management, caseload size, family involvement, key functions, and authority.

Five of the 16 attributes were selected for focused, cross-site analysis. These included two Infrastructure attributes (Interagency Structure and Service Array) and three Service Delivery attributes (Case Management, Family Focus, and Cultural Competence). These attributes generally represented the areas of greatest site activity throughout the grant period. Aspects of other closely related attributes were incorporated into the analysis of selected attributes where possible. For each of the five attributes analyzed, the following discussion of results addresses (a) cross-site patterns, (b) strengths, and (c) challenges. Also analyzed were respondents' perceptions of the ability of the sites to sustain themselves following grant funding, both fiscally and philosophically. Results summarize (a) financial approaches to sustainability, (b) sustainability of program and philosophical aspects of the system of care, (c) challenges to sustainability, and (d) lessons learned over the course of the 5-year grant.

Calculating exact percentages for some findings was difficult given varying levels of detail in the reports and particularly for findings that reflected respondents' impressions about the future of the system of care. Where possible, conservative estimates were indicated. Given these limitations, findings should be interpreted with caution.

SYSTEM-OF-CARE ASSESSMENT FINDINGS

INFRASTRUCTURE IMPLEMENTATION RESULTS

During the 5-year grant period, sites made significant progress in developing their infrastructures. Two infrastructure attributes have been selected to illustrate infrastructure implementation: interagency structure and service array. Interagency structures are often responsible for activity and growth among other related attributes as they are often the central mechanism for establishing goals, target population, and communication patterns, and are the forum for decisionmaking related to human resources, funding, case review, and use of evaluation data.

Interagency Structure

Cross-Site Patterns

The grant-funded sites assessed were governed by either public sector groups (23 sites) or private nonprofit organizations (4 sites). Three primary models categorized according to their primary functions and decisionmaking authority were identified. These models have been named Board of Directors (found primarily in sites governed by private nonprofit organizations), Communications and Networking Forum, and System Management Team (see Table 3). Most sites employ more than one model, so the models should not be considered mutually exclusive. The implementation of multiple aspects from different models offered both benefits and challenges to system management and development of the system of care, as described below.

Board of Directors. Found in only four sites (15 percent), this was the least commonly used model found among the CMHS grantees. Boards of Directors tended to be small, focused groups with direct oversight responsibility for the organization that held the grant. Most major decisions, including staffing, funding, training, target population, and service array, were made or approved by the board membership. Executive directors were typically charged with leading the implementation of the decisions and recommendations of the Boards.

In this model, families tended to be well represented, with most boards designating at least half of their membership slots to family members. However, Boards of Directors tended to have little to no representation from the core public child-serving agencies. In fact, they often had bylaws that specifically restricted the involvement of public agencies to avoid potential conflicts of interest. Because their organizations often entered into contractual relationships with public agencies to provide services, having public agencies on the Boards and involved in deliberations about contract awards and relationships would be viewed as giving an unfair advantage to the organizations. Perhaps because of this lack of interagency representation, of the four sites that were governed by Boards of Directors, three (75 percent) had an additional advisory group with interagency involvement. However, these additional groups typically acted more as an information exchange or advisory group with little decisionmaking power (as described in the Communication and Networking Forum model).

One challenge faced by the nonprofit organizations governed by a Board of Directors was that public agencies did not always see them as a true partner in broader system reform or initiatives. The core child-serving agencies were generally better established in the community, while the nonprofit organizations were often newly incorporated. As nonprofit organizations, it was difficult to gain the credibility, power, and authority necessary to influence the large public agencies and private service providers.

Communications and Networking Forum. The approach used by 23 of the 27 sites (85 percent) included interagency structures that fell under the Communications and Networking Forum model. These multi-agency groups existed to provide a forum for information sharing, networking, and communication. These groups were often large, with diverse representation from a variety of agencies, family organizations, or community groups. In fact, the highest level of participation by families and community organizations was observed in this model. The primary and most active participants, however, were from public-sector child-serving agencies.

These forums tended to have minimal power over the participating agencies and were generally unable to effect changes directly in agency operations or policy. Despite this, respondents valued the groups as an opportunity to share ideas, discuss challenges, and develop potential solutions. Frequently, members were responsible for communicating recommendations or proposals to higher authorities in their respective agencies. In a few sites, these groups were also responsible for implementing and operationalizing the decisions made by such an authority.

This type of group frequently acted in an advisory capacity to the agency that held the grant. Although the group rarely had any governance authority over the grant program, input and recommendations from members were sought and often incorporated into operations. Of the 23 sites that had Communications and Networking Forums, 10 also had another structure that had greater authority to govern the system of care (as described in the Board of Directors and System Management Team models).

System Management Team. Eleven of the 27 sites (41 percent) used a model that could be described as a System Management Team. These multi-agency groups generally had a defined and limited membership of public agency managers and administrators (e.g., county department heads, division directors) who were official designees empowered to make decisions (e.g., funding, strategic planning, operating procedures) on behalf of their agencies. Although some of the structures that fell under this model had oversight for the grant, their jurisdiction encompassed the entire community and, in some cases, all children's services regardless of specific target population.

Almost all of the sites that employed the System Management Team model also had an executive group comprised solely of the top agency administrators. The highest level of decisionmaking authority was present in these executive groups. Note, however, that not even these groups could make decisions that would, without agreement from that agency, profoundly alter the policies of a member agency. In other words, the teams had no mandating power.

Although family members and community representatives were present on over half of the System Management Teams, they tended to be present in much smaller numbers (typically only one designated slot). Moreover, family and community representatives were virtually never present on the smaller executive groups. Seven of the 11 sites that used the System Management Team model, however, also had another interagency body that had broader representation (as described in the Communication and Networking Forum model). In six of those sites, the additional body had a clear relationship with the System Management Team (e.g., the forum would pass ideas and recommendations to the system management team for decisionmaking).

Strengths

The primary advantages or strengths of each model varied widely. The Board of Directors model was characterized by active family involvement and localized decisionmaking authority. The Communications and Networking Forum model enjoyed broad community membership, and many groups had an open-door attendance policy whereby any interested community member was welcome to participate. The System Management Team model had the power and authority to accomplish higher system management functions. This was due in part to the membership—generally top-level leaders and executives of major child-serving agencies in the catchment area. In a few sites, these groups were considered the highest level of management for child health issues in the entire county, aside from elected officials. Sites with both a Communications and Networking Forum and a System Management Team seemed well poised to accomplish a variety of system management functions, through the involvement of key child-serving agencies, while incorporating diverse inputs from families and the community at large.

Challenges

The primary disadvantages or challenges of each model also varied. The main drawback to the Board of Directors model was lack of representation and involvement from the core child-serving agencies, which curtailed close collaboration. Relatedly, the nonprofit agencies had a limited purview and minimal ability to effect systemwide change. They were generally seen as providers of specialized niche services but unlikely as the center of community reform efforts. Communications and Networking Fora were limited in the functions they successfully executed. Although inherently valuable, the main purpose of these bodies was generally limited to information-sharing and informal communication. System Management Teams were unlikely to have significant family or community representation.

Although specific challenges were identified for each model, a few major patterns emerged across sites and models. Chief among these was concern regarding long-term sustainability. This included sustainability of the community service array (a key responsibility of system groups), of system reform efforts, and of the interagency body itself. In some sites, the interagency group underwent restructuring to redefine its mission, breadth of responsibility, or membership in hopes of ensuring the group's longevity. In sites where the mission was broadened to include all children with mental health needs rather than focusing only on those with serious emotional disorders, respondents felt the broader focus would make the group more useful to all participants.

Finally, family involvement in system management functions was a source of struggle and discord for sites. All sites but two had at least one family representative on a system-level group or committee, but family involvement at this level was still lacking. Although many sites were committed to involving families in significant decisionmaking at this level, few sites had established workable partnerships that both families and agencies felt were fair as well as effective.

Service Array

Over the course of the grant, sites dramatically expanded the service array by adding new services and by expanding the capacity of existing services. In a few sites, children's mental health services were severely limited before the grant. With the infusion of grant funds and ongoing efforts to expand the service array, these sites have been able to extend the range of services available to children and families. Although several services were required by the grant program, all sites made active efforts to expand upon this base, creating new services and tailoring others to meet the specific needs of their communities (see Table 4).

In the last year of grant funding, the majority of efforts related to the service array focused on filling remaining service gaps, addressing capacity issues, developing additional cross-agency methods of service delivery, and devising plans to sustain services fiscally once the grant ends. Many sites were actively working to diversify the funding sources for the system-of-care service array. Some communities submitted proposals jointly developed by multiple agencies to institute State-level waivers for community-based or "wraparound" services or to develop additional pilot projects. Several services were retooled to be Medicaid billable (e.g., family advocate staff taking on more care coordination functions).

Cross-Site Patterns

Although a set of nine required services is stipulated by the grant program, grantee sites varied considerably in how they implemented, expanded, and provided services. Ultimately, each site's service array was unique to that site and reflected the service needs, resources, and capabilities of the community. Given these local contexts, it was difficult to organize service arrays into defined models or groups. It was possible, however, to identify several cross-site patterns specific to service arrays that were also important to system-of-care implementation. These are detailed below.

Family Support and Paraprofessional Services. During the final year of funding, sites deeply appreciated and

relied on paraprofessional and support services (e.g., respite, behavioral aides, family support groups, parent aides, family advocacy, support). In almost all sites, these were the services most requested by families, and sites were very creative in finding ways to provide them. For example, many sites used community-based organizations as contract service providers. Frequently, family organizations (or a family team within the grant agency) were the primary providers of respite and family support services.

"Increasingly, professionals are recognizing the helpfulness of natural and informal supports." — Care Coordinator "Parents need parent aides more than they need therapy." — Agency Administrator

In addition, paraprofessional and family support service were those most frequently provided in home- or community-based settings (e.g., community centers). In some sites, these services were Medicaid reimbursable. This helped to ensure their long-term sustainability. In other sites, flexible funds or other non-categorical funds had to be used to pay for the services. Many respondents expressed concerns that after grant funding ended, maintaining these services without billing Medicaid would become more difficult. Despite increased recognition that such services are beneficial and family centered, their long-term future is uncertain.

Service Capacity. Although the grant was initially responsible for most improvements to the service array, sites also independently identified needed services and filled those service gaps with other grants or with State or Federal funds. Each site varied in the services most needed and prioritized its service expansion efforts accordingly. Areas in which sites commonly expanded capacity in the past year included youth-to-adult transition services, substance abuse treatment, paraprofessional services, early intervention and prevention services, and treatment for sexually abused children as well as for sexual offenders. Frequently, new providers (e.g., contractors) were identified to help expand capacity. Collaboration across multiple agencies was also used to ameliorate shortages and to expand the service array (e.g., through joint staffing and programming).

In 1998, sites provided the nine essential services supported and required by the grant initiative, with few exceptions. Seven sites (26 percent) were still working toward fully implementing therapeutic foster care, day treatment, transition to adult services, group homes, or respite care.

Despite the limited number of gaps in the service array, all 27 sites (100 percent) reported concerns about capacity, at least with some services. Nine sites (33 percent) maintained waiting lists for entrance into the project. Eighteen sites (67 percent) reported that demand for respite was extremely high and outstripped the service system's capacities to meet the demand. Nine sites (33 percent) reported insufficient capacity for transition-to-adult services, behavioral aides, short-term inpatient crisis care, substance use, day treatment, transportation, and after-school services (e.g., social skill building, recreation, mentoring, tutoring). Other capacity issues varied more widely by site. In many of the rural sites, psychiatric services (e.g., for assessments and medication monitoring) were often lacking. Specialized childcare services for children with serious emotional disorders were also mentioned as services needing expansion.

Concerns about continuity of care, infrequently mentioned in previous years, were more prevalent in the last year of grant funding. Specifically, sites recognized the need to develop new services or alter existing services to transition youth from out-of-home placement to intensive system-of-care services (e.g., "aftercare" or "step down" services). Equally important was the need to transition youth from intensive system-of-care services back to their homes, schools, and neighborhood providers. Staff recognized that ongoing monitoring and support were often necessary after leaving the system of care and that additional care coordination services should be developed to address this need. In sum, although sites expanded the capacity of many services, services for young adults, informal services, and flexible transition and coordination services were still lacking.

Multi-Agency Contributions to the Array. Eighteen sites (67 percent) actively used co-location, outstationing, or other forms of cross-agency staffing and programming to provide services. With this level of active collaboration, a wider range of services was generally available to children and families. Public-sector collaborative ventures among major child-serving agencies predominated. When the locus of the grant was a private, nonprofit organization, agency collaboration and joint programming was far less common. Generally, the service array available through the nonprofit organization was limited to the services it provided and those of direct contractors. Finally, 20 sites (74 percent) contracted with community-based organizations or local, private provider agencies to help deliver services such as counseling, behavioral aides, respite, or substance abuse treatment. Collaboration with schools continued, with more and more sites offering school-based services. Collaborative efforts to serve young children expanded beyond what had been accomplished in past years. Co-location, outstationing, and other forms of cross-agency staffing and programming are useful means both to provide services and to increase access to services.

Access to the Array of Services. Sites continued to address access to services by expanding transportation assistance options, hours of service delivery (e.g., offering services during evenings and weekends), and the range of service settings (e.g., home, school, community). The most frequently mentioned access problem was transportation, especially in rural areas. Services developed specifically through the grant program (i.e., intensive case management, family support services) were frequently offered in a range of times and settings. Traditional, clinical mental health services tended to be limited in the number of hours and settings offered. Services specifically delivered by other agencies (e.g., child welfare) were similarly limited. Some financial access barriers were cited by respondents. In nine sites (33 percent), respondents reported that families not eligible for Medicaid had difficulty accessing and paying for system-of-care services. For families who had health insurance, behavioral health benefits were often limited and some key system-of-care services were not covered.

Strengths

The range and diversity of services offered were the most notable strengths of the service array. Frequently cited by respondents and observed by site visitors, the expanded service array was also one of the most notable achievements of the grantee sites, across all areas of system-of-care development. Particularly laudable was the commitment to family support and informal services. Other key strengths related to services included a wide and diverse network of service providers that facilitated flexibility and family choice, the ability to individualize services (e.g., by matching service settings, hours of operation, or providers to families' needs and preferences), and collaboratively delivered services (e.g., through joint staffing or co-location).

Challenges

Despite enormous gains made during the 5-year period, a few challenges were identified in this last year of the

grants. Through increased collaboration and coordination, more and more eligible children were identified by sites. Although this was a highly positive pattern, the accompanying demand for services was difficult to satisfy. Overwhelmingly, the service array challenge most frequently cited was inadequate capacity. Family support and paraprofessional services were those most lacking in capacity. Finally, concerns regarding the ability to sustain the full array fiscally were mentioned frequently.

"Respite is the number one most useful service in stabilizing families, but there's not enough money to sustain it. We need to demonstrate to funders that it works." — Site Director

SERVICE DELIVERY IMPLEMENTATION RESULTS

Case Management

The case management process links families with services, ensures good communication among multiple providers, and supports families and helps them identify their particular strengths and existing resources. The general consensus in the field is that the case manager's role is comprised of a core set of functions: assessing child and family strengths, challenges, and needs; planning services to build on strengths and address identified needs; linking to and coordinating planned services; monitoring and evaluating service delivery to ensure continuity and appropriateness of services; and advocating on behalf of the child and family to ensure needed services and entitlements are received (Stroul, 1996; Early & Poertner, 1995; Levine & Flemming, 1986). Additional functions that are sometimes included in the case manager's role are outreach, clinical services, and financial management. The many different models of case management that are used in the field are distinguished by a range of factors such as the degree of emphasis on each of the core functions, the underlying ideology or philosophy, structural elements such as agency auspices and approaches to supervision, training and preparation of case management staff, status and authority of case managers, case load size, and intensity of services (i.e., the amount of time and resources devoted to each child and family). Many but not all of these factors were explored during the system-of-care assessment site visits.

Cross-Site Patterns

All sites had implemented some type of case management process that was consistent with the core functions generally comprising the case manager's role; however, approaches to case management varied across sites. The multiple dimensions or variables that distinguished sites' approaches to case management are described below.

Agency Auspices and Workplace Settings. In 19 sites (70 percent), case managers were employed or supervised by community mental health centers. In four sites (15 percent), case management was provided by a private nonprofit agency, and in the remaining four sites case management was provided by multiple child-serving agencies. Case managers worked in a variety of locations; many were outstationed in other agencies and, in several sites, working in the community and in-home settings was the norm.

Staff Responsible for Case Management. In 16 sites (59 percent), the case management role was a dedicated role with no other responsibilities; in three of these sites this role was filled by family members trained to perform the case management function in a paraprofessional capacity. One site even developed a certification process for the paraprofessional role. In 10 sites (37 percent), case management was provided by the primary therapist for some children (typically those with less intensive needs), while dedicated case managers were assigned to children and families with intensive needs or higher risk. Only one site relied solely on the primary therapist to provide case management.

Levels of Intensity. Seventeen sites (63 percent) reported different levels of case management services that depended on the level of need. These ranged from minimal risk or need case management (usually involving "as needed" service coordination provided by the primary therapist) to intensive case management for the highest risk children and families (involving frequent, typically at least once a week, in-home visits). The remaining 10 sites (37 percent) reported only one level of case management; however, many reported varying levels of intensity within each case manager's caseload depending on the individual needs of the family.

Caseload Size. Caseload size was not reported for all sites, but in the 17 sites (63 percent) that did report caseload size, 11 reported caseload sizes of 15 or fewer children and their families per case manager, which is consistent with optimal sizes cited in the literature (Stroul, 1996; Burns, Gwaltney, & Bishop, 1995). Four of these 11 sites reported larger caseload sizes for children with less-intensive needs. Among the other six sites, four reported caseload sizes that ranged between 16 and 40 children and their families, one reported caseloads of 50, and in one site, case managers often had caseloads of 100 or more—these case manager's roles were limited to planning and authorizing the initial service plan and did not include ongoing coordination.

Service Planning Processes. Three general approaches were used to accomplish service planning. The use of routine interagency or multidisciplinary team meetings for initial planning and follow-up was reported in 10 sites (37 percent). Planning conducted primarily by the case manager together with the family was reported in 14 sites (52 percent); in four of these sites, other agencies were involved as needed, while in the other 10 the involvement of other agencies was rare. In three sites (11 percent) initial planning was conducted by an interagency team, and follow-up and monitoring were conducted by the case manager and family.

Cross-site patterns were also summarized for two additional attributes that are closely linked to case management functions and processes: individualized service planning and communications.

Individualized Service Planning. An individualized service planning process assures that the unique and changing needs of each child and family can be addressed. In this process services are identified for and by children and their families and tailored to them on the basis of their unique strengths and needs. The service plan is created by a combination of inputs from families, youth, and multiple agencies and cross-disciplinary service providers; its content is detailed, with specific, achievable, strengths-based behavior and treatment goals. Typically, responsibility for assuring that service planning is individualized rests with case managers.

Virtually all sites adopted the practice of individualized service planning as a feature of their case management process that incorporated a range of inputs from families, children or youth, or providers from the multiple agencies with which they were involved. Although many sites reported efforts to assess child strengths, only a few sites adopted a comprehensive approach that gathered information across multiple life domains (e.g., residential, financial, vocational, health, leisure, time, social supports). In addition, few sites assessed family strengths. Virtually all sites documented their service plans; however, the level of detail in the plans varied considerably. Furthermore, there was little evidence in the plans of how strengths were specifically incorporated into service delivery strategies.

Communications. Although listed as an infrastructure attribute, good communication is an important characteristic of both domains that entails effective, efficient, systematic dissemination of information among and between the agencies, management, staff, service providers, and families involved in the system of care. Information sharing is accomplished through a myriad of informal and formal mechanisms including meetings, memoranda, e-mail, newsletters, and telephone calls.

At the service delivery level, the ability to coordinate services is dependent on good communication across the multiple providers involved with the child and family. For sites that used interagency team meetings to plan services, the meetings were identified as the primary formal communication mechanism across provider staff. Ad hoc, informal communication was the norm across most sites and was generally reported to be good. The individualized service plans that resulted from the service planning process were also potential mechanisms for communication (with family consent). Although the vast majority of sites had documented service plans, many did not contain sufficient detail, nor did sites typically have a single format or routinely share service plans among multiple providers or agencies.

Strengths

Challenges

Family Focus

Cross-Site Patterns

Role of Family Organizations and Teams. One or more independent nonprofit family organizations operated in 18 sites (67 percent). Frequently these organizations acted both as service contractors to the system of care as well as representatives to intra- and interagency committees and groups. Some of these organizations were chapters of State or national organizations, while others were locally developed grassroots organizations. Multiple family organizations were active in seven sites (26 percent). In these communities, organizations sometimes experienced difficulty collaborating, coordinating, and defining roles in relation to one another. For example, deciding which organizations should be represented on system-level committees was sometimes difficult. In seven sites (26 percent), the family organization was based primarily in another community or part of the State. As a result, close supervision and support of local advocates was difficult and close collaboration with the local service system was a challenge. Thirteen sites (48 percent) had identifiable "family" teams within the grant agency. These groups were often charged with a variety of functions: providing direct service (i.e., respite, support groups, care coordination), representing families on various committees or management groups, training other families or staff, providing outreach, and acting as mentors and guides to new families entering the system of care.

Nine sites (33 percent) had an independent family organization as well as a family team internal to the grant agency. This combination of approaches offered several advantages. Members of the independent organizations were better able to advocate openly for change without fear of losing their jobs or the organization losing its funding (when jobs or funding were not directly tied to the grant agency). Having an internal family team enabled clinical, administrative, and family team staff to work closely together on a day-to-day basis.

Services Provided By or To Families. Twenty sites (74 percent) employed families in some capacity (i.e., peer support, respite providers, data collectors). Some staff positions were associated with an intraagency family team, while others were associated with an external family organization. Most positions were contract or part-time positions, and full benefits were not always available. In almost all sites, families acted as advocates or mentors to provide peer support and guidance to families navigating the system of care. The roles and functions associated with this type of position varied widely. They included assisting with intake, supporting families before and during service planning, attending meetings with the family (i.e., IEP meetings, court hearings), acting as a sounding board for concerns, giving emotional support in times of difficulty, providing information about resources and services (i.e., by word-of-mouth as well as through resource libraries, handbooks, and other means), and providing training or workshops on the system of care or particular mental health issues. Other services provided to families included family therapy, parenting classes, recreation, basic needs purchased with flexible funds (e.g., rent, utilities), transportation, and outreach. Nine sites (33 percent) offered support groups on a regular basis. As mentioned previously in the discussion of service array, family support and paraprofessional services were highly valued by both families and service system staff. Sites reported that using family members to provide these services was especially successful when family members were supported through training and when their efforts were well coordinated with the work of other providers.

Although the final grant year saw families better represented in the grant agency, assisting with hiring, training, evaluation, and quality assurance, the role of family representatives rarely equaled that of agency staff. While family input was respected and actively solicited on some issues, families were rarely empowered to cast a vote for

"It's hard to integrate yourself into a system you have little control over." — Staff Member

significant decisionmaking. In nine sites (33 percent), families were not represented in executive bodies pertaining to system management (generally comprised of agency heads). These groups held the highest authority for making significant budget and policy decisions for the system of care. Since families were rarely represented, they had little role in governing the service system. Nonprofit organizations with a Board of Directors constituted the major exception: bylaws typically required 51 percent or higher family representation on the board, and families were considered part of the voting quorum.

Even when policies or bylaws specifically identified positions for family representatives on boards or management teams, 18 sites (67 percent) were actively struggling to recruit more families to fill designated slots. Most of these sites also recognized a need to provide better training and support to family representatives (e.g., through transportation, childcare, stipends, flexible meeting times, orientation to parliamentary procedures, clarification of representatives' roles). A few sites modified system operations (e.g., meeting times, training) of system groups to facilitate family involvement, but additional changes were required to make it feasible for families to participate regularly.

In sum, family involvement at the system level was highly variable. In some sites, the importance of family involvement was recognized across agencies. In other sites, both confusion and disagreement persisted regarding appropriate roles for family members in service system management. Many sites expressed frustration that few models of successful family partnership at a system level were known. As a result, they were forced to work out their own approach through trial and error.

Strengths

Other strengths varied more widely across sites. These included clear policies on system-level family involvement (e.g., a requirement that 50 percent of board representatives be family members), family representation on a variety of system-level committees (e.g., hiring, training, quality assurance, evaluation), and the presence of an established family organization.

Challenges

Cultural Competence

The system-of-care philosophy calls for systems to be responsive to the cultural, racial, and ethnic differences of the populations they serve. This requires that service providers recognize the cultural complexity and rich diversity of the people they serve, and have the capacity to respond to their unique service delivery needs (Benjamin & Isaacs-Shockley, 1996). The system-of-care assessment uses an expanded definition that includes in addition to race and ethnicity, characteristics such as religion, national origin, sexual orientation, education, and socioeconomic status to reflect how the diverse grantees view culture. All of these characteristics can shape how families define mental health concerns, whom they turn to for help, how they respond to services, and how they manage life cycle transitions. In addition, these same characteristics among administrative and provider staff shape how they respond to the children and families they serve. The following analysis focuses on sites' approaches to responding to racial and ethnic differences among their populations, rather than on the broader aspects of culture.

Cross-Site Patterns

Approaches to building culturally competent systems of care depend in part on the demographic composition of the population served. The diversity of the population served by the Phase I grantees underscores the importance of this attribute. The vast majority of grantee sites had some diversity in the populations they served; 19 sites (70 percent) served populations that included 25 percent or more people of color, and four (15 percent) had some diversity in their populations, with between 10 and 15 percent of their populations from a variety of racial or ethnic backgrounds. Four sites (15 percent) served an almost exclusively Caucasian population.

Grantee sites that served ethnically or racially diverse populations generally recognized the importance of having a diverse staff and bilingual staff when the diversity of their population entailed language differences. Of the 19 sites (70 percent) serving diverse populations, most reported some racial or ethnic diversity in provider staff. Some of these grantees also described efforts to ensure diversity in paraprofessional staff and contractor staff as well.

Benjamin and Isaacs-Shockley (1996), in their assessment of programs that were considered strong in the area of cultural competence, identified a list of cultural competence strategies which characterized these programs. Sixteen grantee sites (59 percent) reported using one or more of these strategies:

• Including cultural competence as an explicit goal and key priority of the system of care
• Creating specialized staff positions or task forces dedicated to the topic of cultural competence.
• Conducting targeted outreach to minority populations.
• Conducting assessments of community needs related to cultural issues.
• Conducting formal cultural assessments of client population.
• Considering cultural issues in developing the service array.
• Using creative approaches to recruiting and retaining culturally diverse or bilingual staff.
• Placing strong emphasis on training that incorporated cultural issues.

A combination of approaches—including training, staff recruitment, and other efforts—distinguished sites' levels of achievement in the area of cultural competence. The greater the variety of approaches and strategies applied, the more successful sites were in meeting their cultural competence goals. This is consistent with the common conceptualization of cultural competence as a continuum of stages that involves an ongoing learning and growing process for individuals and communities. Sites' efforts spanned a continuum from isolated, limited strategies at one end, to multiple, ongoing, dynamic approaches at the other. Along this continuum, the efforts of Phase I sites could be characterized as ranging from limited efforts (e.g., periodic training or some attempt toward staff diversity) to more extensive efforts that included multiple strategies such as routine training on various topics, concerted efforts to recruit and retain diverse staff, and targeted outreach to minority populations. Eleven sites (41 percent) reported minimal efforts to address cultural competence; these sites either provided some cultural competence training or attempted to have some diversity in their staffing, but efforts were limited to one strategy only. Nine sites (33 percent) gave somewhat more attention to the issue of cultural competence by ensuring that their staffing structure included staff who reflected the background of the population served, and by providing ongoing training or using another strategy to address cultural issues such as outreach to minority populations. Seven (26 percent) of the grantee sites put into place a range of strategies that reflected an ongoing, concerted effort to address the dynamic nature of cultural competence.

Strengths

The importance of building culturally competent systems of care was generally acknowledged by the grantees. Most grantees provided some level of training for staff. Even those sites that served an almost exclusively Caucasian population expressed an awareness of other cultures such as rural cultures or poverty. The majority of sites that served ethnically or racially diverse populations acknowledged the importance of having diverse service providers reflective of the population served, and made ongoing efforts to expand and diversify staff as the population they served evolved and changed.

Challenges

Despite these efforts, sites continued to face a number of challenges. Key among these was a lack of emphasis on or recognition of cultural competence as a priority in most sites, and the absence of a strategic plan or approach to assess needs and address issues of culture comprehensively. Other specific unmet needs raised across many sites included sufficient numbers of minority staff, especially bilingual staff, even among those that attempted a variety of approaches; outreach to distinct minority populations; and training that extends beyond an orientation to cultural competence to issues of implementing cultural competence in practice.

FUTURE OF THE SYSTEM OF CARE

Financial Approaches to Sustainability

In many communities across the country, Federal funding (e.g., Child and Adolescent Service System Program, Local Level Infrastructure Development Grant Program, Center for Mental Health Services Children's Initiative) has been largely responsible for the development and proliferation of the system-of-care approach. However, the intent of the CMHS initiative has always been to provide communities with financial and technical support upon which systems of care would continue to build and grow. In fact, the grant required recipients to develop mechanisms for matching Federal dollars that slowly declined over the 5-year funding period. Findings from system-of-care assessments indicated that grantees have used a variety of approaches to maintain and enhance the fiscal viability of their systems of care. Most sites used multiple sources. Table 5 summarizes the number of sites that reported using each source; approaches are described in more detail below.

Medicaid

Medicaid has always been a major source of funding for children's mental health services, but restrictions on which services can be provided, where services can be provided, and who can provide services has historically limited the array of mental health and support services available to children and families. Consequently, the flexibility and creativity encouraged and supported by CMHS funding was embraced in many sites as a way to augment traditional service delivery. Broader and flexible funding allowed sites to increase the number of children who could be served by providing support for the uninsured or underinsured, and by providing individualized and unique services to children and families that previously were not available in their communities.

As grant funding drew to a close, however, the vast majority of sites found themselves having to rely once again on Medicaid. In fact, 20 sites (74 percent) reported maximizing Medicaid as their key approach to sustaining financially their systems of care and their services. In 17 sites (63 percent) where Medicaid managed care was implemented or was planned for, most grantees were able to position themselves to serve as the primary behavioral health provider in the new system or the administrator of the managed care system. In other sites, the organization that held the grant became certified as a Medicaid provider or increased the capacity of existing Medicaid waiver programs.

While maximizing Medicaid may appear to be a fairly obvious solution to the sustainability of the grant communities, it has by no means been an easy one. The expanded service arrays developed with CMHS funding included many nontraditional services typically not reimbursable under Medicaid.

In order to sustain the broader arrays and shifts in service delivery, several grantees set out not only to maximize Medicaid, but to enhance it. As the behavioral health provider in a new Medicaid managed care system, or as the system administrator, or as a participant on a State-level advisory board or panel, 11 sites (41 percent) reported that they had achieved some success in bringing system-of-care values to their State's Medicaid program. Examples of their successes include the addition of respite as a State Medicaid reimbursable service, the certification of paraprofessional staff to serve as Medicaid providers of case management services, the incorporation of cultural competence values and staff training initiatives, and the inclusion of wraparound service delivery approaches. Some sites were able to negotiate a case rate for serving children among the target population. Such a capitated arrangement allowed the site to disperse funds flexibly, making it more likely that services would be sustained. When systems were strictly fee-for-service, reimbursement for meeting time or travel (e.g., to consumer's homes) was more likely to be cut or extremely limited. This restricted the provision of home-based services as well as key activities such as care coordination and collaborative service planning. As a result of grantees' efforts in this area, they have not only succeeded in obtaining financial sustainability for many important service components but have ushered in significant changes to their State's Medicaid system and have helped proliferate system-of-care principles and values.

EPSDT and CHIP

Funding from State Early, Periodic, Screening, Diagnostic, and Treatment (EPSDT) programs and emerging Child Health Insurance Programs (CHIP) was cited by eight grantees (30 percent) as possible sources of funding to sustain services. Several of these sites plan to use EPSDT and CHIP funding to provide services such as wraparound, home-based services, substance use treatment, and day treatment programs. Additional sites plan to use these funding sources to maintain or broaden coverage for previously unserved or under-served children.

State-Specific Resources

State-level resources generated from taxes on tobacco products, closings of hospitals or group homes, general sales taxes, special legislative appropriations, and mental health block grant funds were cited by 21 sites (78 percent) as a major financial resource for their sustainability efforts. Three sites described State-level funding that would continue to support the larger system of care. In these States, legislation and financial appropriations were specifically designed to support the development and provision of mental health service components and system development.

Local-level Support

Local resources were identified by five sites (19 percent) as potential sources for funds. These included county general revenue funds, community mental health reinvestment dollars, and county discretionary funds. In many cases, such monies were expected to be used to replace flexible funds previously available under the CMHS grant.

Support from Other Child-Serving Agencies

Three sites (11 percent) described having access to "pooled" or "blended" funding. Within this type of funding structure, each child-serving agency contributed funds to one pool of resources that was then used to provide needed services to children and families, frequently a well-defined subpopulation (e.g., children requiring or at high risk for out-of-home placement). Although State-level legislation pre-dating the grant often supported or required this pooling of funds, sites reported that having access to this central source of funding would help sustain some service options as well as system infrastructure components such as the interagency structure and stipends for family representatives.

Private Funding

Seven sites (26 percent) reported that they had already received or were planning to apply for funding from local and national private organizations to sustain services or aspects of their systems of care. Private, philanthropic organizations such as the Robert Wood Johnson Foundation and the Annie E. Casey Foundation, which had been historically involved in supporting the development of systems of care, were cited by some grantees as sources of continued fiscal support. Other grantees reported that they were receiving assistance from other organizations, such as the United Way, to support a range of activities and services such as a youth shelter, respite, or system planning efforts.

The Legacy of the CMHS Children's Initiative

Family Involvement in Service Planning and Delivery