This Web site is a component of the SAMHSA Health Information Network
|
This Web site is a component of the SAMHSA Health Information Network |
| | | | | | |  |
||||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This Web site is a component of the SAMHSA Health Information Network. |
|
Annual Report to Congress on the Evaluation of the Comprehensive Community Mental Health Services Program for Children and Their FamiliesChildren and Families: Descriptive and Outcome DataThe two overarching questions addressed by the national evaluation are:
A description of participants was needed to document the total number and characteristics of children and families receiving services in the demonstration projects supported by the Comprehensive Community Mental Health Services for Children and Their Families Program. Tracking outcomes was important to providing information about the effectiveness of these programs. The evaluation design was developed and pilot-tested during the first full year of funding of the 22 system-of-care sites (e.g., 1993–1994). During the second year of funding, group meetings were held with site evaluators to introduce them to the design, to train the evaluators on the administration of specific measures, and to provide technical assistance on coding and data transmission procedures. Assistance was also provided in the development and implementation of systems for following individuals enrolled in the outcome study and for collecting services and costs information from existing management information systems. A data collection and procedures manual (Macro International Inc., 1995) was developed and distributed to each site to assist with ongoing evaluation efforts. Lead evaluators at sites were responsible for training and monitoring evaluation team members. Regular follow-up contact between Macro International and site evaluators occurred through conference calls and evaluation workshops conducted at national grantee meetings. Methods for the Descriptive and Outcome Studies Descriptive Study Sample Sites collected descriptive data on all children receiving services through the system-of-care initiative, including demographic information, diagnostic status, functional characteristics, and service histories. Data from this component of the evaluation is referred to in the remainder of this report as the descriptive study sample. The original intent of this part of the evaluation was to collect information that was fully representative of all children and families who entered systems of care. To date (August 31, 1998), demographic and diagnostic information has been reported on 34,377 children who have entered systems of care. The intended target population for this initiative was children ages 5 to 18 with serious emotional disturbance. Between 15 and 20 percent of all children across the nation may be diagnosed with a mental disorder using the criteria specified by the psychiatric community in the Diagnostic and Statistical Manual for Mental Disorders, Fourth Edition (DSM-IV; APA, 1994). However, it is generally recognized that a smaller subgroup, 6 to 8 percent of all children, present with a serious emotional disturbance (Friedman et al., 1996) and display significant functional impairment, as demonstrated by a decreased ability to meet the challenges presented by day-to-day settings and interactions (in the school, the home, and the community). There was some variability across sites in how the target population was defined. The geographic areas in which the programs were situated varied substantially; some programs extended across an entire State or a large number of counties within a State, while others focused their efforts on single counties/cities or a circumscribed urban area. Additional variability in the definition of target populations occurred across sites as different subgroups of children were identified to receive care, and different agencies took the lead in the development of programs. For example, some sites had a strong focus on children who were referred through the juvenile justice system, while others were primarily oriented toward enhancing mental health-school liaison relationships. Target populations also changed significantly across time at individual sites. Services were in a constant state of development, with relationships between agencies continually cultivated and new service delivery options implemented as systems of care matured. All 22 sites contributed some data for the descriptive study sample, although sites varied in the number of families participating due to differences in the number served. The most complete information obtained was for child age, child gender, racial/ethnic status, and diagnostic status. Outcome Study Sample Outcomes were evaluated through a simple pretest/post-test replacement design that requested sites to evaluate more fully a selected group of children at intake, at 6 months, at 1 year, and annually thereafter for as long as they remained in services. When children and families exited the evaluation or were lost to follow-up, they were replaced with a new family who had just entered system-of-care services. Sites were also requested to collect outcomes data from children and families when they exited services. This design allowed for tracking of children and examination of participation outcomes, although it did not provide a controlled test of the system-of-care approach versus other approaches or no services at all. Child and family outcome data included the child’s clinical and social functioning, educational performance, restrictiveness of child’s placement, and child and family satisfaction collected with standardized instruments that are common to the field of children’s mental health services. The outcome evaluation design requested that grantees serving fewer than 400 children include all children and families who consented to participate. For those grantees serving between 400 and 4,000 children, a sample of 400 children was drawn. For those grantees serving over 4,000 children, a 10 percent sample was drawn. Sites were asked to report the specific reasons why children left services at exit. Data from this part of the evaluation is referred to in the remainder of this report as the outcome study sample. Of the 22 programs, 20 provided data for the outcome study sample. In most sites, however, enrollment was not based on a systematic random sampling scheme. The number of children in the outcome study as a percentage of the descriptive study sample varied across the sites (i.e., 40 to 95.2 percent). Only two sites failed to enroll 70 percent of the children in the descriptive sample. The average percentage of children enrolled in the outcome study sample was 79.5 percent (SD = 13.4%) of those in the descriptive study sample representing a relatively high rate of children and families who participated in both substudies. Some missing outcome data at intake can be attributed to families declining participation or insufficient support for data collection. Completion rates were calculated separately for the three major clinical outcome measures: the Child and Adolescent Functional Assessment Scale (CAFAS), the Child Behavior Checklist (CBCL), and the Youth Self-Report (YSR). Due to the replacement sampling design, data completion rates were based on a formula which limited the total number of potential respondents at each assessment interval to those children and families who could be reliably identified as continuing to participate in services rather than the total number of participants who were initially enrolled in the longitudinal outcome study. Data completion rates at 6 months averaged 64.5 percent (SD = 21.0%) for the CAFAS, 53.6 percent (SD = 20.7%) for the CBCL, and 46.4 percent (SD = 19.6%) for the YSR. Data completion rates at 1 year averaged 43.5 percent (SD = 25.5%) for the CAFAS, 36.5 percent (SD = 23.3%) for the CBCL, and 28.5 percent (SD = 22.1%) for the YSR. The generally low level and high variability in completion rates are due to a number of factors, many of which occurred across sites and many that are specific to individual sites. These include the use of a replacement sampling design, inconsistencies in implementation and use of tracking and tickler systems, and variable allocation of evaluation personnel at each site. Another factor to consider is that children and families may have exited systems of care before the 6-month and 1-year evaluation points due to either improvement or deterioration in their emotional disorders. Due to site variability in the collection of exit data, it is not possible to reliably determine whether improvement had occurred before exit for individual cases who left the program. Therefore, caution is recommended in interpreting the implications of the longitudinal outcome data. For reporting and analysis purposes, data was aggregated across sites for the outcome study sample. Extensive information at intake is based upon a sample of 6,668 children. Smaller samples were available for analyses for subsequent outcomes at 6 months (N = 3,195) and 1 year (N = 1,729). The outcome study sample differed from the descriptive study sample in the following ways: slightly more males participated in the outcome study at intake (66 percent), at 6 months (65 percent), and at 1 year (67 percent) than in the larger descriptive study sample (62 percent); the average age was lower and decreased across time from 11.6 years at intake and 6 months to 11.1 years at 1 year; a relatively higher proportion of African American children was involved in the outcome study at intake (21 percent), at 6 months (22 percent), and at 1 year (23 percent) than in the descriptive study sample (17 percent); there was a corresponding lower percentage of Hispanic children at intake (12 percent), at 6 months (10 percent), and at 1 year (7 percent) than in the descriptive study sample (24 percent). The sites that were included in the descriptive study sample but did not contribute data to the outcome study sample had higher rates of Hispanic children in their systems of care. Minimal differences existed in household composition and family income across the demographic study sample and three outcome study data collection points. In general, the demographic characteristics of the children and families who participated in the outcome study at different evaluation points were similar to the descriptive study sample despite lower numbers of children participating at the 6-month and 1-year follow-up data collection points. Analyses were also computed to compare intake scores on the major clinical measures for the subsamples of children who completed these measures again at 6 months and 1 year. The same average scores on the CAFAS at intake were obtained for children with complete 6-month (M = 62.6; SD = 27.5) and 1-year (M = 62.6; SD = 27.2) outcome data. These scores were slightly higher than the average for all children at intake (M = 61.7; SD = 27.8). A similar pattern was observed with the CBCL. Average scores on the CBCL at intake were nearly equivalent for children with complete 6-month (M = 68.3; SD = 10.5) and 1-year outcome data (M = 68.6; SD = 10.6). These scores were slightly higher than the average for all children at intake (M = 67.9; SD = 10.7). Clinical status at entry into the evaluation was not substantially different for those who completed the evaluation at 6 months and 1 year when compared to all children who completed intake clinical measures. The subsamples providing outcome data at 6 months and 1 year appear to be representative of the clinical status of all children as they enter system-of-care services. The following information in the remainder of this report regarding longitudinal outcomes should be interpreted with cautions. First, a control group was not included in the initial program evaluation design. To rectify this, evaluations are currently underway in three funded sites and their matched comparisons that will yield useful comparison information in the future. Second, completion rates for longitudinal data collection were lower than optimal across sites. This suggests that the generalizability of the current outcome analyses may be limited to a select group of children with serious emotional disorders who remain in services in systems of care. More complete evaluation information will provide better outcome data for determining the relative success of system-of-care interventions for children with serious emotional disorders. Demographic Profile The information presented in this section of the report is from the descriptive study sample data that was collected at intake into systems of care. Due to differences in the data that was collected across sites, the numbers of participants vary across individual items. For the most part, frequency distributions are reported, and the numbers of participants with complete data on each item was used to derive the percentages that are reported. Tables and figures contain notations to the numbers of participants included in each analysis. Table 3 presents the demographic characteristics of the children and families who participated in the descriptive study sample as of August 1998. Demographic characteristics of the children and families were consistent with proportions reported in the children’s mental health literature (Lefkowitz & Tesiny, 1984; Deaux, 1985; Eme, 1984; Farrington, 1987). The mean age of the children enrolled in this evaluation was 12. Nearly two-thirds of the sample were male; 53.5 percent were Caucasian; 17 percent were African American; and approximately 24 percent were Hispanic. The representation of Hispanic and African American families was higher than expected when contrasted to the general population of the United States. This is partially due to the geographic distribution of CMHS-funded system-of-care sites and partially due to providing system-of-care services within the public sector. It is important to note that 61 percent of the children’s families in the descriptive study sample reported incomes below the poverty guidelines for 1996 of the U.S. Department of Health and Human Services. Children who lived in single-parent families were more likely to experience poverty than children who lived in two-parent homes. This association is strongly reflected in the descriptive study sample. Of the children enrolled in the descriptive study, 49 percent were living in families maintained only by mothers, compared to a national average of 27 percent (U. S. Department of Health and Human Services, 1996). Of those children who lived in poverty, 34.4 percent lived in families maintained by mothers, as compared to 38.3 percent of those children who lived with both parents, and 7 percent who lived with their fathers (P2 = 399.126, df = 5, p < .001). Employment opportunities for single-parent households that are challenged by a child’s serious emotional disturbance are likely to be limited (Brennan & Poertner, 1997), and may extend the duration of poverty, adding to overall family stress. Table 4 illustrates the poverty levels for two-parent, mother-, and father-maintained households in the descriptive study sample. For all female-led, single-parent households in the United States, 32.6 percent were reported to live in poverty (DHHS, 1996). As expected, two-parent families reported higher income levels, and single-parent households are more economically disadvantaged. It is important to keep in mind that while the system-of-care approach may be an effective mental health service model, continued economic hardship for families may constrain optimal mental health outcomes.
Figure 4 illustrates the race-ethnicity of children who were referred for services by their parents or themselves (i.e., self-referral) and those who were referred by external sources such as schools and mental health agencies (i.e., external referral). There was a significant difference in the racial-ethnic distributions of the two groups. In the self-referral group, 72 percent of the children were Caucasian; in contrast, 52 percent of the children in the external referral group were Caucasian, 25 percent were African American, and 12 percent were Hispanic. Minority children were referred more often through external referral sources such as schools, mental health, social services and juvenile justice agencies. This finding is consistent with the literature showing that children and youth of color are more likely to be referred for service by external agencies than their Caucasian counterparts (Takeuchi, Bui, & Kim, 1993). Figure 5 illustrates the proportion of children referred for services by each source. Children entered the system of care through a variety of avenues. Schools provided the largest proportion of referrals (22 percent), closely followed by mental health agencies (21 percent), and social services agencies (17 percent). Eleven percent of the children were directed to the system of care through the courts and correctional institutions. Another 16 percent of the children were referred to system-of-care services by their parents or themselves. The remaining 13 percent specified some other referral source. The greatest variability across sites was found for referrals from schools and mental health agencies; sites specializing in school liaison relationships had higher rates of school based referrals. Mental health was the lead agency in most sites, but referrals from mental health decreased as interagency relationships and cross-agency programming increased. Referral sources changed over time (see Figure 6). In 1994, schools referred a greater proportion of the children, while mental health agencies have referred proportionately fewer. By 1997, schools were the major referral source. The proportion referred by social service agencies in 1994 was relatively high and may reflect start-up conditions of the study such as a limited number of sites and children served. Referral patterns are highly specific to each site, and the introduction of a new site or changing conditions at a particular site may drastically affect the year-to-year referral patterns described above. For example, as new programs develop and interagency relationships are solidified referral patterns may shift significantly across time. Diagnoses for Children Challenged With Serious Emotional Disturbance Diagnostic information was collected at each child’s entry into the system of care based on criteria from the Diagnostic and Statistical Manual for Mental Disorders, Fourth Edition (DSM-IV)(APA, 1994). Of the 29,945 children in the descriptive study sample who were assigned a primary diagnosis, 43 percent displayed a disruptive behavior disorder (29.8 percent conduct-related disorders, 13.5 percent attention deficit/hyperactivity disorder). Over 26 percent were diagnosed with depression and approximately 8 percent with an anxiety disorder. Approximately 6 percent were diagnosed with an adjustment disorder and fewer than 2 percent had a psychotic disorder. The remaining primary diagnostic categories (assigned to 15 percent of the sample) included, but were not limited to, substance abuse and eating, somatic, and speech disorders, as well as enuresis, encopresis, abuse/neglect, personality disorders, and learning disabilities (see Figure 7). More children with disruptive behavior disorders and fewer children with depression or anxiety entered systems of care compared to expectations derived from overall population prevalence rates. The number of children with multiple diagnoses was similar to other studies of children with serious emotional disturbance (Greenbaum et al., 1996). Thirty-four percent of the children in our sample (10,141 children) with a primary diagnosis also had a secondary diagnosis. Of these children, 19.2 percent had a secondary diagnosis of a substance abuse disorder, and another 18.2 percent of these children had a secondary diagnosis of a conduct-related disorder (see Figure 8). These findings are consistent with recent research among clinical samples which indicates that adolescents with behavioral and substance abuse disorders tend to have other psychiatric disorders that predate their problems with substances (Grilo, Becker, Fehon, Edell, McGlashan, 1996; Grilo, Walker, Becker, Edell, McGlashan, 1997; Wilens, Biederman, Abrantes, Spencer, 1997). Overall, comorbidity rates varied as a function of age for children receiving services in systems of care; comorbidity was lower for children ages 4 to 5 (18.3 percent) than for children 6 to 11 (32.1 percent), 12 to 15 (34.6 percent) and 16 or over (40.2 percent). The most frequently occurring secondary diagnosis was a substance abuse disorder, found for over 19 percent of children in the descriptive sample with a comorbid presentation at intake. The vast majority of children with a secondary diagnosis of a substance abuse disorder had either a primary diagnosis of a conduct-related disorder (52.8 percent) or depression/dysthymia (34.9 percent). Rates of secondary diagnoses of substance abuse disorders varied as a function of age and gender of the child. Of those children with comorbid presentations, less than 1 percent ages 6 to 11, 20.3 percent ages 12 to 15, and 38.4 percent age 16 or older had a secondary diagnosis of a substance abuse disorder (P2 = 1232.44, df = 3, p < .000). The increasing prevalence of diagnoses as a function of age is consistent with a recently released report on mental health and substance abuse in the general population (SAMHSA, 1999). Rates of secondary diagnoses of substance abuse disorders were similar for males (0.7 percent) and females (0.8 percent) at the 6 to 11 age range, lower for males (19.1 percent) than females (22.3 percent) at the 12 to 15 age range (P2 = 5.1, df = 1, p < .024), and higher for males (43.9 percent) than females (28.4 percent) who were age 16 or older (P2 = 54.64, df = 1, p < .000). Additional information regarding functional impairment related to substance abuse was also obtained for all children participating in the outcomes study through administration of the substance use scale of the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 1990). Severe impairment on this scale is indicative of the presence of behaviors reflecting abuse, dependence, or drug-seeking behaviors pervading lifestyle. Rates of severe impairment varied as a function of age and gender. For all children at intake completing the measure (N = 12,821), fewer than 1 percent ages 6 to 11, 11.3 percent ages 12 to 15, and 24.6 percent age 16 or older were rated at the severe level of impairment in the substance abuse area (P2 = 3175.74, df = 9, p < .000). Rates of severe impairment were similar for males (0.5 percent) and females (0.5 percent) at the 6 to 11 age range, higher for males (12.2 percent) than females (9.9 percent) at the 12 to 15 age range (P2 = 20.012, df = 3, p < .000), and higher for males (29.9 percent) than females (15.8 percent) who were age 16 or older (P2 = 89.712, df = 3, p < .000). Severe impairment rates for the children at intake into the outcomes study indicate that impairment from substance abuse is a significant issue that is not being adequately identified through formal diagnoses. Early intervention and prevention services for children at risk for substance abuse appear to be warranted for the population of children with serious emotional disturbance. Preventive approaches may need to be tailored by gender for children with serious emotional disturbance to address the differential rates of substance abuse disorders by gender during adolescence (SAMHSA, 1999). Specific treatment approaches for substance abuse disorders may need to be developed for females with serious emotional disturbance in early to mid-adolescence and males in late adolescence. Of children in the descriptive study sample, 13 percent did not receive a primary diagnosis, reflecting concerns regarding negative labeling as well as staff qualifications and training. Nationally, minority children were more likely than white children to have a diagnosis (P2 = 399.126, df = 5, p < .001). Asian/Pacific Islander children were less likely to have a diagnosis than other racial/ethnic groups. The probability of having a diagnosis was at least twice as high for children in families with incomes over $100,000, while the differences between the other income groups were generally less substantial (P2 = 46.324, df = 8, p < .00). Risk Factors for Serious Emotional Disturbance Certain child, family, and environmental factors are often associated with the development of serious emotional disturbance. Childhood exposure to violence, sexual abuse, physical abuse and neglect, and substance abuse are linked in the literature with emotional and psychological distress (McLoyd, 1991; Patterson, 1988). Family characteristics, such as a history of mental disorders, family violence, and felony convictions are also believed to be associated with child and adolescent emotional disturbance (Friedman et al., 1996). Risk factors in this evaluation were assessed independent of each other so that percentages do not total to 100 percent. Child Risk Factors Child risk factors include traumatic events that a child may have experienced and aspects of a child’s history that are indicative of heightened severity and risk. Of the children in the descriptive study sample, 59 percent were described by their caregivers as having experienced one or more of these factors including physical abuse, sexual abuse, psychiatric hospitalization, sexual abusiveness, suicide attempt, drug and alcohol abuse, and history of running away. Figure 9 depicts the proportion of children at intake, by risk factor. What is not depicted in the graph is the proportion of children who have experienced more than one risk factor—16 percent had two such factors, and another 19 percent had three or more child risk factors. Figure 10 illustrates the unduplicated count of children, by the number of child risk factors. Family Risk Factors Family risk factors include events occurring within families that may have a direct negative impact on child and adolescent mental health. Overall, 76.3 percent of the families reported the presence of one or more family risk factors, including family history of substance abuse, family history of violence, previous mental health concerns, previous psychiatric hospitalization, felony conviction, sibling history with foster care placement, and sibling history of institutionalization. Of these families, 19 percent identified one risk factor. As illustrated in Figure 11, family history of substance abuse, violence, and mental health concerns were among the most frequently reported risk factors. As with child risk factors, many families reported multiple risk factors—19 percent reported two family risk factors and another 38 percent reported three or more risk factors, indicating high rates of multiple family risk factors in children’s histories (see Figure 12). These high proportions of individual and multiple family risk factors reflect the highly stressed and burdened environments for families who are participating in services through systems of care. In summary, a high percentage of children receiving services in systems of care presented with disruptive behavior disorders, although a large proportion were depressed and/or anxious. Multiple individual and family risk factors characterize children’s histories prior to entering systems of care. These are clearly children with significant difficulties at the time of entry who are in need of integrated, community-based services. Functional and Behavioral Outcome Measures: Clinical Assessment Behavior, emotions and level of functioning were assessed at intake using the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 1990), the Child Behavior Checklist (CBCL) (Achenbach, 1991), and the Youth Self-Report (YSR, completed by children 11 years of age and older) (Achenbach and Edelbrock, 1987). Repeated assessments at 6 months and 1 year provided an opportunity to evaluate the changes in functional impairment, behavior and emotions for children in the system-of-care programs. It should be noted that analyses of changes across time are based on fewer children due to both completion of services and dropout. For all measures reported below, a higher score reflects greater impairment. Declining scores over time, then, represent positive change. The analyses of clinical change to 1 year focus on groups of children and families from the outcome study sample who had complete data at all three evaluation points: intake, 6 months, and 1 year. The changes that are reported reflect average group differences across time rather than individual change patterns. Measuring Social Functioning Functional impairment is an extremely important concept in the study of children’s mental health services. Diagnoses can vary due to a wide range of factors, and emotions and behavior may not truly reflect the extent to which a serious emotional disturbance affects a child’s life. Measuring functional impairment allows one to assess the extent to which functioning in multiple life areas is impaired. As importantly, changes in functioning across time as a result of participation in a system of care can be evaluated. The CAFAS was designed to assess the degree of a child’s or adolescent’s psychosocial functioning across five areas: (1) role performance in the community, school, and home; (2) thinking; (3) behavior towards others and self (i.e., self-harm); (4) moods and emotions; and (5) substance abuse. In most cases, the CAFAS was administered by a clinician who obtained information from multiple informants such as the child, caregivers, school, and official records. A few sites obtained the information by way of a structured interview with the caregiver. The CAFAS yields scores for each of the eight psychosocial subscales (school/work role performance, home role performance, community role performance, behavior toward others, moods/emotions, self-harmful behavior, substance use, thinking) and generates a total score indicating the child’s highest level of functioning during the previous 3 months. The higher the score, the greater the impairment in functioning. The total CAFAS score was used to assess children’s overall functioning. The percentage of children scoring below the clinical cutoff score of 40 was 13.4 percent at intake. This improved to 25.1 percent at 6 months and 27.3 percent at 1 year (see Figure 13). When comparing this population to other populations of children with mental health needs recently investigated, scores at intake are higher, indicating a more severely challenged group in the outcome study sample for the CMHS grantees. Improvement across the first 6 months was lower than reported for the Fort Bragg evaluation (Bickman et al., 1995; Bickman et al., 1997). The Fort Bragg evaluation investigated the effects of a continuum of care on military dependents and their families using a comparison group design with nearly 1,000 participants. The evaluation indicated that services delivered through a continuum of care did not produce significantly better clinical outcomes than regular CHAMPUS-funded services for military dependents (Bickman et al, 1995; 1997). Access to services was greater in the demonstration site with resulting increases in costs. The results of this study were limited to a military population that may not generalize well to children receiving mental health services within the public mental health system. In comparison to findings from the more recent Stark County, Ohio, randomized trial of managed mental health care services for children with serious emotional disturbance (Bickman et al., 1997), children in this sample were more functionally impaired at intake, but also displayed greater gains. This latter effort also supported the conclusion of no significant clinical and functional differences between groups served in systems of care or under treatment as usual, although the children enrolled in this trial may have been minimally functionally impaired, and the number of participants limited the power to detect significant differences. Intake and follow-up scores varied for different subscales of the CAFAS. Figure 14 illustrates the changes in the CAFAS role performance scores from intake to 6 months and to 1 year. The significant decrease in the children’s role performance scores indicated that, on average, children made significant improvement in their behavior at school, at home and in the community from intake to 6 months that remained stable at the 1-year assessment. Changes in functional impairment also varied according to different child characteristics. Figure 15 illustrates the difference in total CAFAS scores at intake, 6 months, and 1 year for male and female children. Although male and female children changed similarly across time, males displayed higher levels of functional impairment at each point in time. CAFAS scores also differed by age (see Figure 16). As expected, older children entered services with more severe challenges than younger children. The 6- to 11-year-old children made significantly more progress from intake to 6 months to 1 year than the other three age groups. The system-of-care model might have its greatest impact at the earliest ages, giving importance to the role of early intervention. The relationship between changes in educational performance and levels of functional impairment on the CAFAS were also examined. Children whose school performance improved between intake and 6 months made the greatest improvement in their total CAFAS score, as illustrated in Figure 17. Children who had no recent (previous 1 year) contacts with law enforcement were also reported to make significant improvement in their overall functioning. Figure 18 shows that, overall, children for whom law enforcement contacts decreased at 6 months made the greatest progress in their total CAFAS score from intake to 6 months to 1 year, as compared to children who reported that contacts with law enforcement stayed the same or increased. Of the children in the CAFAS analyses, 38 percent had multiple diagnoses. Although the rates of improvement were similar between children with a single diagnosis and those with any secondary diagnosis, the rate of improvement for children whose secondary diagnosis was substance abuse is significantly greater (see Figure 19). These children entered the system with more challenges, yet they displayed more improvement in the first 6 months and sustained these improvements to 1 year. Gehavioral and Emotional Outcome Measures The CBCL is the most widely used measure of children’s behavioral and emotional problems in the field of children’s mental health services. Information is collected directly from the primary caregiver concerning the child’s behavior during the previous 6 months. The CBCL is comprised of 118 problem behavior items categorized as internalizing or externalizing behaviors. Internalizing behaviors include sadness, depression, and anxiety. Externalizing behaviors include being oppositional, aggressive, and overactive. In addition, 20 social competence items are used to measure school performance, functioning at home, peer relationships, and social involvement. From these items, the CBCL scoring procedures generate standardized scores for syndrome subscales (i.e., withdrawal, anxious/depressed, somatic, social, attention, delinquent, thought, aggressive) and scale scores for social competence, internalizing behavior, externalizing behavior, and a total problems score. Higher scores indicate more problems. Standardized scores above 63 are considered in the clinical range (i.e., indicating emotional and behavioral problems above the 90th percentile in the population of children as a whole). Figure 20 displays the changes in children’s internalizing, externalizing, and total problem behaviors from intake to 6 months to 1 year. At intake, 57 percent of the children scored above the clinical range for internalizing behavior, 72 percent attained scores in the clinical range for externalizing behavior, and 77 percent of the children attained scores in the clinical range on the CBCL total problem scale. At 6 months, 49 percent of the children scored above the clinical range for internalizing behavior, 64 percent attained scores in the clinical range for externalizing behavior and 65 percent attained scores in the clinical range on the total problems scale. At 1 year, 43 percent of the children scored above the clinical range for internalizing behavior, 60 percent for externalizing behavior and 60 percent for the total problems score. Seventeen percent of the children with total problem scores in the clinical range at intake, had total problem scores in the normal range at the 1-year evaluation point. When comparing this population to other published information on populations of children with mental health needs participating in services, scores at intake are higher among the children served by the CMHS grants, indicating a more severely challenged population (Bickman et al., 1995; Bickman et al., 1997). Improvement across the first 6 months was lower than reported for the Fort Bragg evaluation (Bickman et al., 1995) and the more recent Stark County, Ohio, randomized trial of managed mental health care services for children with serious emotional disturbance (Bickman et al., 1997). Improvement across 12 months was also lower than for the Fort Bragg evaluation (Bickman et al., 1995). Rates of improvement on the CBCL, however, were greater than those reported for a sample of children with serious emotional disturbance who were followed longitudinally for 7 years (Greenbaum et al., 1996), participating in treatment as usual in the community, which consisted of services that were not integrated or coordinated across sectors. Social Competence Scores on the CBCL represent the child’s ability to function successfully in various social arenas, namely in sports or job activities, social organizations and at school. Lower scores represent less competence in these areas, and it is expected that these scores will increase as the child continues to receive services within the system of care. Figure 21 illustrates that while the activities score remained unchanged, the social and school scores improved over time. Table 5 provides comparisons between intake, 6-month, and 1-year CBCL syndrome scales. With the exception of somatic complaints, an index of physically based symptoms that may be related to psychological distress, intake scores were all over the clinical cutoff of 63, which means that the children scored above the 90th percentile for the standardization sample. On average, children made significant improvement in all problem areas. There was improvement in all scores, with the greatest occurring on the scores that were highest at intake: attention, delinquency, and aggressive problems. As previously discussed, disruptive behavior disorders accounted for 43 percent of the primary diagnoses at intake for the descriptive study sample. These diagnoses appear to be supported by the CBCL intake data; compared with the internalizing behavior scores, the externalizing subscales scores are the highest for children in the outcome study sample. Initial analyses revealed that child risk factors were associated with clinical range scores on the CBCL. Children with multiple child risk factors were more likely to have higher scores on the CBCL. Figure 22 illustrates this relationship. It also shows that despite the differences in their scores at intake, children’s mean scores for each risk group improved significantly from intake to 6 months to 1 year. Despite the presence of multiple child and family risk factors, children displayed improvement in behavioral and emotional symptoms across 1 year of services in systems of care, and this improvement in scores was most dramatic for children at greatest risk. Children with more family risk factors had similar improvement profiles (see Figure 23). The Youth Self-Report (YSR) is very similar to the CBCL with the exception that it is a self-report measure for children above age 11. The YSR provides youths’ perspectives on changes in their behavioral and emotional status as they participate in the system of care. Figure 24 presents the mean scores for the YSR total problems, internalizing behaviors, and externalizing behaviors across 1 year in the system of care. As expected, when asking youths to report on their own behavior and emotions, the average score on all three scales was below the clinical range. Adolescents perceived themselves as improving significantly across time on the three scores, which supports the previously parent-reported improvements displayed on the CBCL. Summary Children and adolescents displayed improved functioning and decreased behavioral and emotional symptoms across 1 year of receiving services in systems of care. These positive changes were accompanied by improved school performance and decreased law enforcement contacts. Differences in impairment levels at entry based on age, risk factor histories, and gender exerted some influence on change across time. However, the general trend on average was for all subgroups to display reductions in impairment and symptoms from intake to the 1-year assessment. Education School attendance and grade performance are two important indicators of adaptation to demands placed upon children by parents and other authority figures in the community. It is expected that both attendance and performance would improve significantly for children receiving services in systems of care. Analyses in this section are limited to the group of children and families for which complete data was available for all three evaluation points: intake, 6 months, and 1 year. The changes that are reported reflect average group differences across time rather than individual change patterns. At intake, 75 percent of the children in the outcome study sample attended school on a regular basis (76 to 100 percent of the time), while about 25 percent attended infrequently (1 to 75 percent of the time). Table 6 illustrates the change in school attendance from intake to 6 months to 1 year. There was a significant increase in the number of children who attended school greater than 75 percent of the time at 6 months and 1 year, as compared to those at intake. As expected, school attendance and school performance were related. Overall, children’s school performance improved from intake to 6 months to 1 year. At intake, 55 percent of the children made Average or Above Average school grades, and the percentage increased significantly to 63 percent at 6 months and to 67 percent at 1 year. Table 6 also indicates the relationship between school attendance and grades. Children Participating in Special Education At intake, slightly less than half (47 percent) of the children were in regular classrooms, 39 percent were in special education classes, and 2 percent were suspended or formally dropped out (see Figure 25). Thirty-four percent of the children were assisted with resource rooms and classroom aides; and 66 percent had no classroom aides (see Figure 26). Of the children who were in special education classrooms at intake, within 6 months of entering a system of care, 14 percent were transferred to regular classrooms with some form of special education support. Table 7 shows the changes in school performance for those children in special education classrooms at intake. Children in special education class at intake also significantly improved their school performance from intake to 6 months to 1 year. An Individualized Education Plan (IEP) is an important component in providing effective services through the schools to children with emotional disturbance. These plans clarify service provision and provide yearly goals for measuring progress in behavioral and emotional as well as academic areas. IEP’s facilitate interagency collaboration in meeting the needs of individual children and are consistent with the system-of-care philosophy. In the outcome study sample, children in grades one through three were less likely to have an Individualized Education Plan (IEP) than children in the higher grade levels. Children in grades three through six were more likely to have an IEP than any other group in the sample. Twenty-nine percent of the children had an IEP that reflected emotional disturbance. Sixteen percent of the sample had an IEP for challenges other than mental health concerns, and 40 percent of the children had no IEP. Five percent of the children’s IEP status was pending (see Figure 27). Analyses in this section include two different groups: a larger group with complete data at intake and 6 months, and a smaller group with complete data at intake and 1 year. The changes that are reported reflect average group differences across time rather than individual change patterns. Increased use of IEPs was found for children in the outcome evaluation over time. Fifty-one percent of these children had an IEP at intake, 58 percent at 6 months, and 60 percent at 1 year. Additionally, for the same children, 38 percent were receiving special education services at intake, 43 percent at 6 months, and 45 percent at 1 year. Participation in systems of care may help children to gain greater access to educational supports and interventions. This information can be compared to statistics available from the Office of Special Education Programs’ (OSEP) annual report to Congress for 1997 (OSEP, 1998). According to this report, approximately 11 percent of all children ages 6 through 17 enrolled in public schools were being served for disabilities in public schools during the 1995–1996 school year under the Individuals with Disabilities Education Act (IDEA). Only 8.6 percent of all children with disabilities in the school system, however, were being served under the disability category of emotional disturbance. Nearly 60 percent of children and adolescents participating in systems of care were receiving special education services through an Individualized Education Plan (IEP), although only approximately one-half of those children had an IEP specifically targeting emotional disturbance. Contacts With Law Enforcement Decreased involvement with the legal system is also an indicator of positive change for children and adolescents who received services through systems of care. As noted previously, 11 percent of children entering systems of care were referred from the courts and juvenile correctional system. As expected, 92 percent of the referrals from the juvenile justice and correctional systems were for adolescents above age 12. Parent reports on approximately 29 percent of all children entering systems of care indicated some contact with law enforcement during the 12 months prior to intake. Eleven percent of the sample had one or more adjudicated misdemeanors, 5 percent reported an adjudicated felony, and 23 percent had one or more arrests that led to convictions (see Figure 28). These data were reported for the 12 months before the children entered systems of care. Of these children, 55 percent had subsequent contacts from intake to 6 months. At 1 year, no law enforcement contacts were reported among 55 percent of the children with one or more contacts at the time of entry into services. Figure 29 illustrates the changes in contacts with law enforcement. Of the children who had not had any contact, 96 percent continued to have no new contacts at 6 months and at 1 year. For those children with a history of adjudicated misdemeanors who had both intake and follow-up data, 50 percent had no adjudicated misdemeanors at 6 months, and 55 percent had no adjudicated misdemeanors at 1 year. Of all children who entered systems of care without a prior history of convictions, 90 percent had no new convictions at 6 months, and 91 percent had none at 1 year. According to OJJDP’s Statistical Briefing Booklet, in 1997 law enforcement agencies made an estimated 2.8 million arrests of persons under the age of 18, of whom 32 percent were under the age of 15. There was also a 14 percent increase in total number of juvenile arrests from 1993 to 1997. The overall arrest rate for youth in 1997 was 9,200 per 100,000, or approximately 9 percent (OJJDP, 1998). As expected, the numbers of law enforcement contacts and arrests are higher for children entering systems of care than for the general population. Living Arrangements Analyses in this section include two different groups: a larger group with complete data at intake and 6 months, and a smaller group with complete data at intake and 1 year. The changes that are reported reflect average group differences across time rather than individual change patterns. One of the major principles of the system-of-care initiative is to provide services so that children can remain in their home and in their community. Stability of living arrangements is a reflection of how a system of care is achieving that principle. At intake, 63 percent of the children were reported to have lived in just one living arrangement over the last year; the other 37 percent were reported to have lived in two to eight different living arrangements. Figure 30 illustrates the changes in number of living arrangements at 6 months and 1 year for the group of children who lived in multiple living arrangements in the year before intake. Of the children who lived in multiple living arrangements at intake, 57 percent lived in just one living arrangement after they had been enrolled in the system-of-care services for 6 months, and this proportion rose to 60 percent at 1 year. Among the children who lived in just one living situation before intake, 92 percent remained in just one living arrangement at 6 months; this proportion was 94 percent at 1 year. Thirty-nine percent of the sample at both intake and 6 months were living with two biological parents; 45 percent lived with both parents at intake and 1 year. Twenty-nine percent of the sample were living with a single parent at intake; of these, 60 percent were still with a single parent at 6 months, 49 percent at 1 year. As expected, greater stability in living arrangement was found for children living with both parents compared to children living with a single parent. Satisfaction With Services After a child had been enrolled in the system of care for about 6 months, caregivers were asked their opinions of the services the child received, and whether they were satisfied with their child’s progress. Data for satisfaction analyses included two different groups of caregivers: a larger group with complete satisfaction data at 6 months, and a smaller group with complete satisfaction data at 1 year. The changes that are reported reflect average group differences across time rather than individual change patterns. In general, most of the caregivers were satisfied with the services their children had received, and they felt involved in their children’s treatment program. Three-quarters of the caregivers rated services as good or excellent at both 6 months and 1 year. Nearly 70 percent were very satisfied or satisfied with their child’s progress at 6 months and at 1 year (see Figure 31). These high rates of satisfaction among caregivers are consistent with what has been typically found in the children’s mental health literature. High satisfaction may be due to the system of care’s model of involving caregivers in service planning and treatment. The system-of-care philosophy calls for active engagement of caregivers in the decision-making process, solicitation of their ideas and opinions, and provision of unconditional care. Seventy-six percent of the caregivers indicated high levels of input to their children’s treatment planning at 6 months and 81 percent of the caregivers responding at 1 year also indicated high levels of input. Over three-quarters of the caregivers felt that they were always given a choice in decision making related to their children’s services. The same proportion of caregivers indicated that they were always asked for their ideas and opinions, working with case managers and other service providers to creatively select appropriate services. Further, an additional 20 percent of caregivers indicated that they were usually asked for opinions and generally involved in decision making. Another key indicator of family satisfaction was the high level of comfort caregivers felt with their involvement in the system of care. Most caregivers felt that opposing opinions or dissent would not result in negative repercussions for their children. Over three-quarters of the caregivers felt they would receive care unconditionally and believed that the quality of care would be high (see Figure 31). Families rated services at levels similar to ratings found in other evaluations of mental health services. Sites are doing well in areas promoted by the system-of-care approach, such as family involvement and unconditional care. Summary In summary, descriptive and outcome data from the national evaluation indicate the following:
|
|
|
| Home | Contact Us | About Us | Awards | Accessibility | Privacy and Disclaimer Statement | Site Map |
