Remarks by
A. Kathryn Power, M.Ed.
Director
Center for Mental Health Services
Substance Abuse and Mental Health Services Administration
U.S. Department of Health and Human Services
Measure Twice, Cut Once: The Use of Data in Mental Health Transformation
February 13, 2008
Arlington, VA
PowerPoint Version
Attached is the text prepared for delivery; however, some material may have been added or omitted at the time of delivery.
[Slide 1: Title slide]
Thank you, Joyce [Berry, or John Morrow], for your kind introduction and your invitation to join you this afternoon. Those of you who know me know that I’m a big picture person. I’m deeply committed to transforming our mental health service system at the Federal, State, and local levels, and certainly transformation is a big picture concept.
Indeed, transformation implies profound change—not at the margins of the system, but at its very core. It is about new values, new attitudes, and new beliefs that are expressed in new behaviors of people and institutions. We have embarked on an ambitious agenda to dispel the antiquated myths, replace the outdated science, and transcend the outmoded financing of current mental health care so that adults and children in this country with mental health problems can live, work, learn, and participate fully in their communities. This is a tall order, indeed.
However, big picture thinkers need detail oriented people who can tell them whether they are on the right track. In this case, we cannot adequately define or describe transformation or measure its impact without the data you collect. The renowned physicist Lord Kelvin has said, “When you cannot measure, when you cannot express in numbers, your knowledge is of a meager and unsatisfactory kind.”
We need knowledge about whether our efforts are making a difference in the lives of the people we serve because the old ways of doing business are no longer acceptable. We know that mental illnesses are as treatable as most physical illnesses, and —given the right combination of treatment and support and a voice in decisions concerning their care—people with mental and substance use disorders can and do recover.
We know, too, that we can no longer afford to wait until people become sick before we treat them. Instead, we must embrace a public health approach, one that focuses on promoting health and preventing disease before it occurs, as the cornerstone of a transformed system of care.
Finally, we are keenly aware of the fact that, as Hippocrates reminds us, the mind and the body are one. Physical and mental health conditions impact one another and must be addressed holistically rather than as separate and distinct problems. At a SAMHSA-sponsored dialogue between consumers and primary care representatives, one participant stated this succinctly when she noted, “I am a whole person and all my parts come together.”
But we can’t do any of this without the data to show whether or not we’re being successful. I’m certain you know the adage “measure twice and cut once.” Just as we wouldn’t dream of building our house without carefully measuring the walls that will hold it up, so too do we need to measure our efforts at providing the type and range of services that constitute the very foundation of a transformed system of care.
Of course, measuring such concepts as recovery or social connectedness isn’t the same as measuring the length of a 2 by 4. This is why the data you collect are critically important both at the State and Federal level. As one of my colleagues recently said, “When we can measure transformation, I can understand it.” I’m struck by how true this is. When you collect data on who your State Mental Health Authority serves, where they live, the types of services they receive, and whether the care they receive meets their needs, you are helping us define what it means to create a consumer-driven, recovery-focused, evidence-based system of care.
In the time I have with you today, I’d like to highlight three important topics:
- First, I’d like to point to the wonderful progress you’ve made in reporting URS and NOMs data.
- Second, I’d like to talk about how we can use these data in decision support and policy development at the State and Federal level.
- And third, I’d like to look ahead to our data needs and strategies as we move forward.
Progress in Reporting URS and NOMs Data
[Slide 2: 6 years of data collection]
I’m so pleased to be able to congratulate you on the tremendous progress you have made over the past 6 ½ years in collecting and reporting the URS measures. These data not only allow us to address accountability in public spending, but they also serve as a barometer for the lives of adults and children with mental health problems.
In each year of URS reporting, more States have submitted data and the level of data submitted by States has also improved. For example, in the first year of reporting, 39 States reported on employment status for enrolled clients; in 2007, 55 States provided these data. Reporting on adult perception of care has jumped from 37 States to 52 States, and data on child and family perception of care reported by 22 States in year 1 were available from 47 States in the most recent year. Other URS measures have seen similar large increases in reporting over time.
Overall, this reporting effort demonstrates that, as of 2006, State public mental health systems are serving 6 million men, women, and children each year. These individuals are more often unemployed, receiving Medicaid assistance, and served in community mental health settings. They generally rate their access, appropriateness, and outcomes of services as positive. State mental health agencies expended $30 billion to provide these mental health services.
But these data only tell part of the story. What’s equally exciting is the fact that you’ve begun to report new measures, including social connectedness, level of functioning, criminal justice involvement, and school attendance. Some States are also beginning to report employment by diagnosis. And these data feed into the 10 mental health National Outcome Measures that help us quantify meaningful, real-life outcomes for adults with serious mental illnesses and children with serious emotional disturbances and their families.
[Slide 3: NOMS]
The mental health NOMS include measures that depict how well consumers are managing their illnesses and living and working in the community, with a focus on recovery and resiliency-oriented measures. These include:
- Improved functioning for individuals receiving mental health services;
- Obtaining and keeping a job or enrolling and staying in school;
- Decreased involvement with the criminal justice system;
- Securing a safe, decent, and stable place to live;
- And having social connectedness to and support from others in the community such as family, friends, co-workers, and classmates.
Two other measures directly address the treatment process itself, which we calculate in terms of increased access to services for mental health and decreased inpatient re-hospitalizations for mental health treatment.
The final three measures examine the quality of services provided. They are client perception of care, cost-effectiveness, and use of evidence-based practices in treatment.
Data in these critical areas tell us how we are doing at transforming the mental health treatment system and where we need to improve. The NOMs are an outgrowth of a collaborative process between SAMHSA, the States, and NASMHPD, and are designed to provide the data we need for decision support and policy development at the State and Federal level.
Using Data in Decision Support and Policy Development
Unlike business, the public sector does not have profit as a measure of performance. However, there is a very real need to be able to measure how effectively we provide mental health services to those who seek them. Like business, we can tie performance goals to specific outcomes in terms of growth, costs, quality, and customer satisfaction.
Data mean little, however, unless they are implemented and used for planning and decision making. At the State level, we encourage you to use the data you collect for self-assessment and policymaking. Many of you are doing just that by integrating the NOMs measures into the day-to-day operations of your mental health systems. For example:
- Alabama’s Planning Council uses the NOMS for evaluation of planned goals, strategic planning, and the budgeting process.
- Connecticut uses the NOMS for planning, monitoring, and service quality reviews.
- Guam uses the NOMS for program evaluation.
- Illinois is working with its Planning Council to use some of the data for advocacy and planning.
- Indiana uses the NOMS as part of performance-based contracting with providers to guide resource allocation.
- South Dakota uses NOMS for planning the Mental Health Block Grant and developing goals and objectives for the community mental health system.
- And Washington uses the NOMS in conjunction with the implementation of the State’s transformation grant.
I’m certain that each of you here today is making a similar use of the data you collect. This is transformation in action!
Be sure to stop by the poster session and reception following the meeting to learn about some specific ways in which your colleagues are using URS and other performance data.
At the Federal level, the URS and NOMs data provide a way for us to track progress and trends across all States and territories. Doing so helps us understand where to focus our technical assistance efforts. Let’s examine what some of the 2006 data reveal.
[Slide 4: Increased access to services]
NOM 1 measures increased access to services, and clearly we’ve made some progress. From 2005 to 2006, States served an additional 100,000 consumers. However, we’re still only reaching a quarter of the population that has serious mental illnesses or serious emotional disturbances. We know that fear, misunderstanding, and discrimination can keep adults and children from receiving the care they need and deserve.
[Slide 5: Mental Health Campaign]
To help send the important messages that mental health problems are nothing to be ashamed of and treatment is effective, SAMHSA has launched the Campaign for Mental Health Recovery. Aimed at young people ages 18 to 25, the campaign theme is “What a Difference a Friend Makes.” Our goal is to encourage young people to support their friends who are experiencing mental health problems.
Data tell us this is an important group to reach. Results of SAMHSA’s annual National Survey on Drug Use and Health reveal that the prevalence of serious psychological distress is high among young adults ages 18 to 25, yet this age group shows the lowest rate of help-seeking behaviors. Further, findings from the National Comorbidity Survey indicate that half of all mental illnesses in both men and women occur before age 14 and three-fourths by the age of 24.
I would encourage you to visit the Campaign’s Web site at www.whatadifference.samhsa.gov, where you can view television spots and hear radio ads that send a powerful message of hope and recovery.
[Slide 6: Employment]
We know that employment is critical to a person’s recovery; it helps increase self-esteem and social connectedness and provides a valuable source of income. In 2006, the URS data tell us that 22 percent of individuals with mental illnesses are competitively employed, but nearly half of all consumers are not in the labor force. Though many individuals with serious mental illnesses have significant barriers to employment, we know that supported employment helps individuals overcome these barriers and be successful at work. SAMHSA has published a supported employment toolkit, and we would hope to see an increase in the number of individuals who are employed as more States adopt this evidence-based practice.
[Slide 7: Dual diagnosis]
We also know that substance use disorders, alone or in combination with mental illnesses, may be an impediment to work for many of the individuals we serve. The 2006 URS data indicate that nearly a quarter of adult consumers—and, on average, 6 percent of children—have co-occurring disorders. Though we don’t understand all of the factors that precipitate co-occurring disorders in children, we do know that those with certain mental disorders, in particular, are vulnerable to the development of substance abuse problems. This suggests an important window of opportunity in which we can act to prevent impaired functioning, disability, and other negative consequences of co-occurring disorders for our youth.
For adults who have co-occurring disorders, we know that Integrated Dual Diagnosis Treatment is an evidence-based practice that helps people recover by offering both mental health and substance abuse services at the same time and in one setting. SAMHSA also has a toolkit available on this practice, which a number of States have begun to implement.
Finally, we are now developing prevalence data on depression in 40 States through the DIG Supplements and the SAMHSA/CDC Intra-agency Agreement. This will inform us not only about the prevalence of depression in these States, but also about the association of depression to behavioral risk factors and chronic health problems. Already, the data reveal significant associations between depression and such chronic diseases as heart disease, diabetes, and asthma. These data are critically important to our understanding of what it means to create a public health, integrated approach to the treatment of both behavioral health and medical conditions. And your ability to gather this information responds directly to the first goal of the New Freedom Commission on Mental Health, which acknowledges the importance of helping Americans understand that mental health is essential to overall health.
This is just a small sampling of what the data you collect tell us about the individuals you serve and the services they receive. Taken together they provide us with a rich source of information about the needs of consumers and where our efforts should best be addressed.
I’d like to take a few minutes to highlight another source of data we’re collecting that will address two outcomes that are not measured directly by URS or NOMs but are vital to understanding whether the changes we make in our service systems lead to improvements in the lives of adults and children with mental health problems and their families. These outcomes are the process of infrastructure change and recovery as consumers define it.
[Slide 8: T-SIG States]
Both of these outcomes are being measured as part of the evaluation of the Mental Health Transformation State Incentive Grants or T-SIGs. Nine States were awarded these grants, which are unique in that they are supporting new and expanded planning and development to promote transformation of systems explicitly designed to foster recovery and meet the multiple needs of consumers.
We are beginning to collect data that will help us determine whether infrastructure changes lead to service changes and whether service changes lead eventually to client outcome changes, though we may not see these changes in client outcomes during the life of the grants. Our evaluation will also document factors that contribute to successful transformation in order to inform current and future transformation efforts of other States and SAMHSA.
[Slide 9: Infrastructure indicators]
In order to realize our evaluation goals, we had to develop a way to measure system change, which has never been done in a systematic way before. The T-SIG States are, in essence, the guinea pigs for a new set of GPRA infrastructure indicators. These 7 measures [on slide] address those areas that we believe indicate the type of system-level changes needed to transform the delivery of mental health services to adults and to children and their families.
The 7 indicators are tied to the New Freedom Commission’s 6 goals and must reflect the Comprehensive Mental Health Plan the State develops. States set targets they plan to meet, such as training 100 employment specialists in the supported employment model, and report on their progress in doing so.
We have our first set of data available. They reveal that, within the next 2 years, the first group of 7 States will have:
- Made more than 150 significant policy changes, including more than 35 regarding the financing of mental health-related services;
- Trained about 20,000 providers in best mental health practices;
- Made 65 significant organizational changes to support transformation;
- Expanded data accountability systems across 450 organizations; and
- Implemented State-of-the-art mental health practices in more than 1,000 programs.
In Ohio, the State has trained 2,500 law enforcement personnel in the Crisis Intervention Training (CIT) model and 2,500 early childhood providers using the Incredible Years curriculum.
Oklahoma has developed a framework and collaborative among researchers, State agencies, and consumers to form partnerships between science and service, including a grant process for research. The partnership has already resulted in 5 grant awards being made.
And in Maryland, Consumer Quality Teams have been established in 3 clinical programs. The teams strengthen self-advocacy and critical thinking in consumers and enhance the quality of mental health services by addressing individual consumer concerns and resolving them at the level of the local system or provider. Consumer Quality Teams will be established in 2 additional programs in the coming year. This is transformation in action!
In addition to measuring infrastructure change, we have embarked on a groundbreaking effort to measure recovery and resilience as consumers and families define them. To be certain our efforts are truly consumer-driven, we hired five consumer and family member consultants to help develop the evaluation plan and assist with collecting and interpreting the data.
[Slide 10: National consensus definition]
To measure recovery and resilience, each State will compare the experiences of consumers and families in programs or services that have been or are anticipated to be impacted by the T-SIG grant to the experiences of consumers and families in programs or services that are not anticipated to be impacted by the T-SIG grant. States can choose their own measures of recovery and resilience as long as they are validated and are consistent with SAMHSA’s National Consensus Statement on Mental Health Recovery.
When you look at the types of outcomes that most recovery measures include, you can see what a powerful construct recovery truly is. In addition to social connectedness, which the URS data capture, T-SIG States will be measuring such seemingly intangible values as confidence, hope, optimism, empowerment, meaning in life, and well-being. They will also be examining their systems to determine whether consumers have choice and control, are given hope for recovery, are supported in having positive relationships, and are treated as full partners in developing person-centered plans. To measure the recovery process, which is unique from one individual to another, and to be able to tie it to the types of system-level changes States adopt, truly is revolutionary.
Indeed, I think of our efforts to collect, interpret, and disseminate data on mental health service delivery at the State level to be evolutionary to this point. We have developed, refined, tested, and further improved our measures, and we will continue to do so.
[Slide 11: Albert Einstein quote]
Now, with most of you able to report the URS and NOMs data, and with the T-SIG States beginning to submit data for the national evaluation, we have moved to the next level. It was Albert Einstein who said, “The significant problems that we face cannot be solved at the same level of thinking we were at when they were created.” The problems in our mental health systems are longstanding and deep rooted, but we know that not changing the way we do business is no longer an option.
We must adopt new ways of working together, measure whether we are being successful, adapt those practices that don’t get results, and measure again. We are the vanguard of a movement that’s returning consumers to the center of care and the more we can use data to demonstrate what works, the more successful we, and ultimately the individuals we serve, will be.
Looking Ahead
Finally this afternoon, I would like to touch on several further refinements in data collection and reporting that will move the transformation agenda forward. First, we have a workgroup of DIG States that have volunteered to define a time interval for consistent reporting of employment outcomes across States. Currently, there is no one common time interval used by all States in recording and reporting the employment status of individuals with mental illnesses. This is an important effort because it will help us identify change over time in employment, which may indicate progress toward recovery. Mental health recovery is the ultimate goal of a transformed system of care.
[Slide 12: Client-level pilot]
Second, we also have a client-level pilot project involving 9 States, which is examining what it would take for States to collect client-level data in terms of costs, impact, and value to the States and to SAMHSA. Aggregate data tell us whether we are headed in the direction we want to go; client-level data may help us understand whether that direction is the right one for the adults, children, and families we serve.
These and other efforts to further refine the URS and NOMs measures are in a very real sense emblematic of the type of transformation we are trying to achieve at the State level. They are collaborative, comprehensive, and designed to challenge us to move beyond old ways of thinking about a problem. This is transformation in action!
Wrap-up and Conclusion
[Slide 13: Data photo]
I’d like to leave you with an anecdote from an episode of Star Trek: The Next Generation. The character Data is an artificial life form designed to resemble a human. He has impressive computational abilities, but has difficulty understanding certain aspects of human behavior. However, he may not be entirely lacking in human traits after all.
When the show’s chief medical officer calls him Dah-ta, rather than Day-tuh, he corrects her pronunciation. She asks, “What’s the difference?” and he replies, “One is my name, the other is not.” His comment reflects the deep longing in all of us to be known, respected, and loved for who we are.
The adults, children, and families we serve all have names. They have distinct personalities, goals, and dreams. They want to participate in their care, recover on their own terms, and live in community with their families, friends, neighbors, and coworkers. Most of all, they want to feel that their life has meaning and purpose and that we support and celebrate them in aspiring to their personal best. Even as we collect data about them and the services they receive, we must never forget that there is a person behind the numbers. Collecting data is a means to an end, but it is never an end in itself.
Thank you. I’d be happy to take your questions.
|
