A Review of Recent Evaluation Efforts Associated with Programs and Policies Designed to Promote the Employment of Adults with Disabilities

 

By: Gina Livermore and Nanette Goodman, Mathematica Policy Research

February 2009

 

This research was conducted under the Rehabilitation Research and Training Center on Employment Policy for Persons with Disabilities at Cornell University, funded by the U.S. Department of Education, National Institute on Disability and Rehabilitation Research (Cooperative Agreement H244B040013).  The opinions, conclusions, and errors in this paper are the sole responsibility of the authors and do not represent the official views of the U.S. Department of Education, Mathematica Policy Research, Inc., or Cornell University and do not assume endorsement by the Federal Government (Edgar, 75.620(b)).

Contract No.:H133B040013

MPR Reference No.:6239-620

 

 

 

 

Submitted to:

Cornell University

The Employment & Disability Institute

School of Industrial and labor Relations

331 Ives Hall

Ithaca, NY  14853-3901

Project Officer: Susanne Bruyère

 

Submitted by:

Mathematica Policy Research, Inc.

P.O. Box 2393

Princeton, NJ  08543-2393

Telephone:       (609) 799-3535

Facsimile:        (609) 799-0005

Project Director: David Stapleton

 

Contents

A Review of Recent Evaluation Efforts Associated with Programs and Policies Designed to Promote the Employment of Adults with Disabilities  1

Acronyms  1

I.  Introduction  2

A.        Purpose of the Report 2

B.        Factors Affecting the Employment of People with Disabilities  4

1.        Why the Employment of People with Disabilities Matters  4

2.        An Overview of the Labor Market and Factors Affecting the Employment of People with Disabilities  5

II.  Findings  6

A.        Nature of the Initiatives  6

B.        Adequacy of the Evaluation Efforts  7

C.        Conclusions  8

III.  Descriptions of Initiatives and Evaluation Efforts  9

A.        Legislation  9

1.        American with Disabilities Act of 1990  9

2.        Balanced Budget Act of 1997  10

3.        Ticket to Work and Work Incentives Improvement Act of 1999  10

4.        Workforce Investment Act of 1998  11

B.        Centers for Medicare and Medicaid Services  14

1.        Demonstrations to Maintain Independence and Employment 14

2.        Medicaid Buy-In Programs  16

3.        Medicaid Infrastructure Grants  17

C.        Department of Education  18

1.        Federal/State Vocational Rehabilitation Services  18

2.        Projects with Industry  21

3.        RSA/SSA Systems Change/State Partnership Initiative  23

b.        Evaluation Activities  23

D.        Department of Labor 25

1.        Customized Employment Grants  25

2.        Disability Employment Grants  26

3.        Disability Program Navigators  30

4.        Work Incentive Grants  31

E.         Department of Veterans Affairs  33

1.        Vocational Rehabilitation and Employment Program   33

F.         Internal Revenue Service  34

1.        Business Tax Incentives to Hire and Accommodate People with Disabilities  34

G.        Social Security Administration  35

1.        Accelerated Benefits Demonstration  35

2.        Benefit Offset Demonstration  36

3.        Benefits Planning, Assistance and Outreach/Work Incentive Planning and Assistance  37

4.        Employment Support Representatives/Area Work Incentive Coordinators  38

5.        Florida Freedom Initiative  39

c.         Progress to Date and What Will Be Learned in the Future  40

6.        Mental Health Treatment Study  40

a.         Intervention and Target Population  40

7.        Substantial Gainful Activity Level Increase  40

8.        Ticket to Work Program   42

H.        Substance Abuse and Mental Health Services Administration  43

1.        Employment Intervention Demonstration Program   43

References  44

Acknowledgements  48

 

Acronyms

AB                          Accelerated Benefits Demonstration

ADA                       Americans with Disabilities Act of 1990

AIDS                      Acquired Immune Deficiency Syndrome

AWIC                     Area Work Incentives Coordinator

BAC                       Business Advisory Council

BBA                       Balanced Budget Act of 1997

BOND                    Benefit Offset National Demonstration

BPA                        Berkeley Policy Associates

BPAO                     Benefits Planning, Assistance, and Outreach

CDC                       Consumer Directed Care

CDR                       Continuing Disability Review

CMS                       Centers for Medicare and Medicaid Services

CPS                        Current Population Survey

CWHS                    Continuous Work History Sample

CWIC                     Community Work Incentive Coordinators

DI                           Social Security Disability Insurance

DMIE                     Demonstrations to Maintain Independence and Employment

DOL                       U.S. Department of Labor

DPN                       Disability Program Navigator

ED                          U.S. Department of Education

EEOC                     Equal Employment Opportunity Commission

EIDP                      Employment Intervention Demonstration Program

EN                          Employment Network

ESR                        Employment Support Representative

ETA                        Employment and Training Administration, U.S. Department of Labor

FFI                          Florida Freedom Initiative

FO                          Social Security Field Office

GAO                       U.S. Government Accountability Office

GPRA                     Government Performance and Results Act

HIV                        Human Immunodeficiency Virus

IDA                        Individual Development Account

IRS                         U.S. Internal Revenue Service

JTPA                       Job Training Partnership Act

LSVRSP                 Longitudinal Study of the Vocational Rehabilitation Services Program

MHTS                     Mental Health Treatment Study

MIG                        Medicaid Infrastructure Grant

MPR                       Mathematica Policy Research, Inc.

NCD                       National Council on Disability

ODEP                     Office of Disability and Employment Policy, U.S. Department of Labor

OSERS                   Office of Special Education and Rehabilitative Services, U.S. Department of Education

PART                     Program Assessment Rating Tool

PASS                      Plan for Achieving Self Support

PMPM                    Per Member Per Month

PWI                        Projects with Industry

RSA                        Rehabilitation Services Administration

RTI                         Research Triangle Institute

SAMHSA               Substance Abuse and Mental Health Services Administration

SGA                       Substantial Gainful Activity

SIPP                       Survey of Income and Program Participation

SPI                          State Partnership Initiative

SSA                        Social Security Administration

SSI                          Supplemental Security Income

TANF                     Temporary Assistance for Needy Families

Ticket Act               Ticket to Work and Work Incentives Improvement Act of 1999

TTW                       Ticket to Work Program

VA                          U.S. Department of Veterans Affairs

VCU                       Virginia Commonwealth University

VR                          Vocational Rehabilitation

VR&E                    Vocational Rehabilitation and Employment Program, U.S. Department of Veterans Affairs

WIA                       Workforce Investment Act of 1998

WIG                       Work Incentive Grant

WIL                        Work Incentive Liaison

WISE                      Work Incentive Seminar Event

WIPA                     Work Incentives Planning and Assistance

WOTC                    Work Opportunity Tax Credit

 

 

I.  Introduction

A.   Purpose of the Report

Over the past several years, a number of new programs, policies, and initiatives designed to promote the employment of people with disabilities have been implemented by state and federal agencies, and are being evaluated to greater and lesser degrees. The purpose of this report is to provide a review of the recent evaluation activities being conducted for these new initiatives, as well as some existing programs that serve people with disabilities. The review is intended to provide policymakers, researchers, and others interested in efforts designed to promote the employment of people with disabilities a single source for information on the nature of the initiatives and the evaluation efforts that have been recently completed or are currently under way and the findings to date related to the effectiveness of these initiatives. This broad review is also intended to provide some evidence of the progress we are making. The report also suggests avenues where further efforts and progress might be warranted.

 

Our review focuses on studies and initiatives that—

·      Represent a federally sponsored program, policy, or initiative designed specifically to improve employment of the working-age adult population with disabilities, and

·      Evaluate the effectiveness of the initiative or program (the evaluation must have been completed by 2000, or be pending implementation or completion).

Using the above criteria, we identified 27 initiatives or programs and their associated evaluations for review. These are briefly described in Exhibit 1. Because of resource constraints, we did not review initiatives designed to improve the adult employment outcomes of youth with disabilities, such as the Social Security Administration (SSA) sponsored Youth Transition Demonstrations. We also did not review small-scale studies evaluating the effectiveness of specific clinical, supported employment, or vocational rehabilitation (VR) approaches. We only looked at information related to the major federal programs serving people with disabilities, general legislation and policies, and initiatives that were fairly large-scale in nature.

 

To provide context for the discussion of findings, in the remainder of this introductory section we describe the general theory of the labor market and factors believed to affect whether an individual with a disability becomes and remains employed. The findings from our review are presented in Section II. In the final section, we provide summary descriptions of each of the 27 initiatives reviewed.

 

 

 

Exhibit 1

 

Employment-Related Initiatives Reviewed

 

Program/Policy/Initiative

Oversight/Sponsoring Agency

Year(s) Implemented

Target Population

Employment Barriers Addressed

Legislation

Americans with Disabilities Act*

Congress/EEOC

1999

People with disabilities

Workplace discrimination

Lack of workplace accommodations

Balanced Budget Act*

Congress/CMS

1997

Working people with disabilities

Lack of adequate health supports Loss of public health insurance as earnings rise

Ticket to Work and Work Incentives Improvement Act*

 

Congress/SSA/CMS

 

1999

 

People with disabilities, with particular emphasis on SSI/ DI beneficiaries

Lack of adequate health supports

Loss of public health insurance as earnings rise

Risk of cash benefit loss due to employment

Lack of employment-related supports

Lack of information about SSA work incentives

Workforce Investment Act*

Congress/DOL/RSA

1998

Adults, dislocated workers, and youth

System fragmentation

Lack of consumer control in career development

Centers for Medicare and Medicaid Services (CMS)

Demonstrations to Maintain Independence and Employment

 

CMS

 

Various years since 2000

 

Working adults with disabilities or potentially disabling conditions

Lack of health insurance/adequate health supports to prevent conditions from becoming disabling

Incentives to discontinue work to qualify for public health insurance

Medicaid Buy-In Programs

CMS

Various years since 1992

Working adults with disabilities

Lack of adequate health supports

Loss of public health insurance as earnings rise

Medicaid Infrastructure Grants

 

CMS

 

Various years since 2001

Adults and transition-age youth with disabilities

Variety of barriers addressed, depending on the focus of individual state efforts

Department of Education (ED)

Federal/State Vocational Rehabilitation (VR) Services Program

RSA

 

1973

 

People with disabilities who want to work

Lack of employment-related supports

Lack of skills/training

Projects with Industry

 

RSA

 

1968

 

People with disabilities who want to work

Lack of skills/training

Lack of employment-related supports

Lack of business involvement in employment programs

Systems Change/State Partnership Initiative

RSA/SSA

 

1998

 

SSI/DI beneficiaries & others with disabilities who want to work

System complexity and lack of coordination

 

Department of Labor (DOL)

Customized Employment Grants

DOL

2001-2003

People with disabilities who want to work

Lack of flexible, individualized employment supports

Disability Employment Grants

DOL

1998 and 2002

People with disabilities who want to work

Lack of employment opportunities above entry-level positions

Disability Program Navigator

DOL/SSA/state One-stops

Various years since 2003

One-stop users with disabilities

System complexity and lack of accessibility of the One-stop system to people with disabilities

Work Incentive Grants

DOL

2000-2004

One-stop users with disabilities

Lack of accessibility of the One-stop system to people with disabilities

Internal Revenue Service (IRS)

Barrier Removal Tax Deduction

IRS

1976 (amended in 1990)

Businesses of any size

Costs to employers to hire/retain/accommodate workers with disabilities

Disabled Access Tax Credit

IRS

 

Small businesses

Costs to employers to hire/retain/accommodate workers with disabilities

Work Opportunity Tax Credit

IRS

1996

Businesses of any size

Costs to employers to hire/retain/accommodate workers with disabilities

Department of Veterans Affairs (VA)

Vocational Rehabilitation and Employment Program

VA

1918 (in various forms)

Veterans with service-connected disabilities

Lack of employment-related supports

Lack of skills/training

Social Security Administration (SSA)

Accelerated Benefits Demonstration

 

SSA

 

2007

 

Uninsured DI beneficiaries in the Medicare 24-month waiting period

Lack of health insurance/adequate health supports during Medicare 24-month waiting period

Benefit Offset National Demonstration

SSA

Pending

DI beneficiaries

Loss of DI benefits as earnings rise

Benefits Planning, Assistance and Outreach/ Work Incentives Planning and Assistance

SSA

 

2001

 

SSI and DI beneficiaries

Lack of information about SSA work incentives

Employment Support Representatives/ Area Work Incentive Coordinators

SSA

 

2000

 

SSI and DI beneficiaries

Lack of information about SSA work incentives

Florida Freedom Initiative

 

SSA/CMS/HHS/Florida Agency for Persons with Disabilities

2005

 

Adult SSI recipients with developmental disabilities

Loss of SSI and Medicaid as earnings and assets increase

Lack of employment-related supports

Mental Health Treatment Study

 

SSA

 

2006

 

DI beneficiaries with schizophrenia or affective disorder

Lack of adequate medical, behavioral, and employment-related supports and coordination/integration among these supports

Substantial Gainful Activity Level Increase

SSA

1999

SSI and DI beneficiaries

Loss of DI benefits as earnings rise

Ticket to Work Program

SSA

2002

SSI and DI beneficiaries

Lack of employment-related supports

Substance Abuse  and Mental Health Services Administration (SAMHSA)

Employment Intervention Demonstration Program

SAMHSA

 

1995-2000

 

Adults with serious mental illness

Lack of adequate medical, behavioral, and employment-related supports and coordination/integration among these supports

* The information shown in the exhibit refers exclusively to the provisions relevant to the employment of people with disabilities.

 

B.    Factors Affecting the Employment of People with Disabilities

Recent efforts designed to promote the employment of people with disabilities and attempts to evaluate their effectiveness cannot be assessed without an understanding of the reasons why employment can be beneficial and the factors likely to affect the employment of people with disabilities. In this section, we present a brief discussion of these issues to provide a framework and context for the discussion of the findings.

1.    Why the Employment of People with Disabilities Matters

The employment rates of working-age people with disabilities are low relative to their counterparts without disabilities, and there is evidence that the rates have been declining since the 1990s (Stapleton, Burkhauser and Houtenville 2004). In 2005, the employment rate of people with disabilities was just 38 percent, compared with 78 percent among people without disabilities (Rehabilitation Research and Training Center on Disability Demographics and Statistics 2005).

 

A very low employment rate for any working-age group in our society is cause for concern for several reasons:

·      From the individual perspective, market work is the principal source of income in all modern societies and is key to financial independence and well-being. Employment and contributing to family and societal productivity is also an important component of self-esteem.

·      From the employer perspective, if individuals are unable or unwilling to easily participate in the labor market, it reduces the pool of qualified. This makes it more difficult and costly for employers to access the labor they need to produce their goods and services.

·      From the government and taxpayer perspective, low employment rates are undesirable because local, state, and federal governments (supported by taxpayers) benefit from the taxes levied on earnings and greater consumption in response to higher income. Governments also benefit from high employment rates because people reduce their dependence on public support programs that are financed by government expenditures.

·      From society’s perspective, low employment means that some human capital is not being put to productive use, so aggregate productivity and welfare might not be maximized. When the skills and talents of individuals are not put to productive uses, the lost productivity cannot be recaptured.

Government investments in programs, policies, and initiatives designed to increase the employment of working-age people with disabilities are desirable because they can lead to greater individual well-being, increased government revenues, reduced government spending, and net gains to society.

2.    An Overview of the Labor Market and Factors Affecting the Employment of People with Disabilities

The labor market is comprised of two primary players: individuals who supply labor and firms that demand labor. The interactions of these players, along with many different factors that influence their individual decisions regarding employment, will affect the employment of people with disabilities in the aggregate. Below, we briefly discuss the factors that can affect the willingness of individuals with disabilities to supply labor; the likelihood that employers will hire and retain employees with disabilities; and the efficiency with which individuals are matched to jobs in the labor market.

a.    Individuals

Economic theory posits that the amount of labor individuals are willing to supply (that is, the number of hours individuals are willing to work) will depend on their preferences and an hours/earnings tradeoff. Individuals must choose how to allocate their limited time between market work activities and all other activities. Economists refer to non-work activities as “leisure,” but these include all forms of unpaid work, household work, dependent care, and self-care. Market work is necessary to obtain earnings, which are used to purchase goods and services. Therefore, the allocation of hours to market work and other activities represents a choice based on the individual’s preferences and the tradeoff between the consumption of goods and services and the consumption of leisure. Many factors will affect the perceived net benefits to working by an individual, and will determine whether an individual decides to become employed and the number of hours he or she will decide to work.

 

Compensation and Benefits. How much an individual will earn from work, along with other benefits (for example, health insurance, retirement benefits, paid leave) will certainly influence his or her decision to engage in work. All else constant, the higher the wages and benefits, the more likely an individual will be willing to supply labor. Many factors will influence wages and benefits associated with specific jobs and occupations, and for specific individuals within a job or occupation. In general, anything that affects productivity (or perceived productivity) will affect the wage rate. The more productive an individual is (as measured by the market value of the output produced), the higher the wage the individual will command in the labor market. Factors that tend to affect wage rates include education, skills, training, and experience. If a disability reduces productivity by eroding skills or abilities, it can also reduce the wage rate offered to the individual. At lower wage rates, individuals may be unwilling to work or to work as many hours.

 

Availability of Non-Labor Income and Resources. If an individual has income available from sources other than earnings, it can diminish the need and desire to engage in employment. Other sources of income might include spousal earnings, savings, private disability benefits, and public health insurance and income assistance, such as Social Security Disability Insurance (DI), Supplemental Security Income (SSI), Medicare, Medicaid, food stamps, and housing assistance. The more income and resources available to the individual from other sources, the lower the value of work.

 

Preferences for Work and Non-Work Activities. An individual’s own preferences for spending time working versus non-work activities will influence the likelihood of working and the hours of labor supplied. Time is a limited resource. How one allocates his or her limited time will be based on individual preferences for engaging in paid work (for purposes of earning income that will enable the purchase of goods and services) and non-work activities. Family obligations, time required for health or personal care activities, how one derives enjoyment and self-worth, and expected remaining years of life will affect the value of work relative to other activities.

 

Resources/Investments Required to Engage in Work. Work-related expenses or investments may be required to successfully engage in paid work; for example, transportation to and from work, business attire or uniforms, or specific kinds of tools, equipment, or supplies. All else equal, the higher the cost of work-related items borne by the individual, the lower the net value of work from the individual’s perspective.

 

Information. Information plays an important role in the individual’s decision-making process. If individuals lack adequate, or accurate information about the process and resources available for finding and maintaining employment, the impact of earnings on benefits and income, or other consequences of working or not working, they might make decisions about work that are less than optimal for them. Inaccurate information can lead to the individual to work too little or too much, relative to the optimal amount. For instance, if the individual thinks that earnings will have a more negative impact on public benefits than they actually do, he or she might work too little, but the opposite might be true if the individual fails to anticipate the negative effect of earnings on benefits.

b.    Firms

Economic theory predicts that the amount of labor firms demand will depend on the demand for the firm’s product, the productivity of labor (and other inputs), the wage rate and other costs firms must incur to hire labor, and the costs associated with other inputs that are substitutes or complements to labor. The theoretical profit-maximizing firm will demand the amount of labor at which the value of the additional output produced by the last worker hired is just equal to the cost of hiring that worker. Therefore, anything that affects the cost or productivity of labor will affect the amount of labor demanded by a firm. Factors that can affect the cost or productivity of labor include:

 

Human Capital. Human capital refers to the stock of skills and abilities embodied in labor. Greater skills, knowledge, and experience can lead to greater productivity. Productivity, or the value of what is being produced by labor, is taken in consideration relative to the cost of labor in a firm’s hiring decisions. Demand for labor will be higher when productivity is high relative to the costs. Factors that might erode productivity, such as poor health or changes in technology that make existing skills obsolete, will reduce the demand for labor.

 

Compensation and Benefits. How much the firm must pay labor in the form of wages and benefits will affect the firm’s demand for labor. As noted previously, wages are usually higher the greater the value of the labor output. Factors other than productivity can affect wages and benefits, however, such as: the availability of labor and competition in the local market; state and federal regulations, such as minimum wage and workers’ compensation laws; the influence of labor unions; and wage discrimination by the employer.

 

Accommodations. The accommodations that employers might need to make for workers with disabilities represent a cost of employment. Accommodations might include the purchase of special equipment (such as modified computers for persons with impaired eyesight), providing wheelchair access, flexible hours to accommodate shift work, and allowing employees to work from home. Non-disability related accommodations might also be required by certain individuals; for example, child care, or flexible schedules to accommodate employees with children or long commutes. Economic theory suggests that high accommodation costs would likely result in a reduced demand for labor among individuals requiring high-cost accommodations. In an attempt to avoid having to make such accommodations, employers might avoid hiring such individuals, or might shift the cost of those accommodations to the employees who require accommodation in the form of lower wages.[1]

 

Taxes, Subsidies, and Government Regulations. There are several provisions in the U.S. tax code designed to reduce the employer tax burden and offset the potential or realized costs of hiring individuals with disabilities, thereby increasing a firm’s incentive to employ people with disabilities. These take the form of wage subsidies, tax credits, and tax deductions. A number of other, more general government regulations, such as occupational health and safety standards, family leave requirements, and the FICA tax, serve to increase the cost of labor to the firm.

c.     Job-Employee Matching

Economic theory provides a framework for analyzing job-search behavior. Individuals vary in their abilities, education, work experiences, and characteristics, and jobs vary in their skill and knowledge requirements. Economists often describe the labor market as a matching process in which workers with varying characteristics are matched with appropriate jobs. The economic framework for this process is useful for understanding how environmental factors affect an individual’s job search and ultimately, employment.

 

According to this theory, individuals entering the job market are constrained by their available resources to accept an offer of employment within a specified timeframe. Initial assets available and access to income during the search period will determine resource availability. While searching, individuals must weigh the risk of continuing to search for a job offering their “reservation wage” (the minimum wage that must be received to accept an offer of employment) against the risk of exhausting their resources and being forced to accept the next offer of employment. Other factors also affect the matching process, such as employer perceptions of the qualifications of potential employees, potential employee perceptions of job characteristics, and the amount and nature of other information available to both parties in making their respective employment decisions.

 

Income During the Job Search Process. Access to other sources of income during the job search process can affect the likelihood that an individual will accept a given job offer. Having other resources available allows the individual to be more selective, and can prolong a job search relative to if the individual had few or no other sources of income during the job search process. Because SSI, DI, and workers’ compensation benefits are not time limited, these programs might be expected to increase the time spent searching for employment among those seeking to return to work. When a person begins to receive DI, SSI or workers’ compensation, all else being equal, such an individual can afford to be more selective in searching for employment than a comparable person receiving temporary benefits, for example, under unemployment insurance, or one who is living on limited savings.

 

Information. Information about the other party plays a key role on both sides of the matching process. Individuals need information about where job openings exist, the needed skills and other requirements of the job, job benefits, and the characteristics of the employer in order to find job opportunities that match their interests, capabilities and needs. Some employers will have an infrastructure and culture that are better able to meet the needs of people with specific types of disabilities relative to others. This information, however, can be difficult to discern during the interview process, and some job candidates with disabilities may be reluctant to question employers about accommodations or other disability-related needs for fear of jeopardizing their chances of receiving a job offer. In addition, employers might misrepresent the characteristics of the job or the firm.

 

From the employer perspective, information is important for distinguishing who, among the pool of job candidates, is best suited for the position. Employers judge the potential productivity of job candidates based on both objective and subjective factors. Their assessments can be inaccurate if job candidates over-represent their qualifications, and/or if an employer’s assumptions about the impact of a disability on productivity are incorrect. As with any job candidate, employers have good reason to be skeptical of the candidate’s ability claims. The presence of a disability adds another dimension of uncertainty to the hiring decision, which might discourage some employers from hiring people with disabilities.

d.    Summary

The above discussion suggests that a multitude of factors come into play in determining the employment outcomes of people with disabilities. It also suggests that many potential barriers or disincentives to work can arise for people with disabilities:

·      Poor health or functioning limiting the ability to engage in work or reducing the level of productivity

·      Inadequate education, skills, training, or job-related experience

·      Lack of reliable transportation to and from work

·      Lack of specific supports needed while at work, or at home to prepare for/enable work

·      Loss of public or private cash and in-kind benefits as earnings and assets increase

·      Inaccessible workplaces and inflexible employment situations

·      Costly accommodations

·      Fear of discrimination and employer misconceptions of disability

·      Discrimination and employer misconceptions of disability

·      Lack of information about individual abilities and productivity

·      Insufficient wages or benefits offered with employment

·      Lack of information about employment-related supports and resources available

·      Lack of information about the impact of work on cash and in-kind benefits

·      Inadequate job search and interview skills or information

These barriers are, presumably, among the factors intended to be addressed by the wide variety of federally sponsored initiatives designed to improve the employment of people with disabilities described in the remainder of this report.

 

 

II.  Findings

A.   Nature of the Initiatives

The 27 initiatives (shown in Exhibit 1) use a wide variety of strategies to address many of the barriers to employment faced by people with disabilities. Here, we describe some general observations regarding the nature of these initiatives.

 

Most of the Interventions Focus on the Individual, Rather Than on Employers. The majority of the initiatives shown in Exhibit 1 are directed primarily at individuals, rather than employers. They are designed to increase the net value of work to the individual, increase human capital, make employment and employment-related resources more accessible, and/or assist the individual in the job search process. The disability-related tax provisions, Americans with Disabilities Act (ADA) provisions, Projects with Industry, and selected activities conducted by states under their Medicaid Infrastructure and Disability Employment Grants represent the few  directed at employers and intended to increase the likelihood that employers will hire people with disabilities.

 

Many Efforts Are Designed to Address System Complexity. Several of the initiatives are specifically designed to make information about disability and employment resources more readily available to people with disabilities and reduce the complexity of the service system from the perspective of individuals with disabilities. SSA’s Area Work Incentive Coordinators and Work Incentive Planning and Assistance initiatives are intended to provide better information to SSI and DI beneficiaries about the impact of work on disability benefits. The DOL Disability Navigator program is designed to provide information to people with disabilities using One-Stop Career Centers about the employment-related and other assistance programs and resources available in the community. The RSA/SSA Systems Change/State Partnership Initiative and CMS Medicaid Infrastructure Grant efforts provide support for states to look at their systems more broadly, and develop the means to better coordinate services and reduce system complexity. None, however, directly addresses the inherent complexity of the SSI and DI work incentive provisions.

 

The Onus for Change Appears to Be on the State and Local Systems. Although all of the initiatives are in some way sponsored by federal regulations, agencies, and dollars, most are being designed and implemented by state and local entities. Even some of those directed specifically at beneficiaries of the federal DI program (for example, the Mental Health Treatment Study) rely on state and local providers to implement the intervention. This reflects that fact that most of the initiatives involve some form of service provision, and services are typically provided at the state and local level.

 

There Are New Efforts to Combine/Coordinate Health and Vocational Supports. The importance of access to health insurance and maintaining health insurance coverage as employment status changes is evident in many of the initiatives (for example, Medicaid Buy-In programs, the Accelerated Benefits demonstration, Medicare provisions of the Ticket Act). In addition, a few of the initiatives go beyond simply providing access to health insurance because they are designed specifically to combine and coordinate health and vocational supports in ways that promote employment (one of the treatment groups in the Accelerated Benefits demonstration, the Employment Intervention Demonstration Program, and the Mental Health Treatment Study).

 

There Is Some Emphasis on Asset Development. Though somewhat meager, a few of the initiatives have an explicit focus on asset development and the impact of assets on continued program eligibility. Most of the existing Medicaid Buy-In programs permit higher levels of assets than would be permitted under other Medicaid categories. The Florida Freedom Initiative includes the use of individual development accounts, which permit savings in these accounts to be excluded from SSI eligibility determinations.

 

Few Initiatives Focus on Early Intervention or Preventing Labor Force Withdrawal at Disability Onset. Most of the reviewed interventions focus on individuals who have already left the labor market and entered public programs. This might reflect, in part, the fact that many of the recent initiatives were developed in response to mandates in the Ticket Act, the provisions of which focus primarily on Social Security disability beneficiaries. It also reflects the fact that no federal agency has “jurisdiction” over people with disabilities until they become program participants. With the exception of the CMS demonstrations to maintain independence and employment, programs serving people with disabilities have not generally developed ways to provide supports that are intended to deter working people with disabilities from leaving employment and applying for public assistance benefits. The ADA, requiring employers to make accommodations when employees become disabled, and the laws permitting states to have Medicaid Buy-In programs have the potential to affect labor force withdrawal, but only to the extent that lack of access to accommodations and health insurance represent the primary barriers to ongoing employment.

 

Most Initiatives Represent Small Changes to the Status Quo. Nearly all of the initiatives  represent relatively small-scale or minor tweaks to the existing system of supports. Only the ADA represents a transformative change, in its broader purpose to guarantee equal opportunity and inclusion of people with disabilities in society. The Ticket Act and the Workforce Investment Act have elements that might have been used to bring about transformative changes in the way employment-related services and supports are provided to people with disabilities, but  they have been implemented in ways that effect only relatively small changes to the status quo. It is possible, however, that the experiences gained from the many small-scale efforts currently under way, and the increasing attention being paid to employment issues for people with disabilities will provide a foundation for more fundamental changes in the future. 

B.    Adequacy of the Evaluation Efforts

Although much useful information has been generated by the evaluation of the initiatives, many of these efforts have significant shortcomings or face specific challenges that limit the ability to draw firm conclusions about their impacts on employment. Below, we discuss the most important of these shortcomings and challenges.

 

Data Is Often Unavailable or Inadequate. The lack of adequate data to evaluate the impact of the intervention on employment was an issue in reviewing many of the initiatives. In some cases, the data needed to assess even basic outcomes were very poor or simply did not exist (for example, the three employer tax provisions). In other instances, administrative data from partnering agencies were needed, but they were unwilling or unable to cooperate due to resource or data confidentiality issues (for example, Systems Change/State Partnership Initiatives, Medicaid Buy-In). The general lack of data, in combination with the lack of an adequate comparison or control group (described below) are probably the two primary reasons why there is very little solid evidence on the impacts of the interventions on employment of the interventions. Substantial information exists regarding the implementation process and issues encountered, however, very little can be said with certainty about the impacts of the intervention on the employment of people with disabilities.

 

Many of the Interventions Lack the Comparison or Control Groups Needed to Produce Rigorous Estimates of Their Impacts. The goal of an impact evaluation is to measure the causal effect of a program or intervention. That is, outcomes under the intervention must be compared to estimates of what they would have been under the counterfactual–usually meaning the status quo program or policy. Thus, it is important not only to observe the outcomes (or changes in outcomes) after the intervention, but to compare their levels (or changes) relative to those for a comparable group not affected by the intervention. Evaluations use two types of study designs to accomplish this—experimental (or randomized control group) designs, and quasi-experimental (or comparison group) designs.

 

Experimental designs are considered the more rigorous of the two approaches. Random assignment of study participants (usually recruited volunteers) to treatment and control groups creates two groups that are statistically equivalent to one another. Barring significant violations of the experimental design (for example, some control group members receive the intervention) differences in outcomes between the two groups can be assumed to be entirely due to the impact of the intervention.

 

Although some of the initiatives incorporate experimental designs (Benefit Offset, Accelerated Benefits, Employment Intervention Demonstration Program, Mental Health Treatment Study, four of the State Partnership Initiatives, and the four Demonstrations to Maintain Independence and Employment) a large majority of the interventions reviewed did not. This could be for at least two reasons. First, in the case of many of the initiatives we reviewed (for example, most provisions of the four federal Acts reviewed and the employer tax provisions) the initiatives were implemented in a manner such that random assignment was simply not possible. When Congress passes a law or creates a new program, it is no longer an experiment. It becomes a resource available to all in the target population, and, understandably, cannot easily be withheld from some individuals for purposes of evaluating its effect. However, in some cases, a quasi-experimental evaluation may be possible. We return to these types of evaluations below.

 

The second reason for not implementing an experimental evaluation design is that the sponsors of the intervention chose not to do so. This can be for many reasons: the infrastructure and resources available are insufficient; the initiative is viewed as a “proof of concept” test,[2] rather than a formal test of the impact of the intervention; the sponsors have ethical objections to withholding the intervention from control group members; the technical aspects of providing treatment to one group and not the other are problematic (for example, because members of the two groups might receive services from common providers or advisors); and/or the findings of a high quality evaluation of the intervention are not viewed as critical or worth the cost. From the information available to us for most of the initiatives, we cannot determine the relative importance of the preceding reasons in the cases where an experimental design was not pursued, but was technically possible.

 

In several of the initiatives we reviewed (for example, State Partnership Initiative, Ticket to Work, Disability Employment Grants), a quasi-experimental study design was attempted to evaluate the impacts of the program or initiative. In these examples, the evaluators identified a comparison group of individuals who did not receive the intervention, but who, in many important ways, are very similar to the treatment group who received the intervention. Using a comparison group (rather than a randomized control group) approach can be problematic, however. The evaluators of several projects found that they could not identify comparison groups that did not differ systematically from the intervention groups. For example, in the State Partnership Initiative evaluation, Peikes and Sarin (2005) compared the impact estimates generated from the use of a sophisticated statistical method for constructing a comparison group with the estimates generated using an experimental control group. Using numerous variables, a propensity score matching technique was used to identify an appropriate comparison group. When this approach was applied to one of the state projects that had an experimental design, the evaluators found that the non-experimental method produced estimates of impacts that were different than those produced by the experimental design, and concluded that the non-experimental estimates were misleading. The apparent problem with the non-experimental design is that it was not possible to match comparison group members to treatment group members with respect to factors that motivated the participation of treatment group members in the demonstration.

 

Another example is the Ticket to Work evaluation (Thornton et al. 2007), which attempted to exploit the phased rollout of the program and use a contemporaneous comparison group made up of beneficiaries in states where Ticket to Work had not yet been implemented. The evaluators found that the treatment and control states were not well matched. Differences across states in beneficiary earnings and benefit trends that existed before the Ticket to Work program rollout were so large that they obscured any effects that program might have had on these important outcomes. The problem, from an evaluation design perspective, appears to be that SSA selected states for the first rollout phase on the apparent readiness of state agencies and organizations for the rollout.

 

Low Participation Rates in Some Studies Limited the Ability to Detect Impacts. In a few instances where formal experimental or quasi-experimental evaluation studies were planned, the studies encountered lower than expected participation in the intervention. When that happens, effects for those who participate become harder to distinguish from random noise. Low levels of participation can happen for a variety of reasons: the target population for the intervention is too small or defined too narrowly; the methods used to identify and recruit participants are inadequate; the study enrollment process is overly complex or burdensome; participants distrust the intervention or study; or the intervention is ill-conceived or unattractive to the target population. The Florida Freedom Initiative had so few enrollees (just 35) that plans to evaluate the initiative were terminated. Although the Ticket to Work program has had over 200,000 participants, that number represents only a little over one percent of the target population, and the vast majority of them would have received services under the counterfactual–the pre-Ticket to Work reimbursement program for state vocational rehabilitation (VR) agencies. The low participation rate has contributed to the inability of the evaluation to detect employment impacts, if indeed the program has generated any.

 

Some Studies Pooled Data Across Interventions That Were Dissimilar, Making It Difficult to Draw Conclusions. Many grant programs offer the grantees wide latitude in the design of their interventions. Moreover, each grantee tends to serve a limited number of consumers (for example, Projects with Industry, Disability Employment Grants, Work Incentive Grants, State Partnership Initiative projects) making the sample sizes for each grantee too small to observe differences in outcomes. To offset this limitation, the national evaluations attempted to pool data across grantees, but because of the wide variation in populations served and types of interventions across grantees, pooling the data does not result in a useful assessment of the impacts of a particular intervention.

 

Study Timeframes Are Short, but Target Outcomes and Behavioral Change Might Be Longer Term. Programs are often designed to produce long-term impacts, for example improving someone’s wage trajectory over the course of their working life or keeping them in the workforce. The temporary, and often short, timeframe of project interventions and evaluations often necessitate that the measurement of changes in wages or employment be made over periods as short as six months or one year (Disability Employment Grants, Projects with Industry, VR services). Programs that might demonstrate impacts in the long run may be found to have no significant benefits in short run evaluations. Conversely, short-term impacts identified in an evaluation might not persist beyond the evaluation period.

 

Many Studies Assess the Impact of the Program on Service Users but Do Not Consider the Impact on the Larger Population of People with Disabilities. Many of the studies reviewed focused on measuring participant experiences, the impact of the program on participant outcomes, and in some cases, impacts on program expenditures. Broader analyses of the reasons for participation/nonparticipation and of costs and benefits external to the program were often not undertaken. This is likely for two primary reasons: the data needed to conduct broader participation analyses can be expensive to collect (the study must be able to identify relevant nonparticipants and collect information about their characteristics and reasons for nonparticipation). And if a rigorous impact evaluation for participants is not being conducted (as was the case in most of the studies reviewed) then there is no point in conducting a broader analysis of the costs and benefits associated with the program impacts. Many of the SSA initiatives represent at least partial exceptions to this limitation (for example, Ticket to Work, Mental Health Treatment Study, Accelerated Benefits, Benefit Offset). The target populations for these initiatives are well-defined (so non-participants can be easily identified) and formal impact evaluations, including some manner of cost/benefit component, are included in the evaluations.

 

Measuring the Impact of Systems Change Activities Is Difficult. Fragmentation of services and funding sources is often cited as a barrier to employment supports. Many of the initiatives reviewed were intended to address this barrier by integrating funding, coordinating services, formalizing referral services, and various other approaches (for example, Medicaid Infrastructure Grant, Workforce Investment Act, Disability Employment Grants, Work Incentive Grants, Disability Program Navigators, and the Systems Change/State Partnership Initiatives). However, evaluators have found it difficult to measure the impact of systems change activities on employment outcomes for several reasons. First, system quality indicators are generally not available. Concepts like attitudes, programmatic cohesion, and client understanding are hard to quantify, making the construction of systems change indicators difficult. Second, systems change occurs in many small increments that often take place over an extended period of time. Moreover, one change may have no effect until it is coupled with others. Isolating the effects of individual changes over the period of an evaluation is difficult, and made even more difficult by the fact that a variety of other factors affecting employment are occurring simultaneously (for example, changes in economic conditions, technology, the nature of work, and societal attitudes). For these and other reasons, evaluations of initiatives designed to change the overall support system have not provided convincing evidence of impacts on employment. These studies more often consist of process analyses that report on challenges, experiences, and “best practices” that have arisen from their efforts. Although, many of these best practices are idiosyncratic to the state system or specific target population, several grant programs (for example, Medicaid Infrastructure Grants, the Systems Change/State Partnership Initiatives, and several of the DOL grants) have developed successful strategies to disseminate each state’s lessons learned to other states. In several cases, these practices have been replicated in other states and grant programs. For example, the SSA/DOL Disability Program Navigator initiative was based on a best practice developed by one of the Work Incentive Grantees.

C.   Conclusions

Considerable effort has been undertaken to implement and evaluate programs and special initiatives intended to improve the employment of people with disabilities. But are we making any progress? On the one hand, the employment rate of people with disabilities remains low, few of the initiatives reviewed were able to rigorously demonstrate positive impacts on employment, and many of the initiatives were not sustained after the special funding for them ceased. On the other hand, it does appear that system change is occurring. Greater attention is being paid to the incentives and issues surrounding employment for people with disabilities, including how the currently fragmented system of programs and supports can both hinder and help individuals, and how services from multiple sources might be coordinated to achieve better outcomes. SSA in particular has demonstrated a much stronger focus on employment in its disability programs in recent years than at any time in the past. CMS has actively promoted employment through support for Medicaid Buy-In programs and Medicaid Infrastructure Grant activities, many state Medicaid programs have taken the opportunity to expand coverage to workers with disabilities, many one-stops have substantially increased access to services for those with disabilities, and there are signs of increased cooperation between these programs and state VR agencies. Though there is evidence of progress, transformation is occurring slowly, and as GAO notes, the many federal programs serving people with disabilities continue to operate without a coordinated strategy or centralized mechanism for ensuring that programs and policies are aligned (GAO 2008a).

 

How might we make greater progress toward improving the employment outcomes of people with disabilities? Two complementary strategies are suggested based on the findings of this review: undertake bolder initiatives and generate better evidence on the impacts of the initiatives undertaken. Significant progress is not likely to occur unless we can move beyond incremental changes to a system that, according to many, is in dire need of modernization and transformation.[3] Bolder initiatives would include ones that intervene early, before workers with disabilities have left the labor force and entered DI, and before youth with disabilities have entered a lifetime of dependency on SSI. Bolder initiatives would also provide incentives and mechanisms for multiple federal agencies to coordinate their efforts, pool resources, and strive to achieve common objectives.

 

Better evidence of the impact of the initiatives undertaken is also needed. Experimental evaluation designs and strong non-experimental designs can generate compelling information to policymakers. Strong evidence of impacts is particularly important for justifying more transformative changes to existing programs and policies.

 

The limited evidence of success makes efforts to improve the employment outcomes and self-sufficiency of people with disabilities seem daunting. But the experience of our nation’s welfare system gives cause for optimism. As noted by Butler (2007), there are many parallels between the experiences and factors that drove the 30-year transformation of the welfare system and what has been occurring more recently with our nation’s key disability programs. Changes in societal norms and expectations regarding the ability of single mothers to become employed and contribute to their families’ self-sufficiency, along with strong evidence of policy impacts derived from innovative policy experiments led to the significant changes in the welfare system embodied in the current Temporary Assistance for Needy Families (TANF) program and Earned Income Tax Credit for low-income parents. The focus of TANF is on providing supports and incentives that will help participants achieve greater self-sufficiency and reduce long-term dependency on public income supports. Societal norms and expectations surrounding the employment of people with disabilities have already changed, but we have yet to launch rigorous tests of highly innovative, transformative programs that improve the employment and self-sufficiency of people with disabilities.

 

 

III.  Descriptions of Initiatives and Evaluation Efforts

A.   Legislation

1.    American with Disabilities Act of 1990

a.    Intervention and Target Population[4]

The Americans with Disabilities Act (ADA) was signed on July 26, 1990. The ADA made discrimination based on disability illegal in all employment activities. The goal was to reduce some of the structural and institutional barriers people with disabilities faced in workplaces. By July 26, 1994, the ADA applied to all employers with 15 or more employees.

 

The ADA considers a disability to be “a physical or mental impairment that substantially limits a major life activity,” or “if you have a history of such a disability, or if an employer believes that you have such a disability, even if you don’t”. The ADA is designed to prevent discrimination against people with disabilities, but the law also applies to people who are discriminated against because of their association with a person with a disability (for example, family members). The ADA applies to all aspects of employment, such as hiring, firing, benefits, pay, promotion, and such.

 

The ADA also requires employers to make reasonable accommodations for employees with disabilities. These accommodations could be, for example, allowing flexible work schedules, making the work area physically accessible, or providing an interpreter. However, the ADA does allow for exceptions. If the modification would be too expensive or difficult then the employer is not required to make the change. Additionally, if any employee’s disability renders him or her unable to perform the essential functions of the job, the employer can refuse to hire the person. Inability to perform functions that are not essential to the job cannot be used as a basis for a decision to not hire an applicant with a disability.

b.    Evaluation Activities and Findings

Numerous qualitative and quantitative assessments of the impact of the ADA on the employment of people with disabilities have been conducted since the implementation of the Act in 1992. Here, we provide a brief description of the general findings from a few of the more recent efforts.

 

The empirical evidence regarding the effects of the ADA on the employment of people with disabilities is mixed. Acemoglu and Angrist (2001) used Current Population Survey (CPS) data to analyze the effect of the ADA on the employment of people with disabilities. They found that in the 21-39 age category, employment rates decreased for men with disabilities in 1993, and women with disabilities in 1992; both years correlate closely with the beginning of ADA enforcement. In their analysis, they controlled for the increase in the number receiving SSI. Interestingly, the largest decline in employment for people with disabilities was in medium-sized companies. The authors suggested that this is because small employers were exempt from parts of the ADA, while the larger firms could more easily absorb the costs associated with its enforcement. They also found that rates of employment dropped relatively more in states where the most ADA discrimination charges were filed—further suggesting that the ADA contributed to the lower employment rates. The employment rate effects were less for the 40-58 age group. The authors speculate that this is because people over age 40 were already protected by the Age Discrimination in Employment Act, and women were protected by Title VII.

 

DeLeire (2000a, 2000b) came to similar conclusions as Acemoglu and Angrist (2001), through analysis of data from the Survey of Income and Program Participation (SIPP). After controlling for demographic characteristics, industry, and occupation, DeLeire found that employment rates for people with disabilities were steady until 1989, then began to fall in 1990 and onward. DeLeire suggested that employers were not hiring people with disabilities in order to avoid the costs of accommodations and potential litigation based on the ADA protections. He noted that, while most accommodations are generally inexpensive, as others have shown, a substantial share are not—12 percent cost more than $2,000 and 2 percent cost more than $20,000. He also noted that researchers underestimate the impact of accommodations that do not have a direct cost—such as flexible scheduling.

 

The findings of these early studies have been challenged by many. Like most impact evaluations of new nationwide policies, they have to rely on non-experimental methods that are limited by the availability of comparison groups and data. In this case, the researchers needed to use the pre-ADA period as the comparison period, and data from surveys for that were conducted in both the pre-ADA and post-ADA period. The ADA’s implementation period occurred during a period when changes in other programs were potentially having an impact on employment of people with disabilities, as well as during a significant recession. Hence, these researchers faced the challenge of separating any effects of the ADA from the effects of other changes. They also had to use data that do not cleanly identify the population that the ADA’s employment provisions are applicable to and employment measures that were not arbitrary in important respects.

 

Houtenville and Burkhauser (2004) challenged the conclusions of DeLeire and Acemoglu and Angrist that the ADA was responsible for the observed decline in the employment rate of people with disabilities. They used the model developed by Acemoglu and Angrist (2001), but additionally analyzed data for people with a two-period work limitation (those who reported work limitations each March, two years in a row), who are presumed to have a more long-term disability and are more likely to meet the definition of disability intended by the legislation. They find that the decline in employment for the two-period work limitation group begins well before the ADA, and the start of the decline appears more closely linked to the expansion of eligibility criteria for SSI and DI benefits that began in 1984. Goodman and Waidmann (2003) also argue this point, and suggest that the recession in the 1990s factored into the decrease in employment rates. Houtenville and Burkhauser also showed that the results were very sensitive to changes in the definition of employment. Kruse and Schur (2003) analyzed SIPP data and found that alternative definitions of disability produced different conclusions regarding the effect of the ADA on employment, and suggested that any findings should be taken with a certain degree of caution. Hotchkiss (2004) analyzed SIPP and CPS data and concluded that the apparent decline in employment is partly due to nondisabled individuals who are not in the labor market being reclassified as disabled, rather than due to an increase in the labor force withdrawal of people with disabilities. She postulated that the increased reporting of disability may have occurred as a result of more stringent welfare reform requirements and the availability of more generous federal disability benefits. Jolls and Prescott (2004) exploited state variation in anti-discrimination laws prior to the ADA to estimate the impact of the ADA on employment in states with no pre-ADA law and states with laws against discrimination that, unlike the ADA, did not require employers to provide reasonable accommodations. They found immediate negative effects of the ADA on employment only in the states that had no pre-ADA law or had a law that did not require reasonable accommodations. They also found that the negative effects disappeared after a few years.

 

Qualitative assessments of the impact of the ADA also find mixed evidence regarding its impact on employment. Most recently, the National Council on Disability (NCD) released a large report assessing the impact of the ADA (NCD 2007a). The study based its findings on a review of publicly available documents and data, stakeholder interviews and focus groups, and analyses of data from several of the NOD/Harris surveys of people with disabilities to assess changes in perceptions and attitudes over time. The report concludes there is evidence that people with disabilities are experiencing less discrimination and greater accommodation on the job, but they do not appear to be experiencing increases in hiring. The report also notes that many people with disabilities, employers, and businesses still do not understand major provisions of the ADA, particularly the employment provisions, and that this lack of understanding is reducing the effectiveness of the legislation.

2.    Balanced Budget Act of 1997

a.    Intervention and Target Population

Under Section 4733 of the Balanced Budget Act of 1997 (BBA), states became able to expand Medicaid coverage of working people with disabilities by creating a new optional eligibility group whereby working individuals with disabilities meeting certain medical, income, and asset criteria may be permitted to “buy in” to Medicaid by paying a premium. Medicaid Buy-In programs are intended to help people with disabilities by providing access to health insurance to individuals who have relatively high earnings but do not qualify for Medicaid under one of the other eligibility provisions. Individuals are not required to be on SSI or DI to be eligible for the Medicaid Buy-In. However, the state must determine if the individual would be eligible for these programs if they were not working. The legislation allows states to provide the benefit to individuals with incomes up to 250 percent of the federal poverty standard. States are permitted to deduct certain expenses from income before applying the income test. SSI asset limits are also applied ($2,000 for an individual and $3,000 for a couple).

 

The Medicaid Buy-In was designed to allow low-income workers with disabilities to purchase affordable health insurance, reducing a major incentive to withdraw from the workforce and seek SSI. Because Buy-In eligibility does not, in theory, depend on current or former SSI/DI participation, it weakens the incentive for a person with a disability to withdraw from the workforce or to restrict earnings to receive health insurance benefits based on SSI or DI eligibility. There is still, however, an incentive to restrict earnings to remain within the Buy-In income limits, and the attractiveness of the program to potential participants will depend on its premium levels.

 

The 1999 Ticket to Work and Work Incentives Improvement Act (Ticket Act) expanded state authority originally granted under the BBA to offer Medicaid Buy-In programs to working people with disabilities. As described further below, the Ticket Act eliminated eligibility restrictions on income and assets, and expanded eligibility to a “medical improvement” group—individuals whose medical conditions have improved and are determined to be no longer eligible for SSI or DI, but who still have a severe impairment.

b.    Evaluation Activities and Findings

We describe the evaluation activities and findings to date associated with Medicaid Buy-In programs established under both the BBA and the Ticket Act in Section III.B.2 below.

3.    Ticket to Work and Work Incentives Improvement Act of 1999

a.    Intervention and Target Population

Signed by President Clinton on December 17, 1999, the Ticket to Work and Work Incentives Improvement Act (Ticket Act) is intended to address a number of work disincentives inherent in the DI and SSI programs. We describe the nature of these provisions below.

Ticket to Work and Self-Sufficiency Program

Section 101 of the Ticket Act amended the Social Security Act to establish an SSA Ticket to Work and Self-Sufficiency Program (TTW). TTW provides disability beneficiaries who are appropriate candidates with a voucher, or ticket, to be used to obtain vocational rehabilitation or employment services. The Ticket can be assigned to any pre-qualified provider of these services. The provider is paid based on the earnings and benefit outcomes of the beneficiary. While a beneficiary is using a Ticket, SSA is prohibited from initiating a continuing disability review (CDR), the periodic review that determines whether a beneficiary is still medically eligible to receive disability benefits.

 

The motivation behind TTW is to expand the employment and training opportunities of beneficiaries by expanding their access to a wide variety of vocational services. SSA had previously paid for services provided to DI and SSI beneficiaries by state VR agencies if the beneficiaries met a specific employment goal. TTW expanded the providers that beneficiaries could use to obtain SSA-funded services, but under more stringent outcome criteria. The goal was to allow market forces to reward providers who successfully move people with disabilities into work, through the use of consumer choice and performance-based reimbursement. The provision that suspends continuing disability reviews (CDRs) for individuals participating in the program aims to reduce the risk of benefit loss that a person faces when he or she enters a VR program. The provision ensures that people will not avoid rehabilitation services for fear that their participation will make them ineligible for disability benefits.

Expansion of Health Insurance Eligibility

The fear of losing eligibility for public health insurance is frequently cited as a major disincentive to work for people with disabilities. Because health care costs are often extremely high for people with disabilities, the cost of health care can often outweigh the financial benefits of a job that does not provide comprehensive health insurance coverage. For many individuals with disabilities, particularly those who can work but are not capable of working a full-time job that would provide health benefits, employment is simply not a financially viable option. The health insurance provisions of the Ticket Act are intended to address this situation by improving access to health insurance coverage in the following ways:

·      Medicaid Buy-In Programs. Section 201 of the Ticket Act loosens restrictions on states regarding who is eligible to buy into the Medicaid program. Under this legislation, states have the option to eliminate all income, asset, and resource limitations for workers with disabilities who buy into Medicaid. States can also continue to offer the Medicaid Buy-In to workers with disabilities, even if they are no longer eligible for DI or SSI because of medical improvement. States are authorized to require individuals to pay premiums, or other cost-sharing charges, on a sliding scale. Section 203 of the Act also established a grant program to help states build infrastructures to support working individuals with disabilities. These grants became known as the Medicaid Infrastructure Grant (MIG) and the Comprehensive Employment Opportunity grant. Although intended to encompass a broader range of supports than just health care, the grants were initially tied to the development of state Medicaid Buy-In programs and the provision of expanded personal assistance services under Medicaid, and are administered by the Centers for Medicare and Medicaid Services.

·      Extended Medicare Coverage. Section 202 of the Ticket Act provides for the continuation of Medicare coverage for individuals formerly receiving DI benefits who leave the rolls due to work. Under previous law, the extended period of eligibility allowed DI beneficiaries to continue receiving premium-free Medicare Part A coverage for a total of four years after they returned to work. The Ticket Act extends this period for up to an additional four and one-half years.

·      Suspension of Medigap Policy. Section 205 of the Ticket Act allows workers with disabilities who have a Medigap policy—a commercial insurance policy that provides benefits supplemental to Medicare—to suspend the premiums and benefits of the Medigap policy if they have employer-sponsored coverage. This allows workers to take advantage of employer-sponsored benefits, an important incentive to work, while retaining eligibility for their Medigap policies in case their employment attempts fail.

Other Work-Related Provisions

Several other provisions of the Ticket Act are design to promote employment among SSA disability beneficiaries:

·      Work Activity and CDRs. In addition to the limitations on conducting CDRs for individuals participating in the TTW program, Section 111 of the Ticket Act prohibits the use of work activity as a basis for review of an individual’s disability status. A beneficiary who is working is still subject to CDRs on a regularly scheduled basis, but the initiation of work cannot trigger such a review, and work activity may not be used as evidence that an individual is no longer disabled. This makes work activity less risky and presumably more attractive.

·      Expedited Reinstatement of Benefits. Section 112 of the Ticket Act allows for expedited eligibility determinations for applications of former long-term beneficiaries that have completed their extended period of eligibility. If an individual returns to work, earns enough to be ineligible for disability benefits for longer than the extended period of eligibility,[5] and is later unable to work, he or she will be able to return to the DI rolls without the lengthy eligibility process to which first-time applicants are subjected. Like the limitation of CDRs, this provision reduces the risk beneficiaries face when returning to work.

·      Work Incentives Outreach Program. Sections 121 and 122 of the Ticket Act directs SSA to establish a community-based work incentive planning and assistance program to provide accurate information related to work incentives to beneficiaries with disabilities. Under the program, SSA is to establish a competitive program of grants, cooperative agreements, or contracts to provide benefit planning and assistance to DI and SSI beneficiaries.  Such grant programs are to include information on the availability of protection and advocacy services.  The programs are also to conduct or fund ongoing outreach efforts and establish a corps of work incentive specialists within SSA who specialize in DI and SSI work incentives. In response, SSA established the Benefits Planning, Assistance and Outreach (BPAO) program, a cooperative agreement program with community-based providers, which later became known as the Work Incentive Planning and Assistance (WIPA) program. SSA also piloted the Employment Support Representative (ESR) position within SSA, which later became the Area Work Incentive Coordinator (AWIC) position.

Research and Demonstration-Related Provisions

In addition to the new programs and modifications to the DI and SSI programs described above, the Ticket Act included several research and demonstration-related provisions:

·      TTW Evaluation. The Ticket Act directs SSA to conduct an evaluation of the TTW program to assess the determinants of TTW participation, the characteristics of beneficiaries and providers who participate, the types of services provided and received, the employment outcomes of participants, beneficiary satisfaction with the program, and the costs and impacts of the program.

·      GAO Studies. Section 303 of the Ticket Act directed the Government Accountability Office (GAO)[6] to conduct several studies of various aspects of the DI and SSI programs: a study to assess existing tax credits and other disability-related employment incentives under Federal law;[7] a study to evaluate the coordination of the DI and SSI programs as the programs relate to individuals entering or leaving concurrent entitlement, with a focus on the effectiveness of the work incentive provisions of both programs;[8] a study of the effect of the substantial gainful activity (SGA) level on employment;[9] and a study to assess the results of SSA’s efforts to conduct disability demonstrations authorized under prior law as well as under section 234 of the Social Security Act.[10]

·      Demonstrations to Maintain Independence and Employment. Section 204 of the Ticket Act appropriated funds to support states in conducting demonstrations designed to provide health insurance and supports to working individuals with physical or mental impairments that are reasonably expected to become blind or disabled without adequate medical treatment and supports. The expectation is that the early provision of adequate health insurance and medical supports to those who would otherwise lack such supports will allow individuals with potentially disabling conditions to improve their health and functioning and retain their attachment to the labor force.

·      DI $1 for $2 Benefit Offset Demonstration. Section 302 of the Ticket Act directs SSA to conduct a demonstration and to evaluate the effects of a $1 for $2 reduction in DI payments for earnings over a specified level. This “phase-out” of benefits will make the DI benefit and incentive structure more similar to that of SSI. It eliminates the earnings cliff currently faced by DI beneficiaries, where earnings beyond the SGA level eventually result in a total loss of benefits. The Ticket Act directs SSA to conduct an evaluation of the demonstration to determine the impact of the policy on induced entry into the DI program and the impacts on federal program expenditures.

b.   Evaluation Activities and Findings

A number of evaluation activities have been initiated in response to the new programs and provisions implemented under the Ticket Act. In later sections of this report, we describe the evaluation activities and the findings to date associated with the following programs:

·      Demonstrations to Maintain Independence and Employment (Section III.B.1)

·      Medicaid Buy-In programs (Section III.B.2)

·      Medicaid Infrastructure Grants (Section III.B.3)

·      GAO study of the effectiveness of employer tax incentives to hire and accommodate people with disabilities (Section III.F.1)

·      Benefit Offset Demonstration (Section III.G.2)

·      BPAO/WIPA programs (Section III.G.3)

·      ESR/AWIC staff (Section III.G.4)

·      GAO study of the effect of SGA on employment (Section III.G.7)

·      Evaluation of the TTW program (Section III.G.8)

4.    Workforce Investment Act of 1998

a.    Intervention and Target Population[11]

In 1998, Congress passed the Workforce Investment Act (WIA) to unify a fragmented employment and training system, requiring states to provide most federally funded employment-related services through One-Stop Career Centers. The goals of WIA include improving both the quality of the workforce to sustain economic growth and productivity and competitiveness, and reducing dependency on welfare.

 

Under WIA, states are required to create workforce development plans that describe how the state will meet the needs of major customer groups, including individuals with disabilities, and show how the plans will ensure nondiscrimination and equal opportunity. Each state’s governor is required to:

·      Establish a state workforce investment board to help design, implement, and provide oversight of the new workforce development system. The state board is responsible for organizing the service system to most effectively serve customers with multiple barriers to employment, including individuals with disabilities.

·      Oversee the creation of local workforce investment boards to set policy for the local portions of the statewide workforce investment system.

·      Designate local workforce investment areas. The local workforce investment boards perform many of the same function as the state boards, but on the local level.

·      Develop one-stop delivery systems in all local areas: These delivery systems provide job search, job training, and occupational education programs.

The one-stop centers provide core services to all individuals interested in learning more about the labor market or employment opportunities. Core services include determinations of eligibility for assistance; intake and initial assessment services; job search, placement, and career counseling; provision of program performance information and program cost information; provision of vocational rehabilitation services; assistance in establishing eligibility for Welfare-to-Work activities and education and training; and follow-up services, including counseling. These services are often self-directed, but individuals who experience difficulty or prefer to work with a staff member may receive “core assisted services.”

 

Individuals who fail to find employment after the provision of core services can be eligible to receive intensive services. These services include comprehensive and specialized assessments of the skill levels and service needs; development of an individual employment plan; counseling and career planning; case management for those seeking training; short-term prevocational services; literacy activities related to basic workforce readiness; and out-of-area job search services.

 

If the core and intensive services fail to lead to a job, the one-stop can offer the customer access to classroom or on-the-job training. Training is not an entitlement under WIA, and the statute states that if there are insufficient funds to provide training to all suitable customers, preference is to be given to low-income individuals.

 

WIA specifically addresses the needs of people with disabilities by—

·      Mandating linkage of the State vocational rehabilitation systems to the State workforce investment systems

·      Streamlining the current State vocational rehabilitation systems

·      Improving delivery of services by providing more consumer choice, facilitating self-employment for disabled individuals, ensuring certain core services are available to all eligible individuals, and improving the clients' dispute resolution process

·      Mandating that one-stop centers and services be readily accessible to all Americans

WIA requires that some agencies become partners in this system, including employment services, adult education, post-secondary vocational education, vocational rehabilitation (VR), Welfare-to-Work, and Community Services Block Grant recipients. In addition, state and local boards can authorize or require other partners to participate in the one-stops. Other organizations serving people with disabilities that sometimes serve as partners at the one-stops include state or county mental health agencies, state mental retardation/developmental disabilities agencies, and community-based organizations.

 

In 1999, DOL implemented nondiscrimination provisions of WIA that prohibited denying a qualified individual with a disability the opportunity to participate or benefit from WIA services or affording such individuals lesser, different, separate or segregated opportunities, or otherwise limiting a qualified individual with a disability in the enjoyment of a right, privilege, advantage, or opportunity afforded others. Under these provisions, states must also offer individuals with disabilities services in the most integrated environment possible.

 

Hoff (2002) summarizes the nondiscrimination and comprehensive access requirements of the WIA regulations with respect to people with disabilities as follows:

·      People with disabilities have a right to use the services of the one-stop system.

·      One-stop career centers must be readily accessible to people with disabilities.

·      People with disabilities are entitled to reasonable accommodations and modifications when using one-stop services.

·      People with disabilities should not be automatically referred to the public Vocational Rehabilitation System.

WIA was due to be reauthorized in 2003; however, the House and Senate could not reach an agreement on its terms (U.S. Department of Labor 2006). As of the drafting of this report, the attempts to reauthorize the Act continue.

b.    Evaluation Activities and Findings

No formal evaluation of the impact of WIA on access to services or the employment outcomes of people with disabilities has been conducted.[12] Below, we briefly describe the general Government Performance and Results Act (GPRA) and Program Assessment Rating Tool (PART) indicators used to assess WIA performance, and also summarize the findings of two qualitative assessments of the extent to which one-stop services are accessible to and being used by people with disabilities.

 

GPRA and PART Indicators

WIA programs serving adults and dislocated workers utilize three employment-related outcome measures that states are required to report on a quarterly basis: (1) entered employment rate, (2) employment retention rate, and (3) average earnings. States are also required to provide a variety of other information about the characteristics of service users and the types of services they received, although this information is not used as part of the GPRA and PART performance assessment; only the aggregate outcomes are evaluated. Based on these outcome measures, the WIA met its 2005 targets (Exhibit 2).

Exhibit 2

WIA PART Outcome Measures

Measure

2005 Target

2005 Actual

Entered employment rate

76%

76%

Quarterly earnings of those who are employed (average of first, second, and third quarters after the exit quarter)

$3,400

$4,044

Employment retention rate

81%

82%

Source: www.ExpectMore.gov

 

The PART indicators are a subset of the 17 performance measures that state and local areas report to DOL. These measures include entry into unsubsidized employment, employment retention for six months, earnings six months after employment, attainment of appropriate credentials, and customer satisfaction. States and local areas that achieve or exceed their performance standards can receive additional funds. However, states and local areas can lose funding as well as the right to operate the WIA program if they fail to meet standards for two consecutive years (Holcomb and Barnow 2004). Two studies (described below) have suggested that the PART performance indicators act as disincentives for one-stops to serve customers with disabilities, because they are perceived to be less likely to find employment, and thereby negatively affect  One-stop employment performance metrics (GAO 2004b, Holcomb and Barnow 2004).

GAO (2004b)

The GAO (2004b) conducted a study to address the following questions:

·      What have the DOL, states, and one-stop centers done to facilitate comprehensive access to the one-stop system?

·      What relationships have the one-stops established with disability-related agencies to provide services to persons with disabilities?

·      What has DOL done to ensure that one-stops are meeting the comprehensive access requirements, and what factors have affected efforts to ensure compliance?

·      What is known about the employment outcomes of persons with disabilities who use the one-stop system?

To address these questions, GAO conducted site visits to 18 local workforce investment areas in six states, and interviewed WIA officials, one-stop staff, and officials from organizations representing people with disabilities about issues and practices for providing programs, services, and activities to individuals with disabilities. In addition, they reviewed relevant documents from the DOL, as well as the WIA statute and regulations and other relevant statutory and regulatory provisions.

 

To varying degrees, most of the sites visited by GAO had worked to implement architectural access requirements. One-stops, VR agencies, and other disability-related organizations in the community had established various relationships to provide services to persons with disabilities. GAO concluded that, although DOL has taken steps to ensure comprehensive access to one-stops by persons with disabilities, the efforts may not be sufficient because the DOL has not established a long-term plan for monitoring and enforcing the comprehensive access requirements.

 

The report also notes that the ability to evaluate the employment outcomes of people with disabilities is limited by the extent to which disability data are collected by the one-stops and the overall data collection and reporting methods used under WIA. Although DOL has issued guidance that information about disability status must be collected from job seekers, disclosure by job seekers is voluntary. Finally, the report notes that because WIA performance levels and standards are tied to incentives and sanctions, there may be disincentives for one-stops to serve people with disabilities if their employment outcomes are likely to be poorer because there is no adjustment in the performance measurement system to account for differences in the populations served, including percentage of people with disabilities among job seekers utilizing one-stop services.

Holcomb and Barnow (2004)

In 2004, the Ticket to Work and Work Incentives Advisory Panel contracted with The Urban Institute to examine the extent to which people with disabilities are served through WIA’s one-stop system and to assess the system’s capacity to serve people with disabilities, both in terms of common barriers to access as well as promising strategies to improve service delivery. The study, conducted by Holcomb and Barnow (2004), based its findings on an analysis of selected WIA program data, a review of existing literature, and discussions with a wide range of individuals knowledgeable about different aspects of the one-stop system in connection with people with disabilities.

 

Holcomb and Barnow (2004) note that the WIA program enrolls a number of people with disabilities; however, such individuals comprise a small proportion of the customers served. In program year 2002, only eight percent of individuals who exited the WIA program had identified disabilities. Data limitations precluded tracking enrollment by beneficiary status, but the authors estimate that fewer than 2,400 exiters from the adult program in program year  2002 were SSI beneficiaries. The trend in services to people with disabilities is mixed. For the adult program, the total number of exiters increased each year, but the number of exiters with disabilities only increased between 2000 and 2001. Once enrolled in WIA, the services received for customers with disabilities are close to the figures for the overall population. Finally, exiters with disabilities have lower employment and earnings than other customers who exit the program, but they have greater earnings increases from the pre-program period.

 

During the early years of WIA, anecdotal evidence suggested that the one-stops were not particularly accessible to customers with disabilities. According to Holcomb and Barnow (2004), these barriers included lack of physical access to the facilities, absence of appropriate hardware and software for customers with disabilities to access information on computers and in hard copies, and staff inexperience and lack of knowledge about how to identify and serve customers with disabilities. In addition, customers were generally expected to use the computers and printed material in a resource room on their own. This may have discouraged potential customers with disabilities from using the one-stop centers because, although special assistance was supposed to be available, it was not always offered, at least in part because staff found it difficult to identify people with disabilities with need for assistance.

 

DOL and SSA established several grant programs designed to mediate these and other barriers for people with disabilities, including:

·      Customized Employment Grants funded by the DOL Office of Disability Employment Policy

·      Disability Program Navigators (DPN) funded jointly by the DOL Employment and Training Administration and the SSA Office of Program Development and Research

·      Work Incentive Grants (WIGs) funded by DOL Employment and Training Administration

We discuss each of these programs separately in Section III.D below.

B.    Centers for Medicare and Medicaid Services

1.    Demonstrations to Maintain Independence and Employment

a.    Intervention and Target Population

Under the authority of the Ticket Act, CMS is conducting a set of demonstrations targeting workers who have physical or mental impairments that, without medical assistance, will likely result in withdrawal from the workforce. These demonstrations allow states to provide a health care package similar to Medicaid and a range of employment supports.

 

The Ticket Act identified the target recipients for the demonstrations as individuals with a specific physical or mental impairment who are reasonably expected, but for the receipt of medical care, to become blind or disabled as defined by SSA (Ticket Act, Section 204b). It was designed to offer medical care to those individuals who would otherwise need to leave the labor force and apply for disability benefits in order to gain access to health insurance and health supports.

 

The first two rounds of solicitations for state proposals to design and implement Demonstrations to Maintain Independence and Employment (DMIE) occurred in 2000 and 2002. These solicitations focused on working individuals with chronic or progressive conditions whose conditions had not progressed to the point where they would meet SSA’s medical definition of disability. The four proposals submitted focused on progressive impairments such as HIV/AIDS, multiple sclerosis, and mental illness. Only the District of Columbia successfully established a viable program under these early solicitations.

 

In response to the lack of interest in the DMIE on the part of states, during subsequent grant cycles, CMS clarified the definition of the target population to include any worker with a chronic or progressive condition who is at risk of withdrawing from the labor force and applying for Social Security cash benefits. In other words, it included individuals who might meet SSA’s medical definition of disability, but would not meet the work criteria for disability. CMS also provided states substantially more flexibility to develop a coordinated set of medical and vocational services to investigate the question of whether a program of medical assistance and other supports can forestall or prevent the loss of employment and independence due to a potentially disabling and medically determinable physical or mental impairment. Four additional states were awarded DMIE grants in the later rounds. The nature and status of the DMIE initiatives are shown in Exhibit 3.

b.    Evaluation Activities

CMS contracted with the Research Triangle Institute (RTI) to evaluate the DMIE programs in the District of Columbia and Mississippi. However, because of low enrollment in the Mississippi program, it was discontinued. Thus, the RTI report evaluated only the program in the District of Columbia (Haber and Osber 2005).

 

CMS contracted with MPR to conduct a national evaluation of the four states with viable programs established under the later solicitations (Kansas, Minnesota, Texas, and Hawaii). In contrast to the DMIE program in the District of Columbia, which did not plan for an experimental or quasi-experimental evaluation design, these states are operating programs that incorporate random assignment designs. Beginning in 2008, MPR will produce annual reports on the DMIEs, and in 2010, MPR will submit a final report to CMS on the national findings. The MPR evaluation will focus on estimating the impact of the initiatives on employment, access to and use of health care, and health status. The evaluation will be based on data collected by the states via survey, and maintained in state administrative systems. It will also use Medicaid claims data from CMS, earnings and disability program participation information from SSA administrative data, and qualitative information gained through site visits, interviews, and document reviews.


Exhibit 3

Selected Features of the DMIE Initiatives

State *

Enrollment Start Date

Target Population

Target and Actual Enrollment **

District of Columbia

 

September 2002

Employed individuals who are HIV-infected, with incomes at or below 300% of poverty, assets below the Medicare Buy-In resource limit ($4000 for individuals; $6,000 for couples), and are otherwise ineligible for Medicaid

Target: Treatment: 420; Control: NA

Actual: Treatment: 399

Kansas

 

April 2006

Employed individuals enrolled in the Kansas Health Insurance Association high-risk pool

 

Target: Treatment: 200;

Control: 200

Actual: Treatment:189; Control: 149

Minnesota

 

December 2006

 

Employed individuals with serious mental illness from five counties

 

Target: Treatment: 1,125; Control: 375

Actual: Treatment: 330; Control: 103

Texas

 

April 2007

Employed adults enrolled in the Harris County Hospital District medical program for uninsured residents with either (1) severe mental illness or (2) behavioral health diagnosis co-occurring with a physical diagnosis

Target: Treatment: 800

Control: 625

Actual: Treatment: 351; Control: 280

 

Hawaii

 

June 2007

Employed adults with diabetes who reside in the city and county of Honolulu

Target: Treatment: 356

Control: 178

Actual: Treatment: 0; Control: 0

 

Sources: Department of Health and Human Services, Centers for Medicare and Medicaid Services. Program Announcement: Demonstration to Maintain Independence and Employment. Issued March 17, 2006; Gimm and Weathers (2007); and Gilman et al. (2008).

 

 * In April 2002, Mississippi implemented a DMIE program for people with HIV infection, but because of low enrollment, transitioned all enrollees out of the demonstration by September 30, 2007. Rhode Island and Louisiana also were approved under earlier solicitations, but neither state implemented a program. Iowa received a planning grant in May 2008 to assess outcomes of a program to help adults with mental illnesses who have been recently discharged from correctional facilities.

 

**Actual enrollment figures for the District of Columbia are as of December 2006. Enrollment figures for Kansas, Minnesota, Texas, and Hawaii are as of December 2007.

 

The later-round grantees were also required to include independent evaluations of their programs as part of their grant activities. Each state has contracted with its own evaluator to conduct these activities.


c.     Progress to Date and What Will Be Learned in the Future

The DMIE program in the District of Columbia covers HIV-positive individuals with incomes up to 300 percent of the federal poverty level who work at least 40 hours per month or 120 hours over a three-month period. It differs from the demonstrations implemented under the subsequent grant cycle because it does not provide enhanced case management services to DMIE enrollees and the services do not focus on employment support. In addition, most of the enrollees were already receiving adequate HIV medications and HIV-related care through the AIDS Drug Assistance Program and Ryan White clinics, so the demonstration primarily represented a change in payment source (Haber and Osber 2005). 

 

The RTI evaluation of the District’s program focused on program implementation and operations. It found that within two years of its inception, the program had reached its enrollment cap of 420 individuals and a waiting list for the program was instituted. Program participants and case managers noted that enrollment in the DMIE improved access to treatment for non-HIV-related conditions because participants received coverage for the full range of Medicaid services. The study did not assess the impact of the program on health or employment outcomes.

 

Because of its design, the demonstration in the District of Columbia is unlikely to provide information on the impact of providing improved access to health care on the employment of individuals with HIV/AIDS. It is possible that the RTI evaluators could use individuals on the waiting list as a control group and collect information about employment outcomes from both groups; however, we have no information about RTI’s plans for the District of Columbia evaluation, or the potential for producing impact estimates. The demonstration in the District of Columbia (and other states) is, however, likely to produce useful information about the demand for and feasibility of offering health coverage combined with intensive case management to workers at risk of leaving the labor market because of a disability.

 

The MPR national evaluation is still in its early stages. As it is based on a random assignment design, it is expected to produce rigorous estimates of the impact of the DMIE interventions on health care access and utilization, health status, employment, and SSI/DI program participation and benefits. A potential limitation is that the small sample sizes or low levels of enrollment at some sites could limit the ability of the evaluation to detect some impacts. The first interim evaluation findings are expected to become available in early 2009, and the final evaluation findings approximately one year later.

2.    Medicaid Buy-In Programs

a.    Intervention and Target Population

The 1999 Ticket to Work and Work Incentives Improvement Act (Ticket Act) expanded state authority originally granted under the 1997 Balanced Budget Act (BBA) to provide Medicaid coverage to workers with disabilities who, because of income and assets, would not otherwise qualify for Medicaid coverage. Both the BBA and the Ticket Act authorized states to charge premiums for this coverage, and thus, allow working individuals with disabilities to “buy in” to Medicaid.

 

The program was developed in response to evidence that many individuals with disabilities face the choice of private employer-based insurance, which might not cover needed services, prohibitively expensive coverage in the private market, and the being uninsured. As a result, they often rely on public health insurance. However, because of the link between health insurance (Medicaid and Medicare) and eligibility for disability benefits (SSI and DI), which require that the individual not be working at substantial levels, this system creates a strong work disincentive. The Buy-In programs are designed to break the link between benefits and health care and thus remove the work disincentive inherent in the public programs.

 

As of July 2008, 35 states were operating a Medicaid Buy-In program. Across all states, nearly 190,000 working-age individuals with disabilities have enrolled in a state Medicaid Buy-In program at some point between 1997 and 2006 (Gimm et al. 2008).

b.    Evaluation Activities

CMS contracted with MPR to document the outcomes of the program. MPR has produced multiple reports and policy briefs that describe enrollment trends, participant characteristics, earnings, and Medicaid expenditures.

 

The annual reports for 2002, 2003, and 2004 (Ireys et al. 2003, White et al. 2005, Black and Ireys 2006) were based on summary data submitted by states derived from their various databases. The state-submitted data allowed CMS to track participation in the Buy-In program, however, the states did not have access to SSA or Medicare program data, and therefore lacked information that could be important for program evaluation, such as the participant’s impairment, SSI and DI benefits, and Medicare expenditures. In addition, states could submit earnings data only from their Unemployment Insurance program systems, which are not as comprehensive or as accurate as federal earnings data available from IRS records.

 

To address these shortcomings, CMS worked closely with SSA to modify existing interagency data-use agreements such that person-level data from Medicare, Medicaid, and SSA, including IRS records available to SSA, could be used to evaluate the program.[13] This approach provides more complete and consistent data among states and reduces their reporting burden (Liu and Ireys 2006). The states provided CMS with individual-level enrollment data through “finder files” that provided CMS with identifying information about each Buy-In participant (for example, social security number, date of birth). MPR then linked the finder files to Social Security, Medicare and Medicaid data sources. The combined data file has information on Buy-In participants who had been an SSI or DI beneficiary between 1996 and 2005, including data on their ethnic background, primary disabling condition, benefit receipt, earnings, and dates of SSI or DI program enrollment as well as Medicare and Medicaid expenditures. MPR used these data in subsequent reports to present a snapshot of the program participants in 2005 (Ireys et al. 2007), report the characteristics of high earners (Gimm et al. 2007), compare annual earnings among first-time Buy-In participants before and after their initial enrollment in the program (Liu and Weathers 2007), and assess the determinants of employment and earnings (Gimm et al. 2008).

 

In addition to the MPR studies, there have been several smaller scale research activities. The Office of the Assistant Secretary for Planning and Evaluation contracted with George Washington University to assist stakeholders in designing and implementing Medicaid Buy-In programs. They developed several reports based on the experiences of nine “early implementer states.” While not formally “evaluations” of the program, these reports describe the program design and how these designs affect enrollment, costs, and state fiscal exposure. The reports also describe the programmatic, fiscal, and political context in which these design decisions were made (Folkemer et al. 2002a and 2002b, Jensen et al. 2002).

 

Many states have conducted evaluations of their Buy-In programs using funds from their CMS Medicaid Infrastructure Grants, including, for example, California (Jee and Menges 2003), Iowa (Iowa Department of Human Services 2005), Maine (Clark et al. 2003), , and Wisconsin (APS Healthcare 2003). On behalf of the Ticket to Work and Work Incentives Advisory Panel, Goodman and Livermore (2004) reviewed the information that existed at that time regarding evidence of the effectiveness of Medicaid Buy-In programs in promoting the employment of people with disabilities.

c.     Findings

The major findings from the MPR studies noted above include the following:

·      Program design and enrollment

Ø    There is considerable variation across states in the eligibility criteria and cost sharing policies used by Buy-In programs (Folkemer et al. 2002a, Goodman and Livermore 2004, Ireys et al. 2007). These variations have consequences for the ability of particular Buy-In programs to promote employment, especially among DI beneficiaries (Goodman and Livermore 2004, Black and Ireys 2006).

Ø    States have adjusted the eligibility requirements and operational features of their Buy-In programs over time for various reasons, including to strategically expand or restrict the pool of potential applicants; to allow current participants to earn more money; to alter the structure of premium payments for purposes of increasing revenues or increasing participation; and/or to alter grace periods in order to make it easier or more difficult for participants to stay enrolled if they lose their jobs (Ireys et al. 2007).

Ø    There is large variation in enrollment among states, from a low of one participant in South Dakota’s newly implemented program, to more than 9,000 in Massachusetts in 2006 (Gimm et al. 2008).

Ø    Certain program features have been shown to be associated with participant employment outcomes. Shorter grace periods (work stoppage provisions), work verification requirements, and higher earned income limits were positively associated with both the likelihood of employment and the level of earnings (Gimm et al. 2008).

 

·      Participant characteristics

Ø     Most (71 percent) Buy-In participants in 2006 were DI beneficiaries (Gimm et al. 2008).

Ø     The most common primary disabling condition of Buy-In participants is severe mental illness and other mental disorders. About 30 percent of all Buy-In participants in 2006 had these conditions (Gimm et al. 2007).

Ø     Most 2006 participants were white (76 percent) and over age 40 (58 percent) (Gimm et al. 2008).

 

·      Health care costs

Ø    Average Medicaid expenditures per member per month (PMPM) were $1,076 among Buy-In participants in the 22 states with a program as of 2002; average PMPM Medicare expenditures were $391.

Ø    Medical expenditures vary by type of impairment. For example, PMPM Medicaid and Medicare combined expenditures in 2002 were $1,042 among Buy-In participants with musculoskeletal disorders, and $1,695 among those with mental retardation.

Ø    Medical expenditures also vary by receipt of federal disability benefits. PMPM Medicaid and Medicare combined expenditures in 2002 ranged from $1,179 among Buy-In participants receiving only SSI benefits to $1,491 among those receiving only DI benefits.

 

·      Employment and earnings

Ø    In 2006, 69 percent of Buy-In participants were employed and had earnings during that year. The likelihood of employment was significantly associated with age (negative), SSI/DI program status (negative), certain disabling conditions, and as noted above, certain features of the programs (Gimm et al. 2008).

Ø    Broadly speaking, the data indicate that Buy-In participants engage in levels of employment that allow them to maintain their eligibility for DI benefits. Of the 43 percent of Buy-In participants nationally who had earnings reported in state unemployment insurance data systems, about 7 in 10 had earnings below the SGA level (Black and Ireys 2006).

Ø    Approximately 40 percent of new Buy-In participants increased their earnings, with adjustments for inflation, after they enrolled in the program. The median inflation-adjusted increase in their annual earnings was $2,582. This is substantial relative to the average annual pre-enrollment earnings of $4,844 (Liu and Weathers 2007).

Ø    The rate of earnings growth differed based on participant characteristics and across states: 65 percent of participants under age 21 increased their earnings and the percentage steadily declines to 47 percent for those ages 21 to 44, 33 percent for those ages 45 to 64, and 30 percent for those ages 65 and older (Liu and Weathers 2007).

Ø    The share of people with increased earnings post-enrollment differs substantially across states, from 58 percent in Nebraska to 20 percent in New Mexico. These differences may be in part the result of factors such as state labor markets and policy environments. However, they may also be due to state-specific program features that affect the characteristics of participants (Liu and Weathers 2007).

 

The goal of the Medicaid Buy-In program is to increase employment among people with disabilities by eliminating the risk that employment will cause an individual to lose access to public health insurance. Although studies indicate that Buy-In enrollees increase their earnings, on average, after enrollment in the Buy-In, no strong evidence exists that these programs caused the increase in employment. In rigorous experimental evaluations of employment programs for other populations, it is common to find that control group as well as treatment group earnings increase because those who volunteer to participate are seeking to increase their earnings, and many continue to do so, regardless of whether they are assigned to treatment or control. Hence, it might be that increases in earnings observed for Buy-In participants would have occurred, in whole or in part, in the absence of the Buy-In.

 

Conducting a rigorous evaluation of the impact of Buy-In programs on employment is made difficult by the absence of data on suitable comparison groups (Goodman and Livermore 2004). As the programs were implemented statewide at a given point in time, within-state comparison groups are not available. It is possible that a study design incorporating pre-post comparisons across states with and without a Buy-In over time might detect impacts of the program on employment, but the large degree of variation across states, both in the nature of their Buy-In programs and in factors that likely affect labor market outcomes in general, would make the detection of relatively small impacts unlikely.

 

An important outcome of the MPR studies is that they led to establishment of an interagency agreement between CMS and SSA for linking administrative data from the two agencies for purposes of program and policy evaluation. Although MPR has used the linked data mainly for descriptive analyses, linking data across these programs holds promise for conducting other future program evaluations.

3.    Medicaid Infrastructure Grants

a.    Intervention and Target Population

The Medicaid Infrastructure Grant (MIG) program was established by the Ticket to Work and Work Incentives Improvement Act of 1999 and is administered at the federal level by CMS.

 

Congress authorized the MIG program for 11 years beginning in fiscal year 2001. States apply for funding annually but there is no maximum to the number of years for which a state is eligible. States are expected to use the funds first to change the Medicaid program by establishing Medicaid Buy-In programs and other Medicaid enhancements that would support employment (such as personal assistance services). As states achieve these goals, they can use the funds more broadly to “promote linkages between Medicaid and other employment-related service agencies and to develop a comprehensive system of employment supports for people with disabilities” (CMS 2007).

 

States have used the grants primarily to provide education and outreach for Medicaid Buy-In programs and other work incentives; conduct needs assessments and cost estimates for expanding personal assistance services; conduct ongoing monitoring and evaluation of Buy-In programs; convene leadership councils; and build interagency relationships. As of July 2008, 40 states had MIGs.

b.    Evaluation Activities

The primary outcome of the MIGs is the establishment of the Medicaid Buy-In programs. The evaluations of the Buy-In programs are described in the previous section. There is no federal evaluation of other MIG activities. Grantees are, however, required to report grant-related activities to CMS on a quarterly basis, and innovative practices are shared among states via two technical assistance centers—the National Consortium for Health Systems Development and the Center for Workers with Disabilities. In addition, the Law, Health Policy and Disability Center at the University of Iowa’s College of Law, through a grant from the Rehabilitation Services Administration in the U.S. Department of Education, provides ongoing reports about MIG activities.[14]

c.     Progress to Date and What Will Be Learned in the Future

Although there have been no evaluations and no formal identification of “best practices,” CMS included in the request for MIG proposals several steps to facilitate information sharing among states, including the following:

·      MIG grantees must provide state-to-state technical assistance either by joining one of the two technical assistance networks or developing their own plan.

·      Grantees must attend the annual technical assistance conferences.

·      Grantees must report their activities to an on-line reporting system designed to “facilitate the sharing of technical assistance and contacts”.

·      States are required to post all technical assistance products purchased with grant funding and make them available for use by other states.

·      CMS provided funds to establish a northeast regional network of MIG grantees, which meets regularly

C.   Department of Education

1.    Federal/State Vocational Rehabilitation Services

a.    Intervention and Target Population

Title I of the Rehabilitation Act of 1973 authorizes a federal/state Vocational Rehabilitation (VR) program to provide services to persons with disabilities so that they may prepare for and engage in gainful employment. It provides federal funds to help people with disabilities become employed, more independent, and integrated into the community. The federal funds are chiefly passed to state VR agencies that directly provide services such as guidance, counseling, and job placement, as well as purchase services such as therapy and training from other providers. In 2007, the U.S. Department of Education provided about $2.9 billion in VR grants to the states and territories based on a formula that considers the state’s population and per capita income.[15] The Act generally requires states to match federal funds at a ratio of 78.7 percent federal to 21.3 percent state dollars. Each state and territory designates a single VR agency to administer the VR program, except where state law authorizes a separate agency to administer VR services for individuals who are blind (NCD 2007b).

 

Section 101(a)(5) of the Act requires VR agencies to give priority to serving individuals with the most significant disabilities, many of whom benefit from the specialized services provided under VR (about 86 percent of the individuals served are those with significant disabilities).

b.    Evaluation Activities

There have been several efforts to evaluate the VR program:

·      Annual GPRA and PART indicators

·      The Longitudinal Study of the Vocational Rehabilitation Services Program (LSVRSP)

·      Several GAO studies


GPRA and PART Indicators

The U.S. Department of Education (ED) collects information about all individuals who exit each state VR agency’s program during a particular fiscal year, as reported by each state VR agency. The record for individuals exiting the program includes information such as whether or not they became employed, the weekly earnings and hours worked for individuals if they exited the VR program with employment, the types and costs of services they received, and demographic factors, such as impairment type, gender, age, race and ethnicity, public benefit receipt, and income from work at the time of application. ED also collects summary information on agency expenditures in a number of categories from each state VR agency. ED tracks seven types of case closures, which can be collapsed into individuals who, 1) exited without employment during the application phase; 2) exited without employment with limited services; 3) exited without employment after receiving services under an employment plan; and 4) exited with at least 90 days of employment after receiving services under an employment plan.

 

With this information, ED reports the following measures on the PART evaluation:

·      Percent of state VR agencies (excluding VR agencies for the blind) that assist at least 55.8 percent of individuals receiving services to achieve employment

·      Percent of state VR agencies (excluding VR agencies for the blind) that assist at least 85 percent of individuals with employment outcomes to achieve competitive employment (employment in an integrated setting at/or above the minimum wage)

·      Percent of state VR agencies (excluding agencies for the blind) for which at least 80 percent of the individuals achieving competitive employment are individuals with significant disabilities

·      Percent of general and combined state VR agencies that demonstrate average cost per employment outcome between $6,000 and $16,500

·      Percent of general and combined state VR agencies that demonstrate an average annual consumer expenditure rate of at least 83 percent

LSVRSP

In 1992, RSA initiated a study to assess the performance of the federal/state VR program on a broad spectrum of outcome measures. Between 1995 and the end of 1999, the Research Triangle Institute (RTI) collected data on a nationally representative sample of more than 8,500 state VR consumers who were followed over a period of three years from application to closure to post-service outcomes.[16] The data collected include both survey data and data abstracted from case files. The study’s database has much more detail on VR consumers than what is available through RSA administrative data regarding client characteristics, services received, and work outcomes.

 

RTI collected data through a series of surveys with participants and a process of abstracting information from VR records. The baseline interview conducted at the time of entry into the study obtained information on work history, functioning, vocational interests and attitudes, independence and community integration, and consumer perspectives on their VR participation. Administration of a follow-up survey for three subsequent years varied according to the individual’s stage in the VR process at the time of interview. The records abstraction, which was conducted when the consumer entered the study, then continued quarterly until program exit, generated information on consumer characteristics and detailed information about services.

 

In addition to collecting data about the consumers, RTI collected data about the agencies serving the study participants through mail surveys of office managers, counselors, and other office staff, using a form regarding state policies and procedures. These instruments were administered at initiation and termination of the study’s data collection activities, with annual updates from the local office manager surveys (Hayward and Schmidt-Davis 2003a).

 

The study was designed to evaluate the VR program based on a conceptual framework where outcomes (earnings, employment, consumer satisfaction, community integration, and other economic and non-economic outcomes) are a function of the following:

·      The characteristics of applicants and consumers, including demographics, functional level, work history and earnings, interests and motivations, and type and severity of disability.

·      Services and service costs, including equipment, adaptive devices, supplies, professional services, and other RSA resources such as counselor and staff time.

·      Local economic and population characteristics, including the urban or rural nature of the area, the availability of jobs, and the prevalence of work disability in the state population.

·      The organizational culture and resources in the local agency office, such as internal organization and management philosophy. The organizational culture is viewed as an influencing factor on both quality of services and resulting service outcomes.

The design did not include a method to determine what employment outcomes would have been in the absence of VR services, presumably because it would be problematic to identify a suitable comparison group.

GAO 2007

In 2005, GAO reported that, based on the RSA 911 data, the employment rates for VR clients, particularly for SSA beneficiaries, varied substantially among VR agencies. As many as 68 percent of SSA beneficiaries exited VR with employment in some states compared to as few as 9 percent in other states (GAO 2005). Based on these findings, Congress requested that GAO conduct a study to determine what might account for the wide variation across states in the employment outcomes of VR consumers who are DI and SSI beneficiaries. The study was not intended to be an impact analysis. It was designed to examine the extent to which (1) differences in VR agency outcomes for SSA beneficiaries continued over several years and across different outcome measures, (2) differences in VR agency outcomes were explained by state economies and demographic traits of the clientele served, (3) differences in VR agency outcomes were explained by specific policies and strategies of the VR agencies, and (4) ED’s data allowed for an analysis of factors that account for differences in individual-level (as opposed to agency-level) outcomes (GAO 2007).

c.     Findings

GPRA and PART Indicators

The PART assessment concluded that the VR program is performing adequately based on the annual performance goals.[17] The performance measures focus on whether certain short-term objectives have been achieved, but because there is no counterfactual, they do not indicate the extent to which the program contributed to that achievement. They serve purely as descriptive performance benchmarks.

LSVRSP

RTI developed four primary evaluation reports (Hayward and Schmidt-Davis 2002, 2003a, 2003b, and 2005) with descriptive statistics about the characteristics of VR clients (types of disabilities, functional status, receipt of financial assistance etc), the services they use and their outcomes (employment status, earnings, health insurance, community integration, independence, etc).[18] The evaluators used basic univariate and multivariate approaches to address five questions. We briefly summarize their findings regarding each question.

1.      What short- and long-term economic and non-economic (for example, independent living, community integration) outcomes do VR applicants and consumers achieve as a result of their participation in VR?

On all measures of economic outcomes, VR consumers who achieved a competitive employment outcome fared better than those who achieved a non-competitive employment outcome and those who exited without an employment outcome. For example, at the third annual follow-up, 78 percent of persons exiting into competitive employment were still working, compared with 70 percent of those exiting into non-competitive employment, 37 percent of those who received services but exited VR without an employment outcome, and 40 percent of those who were eligible but dropped out before receiving VR services.

 

On measures of non-economic outcomes, consumers with competitive employment outcomes reported less frequent use of a variety of services and were less likely to report that their disability restricted their ability to participate fully in social and community activities compared to consumers with other outcomes. They were also more likely to believe that VR contributed to their success.

2.      What characteristics of individuals with disabilities affect their (1) access to and receipt of VR services, and (2) short- and long-term outcomes?

Among all VR applicants, 84 percent were offered VR services. Of those who were offered services, 88 percent received services. The remaining 12 percent chose to exit the program prior to receiving services. Of those who received services, at the end of the data collection period, 17 percent were continuing to receive VR services, 45 percent had achieved employment outcomes, and 21 percent had exited VR after services without an employment outcome. Among those with an employment outcome, 78 percent were working at jobs in the competitive labor market. The remaining 22 percent held non-competitive jobs.

 

Using a multivariate approach, RTI found that the probability of each of these outcomes was positively or negatively correlated with a number of consumer characteristics, such as type of disability, psychosocial characteristics, reliance on other government programs for support, work history, career interests, motivation, and demographic characteristics.

3.      To what extent does receipt of specific VR services contribute to successful consumer outcomes?

Overall, consumers who received VR services averaged 12 services during their participation in the program. The most frequent services delivered to consumers were medical/physical functional evaluation (62 percent), cognitive/psychological assessment and services (35 percent), employment development services (33 percent), postsecondary education (33 percent), and miscellaneous support services, such as transportation or maintenance (38 percent). The type of services that a client received was a function of disability type and employment goal.

 

Controlling for differences in consumer characteristics, a number of specific VR services contributed to the likelihood of an employment outcome and a competitive employment outcome. These included job placement, supported employment, on-the-job training, independent living services, driver training/licensing, tools/uniforms/equipment and business/vocational training. Receipt of psychosocial counseling was associated with a decrease in the probability of an employment outcome. The consumer’s rating of the quality of the consumer-counselor relationship was also a significant determinant of employment outcomes.

4.      In what ways and to what extent do local environmental factors influence VR consumer services and outcomes?

Each VR agency operates in a different environment in terms of general population characteristics, the employment environment, and community resources. Although some aspects of the local environment affected the likelihood that consumers would receive certain services and achieve employment outcomes, the identified features of the environment were much less important than consumer characteristics as determinants of these outcomes.

5.      In what ways and to what extent do the operations, resources, and organizational climate of VR agencies influence consumer services and outcomes?

Local offices vary in terms of caseloads, financial resources, office policies and practices, and office culture. While a number of client characteristics were associated with the type or amount of services a consumer received, office factors were the primary determinant.

 

The LSVRSP study has several limitations. Although the study controlled for differences in individual characteristics of VR clients and found some indication of the relationships among services and outcomes, the study design did not include a comparison or experimental control group. Thus, the evaluators could not conclude that specific services caused particular outcomes as there is no information about what would have occurred in the absence of VR services.

 

Another limitation is that, because the study findings reflect the study’s data collection period, which occurred between 1995 and the end of 1999, they do not reflect recent changes that might have affected VR services and outcomes, such as WIA and the Ticket Act.

 

One of the strengths of the LSVRSP is the rich data that was collected. Cornell University has made these data easily accessible and researchers have made use of the data for specific studies. For example, Capella-McDonnall (2005) identified predictors of competitive employment outcomes for blind or visually impaired VR consumers; Stapleton and Erikson (2005) analyzed the impact of SSI and DI recipiency on employment outcomes; and Homa (2005) analyzed the impact of vocational evaluation on VR outcomes.

GAO 2007

The GAO (2007) study found that agencies varied widely across different outcome measures including employment rates and annual earnings. Focusing on earnings outcomes in a multivariate model, the study found that much of the variation in state agency earnings outcomes could be explained by state economic conditions and the characteristics of the agencies’ clients. Together, state unemployment rates and per capita income levels accounted for roughly one-third of the variation across state agencies in the share of SSI and DI beneficiaries with earnings during the year after exiting VR. The demographic profile of clients also accounted for some of the variation across states in terms of the likelihood that SSI and DI beneficiary VR clients would leave the disability rolls—women, older clients, and clients with mental illness or visual impairments were less likely to leave the disability rolls.

 

A few agency practices appeared to be associated with positive earnings outcomes. For example, state agencies with a higher proportion of state-certified counselors had better employment outcomes.

 

GAO was not able to determine which factors might account for differences in earnings outcomes at the individual level, in part because RSA’s administrative data lacked important information. Although the RSA administrative data has extensive client-level data, it does not systematically collect data that research has linked to work outcomes such as detailed information on the severity of the client’s disability—data that some state agencies independently collect for program purposes. Further, other key data are self-reported and might not be verified by state agencies.

2.    Projects with Industry

a.    Intervention and Target Population

The Projects with Industry (PWI) program was designed to create and expand job and career opportunities for individuals with disabilities in the competitive labor market by engaging the participation of business and industry in the rehabilitation process. It is funded and administered by the Department of Education, Office of Special Education and Rehabilitative Services (OSERS).

 

Since its inception in 1968, the program has provided grants to nonprofit organizations, educational institutions, private corporations, state VR agencies, Indian tribes, labor unions, trade associations, and other organizations to help individuals with disabilities obtain competitive employment.[19] All grantees are required to provide job development, job placement, and career advancement services as well as any required support services. Most PWIs also provide a range of optional services such as job readiness training, job skill training, and post-placement assistance.

 

PWIs must also establish Business Advisory Councils (BACs) that include representatives of private industry, organized labor, individuals with disabilities and their representatives, state VR agency representatives, and others. BACs are intended to identify job and career availability within the community, identify the skills needed to perform those jobs, and prescribe for individual PWI participants an appropriate training or placement program. Some BACs also raise the general awareness of the employment potential of persons with disabilities and of the PWI project (Tashjian 2003).

 

In 2006, there were 79 PWI programs that served a total of 7,512 clients (U.S. Department of Education n.d.). PWIs vary considerably in terms of the types of individuals they serve. Participants are referred to individual projects from a wide variety of sources; traditionally a majority of participants are referred from the VR program. Eligibility criteria for PWI services are the same as for the federal/state VR program (Tashjian 2003).

b.    Evaluation Activities

There have been several efforts to evaluate the PWI program:

·      Annual GPRA and PART indicators

·      An evaluation conducted by RTI (Tashjian 2003)

·      Studies conducted on specific programs, of which we describe the methods and findings of one (Fabian 2007) that assessed the employment outcomes of a very large sample of program participants

GPRA and PART Indicators

PWI grantees are required to submit primary and secondary performance indicators to OSERS annually. The primary indicators include placement rate into competitive and average increase in weekly earnings. Secondary indicators include cost per placement; placement rate among people with significant disabilities; and placement rate among those who were previously unemployed. These indicators are measured against a target established by OSERS. Targets are set based on expected project outcomes and take into consideration past program performance and factors such as grantee experience. For example, new grantees are expected to produce fewer outcomes in the first year of the grant. To receive continuation funding for the third or any subsequent year of a PWI grant, a PWI project must pass both primary indicators and at least two of the three secondary indicators.

RTI Evaluation

During 2000-2003, the U.S. Department of Education contracted with Research Triangle Institute (RTI) to “examine the role and performance of the PWI program as one component of the broader set of employment-related services available to individuals with disabilities, with a particular focus on the extent to which PWI projects fulfilled their intended goal to create and expand job opportunities for individuals with disabilities at the project level” (Tashjian 2003).[20] To address this question, the RTI report described the following: structure and operations of PWI projects; the relationship of the PWI program to the employment community; coordination between the PWI and federal/state VR programs; the validity and reliability of compliance indicators; the characteristics of individuals served by PWIs including their demographic and disability characteristics; types of PWI services received; and, employment outcomes obtained.

 

RTI gathered data through two mechanisms: (1) a survey of all PWI projects funded as of October 2000, and (2) on site data collection at 30 randomly selected PWI projects. The on-site data collection included interviews and focus groups with BAC members, local VR staff, employers, and Workforce Investment Board chairs. The onsite data collection also included abstracting data from a randomly selected 20 percent of case files (584 individuals) whose participation in the PWI program ended in FY 2001. RTI weighted these data to represent the universe of PWI users and used the data to describe the characteristics, services, and outcomes for all former PWI participants. They compared these PWI measures to measures for VR consumers based on RSA-911 data (case service records) from those VR districts most proximate to the 30 selected PWI projects (Tashjian 2003).

Fabian (2007)

One recent study (Fabian 2007) assessed the employment outcomes of participants in a particular PWI program, the Marriott Foundation’s Bridges from School to Work Program (Bridges). The goal of the Bridges program is to provide competitive paid work experience for special education youth who are transitioning out of high school to adult life. Since the program’s inception in 1990, Bridges has served more than 10,000 youth in six major urban locations: San Francisco, Los Angeles, Chicago, Atlanta, Philadelphia, and Washington, DC. Bridges offers a standardized one-semester vocational intervention program consisting of three phases: (1) career counseling and job placement; (2) paid work experience with training and support provided by a Bridges employer representative; and (3) follow-along support and tracking of student participants.

 

Fabian (2007) assessed the postsecondary employment outcomes of transition-age youth with disabilities who participated in Bridges from 2000 to 2005. The study explored factors that distinguished the students who obtained employment and the nature of the jobs that are secured using a database of 4,571 students enrolled in the Bridges program containing demographic information (race, age, gender, disability), background information (special education services received, whether the student received SSI), previous employment experience, and whether the student had identified a post-school career goal.

c.     Findings

GPRA and PART Indicators

According to the annual GPRA indicators, on average, PWI projects were close to meeting the employment targets during 2004 and 2005, and exceeded the earnings increase targets in each year from 2003-2005 (Exhibit 4). The projects fell short of the employment targets during 2003.

Exhibit 4

 

GPRA Performance Measures for the Projects with Industry Program

 

 

2003

2004

2005

GPRA Indicator 

Actual

Target

Actual

Target

Actual

Target

Percentage of individuals served who were placed into competitive employment

54.2%

62.4%

61.5%

62.7%

51.9%

63.0%

Percentage of previously unemployed individuals served who were placed into competitive employment

54.0%

63.0%

65.5%

64.0%

62.4%

65.0%

Average increase in weekly earnings

$242

$231

$247

$233

$253

$238

Cost per placement (federal grant funds only)*

NA

NA

$3,139

NA

$3,014

NA

Source:  U.S. Department of Education (2006).

* The PWI data collection instrument was revised for FY 2005 reporting which resulted in a significantly lower reported placement rate compared to previous years.

 

RTI Evaluation (Tashjian 2003)

We briefly summarize the key findings of the RTI evaluation below.

 

Structure and Operations. Individual PWI projects vary considerably with respect to scope, type of consumers served, priority areas addressed, types of services provided, and other factors. These variations tend to reflect the availability of other resources in the community and other programs available at the grantee organizations.

 

Relationship of the PWI Program to the Employment Community. PWI projects also varied in the extent to which the BACs fulfilled their statutory requirement. The vast majority of BACs, according to survey data, pursued their legislated functions of identification of job openings and career availability (99 percent), and identification of necessary skills for those jobs (92 percent); somewhat fewer BACs actually prescribed participant training or placement programs (77 percent). Many PWIs had engaged businesses through other mechanisms such as formal agreements or long-standing relationships with one or more area employers or staffing agencies. The evaluators questioned whether the statutory role of the BAC was reasonable given the voluntary nature of BAC services, turnover in BAC membership, and the infrequency with which most BACs convene.

 

Coordination Between the PWI and Federal/State VR Programs. The relationship between VR and the PWI also varies considerably. While some projects obtained 100 percent of their participants through VR agency referrals, others served few or no VR consumers, and still others (school-to-work projects) served individuals with disabilities prior to their receipt of VR services. Individuals served by both programs tend to receive placement assistance only through the PWI program, with any training usually provided or at least funded by VR. PWI programs tend to emphasize job placement and follow up with participants and employers. Employers identify this follow-up as the reason they believe the PWI program is a better source of qualified employees compared to the VR services program or private employment agencies.

 

Validity and Reliability of Compliance Indicators. About one-quarter of the projects did a poor job in collecting and maintaining the data required to report values for the performance indicators. The most difficult aspect was tracking former participants’ employment status over time.

 

Demographic and Disability Characteristics of Individuals Served by PWIs: RTI found that individuals served by the PWI program, averaged over all grantee sites, do not differ much from those served by VR. Although each PWI project typically serves a specific subset of the disability population, when aggregated across all sites the population looks similar to that of VR.

 

Types of PWI Services Received. PWI services most frequently included job placement (62 percent), job readiness training (59 percent), and job development (42 percent). Less frequently received services included job skills training (25 percent), supportive services (17 percent), worksite modifications (2 percent), and career advancement services (2 percent). On average, participants received PWI assistance for 8.7 months; persons who obtained employment following PWI participation averaged of 9.6 months; and persons who exited without employment averaged 6.5 months.

 

Employment Outcomes Obtained: RTI found that the data collection practices of the PWI projects limited their ability to formally evaluate the program’s impact on employment. However RTI found the following:

·      The outcomes of the PWI program, with respect to the percentage of persons served who exited into employment and the average hourly earnings of those individuals, are comparable to those of the VR services program.

·      Various combinations of program scope, type of consumers served, priority areas addressed, and types of services provided appear to be equally effective. The percentage of persons who obtained employment (as calculated from survey data) varied little when examined by these variables.

·      Although difficult to quantify, RTI noted that PWI effectiveness is likely in part a function of how well the project complements other grantee programs and the extent to which projects coordinate their services with those of other local programs, including especially VR. RTI found that PWI participants who had received VR services more often obtained employment than PWI participants who had not received VR services.

The RTI evaluation was not able to meet the goal of measuring whether the PWI program created and expand job opportunities for individuals with disabilities or whether involvement by the business community leads to better employment outcomes because of several limitations:

·      The programs varied so significantly, even in the core goal of engaging businesses and industry in the rehabilitation process, that very little can be learned about effectiveness of the approach by aggregating data across programs.

·      The data collection practices of the PWI grantees undermine the ability to accurately measure their achievements.

·      The programs were implemented with no control or comparison groups so the counterfactual is not known.

·      There is a great deal of overlap between the PWI programs and the VR program so it is difficult to separate the effect of PWI from that of VR.

·      Several functions of the BAC (for example, identifying careers and the corresponding needed skills) are now functions of the local workforce investment board under the Workforce Investment Act of 1998 (WIA).

In interviews with employers, the RTI study identified potential strengths of the PWI program compared to VR including better candidate screening to ensure a good job match and better follow-up with employers and former participants. It is not clear whether these practices are replicable or have been replicated.

Fabian (2007)

Of the 4,571 youth participating in the Bridges program from 2000 to 2005, 68 percent secured jobs after completing the career counseling and career assessment components of the Bridges intervention. The average rate of competitive job placement was 68 percent across all years, while the average wage for all employed youth was $6.91 per hour. A number of individual characteristics and experiences were associated with the likelihood that a youth with disabilities obtained a job: girls were significantly less likely to secure jobs than boys; those with prior paid vocational experience were more likely to secure jobs; and those receiving SSI were less likely to secure jobs than those not receiving SSI.

 

The study findings suggest that rehabilitation counselors should begin working with special education personnel as early as possible to formulate career-related activities and interventions, even prior to student entry into secondary schools. This directive corroborates other studies, which demonstrate that longer exposure to career and related interventions increases the likelihood of positive outcomes. The study was limited by the lack of a comparison or control group to assess what the outcomes would have been in the absence of the program. The findings also are not necessarily representative of youth with disabilities in general, as selection bias is likely. Students who enrolled in Bridges might represent a group of students with disabilities who expressed a desire to participate in a vocational program prior to exiting school, and represent a subset of youth with disabilities who exhibit characteristics associated with academic persistence and secondary labor market employment aspirations.

3.    RSA/SSA Systems Change/State Partnership Initiative

a.    Intervention and Target Population

Under the State Partnership Systems Change Initiative (SPI), SSA and RSA funded a combined total of 18 demonstrations in 17 states over the 1998-2004 period to identify, implement, and evaluate new strategies to promote employment opportunities for SSI/DI beneficiaries, as well as recipients of other types of public supports. Twelve of the projects were funded primarily by SSA and the other six projects were funded primarily by the RSA. DOL and the Substance Abuse and Mental Health Services Administration provided supplementary funding and support.

 

The amount and type of activities conducted by the SPI projects varied considerably between the SSA-funded demonstrations and the RSA projects. The SSA-funded demonstrations developed intensive, specialized interventions targeted toward a small group of beneficiaries, implemented under the limitations of a formal evaluation design. These projects served SSI and DI beneficiaries exclusively. Although the SSA projects were permitted to pursue systems-change activities, the programs used the most of their resources on research interventions and allocated relatively few resources to systems change efforts. In contrast, the six RSA-funded projects focused more on systems change efforts, both at the local and state level. They were not restricted to serving only SSA beneficiaries and as a result, the states targeted their programs to people with disabilities who receive public assistance benefits such as TANF, General Assistance, and/or Social Security disability benefits (Kregel 2006).

 

Five of the 18 projects targeted people with severe mental illness exclusively, and an additional 3 projects targeted people with mental illness along with people with other disabilities. The remaining 10 projects targeted people with any type of disability. A number of the projects required participants to be enrolled for services at their state’s VR agency.

 

The interventions implemented by the projects addressed the following service and policy gaps:

 

Benefit Policies. Most of the projects offered benefits planning and assistance programs to help beneficiaries understand and take advantage of SSA work incentives. Four states tested waivers to SSI regulations designed to make employment more attractive to beneficiaries. These SSI waivers included provisions that permitted working beneficiaries to keep more of their benefits (benefits were reduced by $1 for every $4 above the applicable earnings disregard, instead of $1 for $2), to accumulate more savings and assets, and to be exempt  from being continuing disability review triggered solely because of their participation in SPI (Peikes and Paxton 2003).

Access to Health Insurance. Many of the SPI projects were instrumental in the development and implementation of Medicaid Buy-In programs in their states, which were designed to allow beneficiaries to work without fear of losing medical benefits, and to provide coverage for some medical care, such as prescription drugs[21] and personal services, not covered by Medicare.

 

Service System Barriers. Most projects tried to improve coordination among state agencies and the various organizations that provide employment supports to people with disabilities. A common approach was to include disability-related services in DOL’s one-stop centers. Some programs developed initiatives to foster interaction among state agencies that share responsibility for encouraging work among people with disabilities (Peikes and Paxton 2003).

 

Human Capital and Personal Barriers. The SSA projects developed programs to help beneficiaries obtain skills that would help them compete in the labor market. These included case management services to help participants obtain needed services and supports, and placement assistance programs that provided participants with job placement and support services to help them find and maintain employment. Services were also provided through job service vouchers that enabled beneficiaries to obtain vocational services from a vendor of their choice. Two projects also tested ways to use peer support to help beneficiaries deal with the world of work. The type and intensity of services varied significantly across projects (Peikes and Paxton 2003). As the RSA projects were more generally focused on systems change, they did not provide these types of direct services to individuals.

 

Employment Market Barriers. A few of the projects provided education, outreach, and direct incentives to encourage employers to hire more beneficiaries with disabilities. Some projects also used non-SPI funding for initiatives to promote employer awareness of the abilities and employment potential of people with disabilities (Peikes and Paxton 2003).

b.    Evaluation Activities

SSA and RSA funded a SPI project office at Virginia Commonwealth University (VCU). VCU and its subcontractor, MPR, conducted the national evaluation. The evaluation was designed to identify the mix of services that were most effective in promoting employment, and identify services that were especially valuable for a particular participant group (Agodini et al. 2002). Because the RSA-funded projects focused on systems change rather than direct services, the core and supplemental analyses described below are for the SSA-funded projects only.

 

In addition to the national evaluation, each SPI project was required to develop an internal evaluation plan. Of the 12 SSA-funded states, four randomly assigned participants to an intervention or control group. The project office synthesized these evaluations in two reports (Peikes and Sarin 2005; Kregel et al. 2005).

 

The state evaluations examined whether the projects accomplished the following:

·      An increase in employment rates and earnings

·      A decrease in the proportion receiving Social Security benefits and the amount of Social Security benefits received

·      An increase in participants' income (defined as earnings plus Social Security benefits)

The national evaluation implemented a four-part study that combined multiple data sets and methods to produce several estimates of the effects of the state project interventions. It then used qualitative information about project implementation to synthesize those estimates and to develop an understanding of the relative performance of the projects. The four components of the national SPI evaluation include:

 

Core Evaluation. The core evaluation component compared key outcomes of the beneficiaries who participated in the SSA-funded projects to outcomes of a comparison group selected to match the participants in terms of the characteristics of the areas in which they live, as well as their demographic characteristics, prior labor market experiences, and prior benefit receipt. The core evaluation used only SSA administrative data and income tax data so that the data and approach could be applied consistently across projects.

 

Supplemental Evaluation. The projects collected data on beneficiaries at study enrollment and on a quarterly basis. These included detailed data about participants’ characteristics, their receipt of project services, and their employment and benefit receipt outcomes. As part of the supplemental evaluation, MPR produced two reports:

·      Characteristics of Participants Enrolled Through March 2003. Based on an analysis of data that state projects collected from participants at intake, the evaluators characterized the participants by demographics, type of disabling condition, education, employment experience, and use of public assistance programs (Peikes and Paxton 2003)

·      Services Delivered Through March 2002. Using information collected from participants on a quarterly basis about the types of services they used, the number of hours associated with each service, the use of work incentives, and other programs, the evaluators analyzed the type and intensity of services used and use of work incentives and Medicaid Buy-In programs (Deke and Peikes, 2003)

State Projects’ Own Evaluations. SSA required each project to use its own evaluation design and data sources to describe the project’s implementation and to assess the project’s impacts. All of the projects completed final reports. Ten of the 12 projects provided final estimates of program impacts. MPR researchers synthesized the findings from SSA-funded projects in Peikes and Sarin (2005).

 

Implementation and Synthesis Analysis. The SPI Project Office at VCU documented and analyzed the interventions fielded by the projects. This information provided context for the evaluators to understand the ways in which the projects changed the services available to participants, as well as the context in which each project operated.

c.     Findings

Generally, the SPI projects providing direct service interventions were able to meet their enrollment targets. From January 1999 through September 2003, they had roughly 10,300 participants. Individual project enrollment ranged from 181 to over 2,000 participants, with an average enrollment of 582 (Peikes and Paxton 2003).

 

The process evaluations indicated that the SPI projects had an effect on system change within states and provided replicable strategies that could be implemented in other states (Kregel et al. 2005). Some examples include:

·      The SPI projects led the way in the establishment of a nationwide system of Benefits Planning Assistance and Outreach (BPAO) Projects,[22] with many staff involved in the on-going training provided to these projects.

·      Several SPI projects were instrumental in facilitating the development and/or implementation of Medicaid Buy-In programs, at first through the Balanced Budget Act and later through the Ticket Act.

·      The model for the Disability Program Navigators initiative within the one-stop career center system that is currently being administered by the Employment and Training Administration of DOL was initially developed through the Colorado RSA-funded SPI project.

·      In a number of SPI projects, the use of benefits planning and assistance services by the state VR agency became a “routine” component of service delivery for SSA beneficiaries.

·      Multiple SPI projects demonstrated effective strategies for coordinating the efforts of employment service projects with local SSA field office staff.

Findings regarding the outcomes and impacts of the SPI projects are mixed. The projects seemed to increase the proportion of participants who attempted to work. However, there is mixed evidence of the effects of SPI on average earnings and little effect on benefits (Peikes and Sarin 2005).

 

For several reasons, the evaluators expected the impact on employment and earnings to be small to modest in the short-term. First, while participants faced many barriers to employment, most projects focused on one or two barriers, and so the projects were expected to be most successful for people who faced only those barriers. Second, two earlier SSA demonstrations with more intensive services than provided under SPI (the Transitional Employment Training Demonstration and Project Network) generated only modest increases in participants’ employment and earnings. Third, the SPI program was fielded in an environment where many other services were available from state VR agencies, community organizations, and the BPAO projects. BPAO was implemented during SPI in some states because of the efforts of SPI project staff. As a result, some of the comparison group members received benefits counseling and employment services similar to those being offered by SPI.

Testing the Validity of Comparison Groups[23]

In the core evaluation, MPR recognized that a beneficiary’s decision to participate in the SPI project might be related to the participant’s observable and unobservable demographic or human capital characteristics. Thus, they expected that the beneficiaries who participated in SPI would differ from the average SSI/DI beneficiary in terms of their motivation to work, work histories, and other characteristics that might be related to employment and earnings outcomes. To account for these differences, MPR used propensity score matching (a statistical matching technique) to develop a comparison group comprised of a subset of SSI/DI beneficiaries who were similar to SPI participants along 250 characteristics. Multiple statistical tests confirmed that the propensity score match yielded comparison groups that were well matched to the SPI participants on the observed characteristics.

 

Using three state projects that had implemented random assignment study designs (New Hampshire, New York, and Oklahoma), MPR analyzed the validity of the propensity score comparison groups by comparing the impact estimates based on random assignment with estimates based on the propensity score comparison groups. They found that although the propensity score comparison groups were similar to SPI participants along the entire spectrum of available characteristics, the estimates based on the propensity-score comparison group were quite different than those based on the randomized control. The former suggested that the SPI interventions increased earnings by between $970 and $5,600 a year, whereas the latter showed that the interventions had no impacts, or even negative impacts on earnings. MPR ultimately concluded that non-experimental propensity score matching method could not provide reliable estimates of the SPI program effects, most likely because the comparison groups did not match the participant group on unobserved characteristics such as motivation to work and informal support from family, friends and others (Peikes and Sarin 2005).

d.    Strengths and Weaknesses of the Evaluation

One of the strengths of the SPI evaluation was its data collection strategy. The program office developed a rigorous data collection system that facilitated the state’s ability to provide common information about each participant and to track each participant’s receipt of 17 types of services using standard definitions and data collection procedures. This allowed VCU and MPR to aggregate data across projects. The projects sent their data quarterly to the SPI Project Office at VCU, where it was assessed for quality and completeness, and VCU generated discrepancy reports. The projects then corrected the discrepancies and submitted any missing data.

 

Despite the strength of the design the evaluators found the following: 1) the projects had an extremely difficult time collecting detailed service data from collaborating agencies and organizations; 2) although the projects were able to access other state administrative data sets, they had to overcome administrative and logistical obstacles to obtain the data; and 3) the projects found that tracking participants over time was much more difficult and time consuming than they had initially anticipated.

 

The evaluators noted the following limitations in the analysis of the SSA SPI projects.

·      Insufficient Follow-Up Period. The follow-up timeframe (6 to 12 months) was too short to assess the effectiveness of the intervention on the goal of reducing reliance on public benefits. It takes a longer period of time to increase participants’ self sufficiency enough so that they feel comfortable leaving the SSA rolls (Kregel 2006).

·      Lack of Control Over Administrative Data. Many states used Unemployment Insurance data to monitor the employment and wages of SPI participants. However, analyses comparing these data with self-reports suggested that the Unemployment Insurance system did not capture some forms of employment or employers (Kregel 2006).

·      Problems Implementing Random Assignment Study Designs. Although random assignment to treatment and control groups is the best way to eliminate bias in research, it proved to be very difficult for the projects to execute. Only four projects had experimental designs with some level of randomization to treatment or control groups. Only one (Oklahoma) randomly assigned subjects to the treatment or control group prior to any contact by the project. New York, New Hampshire, and Illinois randomly assigned volunteers to either the treatment or the control group.

·      Contamination of Control Groups. The BPAO initiative began implementation during the SPI project research cycle, thus providing one of the SPI services to some control and comparison group members.

D.   Department of Labor

1.    Customized Employment Grants

a.    Intervention and Target Population

WIA required states and localities to bring together a range of federally funded employment and training services into a single system called the One-Stop Career Center system. Although one of the core principles of the one-stop system is to provide universal access to all job seekers, most had been oriented towards a self-service model that offers a “one size fits all” approach to customers. This resulted in people with disabilities being referred elsewhere because the One-Stop Career Centers were physically or programmatically inaccessible, or because staff believed that other programs, such as state VR programs, could better serve people with disabilities (Elinson and Frey 2005).

 

In three annual grant cycles between 2001 and 2003, the Office of Disability Employment Policy (ODEP) within DOL provided 20 grants to Workforce Investment Boards to increase the capacity of One-stop Career Centers to provide person-centered, individualized, or customized employment services for people with significant disabilities. Customized employment services could include strategies such as supported employment; supported entrepreneurship; individualized job development; job carving and restructuring; use of personal agents (including individuals with disabilities and family members); development of micro-boards, micro-enterprises, cooperatives and small businesses; and use of personal budgets and other forms of individualized funding that provide choice and control to the individual and promote self- determination [Federal Register, June 6, 2003 (volume 68, number 109)].

 

The grants funded direct customized employment services as well as the development of policies and practices to ensure that these strategies were systemically included in the services available through the one-stop system. The grants were for one year and could be renewed for up to four additional years. The target population was people with significant disabilities, including people who previously might have been considered “nonfeasible” for employment, and people who have been segregated in institutions, nursing homes, and day activity programs [Federal Register, June 6, 2003 (volume 68, number 109)].

 

ODEP also funded three other demonstration programs designed to build the capacity of the workforce development system to serve people with disabilities, including (1) Youth programs to provide services to help youth with disabilities reach their educational and occupational goals; (2) Technical Assistance to the youth and adult demonstration programs as well as to other elements of the workforce investment system; and (3) Olmstead WorkFORCE Action Grants.

b.    Evaluation Activities

ODEP contracted with Westat to conduct an independent evaluation of the customized employment programs as part of a larger evaluation of ODEP’s grant initiatives (Elinson and Frey 2005). Westat collected both quantitative and qualitative data to generate their conclusions and recommendations.

·      Quantitative Data. Westat twice collected information on randomly selected program participants who had been placed in competitive employment. The first round of data collection included 345 customers from 25 programs. The second round included 536 customers from 30 programs. The evaluators did not collect data on program participants who did not become employed, and no control or comparison groups were included in the evaluation design.

·      Qualitative Data. Westat conducted site visits to all ODEP demonstration locations and interviewed the directors of these projects, employees and partners of the projects, customers and their families, and community representatives. To capture changes over time and collect longitudinal data, Westat researchers visited locations multiple times over the course of the demonstration.

c.     Findings

The evaluation focused on two outcomes: employment and capacity building.

 

Employment. Westat reported that the customized employment process can be effective in helping some hard-to-serve individuals with disabilities find better employment and thus allow them to begin exiting income support programs. This conclusion was based on the following findings: In the first round of data collection, 43 percent of those who self-disclosed their disability said that they had psychiatric or emotional disabilities, and 37 percent had never worked or had only worked in a noncompetitive employment environment. More than half (54 percent) of program customers who had been placed in competitive employment obtained jobs with the potential for career advancement. In addition, compared to their status at the beginning of their involvement with the project, the percentage of customers in the ODEP demonstration programs who were on government benefits (such as Food Stamps, SSI, DI, and Temporary Aid to Needy Families) had declined in every category except subsidized housing (which increased as customers of the Chronic Homelessness program were placed in housing as part of their participation in that program). The second round of data collection on competitive employment placements yielded similar findings (Ellison and Frey 2005).

 

Capacity Building. Demonstration programs employed multiple strategies to build capacity (Ellison and Frey 2005):

 

·      Strategic Planning. Although the programs were required to engage in strategic planning activities, many were still using a “plan as you go” approach. Conversely, a number had reinforced their understanding of local needs by gathering additional information from key stakeholders and involving stakeholders as part of advisory committees.

·      Physical and Programmatic Accessibility. Although ODEP funds could not be used for physical modifications, the projects served as an impetus for the sites to modify buildings or purchase equipment for physical and communication accessibility. Other approaches to enhancing accessibility at the one-stops included modifying the client orientation sessions to make them more disability-friendly, offering disability awareness training to staff, and connecting with the local Disability Program Navigator.

·      Training. The demonstration programs provided training (including formal training and technical assistance) on the following topics for their staff, partners, and collaborators: disability awareness, working with people with disabilities, availability of resources and services for people with disabilities, and advising customers on how to obtain financial work incentives. However, as Westat noted, the quality of this training varied from location to locationranging from basic and informational to more thorough and problem-solving. Westat also noted that because of staff turnover at the one-stop centers, it was necessary to repeat staff training to maintain sustainability.

·      Service and resource coordination. In all of the demonstration programs, coordination of services for people with disabilities (such as job accommodations, coaches, personal assistants, funds for training and education, and transportation) began with extensive collaboration among partners and other organizations. In many locations this coordination transitioned to the leveraging of each other’s resources; in others it resulted in the establishment of formal arrangements among the partners so that these leveraged resources could be sustained after the funding for the program ended. The range and types of partners varied across programs, as did the success of these collaborations. Despite some promising practices, the evaluation identified few changes to formal policies, procedures, and staff at any of the project sites, local Workforce Investment Boards, or collaborating one-stop centers, or changes to infrastructure that would facilitate systems change.

·      Sustainability. A number of elements of the demonstration program may prove to be sustainable, including improvements in accessibility at the one-stops, increased collaboration among agencies and organizations, and staff training. However, the ultimate measure of sustainability for the program is if customized employment becomes a part of the menu of services available at one-stop centers. Many project sites reported that it could be difficult to sustain customized employment without additional funding.

d.    Strengths and Weaknesses

In the absence of information on a comparison or control group, it is not possible to gauge the strength of Westat’s findings or the impact of the customized employment demonstrations on the employment outcomes of people with disabilities. At least some of those who became employed after receiving services from the grant programs might have become employed in the absence of the grants, and perhaps all would have managed to find employment. 

2.    Disability Employment Grants[24]

a.    Intervention and Target Population

In 1998, the DOL Employment and Training Administration (ETA) awarded $6.8 million in demonstration grants to 15 nonprofit organizations to provide employment and training services to persons with severe disabilities and to dislocated workers with disabilities.[25] In 2002, ETA awarded an additional $5.5 million to 12 organizations to pursue the same goal.

The initiative gave grantees wide latitude in designing their programs, but encouraged the programs to invest in resources that would support participants’ efforts to develop career goals and skills needed to advance beyond entry-level work. The grant solicitation emphasized that quality of employment outcomes was more important than the number of placements. The project designs varied in terms of approaches, specific goals, target population, number of participants, funding levels, services provided, and service intensity. However, projects typically included recruitment, assessment, service planning, basic education, vocational training, job readiness training, work experience, on-the-job training, job search assistance, post-employment follow-along services, and supportive services.

 

The target population varied across grantees and, in many cases, across sites. About half of the grantees concentrated on providing services to people with severe disabilities, although the definition of “severe” varied across sites. Some grantees focused on people with specific types of disabilities (physical, psychiatric, HIV/AIDS, developmental disabilities, mental retardation and developmental disabilities, and multiple disabilities) while others served people with any type of disability. Some addressed all age groups while others focused on youth transitioning from high school to work. Across all 15 projects receiving grants in 1998, 41 percent of participants were SSI/DI beneficiaries.

b.    Evaluation Activities

In 1999, DOL contracted with Berkeley Policy Associates (BPA) to evaluate the 1998 grantees. BPA’s evaluation of the Disability Employment Grants (Almandsmith et al. 2001) had four goals:

·      Assess the effectiveness of grantees in meeting their enrollment and employment outcome objectives

·      Analyze the grantees’ success in collaborating and coordinating with One-stop Career Centers and state VR agencies

·      Identify innovative practices that grantees have implemented

·      Examine the replicability of innovations, best practices, and project designs both within the workforce development system and for other disability employment service providers

The evaluation design included two major components. The qualitative study focused on grant practices and collaboration strategies. It was based on site visits to the grantee’s central office and two of each grantee’s service sites, as well as review of grantee materials. The quantitative study measured participant characteristics, use of services, and employment outcomes. The analysis was based on reports that grantees were required to submit to DOL quarterly and/or annually, combined with data developed by the grantees at BPA’s request. BPA used univariate and multivariate techniques to explore the influence of participant characteristics, specific types of services, and groups of services on employment outcomes. Outcomes of interest included job placement rates, weeks to placement, hours per week in first job, hourly wage in first job, employed at 90 days, employed at 180 days, and hourly wages at 180 days. In addition, BPA compared demonstration outcomes to outcomes of a comparison group made up of Job Training Partnership Act (JTPA) program participants with substantial impairments, and who were served by the federal program in 1998.[26] However, BPA noted that relative to the JTPA comparison group, the demonstration served individuals with more severe disabilities, who were younger, more likely from minority backgrounds, and had more limited work histories.

c.     Findings

In total, the Disability Employment Grants served 4,619 participants during the evaluation’s data collection period (July 1, 1998, to June 15, 2000).

 

Employment Outcomes. Across the 15 Disability Employment Grants, 47 percent of participants entered employment. Thus, the program as a whole met DOL’s GPRA Program Year 1998 goal of a 47 percent placement rate. Across grantees, placement rates varied from a low of 16 percent to a high of 72 percent. As shown in Exhibit 5, the average hourly wage, number of hours worked, and retention rates varied substantially across grantees.

 

The employment outcomes achieved by participants varied substantially depending on personal characteristics, the types of services individual projects offered, and the grantee’s approach to employment services.


Exhibit 5

Disability Employment Grant Outcomes

 

Average Across All Grantees

Minimum

Maximum

Percent of participants who entered employment

47%

16%

72%

Percent of participants who entered employment 20 hours or more

42%

10%

64%

Average hourly wage

$8.01

$5.69

$12.62

Average hours worked in first job

28 hours

16 hours

39 hours

Job retention rates

 

 

 

90 days

67%

47%

100%

6 months

61%

40%

90%

12 months*

55%

37%

100%

 

Source:  Almandsmith et al. (2001).

*The 12-month results are somewhat difficult to interpret because fewer than 500 of the demonstration’s participants were eligible for 12-month follow-up. This is an artifact of the timing of the evaluationdata were collected only during the project’s first two years of operation.

 

Controlling for other factors, the evaluators found the following personal characteristics to be associated with employment outcomes:

·      Having a history of employment before entering the grant program appears to be the single best predictor for success in the evaluation’s key employment outcome measures.

·      The higher a participant’s education level, the higher his or her hourly wage at placement and at six months, and the more likely the individual would be working at six months.

·      Participants with cognitive and psychiatric/emotional disabilities achieved significantly lower outcomes in terms of wages and hours per week at placement and retention at six months. Individuals with psychiatric disabilities were also less likely to be working six months after entering employment.

·      Participants with more impairments worked fewer hours per week and earned lower hourly wages at both placement and at six-month follow-up than did individuals with a single disability.


Analysis of the types of services provided yielded the following findings:

·      Job search services, on-the-job training and work experience services were significantly associated with job placement and with employment rates both 90 and 180 days post-placement.

Ø     Individuals who participated in on-the-job training worked in jobs that paid significantly less than those held by their peers who did not use on-the-job training.

Ø     Work experience participants worked significantly fewer hours per week than did other participants.

 

·      Individuals receiving assessment and supportive services were less successful than participants who did not receive these services.

Ø     Participants who received nonvocational supports worked fewer hours per week at placement, and earned lower hourly wages six months after they started working, compared to those who did not receive such supports.[27]

 

·      Participants who received basic education services earned higher hourly wages than did individuals who did not use these services.

·      Post-employment services were positively associated with higher participant hourly wages at six months post-placement.

Controlling for participant characteristics, the different project approaches were associated with the following:

·      Participants in programs using a vocational training approach had significantly better outcomes on three of the evaluation’s employment outcome measures. They earned higher wages, worked more hours per week, and were more likely to be employed at 90-day follow-up than were participants served by projects that adopted a different approach to service provision.

·      Work-first projects moved participants into employment more quickly than did grantees that used other service approaches. Participants in these projects, however, earned lower hourly wages than those served by other projects both at placement and six months later.

·      Participants in projects that used the person-centered planning approach had poorer outcomes both in terms of finding a job and job retention at 90 and 180 days post-placement.

·      The project that used a capacity-building service strategy assisted significantly more participants to enter employment than did projects that used other service approaches, but these participants worked significantly fewer hours per week than individuals served by other types of projects.

In its comparison of the Disability Employment Grant programs to the JTPA control group, BPA found that outcomes for demonstration participants were somewhat poorer than those achieved by JTPA participants. Demonstration participants had a lower placement rate, lower hourly wages, and worked fewer hours per week on average than did the individuals in the JTPA sample (Exhibit 6).

Exhibit 6

 

Employment Outcomes of Disability Employment Grant

Demonstration Participants and JTPA Comparison Group Members

 

Outcome

Disability Employment Grant Demonstration Participants

JTPA Comparison Group

Placement 20+Hours

41.5%

55.2%

Hours/Week (1st Job)

27.8

34.7

Hourly Wage (1st Job)

$7.53

$8.11

90-Day Retention

67%

68%

12-Month Retention

55%

N/A

Source: Almandsmith et al. (2001).

Based on their understanding that the Disability Employment Grants served individuals with more severe disabilities than did the JTPA system at large, the BPA evaluators concluded that, despite the presence of participant characteristics that tended to negatively affect employment outcomes, once demonstration participants obtained employment, they had the same rate of employment 90 days later as did people with disabilities served by the mainstream workforce development system. Thus, they concluded that the Disability Employment Grants appeared to have achieved one of the purposes of the demonstrationto improve employment retention among the people with disabilities that they serve.

Collaboration and Capacity Building

ETA encouraged grantees to collaborate and develop linkages with the local workforce development system, the state VR agencies, and local community-based organizations providing employment and training services and/or serving people with disabilities. Most grantees pursued these goals. BPA reported that anecdotal reports from participants, project staff, and collaborators indicated that grantee collaboration efforts resulted in improved outcomes and better coordination of services. However, neither the national evaluation nor any of the grantees formally measured the effects of collaboration on participant employment outcomes. Further, all of the grantees used more than one collaboration method in implementing their projects, thus, the link between specific collaboration techniques and participant employment outcomes is not easy to determine (Almandsmith et al. 2001).

Innovative and Best Practices

BPA identified the common characteristics and service practices of the grantees who achieved the best outcomes. They noted that the following practices seemed to be associated with success in one or more programs:

·      Making systematic and ongoing efforts to understand local market conditions and labor needs

·      Tailoring programs to individual participant skills and needs

·      Ensuring that programs are accessible and address the disability-related needs of participants

·      Recognizing that customer choice and customer-driven services are keys to success

·      Collaborating with formal and informal partners in the workforce development system in order to leverage resources and expand the number of services and resources available to participants

·      Creating new service partnerships

·      Collaborating with employers and the business community, particularly though a BAC

d.    Strengths and Weaknesses of the Evaluation

Several aspects of the Disability Employment Grant program affected its evaluation:

·      Not all populations had access to all services and there was a close correlation between the population and the type of services offered, making it difficult to identify the effect of the intervention.

·      The project approaches varied dramatically. Although the multivariate analysis attempted to exploit these differences, the depth of the variation could not be adequately captured in the analysis. The service approaches were categorized into four types, but within each type, there were notable differences in the services offered. For example, among grantees that used the work-first strategy, one worked with employers to restructure jobs for participants when appropriate and feasible. Others provided job coaches and other on-the-job support for participants in order to help employees with the training and orientation process. Another offered employers incentives in the form of cash grants to provide accommodations or on-the-job training, or a one-month trial period during which the grantee or another agency paid the participant’s salary.

·      For two of the types of services (on-the-job training and work experience) only one project provided that particular service. Thus, it is not possible to distinguish the effect of the particular intervention from other characteristics of the provider and population.

·      DOL required grantees to have sites in two or more states in order to increase the ability to evaluate the effectiveness of each service model in diverse settings. However, the grantees did not implement identical programs in each location, thereby hindering the ability of the evaluation to make useful comparisons between the sites.

·      As BPA noted, the time frame of the study (collecting data during the projects’ first two years of operation), did not provide information to assess the long-term effectiveness of the services provided. This is a significant limitation when considering the outcomes of projects with long-term goals for their participants and a lengthy service period. In addition, the two-year time frame yielded very small sample sizes for the outcome measures six and 12 months after placement (Almandsmith et al. 2001).

3.    Disability Program Navigators

a.    Intervention and Target Population

DOL and SSA jointly funded Disability Program Navigators (DPNs) in 45 states, the District of Columbia, and Puerto Rico. The initiative establishes disability specialists in the one-stop centers to provide information for SSA beneficiaries and other people with disabilities (DOL 2007). The DPN grant is one of three funding mechanisms to support a disability specialist. States have also used Work Incentive Grant funding and WIA funding to support similar positions, often referred to as “navigator-like” staff. (Hall et al. 2005).

 

The DPNs and the navigator-like staff are “systems change agents” and “facilitators.” They operate in the one-stop center and have some face-to-face contact with clients. However, they are not intended to take the role of a case manager or vocational rehabilitation counselor. Generally, they are tasked with the following:

·      Guide one-stop staff in helping people with disabilities access and navigate provisions of various programs that impact their ability to gain/retain employment

·      Provide disability-related training to one-stop staff

·      Ensure the accessibility of the one-stop center by conducting accessibility assessments and developing accessibility plans

·      Reach out to the disability community to assure that clients and recipients of many support programs that are not partners in the one-stops become aware of, and have access to, the employment assistance provided through the one-stop system

·      Develop linkages and collaborate on an ongoing basis with employers to facilitate job placements for persons with disabilities

·      Develop partnerships with other agencies and service providers to integrate services, implement systemic change, and expand the capacity to serve customers with disabilities

b.    Evaluation Activities

DOL contracted with the Rehabilitation Research and Training Center on Workforce Investment and Employment Policy for Persons with Disabilities at the Law, Health Policy, & Disability Center of the University of Iowa School of Law to provide the DPN project with training, technical assistance, and evaluation. The evaluation consists of four components (Schmeling and Morris 2005):

 

Quarterly Report Evaluations. The evaluation instrument is not an attempt to measure individual outcomes, but rather an effort to understand the process of system capacity building to support job seekers with disabilities. It assesses the following: (1) time allocation in eight areas by month; (2) systems relationship activities and outcomes; (3) best practices; (4) linkages between one-stops and other organizations and employers; and (5) navigator needs.

 

Telephone Survey of 14 States. The evaluation includes two rounds of surveys with 819 one-stop staff and representatives from partnering agencies. The survey instrument asks about barriers and facilitator to services, supports, and employment outcomes for individuals with disabilities in the Workforce Development System, and the role of the navigator in reducing these barriers

 

In-Depth Study of Four States. Evaluators conducted interviews with state and local officials in eight sitestwo (one rural and one urban) in each of four states (Massachusetts, Florida, Colorado, and Wisconsin). The evaluators interviewed 117 individuals to document implementation experiences of the DPN initiative, best practices, and short-term systems-change outcomes.

 

Quantitative Study of Outcomes. The planned evaluation will link SSA records to one-stop records from four states covering the 2001–2006 period, for purposes of comparing the employment outcomes of people with disabilities served by one-stops in areas covered by DPNs to those in areas not covered by DPNs (Livermore et al. 2008). In January 2008, DOL issued a contract to MPR to conduct this component of the evaluation.

c.     Findings to Date and What Will Be Learned in the Future

Findings from the quarterly reports provide quantitative information on the following:

·      Description of typical DPN activities, including time allocation by type of activity, system relationships and outcomes, and involvement with organizations

·      Description of changes in DPN activities over quarters by type of activity, system relationships, and involvement with organizations

·      Best practice reports or “mini case studies” on systems relationships, collaborations with employers, and experiences of job seekers with disabilities

Findings from the phone survey indicated that satisfaction with DPN services was high and that the most significant advantages of having a DPN in the system noted are (1) improving interagency coordination, (2) improving program/service access, and (3) improving availability of benefits counseling (Schmeling and Morris 2005).

 

The qualitative findings of the four in-depth case studies (Emery and Bryan 2005) suggest that DPNs had a positive impact on one-stop centers. Respondents reported that centers were more accommodating and better equipped, more fully integrating services and training that are accessible to everyone, providing seamless service to persons with disabilities, and helping to realize the goal of universal access. A goal of the DPN initiative was to better enable the workforce network to help job seekers with disabilities secure and maintain full and meaningful employment. The case study report notes improved quality of referrals; less turf-protecting behavior and better leveraging of resources; increased awareness and more appropriate referrals between one-stops and community agencies; and bridging of gaps between agencies to coordinate services to clients. Though not a representative sample, customers interviewed during the site visits generally reported high levels of satisfaction with the DPNs.

 

To date, the various evaluation activities have collected a large amount of descriptive information about the activities of DPNs, but have generated only limited analysis of data. As a result, the reports are most useful within the DPN project to inform other DPNs rather than to evaluate the impact of the project on the employment of people with disabilities. The qualitative findings of the case studies and phone survey are suggestive of the DPN having an impact on the broader service delivery system.

 

If undertaken, the planned quantitative evaluation of outcomes using the linked SSA and one-stop data will provide useful information about the extent to which Social Security disability beneficiaries utilize one-stop services, the types of services they receive, and whether there have been changes in such utilization that might be attributed to the implementation of the DPNs. Estimation of impacts is likely to be problematic because DPN implementation in the states to be evaluated did not support the identification of a comparison. Descriptive information about outcomes will, however, yield substantial new information about the extent to which Social Security disability beneficiaries utilize the one-stop system and the characteristics and employment outcomes of users. To date, the quantitative evaluation of DPN outcomes has been delayed because of difficulties in obtaining permission to match SSA and state one-stop system data.

4.    Work Incentive Grants[28]

a.    Intervention and Target Population

ETA established the Work Incentive Grant (WIG) program to address concerns about the ability of the one-stop system to meet the needs of people with disabilities. With four solicitations between 2000 and 2004, ETA awarded grants to 113 state and local entities totaling $65 million. The first two solicitations were relatively broad and permitted the grantees to use the funds for a range of activities, including assessing one-stops’ architectural accessibility; acquiring assistive technology; conducting outreach to the disability community; linking and coordinating with community disability-related agencies; training existing one-stop staff on disability issues; and making available staff who have the experience, knowledge, and skills necessary to address a broad range of disability-related issues (GAO 2004b).

 

Experience with the initial grants showed that a number of local workforce investment areas were using disability resource specialists or navigators and that this approach seemed successful in improving overall service delivery for people with disabilities. As a result, in the subsequent rounds ETA focused its priorities more narrowly—though not exclusively—on placing a disability specialist in the one-stops. The grantees were also given the option to address other issues, such as meeting the architectural access requirements. To supplement the WIGs in placing staff in the one-stops, ETA, in conjunction with SSA, funded cooperative agreements to establish and train disability program navigators (discussed in the previous section).

b.    Evaluation Activities

DOL contracted with the Rehabilitation Research and Training Center on Workforce Investment and Employment Policy for Persons with Disabilities at the Law, Health Policy & Disability Center at the University of Iowa to provide technical assistance to WIG grantees and to conduct a process evaluation. The evaluation was designed to document the progress of WIG activities, and identify successes, best practices, challenges, obstacles, and future needs. The analyses were based on annual surveys of grantees.

 

The survey instrument changed slightly over the course of the evaluation, but generally grantees reported five types of information: (1) types of activities undertaken, (2) the amount of effort (none, limited, significant) and outcome (none, limited, significant) of a variety of activities within each of the major focus areas, (3) status of selected one-stop operations, (4) self-assessment of the progress made in improving access and opportunities for people with disabilities, and (5) major accomplishments and barriers.

c.     Findings

WIG grantees undertook a range of activities to increase the participation and success of people with disabilities in the one-stop system. The activities fell into five broad categories: (1) accessibility, (2) identification of job seekers with hidden disabilities, (3) coordination with employers, (4) training and education, (5) marketing and outreach, and (6) service and interagency coordination.

 

The evaluation focused on process rather than outcome, and did not rigorously identify the impacts of activities on outcomes. In each year the evaluators report the number of WIGs that reported “significant activity with significant outcomes” and “significant activity with limited outcomes” for roughly 80 activities. However they do not report any conclusions based on this information.

 

The WIG evaluations identified the following key obstacles to success of the WIGs in promoting systems change:

·      State of the Economy. DOL awarded the first two rounds of grants during a period of state budget cuts, making it difficult for systems change programs to fully engage other partners and implement the proposed activities. It was also a period of high unemployment and the large pool of qualified job seekers added to the competition for limited job openings for individuals with disabilities.

·      Philosophies and Organizational Beliefs. Workforce development professionals tended to believe that people with disabilities required too much time and resources and that referring the client to other agencies (such as VR) rather than collaborating with VR was the most appropriate solution.

·      Performance Measures. The performance standards for the one-stop system focus on the percent of consumers who find jobs and increase their wages. Many stakeholders in the workforce system believed that individuals with multiple barriers to employment would be less successful in finding a job and would adversely affect the overall performance measures for the workforce investment area.

·      Staff Turnover. The WIGs invested in training staff about assistive technology, reasonable accommodations and general disability awareness, but frequent staff turnover at the one-stops reduced the effectiveness of this strategy.

·      Service Coordination. “Turfism” remained a systemic problem among both mandated and nonmandated partners at the one-stops.

·      Employer Interest and Investment. Although a majority of the WIGs implemented strategies to attract the interest of the business community, engaging employers was difficult.


d.    Overall Strengths and Limitations

The WIG evaluations identified the disability program navigator as a promising approach to addressing many of the systemic barriers faced by people with disabilities in using the one-stop system. This approach was replicated by other WIG grantees and prompted the introduction of the DPN cooperative agreements. The evaluations did not provide specific evidence of the impact of any particular approaches on the employment of people with disabilities. They did, however, identify several key obstacles to programmatic success.

E.    Department of Veterans Affairs

1.    Vocational Rehabilitation and Employment Program

a.    Intervention and Target Population[29]

The purpose of the VA Vocational Rehabilitation and Employment (VR&E) program is to assist veterans with service-connected disabilities to prepare for, find, and maintain employment. The program also offers independent living services for veterans with service-connected disabilities that are so severe that they cannot immediately consider work. To be entitled to services, a veteran must have received a discharge that is other than dishonorable, have at least a 20 percent service-connected disability rating, and have an employment handicap.[30] The basic period of eligibility in which VR&E services may be used is 12 years from the latter of the date of separation from active military service or the date the veteran was first notified by the VA of a service-connected disability rating. If the service-connected disability rating is less than 20 percent or if the veteran is beyond the 12-year basic period of eligibility, then a serious employment handicap must be present to establish entitlement to VR&E services. Those found eligible may receive the following types of services:

·      Assessment to determine abilities, skills, interests, and needs

·      Vocational counseling and rehabilitation planning

·      Employment services such as job-seeking skills, resume development, and other work readiness assistance

·      Assistance finding and keeping a job, including the use of special employer incentives

·      Training such as on-the-job training, apprenticeships, and unpaid work experiences

·      Post-secondary training at a college, vocational, technical, or business school

·      Supportive rehabilitation services, including case management, counseling, and referral

·      Independent living services

b.    Evaluation Activities

No formal evaluation to assess the impact of the VR&E program on the employment of veterans with disabilities has been conducted. However, several recent efforts have assessed the general performance of the program. We describe these efforts below.

VR&E Task Force 2004

In May 2003, the VA Secretary established a task force to conduct an independent evaluation and analysis of the VR&E program. The task force was asked to:

·      Conduct a functional and organizational assessment of the VR&E

·      Evaluate eligibility criteria, procedures, and processes for determining how a veteran is approved for training, employment, or independent living services

·      Appraise current VR&E processes, information systems, and management controls

·      Determine consistency in the administration of the VR&E Program across regional offices

·      Examine clinical rehabilitation practices and employment placement services used by other federal, state, local, or private organizations serving disabled persons, including veterans

The task force based its review on a series of public fact-finding sessions, site visits, and analyses of previous studies and reports on the VR&E program. The fact-finding sessions included public meetings to solicit the comments and recommendations of Congressional committee staffs, GAO, veterans service organizations, Veterans Benefits Administration, and partnership organizations such as the Veterans Health Administration and DOL. The task force also received comments from VR&E staff and a variety of professional organizations and private sector firms prominent in the fields of disability, rehabilitation, and employment of people with disabilities (VA VR&E Task Force 2004).


PART

In 2006, OMB conducted a PART assessment of the VR&E program. Included in the PART assessment were rehabilitation rate measures.[31]

Office of Inspector General Audit 2007

In 2007, the VA Office of Inspector General (OIG) performed an audit of VR&E program operations to evaluate program results and performance. The objectives of the audit were to (1) evaluate aspects of program results and performance, including accuracy of performance measurement and reporting; (2) assess information on the reasons veterans discontinue participation in the program and actions taken to reduce the probability of veterans dropping out of the program; and (3) evaluate and determine the effects of the statutory annual cap on veterans eligible for independent living benefits. The audit was based on information site visits and interviews with VA staff, review of documents and reports, and review of a large sample of case files including a comparison of the hard copy files to electronic administrative data (VA OIG 2007).

c.     Findings

VR&E Task Force 2004

The general conclusions of the task force were that (1) the VR&E program has not been a priority of the VA; (2) the program has limited capacity to manage its growing workload; and (3) the VR&E program must be redesigned for the 21st century employment environment (VA VR&E Task Force 2004). The task force’s report included 110 specific recommendations covering a wide variety of management, process, and service delivery activities. As of 2007, the VR&E program reported that it had implemented 89 of the recommendations (VA OIG 2007).

 

One of the key recommendations of the task force led to the program developing a new Five-Track Service Delivery System designed to make VR&E services more employment-focused. Veterans choose one of five tracks to achieve their employment goals, with each track consisting of specific services and strategies for employment and independent living that are most suited to the employment objectives. The five tracks include reemployment; rapid access to employment; self-employment; employment through long-term services; and independent living services (Crane et al. 2008).

PART

The 2006 PART assessment concluded that the VR&E program is performing adequately based on the annual performance goals.[32] Like PART measures used in other programs, the performance measures focus on whether certain targets have been achieved, but because there is no counterfactual, they do not indicate the extent to which the program contributed to that achievement; they serve purely as descriptive performance benchmarks. Exhibit 7 shows the rehabilitation rate program performance measures. Although it appears that the program met its targets in 2007, the OIG audit report (discussed below) criticized the measures because they excluded participants who dropped out of the program before becoming rehabilitated. If these individuals had been included in the calculations, the rehabilitations rates would be much lower, as the majority of initial participants subsequently dropped out of the program.

Exhibit 7

VR&E Program Performance Measures

Year/Measure

Target

Actual

Rehabilitation Rate

2005

66%

63%

2006

69%

73%

2007

73%

73%

Serious Employment Handicap Rehabilitation Rate

2005

N/A

64%

2006

65%

73%

2007

74%

73%

Source: www.ExpectMore.gov.

 

Office of Inspector General Audit 2007

The OIG report criticized the VR&E performance measures reported in the agency’s performance and accountability report (PAR). Specifically, the VR&E rehabilitation rate calculations were not fully explained in the PAR report. The fact that the measures excluded participants who discontinued participation was not disclosed. The report noted that the PAR should provide information on total program participants, including those who discontinued participation in the program, those who obtained and maintained employment, and those who achieved independent living goals. It also noted that most veterans discontinued participation in the program and were not rehabilitated, however, no information was available for the VA to analyze the reasons for the high rate of program discontinuation. The OIG report recommended that the VA better explain the methodology used for VR&E rehabilitation rate calculations reported in the PAR; ensure that the PAR includes data on total program participants; and develop methods and procedures to determine why veterans discontinue participation in the program before they are rehabilitated (VA OIG 2007).

F.    Internal Revenue Service

1.    Business Tax Incentives to Hire and Accommodate People with Disabilities

a.    Intervention and Target Population

Three federal tax incentives were created to help businesses with the costs of making their facilities accessible, and to encourage the employment of people with disabilities.

 

The Work Opportunity Tax Credit (WOTC), established under the Small Business Job Protection Act of 1996, applies to many economically disadvantaged workers, including those with disabilities using veterans, vocational rehabilitation, or Ticket to Work program services, or receiving SSI.[33] The WOTC gives employers a tax credit when they hire or retain economically disadvantaged individuals. A firm hiring a qualified individual who remains employed for 400 or more hours during the first 12 months can receive a credit of 40 percent of the individual’s qualified wages up to a maximum on $6,000. If the individual is employed for at least 120 hours, but fewer than 400 hours during the first 12 months, the business can receive a tax credit of 25 percent of the individual’s qualified wages, again to a maximum of $6,000.[34]

 

The Disabled Access Tax Credit (Section 44 of the IRS Code), introduced in 1990, gives a tax credit of up to $5,000 to small businesses for certain costs they incur when making their workplace ADA compliant. This includes costs incurred to:

·      Remove architectural, communication, physical, or transportation barriers that prevent a business from being accessible to, or usable by, individuals with disabilities

·      Provide qualified readers, taped texts, and other effective methods of making materials accessible to people with visual impairments

·      Provide qualified interpreters or other effective methods of making orally delivered materials available to individuals with hearing impairments

·      Acquire or modify equipment or devices for individuals with disabilities

·      Provide other similar services, modifications, materials, or equipment[35]

The Barrier Removal Tax Deduction (Section 190 of the IRS Code), introduced in 1976 (and amended in 1990), allows a deduction of up to $15,000 per year for capital expenses incurred when a business of any size makes its building or transportation more accessible for people with disabilities. This deduction can be used in combination with the Disabled Access Credit if the expenses qualify under both provisions.[36]

b.    Evaluation Activities

In 2002, GAO published an evaluation of the effectiveness of the three business tax incentives (GAO 2002c). The evaluation specifically focused on the effectiveness of the provisions in promoting the employment of people with disabilities. GAO analyzed tax data from the Internal Revenue Service (IRS) Statistics of Income Programs for 1999. However, the findings of that component of the study were extremely limited for a number of reasons. First, there was no information on the barrier-removal deduction. For the other incentives, only the credit amount was listed, so it could have applied to accommodations not related to the employment of people with disabilities; the WOTC can apply to other groups—not just people with disabilitiesand the Disabled Access Credit can be used for improvements in customer accessibility. Additionally, the tax return data were not adequate for assessing how the tax incentives affected employer attitudes toward hiring, retaining, and making accommodations for people with disabilities.

 

To complement the very limited data available from the IRS, the GAO performed a literature review and conducted interviews with many people with interests relevant to the tax incentives. In its literature review, GAO was only able to find studies pertaining to the WOTC. Interviewees included officials from the Departments of Education, Labor, Justice, and the Treasury, and the Equal Employment Opportunity Commission; state officials in New York and California; businesses of various sizes; tax preparers; disability organizations; academic experts; and business groups.

c.     Findings

Based on the analyses of IRS data for 1999, GAO found that only a very small share of corporations or individuals with business affiliations (for example, sole proprietorships, partnerships, farms, and individuals with rental properties) used the two business tax provisions for which IRS data were available:

·      One out of every 790 corporations and one out of 3,450 individuals with a business affiliation reported the WOTC on their tax returns. Corporations in the retail and service industries accounted for the largest share of the credits. The total value of the credits was approximately $254 million, with the average credit being about $106,000 among corporations and $3,800 among individuals with a business affiliation reporting the credit on their tax returns.

·      One out of every 680 corporations and one out of 1,570 individuals with a business affiliation reported the Disabled Access Credit on their tax returns. Individual taxpayers with a business affiliation accounted for the majority of the credits. Individual and corporate taxpayers associated with the providers of health care and other social assistance services accounted for the largest share of the disabled access credits. The total value of the credits was approximately $59 million, with an average credit of about $2,800.

Through interviews and literature reviews, GAO found some indication that employers do take tax incentives into consideration, but could not determine if the incentives directly helped people with disabilities find or retain employment. Additionally, it was unclear if the credits given to employers were for people they would have hired anyway, without the credit.

 

Interviewees suggested ways for improving the effectiveness of the tax credits: increasing outreach so more people would be aware of the incentives; making the requirements clearer; raising the maximum claim amount; and expanding eligibility. But GAO was unsure if the suggested changes would lead to increased effectiveness of the provisions at an acceptable cost to the government.

 

GAO’s evaluation of the effectiveness of the business tax incentives was severely hampered by a lack of data and an inability to determine the effects on employment given the data available and the manner in which the provisions were implemented. The study was unable to definitively determine whether the incentives had any real effect on the hiring or retention of people with disabilities.

G.   Social Security Administration

1.    Accelerated Benefits Demonstration

a.    Intervention and Target Population

In January 2006, SSA awarded a contract to MDRC to conduct the Accelerated Benefits (AB) Demonstration. The AB demonstration is a study designed to assess whether providing new DI beneficiaries who are uninsured with health insurance benefits and other supports will stabilize or improve their health and help them return to work. In this demonstration, newly entitled DI beneficiaries (that is, those who have at least 18 months of the 24-month waiting period for Medicare eligibility remaining) ages 18 to 54 and who have no health insurance coverage were randomly assigned to three groups: a control group that just received their regular DI benefits; a treatment group that received immediate access to health care benefits (AB Basic); and a second treatment group that received health care benefits and additional care management and employment-related supports (AB Plus). The study will assess whether the provision of AB and AB Plus for a period of two to three years will help new beneficiaries improve their health and functioning and return to work. The study will be conducted over a five and a half-year period.


b.    Evaluation Activities

The AB demonstration is using an experimental design to evaluate how the intervention affects the health, health care utilization, employment, and DI benefit receipt of a sample of new DI beneficiaries who are randomly assigned to one of two treatment groups (AB Basic and AB Plus). A total of 2,000 DI beneficiaries residing in 53 large metropolitan areas will participate in the study (Wittenburg et al. 2008). While the demonstration features and evaluation design are not fully specified in public, based on the RFP issued by SSA [37] and early information provided by MDRC,[38] it appears that the evaluation will include process, impact, and cost-benefit analysis components. Some of the key research issues to be addressed by the evaluation include the following:

·      What are the challenges faced in creating and implementing a system of medical benefits?

·      How does accelerated access to health benefits affect health care service utilization, health outcomes, employment, and reliance on DI benefits?

·      What is the added effect of providing care management in addition to health benefits?

·      Does accelerated access to health benefits have larger effects for some groups of DI beneficiaries than for others?

In addition, SSA has requested that a cost-benefit analysis be conducted in order to understand the annual cost of the project and whether the program’s cost can ultimately be justified by the long-term benefits to the beneficiaries and by potential savings to the Trust Funds and other federal programs.

 

The evaluation will rely on data obtained from SSA administrative files, baseline and follow-up surveys of study participants, demonstration operations data, and site visits and interviews with intervention sites.

c.     Progress to Date and What Will Be Learned in the Future

The initial phase of the demonstration began in November 2007 and full rollout began in March 2008. Evaluation reports are expected to be completed in February 2009, October 2009, and January 2011 (Wittenburg et al. 2008).

 

Based on the information currently available, it appears that the impact evaluation will provide substantial information on the short- and longer-term impacts of providing public health insurance to DI beneficiaries earlier than under current law. The evaluation will focus on the impact of the two variations of the intervention on health status, health care utilization, employment, earnings, duration of employment, and dependence on DI benefits. The findings from the evaluation should also be able to provide information about the determinants of successful return to work and the beneficiary characteristics most correlated with successful outcomes.

 

Based on the RFP issued by SSA, it appears that the cost-benefit analysis will focus on estimating the annual costs (including net cost) of the demonstration project and the annual cost (including net cost) that would have been incurred in the absence of the project. The evaluation will consider the costs and savings to the Trust Funds and other federal programs including long-term costs and savings. It does not appear that costs or benefits occurring outside of federal programs (for example, state expenditures on Medicaid and other assistance programs, or costs to employers) will be considered.

 

It does not appear that the evaluation will assess whether the provision of immediate health benefits leads to any induced demand for DI benefits. As the demonstration is targeting only uninsured DI beneficiaries, is also unlikely that the evaluation will assess the degree to which accelerated access to health benefits crowds out other sources of insurance (such as employer-sponsored insurance or Medicaid).

2.    Benefit Offset Demonstration

a.    Intervention and Target Population

The Ticket Act requires SSA to conduct demonstration projects for the purpose of evaluating a program for DI beneficiaries under which benefits are reduced by $1 for each $2 of the beneficiary’s earnings above a specified amount. The Benefit Offset National Demonstration (BOND) will test a $1 reduction in benefits for every $2 in earnings above SGA, with the goal of enabling more beneficiaries to return to work and maximize their employment, earnings, and independence. Currently, DI beneficiaries lose their entire cash benefit if their earnings exceed the Substantial Gainful Activity (SGA) threshold by any amount for a sustained period. BOND will also test the provision of intensive benefits counseling. Some treatment groups will be provided with more intensive versions of currently available benefit counseling services, allowing more personalized support to help with the complications of returning to work. Some evidence suggests that financial incentives have a limited impact on employment and earnings because participants lack understanding of the benefit schedules; provision of more comprehensive benefits counseling might therefore increase the impact of the incentives. The more intensive counseling will be tested both in combination with the offset and as the sole addition to the current DI work incentives. The offset component of the treatment will be offered to samples of DI-only and concurrent beneficiaries residing in 10 sites across the country. The intensive benefits counseling component will be offered only to a sample of DI-only beneficiaries in these sites. The contract for the design of the national study was awarded to Abt Associates on September 30, 2004 (SSA 2007).

 

Concurrent with the design of the national demonstration, SSA sponsored a four-state pilot offset demonstration in Connecticut, Utah, Vermont, and Wisconsin. The purpose of the four-state pilot was to collect early information on the benefit offset demonstration that will be useful in developing the national study. The four-state pilot cooperative agreements were awarded in April 2005, and the states began recruiting participants for the demonstration in August 2005.[39] Treatment group participants in the state pilots receive a $1 for $2 DI benefit offset for earnings above SGA for up to 72 months following the usual nine-month DI trial work period. Each state in the pilot instituted its own methods for participant recruitment, informed consent, and the nature of any complementary intervention services that would be provided to pilot participants (for example, benefits counseling or referrals to local employment supports). The four-state pilot will terminate in January 2009..[40]

b.    Evaluation Activities

Details of the demonstration and evaluation designs for BOND are not yet public. Based on information contained in the original RFP, it appears that BOND will be conducted using a random assignment design at a set of sites selected such that they are nationally representative of all DI beneficiaries in the BOND target population. Large sample sizes will be needed to detect fairly small impacts of the interventions on employment and benefit receipt outcomes. The evaluation will likely include impact, participation, process, and cost-benefit analysis components and rely on data collected from a variety of sources, including SSA and CMS administrative data, process data generated by the intervention, beneficiary surveys, and site visits and interviews with demonstration staff and other community stakeholders. The evaluation will focus on outcomes related to participation, employment, earnings, SSA benefit receipt, income, and reliance on other public assistance programs.

 

Each state involved in the four-state pilot designed its own evaluation and data collection methods. All were required to implement a random assignment design, but otherwise had a large degree of flexibility in how to implement the intervention, collect data, and conduct evaluations of the implementation process and demonstration outcomes. Project enrollment information, state program administrative data files (including Medicaid, unemployment insurance, and vocational rehabilitation), and interviews with participants were the primary sources of data to be used to assess the pilot projects.

c.     Progress to Date and What Will Be Learned in the Future

BOND was initially scheduled to begin in late 2007,[41] but design changes have delayed the start of the demonstration until a currently unknown date. As the demonstration will use an experimental research design, the evaluation is expected to produce rigorous estimates of the impact of the offset and intensive benefits counseling on employment and benefit receipt outcomes.

As noted above, the four-state pilot has been underway since late 2005. Each of the four states presented preliminary evaluation findings at a March 2007 meeting sponsored by SSA. Some of the general findings from the states’ experiences include the following (Silverstein and Jensen 2007):

·      Each of the states experienced difficulty recruiting and enrolling large numbers of participants. Although the projects found it useful to use existing disability service providers (such as VR, mental health, and development disabilities providers) to recruit participants, many candidates were not eligible for the demonstration. The 72-month extended period of eligibility limit in the demonstration served to exclude many DI beneficiaries who had demonstrated a capacity to work from participating in the project. In addition, the informed consent process was very time intensive.

·      The role of benefit counseling was critical in the demonstration. The complexity of the offset, the fact that benefit adjustments were not made in a timely manner by SSA, and information about how and when to report earnings were all factors that frequently needed to be addressed by benefits counselors.

·      All states experienced delays with respect to SSA conducting work activity reviews and accurate benefit adjustments.

States had a number of suggestions for making BOND as effective as possible. Primary among them was the need to have sufficient benefits counseling resources available for participants, to try to keep the intervention as simple as possible for beneficiaries to understand, to have the mechanisms in place for SSA to provide timely and accurate information about benefits and eligibility status, and to make benefit adjustments in response to earnings changes in a timely and accurate manner so that over- and underpayments are avoided. More detailed participation and outcome information may be forthcoming from the state pilots, which will terminate in January 2009.

3.    Benefits Planning, Assistance and Outreach/Work Incentive Planning and Assistance

a.    Intervention and Target Population

SSA, as authorized by Section 121 of the Ticket Act, awarded 116 cooperative agreements nationwide to a variety of community organizations under the auspices of the Benefits Planning, Assistance and Outreach (BPAO) program beginning in 2000.[42] The purpose of the BPAO initiative is to provide SSA disability beneficiaries (both SSI and DI beneficiaries) with accurate and timely information about SSA work incentives and other federal efforts to remove regulatory and programmatic barriers to employment for persons with disabilities. Trained benefits specialists in local BPAO programs work with individual beneficiaries to explain the many regulations, provisions, work incentives, and special programs that complicate an individual’s decision to enter or reenter the workforce. Benefits specialists are instructed not to tell beneficiaries what to do or to make specific recommendations. Instead, they are to allow beneficiaries to make their own informed decisions based on complete and accurate information. In addition, they support individuals who choose to enter employment by assisting them to comply with all relevant regulations and reporting procedures. The primary purpose of the programs is to educate beneficiaries about work incentives. Although increased employment would be a desirable outcome for the BPAO program, the impact of the initiative on employment is a secondary consideration and need not be positive.

 

Each BPAO project conducts an outreach program in its area to identify beneficiaries who might be eligible for their services. SSA does not provide any lists regarding current beneficiaries but may make referrals to a local BPAO project. The beneficiaries are given counseling by benefits specialists during one or more contacts. These contacts may take place on the telephone or in person. The effect of their work activity is discussed, and the ramifications of this work for their benefits are explained. Benefits specialists may contact third parties on behalf of beneficiaries to verify information or to assist beneficiaries in carrying out the actions required of them. As of September 2006, over 250,000 beneficiaries had received assistance from one of the BPAO organizations operating nationwide.[43]

 

Benefit specialists under the BPAO program received two weeks of intensive training on work incentive issues. The training is provided by one of three BPAO technical assistance centers funded by SSA.[44] After the initial training, benefit specialists receive ongoing technical assistance and support through these centers.

 

In 2006, SSA renamed the BPAO initiative the Work Incentive Planning and Assistance (WIPA) project and issued a new Request for Applications for WIPA projects. Benefit specialists under the WIPA program are referred to as Community Work Incentives Coordinators (CWICs). The WIPA projects operate similarly to the BPAO programs, but will emphasize improved community partnerships. One of the ways in which this will be done is by conducting periodic Work Incentives Seminars (WISE) to provide beneficiaries the opportunity to meet directly with WIPAs, Employment Networks, and public and private community-based organizations that provide services to people with disabilities. Each WISE is expected to last approximately two hours and will provide beneficiaries with information about available work incentives and job supports needed to either assign their Ticket or pursue other employment options. SSA made the first round of WIPA awards in September 2006, with awards to 99 projects in 49 states. A second-round application process was implemented in late 2006 to recruit additional programs for uncovered areas of the country. As of mid-2007, 104 WIPA projects were operating nationwide. Most had previously operated as a BPAO program, but a number of new organizations have become involved as WIPAs.[45], [46], [47] In August 2007, SSA awarded a contract to Virginia Commonwealth University to act as the training and technical assistance center for the WIPA projects.

b.    Evaluation Activities

We have identified two efforts to evaluate aspects of the BPAO programs, and a third evaluation effort for the WIPA projects that is currently in the planning stage.

Evaluation of Vermont’s Benefits Counseling Program (Tremblay et al. 2004)

Vermont established a specialized benefits counseling program in 1999 under its SSA-sponsored State Partnership Initiative. Although not technically a BPAO program when it began in 1999, it operated in much the same way and subsequently obtained funding from SSA to operate as a BPAO.

 

Staff of the Vermont Division of Vocational Rehabilitation conducted an evaluation of the impact of benefits counseling on a sample of 672 SSA disability beneficiaries who were state VR service users and who had received benefits counseling services between 1999 and 2002. The study used a quasi-experimental design to estimate the impact of the benefits counseling on earnings. The earnings of treatment group members over a four-year period (two years before the intervention and two years after) were compared to the earnings of members of two matched comparison groups who did not receive benefits counseling services: a “contemporaneous” comparison group comprised of VR service users who received services at the same time as members of the treatment group; and an “historical” comparison group made up of VR service users who had received services approximately two and a half years earlier than the treatment group members. The historical comparison group was used to confirm findings from the contemporaneous group, as the latter was susceptible to treatment spillover effects. Both comparison groups were matched to treatment group members based on a variety of VR and SSA program-related characteristics.

 

The analysis used quarterly earnings data on treatment and comparison group members over a four-year period obtained from Vermont’s Unemployment Insurance program. Regression modeling was used to analyze differences in the longitudinal earnings patterns between treatment and comparison group members.

2002 BPAO Satisfaction Survey (SSA 2004)

Between November and December 2002, SSA conducted a phone survey with 1,764 individuals who were identified as having received services from one of the 116 operating BPAO programs between October 2001 and August 2002. The purpose of the survey was to obtain feedback about the experiences and opinions of BPAO program participants. The study was not intended to assess impacts of the program on beneficiary knowledge, behavior, or employment.

 

Using SSA’s standard six‑point rating scale that ranges from “excellent” to “very poor,” beneficiaries were asked to evaluate their overall satisfaction with the information and services provided, as well as the more tangible aspects of the experience such as the convenience, accessibility, and privacy of the location where they met with the benefits specialists. Beneficiaries also provided their level of satisfaction (“very satisfied” to “very dissatisfied”) with access to the benefits specialist, both in person and by telephone. The clarity and helpfulness of the information and actions of the benefits specialists were assessed using a four‑point scale from “strongly agree” to “strongly disagree.” The survey also contained questions about the participant’s work activity and education.

SSA’s WIPA Evaluation

In 2007, SSA awarded a contract to MPR to conduct a process evaluation of the WIPA program. The evaluation will focus on the initial stages of the WIPA program, including the transition out of the BPAO program. The process analysis is intended to serve as a forerunner to an outcomes-oriented evaluation that might be undertaken in the future, and as a means to capture any data that might be lost after the early phase of WIPA implementation is complete (Buschman et al. 2007).

 

The process evaluation will document how the WIPA program was implemented, how the program is monitored by SSA, the characteristics of the organizations acting as WIPAs and the services they provide, the characteristics of the beneficiaries served by WIPAs, interactions and competition with other local organizations, the kinds of data that are collected and maintained by the WIPAs, and stakeholder views about what constitutes program success. The evaluation will be based on interviews, site visits, and focus groups conducted with SSA staff, WIPA staff, and beneficiaries served by the program. It will also include a review of documents, files, and data maintained by the WIPA programs (Buschman et al. 2007).

c.     Findings and What Will Be Learned in the Future

From the Tremblay et al. (2004) study, there is some evidence that benefits counseling had a positive impact on the earnings of SSA disability beneficiaries in Vermont. Although there are a number of limitations associated with the study in attempting to draw conclusions about the effectiveness of BPAO programs in general (comparability of the treatment and comparison groups, potential effects of selection bias, comparability of the intervention and service population to that of other BPAO programs), the findings of a significant positive impact on employment cannot be completely discounted.

We also know from the SSA (2004) satisfaction study that, in general, beneficiaries receiving benefits counseling services from BPAOs were satisfied with the services they received (89 percent rated the services as excellent, very good, or good). While most beneficiaries were generally satisfied overall and with various aspects of their services, a minority expressed complaints related to the inability of benefits specialists to provide complete and accurate information, failure to return calls promptly, and lack of tangible help in finding employment. The convenience of the meeting location was one of the lowest-rated aspects of service (84 percent rated it excellent, very good, or good) because of the long distance respondents had to travel, the lack of transportation, and inadequate parking. Respondents also reported the lowest level of agreement (87 percent) with the statement that, after talking with the benefits specialist, they understood what they were supposed to do or what was supposed to happen next.

 

Findings from the WIPA process evaluation are expected to be released in early 2009. From this evaluation, we will gain a better understanding of how the benefit specialists are operating, stakeholder perceptions of how well the program is performing, and the potential for an assessment of program outcomes. Given the manner in which the WIPA programs are being implemented, it is unlikely that a rigorous impact analysis of the effects of the program on employment could be conducted on a national scale. Smaller-scale assessments, however, might be possible with the cooperation of specific WIPA programs. While much could be learned from a small-scale, rigorous assessment of the impact of WIPA programs on employment, given the potentially large degree of variation across programs (for example, in service quality and focus, characteristics of clients, and local factors affecting program performance), it would be difficult to extrapolate the findings to the national program. A broader but less rigorous investigation of employment outcomes, combined with the information of the nature that is planned for the WIPA process evaluation, might be useful in identifying the factors that appear to be associated with improved employment outcomes.

4.    Employment Support Representatives/Area Work Incentive Coordinators

a.    Intervention and Target Population

Section 121 of the Ticket to Work and Work Incentive Improvement Act of 1999 mandates that SSA shall “. . . establish a corps of trained, accessible, and responsive work incentives specialists . . .” to assist beneficiaries with disabilities who want to start or continue working. The mandate came in response to criticisms of SSA by beneficiaries with disabilities who want to start or continue working, their families, advocates, and service providers, who complained that SSA field staff lacked adequate capability to provide accurate and accessible information about SSA’s employment support programs and to process disability work-issue workloads timely and accurately.

 

In response to the Ticket Act mandate, SSA began piloting a new, temporary position, the Employment Support Representative (ESR), with 32 ESRs serving 54 sites nationally. SSA piloted the ESR position by testing three different models to determine whether and how best to implement the position nationally. The ESR pilot began in July 2000 and concluded in September 2001. ESRs were selected from among claims representatives, underwent six weeks of intensive training on SSA work incentive provisions and related issues, and received a promotion to the ESR position. In addition to the role of providing beneficiaries information about work incentive provisions, the 32 ESRs were tasked with conducting outreach and providing information to the general disability community. The ESR pilot ended in September 2001, and SSA issued an evaluation of the pilot in November 2001 (SSA 2001).

 

Based in part on the findings of the ESR pilot, in May 2003 SSA implemented two new types of staff positions: an area work incentive coordinator (AWIC) and a work incentives liaison (WIL). AWICs are responsible for providing assistance to SSA field office staff on employment support and outreach issues by coordinating and/or conducting local public outreach on work incentives; providing, coordinating, and/or overseeing training for all personnel on SSA’s employment support programs; handling sensitive or high profile disability work-issue cases; and monitoring the disability work-related issues in their respective areas. The 58 AWICs coordinate with the 1,335 WILs in local offices, the public affairs specialists, the Plan for Achieving Self-Support Cadre members, and other SSA personnel to provide improved services and information on SSA’s employment support programs. Each AWIC provides technical support and training to 20 to 30 field offices and networks with community agencies and other organizations that provide employment services to people with disabilities. The WILs provide technical assistance to staff and field office management, including assistance on complex cases. The WILs also process cases involving work incentives and maintain ongoing contact with beneficiaries. WILs are also involved in TTW marketing and outreach, but their level of involvement varies according to the needs of the SSA field office.[48]

b.    Evaluation Activities

As noted above, SSA began a pilot of the ESR position in July 2000, and completed an internal evaluation of the pilot in November 2001 (SSA 2001). The pilot tested three variants of an ESR model:

·      Model 1, Expanded Field Office (FO) Model: ESRs were stationed in an SSA FO and served that FO and, in some instances, affiliated FOs. The ESRs were expert in all of SSA’s employment support programs, and adjudicated Plans for Achieving Self Support (PASSs).

·      Model 2, Expanded Cadre Model: ESRs were stationed in an existing PASS Cadre. They served a specific FO service area(s) within the PASS Cadre service area. These ESRs needed to travel to the FOs that they served. They were expert in all of SSA’s employment support programs, and adjudicated PASSs.

·      Model 3, Field Office/Cadre Model: ESRs were stationed in an SSA FO and served that FO and, in some instances, affiliated FOs. The ESRs were expert in all of SSA’s employment support programs, but unlike under the other models, these ESRs did not adjudicate PASSs. The PASS Cadre continued to be responsible for the adjudication of PASSs.

The evaluation of the pilot used data from several sources to assess the impact of the new position on service delivery, productivity, and customer satisfaction. The impact on beneficiary employment was not in the scope of the evaluation. The sources of data used included:

·      Surveys sent to five groups affected by ESRs: customers and community-based organizations that received ESR services; ESRs; ESR pilot site managers and area or state directors; and other SSA field employees at ESR sites

·      Information collected during a teleconference conducted with all ESRs in August 2001

·      ESR work activity as captured on an electronic work ticket” and through time and attendance data

·      ESR outreach costs

c.     Findings

The SSA (2001) ESR evaluators concluded that the position was a success. The feedback from the beneficiaries and staff of community organizations who were surveyed indicated overwhelming appreciation for and satisfaction with the single point of contact and the responsiveness of ESRs. The location of the ESR was viewed as crucial to the success of the position in dealing with customers, community organizations, and fellow employees. Models 1 and 3, where ESRs were located in and dedicated to a specific FO, performed notably better in terms of work processing, customer satisfaction, and response time.

 

Work-year projections indicated that, when all types of employment support activities (that is, work CDRs, SGA determinations, employment support inquiries, and post-entitlement actions) were considered, Model 1 was the most productive. However, Model 3 produced the highest number of employment support activities in the shortest average time.

 

In light of the success of the pilot, the evaluation recommended that SSA make the ESR a permanent position and ensure that as many service areas as feasible have ESR services. Due to cost and resource considerations, however, SSA decided to implement the ESR concept in the form of the AWIC/WIL model described previously, a more diluted version of the ESR. AWICs have a high skill level and expertise related to work and post-entitlement issues generally equivalent to that of the ESRs, but act more as technical assistance resources and trainers for the WIL staff located in the FOswho in turn deal directly with beneficiaries but generally have less expertise in work-related issues than the AWICs.

5.    Florida Freedom Initiative[49]

a.    Intervention and Target Population

The Florida Freedom Initiative (FFI) is a demonstration designed to test whether the provision of employment and microenterprise services, along with certain SSI waivers, will lead to greater employment, assets, and self-sufficiency among Social Security disability beneficiaries participating in the program. The target population is adult SSI recipients with developmental disabilities who participated in Florida’s Cash & Counseling program, called Consumer Directed Care (CDC)-plus. CDC-plus provides people receiving personal assistance services an allowance to hire personal care workers and to buy other disability-related supports in lieu of the Medicaid home- and community-based benefits they would otherwise have received.

 

Under FFI, volunteer participants receive information and training about employment and microenterprise development, and assistance with addressing human capital barriers to employment. In addition, FFI participants are subject to several waivers of SSI regulations, which are intended to promote employment and encourage asset accumulation. These waivers include the following:

1.      Exclusion of Medicaid Payments being saved for the purchase of medical or social services from the participants’ countable income in determining SSI payments and eligibility. This waiver is required to allow SSI beneficiaries to keep SSI eligibility while participating in the CDC-plus program (which provides Medicaid beneficiaries with a cash allowance so that they can manage their own care instead of receiving Medicaid’s home and community-based waiver services).

2.      Expansion of Individual Development Accounts (IDAs). The FFI IDAs are savings accounts excluded from SSI resource limits when determining SSI eligibility. Although some IDAs are set up to contribute federal or state matching funds to reward participants for saving, the IDAs offered under the FFI program did not have this feature (as of program implementation).

3.      Increased Exclusion for Earned Income. Currently, the monthly benefit checks of SSI recipients who work are reduced by 50 cents for every dollar earned after a $20 income exclusion and a $65 earned-income exclusion. This waiver raises the earned-income exclusion from $65 to $280 per month, in effect allowing those who earn more than $280 to keep about $107.5 more of their earnings each month.

4.      Modified Goal for a Plan for Achieving Self-Support (PASS). A PASS allows a beneficiary to set aside income or resources for a specified time to save for a work goal. Under the usual PASS rules, someone who establishes a PASS for education must specify the work goal. This waiver allows FFI participants to specify post-secondary education as a PASS goal without initially specifying the work goal, as long as the PASS includes a step for specifying a work goal at least one year before completing the coursework.

5.      Suspension of Continuing Disability Reviews (CDRs). SSA must periodically review medical and other evidence to determine whether an individual continues to meet the eligibility requirements for benefits. This waiver suspends CDRs for FFI participants while they participate in the demonstration.

The FFI demonstration was collaboratively sponsored by federal and state agencies, including Florida Agency for Persons with Disabilities; the Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of Health and Human Services; SSA; and the CMS. The FFI program was funded from September 2003 through September 2007. FFI enrolled its first participant on September 30, 2005.

b.    Evaluation Activities

The Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of Health and Human Services issued a contract to MPR to design an evaluation of the FFI demonstration. The proposed design (completed in May 2005) involved an analysis of the FFI program implementation, a participation analysis, and an analysis of employment, earnings, asset accumulation, and benefit receipt outcomes based on a pre-post comparison of outcomes among FFI participants. The impact evaluation was designed to be based on SSA administrative data and on data collected via baseline and follow-up surveys of FFI participants.

 

The proposed pre-post design for estimating the impact of FFI on employment, earnings, assets, and benefits was arrived at based on several considerations: due to low initial enrollment and the small target population from which participants would be recruited (700 CDC-plus participants), it was believed that sample sizes would be too small to support a more rigorous, random assignment design; and because the target population had just recently undergone a random assignment process (to become participants in the CDC-plus program), FFI program officials did not want to adopt a randomized design for FFI. Non-experimental designs were also considered, but rejected due to concerns about selection bias, low enrollment, and the costs associated with collecting data from comparison group members.

c.     Progress to Date and What Will Be Learned in the Future

FFI program enrollment lagged far below expectations. As of February 2007, one source noted that only 35 people had enrolled in FFI (Center for Workers with Disabilities 2007). This source and another (GAO 2008b) noted that evaluation of the FFI program was deemed infeasible with so few participants and so no formal evaluation has been conducted.

6.    Mental Health Treatment Study[50]

a.    Intervention and Target Population

In September 2005, SSA awarded a contract to Westat to design, implement, and evaluate a demonstration that tests whether better access to mental health treatment and employment supports will lead to improved health and functioning, increased employment and earnings, and reduced reliance on public benefits among DI beneficiaries with schizophrenia and affective disorders. The rationale for the study is that DI beneficiaries with schizophrenia and affective disorders represent a large and growing share of all DI beneficiaries, and many have conditions that are treatable, but they do not receive adequate treatment or fulfill their employment potential due to lack of health insurance coverage and a lack of access to needed behavioral health and employment supports.

 

The Mental Health Treatment Study (MHTS) is being conducted in 22 sites nationwide. Over an 18-month recruitment period, the study plan calls for enrollment of 3,000 DI beneficiaries with schizophrenia or affective disorders, ages 18 to 55. Half of the enrollees were to be randomly assigned to treatment and half to control. Treatment group members are to receive a 24-month intervention comprised of the following components:

·      Health insurance coverage for participants without insurance, and supplemental coverage or assistance for those with inadequate insurance

·      Coverage of all out-of-pocket costs associated with all behavioral health services

·      Systematic management of medications

·      Employment supports that are based on the Individual Placement and Support supported employment model (Drake 1998; Drake et al. 1999), which includes placement in competitive employment, employment supports integrated with treatment, and continuous follow-along supports

A key feature of the intervention is the integrated treatment approach. Under this approach, all of a participant’s supported employment and behavioral health services are coordinated and provided by a single treatment team located at the local demonstration site.

b.    Evaluation Activities

The planned MHTS evaluation will include implementation, participation, impact, and cost analyses.

 

The implementation analysis will document each site’s adherence to evidence-based practices and the study protocol. The participation analysis will assess the beneficiary characteristics that predict study enrollment among members of the target population.

 

The impact analysis will be based on a randomized control group design and will measure the impact of the intervention on a variety of outcomes, particularly those related to employment, employment stability, and job quality. The analysis will also examine the impact of the intervention on benefit receipt, health care utilization, health and functioning, and quality of life. The impact analysis will include an assessment of the degree to which treatment impacts varied across treatment group members with different characteristics.

 

The cost analysis will document the costs associated with the various types of services and medications provided to treatment group members, and assess the relationships between participant characteristics and variations in intervention costs.

 

The evaluation will rely on SSA administrative data, data collected via a baseline survey of treatment and control group members, seven quarterly interviews with treatment group members, and a final follow-up survey of treatment and control group members.

c.     Progress to Date and What Will Be Learned in the Future

MHTS enrollment began in October 2006. If the study proceeds as originally scheduled, the findings from the final evaluation should become available sometime in 2011. Assuming that the study is able to meet its sample size targets, the evaluation will be able to produce rigorous estimates of the impact of the intervention on functioning, employment, and benefit receipt. The MHTS will demonstrate whether the provision of adequate health care and employment supports can result in significant changes in health, employment, and benefit receipt outcomes among beneficiaries with schizophrenia or affective disorders. From the information available, it is unclear to what extent the evaluation will consider impacts of the intervention on other public programs, or consider more generally the social costs and benefits of the intervention.

7.    Substantial Gainful Activity Level Increase

a.    Intervention and Target Population

On July 1, 1999, the definition of the substantial gainful activity (SGA) level was raised from $500 to $700 of earned income per month. The SGA level is the amount of income that an individual with a disability applying to the DI and SSI program may earn to be initially considered eligible for the programs. It is also the amount that DI beneficiaries may earn without losing disability benefits. The SGA level provides a substantial incentive for DI beneficiaries to work only up the income level of the threshold itself to avoid the complete loss of benefits.[51] The adjustment in 1999 was the first of its kind since 1990, and reflected the growth in average wages since that time. In December 2000, SSA finalized a rule that allows for the annual indexing of the SGA level. Adjustments to SGA are now applied annually to reflect annual growth in national average wages. The nonblind SGA level in 2008 was $940 per month.

 

There are at least two potential effects of this change, depending on the employment status of an individual before the change went into effect:

·      For beneficiaries earning below the former SGA level of $500, the change represents an opportunity to increase earnings without loss of benefits. Those already working might increase the number of hours they work to take advantage of this opportunity, and those who did not perceive the effort of finding a job worthwhile under the old limit might decide to seek employment under the new limit.

·      For nonbeneficiaries earning more than the former $500 threshold but at or near the current $940 level, there is an incentive to restrict earnings to the new threshold to become eligible for disability benefits, which might result in fewer program exits and a decrease in earnings among members of this group.

The presumption is that the first effect will dramatically outweigh the second—that is, there are far more individuals with disabilities working just below the threshold than there are working just above it. It is anticipated that this change will facilitate meaningful work among beneficiaries by reducing the likelihood that their return-to-work efforts will result in the immediate loss of cash benefits.

b.    Evaluation Activities

In 2002, GAO released a report that assessed the effects of the SGA level on the DI program (GAO 2002). The assessment by GAO was mandated under the Ticket Act. GAO’s study considered the effects of the SGA level on the work patterns of DI beneficiaries as well as the effects of the SGA level on DI program entry and exit rates. The study was based on a review of the economic and disability literature related to the effects of the SGA; an analysis of DI program data covering the period of 1985 through 1997; and interviews with SSA policy officials, academic experts, and representatives from disability advocacy groups.

 

GAO’s quantitative analysis relied on data from SSA’s Continuous Work History Sample (CWHS). The CWHS is a file representing a longitudinal sample of one percent of all active Social Security accounts. The file contains data on earnings for purposes of analyzing the lifetime earnings patterns of individuals. The earnings data are annual, however, so monthly SGA levels had to be annualized, with the analysis based entirely on the annualized SGA amounts. The CWHS also contains data abstracted from SSA administrative files, including information on DI program eligibility and demographic characteristics. From the CWHS file, GAO selected a subsample of approximately 90,000 DI beneficiaries eligible for DI benefits at some time between 1984 and 1998. During the study period, the SGA level for nonblind individuals increased from $300 per month (from 1980 to 1989) to $500 per month (from 1990 to mid-1999). The SGA level for blind individuals increased steadily over the entire period, from $580 per month in 1984, to $1,050 per month in 1998. GAO also drew a subsample of about 10,000 DI beneficiaries who reached age 65 during the 1987 to 1993 period to examine beneficiary work activity after conversion to the retirement program, where SGA earnings restrictions do not apply.

 

GAO analyzed the annual earnings data to determine if trends observed over the 1984 to 1998 period were suggestive of any effects of the SGA level on the earnings of DI beneficiaries. If the SGA level does indeed affect earnings, a priori, one would expect to see evidence of the following as a consequence of the monthly nonblind SGA level increase from $300 to $500 in 1990:

·      A greater share of DI beneficiaries working during the post-1990 period

·      Among working DI beneficiaries, higher levels of earnings in the post-1990 period

·      An increase in the share of working beneficiaries with earnings between $300 and $500 per month during the post-1990 period

One might also expect to see blind beneficiaries generally working at higher rates and levels than nonblind beneficiaries, as in each year, they were subject to substantially higher SGA limits. In addition, one might expect to find relatively higher levels of employment after conversion to the retirement program, when a beneficiary is no longer subject to the SGA earnings limits.

 

With respect to effects of the SGA on program entry and exits, an increase in the SGA level is hypothesized to increase program entry because those working between the old and new SGA limit would now qualify for benefits, and those working just above the new limit might restrict their earnings to qualify for the program. With respect to program exit, the hypothetical effect of an increase in the SGA level is ambiguous. On the one hand, beneficiaries can earn more and remain eligible for DI, and thus fewer may leave the program due to earnings. On the other hand, allowing beneficiaries to engage in greater work activity could, in the long run, result in more beneficiaries becoming capable of earning at levels leading eventually to exit from the disability rolls.

c.     Findings

In summary, GAO found only limited evidence of an effect of SGA on earnings and no conclusive evidence of any impact on program entry or exit.

 

GAO concluded that the SGA level affects the work patterns of only a small proportion of DI beneficiaries. It found that, on average, only about 7 percent of DI beneficiaries who worked in any given year during the study period had earnings near the annualized SGA level (between 75 and 100 percent of SGA). These beneficiaries comprised only about 1 percent of all DI beneficiaries. Even among beneficiaries who had earnings near the SGA level in any given year, most experienced a substantial decline in earnings over time. For example, almost half of those with earnings near the SGA level in 1985 had no earnings by 1989. GAO did find some evidence that the SGA might affect the earnings of some beneficiaries. About 13 percent of beneficiaries with earnings near the SGA level in 1985 still had earnings near the SGA level in 1995, even though the level was increased during that period. Among beneficiaries converting to the retirement program, GAO found that, among those with no earnings during the three years prior to conversion, about 7 percent had earnings in one or more years following conversion (between ages 66 and 68). The rate of employment for this group was more than double the rate of a similarly defined comparison group of DI beneficiaries ages 58-60 (3 percent had earnings in one or more years), suggesting that the SGA has an effect on earnings. Although trends in program entry and exit were analyzed, the study could draw no conclusions from the findings regarding the effect of the SGA on entry or exit given significant data limitations and a variety of factors unrelated to the SGA level that likely influenced the aggregate trends observed.

 

The GAO study is flawed in many respects, primarily due to data limitations. The data available only permitted analyses of annual, rather than monthly, earnings. This makes it impossible to precisely identify any effects of the SGA on earnings because the earnings are averaged over a 12-month period. The averages may or may not correspond to the actual monthly earnings of beneficiaries and will be particularly poor indicators of monthly earnings among beneficiaries who work sporadically. The data also could not distinguish beneficiaries who (1) had not completed their trial work period, during which time any level of earnings is permitted); (2) had completed their trial work period and had not reach the 36th month of the Extended Period of Eligibility, when, after a three-month grace period, earnings above SGA in any month result in suspension of benefits for that month only; and (3) had reached or gone past the 36th month, when earnings above SGA in any month result in termination of DI benefits.[52] Thus, the analysis includes a mix of earnings observations from among beneficiaries who (1) had no incentive to keep earnings below SGA; (2) had an incentive to keep earnings below SGA, to avoid temporary benefit loss; and (3) had an even stronger incentive to keep earnings below SGA, to avoid benefit termination. The data used in the study also could not distinguish between blind and nonblind beneficiaries. Therefore, the analysis includes a mix of beneficiaries subject to different SGA levels. In addition, comparisons that might have provided further evidence of the effect of the SGA on earnings could not be undertakenfor example, comparisons between the two groups subject to very different SGA levels and comparisons over time of blind individuals subject to a steadily increasing SGA. Finally, the data were rather old. It is unclear whether the findings from the 1984 to 1998 period would still be applicable today given the considerable changes around employment issues that have happened with the Social Security disability programs since the passage of the Ticket Act in 1999. The GAO did not conduct an evaluation of the 1999 SGA increase.

 

Despite the data limitations, there are other analyses the GAO study could have undertaken that might have provided additional, or more compelling, evidence of the effect of the SGA level on earnings. GAO chose to focus much of its analysis on the number and share of beneficiaries (and working beneficiaries) with earnings far below, near, and above SGA. While interesting, such an analysis provides little information about the effect of SGA on earnings and makes little use of the natural experiment provided by the 1990 SGA increase. It would have been interesting to see the data presented in a manner that allowed one to examine the percent of beneficiaries (and working beneficiaries) earning at the old and new SGA levels (absolute dollar levels) both before and after the policy change. Similarly, the report presents a table of the earnings distributions in 1985 and 1997. The table shows the number of beneficiaries at various annual earnings levels after adjustment for inflation. The report concludes that “even with the 67 percent increase in the SGA level in 1990, the earnings distribution of DI beneficiaries did not change considerably from 1985 to 1997” (p. 12). This is hardly evidence of the lack of an effect of the SGA on earnings given that, after adjustment for inflation, the 67 percent nominal increase in SGA (from $300 in 1985 to $500 in 1997) amounts to a five percent decrease in inflation-adjusted dollars.[53] It would have been more interesting if the GAO report had included a percent distribution of beneficiaries by both inflation-adjusted and nominal dollars

 

The study emphasizes the relatively small share of beneficiaries who work, and even smaller share who work near SGA. The conclusion from these observations that the SGA level affects very few beneficiaries seems to ignore the obvious: if a program places an extreme restriction on earnings to maintain eligibility, very few participants are likely to bother attempting work. Perhaps the most persuasive evidence of the effect of the SGA level on earnings was GAO’s findings with respect to beneficiary work activity after conversion to the retirement program. To find that employment activity among this group was more than double that of a comparison group of older DI beneficiaries is quite compelling. If post-retirement age individuals with disabilities are responsive to the elimination of the SGA restriction, one can speculate that the response among younger beneficiaries who have significantly more years of potential earnings ahead of them might be considerable.

8.    Ticket to Work Program

a.    Intervention and Target Population

The Ticket to Work and Work Incentives Improvement Act of 1999 established the Ticket to Work program (TTW). TTW is a national program that provides eligible DI and SSI disability beneficiaries with a Ticket, which can be used to obtain vocational rehabilitation (VR) or employment services through a participating provider, called an Employment Network (EN). ENs can be any type of entity willing and able to provide employment-related services to beneficiaries. There are very few restrictions on the types of entities that can become ENs. SSA, through a contractor, is responsible for recruiting a large network of ENs that can accommodate beneficiary demand for services in all states and territories.

 

Under TTW, a beneficiary can assign a Ticket to any EN willing to accept it. ENs are paid by SSA only if the beneficiary goes to work and meets or exceeds specific employment and earnings targets, and provided that the EN supplies sufficient evidence of the beneficiary’s earnings. The earnings targets and payments differ depending on in which of two TTW payment systems the EN chooses to participate (milestone-outcomes or outcomes only). Under both systems, however, most (or all) payments are not made to ENs unless a beneficiary is working and earning enough to reduce SSI and/or DI benefits to zero for 60 months. State VR agencies may choose to participate in either of the two TTW reimbursement schemes, or to receive payment based on the traditional system by which SSA has for many years reimbursed state VR agencies for services provided to beneficiaries if the beneficiary earns more than SGA for at least nine months. There is no benefit reduction requirement under the traditional system.

 

The underlying rationale for the program is that some beneficiaries currently lack the resources necessary to return to work at a level above the SGA level, either because they do not have easy access to such services or because they and their providers lack the incentive to invest resources in return-to-work activities that will lead to SSI/DI program exit. TTW confers upon a beneficiary a means to access those resources in a less restrictive manner than under SSA’s traditional VR payment program. TTW was expected to increase beneficiary demand for employment-related services and activities, to increase the number and diversity of providers serving SSI and DI beneficiaries, and to create incentives for providers to help beneficiaries achieve a level of earnings that will result in exit from the disability rolls, and thereby reduce Social Security disability program expenditures.

 

Individuals ages 18 to 64 who are currently receiving either DI or SSI disability benefits are eligible to participate in TTW. Such individuals may have either permanent impairments (impairments for which medical improvement is not expected) or temporary impairments expected to last at least 12 months (impairments where medical improvement is expected). Originally, to be eligible for TTW, those individuals with temporary impairments must have first undergone a CDR, and have been found to have experienced insufficient medical improvement to allow a return to work. In May 2008, the regulations were amended to allow those with medical improvement expected who had not yet undergone the first CDR to receive Tickets and be eligible to participate in TTW.

 

TTW was implemented in three phases: between February and June 2002, TTW was implemented in 13 states; between November 2002 and September 2003, TTW was implemented in an additional 20 states and the District of Columbia; between November 2003 and September 2004, TTW was implemented in the remaining 17 states and U.S. territories.

b.    Evaluation Activities

In June 2003, SSA contracted with MPR to conduct a five-year evaluation of TTW, based on an evaluation design that had been previously developed by The Lewin Group.[54] The evaluation is intended to address five broad issues:

·      Does TTW improve the supply of employment-related services to beneficiaries?

·      Does TTW increase beneficiary use of employment services?

·      Are TTW services effective in improving employment outcomes?

·      Does TTW generate net disability program savings?

·      Are there other social consequences associated with TTW?

The Evaluation has Three Primary Components:

The process evaluation uses survey, administrative, and qualitative data to document how the program was implemented; assess the effect of the program on the market for employment-related services (beneficiary demand, provider supply, dynamics, funding sources); and provide contextual information to help interpret impact analysis findings.

 

The participation analysis uses administrative and survey data to assess who participates in TTW and who does not; the relationships between participation and both beneficiary and area characteristics; and reasons for nonparticipation.

 

The impact analysis uses administrative data to produce estimates of impacts on enrollment in services, employment, annual earnings, exits from the disability programs for work, and net disability program savings. The impact estimation relies on a quasi-experimental design, where the outcomes of pre-implementation states are compared with the outcomes of post-implementation states, taking advantage of the phased implementation of the program.

 

Additional outcome analyses will supplement the impact analyses by documenting outcomes for which it is not possible to produce rigorous impact analyses because the data required for impact estimates are not available. These outcomes include the nature of employment services received by TTW participants and by other beneficiaries, the cost of services provided by ENs, beneficiary experiences with use of TTW and satisfaction with their experience, beneficiary income, beneficiary use of other programs, and impacts on other programs. The beneficiary outcomes are measured via the survey, and other outcomes are measured from various sources.

c.     Findings

To date, four extensive TTW evaluation reports have been released: a preliminary process evaluation report developed by The Lewin Group (Livermore et al. 2003) and three evaluation reports developed by MPR (Thornton et al. 2004, 2006, and 2007). The major findings from the most recent of these reports include the following:

·      From SSA’s perspective, TTW was challenging to implement for several reasons: the short timeframe; limited resources available to implement the Ticket Act provisions; administrative and system inadequacies requiring substantial effort to address; and the considerable effort involved in EN recruitment.

·      Beneficiary participation in the program has been limited but continues to grow. As of December 2004, 1.4 percent of all TTW-eligible beneficiaries in Phase 1 states had assigned a Ticket to a provider.

·      Beneficiary survey data suggest some reasons for the limited beneficiary participation in TTW: lack of awareness of the program and lack of interest in work among beneficiaries. Only about one-third of all beneficiaries had heard of the program at the time they were interviewed in 2004. Among all beneficiaries, only 30 percent indicate having goals that include work or career advancement, and only 26 percent see themselves working within the next five years.

·      Although participation in TTW has been low, the survey data suggest that there is potential demand for the program. About 13 percent of all beneficiaries worked at some time during the previous year, and a substantial share of all beneficiaries have goals that include work or career advancement and see themselves working within the next five years. However, only 15 percent of all beneficiaries and 52 percent of TTW participants see themselves working and earning enough to leave the disability rolls within the next five years. Thus, while potential demand for the program might be substantial, the ability to succeed in the program might be limited, given the program’s implicit definition of successwork at levels sufficient to reduce disability benefits to zero.

·      Most Tickets (over 90 percent) have been assigned to state VR agencies and the vast majority of these have been assigned under the traditional reimbursement method over one of the two new TTW reimbursement schemes. This suggests that TTW has had little impact on the delivery of services to beneficiaries, as state VR agencies have historically been the principal source of VR and employment-related services for beneficiaries.

·      While a large number of providers have enrolled in the program as ENs (approximately 1,300), few are accepting any or many Tickets. Only about 40 percent of all ENs have taken any Ticket assignments, and only about 20 percent have accepted more than five Tickets. Reasons for the low levels of participation among ENs include lack of up-front funding within the EN payment systems; the high cost of screening beneficiaries; administrative burden and complexity of the program; discomfort with the low level of risk-sharing within the EN payment systems; a belief that the value of EN payments is low relative to payments from other sources; concerns about the ability to mingle Ticket payments with other sources of funding; a belief that few potential clients will work at levels sufficient to generate payments; and lack of capacity to serve significant numbers of Ticket holders.

·      TTW appears to have had an impact on the number of beneficiaries enrolled for employment services but has had no measurable impact on earnings or benefit receipt. The lack of measurable impacts on earnings and benefits might reflect the fact that the quasi-experimental evaluation design, which relied on comparisons of outcome changes for early and late implementation states, cannot detect effects that, although important, are small relative to historical state-level variation in earnings and benefits. Even if the TTW was successfully leading to increases in earnings and exits from DI, effects would likely be small in the first two years, as participants find work, complete their trial work periods, and eventually experience earnings reductions. Unfortunately, the quasi-experimental method cannot be extended beyond the first two years, because TTW was in place in all states after two years.

In July 2008, SSA implemented very substantial revisions to TTW regulations. The changes are expected to increase the incentives that providers have to serve beneficiaries and might increase beneficiary interest in earning their way off the rolls. SSA expects to continue the TTW evaluation. Although the ability to conduct an impact evaluation is limited by the lack of a suitable comparison group, it will not be difficult to detect a marked increase in the number of beneficiaries who exit the rolls because of work, as historically so few have done so. While such an increase would not necessarily be entirely attributed to TTW, it might be an indicator of impact.

H.   Substance Abuse and Mental Health Services Administration

1.    Employment Intervention Demonstration Program

a.    Intervention and Target Population

The Employment Intervention Demonstration Program (EIDP) was a multisite randomized controlled trial of the effectiveness of supported employment for people with psychiatric disabilities in eight locations across the U.S. SE programs use a rapid job search approach to help clients obtain jobs directly (rather than providing lengthy assessment, training, and counseling) and provides them with ongoing support to maintain and improve their earnings after they start work. The study was funded by the Center for Mental Health Services of the Substance Abuse and Mental Health Services Administration. The purpose of the study was to generate knowledge about effective approaches for enhancing employment among adults with severe mental illnesses (Cook et al. 2002). Half of all participants had a schizophrenia spectrum diagnosis and another 40 percent were diagnosed with major depression or bipolar disorder. Over half had a secondary diagnosis of substance abuse.

 

The experimental study group received services under different supported employment service models designed specifically for people with psychiatric disabilities. All of the experimental conditions had five characteristics: (1) integrated services delivered by a multidisciplinary team that met three or more times per week to plan and coordinate employment interventions with case management and psychiatric treatment; (2) placement into competitive employment, defined as jobs paying at least minimum wage, in regular, socially integrated community settings; (3) development of jobs tailored to personal career preferences; (4) use of a job-search process beginning immediately after program entry and moving as quickly as the individual desired; and (5) provision of ongoing vocational supports freely available throughout the entire study period.

 

The control groups received services as usual (that is, whatever was typically available in the participants’ local communities), unenhanced versions of the experimental models, or Clubhouse services. Generally, individuals in the control group received lower amounts of vocational services although they received equivalent amounts of psychiatric services in comparison to experimental group participants.


b.    Evaluation Activities

The study randomly assigned over 1,600 participants to experimental and control groups at the eight EIDP study sites, and the researchers followed them for two years. The study documented vocational outcomes, including competitive employment, earnings, employment status, benefit receipt, and number of hours worked.

c.     Findings

The EIDP evaluation findings have been reported in a number of published papers (Burke-Miller et al. 2006; Cook et al. 2005a, 2005b, 2006; Leff et al. 2005; and Razzano et al. 2005). Some of the key general findings of the study include the following (Cook 2007):

·      Experimental group participants were more likely than the control group participants to achieve competitive employment, work more than 40 hours in a given month, and earn more money. People with severe mental illness who received well-integrated and coordinated vocational and clinical services had significantly better employment outcomes than those who received nonintegrated services.

·      Integrated employment services resulted in positive employment outcomes regardless of consumers’ personal characteristics, diagnoses, work histories, receipt of SSA disability income, and functioning levels.

·      The more vocational services participants received, the better the employment outcomes.

·      Personal characteristics such as type and intensity of mental disorders and psychiatric symptoms influenced employment outcomes within vocational programs.

·      Employment outcomes for all study subjects were negatively related to the level of unemployment in the study area. Impacts were largest in the study areas with the lowest unemployment, but were substantial even in areas with high unemployment.

The study clearly demonstrated positive impacts on employment of supported employment approaches that integrate health and vocational supports for persons with severe mental illness. In addition, the advantage of supported employment over other programs increased over the 24-month study period, making it apparent that programs offering ongoing support and services that build on career achievements had greater success. The findings support the importance of providing ongoing supported employment services as a best practice in vocational rehabilitation for people with psychiatric disabilities.

References

Acemoglu, D., and J. Angrist. “Consequences of Employment Protection? The Case of the Americans with Disabilities Act.” Journal of Political Economy, vol. 109, no. 5, 2001, pp. 915-957.

Agodini, R., C. Thornton, N. Khan, and D. Peikes. “Design for Estimating the Net Outcomes of the State Partnership Initiative: Final Report.” Princeton, NJ: Mathematica Policy Research, Inc., 2002

Almandsmith, S., K. Magill, L. Barker, and M. Vencill. “Evaluation of the DOL Disability Employment Grants, Revised Final Report.” Submitted to U.S. Department of Labor, Employment and Training Administration, Office of National Programs. Oakland, CA: Berkeley Policy Associates, 2001.

APS Healthcare, Inc. “Medicaid Purchase Plan Evaluation Annual Report.” Madison, WI: Center for Delivery System Development and the Division of Health Care Financing, Wisconsin Department of Health and Family Services, 2003.

Black, W., and H. Ireys. “Understanding Enrollment Trends and Participant Characteristics of the Medicaid Buy-In Program, 2003-2004.” Washington, DC: Mathematica Policy Research, Inc., 2006.

Burke-Miller, J., J. Cook, D. Grey, L. Razzano, C. Blyler, H. Leff, et al. “Demographic Characteristics and Employment Among People with Severe Mental Illness in a Multisite Study.” Community Mental Health Journal, vol. 42, no. 2, 2006, pp. 143-159.

Buschman, R., K. Harrison, C. DeSimone, and M. Au. “Evaluation of the Ticket to Work Program: Work Incentives Planning and Assistance (WIPA) Program Implementation Study Design Report.” Draft Report. Washington, DC: Mathematica Policy Research, Inc., 2007.

Butler, D. “Instilling a Culture of Work: Applying Lessons from Welfare Policy to Disability Policy.” Presentation at the annual meeting of the Association for Public Policy Analysis and Management, New York, NY, 2007.

Capella-McDonnall. M. “Predictors of Competitive Employment for Blind and Visually Impaired Consumers.” Journal of Visual Impairment & Blindness, vol. 99, no. 5, 2005, p. 303.

Center for Workers with Disabilities. “Florida Freedom Initiative.” Washington, DC: Center for Workers with Disabilities, February 2007. Available at [http://cwd.aphsa.org/Home/docs/CWD_Florida_Freedom_Initiative.pdf ].

Centers for Medicare and Medicaid Services. “2008 Edition Announcement, Medicaid Infrastructure Grant to Support the Competitive Employment of People with Disabilities.” Baltimore, MD: Centers for Medicare and Medicaid Services, Center for Medicaid and State Operations, 2007.

Clark, R., K. Swain, and W. Peacock. “Economic Evaluation of the Medicaid for Employed Adults with Disabilities (MEAD) Program: February 1, 2002 through January 31, 2003.” Concord, NH: Office of Health Planning and Medicaid: New Hampshire Department of Health and Human Services, 2003.

Cook, J. “Executive Summary of Findings from the Employment Intervention Demonstration Program.” Chicago, IL: University of Illinois, Center on Mental Health Services Research & Policy, 2007.

Cook, J., M. Carey, L. Razzano, J. Burke, and C. Blyler. “The Pioneer: The Employment Intervention Demonstration Program.” New Directions for Evaluation, no. 94, 2002, pp. 31-44.

Cook, J., H. Leff, C. Blyler, P. Gold, R. Goldberg, K. Mueser, et al. “Results of a Multisite Randomized Trial of Supported Employment Interventions for Individuals with Severe Mental Illness.” Archives of General Psychiatry, no. 62, 2005a, pp. 505-512.

Cook, J., A. Lehman, R. Drake, W. McFarlane, P. Gold, H. Leff, D. Blyler, M. Toprac, L. Razzano, J. Burke-Miller, L. Blankertz, M. Shafer, S. Pickett-Schenk, and D. Grey.  “Integration of Psychiatric and Vocational Services: A Multi-Site Randomized, Controlled Trial of Supported Employment.” American Journal of Psychiatry, vol. 162, no. 10, 2005b, pp. 1948-1956.

Cook, J., G. Mulkern, D. Grey, J. Burke-Miller, C. Blyler, L. Razzano, R. Balser, P. Gold, M. Shafer, K. Dudek, C. Chow, and P. Steigman. “Effects of Unemployment Rate on Vocational Outcomes in a Randomized Trial of Supported Employment for Individuals with Severe Mental Illness.” Journal of Vocational Rehabilitation, vol. 22, 2006, pp. 1-13.

Crane, B., C. Scott, and C. Davis. “Veterans’ Benefits: The Vocational Rehabilitation and Employment Program.” Washington, DC: Congressional Research Service, 2008.

Deke, J., and D. Peikes. “State Partnership Initiative: Services Delivered Through March 2002.” Princeton, NJ: Mathematica Policy Research, Inc., 2003.

Deleire, T. “The Unintended Consequences of the Americans with Disabilities Act.” Regulation, vol. 23, no. 1, 2000a, pp. 21-24.

DeLeire, T. “The Wage and Employment Effects of the Americans with Disabilities Act.” Journal of Human Resources, vol. 35, no. 4, 2000b, pp. 693-715.

Drake, R. “A Brief History of the Individual Placement and Support Model.” Psychiatric Rehabilitation Journal, vol. 22, no. 1, 1998, pp. 3-7.

Drake, R., D. Becker, R. Clark, and K. Mueser. “Research on the Individual Placement and Support Model of Supported Employment.” Psychiatric Quarterly, no. 70, 1999, pp. 289-301.

Elinson, L., and W. Frey. “Evaluation of the Disability Employment Policy Demonstration Programs: Task 10: Interim Report on ODEP Demonstration Programs: Accomplishments and Issues Identified by the Independent Evaluation.” Report prepared for the Office of Disability Employment Policy, U.S. Department of Labor. Rockville, MD: Westat, 2005.

Emery, J., and M. Bryan. “Disability Program Navigator Project: Four-State Study: National Overview Report.” Iowa City, IA: University of Iowa Law, Health Policy & Disability Center, 2006.

Fabian, E. “Urban Youth With Disabilities: Factors Affecting Transition Employment.” Rehabilitation Counseling Bulletin, vol. 50, no. 3, 2007, pp. 130-138.

Folkemer, D., A. Jensen, R. Silverstein, and T. Straw. “Medicaid Buy-In Programs: Case Studies of Early Implementer States.” Washington, DC: U.S. Department of Health and Human Services Assistant Secretary for Planning and Evaluation Office of Disability, Aging and Long-Term Care Policy, 2002a.

Folkemer, D., A. Jensen, R. Silverstein, and T. Straw. “The Medicaid Buy-In Program: Lessons Learned From Nine “Early Implementer” States. Washington, DC: U.S. Department of Health and Human Services Assistant Secretary for Planning and Evaluation Office of Disability, Aging and Long-Term Care Policy, 2002b.

Frey, W. “The Mental Health Treatment Study.” Presentation and materials distributed at the fall meetings of the Association for Public Policy, Analysis and Management, Madison, WI, November 2006.

Frey, W., H. Goldman, S. Kalasunas, D. Salkever, A. Miller, G. Bond, and R. Drake. “The Mental Health Treatment Study.” Psychiatric Rehabilitation Journal, vol. 31, no. 4, 2008, pp. 306-312.

Gilman, B. T. Anderson, L. Trebino, G. Gimm, and H. Ireys. “Status Report on the National Evaluation of the Demonstration to Maintain Independence and Employment.” Washington, DC: Mathematica Policy Research, Inc., 2008.

Gimm, G., H. Ireys, and C. Johnson. “Who Are the Top Earners in the Medicaid Buy-In Program?” Working with Disability (no. 3). Washington, DC: Mathematica Policy Research, Inc., 2007.

Gimm, G., S. Davis, K. Andrews, H. Ireys, S. Liu. “The Three E’s: Enrollment, Employment, and Earnings in the Medicaid Buy-in Program, 2006.” Washington, DC: Mathematica Policy Research, Inc., 2008.

Gimm, G. and R. Weathers. “What Is the Demonstration to Maintain Independence and Employment (DMIE) and Who Is Participating?” Working with Disability (no. 6). Washington DC: Mathematica Policy Research Inc., 2007

Goodman, N. and G. Livermore. “The Effectiveness of Medicaid Buy-in Programs in Promoting the Employment of People with Disabilities.” Washington, DC: Ticket to Work and Work Incentives Advisory Panel, 2004.

Goodman, N., and Waidmann, T. “Social Security Disability Insurance and the Recent Decline in the Employment Rate of People with Disabilities.” In The decline in the employment of people with disabilities: A policy puzzle, edited by D. Stapleton and R. Burkhauser, Kalamazoo, MI: W.E. Upjohn Institute, 2003.

Haber, S., and D. Osber. “Evaluation of the DC DMIE and 1115 Waiver Demonstration First Annual Report.” Prepared for the Centers for Medicare and Medicaid Services. Waltham, MA: RTI International, 2005.

Hayward, B., and H. Schmidt-Davis. “Longitudinal Study of the Vocational Rehabilitation Services Program Draft Report 4: Results of the VR Program.” Research Triangle Park, NC: Research Triangle Institute, 2002.

Hayward, B., and H. Schmidt-Davis. “Longitudinal Study of the Vocational Rehabilitation Services Program Final Report 1: How Consumer Characteristics Affect Access to, Receipt of, and Outcomes of VR Services.” Research Triangle Park, NC: Research Triangle Institute, 2003a.

Hayward, B., and H. Schmidt-Davis. “Longitudinal Study of the Vocational Rehabilitation Services Program Final Report 2: VR Services and Outcomes.” Research Triangle Park, NC: Research Triangle Institute, 2003b.

Hayward, B., and H. Schmidt-Davis. “Longitudinal Study of the Vocational Rehabilitation Services Program Final Report 3: The Context of VR Services.” Research Triangle Park, NC: Research Triangle Institute, 2005.

Hoff, D. “One-Stop Career Centers: Serving People with Disabilities.” OneStops.info (Issue no.1). Boston, MA: Institute for Community Inclusion, 2002. Available at [http://www.communityinclusion.org/pdf/os1.pdf].

Holcomb, P., and B. Barnow. “Serving People with Disabilities through the Workforce Investment Act’s One-Stop Career Centers.” Washington, DC: Ticket to Work and Work Incentives Advisory Panel, 2004.

Homa, D. “The Impact of Vocational Evaluation on Outcomes in the Vocational Rehabilitation Services Program.” Dissertation Abstracts International, vol. 65, no. 9-B, 2005, p. 4833.

Hotchkiss, J. “A Closer Look at the Employment Impact of the Americans with Disabilities Act.” Journal of Human Resources, vol. 39, no. 4, 2004, pp. 887-911.

Houtenville, A., and R. Burkhauser. “Did the Employment of People with Disabilities Decline in the 1990s, and Was the ADA Responsible? A Replication and Robustness Check of Acemoglu and Angrist.” Ithaca, NY: Rehabilitation Research and Training Center for Economic Research on Employment Policy for Persons with Disabilities, 2004.

Ireys, H., S. Davis, and K. Andrews. “The Interaction of Policy and Enrollment in the Medicaid Buy-In Program, 2005.” Washington, DC: Mathematica Policy Research, Inc., 2007.

Ireys, H., J. White, and C/ Thornton, “The Medicaid Buy-In Program: Quantitative Measures of Enrollment Trends and Participant Characteristics in 2002.” Report prepared for the Centers for Medicare and Medicaid Services. Washington, DC: Mathematica Policy Research, Inc., 2003.

Jee, J., and J. Menges. “The California Working Disabled Program: Lessons Learned Looking Ahead.” Oakland CA: Medi-Cal Policy Institute, 2003.

Jensen, A., R. Silverstein, D. Folkemer, and T. Straw. “Policy Frameworks for Designing Medicaid Buy-In Programs and Related State Work Incentive Initiatives.” Washington, DC: U.S. Department of Health and Human Services Assistant Secretary for Planning and Evaluation Office of Disability, Aging and Long-Term Care Policy, 2002.

Jolls, C., and J. Prescott. “Disaggregating Employment Protection: The Case of Disability Discrimination.” NBER Working Paper 10740. Cambridge, MA: National Bureau of Economic Research, 2004.

Hall, A., H. Boeltzig, J. Timmons, D. Hamner, and S. Fesko. “Analysis of Change: A Longitudinal Study of Six One-Stop Systems Serving Individuals with Disabilities.” Boston, MA: University of Massachusetts, Institute for Community, 2005.

Iowa Department of Human Services. “Medicaid for Employed People with Disabilities: A Client Profile and Program Evaluation.” Des Moines, IA: Iowa Department of Human Services, 2005.

Kregel, J. “Conclusions Drawn from the State Partnership Initiative.” Richmond, VA: State Partnership Initiative Evaluation and Information Office, Virginia Commonwealth University, 2006.

Kregel, J., L. Tucker, B. Bader, and M. West. “State Systems Change Project 1998-2003: Final Evaluation Report.” Richmond, VA: Rehabilitation Research and Training Center, 2005.

Kruse, D., and Schur, L. “Does the Definition Affect the Outcome?” In The decline in the employment of people with disabilities: A policy puzzle, edited by D. Stapleton and R. Burkhauser, Kalamazoo, MI: W.E. Upjohn Institute, 2003.

Leff, H., J. Cook, P. Gold, M. Toprac, C. Blyler, C., R. Goldberg, W. McFarlane, M. Shafer, I. Allen, T. Camacho-Gonsalves, and B. Raab. “Effects of Job Development and Job Support on Competitive Employment of Persons with Severe Mental Illness.” Psychiatric Services, vol. 56, no. 10, 2005, pp. 1237-1244.

Liu, S., and H. Ireys. “Participation in the Medicaid Buy-In Program: A Statistical Profile from Integrated Data.” Washington, DC: Mathematica Policy Research, Inc., 2006.

Liu, S., and R. Weathers. “Do Participants Increase Their Earnings After Enrolling in the Medicaid Buy-In Program?” Working with Disability (no. 4). Washington, DC: Mathematica Policy Research, Inc., 2007.

Livermore, G., M. Nowak, D. Stapleton et al. “Evaluation Design for the Ticket to Work Program: Preliminary Process Evaluation.” Falls Church, VA: The Lewin Group, March 2003.

Livermore, G., D. Stapleton, B. Kyler, J. Ladinsky, and A. Roche. “Study Design for the Individual Outcome Evaluation of the Disability Program Navigator Initiative: Draft Report.” Washington, DC: Mathematica Policy Research, Inc., 2008.

National Council on Disability. “The Social Security Administration’s Efforts to Promote Employment for People with Disabilities: New Solutions for Old Problems.” Washington, DC: National Council on Disability, 2005.

National Council on Disability. “The Impact of the Americans with Disabilities Act: Assessing the Progress Toward Achieving the Goals of the ADA.” Washington, DC: National Council on Disability, 2007a.

National Council on Disability. “The Vocational Rehabilitation Act: Transition Outcomes and Effects, Request for Proposals.” Washington, DC: National Council on Disability, 2007b.

MDRC. “Accelerated Benefits (AB) Demonstration Overview.” Handout distributed at the SSA-sponsored meeting on Building Data Resources to Support OPDR’s Research Efforts. Washington, DC, November 28, 2006.

Peikes, D., and S. Dale. “Design for Evaluating the Florida Freedom Initiative: Final Report.” Princeton, NJ: Mathematica Policy Research, Inc., 2005.

Peikes, D., and N. Paxton. “State Partnership Initiative: Characteristics of Participants Enrolled Through March 2003.” Princeton, NJ: Mathematica Policy Research, Inc., 2003.

Peikes, D., and A. Sarin. “State Partnership Initiative: Synthesis of Impact Estimates Generated by the State Projects’ Evaluations.” Princeton, NJ: Mathematica Policy Research, 2005.

Razzano, L., J. Cook, J. Burke-Miller, K. Mueser, S. Pickett-Schenk, D. Grey, et al. “Clinical Factors Associated with Employment Among People with Severe Mental Illness: Findings from the Employment Intervention Demonstration Program.” Journal of Nervous and Mental Disease, vol. 193, no. 1, 2005, pp. 705-713.

Rehabilitation Research and Training Center on Disability Demographics and Statistics. “2005 Disability Status Reports.” Ithaca, NY: Cornell University, 2005.

Schmeling, J., and M. Morris. “Disability Program Navigator Fourteen State Evaluation: Telephone Survey Key Findings.” Iowa City, IA: University of Iowa Law, Health Policy & Disability Center, 2005.

Silverstein, R. “A Description of the Workforce Investment Act from a Disability Policy Perspective.” Washington, DC: Center for the Study and Advancement of Disability Policy, 2000.

Silverstein, R., and A. Jensen. “Thematic Review of State Memos Describing Lessons Learned from Implementing Benefit Offset Pilot Demonstration Projects.” Discussion paper presented at the 4-State Pilot Offset Conference, Baltimore, MD, March 2007.

Social Security Administration. “Social Security Administration Quarterly Update on the Ticket to Work and Work Incentives Improvement Act Implementation. Third Fiscal Quarter of 2007.” Baltimore, MD: Office of Disability and Income Security Programs, July 2007.

Social Security Administration. “Benefits Planning Assistance and Outreach (BPAO) 2004 Customer Satisfaction Survey Results.” Baltimore, MD: Office of Employment Support Programs, 2004.

Social Security Administration. “Final Report for Employment Support Representative (ESR) Pilot.” Baltimore, MD: Evaluation Team, Work Incentives Service Delivery Workgroup, November 2001.

Social Security Advisory Board. “A Disability System for the 21st Century.” Washington, DC: Social Security Advisory Board, 2006.

Stapleton, D., R. Burkhauser, and A. Houtenville. “Has the Employment Rate of People with Disabilities Declined?” Policy Brief. Ithaca, NY: Cornell University Rehabilitation Research and Training Center for Economic Research on Employment Policy for Persons with Disabilities, 2004.

Stapleton, D., and W. Erickson. “Characteristics or Incentives: Why Do Employment Outcomes for the SSA Beneficiary Clients of VR Agencies Differ, on Average, from Those of Other Clients?” Ithaca, NY: Cornell University Rehabilitation Research and Training Center for Economic Research on Employment Policy for Persons with Disabilities, 2005.

Stapleton, D., and G. Livermore. “Evaluation Design for the Ticket to Work Program: Preliminary Evaluation Design.” Falls Church, VA: The Lewin Group, 2002.

Stapleton, D., D. Long, J. Feins, L. Orr, and M. Green. “The Design of the Social Security Administration’s Benefit Offset/Early Intervention Demonstration.” Presentation at the fall meetings of the Association for Public Policy, Analysis and Management, Madison, WI, November 2006.

Stapleton, D., B. O’Day, G. Livermore, and A. Imparato. “Dismantling the Poverty Trap: Disability Policy for the 21st Century.” Milbank Quarterly, vol. 84, no. 4, 2006, pp. 701-732.

Tashjian, M. “Evaluation of the Projects with Industry Program: Final Report Prepared for Rehabilitation Services Administration, U.S. Department of Education.” Research Triangle Park, NC: RTI International, 2003.

Thornton, C., T. Fraker, G. Livermore, D. Stapleton, B. O’Day, T. Silva, E. Sama Martin, J. Kregel, and D. Wright. “Evaluation of the Ticket to Work Program: Implementation Experience During the Second Two Years of Operation (2003-2004).” Washington, DC: Mathematica Policy Research, Inc., 2006.

Thornton, C., G. Livermore, T. Fraker, D. Stapleton, B. O’Day, D. Wittenburg, R. Weathers, N. Goodman, T. Silva, E. Martin, J. Gregory, D. Wright, and A. Mamun. “Evaluation of the Ticket to Work Program: Assessment of Implementation and Impact During 2002–2004.” Washington, DC: Mathematica Policy Research, Inc., 2007.

Thornton, C., G. Livermore, D. Stapleton, J. Kregel, T. Silva, B. O’Day, and T. Fraker. “Evaluation of the Ticket to Work Program: Initial Evaluation Report.” Washington, DC: Mathematica Policy Research, Inc., 2004.

Tremblay, T., J. Smith, H. Xie, and R. Drake. “The Impact of Specialized Benefits Counseling Services on Social Security Administration Disability Beneficiaries in Vermont.” Journal of Rehabilitation, vol. 70, no. 2, 2004, pp. 5-11.

U.S. Department of Education. Projects with Industry: 2006 Program Performance Data. Available at [ http://www.ed.gov/programs/rsapwi/performance.html]. Undated

U.S. Department of Labor. “Workforce Investment Act Amendments, Proposed Rules.”  Available at [http://www.dol.gov/eta/regs/fedreg/proposed/2006021766.htm]. 2006.

U.S. Department of Labor, Employment and Training Administration. “Workforce Investment Act Reform.” Available at [http://www.doleta.gov/reports/pdf/WIA%20Reform%202-page%20Fact%20Sheet%20Final.pdf]. 2007.

U.S. Department of Veterans Affairs, Office of the Inspector General. “Audit of Vocational Rehabilitation and Employment Program Operations.” Report No. 06-00493-42. Washington, DC: VA Office of the Inspector General, 2007.

U.S. Department of Veterans Affairs Vocational Rehabilitation and Employment Task Force. “The Vocational Rehabilitation and Employment Program for the 21st Century Veteran.” Washington, DC: Department of Veterans Affairs, 2004.

U.S. Equal Employment Opportunity Commission. “The ADA: Your Employment Rights as an Individual with a Disability.” Available at [http://www.eeoc.gov/facts/ada18.html]. 2005.

U.S. General Accounting Office. “SSA Disability: SGA Levels Appear to Affect the Work Behavior of Relatively Few Beneficiaries, but More Data Needed.” GAO-02-224. Washington, DC: GAO, 2002a.

U.S. General Accounting Office. “SSA Disability: Enhanced Procedures and Guidance Could Improve Service and Reduce Overpayments to Concurrent Beneficiaries.” GAO-02-802. Washington, DC: GAO, 2002b.

U.S. General Accounting Office. “Business Tax Incentives: Incentives to Employ Workers with Disabilities Receive Limited Use and Have an Uncertain Impact.” GAO-03-39. Washington, DC: GAO, 2002c.

U.S. General Accounting Office. “High-Risk Series: An Update.” GAO-03-119. Washington, DC: GAO, 2003.

U.S. Government Accountability Office. “Social Security Disability: Improved Processes for Planning and Conducting Demonstrations May Help SSA More Effectively Use Its Demonstration Authority.” GAO-05-19. Washington, DC: GAO, 2004a.

U.S. Government Accountability Office. “Workforce Investment Act: Labor Has Taken Several Actions to Facilitate Access to One-Stops for Persons with Disabilities, but These Efforts May Not Be Sufficient.” GAO-05-54. Washington, DC: GAO, 2004b.

U.S. Government Accountability Office. Vocational Rehabilitation: Better Measures and Monitoring Could Improve the Performance of the VR Program.” GAO-05-865. Washington, DC: GAO, 2005.

U.S. Government Accountability Office. Vocational Rehabilitation: Improved Information and Practices May Enhance State Agency Earnings Outcomes for SSA Beneficiaries.” GAO-07-521. Washington, DC: GAO, 2007.

U.S. Government Accountability Office. Federal Disability Programs: More Strategic Coordination Could Help Overcome Challenges to Needed Transformation.” GAO-08-635. Washington, DC: GAO, 2008a.

U.S. Government Accountability Office.Social Security Disability: Management Controls Needed to Strengthen Demonstration Projects.” GAO-08-1053. Washington, DC: GAO, 2008b.

White, J., W. Black and H. Ireys. “Explaining Enrollment Trends and Participant Characteristics of the Medicaid Buy-In Program, 2002-2003.” Washington, DC: Mathematica Policy Research, Inc., 2005.

Wittenburg, D., P. Baird, L. Schwartz, and D. Butler. “Health Benefits for the Uninsured: Design and Early Implementation of the Accelerated Benefits Demonstration.” New York, NY: MDRC, 2008.

 

This research was conducted under the Rehabilitation Research and Training Center on Employment Policy for Persons with Disabilities at Cornell University, funded by the U.S. Department of Education, National Institute on Disability and Rehabilitation Research (Cooperative Agreement H244B040013).  The opinions, conclusions, and errors in this paper are the sole responsibility of the authors and do not represent the official views of the U.S. Department of Education, Mathematica Policy Research, Inc., or Cornell University and do not assume endorsement by the Federal Government (Edgar, 75.620(b)).

Contract No.:H133B040013

MPR Reference No.:6239-620

 

 

 

 

Submitted to:

Cornell University

The Employment & Disability Institute

School of Industrial and labor Relations

331 Ives Hall

Ithaca, NY  14853-3901

Project Officer: Susanne Bruyère

 

Submitted by:

Mathematica Policy Research, Inc.

P.O. Box 2393

Princeton, NJ  08543-2393

Telephone:       (609) 799-3535

Facsimile:        (609) 799-0005

Project Director: David Stapleton

 

Acknowledgements

We gratefully acknowledge Cindy Gruman and Allison Roche for their assistance in preparing this report. We also appreciate the helpful comments on an early draft of the report provided by David Stapleton.

 

This research was conducted under the Rehabilitation Research and Training Center on Employment Policy at Cornell University, funded by the U.S. Department of Education, National Institute on Disability and Rehabilitation Research. The opinions, conclusions, and errors in this paper are the sole responsibility of the authors, and do not represent the official views of the U.S. Department of Education, Cornell University, or Mathematica Policy Research.

 



[1] The ability of employers to do this for disability-related accommodations is limited by the Americans with Disabilities Act, which prohibits job-related discrimination against people with disabilities and requires that reasonable accommodation be provided for people with disabilities unless the difficulty or expense of accommodation would result in undue hardship for the firm.

[2] Proof of concept generally refers to a small, short, and/or incomplete test of a method or idea to demonstrate its feasibility and to provide some evidence that it might be capable of achieving the outcomes desired.

[3] Shortcomings of the current system and the need for transformation have been noted by GAO 2003, NCD 2005, Social Security Advisory Board 2006, Stapleton et al. 2006, and GAO 2008a.

[4] This section is based on information in U.S. Equal Employment Opportunity Commission (2005).

[5] The Extended Period of Eligibility (EPE) is a period of at least 36-months in which a DI beneficiary is eligible to receive DI cash benefits for any month in which countable earnings are below the SGA level.

[6] In 2004, the GAO’s legal name became the Government Accountability Office. Provisions that reference the GAO in the Ticket Act use the agency’s former name, the General Accounting Office.

[7] See GAO (2002c) for the completed study.

[8] See GAO (2002b) for the completed study.

[9] See GAO (2002a) for the completed study.

[10] See GAO (2005) for the completed study.

[11] The description of WIA is based on Silverstein (2000).

[12] In June 2008, the DOL awarded MPR a contract to conduct an evaluation of the impact of WIA. The study is currently in the early design stages.

[13] Only SSA staff are allowed access to the IRS data. Hence, analyses that required use of the IRS earnings data were completed by SSA staff.

[14] See http://www.Medicaidbuyin.org.

[15] http://www.ed.gov/about/overview/budget/statetables/index.html.

[16] The study implemented a multistage design that involved selection of a random sample (with probability proportional to size) of 40 local VR offices (located in 32 state VR agencies in a total of 30 states), and among those offices, a sample of 8,500 applicants and current and former consumers of VR services. The study implemented a cohort design that involved randomly selecting 25 percent of the sample from all persons at application to VR, 50 percent of the sample from all persons who were already accepted for and receiving services, and 25 percent of the sample from among all persons at or after they exited VR services.

[17]  http://www.ed.gov/rschstat/eval/rehab/standards.html?exp=1

[18] Additional interim and special topic reports were also developed for the study. See http://www.ed.gov/policy/speced/leg/rehab/eval-studies.html.

[19] Rehabilitation Act of 1973, as amended, Title VI, Part A, Section 611.

[20] Previous evaluations of the PWI program were conducted in 1985 and 1994.

[21] The SPI projects were implemented prior to implementation of the Medicare Part D drug benefit.

[22] Now called Work Incentives Planning and Assistance (WIPA) projects.

[23] This discussion is based on Peikes and Sarin (2005).

[24] This section is based on information from Almandsmith et al. (2001) except where otherwise noted.

[25] The grants ranged from ranged from $300,000 to $640,000 per program year. Grants were originally one year in duration, with the option of extending each over an additional two years. Although the Disability Employment Grant Initiative was planned as a Title IV program, in 1998 Title IV funds were supplemented with Title III funds to allow the demonstration to expand to include more projects. However, Title III funds were restricted to dislocated workers.

[26] Data for the comparison group were available from the 1998 Standardized Participant Information Report data which includes employment outcomes for the mainstream workforce development system, funded by the U.S. Department of Labor.

[27] BPA notes that this finding is consistent with grantee service designs that targeted these services to individuals with greater needs and barriers to employment, indicating that the analysis did not control adequately for variation in participant characteristics.

[28] Unless otherwise noted, information in this section is based on several WIG evaluation reports produced by the Rehabilitation Research and Training Center on Workforce Investment and Employment Policy for Persons with Disabilities, Law, Health Policy & Disability Center, University of Iowa College of Law. These reports are available at http://www.doleta.gov/disability/partevidence/part4.cfm.

[29] This section is based on information from http://www.vba.va.gov/bln/vre/vrs.htm .

[30] An employment handicap is defined as an impairment of a veteran’s ability to prepare for, obtain, or retain employment consistent with his or her abilities, aptitudes, and interests. The impairment must result in substantial part from a service-connected disability.

[31] See www.ExpectMore.gov .

[32] http://www.whitehouse.gov/omb/expectmore/summary/10003220.2006.html 

[33] See IRS Form 8850 and instructions for definitions for the groups covered by the WOTC.

[34] Different limits and time periods apply if the employee is a member of the “Long-term Family Assistance Recipient” or “Summer Youth” categories of the WOTC.

[35] See IRS Form 8826.

[36] See IRS Publication 535, Business Expenses.

[37] SSA-RFP-05-1045 issued July 19, 2005.

[38] MDRC (2006) and Wittenburg et al. (2008).

[39] http://www.socialsecurity.gov/disabilityresearch/offsetpilot2.htm

[40] Federal Register, Volume 73, Number 239, pp. 75492-75494, December 11, 2008.

[41] http://www.socialsecurity.gov/disabilityresearch/offsetnational2.htm

[42] In the initial competition for FY 2000, 43 BPAO projects were funded for the first five-year cycle ending on September 30, 2005. Fifty-one projects were funded in the second cycle, which began on January 1, 2001, and ended on December 31, 2005. The third cycle of 22 projects began on July 1, 2001, and ended June 30, 2006.

[43] http://www.vcu-barc.org/NatReport/raw060930.html

[44] The three organizations awarded contracts by SSA to conduct program training and provide technical assistance to BPAOs were Virginia Commonwealth University, Cornell University, and the University of Missouri.

[45] http://www.ssa.gov/work/ServiceProviders/wipafactsheet.html

[46] http://www.ssa.gov/work/WIPARFA_FAQ.html

[47] http://www.socialsecurity.gov/work/ServiceProviders/WIPADirectory.html

 

[48] Based on information from Thornton et al. (2004) and from http://www.socialsecurity.gov/ work/Beneficiaries/awic.html.

[49] Unless otherwise noted, the information in this section is based on Peikes and Dale (2005).

[50] Based on information from: SSA (2007); http://www.ssa.gov/disabilityresearch/mentalhealth2.htm; SSA-RFP-05-1044 issued July 8, 2005; Frey (2006); and Frey et al. (2008).

[51] SSI recipients lose benefits gradually as earnings rise, whereas DI beneficiaries lose all benefits after earnings have exceeded SGA for a specified period of time.

[52] The trial work period allows beneficiaries to earn any amount without losing benefits for a period of nine months within a 60-month period. The Extended Period of Eligibility starts in the first month after the trial work period ends.

[53] Assuming an average annual rate of wage growth of five percent from 1985 to 1997, $500 in 1997 would be equivalent to about $280 in 1985 constant dollars.

[54] See Stapleton and Livermore (2002).