Cornell University
ILR School
Employment and Disability Institute
Rehabilitation Research and Training Center
on Disability Demographics and Statistics
Disability Statistics User Guide Series
A Guide to Disability Statistics
from the National Assessment of Educational Programs (NAEP)
S. Antonio Ruiz-Quintanilla
Larry W. Featherston
Cornell University.
Andrew J. Houtenville
New Editions Consulting, Inc.
For additional information about this paper contact:
Larry Featherston
Employment and Disability Institute
201 Dolgen Hall
Cornell University
Ithaca, NY 14853
Ph: (607) 255-8348
Fax: (607) 255-2763
Lwf27@cornell.edu
This paper is being distributed by the Rehabilitation Research and Training Center on
Disability Demographics and Statistics at Cornell University.
This center is funded to Cornell University by the U.S. Department of Education, National Institute on Disability and Rehabilitation Research (No. H133B031111). The contents of this paper do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the Federal Government (Edgar, 75.620 (b)).
The Project Director is:
Harry C. Katz – The Kenneth F. Kahn Dean and Jack Sheinkman Professor, ILR School, Extension Division, Cornell University
The Co-Principal Investigators are:
Susanne M. Bruyere -- Director, Employment and Disability Institute, ILR School, Extension Division, Cornell University
Richard V. Burkhauser -- Sarah Gibson Blanding Professor, Department of Policy Analysis and Management, College of Human Ecology, Cornell University
David C. Stapleton – Director, Center for Studying Disability Policy, Mathematica Policy Research, Inc., Washington, DC.
TABLE OF CONTENTS
Conceptual Model of Disability
NAEP Background, Methods and Definitions
Coverage: Universe and Sample Design
Data Collection and Methodology
Accessing of Data and Statistics
Description of the NAEP Survey Instrument and Topics
Definitions of Disability and Other Variables
Suggested NAEP Participation (With or Without Accommodations)
Eligibility for Free and Reduced-Price School Lunch
Population and Prevalence Estimates
NAEP Description of Disability Population
Comparisons to Other Data Sources
|
Accommodation |
Table 1. Frequently-provided accommodations for students with disabilities |
15 |
|
Accommodation |
Table 2. Frequently-provided accommodations for English Language Learners |
16 |
|
Demographics |
Table 3. Number and prevalence of students with disabilities in the fourth and eighth grade, by grade and demographic characteristics |
27 |
|
Demographics |
Tables 4a and 4b. Prevalence of students with disabilities in the fourth and eighth grades, by grade and location |
30 & 53 |
|
Disability |
Table 5. Percentage distribution of students with disabilities in the fourth and eighth grades across various disability-related characteristics |
32 |
|
Economic |
Table 6. Percentage of fourth- and eighth-grade students eligible for free and reduced-price lunch programs and percentage of students in public, private, and Catholic schools, by grade and disability status
|
34 |
|
Participation |
Table 7. Number of fourth- and eighth-grade students with disabilities and the percentage of students included in the NAEP assessment, by grade and various disability-related characteristics |
36 |
|
Participation |
Tables 8a and 8b. Number of fourth- and eighth-grade students with disabilities and the percentage of students included in the NAEP assessment, by grade and location |
37 & 56 |
|
Comparisons OSEP |
Table 9. Comparison of the NAEP and OSEP percentage distribution estimates over type of disability |
40 |
|
Comparisons OSEP |
Table 10. Comparison of the NAEP and OSEP disability prevalence estimates |
41 |
|
Comparisons ACS |
Table 11a. Comparison to the NAEP fourth-grade prevalence rates and prevalence rates obtained from the 2003 American Community Survey (ACS) for children ages 9-10. |
44 |
|
Comparisons ACS |
Table 11b. Comparison to the NAEP eighth-grade prevalence rates and prevalence rates obtained from the 2003 American Community Survey (ACS) for children ages 13-14. |
46 |
|
Comparisons ACS |
Table 12a. Correlation coefficients between fourth-grade state prevalence rates from the NAEP and ACS disability categories |
47 |
|
Comparisons ACS |
Table 12b. Correlation coefficients between eighth-grade state prevalence rates from the NAEP and ACS disability categories |
48 |
Policymakers, service providers, disability advocacy groups and researchers use disability statistics for a wide variety of purposes. A common problem that these groups encounter is finding a data source, a disability definition and/or a statistical method that provides them with a disability statistic that is both relevant to their needs and useful.
The mission of the Cornell StatsRRTC is to bridge the divide between the sources of disability data and the users of disability statistics. One product of this effort is a series of User Guides to national survey data sets that collect information on the disability population. The purpose of each of the User Guides is to provide disability data users with:
1. An easily accessible guide to the disability information available in the nationally representative survey;
2. Estimates of the population with a disability, the disability prevalence rate, and estimates of participation-related statistics;
3. A description of the unique features of the dataset that will help disability statistics users determine whether the dataset can provide them with the statistic that they need; and
4. A description of how the dataset compares to other national data that are used to describe the population with disabilities.
This User Guide contains information on the National Assessment of Educational Programs (NAEP), also known as “the Nation's Report Card.” Unlike the other data sources addressed by the User Guide Series, the NAEP focuses solely on children. As a result, the focus of this Guide will shift to the inclusion and accommodation of educational activities of children with disabilities, as opposed to adult employment and economic well-being, which are the focus of many of the other User Guides
The NAEP is a nationally representative and continuing assessment of U.S. students in various subject areas (reading, mathematics, science, writing, U.S. history, civics, geography, and the arts).[1] The two main goals of the NAEP are (a) to compare student achievement across states and other jurisdictions, and (b) to track changes over time in achievement of fourth-, eighth-, and twelfth-grade students.
The NAEP has been administered since 1969 by the Commissioner of Education Statistics, who heads the National Center for Education Statistics in the U.S. Department of Education. The National Research Council recently described NAEP data as "an unparalleled source of information about the academic proficiency of U.S. students, providing among the best available trend data on the academic achievement of elementary, middle, and secondary students in core subject areas” (http://nces.ed.gov/pubs97/web/97045.asp). However, as with other data sources, periodic changes in the way the NAEP data are collected limit comparisons of some information over longer time periods.
This Guide utilizes data from the 2003 NAEP Mathematics data. Disability-related information is derived from the 2003 Students with Disabilities/Limited English Proficiency (SD/LEP) Questionnaire (see Appendix B). The SD/LEP survey is completed by a school official who is knowledgeable about the child’s ability to perform on standardized examinations. Teacher information is collected using a supplemental questionnaire for only those students who are rated as being capable of being included in testing (with or without accommodations). Therefore, information from the student and teacher questionnaires is not available for students with disabilities who were excluded from the assessment, and we do not draw from those sources.
The following section describes the conceptual model of disability used in the User Guide Series. Subsequently, we present a summary of the development and purpose of the NAEP, an outline of the data collection methodology, and a description of the various definitions used in this Guide.
One purpose of the User Guides is to describe the information on disability available in the various national data sources. An operational definition of disability is required to fulfill this purpose. Unlike age and gender, which are for the most part readily identifiable individual attributes, disability is usually defined as a complex interaction between a person’s health condition and the social and physical environment. An environment that provides accommodation may allow a person with a health condition to function at the level of a person without a health condition. In this instance, the person may not consider her health condition a disability.
Two major conceptual models of disability are the World Health Organization’s (WHO, 2001) International Classification of Functioning, Disability and Health (ICF) and the disability model developed by Saad Nagi (1965, 1976). Both of these conceptual models recognize disability as a dynamic process that involves the interaction of a person’s health condition, personal characteristics, the physical environment and the social environment. Changes to any one of these factors over time can have an impact on a person’s ability to function and participate in activities. Detailed descriptions of these models, as well as a comparison, can be found in Jette and Badley (2000) and Altman (2001).
A version of the International Classification of Functioning and Health for Children and Youth (ICF-CY) has been developed. The ICF-CY maintains the structure of the ICF but modifies content to document functioning, disability and environmental factors that exist from infancy through adolescence (0-18 age group). Alterations to the original ICF included (a) modifying or expanding descriptions, (b) removing codes outside the range of childhood and adolescence, (c) assigning new content to unused codes, (d) modifying inclusion and exclusion criteria, and (e) expanding generic qualifiers to include developmental aspects. All ICF codes applicable for children were retained with needed entries added such as sensory exploration; pre-verbal communication; symbolic and social play; and home, school and community environments (Simeonsson & Leonardi, 2004).
In the following discussion, we adopt ICF concepts to create operational definitions of disability. The concepts used include impairment, activity limitation, participation restriction, and disability (see WHO, 2001). A prerequisite to each of these concepts is the presence of a health condition. Examples of health conditions are listed in the International Classification of Diseases, Tenth Edition (ICD-10, WHO, 1992, 1993) and they encompass diseases, injuries, health disorders, and other health-related conditions. An impairment is defined as a significant deviation or loss in body function or structure. For example, the loss of a limb or vision may be classified as impairments. In some surveys, impairments are defined as long-lasting health conditions that limit a person’s ability to see or hear, limit a person’s physical activity, or limit a person’s mental capabilities. An activity limitation is defined as a difficulty an individual may have in executing activities. For example, a person who experiences difficulty dressing, bathing or performing other activities of daily living due to a health condition may be classified as having an activity limitation. In some surveys, activity limitations are identified based upon a standard set of activities of daily living questions (ADLs) and/or instrumental activities of daily living questions (IADLs). A participation restriction is defined as a problem that an individual may encounter in involvement in life situations. For example, a student with a severe health condition may have difficulty participating in school activities as a result of the physical environment (e.g., lack of reasonable accommodations) and/or the social environment (e.g., discrimination). In some surveys for adult populations, participation restrictions are identified by questions that ask whether the person has a long-lasting health condition that limits his or her ability to work, or whether a health conditions affects his or her ability to go outside his or her home, to go shopping, to church or to the doctor’s office.
The final ICF concept that we will introduce is a disability. Unlike most modern medical texts, the World Health Organization (2002, 2007) defines a disability as a socially-contrived problem and not just a health problem. The World Health Organization stresses that society has created an environment for individuals with disabilities that is filled with both physical and attitudinal barriers. The negative effects of a disability are not just due to an individual’s condition but are also due to environmental factors such as the lack of social integration, social action, individual and collective responsibility, environmental manipulation, attitude, human rights, politics, and overall social change. Simply stated, the term disability is used to describe the presence of an impairment, an activity limitation and/or a participation restriction. This concept is similar to the definition used in the Americans with Disabilities Act of 1990 (ADA). The ADA defines a disability as “a physical or mental impairment that substantially limits one or more of the major life activities, a record of such an impairment, or being regarded as having such an impairment.”
While these concepts may seem to follow a progression—that is, an impairment leading to an activity limitation leading to a participation restriction—this is not necessarily the case. It is possible that a person may have a participation restriction without an activity limitation or impairment. For example, a person diagnosed as HIV positive may not have an evident impairment or activity limitation but may not be able to find employment due to discrimination resulting from his health condition. Similarly, a person with a history of mental illness, but who no longer has a loss in capacity or activity limitation, may also be unable to finding employment due to discrimination resulting from his past health condition.
As a final comment about the ICF, it is important to note the emerging shift in the overall conceptualization of disability. Rather than viewing children and adults with disabilities as being labeled as having a particular health condition or impairment, they are instead being viewed on a “continuum of functioning in three dimensions (the person, the activity, and the environment in which the activity takes place)” (Clauser and Bierman, 2003; p. 6). That is to say that functioning is not static and can change daily, dependent upon the confluence of physiological and environment variables and will most certainly change developmentally over the lifespan. Therefore, to properly employee the World Health Organization’s definition of disability and the concepts of the ICF specifically to children, the overall impact of the disability or impairment on the child’s level of functioning must continually be assessed.
Figure 1 provides a useful summary of the ICF concepts. It illustrates that while there is an overlap across these concepts, it is possible that one of them can occur without the others being present. The shaded area of Figure 1 illustrates the ICF concept of a disability.
Figure 1. Simplified Conceptual Model of Disability Using ICF Concepts: Figure one shows a large circle indicating the sphere of health conditions (e.g. diseases, disorders, injuries, and traumas. Within that circle is a diagram of three interlocked and overlapping circles: impairment, activity limitation and participation restriction.
Translating the ICF concepts into operational definitions for survey use or administrative data collection procedures is not a straightforward task. Decisions to classify the questions into one of the three specific ICF categories (impairment, activity limitation, participation restriction) were made based upon judgments and are not based upon well-defined rules from the ICF. In some cases, the classification is readily evident. In other cases, for example, the survey questions may be interpreted as both an activity limitation and participation restriction. Our approach in these cases was to make consistent judgments so that comparisons across the datasets would be possible. Using this approach would provide a framework for comparisons across surveys and for comparisons to ICF concepts.
When considering the U.S. Office of Special Education Programs’ (OSEP) definitions in the context of the ICF, it is clear that the school environment has been added in some domains (see the subsection entitled Definition of Disability).
The NAEP has two major goals: to compare student achievement in states and other jurisdictions and to track changes in the achievement of fourth-, eighth-, and twelfth-graders over time in mathematics, reading, writing, science, and other content domains. To meet these dual goals, the NAEP selects nationally-representative samples of students who participate in either the main NAEP assessments or the long-term trend NAEP assessments, where the latter is an examination that has retained the same set of questions over time. In addition, the NAEP conducts both national- and state-level assessments.
The survey methods can have an important impact on the information that a survey collects on the population with disabilities. Mathiowitz (2000) provides a good review of the general survey method issues as well as issues specific to the population with disabilities. The purpose of this section is to describe the development of the NAEP, the methods used by the NAEP to collect information on the population, and the definitions used to describe the population with disabilities.
NAEP samples are drawn from fourth-, eighth-, and twelfth-graders in public and private schools, as well as Domestic Dependent Elementary and Secondary Schools (DDESS) and Department of Defense Dependents Schools-Overseas Schools (DoDDS).
The random samples within each state are designed (with one exception mentioned below) to be proportionately representative of all the students in the state. No students are targeted for oversampling based on their student characteristics. However, in a state that contains one or more districts that are participating in the Trial Urban District Assessment (TUDA), students from the districts involved are sampled at a greater rate than those in the remainder of the state. The NAEP's sample weighting procedures ensure that the final results for the state contain the correct proportional contribution from such districts despite this difference in sampling rates within the state (http://nces.ed.gov/nationsreportcard/about/samplesfaq.asp).
The NAEP uses a complex stratified representative sample design. It collects data via a multi-stage, clustered sampling design involving unequal selection probabilities. First, schools are selected and then typically 30 students per subject per grade are selected randomly in each school. The numbers of schools and students vary from year to year depending on the number of subjects and items assessed. Schools are selected in a stratified manner.
For states that have agreed to participate in the NAEP state assessment, a sample of schools and students is selected to represent a participating state. In an average state, 2,500 students in approximately 100 public and non-public schools are selected per grade per subject assessed. Schools are selected randomly, within classes of schools with similar characteristics. But some schools or groups of schools (districts) may be selected if they are unique in the state. For instance, a particular district may be in the only major metropolitan area of a state or have the majority of a minority population in the state. If a state decides not to participate at the state level, schools in that state that are identified as part of the national sample will still be asked to participate.
For the national assessment, a national sample is drawn to yield a sufficient number of schools and students from public and non-public schools and each of the four NAEP regions of the country. Care is taken to ensure representation for characteristics such as sex, race, degree of urbanization of school location, parent education, and participation in the National School Lunch Program (NSLP). In years with both national and state assessments in the same subjects, the national sample is a subset of the combined sample of public and non-public school students assessed in each participating state, plus an additional sample from the states that did not participate, and a national private school sample. This national sample of public and non-public schools is selected for the fourth-, eighth- and twelfth-grades.[2] The sample is designed to produce national and regional estimates of student performance.
Beginning with the 2002 NAEP assessments, a combined sample of public schools was selected for both the state and national NAEP. This was done in response to the NAEP redesign of 1998. It was thought that drawing a subset of schools from all of the state samples to produce national estimates would reduce the burden by decreasing the total number of schools participating in the state and national NAEP. From this group of schools, representing the fifty states and the District of Columbia (DC), a sub-sample was identified as the national subset. (Please note that the long-term trend assessments, given every four years and most recently given during the 2003-2004 school year, use a nationally-representative sample and do not report results by state.)
The NAEP has always endeavored to assess all students selected as a part of its sampling process, including students with disabilities (SD) and Limited English Proficiency (LEP). (Beginning in 2005, English Language Learners, or ELL, replaced LEP.) The decision to exclude any of these students is made by school personnel. School personnel are encouraged to use inclusion criteria provided by the NAEP and may discuss their inclusion decisions with NAEP field staff. Some students may participate with testing accommodations.
According to the current criteria, a student with a disability is to be included in the NAEP assessment except in the following cases (see inclusion of special needs students: http://nces.ed.gov/nationsreportcard/about/inclusion.asp):
1. The student's IEP team determines that the student cannot participate;
2. The student's cognitive functioning is so severely impaired that she or he cannot participate; or
3. The student's IEP requires that the student has to be tested with an accommodation or adaptation that the NAEP does not allow (see the list of NAEP accommodations).
All LEP students who have received academic instruction in English for three years or more, including the current year, are included in assessments, if selected. Those LEP students who have received instruction in English for fewer than three years are included, if selected, unless school staff judges them to be incapable of participating in the assessment in English.
All special-needs students may use the same accommodations in the NAEP assessments that they use in their usual classroom testing unless the accommodation would make it impossible to measure the ability, skill, or proficiency being assessed, or the accommodation is not possible for the NAEP program to administer. For instance, in the reading assessment, reading the passage and questions aloud to a student is not permitted because the NAEP assessment is intended to measure the student's ability to decode the written word as well as understand the meaning of the passage. Also, extending testing over several days is not allowed for the NAEP because the NAEP administrators are in each school only one day.
Accommodations in the testing environment or administration procedures are provided for SD and LEP students (see Appendix; Questions 3, 8-12). Examples of accommodations permitted by the NAEP are extra time, testing in small-group or one-on-one sessions, reading aloud to a student, and scribing a student's responses. Table 1 lists the testing accommodations frequently offered to SD students and whether the NAEP provides such an accommodation. Table 2 does the same for LEP students.
Table 1. Frequently-Provided Accommodations
for Students with Disabilities
|
|
|
Description of Accommodation |
Permitted by NAEP? |
|
Presentation Format |
|
|
Read directions aloud/Repeat directionsa |
Yes |
|
Assist with interpretation of directions |
Yes |
|
Large-print edition of test |
Yes |
|
Use magnifying equipment |
Yes |
|
Person familiar to student administers test |
Yes |
|
Sign directions |
Yesb |
|
Read problems, passages, test questions, |
Yes, except for reading exams |
|
Use audiotaped version of the test |
No |
|
Braille edition of test |
Nob |
|
Response Format |
|
|
Respond in Braille |
Yes |
|
Point to answers |
Yes |
|
Oral responses |
Yes |
|
Use computer or typewriter to respond |
Yes |
|
Use template to respond |
Yes |
|
Use large marking pen or special writing tool |
Yes |
|
Write directly in test bookleta |
Yes |
|
Respond in sign language |
Nob |
|
Tape record answers |
No |
|
Use talking, Braille or other calculators |
No |
|
Setting Format |
|
|
Test in small group or individually |
Yes |
|
Administer test in separate room |
Yes |
|
Preferential seating, special lighting or furniture |
Yes |
|
Other Accommodations |
|
|
Extended time |
Yes |
|
Breaks during test |
Yes |
|
Test sessions over several days |
No |
|
Abacus, Arithmetic tables, Graph paper, Thesaurus |
No |
|
Spelling and grammar checking software and devices |
No |
|
Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2005 Reading, Mathematics, and Science Assessments a Standard NAEP practice. Not considered an accommodation. b Not provided by NAEP, but school, district, or state may provide after fulfilling NAEP security requirements. |
|
NAEP data are collected using a closely monitored and standardized process. The tight controls that guide the data collection process help ensure the comparability of the results generated for the national and the state assessments. All NAEP sessions use the same assessment booklets and identical administration procedures, and contractor staff members direct all sessions during a single calendar assessment period.
In addition to assessing subject area knowledge and abilities, the NAEP collects information from participating students, teachers, and principals about contextual or background variables that are related to student achievement. When developing the questionnaires used to gather this information, the NAEP ensures that the questions do not infringe on respondents' privacy, that they are grounded in current educational research, and that the answers can provide information relevant to the discussions about educational reform.
These questionnaires were developed using a framework and process similar to that used for cognitive questions. This process includes reviews by external advisory groups, pilot testing, and reviews by NCES, NAGB, and the Office of Management and Budget (OMB). For the main and state NAEP, the student questions appear in non-cognitive blocks. The background characteristic questions vary somewhat by grade level within a subject, and the subject area experience questions differ slightly by grade level within a subject. Unlike the cognitive blocks, these non-cognitive blocks do not differ among the assessment booklets for a given grade and subject. The teacher questionnaires vary based on subject area and may differ by grade level. The school questionnaires are completed by a school official for each grade of students participating in the assessment.
Some of the questionnaires, such as the student questionnaires, appear in separately printed and timed blocks of questions in the assessment booklets. Four general sources provide context information for the NAEP results as follows:
·
Student
questionnaires, which examine background characteristics and subject area
instructional experience;
·
Teacher
questionnaires, which gather data on teacher training and classroom
instruction;
·
School
questionnaires, which gather information about school characteristics and
policies; and
·
SD/LEP
(students with disabilities or limited English proficiency) questionnaires,
which provide information about students within the sample who have
disabilities or limited proficiency in English.
Summary data that is collected by the NAEP is publicly available from the NAEP website (http://nces.ed.gov/nationsreportcard/), and customized tables can be produced using the NAEP data tool. However, only limited disability data is available using the Web data tool. NAEP databases have been released since 1969, and are available on CD-ROM for secondary researchers. An assortment of other information, including sample questions for each subject area, survey and background questionnaires, sampling information and manuals, can be downloaded from the NAEP website.
The National Center for Educational Statistics also produces a number of reports, based on data collected by the NAEP, which are available publicly; NAEP statistics and reports are published on the NAEP website. Additionally, NAEP has developed a number of different publications and Web-based tools that provide direct access to assessment results at the state and national level. For every major assessment release, Web-specific content is developed that is suitable to the Web environment.
·
Results
pages, such as those developed for the U.S. history assessment, highlight
assessment results.
·
State
Profiles present state-level results and a history of state participation in
the NAEP.
·
The
NAEP Data Tool provides comprehensive information on student performance.
·
The
NAEP Questions Tool links users to questions, student responses, and scoring
guides that are released to the public.
Several types of printed reports published by the NAEP can be found under publications on the NAEP website (http://nces.ed.gov/pubsearch/getpubcats.asp?sid=031). These range from the NAEP Report Card, a comprehensive report that contains all the major results for each assessment, to technical reports that contain psychometric details of a national or state assessment.
The NAEP SD/LEP questionnaire for 2003 consists of 24 multiple choice items. The first fourteen items are to be completed if the student has a disability and the remaining ten items are to be completed if the student is LEP. The fourteen disability-related items cover the following topics: type of disability; severity of disability; IEP test participation; grade level of language arts and mathematics instruction; inclusion in language arts and mathematics curriculum content of non-disabled students; testing accommodations allowed; appropriateness of testing with or without accommodations; and type of presentation, response, setting or timing accommodation given. The ten LEP-related items cover the following topics: student’s native language, number of academic years receiving language arts and mathematics instruction in English, amount of instruction given in student’s native language, grade level of language arts and mathematics instruction, testing accommodations allowed, appropriateness of testing with or without accommodations, and type accommodation given.
Under the National Assessment of Educational Progress Authorization Act (Public Law 107-279 III, section 303), the Commissioner of the National Center for Education Statistics (NCES) is charged with ensuring that NAEP tests do not question test-takers about personal or family beliefs or make information about their personal identity publicly available. The NCES provides Micro-level NAEP data to researchers in raw format for the purpose of secondary analysis only after the researcher’s organization applies for and receives a restricted-use data license from the NCES. The license is intended to protect the confidentiality of individual students and schools.
A description of the survey questions and the methods used to produce data on disability, demographics, and economic background is shown in Tables 3-6.
Definition of Disability. All students with information on the type of disability question (Question 1 on the SD/LEP survey) were defined as our population with disabilities.[3] Therefore, students with disabilities are those students for which an SD/LEP survey was assigned and for whom the school official has categorized as a student with one of the following conditions: a specific learning disability, hearing impairment/deafness, visual impairment/blindness, speech or language impairment, mental retardation, emotional disturbance, orthopedic impairment, traumatic brain injury, autism, developmental delay (age 9 or younger), other health impairment, or other disability.
Disability terms used in the questionnaire are adopted from the OSEP and should be familiar to the administrators filling out the SD/LEP survey. The Individuals with Disabilities Education Act Amendments of 1997 (P.L. 105-17) defines them as follows:
Autism: A developmental disability significantly affecting verbal and nonverbal communication and social interaction, generally evident before age 3, that adversely affects a child's educational performance. Other characteristics often associated with autism are engagement in repetitive activities and stereotyped movements, resistance to environmental change or change in daily routines, and unusual responses to sensory experiences. The term does not apply if a child's educational performance is adversely affected primarily because the child has a serious emotional disturbance as defined below.
Deafness: A hearing impairment so severe that the child is impaired in processing linguistic information through hearing, with or without amplification that adversely affects a child’s educational performance.
Deaf-Blindness: A combination of hearing and visual impairments, the combination of which causes such severe communication and other developmental and educational needs that they cannot be accommodated in special education programs solely for children with deafness or children with blindness.
Hearing Impairment: An impairment in hearing, whether permanent or fluctuating, that adversely affects a child's educational performance but that is not included under the definition of deafness as listed above.
Mental Retardation: Significantly sub-average general intellectual functioning existing concurrently with deficits in adaptive behavior and manifested during the developmental period that adversely affects a child's educational performance.
Multiple Disabilities: A combination of impairments (such as mental retardation-blindness or mental retardation-orthopedic impairment), the combination of which causes such severe educational needs that they cannot be accommodated in special education programs solely for one of the impairments. Multiple disabilities does not include deaf-blindness.
Orthopedic Impairment: A severe orthopedic impairment that adversely affects a child’s educational performance. The term includes impairments caused by a congenital anomaly, impairments caused by disease (e.g., poliomyelitis, bone tuberculosis), and impairments from other causes (e.g., cerebral palsy, amputations, and fractures or burns that cause contractures).
Other Health Impairment: Having limited strength, vitality, or alertness, including a heightened alertness to environmental stimuli, that results in limited alertness with respect to the educational environment, which adversely affect educational performance.
Serious Emotional Disturbance: A condition exhibiting one or more of the following characteristics over a long period of time and to a marked degree that adversely affects a child’s educational performance: (A) An inability to learn that cannot be explained by intellectual, sensory, or health factors. (B) An inability to build or maintain satisfactory interpersonal relationships with peers and teachers. (C) Inappropriate types of behavior or feelings under normal circumstances. (D) A general pervasive mood of unhappiness or depression. (E) A tendency to develop physical symptoms or fears associated with personal or school problems.
Specific Learning Disability: A disorder in one or more of the basic psychological processes involved in understanding or in using language, spoken or written, that may manifest itself in the imperfect ability to listen, think, speak, read, write, spell, or to do mathematical calculations, including conditions such as perceptual disabilities, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia.
Speech or Language Impairment: A communication disorder such as stuttering, impaired articulation, a language impairment, or a voice impairment, that adversely affects a child's educational performance.
Traumatic Brain Injury: An acquired injury to the brain caused by an external physical force, resulting in total or partial functional disability or psychosocial impairment, or both, that adversely affects a child’s educational performance. Traumatic brain injury applies to open or closed head injuries resulting in impairments in one or more areas, such as cognition; language; memory; attention; reasoning; abstract thinking; judgment; problem-solving; sensory, perceptual, and motor abilities; psychosocial behavior; physical functions; information processing; and speech. Traumatic brain injury does not apply to brain injuries that are congenital or degenerative, or to brain injuries induced by birth trauma.
Visual Impairment, Including Blindness: An impairment in vision that, even with correction, adversely affects a child's educational performance. The term includes both partial sight and blindness.
We map the aforementioned responses to Question 1 into their corresponding ICF concepts of impairment to define disability types. While some of the disability questions in the ACS and NHIS are activity and/or participation based (ADLs and IADLs), the NAEP questions are rooted in body function or the impairment classification of disability. The only exception is the “specific learning disability” which corresponds to the ICF concept of activity restrictions.
Definition of Inclusion. We utilize whether a student is included in the NAEP mathematics assessment to define inclusion. As an alternative, we could have used the recommendation of the school official filling out the SD/LEP questionnaire, Question 14 (see Appendix A). As will be shown in our descriptive analysis, the NAEP largely follows the recommendations of the school official filling out the SD/LEP questionnaire with regard to inclusion.
Curriculum content in mathematics in the NAEP is based on Question 7 of the SD/LEP survey for students with disabilities. Students could be categorized as not receiving instruction in mathematics, receiving the same curriculum instruction in mathematics as non-disabled students, or receiving different curriculum instruction in mathematics than non-disabled students.
The appropriateness of taking the NAEP mathematics assessment was assessed with question 14 of the SD/LEP survey. Students with disabilities could be identified as being able to take the NAEP mathematics assessment without accommodations or adaptations, with accommodations or adaptations, or not able to take the mathematics assessment as recommended by the IEP team or an equivalent group.
Race/ethnicity data in the NAEP are obtained from school records. When the school data are missing, student-reported information is used. The racial/ethnic categories are the following mutually exclusive categories: White (non-Hispanic), black (non-Hispanic), Hispanic, Asian/Pacific Islander, American Indian/Alaska Native, and Unclassified. Hispanic students could be of any racial category. The “unclassified” category includes those who self-report more than one race category, or whose school reported the student’s race as “unavailable,” “other” or did not supply a response. Gender is self-reported by students.
A school’s “type of location” is determined by its physical location and the characteristics of the population of that location. There are three categories (1) “Central city” schools, which are located in a large city (a principle city with a population of at least 250,000) or mid–size city (a principle city of a metropolitan area with a population of less than 250,000); (2) urban fringe/large town schools, which are located in a metropolitan area of a large or mid-sized city, in a city or town classified by the Census Bureau as urban that does not qualify as a principle city, or in large towns that include schools located outside a metropolitan area in a town with at least 25,000 in total population; and (3) rural/small town schools, whose locations are classified as rural by the Census Bureau or are located outside a metropolitan area in small towns with total populations between 2,500 and 25,000 (http://nces.ed.gov/nationsreportcard/glossary.asp).
The Department of Agriculture’s national School Lunch Program provides subsidies and donated commodities to schools that offer free or reduced-price lunches to eligible children. Eligibility is based on school records and is determined by the family’s income as it relates to the federal poverty level. In order to qualify for free lunch the family must earn 130 percent of the poverty level or less. To qualify for reduced-price lunch the family income needs to be between 130-185 percent of the poverty level.
Sample Restrictions. In the tables below, we limit our analysis to students in the fifty states and District of Columbia (DC). (Note: DC is referred to as a state in the following discussion and in all tables.) Since there is variation in private and Catholic school attendance across states, and students with disabilities are less likely to attend private schools as revealed in the data, private and Catholic school students are included in our analytical sample. We excluded from our analytical sample students in the Department of Defense Domestic Dependent Elementary and Secondary Schools (DDESS) and Department of Defense Dependents Schools-Overseas (DoDDS). In our descriptive analysis we use all observations, and for each variable that allows missing values we provide estimates for those with missing values.
Demographics
In Tables 3-6 we provide estimates for the mathematics assessment portion of the NAEP for both fourth and eighth graders addressing the demographic characteristics of who is classified as having a disability, where the students are located, specific disability-related characteristics, and the limited economic indicators available (free or reduced-price lunch eligibility and school type).
Composition of the Population with Disabilities
Table 3 provides the number and prevalence rates of students in the fourth and eighth grades by demographic characteristics. Note that most of the characteristics listed in the tables are remarkably consistent between the fourth and eighth grades with the exception of race and ethnicity. Of the nearly 4 million students in both fourth and eighth grades, approximately 11 percent of each grade (437,000) is comprised of students with disabilities. In terms of gender, nearly twice as many males have a disability in both the fourth and eighth grades, resulting in males comprising approximately two-thirds of the total population with a disability in both grades. The NAEP reveals that 14.2 percent of fourth-grade males and 14.5 percent of eighth-grade males have a disability compared with only 7.5 percent of females in fourth and 7.6 percent of females in eighth grade.
Race and ethnicity varies more between the two grades, although the differences are not substantial and may be related to the degree of missing responses (7.6 percent of fourth and 9.6 percent of eighth grade students lack this information). For both fourth and eighth graders, Asian and Pacific Islanders have the lowest disability prevalence rate (6.7 percent of fourth graders, 5.6 percent of eighth graders) while American Indian/Alaska Natives have the highest prevalence rate (15.2 percent of fourth graders, 16.3 percent of eighth graders). The disability prevalence rates of the other groups (White, Black/African American and Hispanics) range from 10.2 percent to 13.2 percent.
There are minor differences by school location with Central city locations reporting lower prevalence rates (10.1 percent of fourth graders, 10.5 percent of eight graders) than those located in the “urban fringe” (10.9 percent of fourth graders, 10.6 percent of eighth graders). Children attending rural schools have the highest prevalence rates with 12.1 percent of fourth graders and 12.4 percent of eighth graders having a disability.
Prevalence of Disability Across States
The prevalence of disability varied greatly across states. As is shown in Table 4a below, and Table 4b in Appendix A, 11 percent of fourth-grade students in the United States in 2003, were students with disabilities. In other words, roughly 437,000 out of 4,000,000 fourth graders were students with disabilities. Among fourth-grade students, the prevalence of disability varied from 7.2 percent to 16.1 percent across states. The states with the five highest percentages of fourth-grade students with disabilities were Rhode Island (16.1 percent), New Hampshire (15.8 percent), Florida (15.4 percent), Alaska (14.7 percent), and Massachusetts (14.6 percent). The states with the five lowest percentages of fourth-grade students with disabilities were California (7.2 percent), Mississippi (8.1 percent), Hawaii (8.5 percent), Maryland, and Connecticut (both with 9.2 percent).
Among eighth-grade students in the United States in 2003, students with disabilities accounted for 11.1 percent. Among eighth-grade students, the prevalence of disability varied from 7.8 percent to 16.7 percent across states. The states with the five highest percentages of eighth-grade students with disabilities were New Hampshire (16.7 percent), New Mexico (16.5 percent), Vermont (15.0 percent), Rhode Island (14.7 percent), and Maine (14.5 percent); note that four of these five were New England states. The states with the five lowest percentages of eighth-grade students with disabilities were Mississippi (7.8 percent), California (8.3 percent), Idaho (8.9 percent), Georgia (9.0 percent), and Arizona (9.2 percent).
The prevalence of disability at the state level was positively correlated between fourth and eighth grades—a correlation coefficient of 0.60. The states with the greatest increase in percentage change in disability prevalence from fourth graders to eighth graders were Hawaii (43.5 percent), New Mexico (39.8 percent), Connecticut (35.9 percent), New York (19.1 percent), and Virginia (16.4 percent). The states with the greatest decrease in percentage change from fourth graders to eighth graders were South Dakota (-28.5 percent), the District of Columbia (-28.1 percent), Louisiana (-24.0 percent), Florida (-23.4 percent), and Nebraska (-20.6 percent). The states with the least percentage difference from fourth graders to eighth graders were Nevada (0.0 percent), Maine (0.7 percent), Oklahoma (-1.4 percent), Kansas (-1.7 percent), and South Carolina (-2.1 percent).
Distribution by Disability Across Disability-Related Characteristics
Table 5 shows the percentage distribution of students with disabilities in the fourth and eighth grades across various disability-related characteristics. With regards to the type of disability, children with specific learning disabilities account for more than half of all students with disabilities (they comprise 50.7 percent of fourth graders with disabilities and 64.4 percent of eighth graders with disabilities). The next most common specified disability types for fourth graders are speech or language impairment (16.2 percent) and mental retardation (6.6 percent). For eighth graders the most common types after LD are mental retardation (6.6 percent) and emotional disturbance (4.7 percent).
The majority of the students are defined as having mild or moderate disabilities. Forty percent of fourth graders have moderate disabilities while 47.1 percent have mild disabilities; 36.8 percent of eighth graders have moderate disabilities while 53.1 percent have mild disabilities. Nearly one in ten of the fourth graders with disabilities are defined as having severe disabilities (9.6 percent) as compared with 6.8 percent of eighth graders with disabilities. Most students with disabilities receive the same math curriculum as those without disabilities (75.6 percent of fourth graders, 66.3 percent of eighth graders). With regards to the suggested math NAEP participation, approximately one in ten are deemed unable to participate (11.3 percent of fourth graders, 9.6 percent of eight graders). The majority can participate with an accommodation (60.6 percent of fourth graders, 62.7 percent of eighth graders), with approximately a quarter able to participate without an accommodation (24.1 percent of fourth graders, 23.5 percent of eighth graders).
Economic Indicators: Free and Reduced-Price Lunch Program Eligibility and School Type
Table 6 provides a sense of the economic status of the students. It presents information regarding eligibility for free and reduced-price lunch programs, as well as percentage of students in public, private, and Catholic schools, by grade and disability status. A larger proportion of children with disabilities than those without disabilities are eligible for free and reduced-price lunch programs in both fourth (53.3 percent with disabilities compared with 39.7 without disabilities) and eighth grade (47.9 percent with disabilities compared with 32.5 percent without disabilities). The rate of eligibility for this program is indicative of a higher poverty rate among children with disabilities. A much higher proportion of students with disabilities attend public schools for both grades than those without disabilities, again indicative of the relative economic situation of the children with disabilities. A much smaller proportion of fourth graders with disabilities than those without disabilities attend private schools (1.1 percent compared with 5.0 percent) or catholic schools (1.3 percent compared with 5.3 percent). The same pattern is shown for eighth graders: private schools (0.7 percent compared with 4.7 percent) or Catholic schools (0.9 percent compared with 5.4 percent).
Inclusion Rates Estimates by Disability and Other Characteristics
Table 7 presents the number of students included in the NAEP mathematics assessment, by grade level and disability-related characteristics. As evidenced by the table, the inclusion rate varies widely depending on the type of disability. The percent included with autism and mental retardation is quite low, around 40 percent in the fourth and eighth grades. The disability type most likely to be included in the NAEP math assessment were those with speech or language impairments with 94.3 percent of fourth graders included and 88 percent of eighth graders included. Those with specific learning disabilities are also included at a fairly high rate: 82.7 percent of fourth graders and 84.6 percent in eighth grade. There is a fairly large change in the percentage included for those with visual impairments/blindness with 74.1 percent of fourth graders included as compared to 89 percent of eighth graders.
Disability severity clearly impacts the inclusion rate. Slightly less than half of those with profound/severe disabilities are included while around three quarters of those with moderate and 88.3 percent of those with a mild degree of disability were included in the NEAP math assessment. With regards to curriculum content, the majority of students with disabilities who had the same curriculum as those without disabilities were included in the NAEP math assessment (86.7 percent of fourth graders, 87.3 percent of eighth graders). Nearly all of those who were recommended to take the NAEP math assessment without an accommodation did so: 97.4 percent of fourth graders and 95.7 percent of eighth graders. The majority of those who were encouraged to participate with accommodations also participated (86.5 percent of fourth graders and 85.0 percent of eighth graders).
Inclusion Rates Estimates by Disability and State
Table 8a below and Table 8b in Appendix A present the number and percent of students with disabilities included in the NAEP math assessment by state. Overall, nearly four out of five students with disabilities in both grades (79.4 percent of fourth graders, 79.8 percent of eighth graders) participated in the NAEP at the national level. However, the proportion participating in the assessments vary considerably by state. In Mississippi less than half of students with disabilities participated in both grades. Over 90 percent of the students with disabilities participated in both grades in Wyoming, Alaska, New Jersey and Idaho. The largest decrease in percentage change in the participation rate between fourth grade and eighth grade was for three states: Delaware, Louisiana, and South Carolina which all showed a reduction of more than 16 percent between the two grades.
Comparison to OSEP Statistics. An important validity check of the NAEP data and our analytical sample is to compare statistics derived from our sample to statistics reported by OSEP. In Tables 9 and 10, we compare OSEP- and NAEP-based statistics with respect to the prevalence of disability across states and the distribution of students with disabilities over the types of disability. Despite substantial differences in the way OSEP statistics and our NAEP statistics were calculated (see table notes), a high degree of correlation across states and disability types exists. From the results of these comparisons, we conclude that the NAEP is a valid source of disability statistics. For both the NAEP and the OSEP, children with specific learning disabilities comprise the greatest share of youth with disabilities. Speech or language impairment is 16.2 percent of the NAEP fourth-grade population as compared to 19.6 percent of the OSEP population, although this proportion drops significantly in the eighth-grade NAEP population.
Table 10 compares the NAEP fourth- and eighth-grade estimates to the OSEP estimates for students ages 6-17 at the state and national levels. Even given the broader age range included in the OSEP figures, at the national level the OSEP estimates are quite similar to the NAEP estimates (NAEP: 11 percent of fourth graders, and 11.1 percent of the eighth graders have a disability; OSEP: 11.6 percent of students ages 6-17 have a disability). At the state level, a correlation of 0.59 exists for the fourth graders and 0.54 for the eighth graders between the NAEP and the OSEP estimates, despite the differences in the defined OSEP age groupings.
Comparison to American Community Survey Statistics
Table 11a compares the fourth-grade NAEP numbers to estimates derived from the 2003 ACS Public Use Micro-data Sample (PUMS) data, limited to children ages 9-10, the typical ages of those in fourth grade, at the national and state levels. Table 11b displays the same for eighth graders (ages 13-14). The prevalence rates are significantly smaller for the ACS in both tables. The disparity between the NAEP and ACS estimates may be attributed to the limited question set used for defining disability in the ACS and the broader disability determination used in the NAEP. The ACS questionnaire is also more likely to be completed by the parents who may be hesitant to identify or label their child as having a disability in the ACS form, or who may decide that their child does not fit into the ACS disability categories.
In terms of state-level correlations, there is a 0.26 correlation between the fourth-grader NAEP and the overall ACS disability measure, and a 0.34 correlation with the ACS mental disability category. Given that the majority of the NAEP population is children with LD, the correlation with the mental disability category makes sense as the ACS “mental disability” question used is focused on difficulty with “learning, remembering, or concentrating." For the eighth-grade NAEP population, these correlations drop to a 0.11 correlation with the overall ACS disability measure and a correlation of 0.07 with the ACS mental disability category.
Table 12a. Correlation coefficients between fourth-grade state
prevalence rates from the NAEP and ACS disability categories
|
||||||
|
|
NAEP Fourth Graders |
ACS Children Ages 9-10 |
||||
|
|
Overall |
Sensory |
Physical |
Mental |
Self-Care |
|
|
NAEP |
1.00 |
|
|
|
|
|
|
Overall |
0.26 |
1.00 |
|
|
|
|
|
Sensory |
-0.08 |
0.40 |
1.00 |
|
|
|
|
Physical |
-0.16 |
0.46 |
0.26 |
1.00 |
|
|
|
Mental |
0.34 |
0.92 |
0.19 |
0.22 |
1.00 |
|
|
Self-Care |
-0.22 |
0.42 |
0.20 |
0.71 |
0.33 |
1.00 |
|
Source: Authors' calculations using the NAEP 2003 Mathematics Assessment data, using the state samples, and the 2003 American Community Survey Public Use Microdata Sample (PUMS). |
||||||
This Guide carefully describes the information on the disability population from the National Center for Educational Statistics’ (NCES) National Assessment for Educational Progress (NAEP) - Student with Disabilities/Limited English Proficiency (SD/LEP) questionnaire. It began with a description of the ICF and ICF-CY conceptual model of disability. The ICF provides a framework that may be used to assess the disability information in the SD/LEP questionnaire as well as disability information in other surveys.
The Guide then presented an overview of the survey methodology and definitions. The design of the NAEP SD/LEP questionnaire provides several advantages over other data collection efforts. The NAEP SD/LEP questionnaire contains the largest amount of information on disability-related data for students with disabilities compared to all of the other major surveys, including particularly unique and extensive data on accommodations, school location and type, and participation in curriculum instruction and testing. The NAEP SD/LEP questionnaire also contains a broad set of data on disability type and disability severity.
We illustrate the utility of the NAEP SD/LEP questionnaire with estimates based on the NAEP SD/LEP questionnaire data for the students with disabilities population, including: the size of the population, the prevalence rate, the demographic composition, level of accommodation, participation in curriculum instruction, and eligibility for the federal Free and Reduced-Price Lunch program. At the national level, 11 percent of fourth-grade students and 11.1 percent of eighth-grade students have a disability. Fourth- and eighth-grade students with disabilities tend to be Black/African American, White (non-Hispanic), and Hispanic, and attend schools in rural areas. Specific learning disabilities are the most common disabilities among fourth and eighth grade students with disabilities. The majority of fourth- and eighth-grade students with disabilities have a mild or moderate disability, and are able to receive the same curriculum instruction as non-disabled students.
Finally, the User Guide compares estimates from the NAEP SD/LEP questionnaire to other national surveys that collect information on children with disabilities. Despite substantial difference in the way OSEP statistics and our NAEP statistics were calculated, there is a high degree of correlation across states and disability types. The lower ACS estimates are not surprising given the limited question set used for defining disability in the ACS and the broader disability determination used in the NAEP.
In conclusion, while there are limitations to the disability data collected in the NAEP SD/LEP questionnaire, the SD/LEP questionnaire provides substantial information about the numbers and characteristics of fourth- and eighth-grade students with disabilities in the U.S. The extensive data on disability type, curriculum and testing accommodations, and test participation are particularly valuable, because they capture information that is not available from other major surveys. The use of the SD/LEP questionnaire to monitor the status of the population of students with disabilities to provide insights into the level of accommodations needed by students with disabilities will be an important component of the Department of Education’s effort to meet the goals set forth by the Individuals with Disabilities Education Act and No Child Left Behind.
Altman, B. M. (2001). Disability Definitions, Models, Classification Schemes, and Applications. In G. L. Albrecht, K. D. Seelman, and M. Bury (Eds.) Handbook of Disability Studies. Sage Publications: Thousand Oaks CA.
Americans with Disabilities Act. Public Law 101-336, 42 U.S.C. ss 12101 et seq. (1990).
Clauser, S.B. & Bierman, A.S. (2003). Significance of Functional Status Data for Payment and Quality. Health Care Financing Review, 24(3), 1-12.
Individuals with Disabilities Education Act Amendments of 1997 (P.L. 105-17), 111 Stat. 37-157. (1997).
Jette, A., & Badley, E. (2000). Conceptual Issues in the Measurement of Work Disability. In N. Mathiowetz and G.S. Wunderlich (Eds.) Survey Measurement of Work Disability: Summary of a Workshop. Washington DC: National Academy Press: 183–210.
Mathiowitz, N. (2000). Methodological Issues in the Measurement of Work Disability. In N. Mathiowetz and G. Wunderlich (Eds.) Survey of Measurement of Work Disability: Summary of a Workshop. National Academy Press: Washington D.C.
Nagi, S. (1965). Some Conceptual Issues in Disability and Rehabilitation. In M.B. Sussman (Ed.) Sociology and Rehabilitation. Washington, DC: American Sociological Association.
Nagi S. (1976). An epidemiology of disability among adults in the United States. Milbank Memorial Fund Quarterly: Health and Society, 54, 439–467.
National Assessment of Educational Progress Authorization Act. Public Law 107-279 III, section 303 (2002).
Simeonsson RJ & Leonardi M. (2004, October). ICF Version for Children and Youth: Development and Field Trial. Paper presented at WHO Family of International Classifications Network Meeting, Reykjavik.
World Health Organization (1992). The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. Geneva, Switzerland: Author.
World Health Organization (1993). The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. Geneva, Switzerland: Author.
World Health Organization. (2001). International Classification of Functioning, Disability, and Health. Geneva, Switzerland: Author.
World Health Organization. (2002). Towards a Common Language for Functioning, Disability, and Health: ICF. Geneva, Switzerland: Author.
World Health Organization (2007). International Classification of Functioning, Disability and Health - Children and Youth Version. ICF-CY.