UARK

 

 

Education Working Paper Archive   

Keyword: School Choice
.pdf version adobe
 

Who Chooses, Who Uses?
Initial Evidence from the D.C. Opportunity Scholarship Program


Patrick Wolf

University of Arkansas

Nada Eissa
Georgetown University

Babette Gutmann
Westat

July 6, 2006


Abstract

The federal government recently enacted its first school voucher program as a pilot project in the District of Columbia. To be eligible, students need to be entering grades K-12 and have a family income at or below 185 percent of the poverty level. Although a rigorous analysis of the Opportunity Scholarship Program’s impact on student achievement and other outcomes remains a prospect for the future, at this early point initial data exists regarding the families that are applying for the program and the students that are using and not using the voucher when offered. Here we present a preliminary analysis of those data. We find that program applicants are somewhat disadvantaged relative to non-applicants regarding educational characteristics and family income, and are more likely to be African American, than non-applicants. The fact that the program is means-tested appears to be central to the finding that it is reaching a more disadvantaged population of students. When we examine all students that received a voucher award, and compare the group of voucher users with the group of voucher decliners, we find two significant differences. First, scholarship users are educationally advantaged in important ways relative to scholarship decliners. They are much less likely to have learning or physical disabilities, and younger scholarship users evidence somewhat higher test scores than non-users in similar grades. Second, we find that scholarship non-users are more likely to report that their existing school has various specialized educational programs and more extensive facilities. Although these results suggest some measure of selectivity in the group of actual program participants, the data do not indicate conclusively if that selectivity is a function of the decisions and behavior of participating private schools or the result of the rational decisions of consumers in a newly-expanded education market.


On January 23, 2004, President Bush signed the DC School Choice Incentive Act into law. This landmark piece of legislation included $14 million in funding for what would come to be called the DC Opportunity Scholarship Program (OSP).[1]  The OSP is the first federally funded K-12 scholarship program in the country and is designed to provide approximately 1,700 low-income DC children with tuition scholarships worth up to $7,500 to cover the costs of attending participating K–12 nonpublic schools in the District.  The pilot program is authorized to operate for five years and is being implemented by a group of non-profit organizations headed by the Washington Scholarship Fund.

In addition to extending educational choices to an economically disadvantaged group of DC families, the OSP also provides the opportunity to learn more about what happens when more families have the opportunity and responsibility to choose a private school for their children.  The U.S. Department of Education, through the Institute for Education Sciences, has contracted for the conduct of a rigorous experimental evaluation of the impact of the program on a number of student outcomes including student achievement.  Here we provide information from a separate, independent assessment of the characteristics of the initial cohort of eligible families that appear to influence their decisions to (1) apply for the program and (2) use a scholarship to attend a private school of choice if awarded a scholarship. 

The information presented in this report was gleaned from data obtained from program applicants and the District of Columbia Public Schools during the initial year of program implementation in 2004.  The authors are all members of the research consortium selected by the U.S. Department of Education, Institute for Education Sciences, to provide technical assistance to the program in the first year and conduct a rigorous, comprehensive, experimental impact evaluation of the program during its statutory 5-year pilot period.  Although the data that inform this particular analysis were all collected as part of that broader evaluation, and the initial results presented here are all drawn from the evaluation’s baseline report to Congress[2], the interpretations and conclusions presented here are solely the independent judgments of the authors and should not be ascribed to the official evaluation, the U.S. Department of Education, or any of the institutions with which the authors are affiliated.

Although this study is not part of the official evaluation of the Opportunity Scholarship Program (OSP), it does provide a concise and revealing description of what sorts of eligible families initially applied for the program, did not apply for the program, and used a scholarship if offered.  Thus, this analysis speaks to important policy questions of the extent to which the program is reaching its target population and what potential barriers might be preventing targeted families from fully participating in the nation’s first federally-sponsored school voucher program.

As a preliminary analysis, this study has important shortcomings.  First, it examines only the initial cohort of participating families recruited in April and May of 2004.  Since then, a large second cohort of eligible participants has joined the program, filling it to capacity and generating a sizable randomized control group.[3]  The pattern of who chooses and uses from that second cohort of participants may be substantially different from the pattern uncovered for the first cohort in this paper.  Second, our preliminary analysis merely involves comparing the means of various relevant populations of eligible program applicants, non-applicants, scholarship users and non-users.  Once we have collected longitudinal data on program participation, we will be able to conduct more sophisticated statistical analyses of the factors that influence persistence in the program.[4]  

Theory and Previous Research
In the 50 years since economist Milton Friedman published “The Role of Government in Education” scholars and policy makers have been debating how parental choice through market mechanisms can and does operate in education.[5]  Market “optimists” argue that education is a service that can be produced under a variety of arrangements and that parents are natural education consumers.[6]  Market “pessimists” argue that education is a public good best produced in government-run schools, and that school choice programs are prone to “market failure” because advantaged families are more likely than disadvantaged ones to have the motivation, resources, and experience to choose schools effectively.[7]

Fortunately, researchers have begun to study how market mechanisms like school choice work in practice, shedding light on a number of academic disputes and policy questions.  Still, much remains to be learned about how low-income families respond to the new opportunities made available to them through school choice programs.[8]  Moreover, how school choice programs are designed, and the real-world context in which they are implemented, appears to strongly influence their success.[9]
   
Our purpose in examining the characteristics of parents and students who are participating in the OSP is precisely to better understand the reality within which this new school choice program is operating.  The OSP, therefore, offers a rare contemporary opportunity to examine prominent theories and advance research on what kinds of parents and students avail themselves of school choice. 

Why are School Choice Programs Important? 
The OSP is a parental school choice program targeted to low-income families living in the District of Columbia.  It permits parents to send their children to private schools, of their choosing, at public expense.  School choice is not limited to places with K-12 scholarship programs, such as Milwaukee, DC, and the states of Ohio, Florida, and Utah.  As Elmore and Fuller point out, “Choice is everywhere in American education.  It is manifest in the residential choices made by families with school-age children; it is capitalized in the housing prices found in neighborhoods.”[10]  Since most school children are assigned to a public school based on where they live, housing markets play a very important role in American education, so much so that Patrick Wolf has argued: “School choice and residential assignment are both market-based methods for allocating students to schools.”[11] 
   
As an alternative to school assignment by residence, school choice programs hold the prospect of correcting some of the “market failures” associated with a reliance on real estate markets to match students with schools.  After all, low-income families may not view their neighborhood public school as a desirable place for their children to be educated, and they tend to lack access to private schooling or high-priced residential areas with more attractive schools.
However, the extent to which school choice programs operate democratically and effectively to place students in appropriate schools depends heavily on the willingness and ability of participating families to operate as effective education consumers.  Thus, the first issue we will examine in this paper is the extent to which applicants to the OSP are representative of the larger population of DC students.  

How Selective are the Families that Actually Use Scholarships?
A debate continues to rage in the policy world regarding the extent to which school choice programs will primarily serve the proverbial “cream of the crop” from among scholarship recipients.  Theoretical works by a number of prominent academics have predicted that cream-skimming will dominate school choice programs.[12]  Other theorists argue that disadvantaged families will be more likely to avail themselves of new school choice programs, since their schooling options are the most constrained by systems of assigning students to schools based on residence.[13] 

In general, the results of previous studies suggest that the selectivity of a choice program depends heavily on how it is designed and implemented.  Hamilton and Guin conclude that “Parental choice is obviously constrained by the options available to them, and in many cases the options are quite limited.”[14]  When parents have a variety of schooling choices and choice schools have a limited to select their choice students, parents are more likely to find a school that is a good fit for their child’s particular needs.  When choices are limited and the schools themselves make the admission decision, the chances of a desirable placement can be significantly reduced.  For example, Fiske and Ladd report that not all parents in New Zealand’s universal choice program had viable choices, given a shortage of alternatives and out-of-pocket expenses required to enroll in desired schools.  Since oversubscribed schools were allowed to choose their students, “Parental choice, in short, gave way to school choice.”[15]

When school choice programs are limited to low-income families, as is the case with the OSP, previous research indicates that most participants are highly disadvantaged.  For example, the official evaluators of the Milwaukee and Cleveland school voucher programs concluded that those means-tested choice programs were reaching the disadvantaged populations of families intended by policy makers.[16]  Evaluations of voucher-like private scholarship programs have reported some evidence that the educational resources of eligible families -- such as family income, mother’s education, and student’s prior academic achievement -- appear to only modestly and inconsistently influence their ability to use a scholarship.[17]  In the American context, race is often a very important consideration when examining participation in an education program and previous analyses of voucher programs in the U.S. have reached somewhat inconsistent findings regarding the extent to which race is a factor in influencing scholarship use.[18] 

Probably the toughest test for the selectively of voucher programs is the extent to which they enroll students with education-related disabilities.  Particular private schools have long served students with disabilities and some, such as the residential schools for hearing impaired students that are present in many states, are designed to serve such a clientele exclusively.  Urban public school districts regularly contract with private schools to educate students with disabilities at public expense – a practice generally referred to as “non-public placements” – and America’s largest school voucher program is Florida’s McKay Scholarship Program for students with disability-related educational needs.[19]  However, private schools are not legally required to enroll or accommodate students with disabilities of a degree or kind that they are ill-equipped to serve effectively, and most of the urban private schools that participate in voucher programs are low-tuition religious schools that lack the resources and staff to teach students with more than modest educational disabilities.  Thus, the second question we will explore is the extent to which such factors as race, income, parental education, prior academic achievement, and special education status affect scholarship usage in the OSP.

What Motivates Families to Participate?
If scholarship users tend to differ from non-users, is it because users are more motivated to initiate an educational change than are non-users?  There is general agreement that parents seek educational choices beyond their assigned neighborhood public school because they are looking for a better educational alternative for their children.  Fuller and his colleagues conclude that, “When benefits are targeted to low-income families, many parents do actively choose a school that they believe better fits their educational agenda than does the neighborhood school.”[20]  As Henig writes:

Even under the best of circumstances, the neighborhood public school will not adequately serve the needs of every neighborhood child.  This can be due to the particular characteristics of the child, the particular limitations of the school, or a simple lack of fit between one and the other.[21]   


There is some disagreement, however, regarding whether choice families tend to be primarily running away from a bad schooling situation or running toward a good one.  Low-income parents and students in urban neighborhood public schools frequently report disturbing levels of violence in their schools, and resulting concerns about student safety.[22]  At the same time, a survey of the literature on why parents choose schools suggests that a better educational program, a religious educational environment, and a better match between the cultural values of the family and school are often motivators for choosing.[23]  Obviously, school choosers are looking for a change.  Whether they are mainly seeking to escape from a bad situation or are simply interested in a distinct school that might be a better fit for their child remains an open question.  Thus, the third question we will explore in this paper is the extent to which families who are concerned about the safety or program offerings of their previous public schools are more likely to use a scholarship when offered.

Research Design and Data         
The conceptual foundation for our analysis is quite simple.  A defined population of families was offered the opportunity to apply to the OSP.  To what extent and regarding what factors did the sub-group of program applicants differ from the sub-group of non-applicants?  In the initial year of the program, 1,366 eligible applicants were offered scholarships.  A total of 1,027 scholarship recipients “took up” or used their scholarship to enroll in a private school of choice as of September 10 of 2004, leaving 339 initial scholarship non-users or “decliners.”  To what extent and regarding what factors did the sub-group of scholarship users differ from the sub-group of scholarship decliners?  For all of these comparisons, simple two-tailed t-tests are used to identify whether group differences are statistically significant at or beyond the 95 percent confidence level.

Our data for this analysis come from two sources.  The comparison of program applicants to non-applicants is based on information in the 2003-04 Accountability Student Database provided by the DCPS Office of Communications and Public Information.  The DCPS accountability database includes demographic information on 72,511 students in DCPS traditional and public charter schools, of which 1,077 could be definitively identified as program applicants.  The group of applicants successfully matched in the DCPS database represents 80 percent of all public school applicants.  The DCPS database contains variables for student test scores, enrollment in the federal lunch program, grade, race, gender, and whether or not the student has an Individualized Education Plan as a result of special educational needs linked to a disability.  This part of the analysis necessarily excludes all 515 initial private school eligible applicants to the program, since no aggregate data are available regarding private school non-applicants.

Test scores are a crucial element of most school choice analyses, even if the question at hand is not program impact but the degree of selectivity of program clientele.  The accountability database contains the results of DCPS administration of the Stanford Achievement Test, Version 9 (SAT-9) to all students in grades 3 through 11, and some students in grades 1 and 2, in the spring of 2004.  These tests covered the areas of reading and mathematics, with results expressed in terms of National Percentile Ranks (NPRs).  The NPR for a particular student test score describes the percentage of students who scored below that level on a nationally normed test for the specific grade level, subject, and testing period that applies to the student score.[24] Expressing test-score norms in this way makes them approximately comparable across grade levels.

Since our sample of program applicants matched in the DCPS database does not include 20 percent of public school applicants, we conducted an additional analysis to test for sample bias.  This sample bias analysis draws from the second database used in this paper – the 2004 Program Database.  The Program Database includes data on all applicants to the program in the first year, taken from the program application form and baseline survey.  Since all public school applicants are in the program database, 80 percent are in both the program and accountability database, and 20 percent are only in the program database, a comparison of the characteristics of the subgroup that is in both databases with those of the subgroup only in the program database provides information about possible bias in our “applicants versus non-applicants” analysis which necessarily draws exclusively from the accountability database.

We compared the characteristics of matched versus unmatched applicants on 20 factors associated with the amount and sources of family income, parent marital status, and mother’s level of educational attainment.  The two samples differed significantly on only one factor – average amount of child support received.  One might expect a comparison of two samples to differ significantly on one of 20 factors merely by chance.  Thus, this high level of comparability between the matched and unmatched groups of public school applicants suggests that the matched sample which is used in the analysis is representative of the population of the all public school applicants.

The program database used to disconfirm the presence of sample bias in the accountability database also is used in this paper to analyze the selectivity of voucher users compared with voucher decliners.  This database includes observations on all 1,366 students awarded scholarships in the first year of the program, of which 1,027 used the scholarship – a moderately-high usage rate of 75 percent – and 339 declined or failed to use.  Although we cannot perfectly distinguish the non-users who merely decided not to use a scholarship from those who tried to use one but failed to gain access to a preferred private school, we can get some sense of the likely degree of exclusionary behavior on the part of participating private schools by the level of exclusivity of the group of scholarship users relative to non-users.

Analysis and results     
Our analysis proceeds in two stages.  First we examine program applicants who were attending public schools at the time of application and compare them to their public school peers who did not apply for the program.  Due to the limitations in the variables contained in the DCPS accountability database, this first set of comparisons will focus on a small set of educational characteristics and background factors.  The second stage of the analysis will draw upon extensive data about all scholarship recipients and address the question of what factors appear to distinguish scholarship users from non-users.

Who Applied to the Scholarship Program?
The first set of question we explore is whether program applicants differ from non-applicants, and what that might suggest about the motivations of parents and the broader impact of the program on the DC educational landscape.  Here we draw from the DCPS accountability database to compare program applicants entering grades K-12 with DCPS non-applicants in similar grades.  The grade-restriction on the data reduces the total number of observations to 67,945.

The non-applicant sample in this analysis includes DCPS students ineligible for the program due to family incomes above the statutory limit of 185 percent of the poverty line.  Later we will conduct a similar analysis restricted to students that were income-eligible for the program.  However, policy design determines who is and who is not eligible for school voucher programs, and it is useful to assess the extent to which the design of this particular school choice program may have mitigated the selectivity of the group of program applicants. (see Table 1)

The data indicate that program applicants are somewhat disadvantaged educationally and economically, and more likely to be African American, compared with non-applicants (Table 1).  Applicants and non-applicants are similar, on average, regarding both their reading and math test scores at baseline; however, 16 percent of the applicant sample is comprised of students in special education compared with only 14 percent of the non-applicant sample.  Applicants are more likely than non-applicants to be African American and less likely to be Hispanic or of another race – in this case, generally white or Asian.  The two groups are closely matched regarding student gender; however, applicants are much more likely than non-applicants to be participants in the federal lunch program for low-income students. (see Table 2)

The picture changes in important ways when the two samples are limited to students participating in the federal lunch program.  The lunch program was selected as an inclusion screen because its income restriction closely matches the income ceiling for eligibility for the voucher program.  Limiting the comparison groups to students in the lunch program has the effect of reducing the size of the public school applicant group to 894 students and the size of the non-applicant sample to 44,740.[25] 

When we look only at income-eligible students that either did or did not apply for the program, the program applicants no longer appear to be disadvantaged overall relative to non-applicants (Table 2).  The lunch program restriction automatically equalizes the two groups on our income measure, and the two groups remain similar regarding their gender composition.  However, the race differences across the groups change somewhat.  Hispanic students remain somewhat under-represented in the group of program applicants, and African Americans somewhat over-represented, but the proportion of students in the other race category (i.e. white or Asian) becomes similar across the comparison groups.  Among income-eligible students, the percentages of students in both groups who are in special education are statistically similar.  Importantly, the baseline test-scores of program applicants compared with income-eligible non-applicants average nearly four percentile points higher in both reading and math. (see Table 3)

Finally, we conduct comparisons of applicants with income-eligible non-applicants within relevant grade groupings.  Applicants were much more likely than non-applicants to be entering the early elementary grades of K-1 and much less likely to be entering high school at the time of application in the spring of 2004 (Table 3).   Thus, some of the differences between applicants and program-eligible non-applicants might be a result of this compositional difference – younger students are a greater proportion of the applicant group and may be different than older students in certain respects.

Our analysis largely bears out that expectation (see Table 4).  The modest test-score advantage of applicants relative to program-eligible non-applicants is confined to students in the younger elementary grades.  Applicants entering junior high or high school are similar to non-applicants regarding their average baseline test scores.  The group of junior high applicants also is similar to the group of junior high non-applicants regarding racial demographics, though the early elementary and high school groups of applicants remain more likely to be African American than grade-comparable non-applicants.  The groups of program applicants in all three grade-bands are similar to their respective grade-band groups of non-applicants regarding special education status, gender, and, by analytic design, participation in the federal lunch program.

In summary, the extent to which the initial applicants to the Opportunity Scholarship Program appear to be selective regarding educational, social, and economic factors depends on the composition of the group of non-applicants with which we compare them.  Compared to all DCPS students, program applicants appear to be somewhat disadvantaged.  When the comparison is limited to program-eligible non-applicants, the applicants are somewhat advantaged educationally and remain more likely to be African American.  When even finer comparisons are made, between program applicants and program-eligible non-applicants within defined grade-ranges, we see that junior high applicants are statistically similar to non-applicants, early elementary applicants are somewhat advantaged educationally and more likely to be African American, and high school applicants are more likely to be African American.

Who Uses an Opportunity Scholarship?
In the next stage of our analysis, we examine the characteristics of scholarship users and non-users.  The sample includes all 1,366 students offered scholarships during the initial year of program implementation, including 216 eligible applicants who were attending private schools at the time of application.  We compare the two groups – users versus non-users -- based on four sets of factors:  educational characteristics of students, family background, student ethnicity and gender, and previous educational environment.

The students who used their scholarships differed from non-users on some but not all key educational characteristics (Table 5).  In the early elementary school grades of K-5, on average, users outperformed non-users in reading by nearly 9 NPR and in math by nearly 10 NPR.  Users in the junior high and high school grades did not differ significantly from non-users in those grades regarding reading and math baseline scores.  Students whose parents said that they had a learning or physical disability were much less likely to use a scholarship if offered.  Students with disabilities comprised just nine percent of the user group but 29 percent of the non-user group.  Students were much more likely to use a scholarship if entering the early grades of K-3 but were much less likely to use a scholarship if entering the junior or high school grades of 6-12. (see Table 4 and Table 5)

Regarding family background, users differed from non-users in some important respects but not others.  There is little distinction between users and non-users regarding family income, as both groups represent populations that are about equally disadvantaged economically, with annual family incomes averaging less than $20,000.  Mothers of scholarship users have slightly more education, averaging two-tenths of a year more formal education than the mothers of non-users.  However, users are indistinguishable from non-users regarding the likelihood of their mothers being employed full time or being married.  One surprising initial finding regarding scholarship users is that they reported less residential stability, averaging 1.6 fewer years at their current residence than non-users. (see Table 6))

The user and non-user groups do not differ significantly regarding student race, ethnicity, or the primary language spoken in the home (Table 7).  Although gender was not a factor in distinguishing applicants from non-applicants, it was a factor in shaping the decision to use a scholarship, as the composition of the user group was 9 percentage points more female than the composition of the decliner group. (see Table 7)

The previous educational behaviors and experiences of users and non-users might differ in meaningful ways.  First, we consider levels of parental involvement in the child’s education (Table 8).  We see that both groups of parents, scholarship users and non-users, reported being heavily involved in their child’s education at baseline.  However, scholarship users were somewhat more likely than non-users to say that they helped their child with reading or math work not related to homework, assisted with homework, and attended school activities with their child.

Next, we analyze differences between users and non-users regarding problems at their child’s previous school.  Parents whose children encountered disorder or danger regularly in their previous school are probably more likely to use a scholarship as a means to escape such an environment.  We see some evidence that problems at the previous school were an extra motivator for scholarship users (Table 9).  Scholarship users were somewhat more likely than non-users to report that drug distribution and drug and alcohol use were serious problems at their child’s previous school, though there was no difference between the two groups in reports of several other categories of danger and disruption. (see Table 8 and Table 9)

Finally, we consider the extent to which non-users may have felt motivated to decline the scholarship and remain at their previous school due to the school’s facilities or programs.  Here we see some of the largest and most consistent differences between users and non-users (Table 10).  Scholarship non-users consistently reported that their child’s previous school contained more extensive facilities and specialized programs than scholarship users.  For example, 17 percent more non-users than users reported that their child’s previous school had a special program for non-English speakers, 15 percent more said that special education programs were offered, 12 percent more reported that the school included a special program for advanced learners, and eight percent more said that individual tutors were available to students.  The parents of scholarship non-users were more likely than those of scholarship users to report that their previous school had a gym, nurse’s office, cafeteria, prepared lunches, child counselors, and library.  If parent reports are to be believed, the schools that scholarship users decided to leave contained fewer special educational programs and facilities than the schools that non-users decided to remain in. (see Table 10)

Discussion
This initial investigation into who chooses and who uses an Opportunity Scholarship in the District of Columbia offers an interesting, complex, though tentative glance at the characteristics, motivations, and possible obstacles faced by parents and students.  Far from being a privileged sub-section of the District’s educational system, initial applicants coming from DC public schools were similar to non-applicants in their baseline test scores and more likely to be enrolled in special education, African American, and participants in the federal lunch program for low-income students.

The disadvantaged nature of the population attracted to the program appears to be largely a function of the policy decision to restrict program eligibility to students with family incomes at or below 185 percent of the poverty line.  When the comparison is made with program-eligible non-applicants, applicants have slightly higher test scores and remain slightly more likely to be African American than similarly impoverished non-applicants.  Parents appear to be much more motivated to seek educational options for younger children, as evidenced by the fact that students entering kindergarten or first grade were much more likely to apply for the program and students entering junior high or high school were much less likely to apply. 

When comparing applicants to income-eligible non-applicants within relevant grade bands, we see that the test-score advantage of applicants relative to non-applicants is limited to younger elementary students and is not present among either the junior high or high school grade bands.  In fact, junior high students appear to be statistically similar to eligible non-applicants regarding educational and demographic characteristics and high school applicants only differ from comparable non-applicants in that they are more likely to be African American.  This largely confirms the results of previous studies that have found that low-income African American students are more likely to apply for voucher programs when given the opportunity to do so.[26]

To the extent that an element of selectivity has emerged in the OSP it has been at the stage of scholarship use.  Student race, primary language, and family income do not distinguish users from non-users, but some educational characteristics do set users apart from non-users.  Scholarship users are much more likely to be at the early stage of their education, K-4, and users at that stage evidence reading and math test scores that are moderately higher than non-users.  Older students who use the scholarship have initial test scores that are statistically similar to non-users.  However, students with a learning or physical disability were much less likely to use a scholarship in the first year of program implementation – the single largest distinction between the two groups.

It is impossible to know at this point if the sizable difference between scholarship users and non-users regarding learning and physical disabilities is the result of parental decisions regarding what is best for their child or limitations in the ability of participating private schools to accommodate the special educational challenges that some scholarship students face.  The District of Columbia Public Schools has a large “private placement” program for students with special education needs that sent 2,595 students to private or suburban public schools at public expense in 2004, at an average tuition cost of $27,575 per student.[27]

When District parents decide on their own to enroll their special needs child in a private school, the child’s Individual Education Plan and the resources attached to it do not transfer to the parentally chosen private school.  For parents of students with a moderate-to-severe educational disability, the $7,500 voucher may provide insufficient funds, in their view, to obtain the array of educational services necessary to educate their special needs child.  Although two-thirds of the private schools participating in the OSP in the first year reported that they accept students with learning disabilities, the overwhelming majority of such schools fully include such students in their standard educational environment, with no separate special education program.[28]  Parents may plausibly be deciding that a public school environment, with an extensive array of support services, is better for their special needs child than a private school environment with no special programs, in spite of the award of the voucher.

There is substantial additional evidence, though preliminary, that scholarship non-users may largely be declining to use the scholarship due to the special programs and more extensive facilities at their existing schools.  Non-users were more likely than users to report that their child’s school offered special programs for both disadvantaged and advantaged students, and a more extensive physical plant including a gym, cafeteria, and library.  Many parents offered a scholarship might have compared their child’s existing school to the schools available to them in the program and decided that their current school was the better option.  Although the families of scholarship users were slightly more advantaged in terms of mother’s education, residential stability, and parental involvement, scholarship non-users appear to be much more advantaged in terms of the features and programs of the schools that they previously attended.

All of these findings should be taken with a grain of salt.  The initial cohort of eligible applicants to the OSP was modest in size, numbering only 1,848 from both public and private schools.  The second cohort of applicants was slightly larger, including 2,199 eligible applicants.  The characteristics, motivations, preferences, and experiences of this second cohort of applicants could be decidedly different from the original group, painting a contrasting picture of who chooses and uses an OSP scholarship.  Moreover, with a larger and richer set of data regarding OSP applicants, non-applicants, users, and non-users, in the future we will be able to conduct more sophisticated statistical analyses of the factors that predict the decisions to apply for the OSP and use the scholarship once offered.[29]

For now, our initial analysis suggests that disadvantaged students and families are applying for the first ever federal school voucher program.  Among scholarship recipients, somewhat educationally advantaged students are more likely to use the scholarship and students attending schools with more educational programs and more extensive facilities are more likely to decline to use the scholarship when offered.


Footnotes

[1] Title III of the District of Columbia Appropriations Act of 2004, Division C of HR 2673, 118 Stat. 117, D.C Code Sec. 38-1851.01.
[2] Patrick Wolf, Babette Gutmann, Nada Eissa, Michael Puma, and Marsha Silverberg, Evaluation of the DC Opportunity Scholarship Program: First Year Report on Participation U.S. Department of Education, National Center for Education Evaluation and Regional Assistance (Washington, DC: U.S. Government Printing Office, 2005).
[3] Patrick Wolf, Babette Gutmann, Michael Puma, and Marsha Silverberg, Evaluation of the DC Opportunity Scholarship Program: Second Year Report on Participation, U.S. Department of Education, Institute of Education Sciences (Washington, DC: U.S. Government Printing Office, 2006).
[4] See, for example, William G. Howell, “Dynamic Selection Effects in Means-Tested, Urban School Voucher Programs,” Journal of Policy Analysis and Management, 23:2 (2004).
[5] Milton Friedman, “The Role of Government in Education” in Robert Solo (ed.), Economics and the Public Interest (Rutgers, NJ: Rutgers University Press, 1955).
[6] See Milton Friedman, Capitalism and Freedom (Chicago: University of Chicago Press, 1962); John E. Chubb and Terry M. Moe, Politics, Markets, and America’s Schools (Washington: Brookings, 1990).
[7] See John Dewey, Democracy and Education (New York: Macmillan, 1916); Amy Gutmann, Democratic Education (Princeton, NJ: Princeton University Press, 1987); Helen F. Ladd, Market-Based Reforms in Urban Education (Washington, DC: Economic Policy Institute, 2002); Henry L. Levin, “Educational Vouchers: Effectiveness, Choice, and Costs,” Journal of Policy Analysis and Management. 17, (June 1998).
[8] See, for example, David E. Campbell, Martin R. West, and Paul E. Peterson, “Participation in a National Means-Tested School Voucher Program,” Journal of Policy Analysis and Management, 24:3 (2005); Edward Fisk and Helen Ladd, When School Compete: A Cautionary Tale (Washington: Brookings, 2000); Brian P. Gill, P. Michael Timpane, Karen E. Ross, and Dominic J. Brewer, Rhetoric Versus Reality: What We Know and What We Need to Know About Vouchers and Charter Schools (Santa Monica, CA: RAND, 2001).
[9] Jeffrey R. Henig, Rethinking School Choice: Limits of the Market Metaphor (Princeton, NJ: Princeton University Press, 1994), p. 21; Bruce Fuller, Richard F. Elmore, and Gary Orfield, “Policy Making in the Dark: Illuminating the School Choice Debate,” in Bruce Fuller and Richard F. Elmore (eds.), Who Chooses? Who Loses? Culture, Institutions, and the Unequal Effects of School Choice (New York: Teachers College Press, 1996), pp. 12-13; Richard F. Elmore and Bruce Fuller, “Empirical Research on Educational Choice: What are the Implications for Policy-Makers?” in Bruce Fuller and Richard F. Elmore (eds.), Who Chooses? Who Loses? Culture, Institutions, and the Unequal Effects of School Choice. (New York: Teachers College Press, 1996), p. 200; James Harvey and Lydia Rainey, Doing Choice Right: Proceedings of a Meeting on Communities and Choice (Center on Reinventing Public Education: University of Washington, 2004) p. 10.
[10] Richard F. Elmore and Bruce Fuller, “Empirical Research on Educational Choice: What are the Implications for Policy-Makers?” in Bruce Fuller and Richard F. Elmore (eds.), Who Chooses? Who Loses? Culture, Institutions, and the Unequal Effects of School Choice (New York: Teachers College Press, 1996), p. 187.
[11] Patrick J. Wolf, “Comment on ‘School Choice: How an Abstract Idea Became a Political Reality,’” in Diane Ravitch (ed.), The 2004 Brookings Papers on Education Policy (Washington: Brookings, 2005), p. 164.
[12] See, for example, Albert O. Hirshman, Exit, Voice, and Loyalty (Cambridge: Harvard University Press, 1970); Bruce Fuller and Richard F. Elmore (eds.), Who Chooses? Who Loses? Culture, Institutions, and the Unequal Effects of School Choice (New York: Teachers College Press, 1996); Alan Wolfe (ed.), School Choice: The Moral Debate (Princeton: Princeton University Press, 2003).
[13] See for example Caroline M. Hoxby, “Introduction,” in Caroline M. Hoxby (ed.), The Economics of School Choice (Chicago: University of Chicago Press, 2003).
[14] Hamilton and Guin, “The Demand Side of School Choice…, p. 20.
[15] Fiske and Ladd, When Schools Compete…, pp. 8-9.
[16] John F. Witte, The Market Approach to Education (Princeton: Princeton University Press, 2000), pp. 60-61, 67; Kim K. Metcalf, “Evaluation of the Cleveland Scholarship and Tutoring Program” (Bloomington: Indiana Center for Evaluation, University of Indiana, 1999), p. 14.
[17] William G. Howell and Paul E. Peterson, with Patrick J. Wolf and David E. Campbell, The Education Gap: Vouchers and Urban Schools (Washington: Brookings, 2002), pp. 56-89.
[18] Campbell et al., “Participation in a National, Means-Tested School Voucher Program…”; Howell, “Dynamic Selection Effects…
[19] Jay P. Greene and Greg Forster, Vouchers for Special Education Students: An Evaluation of Florida’s McKay Scholarship Program, Manhattan Institute for Policy Research, Civic Report No. 38 (New York, June 2003).
[20] Fuller et al, “Policy Making in the Dark…, p. 9.
[21] Henig, Rethinking School Choice…, p.206.
[22] Wells, “African-American Students’ View of School Choice…,” pp. 43, 45; Wolf et al, Evaluation of the DC Opportunity Scholarship Program…, p. C-5.  William G. Howell and Paul E. Peterson, with Patrick J. Wolf and David E. Campbell, The Education Gap: Vouchers and Urban Schools (Washington: Brookings, 2002), pp. 110-112.
[23]  Valerie Martinez, Kenneth Godwin, and Frank R. Kemerer, “Public School Choice in San Antonio: Who Chooses and with What Effects?” In Bruce Fuller and Richard F. Elmore (eds.), Who Chooses? Who Loses? Culture, Institutions, and the Unequal Effects of School Choice (New York: Teachers College Press, 1996), p. 54.
[24]For example, the score average of nearly 47 NPRs for all K to 5 applicants in reading is about 3 percentile points below the national median for those grades in reading, as approximately 47 percent of students across the country who took the reading test last spring in those grades scored lower than the average for the program applicants.
[25] Although all 1,077 applicants matched in the DCPS database were confirmed to be income-eligible for the voucher program and thus almost certainly also eligible for the lunch program, it is well known that not all families eligible for means-tested programs such as the federal lunch program actually enroll in the program.  To be included in the sample for this analysis, students had to actually be participating in the federal lunch program at baseline.
[26] Campbell, West, and Peterson, “Participation in a National, Means-Tested School Voucher Program…”; Howell, “Dynamic Selection Effects in Means-Tested, Urban Voucher Programs…”
[27] www.dcschoolsearch.dc.gov/schools/report_results.asp?report_id=14/ (accessed November 11, 2004).
[28] Wolf et al, “Evaluation of the DC Opportunity Scholarship Program…”, pp. 14-15.
[29] See, for example, the nested logit analysis employed by Campbell, West, and Peterson, “Participation in a National, Means-Tested School Voucher Program…”, and the survival models estimated in Howell, “Dynamic Selection Effects…”


Table 1. Characteristics of DC Public School Students, Program Applicants
and Non-applicants: Spring 2004

Characteristic

Applicants

DCPS Sample

Difference

Baseline Test Scores1

 

 

 

 

 

 

 

Average Reading Percentile

41.5

40.1

1.4

Percent missing

26

26

 

 

 

 

Average Mathematics Percentile

47.2

46.3

1.0

Percent missing

25

25

 

 

 

Percent in Special Education

16

14

2*

Percent missing

24

23

 

 

 

Percent, by Race

 

 

 

African American

92

85

8**

Hispanic

6

9

-4**

Other race2

2

6

-4**

Percent missing

1

1

 

 

 

Percent, by Gender

 

 

 

Female

51

50

0

Percent missing

0

1

 

 

 

Percent Participating in Free/Reduced-price Lunch Program

85

68

17**

Percent missing

2

2

 

 

 

 

Sample size

1,077

66,868

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
1. Test-score results are in terms of National Percentile Ranks, with 50 as the median score.
2. "Other race" includes students who were identified as white, Asian, American Indian, or Alaska Native.

NOTE: Detail may not sum to totals because of rounding.  Applicant sample includes all applicants identified in the DCPS database, regardless of whether or not they participated in the accountability testing.
SOURCE: Accountability testing database for District of Columbia public and charter schools, DCPS Office of Communications and Public Information.

Table 2. Characteristics of DC Public School Federal Lunch Program Students,
Program Applicants and Eligible Non-applicants: Spring 2004

Characteristic

Applicants

DCPS Sample

Difference

Baseline Test Scores

 

 

 

 

 

 

Average Reading Percentile

40.2

36.4

3.9**

Percent missing

25

26

 

 

 

 

Average Mathematics Percentile

46.7

43.0

3.7**

Percent missing

25

24

 

 

 

Percent in Special Education

17

15

1

Percent missing

24

22

 

 

 

Percent, by Race

 

 

 

African American

93

88

5**

Hispanic

6

10

-5**

Other race2

2

2

0

Percent missing

0

0

 

 

 

Percent, by Gender

 

 

 

Female

51

51

0

Percent missing

0

0

 

 

 

Percent Participating in Free/Reduced-price Lunch Program

100

100

0

Percent missing

0

0

 

 

 

 

 

Sample size

894

44,740

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
1. Test-score results are in terms of National Percentile Ranks, with 50 as the median score.
2. "Other race" includes students who were identified as white, Asian, American Indian, or Alaska Native.
NOTE: Detail may not sum to totals because of rounding. Applicant sample includes all eligible applicants identified in the DCPS database that were participating in the free and reduced price lunch program.
SOURCE: Accountability testing database for District of Columbia public and charter schools, DCPS Office of Communications and Public Information.

Table 3. Percent of Students Entering Various Grades, Eligible Applicants and Eligible Non-Applicants: Spring 2004

Grade Level1

Applicants

Non-Applicants

Difference

Kindergarten -1

20

11

9**

2-3

18

17

1

4-5

18

17

1

6-7

19

18

1

8-9

15

15

0

10-12

10

21

-11*

Percent missing

0

0

  

 

 

 

Sample size

1,848

71,434

 

*Significantly different from the average of the other subgroups at the 95 percent confidence level.
**Significantly different from the average of the other subgroups at the 99 percent confidence level.
1. Grade levels were combined because cell sizes were too small to report separately by grade.

SOURCES: Eligible Applicant Database; DCPS Accountability Testing Database.

Table 4 (click on the table for a larger view)

table

Table 5.        Student’s Educational Characteristics, Users and Nonusers: Spring 2004

Characteristic

Users

Nonusers

Difference

Baseline Test Scores1

 

Average Reading Percentile

 

 

 

Grades K-5

49.12

40.30

8.82**

Grades 6-8

 39.82

38.06 

1.76 

Grades 9-12

30.00 

23.57 

6.43 

Percent missing

56

41

 

 

 

 

Average Mathematics Percentile

 

 

 

Grades K-5

53.47 

43.75 

9.72**

Grades 6-8

44.91 

42.26 

2.65 

Grades 9-12

40.30 

43.83 

-3.53 

Percent missing

56

40

 

 

 

Percent with a Learning or Physical Disability

 9

29 

-20**

Percent missing

6

8

 

 

 

 

Grade Level

 

 

 

K-1

27

14

13**

2-3

23

18

5*

4-5

22

22

0

6-7

15

22

-7**

8-9

9

15

-6*

10-12

 3

 10

 -7**

Percent missing

0

0

 

 

 

 

Sample size

1,027

339

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
Percentages may not sum to 100 due to rounding.
1. Test-score results are in terms of National Percentile Ranks, with 50 as the median score.
SOURCE: Eligible Applicant Database. Test-scores obtained from the accountability testing database for District of Columbia public and charter schools, DCPS Office of Communications and Public Information.

Table 6.  Family Background Information, Users and Non-users: Spring 2004

Characteristic 

Users

Nonusers

Difference

Percent of Students whose Family Income is

 

Less than $5,000

11

11

0

$5,000-$10,999

18

24

-6*

$11,000-$24,999

43

38

5

$25,000-$39,999

25

24

1

$40,000 or more

3

4

0

 

 

 

 

Average Family Income

$18,652.49

$17,840.70

$811.79

Percent missing

0

0

 

 

 

 

Percent of Students whose Mothers Attained the Following Level of Education

 

No high school diploma

11

19

-7**

GED

6

6

0

High school diploma

25

25

0

Some college

35

30

5

Vocational (2 yr) degree or certificate

15

14

1

College graduate

7

4

3

Graduate degree

1

2

-2*

Percent missing

11

19

 

 

 

 

Average Years of Mother’s Education

12.73

12.50

.23*

Percent missing

11

19

 

 

 

 

Percent of Students with Mothers Employed Full Time

46

44

2

Percent missing

13

23

 

 

 

 

 

Percent of Students with Married Mothers

19

16

2

Percent missing

10

18

 

 

 

 

 

Average Years at Current Residence

6.11

7.69

-1.58**

Percent missing

2

3

 

 

 

 

Sample size

1,027

339

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
SOURCE: Eligible Applicant Database.

Table 7.        Student’s Ethnicity and Gender, Users and Non-users: Spring 2004

Characteristic

Users

Nonusers

Difference

Percent, by Race

 

 

African American

94

96

-2

Other race1

6

4

2

Percent missing

6

6

 

Percent Hispanic (any race)
Percent missing

 

7
4

 

7
7

 

1

Percent, by Language Most Spoken in Home

 

 

 

English

93

96

-3

Other language2

7

4

3

Percent missing

4

4

 

 

 

 

Percent, by Gender

 

 

 

Female

53

45

9**

Percent missing

1

2

 

 

 

Sample size

1,027

339

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.

1. "Other race" includes respondents who were identified as white, Asian, American Indian, Alaska Native, Pacific Islander, or multiracial. Respondent classified as "multiracial" if more than one race category selected.
2."Other language" includes all other languages including Spanish and Amharic.
NOTE: Detail may not sum to total because of rounding.
SOURCE: Eligible Applicant Database.

Table 8. Parental Involvement with Child’s Education, Users and Non-users: Spring 2004

Characteristic

Users

Nonusers

Difference

Percent of Parents who Participated in the Following Activities with Their Child in the Past Month

 

 

 

Discussed experiences at school

99

98

0

Helped with mathematics or reading not related to homework

94

90

4**

Worked on homework

97

92

5**

Worked on a school project

86

85

1

Attended school activities with child

90

85

5*

Average percent missing

8

6

 

 

 

Sample size

1,027

339

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
SOURCE: Eligible Applicant Database.

Table 9.        Problems at Child’s School, Users and Nonusers: Spring 2004

Characteristic

Users

Nonusers

Difference

Percent of Parents who Believe the Following Problems at School are Serious

 

Fighting

50

46

4

Tardiness

48

48

1

Destruction of property

35

35

1

Truancy

34

39

-4

Cheating

26

22

4

Guns or other weapons

20

15

4

Drug distribution

16

10

6*

Drug and alcohol use

14

10

5*

Average percent missing

10

9

 

 

 

Sample size

1,027

339

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
SOURCE: Eligible Applicant Database.

Table 10.      School Facilities and Homework, Users and Non-users: Spring 2004

Characteristic 

Users

Nonusers

Difference

Percent of Parents Reporting that Students have the Following Resources at Their School

 

Special programs for non-English speakers

37

54

-17**

Nurse’s office

80

89

-9**

Prepared Lunches

70

77

-7*

Cafeteria

81

90

-9**

Special programs for advanced learners

33

46

-12**

Arts program

67

72

-4

Special education programs

57

73

-15**

Computer lab

75

79

-5

Gym

58

68

-10**

Child counselors

78

84

-7*

Library

80

86

-6*

After-school program

82

78

4

Music program

69

72

-4

Individual tutors

37

45

-8*

 

 

 

 

Average Hours of Daily Homework

.93

.96

-.04

Percent missing

9

7

 

 

 

Sample size

1,027

339

 

*Statistically significant at the 95 percent confidence level.
**Statistically significant at the 99 percent confidence level.
SOURCE: Eligible Applicant Database.

 

 

 

Department of Education Reform
University of Arkansas
201 Graduate Education Building
Fayetteville, AR 72701
http://www.uark.edu/ua/der
 
Ph: 479/575-3172
Fax: 479/575-3196
edreform@uark.edu