To measure the extent to which schools are properly informing parents about NCLB school choice we sent emails to choice-eligible schools requesting information. The emails were made ambiguous in their origin and purpose so that schools could believe that they came from parents. What we found was widespread lack of cooperation. The vast majority of schools failed to reply at all. Those that did reply were mostly concerned with who we were and why we were asking. Only a tiny minority of schools provided us with the information requested. It is clear that schools are failing to properly inform parents about NCLB school choice. It is also obvious that little is being done to monitor or enforce compliance with these provisions of NCLB.
There can be no accountability without meaningful consequences. Unless educators and students believe it is reasonable to expect that they will be rewarded for desirable behavior and punished for undesirable behavior, we shouldn’t expect substantial changes in their behavior. Previous work has raised serious questions about whether the federal No Child Left Behind (NCLB) accountability regime imposes meaningful consequences.
In particular, researchers have questioned whether the provisions contained in NCLB are sufficiently strong and whether those provisions are being properly implemented.
An essential component of the accountability structure of NCLB is whether parents are properly informed of their educational options when public schools have to make those options available. Under NCLB, public schools that fail to make adequate yearly progress (AYP) two years in a row must offer families the option to send students to a better performing public school. Offering families at chronically failing schools the option to transfer to a different public school is one of the consequences schools face under NCLB for poor performance. Requiring that choices be made available not only offers transferring students the possibility of a more effective school, it also threatens underperforming schools with the loss of students and the per-pupil funding those students bring with them. Having to compete to retain students and the revenues they generate might inspire improvements in chronically failing public schools.
Imposing this sanction on schools under NCLB is dependent upon their cooperation. Schools must tell parents that they are eligible for these options, but there’s the rub: schools have little desire to help parents pull students from their school. Schools would rather not sanction themselves. Unless the cooperation of schools in telling parents of their options is strictly monitored and enforced, schools may undermine the effectiveness of this sanction by failing to properly inform parents of their options.
To measure the extent to which schools are properly informing parents about NCLB school choice we sent emails to choice-eligible schools requesting information. The emails were made ambiguous in their origin and purpose so that schools could believe that they came from parents. Our goal was to assess how cooperative schools would be in providing what they might believe to be parents with information about their educational options under NCLB. What we found was widespread lack of cooperation. The vast majority of schools failed to reply at all. Those that did reply were mostly concerned with who we were and why we were asking. And even when told that we were studying the ease of obtaining information on options under NCLB as part of a research project, only a tiny minority of schools that replied at all provided us with the information requested.
It is clear that schools are failing to properly inform parents about NCLB school choice. It is also obvious that little is being done to monitor or enforce compliance with these provisions of NCLB. While passage of NCLB marked an important step in developing an effective accountability regime, we have a long way to go to produce meaningful accountability. Because the consequences for schools and students under NCLB are weak and further undermined by widespread non-compliance, NCLB provides relatively little accountability.
FEW CHOOSE, BUT WHY?
All close observers of NCLB agree that participation in the choice provisions of the law has been very low. According to a 2004 General Accounting Office (GAO) report, only about 1 percent of students eligible for choice have transferred to another school. A more recent Center on Education Policy report found that “the percentage of eligible students actually taking advantage of NCLB choice has remained very low, just 1.6% of those eligible in 2005-6.” Why are so few students leaving chronically underperforming schools? While the GAO report concedes that “little is known about the factors that affect transfer decisions,” a number of possible explanations have been offered.
First, the GAO report noted that seven of the eight districts they examined did not receive final results of school performance from the state in time to notify parents of their options effectively. NCLB requires that parents be informed of their eligibility to transfer no later than the first day of classes of the following academic year. Yet only one of the districts examined by GAO could manage to have the information finalized and disseminated to parents in the roughly five month window between the administration of tests and the start of the next school year. The GAO dryly reports the challenge posed by this “short timeframe” without critically examining the necessity of the cumbersome bureaucratic procedures that make the process take at least five months. Does it really take five months to score standardized tests, analyze the results, and inform parents of their eligibility for choice options?
Second, the GAO report suggested that the “lack of sufficient capacity” limited the number of students participating in NCLB school choice. Numerous media reports echoed this explanation. Los Angeles Superintendent, Roy Romer, told National Public Radio: “"We're 160,000 seats short. Where do you transfer to?" An ABC News report in Chicago told the same story: “Children who want to transfer to better schools will find there is not enough room for them to do so… [T]he more than 200,000 students eligible to transfer to better schools under no child left behind, are competing for only 457 spaces.”
While there is no doubt that in many large cities there were few seats made available in schools that were not already deemed as “in need of improvement,” this is not a fully satisfactory explanation for low participation rates in the choice option. According to the GAO’s description of the NCLB Act, “Districts are not required to give parents their first choice among the transfer options provided, but may not deny transfer requests based on lack of physical capacity, such as lack of space within a building or classroom, according to federal regulations.”
This is puzzling. Districts may not deny choice because of lack of capacity but the lack of capacity is cited as a primary explanation for the lack of choice participants. Perhaps even more puzzling is this: how, exactly, did federal officials expect schools and districts to offer choices regardless of the availability of physical capacity? In the absence of clear federal guidance on how to handle limited physical capacity, districts have essentially been allowed to resolve this issue in whatever way they choose. And quite often their decision involves failing to make choices available -- even when districts could immediately take steps to expand capacity.
At the very least, almost all districts should have the ability to offer sufficient choice options if they work with neighboring districts to utilize any spare capacity those districts may have. In fact, districts are required under NCLB to do this, but apparently none of them have. As the GAO report states: “Where districts lack other schools to offer as transfer options, districts are required, to the extent practicable, to make transfer agreements with neighboring districts, but we are not aware of any locations where such interdistrict transfers have occurred.” The fact that districts have not bothered to pursue this avenue for providing transfer options even though it is required of them and would largely solve their capacity constraints suggests that the limiting factor is not as much the lack of physical capacity as the lack of desire to assist parents in leaving their schools.
Districts also “have the option of offering supplemental educational services [SES] to students” if they lack physical capacity for transfers. One of the eight districts visited by GAO researchers had done precisely that. It was a rural district in Mississippi and its only middle school was identified as in need of improvement. Students at that district had to be offered choice options, but none were available in the district, since there were no other schools of the same grade levels, and neighboring district schools would involve traveling great distances. So they offered students tutoring options instead of transfer options. And a later GAO report on supplemental services noted that four districts in Virginia were participating in a US Department of Education pilot program that allowed them “to offer SES instead of school choice in schools that have missed state performance goals for 2 years and are in their first year of needs improvement.” It is clear that districts lacking the physical capacity to offer sufficient choices could at least offer supplemental services, but it is also clear that only a small number of districts bothered to do so. Again, schools appear to have little interest in assisting the accountability regime in imposing sanctions on themselves.
A third explanation for why so few students exercise school choice under NCLB is that there is very little demand from families for those options. A survey of school officials by the Center on Education Policy draws this conclusion: “Use of the NCLB choice option depends on parents’ decisions as well as schools’ actions. In our case studies, many district and school officials speculated that parents and students may not want to transfer for a variety of reasons, including satisfaction with current schools.” Lisa Snell at the Reason Foundation confirms that this notion about the lack of parental demand for choice is widely held: “Teachers unions, school administrators, and journalists have argued that the low transfer rates prove parents do not want more choices and that they prefer their local schools.”
Some observers have suggested that parental demand for alternatives, in general, is low and has already largely been sated by existing choice programs. The Harvard Civil Rights Project, for example, issued a report making this argument, noting: “Nationwide, student enrollment in public school choice programs, charter schools, and the use of vouchers to attend a private school is low.”  The report goes on to suggest that parental demand for transfers may be low because “the available options were not appealing. For example, among the 90 receiving schools in Chicago, the majority were ‘only marginally better than the failing school,’ and some of Chicago’s receiving schools were already on a state list for ‘chronically poor performance.’”
But all three of these explanations for low participation in NCLB choice – lack of timely notification, lack of capacity, and lack of demand – presuppose that parents are being informed at all that they have choice options. If parents are not told about NCLB school choice they can’t very easily participate, even if there were space or they wanted to pursue alternatives. This gives rise to a fourth and more fundamental explanation: perhaps families are failing to participate in NCLB school choice because they are not being properly informed about their eligibility or their options.
There is good reason to suspect that parents have been poorly informed of their options. In Buffalo a survey by Brighter Choice School Project found that 75% of parents were not aware that they were eligible for NCLB school choice. In Boston William Howell of Harvard University found, “Seventy-one percent of parents with children in underperforming schools do not know that the schools are underperforming, and so do not know that they qualify for choice.”  Of course, it is possible that parents might report that they didn’t know about their eligibility for choice even if they had been properly informed of their options. NCLB requires that all parents in choice-eligible schools receive a letter explaining their options, but those letters may not have been sent or may not be an effective method of communicating with parents. The fact that parents are unaware of their options suggests that schools are not properly informing them of those options, but it is always possible that parents remain unaware despite the best efforts of schools.
To discover whether schools in fact are making their best effort to properly inform parents we devised a method of measuring this directly. We sent emails to a large number of schools that had to make NCLB school choice available to their students, requesting information on options under NCLB. The email was ambiguous in its origin and purpose so that school officials could believe that it was coming from a parent at their school. We then tracked whether we received a response to our request for information and whether that response addressed the issue to gauge the extent to which schools are properly informing parents.
Technically, it is the legal responsibility of the school district, not the school, to inform parents of their options, but schools are simply the local extension of school districts. Presumably districts would establish procedures for schools on how to handle requests for information. In addition, schools are the most common level at which parents could be expected to seek information, so districts wishing to inform parents about NCLB school choice would be sure to prepare schools for fulfilling this responsibility. While most parents would not seek information from their school by email, it is the preferred form of communication of some parents, particularly those with work schedules that do not easily allow them to visit or call schools during work-hours. In general, the responsiveness of choice-eligible schools to email requests for information on options under NCLB is a reasonable test of compliance with at least the spirit of their responsibilities.
First, we had to identify the universe of schools whose students were eligible for NCLB school choice. We relied upon the U.S. Department of Education’s 2005 Annual Report to Congress on schools receiving support under Title I of the Elementary and Secondary Schools Act (ESEA), which provided the most recent comprehensive list of choice eligible schools. But because this list was as of 2003-4 and we conducted our study during 2005-6, it is possible that some of the schools had improved so that their students were no longer eligible for choice. To end choice-eligibility, schools need to make annual yearly progress two years in a row. We eliminated from our sample all schools that were choice eligible as of 2003-4 but made AYP that year. If those schools also made AYP during 2004-5, they would no longer be choice-eligible when we contacted them during 2005-6. Removing those schools from our sample ensured that all schools in our sample would still be required to offer families choices when we contacted them. We also removed all schools from Louisiana from our sample. Given Hurricane Katrina, any failure on their part to provide proper information on choices could be attributed to the special circumstances caused by the storm. In the end we were left with a potential sample of 5,409 schools.
We then searched state, district, and school web sites to find email addresses for contacts at those 5,409 schools. When more than one email address was available, we chose the one for the highest ranking official at the school; if an official was named as one who deals specifically with No Child Left Behind or Title I, that e-mail address was used. We were able to find email addresses for 2,887 choice-eligible schools, more than half of our potential sample.
Next we had to develop an email message that could test the willingness of school officials to provide information on NCLB school choice. Working with the University of Arkansas’ Institutional Review Board for research involving human subjects, we sought to craft an email message that might allow its recipients to believe that it came from a parent at their school without explicitly saying so. Falsely claiming to be a parent would have been logistically complicated and involved a level of deception with which we were not comfortable. Instead we settled on a very brief email message that might allow school officials to believe that it came from a parent at their school. The text was: “I’ve heard something about there being options for students under No Child Left Behind. Could you tell me about what those options might be? Thanks.” Of course, such a brief and ambiguous message might confuse school officials, decreasing their responsiveness, but we believed that it was sufficient to inspire schools interested in providing information to do so.
We then created email accounts with free email services so that we could send our messages without identifying ourselves as researchers at the University of Arkansas. Our email addresses began with “javelina@” in a veiled reference to the University of Arkansas’ beloved mascot, the Razorback, but recipients would have been unlikely to make the connection. All they would know is that someone emailed their choice-eligible school asking for information about “options for students under No Child Left Behind” without knowing precisely who was emailing or why. They certainly would have reason to believe that a parent at their school might want to know about options given that they were required to provide NCLB school choice to all of their students.
We began sending emails on April 5, 2006 and completed sending by April 12. We gave schools until May 12, 2006 to respond to our request for information. On May 22, 2006 we sent a “debriefing” email to all subjects explaining that our earlier email was part of a study we were conducting. We assured all recipients that the identities of individuals, schools, and districts would be kept strictly confidential and that only aggregated results would be reported. According to our research protocol, anyone responding to our initial email who inquired about who we were or why we were contacting them would receive the same “debriefing” email that was later sent to all subjects.
Of the 2,887 emails we sent 439 “bounced back” as either having an invalid email address or because delivery was blocked. This left us with a sample of 2,448 choice-eligible schools that we believe received an email requesting information on options for students under NCLB. A summary of our results can be found in Table 1.
We received responses of some kind from 363 of those 2,448 schools, meaning that more than 85% of schools did not respond at all to a request for information about options. As can be seen in Table 2, the lack of response was fairly uniform across the states. In no state did more than 40% of choice eligible schools with valid email addresses provide any reply. In 13 states no choice eligible schools responded.
Roughly half of those schools that did reply (181) only asked about who we were and why we were contacting them. These 181 schools did not provide any information even after receiving our “debriefing” email explaining that our request was part of a research project. Some of these replies had a more hostile tone. For example, one school official’s entire reply was: “Who are you? I don't recognize this e-mail address.” Another simply wrote, “???” Yet another wrote, “I am confused. Do I know you?”
Some replies were more solicitous. For example, one school official said, “I'll be happy to respond to you if you will confirm who you are. If you are a parent or family of a [X] School student, I would appreciate knowing your specific needs or concerns as it relates to services provided by our school.” Another wrote, “I have no problem answering your question however, please inform me as to who you are.” Whether they responded courteously or not, none of these school officials provided the requested information.
Another 29 schools provided irrelevant or non-responsive responses that did not provide any information about NCLB school choice. The non-responsive responses came in a few different kinds. Some made unhelpful suggestions, like “The question concerning the ‘options’ for the NCLB is a rather vast area. I would suggest that you go to ‘google’ on the website and type in the No Child Left Behind…I am sure you will be able to find out just what you are looking for…be prepared, there is a lot of material. Good luck…” This reply is no more responsive to our request for information than if a doctor replied to a patient’s question about heart disease by saying that there are many books in the library on that topic. The patient should browse the library shelves to get an answer, but be prepared – there are a lot of books there! Another kind of non-responsive response involved the school official editorializing about NCLB. One school official wrote, “[NCLB’s] reality has been to provide little if anything for students. It has mostly added to our bureaucracy and provided us with nothing but more paper work to do.” And yet another kind of non-responsive response took issue with the study rather than providing the requested information. For example, one school official wrote: “Since this study did not go any further then [sic] the original email, that did not identified [sic] the sender, I consider it unreliable in it’s [sic] data, with a very big margin of error.”
An additional 13 schools provided information in response to our email, but we believe the information they provided was inaccurate. Those schools claimed that there were no options available at all under NCLB or that the requirement to provide NCLB school choice did not apply to their school.
Only 140 of the 2,448 schools (5.7%) we contacted actually offered information in response to our request. 53 of these 140 schools provided a phone number to contact to get the information but did not provide the information in their email response. We did not call those phone numbers but are willing to give them the benefit of the doubt that they all would have provided us with accurate information if we had called them. The other 87 schools provided information in response to our request in their email text rather than simply offering to provide information by phone.
The fact that 87 schools provided informative email responses should give us confidence that it was possible for school officials to understand and productively respond to our request for information. For example, one school official wrote, “Students who are enrolled at schools identified as in School Improvement Status have the option of applying to attend another school. The schools that [our] students would be able to apply to attend under the NCLB legislation are: [X] and [Y].” Other school officials sent us links to web sites that had detailed and clear information about choice options under NCLB. Developing a web site with clear and current information that would answer parents’ questions is an efficient method of handling requests for information. School officials could refer parents to web sites without having to burden themselves with drafting unique replies to frequently asked questions. But, again, schools did not frequently take advantage of this more efficient method of addressing requests for information.
While some school officials understood and were able to reply to our request for information, very few did. It is possible that so few schools responded because the email was vague and potentially confusing. To test this possibility we sent similar emails to eight businesses to see what kind of response we would get. Of course, with only eight subjects this is not a rigorous comparison group, but the difference in the frequency and tenor of the response is illustrative. Three of the businesses were companies that provide supplemental educational services. The other five businesses were large, consumer companies. In total, five of the eight businesses we contacted responded to our requests for information. All of those responses were detailed and accurate.
Our emails to these eight businesses were almost certainly more vague and confusing than the ones we sent to schools. With schools at least it would be reasonable for a parent to be puzzled about a new and changing program and ask for information. By making our emails to the businesses as close as possible to the text we used for schools, they come off as remarkably silly requests about things that reasonable people probably wouldn’t ask. For example, we wrote to Starbucks: “I’ve heard about your coffee. Could you tell me about options for getting it? Thanks.” This is very close to our email to schools (“I’ve heard something about there being options for students under No Child Left Behind. Could you tell me about what those options might be? Thanks.”), but one could hardly blame Starbucks for ignoring such a silly-sounding email.
The amazing thing is that unlike more than 94% of schools, most of the businesses we contacted were very cooperative in providing information, even in response to silly questions. Starbucks, for example, provided a link to a web site with the locations of their stores, an 800 number to call with any questions, and a link to a quality-control department to ensure that their answer was responsive to our question. Sylvan Learning Centers, a provider of supplemental education services, was similarly helpful in response to our odd and ungrammatical email: “I’ve heard about your tutoring services. Could you tell me about options for getting it? Thanks.”
It’s true that one of the three education companies and two of the other five businesses failed to respond, but most responded and did so very helpfully. It was clear that they had developed efficient systems for providing information about their services, including clear and informative web sites and courteous customer service personnel. No one asked who we were or why we wanted information. No one offered opinions about the foolishness of their company’s policies or our request for information.
The cheerful responses we received from companies to our odd email requests for information are reminiscent of the Lazlo Letters published by Don Novello. Novello, better known for his recurring role as Father Guido Sarducci on Saturday Night Live, wrote letters to politicians and major companies with odd questions and suggestions, always receiving polite responses. In one letter he wrote the Mr. Bubble Company complaining that the box said “Keep Dry.” How was he supposed to use it, he asked the Mr. Bubble Company, if he kept it dry? As usual in the Lazlo Letters, the company replied courteously and helpfully even if the question was inane. Why don’t schools similarly provide information courteously and helpfully, even when the requests are more reasonable?
There is an extensive business literature on responsiveness to customer needs and customer satisfaction, and their importance in retaining customers and increasing profits. It may be stating the obvious, but a Wharton School researcher summarizes the literature: “In searching for improvements in organizational performance, scholars advocate increased responsiveness to customers.” While such observations are commonplace in business research, they are noticeably absent from education. And the different emphasis placed by businesses and schools on responsiveness is evident in this study.
Of course, the reason why schools are not as likely to provide information is that it is not in their interest to do so. Businesses provide information because it helps them sell their products and make money. Even businesses in the education industry, such as Sylvan, are eager to provide information to attract clients and improve their profits.
Public schools do not face similar incentives. They are essentially guaranteed a stream of revenue regardless of how they treat current or potential students. And individuals who work in public education, such as teachers, administrators, and support staff, do not normally experience variation in their incomes related to the quality of their service.
Educators receive no external rewards for providing helpful information to parents asking about their options under NCLB. In fact, their school might lose students and revenue if they were to assist parents in pursuing a transfer. So it should come as no surprise that they are not particularly helpful in providing information about choice options. To be sure, schools have numerous good-willed people working in them who are eager to help children, which helps explain why schools are as successful as they are given all of their challenges. But even good-willed people do better when they are recognized and rewarded for their contribution. Without the proper incentives, schools will perform sub-par in a variety of ways, including helping parents with NCLB information.
The designers of NCLB were not unaware of this problem of incentives. They believed that they could obtain cooperation from schools in NCLB’s implementation, including its sanctions, by commanding that cooperation.
Commands alter incentives by offering rewards or sanctions for following or disobeying the commands. But command incentives only work if there are meaningful consequences attached to those commands. We are then back to the central problem with NCLB – it imposes few and weak consequences and little has been done to ensure that those consequences are implemented properly.
Secretary of Education Margaret Spellings acknowledges this challenge but it is not clear that she can or will produce a solution. She concedes that “only 17 percent of eligible students nationwide signed up for free tutoring. And of the four million students in the country eligible for school choice, only 38,000 students—less than one percent—actually transferred to a higher-performing school.” She admits that “more than half of school districts didn't even tell parents that their children were eligible for these options until after the school year had already started. That delay makes it virtually impossible for students to transfer schools without disrupting their education.” And she declares: “All of us—from the federal government to the states to districts to schools—must do a better job of reaching out to inform parents about their options.”
We must or … what? The emptiness of the threats behind NCLB helps explain its weak effects and weak implementation. Secretary Spellings has “made every state superintendent aware that she's poised to take action, including withholding funds from noncompliant districts and states, if need be.” But even after widespread non-compliance, documented by the GAO, reported in the media, and acknowledged by the Secretary herself, this threat has never been executed. Any student facing a flustered substitute teacher has seen this routine before.
Part of the challenge with proper implementation of NCLB is that command incentives can be unwieldy. The Secretary can threaten to withhold Title I funds from non-compliant states and districts, but they will howl that the denial of funds only hinders their ability to implement NCLB effectively and to serve disadvantaged students. In general, it is hard to get a government agency to do something by threatening to deprive it of the funds earmarked for that task. It is politically challenging to take the position that schools can be motivated to do a better job by sometimes losing revenue. In the logic of the political world, only the variation in money devoted to a task can explain variation in outcomes – not variation in the motivation to use that money effectively.
WHAT CAN BE DONE?
First, school systems should be encouraged, perhaps even required, to imitate information dissemination practices that are common in business. For example, states and school districts could develop web sites that answer common questions about NCLB and provide a complete and up-to-date list of options available to parents at each school in the state with details on hot to access those services. States and school districts could develop toll-free phone numbers with knowledgeable and helpful staff to answer parents’ questions. Local school officials could be trained to refer all questions to the web site or toll-free phone number so that all parents. Developing centralized sources of information can help ensure its accuracy while also reducing the burden on local school officials to handle parental questions.
Second, the dissemination of information by school systems has to be independently monitored. Someone outside of the school system – perhaps the General Accounting Office and/or similar organizations at the state level – should have the responsibility of checking the accuracy and responsiveness of the information about NCLB provided to parents by school systems. Again, this is a common practice in business and could easily be replicated in education. The independent auditors could contact schools, visit web sites, and call toll-free numbers to ensure the quality and accuracy of the information provided.
Third, and most importantly, there have to be meaningful sanctions for failing to provide helpful information about NCLB. The administration has to be willing to pay the political price of actually following through on its threats of withholding funding for non-compliance. Making an example of even a few districts could be very motivational to all of the others. Of course, given the political difficulty of imposing sanctions it may be necessary to have some automatic penalties written into law rather than rely entirely on the regulatory discretion of the U.S. Department of Education.
With these three steps, the information provided to parents about NCLB school choice could be significantly improved. Whether better informed parents participate in NCLB school choice at higher rates or not, at least their participation will be their choice rather than the choice of school officials who fail to inform them of their options. As it stands now, NCLB school choice is largely ineffectual and irrelevant because of the absence of accurate and helpful information for parents.
Of course we should appreciate how much progress has been made by NCLB despite the shortcomings detailed in this paper. NCLB has entrenched in the educational landscape the idea that we ought to measure student achievement every year in almost every grade. It has solidified the notion that results should be reported by sub-groups. And it has made headway with the proposal that meaningful rewards and sanctions should be attached to student outcomes. While it is important to acknowledge that the glass has at least some water in it, we can’t be called pessimists for observing that the glass is still mostly empty.