(Guest post by Greg Forster)
On Friday, the Friedman Foundation released my new report, “A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public Schools.” It goes over all the available empirical evidence on . . . well, on how vouchers affect public schools.
Here’s the supercool graphic:
Worth a thousand words, isn’t it? I mean, at what point are we allowed to say that people are either lying, or have been hoodwinked by other people’s lies, when they say that the research doesn’t support a positive impact from vouchers on public schools?
There’s always room for more research. What would we all do with our time if there weren’t? But on the question of what the research we now have says, the verdict is not in dispute.
Here’s the executive summary of the report:
This report collects the results of all available empirical studies on how vouchers affect academic achievement in public schools. Contrary to the widespread claim that vouchers hurt public schools, it finds that the empirical evidence consistently supports the conclusion that vouchers improve public schools. No empirical study has ever found that vouchers had a negative impact on public schools.
There are a variety of explanations for why vouchers might improve public schools, the most important being that competition from vouchers introduces healthy incentives for public schools to improve.
The report also considers several alternative explanations, besides the vouchers themselves, that might explain why public schools improve where vouchers are offered to their students. It concludes that none of these alternatives is consistent with the available evidence. Where these claims have been directly tested, the evidence has not borne them out. The only consistent explanation that accounts for all the data is that vouchers improve public schools.
Key findings include:
- A total of 17 empirical studies have examined how vouchers affect academic achievement in public schools. Of these studies, 16 find that vouchers improved public schools and one finds no visible impact. No empirical studies find that vouchers harm public schools.
- Vouchers can have a significant positive impact on public schools without necessarily producing visible changes in the overall performance of a large city’s schools. The overall performance of a large school system is subject to countless different influences, and only careful study using sound scientific methods can isolate the impact of vouchers from all other factors so it can be accurately measured. Thus, the absence of dramatic “miracle” results in cities with voucher programs has no bearing on the question of whether vouchers have improved public schools; only scientific analysis can answer that question.
- Every empirical study ever conducted in Milwaukee, Florida, Ohio, Texas, Maine and Vermont finds that voucher programs in those places improved public schools.
- The single study conducted in Washington D.C. is the only study that found no visible impact from vouchers. This is not surprising, since the D.C. voucher program is the only one designed to shield public schools from the impact of competition. Thus, the D.C. study does not detract from the research consensus in favor of a positive effect from voucher competition.
- Alternative explanations such as “stigma effect” and “regression to the mean” do not account for the positive effects identified in these studies. When these alternative explanations have been evaluated empirically, the evidence has not supported them.
Jay P. Greene's Blog 
This was kind of an interesting report, but then I’ve always been fascinated by skewed research methods and blatant marketing and propaganda techniques. Full disclosure: I’m a public school teacher. Does that make me biased? Yes it does and I admit it, but it also makes me enormously better informed than a few bought-and-paid-for Ph.Ds checking their emails and downloading state test score data. Researchers like Yale graduate Dr Forster, who can’t be more than 25 years old by his photo, simply haven’t the wisdom or experience required to really understand public education, and in my opinion have only a gossamer-thin credibility on the topic, not to mention a crystal clear prejudice. I really don’t have the energy nor the inclination to point out every flaw in this “study”, but I’ll start by saying I don’t think I’ve ever seen a more classic example of someone with an agenda going out and conducting “research” that has as its sole purpose the furthering of that agenda. Dr Forster you should be ashamed to purport that this has even the slightest pretense of objectivity. Paid for by the Friedman Foundation for School Choice? That’s like Exxon paying for a study of environmental damage due to oil spills.
Next, how can any serious, broadly applicable conclusions be drawn from only 17 studies done over just a few years? We don’t even get to look at the studies themselves, just your highly personal interpretation of some of the results. Your admonition about only considering rigrous scientific methods was an insult, and the height of hypocrisy. I hope you at least had a good laugh at it. At the end of this piece should have been the disclaimer: “More research must be done before any real conclusions can be reached.”
On to a few specifics. The claim that students leaving the public school system leaves more money to be spent on the remaining students is absurd! The overwhelming majority of school districts in the US get Average Daily Attendance money from the government as their source of funding, and it’s pretty simple: fewer students = less money. A variation on this formula applies to categorical monies also, but not private donations. Only in a tiny fraction of school districts does the property tax base supercede ADA, the so-called “Basic Aid” districts. In only these few cases can it be said that vouchers could possibly help public schools monetarily.
Next, upon what basis is it said (and repeated ad nauseum) that schools are improved by the proximity of a voucher program? Answer: test scores! When it comes to standardized test scores of American teenagers, never has so much been made of so little. A very large percentage of students do not take these tests seriously, and why should they? Students have zero incentive to prepare for them and apply themselves. After all, they’ve been taking them since they were in second grade and they’ve never once recieved anything other than a pat on the back for doing well, and absolutely no consequence for a poor performance. It’s just a wasted day (or week) at school for most of them. More importantly, standardized test score data is significantly overblown as a measure of how well schools teach. With every year’s passing under NCLB, schools have come closer and closer to “teaching to the test”, just to survive. It has forced schools to adjust their curriculums to the tests, which is not always a bad thing, and waste a lot of energy “incentivizing” the student population to take the tests with some degree of diligence.
Even if they did take them seriously, and some do from time to time, it would be impossible to ascribe to a single cause the variations in scores from one year to the next. Student populations change, accounting methods change, teaching methods change, etc. None of these considerations, and there are many more, were addressed in this report as possible causes for changes in test score data.
The author’s contention is mainly that the threat of a voucher program causes schools to try harder, to “compete” for those students who might leave to a better private school if given a voucher, which only begs the unanswered question of exacly *how* does a school try harder? Exactly HOW does the presence of a nearby voucher program improve a school? He doesn’t say. Somehow if a school feels threatened by vouchers it will improve its test scores, but we never learn how it is accomplished. We are left to assume that when schools are threatened they will improve, but he dismisses the idea that a stigma associated with low scores could produce improvement. This is contradictory, and only possible by his facile and self-serving defining of the “stigma” effect so narrowly that it only applies to the Texas “A+” schools. Baloney! Every school wants to have good test scores to brag about, every administrator wants to be the one that “turned things around” for their school or district, and there will always be a gullible segment of the parent population that believes higher test scores mean better education, or that their child shouldn’t mix with “those” students. Having low test scores published in the newspaper *always* creates a stigma that schools would rather avoid, whether vouchers are available or not.
Finally, nowhere in the report was a consideration of the vast improvements made in the last few years in schools’ ability to analyze data and respond with improved teaching techniques. Perhaps better teaching improves test scores? Maybe schools are teaching the standards that are being tested once the state figures out what those standards are? Gee, might textbook publishers be aligning their materials to standardized test standards? Hmmm. Nah, it must be vouchers. Come on! Your “Alternative theories” was three little straw men in a row and a complete joke.
Let’s face it: vouchers will never be a viable alternative for parents on a large scale, and so they’re not really much of a threat to public education. They are simply a way for a) conservative, anti-government types to withdraw from the accountability in the PS system and not have to pay for it, and b) extremely religious parents to sheild their children from the rampant secularism of the real world without having to pay for the privilege. In the first place, it is a myth that private schools provide a better education, per se. Some do, some don’t, and we’ll probably never know the difference because there’s no way to really compare them. It’s not just apples & oranges, more like an entire jungle of various fruits. Neither curriculums nor teachers have to meet any kind of widely held standards, even though many do take state tests. Secondly, school vouchers cannot pay for private schools because the schools are too expensive.
A better alternative to vouchers, one that you SHOULD be advocating for, is for more districts to relax school boundaries and allow more open enrollment within a district. That way takes advantage of the “competition” effect, but still provides free education to the public. Calling them “government schools” is a gratuitous insult, and implies some kind of indoctrination is happening. People who believe that just don’t know what they’re talking about, and I think they really don’t even want to know.
Thanks for saying I look so young! I’m actually 35. The photo’s about five years old at this point; I suppose I should get a new one.
Normally I make it a policy not to respond to anyone who uses ad hominem attacks (which are a violation of this website’s use policy), but there are some important errors of fact here that need to be corrected. In order to minimize the collatoral troll-feeding, I will mostly confine myself to addressing the errors of fact – tempted as I am to point out Mr. Taylor’s even more serious errors of logic, which I suppose I must trust the reader to detect without my assistance.
Mr. Taylor attributes all the research in this report to me, but the report reviews studies conducted by a wide variety of researchers, most of whom are university professors rather than employees of advocacy organizations. (I would observe that they’re professors at some pretty prestigious universities, but it appears that Mr. Taylor considers an Ivy League affiliation to be a bug rather than a feature.) Some of those researchers (e.g. Carnoy and his team) are actively hostile to vouchers and have done their best to downplay the significance of their own data. Nonetheless, their own empirical findings support vouchers even though they don’t.
Of course, who employs them and whether they support vouchers is irrelevant to the question of whether their research uses proper scientific methods. On the subject of methodology Mr. Taylor offers much heat but no light.
Mr. Taylor complains that he cannot look at the studies themselves to judge their methodology. I don’t know what’s stopping him; they’re all easily obtained and the necessary citations are provided in the report.
Most voucher programs reduce school costs more than they reduce school revenue, leaving schools with more, not fewer, dollars per student. See for example here.
Mr. Taylor’s arguments against testing (that the scores are meaningless, that “teaching to the test” is at odds with real learning, etc.) would take us into a long discussion that has been had so many times before in so many forums that it would be gratuitous troll-feeding to rehash it all here.
The studies in the report do take into account the problem of isolating the impact of vouchers from other factors that affect changes in test scores. This is discussed in the report under the heading “Why Scientific Methods Matter.”
Mr. Taylor writes that I “assume” vouchers improve public schools without having an explanation for how this happens. It’s true that my report does not look at the body of research on what happens inside schools when they’re threatened with vouchers; it is addressed to the narrower question of whether or not vouchers do in fact bring about improvements in outcomes. That they do is not an assumption, but a consistent finding of the empirical research.
Mr. Taylor thinks it is “contradictory” to say that voucher competition improves schools but “stigma” does not. But the report does not say that stigma doesn’t improve schools; while most of the research reviewed in the report does not find a positive effect from stigma, some of it does. The report does not reach a conclusion about whether stigma improves schools; it only concludes that voucher competition has a positive impact independent of any stigma effect that arises merely from labeling schools with an F. (Mr. Taylor appears to be confused about what is meant by “stigma” in the context of the debate over vouchers. The more generalized “stigma” to which he refers is not denied by the report, as he thinks, but is affirmed by it. The “stigma hypothesis” discussed in the report, as the report itself explains, is the hypothesis that the A+ program would have produced the same improvements even without the voucher component.)
Mr. Taylor writes “Texas” when he means “Florida.”
If, as Mr. Taylor thinks, public schools are getting better and/or more aligned to state standards over time in various ways, that doesn’t explain why studies that compare sets of public schools to one another at the same times consistently find a positive effect from vouchers. And Mr. Taylor’s hypothesis works against him in the case of the Florida study that compares results in different years, where school improvements got smaller after the voucher component of the program was removed.
There are plenty of ways to compare public and private schools. The best way is to use random assignment studies that compare the results of students who either won or lost a random lottery to get the opportunity to use vouchers. The kids are exactly the same in the treatment and control groups, except for random chance. There have been a whole bunch of studies that use this design. Guess what they find?
Private schools are actually much cheaper than public schools. The average private school tuition is something like $6,000, whereas it costs over $10,000 to educate a student in public school.
Mr. Taylor-
You have access to the internet- feel free to create your own list of studies from a variety of academics and right and left of center research organizations. You might have a chance to produce something worthy of discussion, rather than an online temper tantrum.
I’ll give my carefully reasoned comment — Control G!
Mr. Taylor confirms my belief that the prevalent research methodology in schools of education is polemics.
Yes, Mr. Taylor made some mistakes…but if you accept his assertions as factual, Greg, you get to be 25 again!
Too bad I missed this debate – as a former High School teacher who supports vouchers, tax credits, charter schools and the termination of government school monopoly power…I could have thrown in a whole bunch of anecdotal evidence myself. 😛
I have been an inner-city secondary teacher for 33 years. I teach Free Market Economics and Constitutional Government. Mr. Taylor is a perfect example of the cause of the problem in the public schools: teachers who cannot think. He discloses, in so many words, “I am too stupid and too lazy to read the research. I just disagree with it.”
I am embarrassed to be his colleague.
Mr. Taylor. What would the KKK do differently if their goal was to destroy the black community? This is what the teachers unions have done.
Your obfuscating drivel about “changing populations” and the myriad of excuses are still just excuses. Mr. Taylor, excuses are for losers!
The democrat party has ruled over the ghettos of America for fifty years. If I lived where I teach, I would vote for someone else.
Look at the black churches that would love to start their own private schools. In LA, we have Crenshaw High that lost their accreditation. Then, because there was nowhere to send the kids, the state gave it back to them, even though they had not improved. In the same neighborhood is Crenshaw Christian Center, with their school, Price Academy. We also have Serra High, Verbum Dei in Compton, Loyola, and Marcus Garvey.
What is your opinion of Obama sending his kids to private school while he takes away the vouchers that allowed poor kids to go to the same schools his daughters go to? Antonio Villaraigosa said, “I would never sacrifice my children by sending them to LAUSD.” You are disgusting!
Your excuse that inner city kids just can’t learn is repudiated by these examples. Keep your safe union job protected by the democrat party, but remember, you are screwing innocent children.
[…] can tell.” According to the evidence, Yglesias’ breezy, offhand accusation is demonstrably wrong. Increased competition from private schools actually improves public school […]
[…] this successful teacher followed union rules, his students would most likely not have prospered. Voucher programs are a win-win for these students yet teacher unions vehemently oppose them for obvious reasons. And union mandates tie […]
[…] this successful teacher followed union rules, his students would most likely not have prospered. Voucher programs are a win-win for these students yet teacher unions vehemently oppose them for obvious reasons. And union mandates tie […]
Just wondering why Mr. Barber did not say choice was a factor in successful systems when asked at the DC
Summit. He said there was not enough research to support it. I was there. Are you saying he was wrong?
[…] Their prime directive is that the competition of the market always produces progress. It is also in the words of sometime Caire collaborator Jay P. Green of the Arkansas Project a win/win assertion for them. Had the voucher schools performed better this would be evidence of […]