A Simple Game

I have always been interested in the group dynamics that exist in a classroom--I think it's a topic that is largely ignored by those currently researching education. We spend so much time thinking about the effect of inputs via teacher quality, school quality, student-specific demographics and not enough time thinking about how all of these things are interdependent. Economists, specifically, tend to think about a classroom as though it were a machine. We put various things in, they are combined to together and we get a "product" out of it. It is a production function--the focus is on "output". This "output" is almost always tied to test scores. If the "output" isn't explicitly test scores, such as studies that tie education to future income, that "output" is still a function of test scores, so it's essentially the same thing. Test scores are wonderful data for economists and other researchers because it's easy to collect, readily available, and usually standardized. In short, it's already in the correct language for economists. While ease of analysis can be an important characteristic of data, however, it does not necessarily mean that the data is useful. This is a major issue I have with current education research--I don't think the models and data being used are giving us the most useful information. Instead, I think our obsession with test scores stems from researchers' love of quantifiable information. Education is a complex system. Learning, in terms of the "product" our schools "produce", is not easily understood. But we very much want it to be so. As a result, we collect test score data and we squeeze, twist, and torture those numbers until they fit nicely into the models we have created. We then sing our own praises and claim that we have finally figured out how to "fix" our schools. We just need to fire bad teachers. We just need to hire better teachers. We just need to develop universal standards. We just need to spend more money. We just need to pay teachers more. We just need to use more technology. We just need to test students more. We just need to test students less. We just need to hold teachers accountable. We just need to hold parents accountable. We just need to hold communities accountable. We just need to open more charter schools. We just need to close charter schools. We just need to collect more data. It's all so damn simple, if you'd just look at what our model tells us. Except it's not simple. Let me use an analogy and a story to explain.

First, the analogy. In his book, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale professor James Scott argues that the desire to organize and operate society according to rigidly defined scientific laws leads to unintended consequences that undermine the success of the system. In the first part of the book, he describes the history and development of German forestry science:

The great simplification of the forest into a “one-commodity machine” was precisely the step that allowed German forestry science to become a rigorous technical and commercial discipline that could be codified and taught. A condition of its rigor was that it severely bracketed, or assumed to be constant, all variables except those bearing directly on the yield of the selected species and on the cost of growing and extracting them. As we shall see with urban planning, revolutionary theory, collectivization, and rural resettlement, a whole world lying “outside the brackets” returned to haunt this technical vision.

I believe education has been turned into a "one-commodity machine", for precisely the same reason and with the exact same effect. In an attempt to make this complex system into a "rigorous technical and commercial discipline", we have created these bracketed models. We have looked at these very complex variables and decided that for the sake of simplicity and universality of our models, we should just assume them to be constant. We shall, in effect, ignore them. And our justification for this treatment is that "it's not correlated with our other variables". We treat it like random noise--something that exists but doesn't really matter to our "output". And because we have are very good at math, we can do that. 

The effects on these German forests were disastrous:

It took about one century for them [the negative consequences ] to show up clearly. Many of the pure stands grew excellently in the first generation but already showed an amazing retrogression in the second generation...A new term, Waldsterben (forest death), entered the German vocabulary to describe the worst cases. An exceptionally complex process involving soil building, nutrient uptake , and symbiotic relations among fungi, insects, mammals, and flora—which were, and still are, not entirely understood— was apparently disrupted, with serious consequences. Most of these consequences can be traced to the radical simplicity of the scientific forest.

This is remarkably similar to what we see in education. Each time a new "solution" or "system" is implemented, we observe great results at the outset. We believe our efforts have finally paid off, that we have finally solved the problem. But then it becomes abundantly clear that those gains are but an illusion. In our simplified models we neglected those intricacies and interdependent relationships, and over time those variables come back to haunt us. Things like peer effects, teacher morale, and classroom culture are ignored. But they matter-- they matter a lot. And every time we neglect them in our model (either by omitting them or "controlling" for them), we end up with policy that sucks. And it's easy to tell that it sucks, because teachers tell us that it sucks. "But it fits our model, it's proven by the data", we say. And in the end, who are we going to believe, the dignified and universally-acclaimed academics or the teachers (those damn unionized lazy teachers)?

Now to the story. As I mentioned in my last post, this summer I am an adviser for two new teachers, which means that I sit in the classroom, observe their lessons, and provide them with feedback and coaching. This gives me the chance to see the complexities of the classroom from a different perspective--without being the one who is responsible for delivering content and managing behavior. I am a fly on the wall, and I see a lot of interesting things each day. Today, one of the teachers was leading the students through a math game. The game is called "buzz". It is a simple game. Students form a circle and each count off starting from "one". The object of the game is to get to 100. There is only one rule to the game. The teacher gives the students a number from 1 to 10. That number is the "multiple", and any time the students reach a "multiple" of that number in their counting, they must say "buzz". If anyone messes up, the entire class goes back to zero. For example, if the multiple is "5", then the students are supposed to count 1, 2, 3, 4, BUZZ, 5, 6, 7, 8, 9, BUZZ...and so on. In theory, this game only necessitates that students be able to do two things: count from 1 to 100 and know their basic multiplication facts. In theory, this game should be quite simple for 7th grade students. Individually, they have the capability to do these tasks. In theory, if each individual can do these tasks, the group should have no problem completing this game. However, as you might have already guessed, it was quite the struggle for this group of 25 students. Most of the time, they got to around 30 before someone messed up. There were screams and groans as the class berated their classmates for a silly mistake. "Can't you multiply? It's so easy!" As disrespectful as that sounds, it's actually not that far from what the education "models" tell us. The "product", in this case, is successful completion of the game. Stuff goes in, product comes out. If the product ain't coming out, there must be something wrong with the inputs (not the machine). If the kids can't complete the game, it must mean that they can't multiply or count to 100 or "think critically" or whatever other condescending thing you want to throw in there. But we can't blame the kids completely-- it's those damn lazy teachers! They didn't teach the kids how to multiply! It's obvious, just look at what's happening in this game! As a passive observer in this situation who also happens to be a middle school teacher, it's pretty obvious to me that the reason students are struggling has little to do with their ability to multiply or count to 100, and everything to do with group dynamics, teamwork, and classroom culture. It's a similar phenomenon to any situation where many interdependencies exist. Students do not learn on command nor do they demonstrate knowledge on command. A classroom is much like a forest-- there exist many variables that are poorly understood, interrelated, and likely impossible to measure. We make this ridiculous assumption that our schools and classrooms run like machines, and that if we take inventory of the individual parts of that machine and throw in a specific combination of inputs that we will be able to accurately predict what comes out. That just couldn't be further from the truth. 

Are we talking about the right issues?

My first year of teaching is over. It's actually been over for about a month already, but it took a while to truly disengage myself from "teacher mode". This summer, I am a faculty adviser for the Teach For America summer school program hosted by my school district-- the exact program I went through as a corps member last summer. My experience in this position so far is enough for an entire post, so I'll leave that alone for now, but I mention it because I spend a lot of time sitting in a classroom each day, which gives me plenty of time to think about teaching and reflect on the past year. It's particularly interesting to reflect given the current context-- watching new teachers encounter the same struggles I fought with only months before. But ultimately, what I am most interested in exploring in relation to the past school year is student learning. "Are students really learning?"-- this is perhaps the most important question in education today, but to me it really seems like the elephant in the room, in that we are reluctant to face realities about the unfair way we assess students. Instead, we prefer to focus on (inadequate) proxies like teacher accountability and the common core. Not that these topics aren't important, but rather they are a red herring allowing politicians, bureaucrats, and pundits alike to talk tough and act tough on education reform without addressing the fundamental issues-- the issues that require tough conversations and painful sacrifices.

Take the issue of teacher tenure, where a California judge recently ruled that the practice was unconstitutional, stating in his decision that, "Substantial evidence presented makes it clear to this court that the challenged statutes disproportionately affect poor and/or minority students. The evidence is compelling. Indeed, it shocks the conscience."

The judge references previous rulings on racial segregation of schools, funding disparities, and term length disparities, describing them as issues of a lack of equality of education; whereas teacher tenure presents an issue relating to the quality of the educational experience. Of particular interest to me is the judge's statement regarding the legal positions of his decision:

"[This court] is not unmindful of the current intense political debate over issues of education. However, its duty and function as dictated by the Constitution of the United States, the Constitution of the State of California and the Common law, is to avoid considering the political aspects of the case and focus only on the legal ones. That this Court's decision will and should result in political discourse is beyond question but such consequence cannot and does not detract from its obligation to consider only the evidence and law in making its decision."

Ostensibly, this seems like a very reasonable (and necessary!) thing-- to consider only the evidence and law in making the decision. However, the very nature of "evidence" changes dramatically as we move from issues of equality of education to quality of the educational experience. Furthermore, the judge's choice to consider certain types of information as "evidence" is an inherently political action. The primary evidence cited in the judge's decision was the expert testimony of Harvard professors (economists) Raj Chetty and Thomas Kane. Chetty's testimony, based on his 2011 paper attempting to quantify the long-term impact of teacher effectiveness on students' adult income, was that a grossly ineffective teacher costs students $1.4 million in lifetime earnings. Kane, based on his own study, testified that students in the LA Unified School District taught by a grossly incompetent teacher lost 9.54 months of learning in a single year compared to students with average teachers. Wow! Such impressive and clear data! It's no wonder the judge was moved enough to comment that it "shocks the conscience". But when we look behind the economic models we find an ominous motif: teacher effectiveness and student achievement that is almost exclusively measured through state-mandated standardized tests. And so we return to the judge's original claim: that decisions are made based on evidence and the law, no politics involved. By regarding the aforementioned studies as evidence (attributing them as objective truths), the judge is effectively making a quite controversial: that the current standardized testing regime provides accurate data about student learning. So much for the absence of politics.

My argument here is not that there is zero value in standardized test scores or value-added models of teacher effectiveness; it is not that laws protecting incompetent teachers are harmful to students. Rather, my worry is that our continued focus on these issues crowds out more important (and potentially more fruitful) issues: de facto school segregation, poverty, privatization of education, to name a few. Why are our priorities so misaligned? Maybe we should ask for expert testimony of our teachers (in contrast to that of the Harvard economists, who have never spent a single day in front of 35 screaming 7th graders), rather than trying to continue playing this ridiculous blame game that devalues those tasked with cultivating our nation's most valuable resource, it's children.