A Simple Game

I have always been interested in the group dynamics that exist in a classroom--I think it's a topic that is largely ignored by those currently researching education. We spend so much time thinking about the effect of inputs via teacher quality, school quality, student-specific demographics and not enough time thinking about how all of these things are interdependent. Economists, specifically, tend to think about a classroom as though it were a machine. We put various things in, they are combined to together and we get a "product" out of it. It is a production function--the focus is on "output". This "output" is almost always tied to test scores. If the "output" isn't explicitly test scores, such as studies that tie education to future income, that "output" is still a function of test scores, so it's essentially the same thing. Test scores are wonderful data for economists and other researchers because it's easy to collect, readily available, and usually standardized. In short, it's already in the correct language for economists. While ease of analysis can be an important characteristic of data, however, it does not necessarily mean that the data is useful. This is a major issue I have with current education research--I don't think the models and data being used are giving us the most useful information. Instead, I think our obsession with test scores stems from researchers' love of quantifiable information. Education is a complex system. Learning, in terms of the "product" our schools "produce", is not easily understood. But we very much want it to be so. As a result, we collect test score data and we squeeze, twist, and torture those numbers until they fit nicely into the models we have created. We then sing our own praises and claim that we have finally figured out how to "fix" our schools. We just need to fire bad teachers. We just need to hire better teachers. We just need to develop universal standards. We just need to spend more money. We just need to pay teachers more. We just need to use more technology. We just need to test students more. We just need to test students less. We just need to hold teachers accountable. We just need to hold parents accountable. We just need to hold communities accountable. We just need to open more charter schools. We just need to close charter schools. We just need to collect more data. It's all so damn simple, if you'd just look at what our model tells us. Except it's not simple. Let me use an analogy and a story to explain.

First, the analogy. In his book, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale professor James Scott argues that the desire to organize and operate society according to rigidly defined scientific laws leads to unintended consequences that undermine the success of the system. In the first part of the book, he describes the history and development of German forestry science:

The great simplification of the forest into a “one-commodity machine” was precisely the step that allowed German forestry science to become a rigorous technical and commercial discipline that could be codified and taught. A condition of its rigor was that it severely bracketed, or assumed to be constant, all variables except those bearing directly on the yield of the selected species and on the cost of growing and extracting them. As we shall see with urban planning, revolutionary theory, collectivization, and rural resettlement, a whole world lying “outside the brackets” returned to haunt this technical vision.

I believe education has been turned into a "one-commodity machine", for precisely the same reason and with the exact same effect. In an attempt to make this complex system into a "rigorous technical and commercial discipline", we have created these bracketed models. We have looked at these very complex variables and decided that for the sake of simplicity and universality of our models, we should just assume them to be constant. We shall, in effect, ignore them. And our justification for this treatment is that "it's not correlated with our other variables". We treat it like random noise--something that exists but doesn't really matter to our "output". And because we have are very good at math, we can do that. 

The effects on these German forests were disastrous:

It took about one century for them [the negative consequences ] to show up clearly. Many of the pure stands grew excellently in the first generation but already showed an amazing retrogression in the second generation...A new term, Waldsterben (forest death), entered the German vocabulary to describe the worst cases. An exceptionally complex process involving soil building, nutrient uptake , and symbiotic relations among fungi, insects, mammals, and flora—which were, and still are, not entirely understood— was apparently disrupted, with serious consequences. Most of these consequences can be traced to the radical simplicity of the scientific forest.

This is remarkably similar to what we see in education. Each time a new "solution" or "system" is implemented, we observe great results at the outset. We believe our efforts have finally paid off, that we have finally solved the problem. But then it becomes abundantly clear that those gains are but an illusion. In our simplified models we neglected those intricacies and interdependent relationships, and over time those variables come back to haunt us. Things like peer effects, teacher morale, and classroom culture are ignored. But they matter-- they matter a lot. And every time we neglect them in our model (either by omitting them or "controlling" for them), we end up with policy that sucks. And it's easy to tell that it sucks, because teachers tell us that it sucks. "But it fits our model, it's proven by the data", we say. And in the end, who are we going to believe, the dignified and universally-acclaimed academics or the teachers (those damn unionized lazy teachers)?

Now to the story. As I mentioned in my last post, this summer I am an adviser for two new teachers, which means that I sit in the classroom, observe their lessons, and provide them with feedback and coaching. This gives me the chance to see the complexities of the classroom from a different perspective--without being the one who is responsible for delivering content and managing behavior. I am a fly on the wall, and I see a lot of interesting things each day. Today, one of the teachers was leading the students through a math game. The game is called "buzz". It is a simple game. Students form a circle and each count off starting from "one". The object of the game is to get to 100. There is only one rule to the game. The teacher gives the students a number from 1 to 10. That number is the "multiple", and any time the students reach a "multiple" of that number in their counting, they must say "buzz". If anyone messes up, the entire class goes back to zero. For example, if the multiple is "5", then the students are supposed to count 1, 2, 3, 4, BUZZ, 5, 6, 7, 8, 9, BUZZ...and so on. In theory, this game only necessitates that students be able to do two things: count from 1 to 100 and know their basic multiplication facts. In theory, this game should be quite simple for 7th grade students. Individually, they have the capability to do these tasks. In theory, if each individual can do these tasks, the group should have no problem completing this game. However, as you might have already guessed, it was quite the struggle for this group of 25 students. Most of the time, they got to around 30 before someone messed up. There were screams and groans as the class berated their classmates for a silly mistake. "Can't you multiply? It's so easy!" As disrespectful as that sounds, it's actually not that far from what the education "models" tell us. The "product", in this case, is successful completion of the game. Stuff goes in, product comes out. If the product ain't coming out, there must be something wrong with the inputs (not the machine). If the kids can't complete the game, it must mean that they can't multiply or count to 100 or "think critically" or whatever other condescending thing you want to throw in there. But we can't blame the kids completely-- it's those damn lazy teachers! They didn't teach the kids how to multiply! It's obvious, just look at what's happening in this game! As a passive observer in this situation who also happens to be a middle school teacher, it's pretty obvious to me that the reason students are struggling has little to do with their ability to multiply or count to 100, and everything to do with group dynamics, teamwork, and classroom culture. It's a similar phenomenon to any situation where many interdependencies exist. Students do not learn on command nor do they demonstrate knowledge on command. A classroom is much like a forest-- there exist many variables that are poorly understood, interrelated, and likely impossible to measure. We make this ridiculous assumption that our schools and classrooms run like machines, and that if we take inventory of the individual parts of that machine and throw in a specific combination of inputs that we will be able to accurately predict what comes out. That just couldn't be further from the truth.