Do We Count in #rhizo15?

I will eventually return to finish my series of posts on ethics in MOOCs, or swarm ethics, or rhizo-ethics, but Dave Cormier has issued a second challenge for #rhizo15, and I want to respond.

His challenge:

Get out there and count! What can we measure that isn’t learning? Think about all the other facets of the human experience… can we do better? What about all the fancy tools we’ve seen… can they help? Should we throw it out all together? Can we help people measure themselves? Is there a better way of looking at it? Be theoretical. Be practical… but GRADE ME!

Measuring has always been problematic for me, in part I suppose because of my discipline: writing. I do not approve of counting writing as a way of assessing students' abilities to write. A 76 on an essay has always struck me as meaningless and wrong-headed, and my readings in complexity theory cause me even more concerns.

For instance, I'm currently reading Timothy Morton's challenging book Hyperobjects: Philosophy and Ecology after the End of the World (2013). Morton, by the way, is an English professor at Rice University who seems to share some of my own interests. At any rate and without getting into the meaning of hyperobjects themselves, which I'm not yet ready to discuss, Morton makes some interesting observations about the problems of observation and measurement in relativistic and quantum sciences. These ideas resonate with me.

First, he notes that measurement is always incomplete. Any measurement reveals and blinds at the same time. As we focus on some aspect of an object—say, an electron or a fourth grade boy—we lose sight of some other aspect of the boy or electron. This is not some mental trick; rather, it is the nature of reality as far as we can currently describe reality: Morton says, "This isn’t about how a human knows an object, but how a photon interacts with a photosensitive molecule. … The way an electron encounters the nucleus of an atom involves a dark side. Objects withdraw from each other at a profound physical level" (Kindle Locations 756-758). Think of Heisenberg's Uncertainty Principle here. Morton elucidates:

Quantum theory specifies that quanta withdraw from one another, including the quanta with which we measure them. … Thus, when you set up quanta to measure the position of a quantum, its momentum withdraws, and vice versa. Heisenberg’s uncertainty principle states that when an “observer”— not a subject per se, but a measuring device involving photons or electrons (or whatever)— makes an observation, at least one aspect of the observed is occluded. (Kindle Locations 748-753)

There are no complete observations, whether by human or instrument. Thus, there are no complete measurements. At best, measurements are adequate, but this always introduces the question: adequate for what? In education, especially, we can never have complete confidence in our measurements of our fourth-grade boy, and we must always revisit our reasons for taking the measurement in the first place. Any measurement of our fourth-grader will, at the same time, reveal and hide salient aspects of the student. We ignore those hidden aspects at peril to both ourselves and the student.

Then Morton notes that measurements always entangle the observer with the observed. This entanglement is very problematic. Morton says, "[Nils] Bohr argued that quantum phenomena don’t simply concatenate themselves with their measuring devices. They’re identical to them: the equipment and the phenomena form an indivisible whole" (Kindle Locations 760-762). This should sober all of us, if not frighten the piddle out of us. Let's say this more practically: In some important sense, I become what I measure—both the things I measure and the tools with which I measure—and my tools and the things I measure become me. We all become entangled in a larger identity (this starts suggesting, I think, what Morton means by hyperobjects, but I'm not pursuing that here). If I measure our fourth-grade boy with a standardized test, then I become that test and that boy. I do not mean that I become ONLY that test and that boy, but that test and boy become entangled in my identity—indeed, for the boy, that test may be all that I am to him. There is no meta-language or privileged position outside the entanglement of me, test, and boy from which I can safely conduct my measurement. If I measure, then I am included in the measurement. In education, we should choose our measurements most thoughtfully, for we become our measurements. Moreover, our students become our measurements. That is an awful burden.

This entanglement suggests to me that any measurement changes the observer, the instruments, and the observed, and this change feeds imprecision back into the incompleteness of observations and measurements. The act of observing our fourth-grade boy changes that boy. He would have been different if not measured. Of course, the boy—along with everything else—is constantly observed and measured by other objects, and that constant observation and measurement and its resulting entanglement is in part what makes the boy what he is. In some sense, then, we cannot avoid making observations and taking measurements of others or being observed and measured in turn, but we seem to have some freedom in choosing the kinds of observations and measurements that we make. We should choose wisely. I wish I knew what was wise.

Of course, I am not suggesting here that education is in any way special. We all constantly measure our worlds, if only informally and loosely: that's more, this is less; you are prettier than he; you are big, you little; this is near, that is far away; you have more money than I. Counting numerically is more precise and handy than counting in natural language, but Morton is reminding us that measurement, counting, is not the benign activity that we imagine it to be. It implicates and entangles. What counts in life is what we count and how we count it. Counting has two entangled meanings here: enumeration and evaluation. We count what counts, and what we count comes to count.

Awareness of this entanglement is not new, I think. In his Sermon on the Mount, Jesus says: "Judge not, that ye be not judged. For with what judgment ye judge, ye shall be judged: and with what measure ye mete, it shall be measured to you again" (KJV Bible, Matthew 7:1,2). This passage is usually read as a moral lesson to avoid hypocrisy and censure, but I think we can give it a more profound reading: what and how you count determines how you yourself count. Perhaps the spiritually advanced among us, the deep meditators of the various religious and philosophical traditions, can measure without judging and without becoming their measurements, but I cannot. If I want to know who I am, then I can get a reliable read by noting what and how I measure my world, especially others, and how I assess the value of those measurements. That stuff counts.

I don't think comma faults count much, so I don't count them, but I know people who do. A comma fault—what a sad thing to become. It's really hard to have a pleasing conversation with someone who's always correcting your grammar, so I usually don't. What do you measure?

Assessing Complex Systems and Sonnet 73

As my own views about education continue to emerge, I understand them best within the context of the conversation about complexity—complexity as a large, transdisciplinary conversation that has been emerging for centuries, but that was made unavoidable by the emergence of relativity and quantum physics at the beginning of the 20th century. The fact that I just used a form of the term emerge three times in a single sentence suggests how much Complexity has informed my thinking. As I am so very fond of following rabbit holes, it helps me from time to time to gather my thoughts to see if something coherent emerges. See?

I've been reading through a series of articles about complexity and the limits of knowledge from a 2005 special edition of Futures. I recommend it to anyone interested in either complexity or knowledge or the knowledge of complexity or the complexity of knowledge. You can really get tangled up, or at least I can. So I want to do a bit of untangling.

At the largest scale I can think about, complexity is that zone of engagement between the open-ended future and the closed past. We call that zone of engagement the now or the present. I could refer to it as The Now and perhaps win an honorable mention in the next Eckhart Tolle book or a few minutes on Oprah, but I'm feeling sober this morning, so I'll just stick with the now. Complexity is the activity that emerges between the juxtaposition of the hot, open-ended potential of the future and the cold, fixed certainty of the past. Complexity is the result of the tension between hot and cold, or to borrow a phrase from David Foster Wallace, it is the result of the miscegenation between a hot air mass and a cold air mass. That image works for me: we exist in the thunderstorm of the now, and though we may long for the potential of the future or the certainty of the past, we cannot live in either place. Life, and by extension knowledge, cannot exist in the chaotic order of the future or the fixed order of the past, but only in the dynamic, emerging order of the now as the heat of the future slides by and is transformed into the cold of the past (I'm perfectly willing to believe that the transition from hot activity to cold fixity only gives us the illusion of movement, but the visual metaphor is appealing to me). The complexity of the now is all we get, all we have, but because the now is a complex system, it is profoundly affected by and interacts with both the future and the past. Both the future and the past inform the now, and the dynamism of the now informs both the future and past in turn.

So for me, complexity is about as big an idea as I can have—sort of a God idea, but I don't intend to talk about God in this post; rather, I want to talk about knowledge and education and what the overarching concept of complexity has to do with them. How does it inform my ideas of knowledge and education? That's the question.

In their Introduction: Complexity and Knowledge (Futures, 2005, Vol. 37, pp. 581-584), Peter Allen and Paul Torrens note that the study of open systems proved to be very problematic for scientific knowledge in both the hard and soft sciences. They say:

For isolated and closed systems classical thermodynamics gave us the knowledge to predict the transformations and final equilibrium states of a system. Obviously, for frictionless systems such as those involved in planetary motion, Newton’s Laws allowed the prediction of orbits and eclipses, both forwards and backwards in time. Knowledge was complete and related directly to prediction. But, open systems were much more problematic. (581-582)

Closed systems, it seems, function in the simple and complicated domains, as defined in the Cynefin Framework. The simple and complicated domains afford us "the knowledge to predict the transformations and final equilibrium states of a system … both forwards and backwards in time." In closed systems, we can arrive at complete knowledge with reliable—testable and verifiable—predictions. Open systems do not allow such affordances.

This is a big problem, as Allen and Torrens note. So what's wrong with open, complex systems? First, we have a boundary issue. Open systems do not have discrete boundaries. I can see this quite clearly when I try to imagine the boundary between now and the future. The boundary has a thickness. I can feel the future coming and the past slipping, sometimes quite strongly, but I can never quite put my finger on the exact line between the future and now, and as soon as I fix my finger to a line, it slips into the past (the line, not my finger, which fortunately stays with me in the now). So the boundary also has an incredible thinness. So which is it—thick or thin? Well, both, of course. The boundary is open, and the exchanges between the system inside (now, for instance) and the systems outside (future and past, for instance) modify all systems. I really am speaking universally here; thus, I include those systems within the simple and complicated domains. From my point of view, everything belongs to the complex domain, and the simple and complicated are but temporary arrangements that we form for our convenience—like a sock drawer, or a classroom. We can pretend for a moment that our classrooms belong to the simple or complicated domains, but they don't. The classroom is a complex system of complex systems, and to treat them otherwise is to risk complete misunderstanding.

The dynamic interaction at the boundaries among complex, open systems means that it is very difficult to limit ourselves to local causality. In other words, the events in any one classroom are the result of remote causes (familial, social, economic, political, etc.) just as much, sometimes more so, as local causes (say, a classroom lecture or demonstration), and we are unlikely to be able to assess exactly what caused any given behavior in our students. Nor can we predict reliably the effects of any applied intervention or instructional design. As Allen and Torrens put it:

The simplest definition of a complex system is one that can respond in more than one way to its environment. … So, ‘knowledge’ about the future trajectory of the system can be both quantitatively and qualitatively wrong. … Innovation can occur, and it may have untold implications for the future evolution of both the ‘inside’ and the ‘outside’ the system. Similarly, the same ‘intervention’ may produce two different results on what were believed to be similar systems, since a single complex system can respond to an intervention in different possible ways. The outcomes could differ qualitatively and this surely must therefore introduce some doubt into the ethical basis for the intervention. … These new ideas force us to accept a significant reduction in our powers of prediction, and even in our ability to frame a useful question.

I find myself, here, slipping into considerations about evaluation and assessment in education, and I'm reminded of the recent words by Christina Hendricks, Stephen Downes, and Keith Brennan about how to assess a MOOC. I won't go into the details of their discussion, but I will say that from my vantage point measuring a MOOC, or any other classroom, is more like measuring a thunderstorm than measuring an automobile. That being said, I think we are beginning to develop some useful metrics for measuring open, complex systems. I may need to complete one of Siemens' learning analytics MOOCs to learn what some of those metrics.

I want to add, as well, that I think we literary scholars have been confronting open, complex systems for a long time. Consider a Shakespearian sonnet—Sonnet 73 will do. Almost all the data that I can gather from traditional measurement (meter, rhyme, number of lines, number of feet, etc.) says so very little about the poem. That data can enrich my understanding and appreciation of the poem, but by itself, that data reduces the poem to a closed system, a handy sock drawer, some trivia to answer on a test, and I would never read the poem again if that's all I had. Only when I open the poem to its environment, allow it to breathe, allow it to help me make connections to grandma, winter freezes, and dying embers, to my hopes and fears, only then do I find value and meaning. I find that value and meaning difficult to measure and assess, but I'm hopeful that we are developing the tools that will help us do so. Some very interesting things are happening in the digital humanities that point this way. I'll have to read some more.