big data and pedagogy

Discussions of big data and pedagogy typically focus on the relative merits of analytics for assessing and improving curriculum and teaching practices. Michael Feldstein has a good piece on this from a few months back where he argues,

Right now, what we’re trying to do is a little like trying to conduct physics research before somebody has invented calculus. You can do some things around the edges, but you can’t describe the really important hypotheses about causes and effects in learning situations with any precision. And if you can’t describe them with precision, then you can’t test them, and you certainly can’t get a machine to understand them.

In other words, maybe but not yet. I’ve wondered about this with writing assessment. I’m not sure if there is anyone out there using the methods developed in the digital humanities to study literary corpora for studying student writing. Such research is not suited, at least not initially, to determining the quality of writing, and as such whether or not students are meeting some standard, which is typically what assessment is investigating. However, it could tell us things about linguistic diversity, topic modeling, use of citation, and other textual features. In other words, it could tell us something about how student writing is changing over time. Maybe we could, through some secondary analysis, connect shifting textual features (e.g. paragraph or sentence length to give a basic example) with “good writing.” Maybe. Of course, turning this into a measure of pedagogy is another matter (turning into a mechanism for machine grading is an even more distant step in my view). As Feldstein says, maybe, but not yet.

However, I have another point that is really about reversing the relationship between big data and the individual student writer. Pedagogy begins (and ends) with the belief that individual students learn and that the learning experience of individuals is ultimately what we want to measure and value. It makes sense. As a student, I pay to learn; I get a degree; and when I leave college, I want to take something with me. At the same time, we recognize that learning is social and environmental; if it weren’t, then we wouldn’t have schools in the first place. So even though it is meant to have predictable effects upon individuals, learning is relational. (Indeed, I would argue that learning is a cognitive activity and all cognition arises from relation; thinking is not “inside.”) The more we start to think about thinking as a relational, networked activity, the more we might also want to shift our focus toward understanding the collective activity rather than the individual one. As such the point is that rather than being concerned about what big data can tell us regarding individual experience, maybe we should turn toward thinking about pedagogy as something that shapes a massive, collective activity that we are now getting a better ability to see. Think of this in terms of climate change. What would it mean to think of pedagogy as shaping the climate of learning? Individual activities, like changing our consumer habits, can have an impact on the climate, but changing individual behaviors is a means to an end that cannot be seen on the individual scale.

How does this apply to writing pedagogy? Though there are a wide variety of teaching practices out there, I’d argue they all share a focus on changing the individual behaviors of student writers. We may contend that there are social-cultural-ideological factors to discourse communities or activity systems or whatever term we want to use for the context in which we write, but we still end up focusing on the writing processes (or products) of individual students. If a student’s essay doesn’t met a standard, then the problem is addressed by focusing on that student’s writing process/behaviors. Even when we acknowledge that some of the causes for the problem are systemic, we still identify the problem as manifesting on the individual level. We pose the problem and solution on the individual scale. So what would writing pedagogy look like if it were designed to teach the collective rather than the individual? I know this sounds, well, inhuman, but if, as I have asserted elsewhere, writing is not a strictly human activity, and the goal is better writing, then why focus on individual humans? It seems to me that when students struggle with writing that it is because they are caught up in networked activities or assemblages that perhaps were productive once upon a time for some purpose but are no longer. We commonly recognize as instructors how difficult it is to shift students’ writing practices. In my view this is partly because we are seeking to make changes at the wrong site. This is the recognition of activity theory, though I believe activity theorists continue to put too much focus on the humans in their systems, which is fine if your interest is in studying human activity but is less useful if one’s interest is in the system itself. Writing is a systemic activity and the logical extension is that we would alter it on that scale.

This doesn’t mean that students don’t “learn to write;” it just changes what that phrase means to something like learn to operate within a compositional network. However it also means that the performance of students within that network cannot be attributed solely to the students. Understanding that big data allows us to see writing on a new “real” level in the same way that information technologies have allowed us to see climate is a significant change. It doesn’t mean that the individual student’s writing isn’t real anymore than climate means that the raindrops on your head aren’t real. It just gives us a different (and compelling) explanation for how those local phenomena arise.