robo-readers do what teachers can't! Or, how we turned writing literacy into an algorithm

robo-readers do what teachers can’t! Or, how we turned writing literacy into an algorithm

Jun 7th, 12 / 1 Comment /

The Canadian Broadcasting Corporation has this weekly radio show called “Spark.” It is, in my opinion, the best technology-focused show that Americans don’t know about.

This week’s show included a story on the use of computer tools to read and score student writing on standardized tests. Spark host Nora Young interviewed Mark Shermis, the Dean of the College of Education at the University of Akron, who had this to say about how so-called “robo readers” assess writing:

They don’t necessarily use the same logic that you or I would in evaluating an essay. Those that use natural language processing will look for structures of arguments, so for example they might be looking for key words like ‘first,’ ‘second,’ ‘third,’ and ‘finally.’ Or if they’re looking for the main point, they might look at the topic sentence and try to identify key words that would be associated with an essay that was on topic. Or if they were looking for a conclusion they might be looking for something like ‘in conclusion’ or ‘in summary.’

Guess what, dudes–I’m not down with this approach to assessing writing. As a pretty decent writer myself, a former college composition instructor and a current instructor of preservice teachers who will one day be teaching our young people how to write, I tell my students the following:

Don’t use the phrase “in conclusion” to end your essay–any reader who’s paying attention can tell that they’ve hit the end of your paper, and “in conclusion” is therefore redundant, throwaway information.
Key words like ‘first,’ ‘second,’ third,’ and ‘finally’ are often lazy transitions, and ones that extremely strong, creative writers almost never use. While they work just fine, they often make a reader feel like s/he’s reading a set of driving directions.

Then I tell my students some things about how good writing is writing that meets the needs and interests and expectations of the reader while also jarring or provoking the reader in some significant way. Often, this jarring can be brought about through creative, unexpected use of language–precisely the sort of thing that robo-readers cannot detect.

Young asks Shermis about the creativity aspect, noting that computers can’t effectively assess this element of good writing. Shermis concedes the point, but adds that

this notion of creativity is kind of a curious one. If you take a look at the actual curriculum, when kids graduate and they go to the university, most colleges and schools aren’t looking for creative writing; they’re looking for somebody who can string together a subject, predicate, and nominative–that is, that they can communicate effectively. When you actually do an analysis of 95% of the writing that goes on even at the university level, it’s not creative. It’s simply a communication pattern that’s been ascribed to by professionals in the field.

Sure. But given that so much of the writing that young people are doing is not directed toward professional or career goals, given that so much of how young people learn, communicate, and participate in social and civic life is through written but informal text online, it just seems silly to put faith in the idea that writing skills are, at their base, about following ‘conventions’ agreed upon long before the internet was even a glimmer in anyone’s eye.

Anyway, go take a listen to Spark if you’re into technology. It’s a cool, fun show that deserves more listeners on the American side of the North American borders.

Tweet This Post