Madeline Sorapure addresses the problem of assessing new media compositions, emphasizing the need to assess the “relations between modes in a multimodal composition” (4). The two most common problems faced by teachers assessing these relations are, she says, a) an overmatching of modes—the visuals, for example, simply repeat the text, as when the textual lyrics of a song are accompanied by the sound of the song, and b) an undermatching of modes (modes that have little or no relationship)—the visuals, for example, have no apparent connection to the text. She calls these the two problems of “pure repetition or pure arbitrariness.”
How, I wonder, does Sorapure’s own multimedia article fair in terms of the relationship between modes?
I confess I don’t feel at all qualified to assess multimedia presentations. However Sorapure, citing Yancey (who acknowledges this “discomfort”), says instructors “are indeed qualified to look at the relations between modes and to assess how effectively students have combined different resources in their compositions.” So here goes.
The three modes Sorapure uses are text, visuals, and sound. She provides three sound options: bird song, indistinguishable human vocalizations, and clock chimes, with an option to turn the sound off. The visuals move between four parts of a photograph of a building with statues in front of it (one part for each section of the text), though the details are difficult to see because either the picture is in motion or it is blocked by the text. The text itself is presented in blocks about the size of one and a half paragraphs with a scroll function combined with a button function (the latter for moving to a new block of scroll). She also provides a unimodal textual alternative–in effect, an option to turn off not just the sound but the visuals, too.
I am afraid that I cannot see any relationship between the three modes, either between the visual and the auditory or between those two modes and the text. What do birdsong, human chatter, clock chimes, and an anonymous building only half seen have to do with ways of assessing multimodal presentations? I don’t know. In addition, the limited access to the text I found limiting in terms of comprehension.
Yancey, in her famous 2004 CCC Chair address (or was it, as Chris Farris observed, more a dramatic performance?), says that she “designed a multi-genred and mediated text that would embody and illustrate the claims of the talk” and that the “images…did not simply punctuate a written text; together words and images were…the materials of composition.”
Actually, if we are talking purely about visuals, this is nothing new. Business writing textbooks have long taught students how to design documents featuring visuals—for example, how the visual must relate directly to the text and vise versa.
Actually (again), I’m not sure how well Yancey succeeds with her own visuals. Using the metaphor of tectonic shifts in higher education is one thing, but illustrating it with a map of the continental tectonic plates is another. Beyond a trite parallel, I don’t see much value in the visual.
So assessing the use of visuals is nothing new in composition. The use of audio, however, is relatively new, at least to me. And this seems a more complicated assessment challenge. While I don’t see any relevance in Sorapure’s audio, I’m not sure what I would suggest instead (in the role of the instructor who makes suggestions for revision). Simply to have an oral repetition of the text doesn’t work (according to Sorapure’s own criteria). So what would?
It’s a fair question.
As a practical assessment of Sorapure’s article, I note that I first switched off the audio because it didn’t mean anything to me–indeed, I found it distracting (not only is it unrelated to the text but it loops and repeats itself interminably). Then I switched to the unimodal textual version to actually read the text because I found the visual design got in the way of my comprehension. Thumbs down?