Assessing Multimodal Performances

Madeline Sorapure addresses the problem of assessing new media compositions, emphasizing the need to assess the “relations between modes in a multimodal composition” (4). The two most common problems faced by teachers assessing these relations are, she says, a) an overmatching of modes—the visuals, for example, simply repeat the text, as when the textual lyrics of a song are accompanied by the sound of the song, and b) an undermatching of modes (modes that have little or no relationship)—the visuals, for example, have no apparent connection to the text. She calls these the two problems of “pure repetition or pure arbitrariness.”

How, I wonder, does Sorapure’s own multimedia article fair in terms of the relationship between modes?

I confess I don’t feel at all qualified to assess multimedia presentations. However Sorapure, citing Yancey (who acknowledges this “discomfort”), says instructors “are indeed qualified to look at the relations between modes and to assess how effectively students have combined different resources in their compositions.” So here goes.

The three modes Sorapure uses are text, visuals, and sound. She provides three sound options: bird song, indistinguishable human vocalizations, and clock chimes, with an option to turn the sound off. The visuals move between four parts of a photograph of a building with statues in front of it (one part for each section of the text), though the details are difficult to see because either the picture is in motion or it is blocked by the text. The text itself is presented in blocks about the size of one and a half paragraphs with a scroll function combined with a button function (the latter for moving to a new block of scroll). She also provides a unimodal textual alternative–in effect, an option to turn off not just the sound but the visuals, too.

I am afraid that I cannot see any relationship between the three modes, either between the visual and the auditory or between those two modes and the text. What do birdsong, human chatter, clock chimes, and an anonymous building only half seen have to do with ways of assessing multimodal presentations? I don’t know. In addition, the limited access to the text I found limiting in terms of comprehension.

Yancey, in her famous 2004 CCC Chair address (or was it, as Chris Farris observed, more a dramatic performance?), says that she “designed a multi-genred and mediated text that would embody and illustrate the claims of the talk” and that the “images…did not simply punctuate a written text; together words and images were…the materials of composition.”

Actually, if we are talking purely about visuals, this is nothing new. Business writing textbooks have long taught students how to design documents featuring visuals—for example, how the visual must relate directly to the text and vise versa.

Actually (again), I’m not sure how well Yancey succeeds with her own visuals. Using the metaphor of tectonic shifts in higher education is one thing, but illustrating it with a map of the continental tectonic plates is another. Beyond a trite parallel, I don’t see much value in the visual.

So assessing the use of visuals is nothing new in composition. The use of audio, however, is relatively new, at least to me. And this seems a more complicated assessment challenge. While I don’t see any relevance in Sorapure’s audio, I’m not sure what I would suggest instead (in the role of the instructor who makes suggestions for revision). Simply to have an oral repetition of the text doesn’t work (according to Sorapure’s own criteria). So what would?

It’s a fair question.

As a practical assessment of Sorapure’s article, I note that I first switched off the audio because it didn’t mean anything to me–indeed, I found it distracting (not only is it unrelated to the text but it loops and repeats itself interminably). Then I switched to the unimodal textual version to actually read the text because I found the visual design got in the way of my comprehension. Thumbs down?


2 comments on “Assessing Multimodal Performances

  1. I think I may have missed the instructions that we were supposed to listen to the sound effects while reading the Sorapure article. While I agree that her article could have demonstrated her own visual rhetoric skills with more effectiveness, I found the message she was conveying helpful. The techniques she discusses are simplistic, but I found them to be a good starting point for re-thinking assignments. I enjoy being challenged, but I, like my students, enjoy receiving suggestions that are “success-oriented.” What I liked about Sorapure’s article is that it was so straightforward that I felt like I could use her methods for an assignment tomorrow.

    Not a lot of times in academic articles do you get the example brought to life. So often, I’m left with the question in my mind, “And what does this look like?”

  2. I liked your red thumbs down image which immediately tells us that you did not approve of the article. I am reminded of the movie Gladiator where Commodus would put either a thumbs up or thumbs down to determine whether someone should be executed. My attention was captured and I felt compelled to read your blog!

    Yancey’s use of tectonic plates to me was more a direct representation image rather than a metaphorical image on the tectonic shifts in higher education. The picture, however, brings me back to a world history class which I took in high school with the map hanging down the white board. I remember learning about current events in the world, finding a news article and presenting the news to classmates as part of an assignment and talking about it. I also remember that it was an intellectually stimulating class where we did all of our assignments in Interactive Notebooks, which were spiral notebooks with our class assignments printed and glue inside, or written with markers and gel pens which the teacher would collect at the end of the semester. It was a lot more convenient than turning in individual assignments and sort of like a portfolio for the teacher to grade.

    The point is that even if pictures do not quite complement the text, it brings up a series of episodic moments in my life that I could use to connect to this article.

    You mention Sorapure’s lack of relationship between the three models of sound, visuals, and text—do you think her purpose is to make it imperfect just so people can criticize and think of ways to improve it?

    I also think environment is important. What if this article was on a large interactive screen in a technological museum? Would it be more interesting than reading a plaque? Is this disconnect in text to new media a result of being on the laptop? Sometimes I think in using laptops, we are primed for the need for speed, and anything creative that slows us down is automatically seen as irrelevant.

    Additionally, Yancey mentions three modes—sound, visuals and text. I wonder if there are any other modes, such as touch? Touch and texture is a normal part of our electronics, such as laptop keyboards and this needs to come into play. Perhaps new media that causes us to run our hands across the keyboard in certain ways may light up areas in our brain—a new media approach that involves movement and sound and vision and text!

    Also, the ineffectiveness of Sorapure’s article may be due to the fact that it lacks interactivity and participation…how about some posts to go along with it? How about live chats on it?

    Ultimately, I believe there are ulterior motives to utilizing new media—that it is a call for English teachers to become interdisciplinary, to get more into the arts.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s