Assessing Industry-Standard Software? Or Multimodal Assignments?

03 Oct

Are students without a background in industry-standard design software rated more harshly in digital multimodal compositions?

I have wondered for a while whether digital multimodal composition assignments, from an assessment/response perspective, favor students who have a background in industry-standard design software like Adobe Photoshop or Adobe Illustrator. To measure this, I could modify the study conducted by Johnson and VanBrackle “Linguistic Discrimination in Writing Assessment.” Into a corpus of multimodal projects to be assessed, I could insert projects with typical design “errors” that result from ignorance of the software: pixelated images from improper resizing, etc. Then, using raters who are both knowledgeable and not knowledgeable about design, I could compare the texts with “mistakes” against texts without “mistakes.” I could even separate sample texts into three classes: expertly designed, designed with some software knowledge, and designed with no software knowledge.


Posted by on October 3, 2013 in Uncategorized


5 responses to “Assessing Industry-Standard Software? Or Multimodal Assignments?

  1. brucebowlesjr

    October 8, 2013 at 6:39 pm

    This is a really interesting topic to explore; I think from an ethical standpoint, this is important to know.

    What about when students use unfamiliar software rather well? Does this still cause instructor apprehension?

    I think you might want to define what types of errors are typical of inexperienced designers more specifically. I’d be curious as to how you approached that aspect of the study.

  2. jc12t

    October 9, 2013 at 3:08 pm

    I think this is the start to something interesting, but I have a few questions (which kind of guide my suggestions). So, just as a general question, what’s the context of this these multimodal projects? For example, is it FYC where familiarity with the software may not be a major concern or is it at the end of an EWM major’s degree? And then, why is this important to study for that context? Like if its for the EWM major to see if they’ve accomplished a certain outcome, are raters assessing fairly? Just stuff to consider.

  3. amypiotrowski

    October 9, 2013 at 5:09 pm

    This is an interesting question, and I agree with Bruce that it’s important to ask this question as a matter of ethics and access. How might you account for some errors being more egregious (and noticeable to raters who are novices) than other design problems might be? You may also want to define what counts as a rater who is knowledgable about design. How much experience with certain software programs would a knowledgable rater need?

  4. jasonecuster

    October 10, 2013 at 1:07 pm

    Suggestion: The title of this threw me off a bit, since I read everything through the lens of whether or not it was the software being assessed or the composition (like that we liked work from Photoshop more than GIMP, so we were awarding higher assessments for students with better access to software), not experience levels with both. My bad for reading it wrong! I’m editing this now to make more sense and coming up with new suggestions. That said, a different title might help!

    Question: This may seem like a silly question, but, don’t these assessments favor students with experience by default? For that reason, I think your topic makes sense and merits exploration, but I must say that I struggle to see a world where this isn’t true.

    Comment: (Continued from the point above) Even in my own limited experience with asking students to compose with image editing software (like using Photoshop, GIMP, and/or Pixlr to make game posters/boxes for their own games), I am forced to assume that had we not had an instructor like myself present to help students with Photoshop in class and resources like the Digital Studio to support instructors/students, that anyone without experience with that software wouldn’t have much of a chance at being successful. I do also allow students to make their posters by hand if they so desire, to not force them to have expertise with the software should it not be a real option in the time constraints. Again, for students asked to create a visual without any knowledge or support, I can’t imagine the students with design experience not faring better. In that way, I imagine this could provide some surprising and helpful results, especially depending on how it plays out. I wonder as well, then, if it would be possible to break this down to examine some other variables, like what affect receiving assistance from a resource like the Digital Studio has for the students without previous experience, and whether having an instructor with experience with the software has an impact. Those variables all seem crucial to this kind of examination, since I imagine shifts in any of them could radically alter the results.

  5. jacobwcraig

    October 20, 2013 at 1:48 am

    I’ve been interested in this question for some time. I think that Joe raises some good points. In my thinking, I can’t escape the idea that the context of the assignment really matters: how the instructor talks about the assignment, if it is a frequently used technology in the program, and even if there are students in the classroom who consult with their peers about ways of using the software. And like Joe, I think that there is a validity question related to the assessment. If it’s an EWM class, there is the grade at the end of the course, but students are also trying to develop a professional portfolio. I think that students do a better job of thinking about grades than we do, and they are really responsive to the stake. What’s the stake of the assessment? What’s it measuring? Skills for a resume, rhetorical knowledge?


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: