To give us a concrete portfolio assessment example, I’d like you to look briefly at the Carleton College example, which has been around for over a decade. It’s a mid-career print portfolio. We’ll talk about it more in class.
Monthly Archives: October 2013
Hamp-Lyons and Condon discuss their process of developing a portfolio assessment at Colorado-Denver. They devote special attention to the ways that portfolios are assessed in the program –  how readers see one genre v. multiple genres;  what counts as a “broader reading” or a reading that invites judgments that are compatible with holistic scoring (as opposed to paper-by-paper judgments);  whether the portfolio makes the students’ process visible; and  whether portfolios create a community of assessment on a campus.
It is a project that they describe having several different iterations. The first iteration of the project was a piece designed to “help faculty reach consensus about what they mean by argumentation” (189). This is important, because two of the four projects included in the portfolio are arguments, so any assessment decision made about the portfolio is significantly informed by students’ argument papers. The second iteration of the project was designed to help faculty “define criteria” for assessing portfolios (189). Through this project, faculty started describing their practices. The third project, the project they represent here, shows that the process of implementing portfolios can have benefits that far outweigh “the assessment reason” — communication among faculty, faculty development as an “outgrowth” of teaching, democratization of faculty (regardless of rank), consensus and collaboration (189).
Elbow and Belanoff reflects on their use of portfolio to assess students’ proficiency in FYC.
E&B talk about how there are multiple stakeholders in this kind of assessment beyond the individual student: the teacher of the FYC class and other “judges” who offer a simple binary evaluations on the portfolio (pass/fail). These roles are interesting particularly because the judge’s role is not to offer assessment for learning, but simply to give a grade. The teacher’s role is to either accept or deny this judgement and to offer help to the student. What I found interesting, was the emphasis on collaboration–E&B mention how this collaboration is the central advantage to this kind of assessment.
They discuss how the students’ focus on the “likes” and “wants” of the teacher is de-emphasized because the evaluation is distributed across multiple stakeholders. The “likes/wants” of the teacher are now in dialog with other collaborators and an evaluation is made by a community of readers than one individual to another individual.
Chapter 7: “Showcase Hybridity: a Role for Blogfolios” and Lam’s article “Two portfolio systems: EFL students’ perceptions of writing ability, text improvement, and feedback”
From ePortfolio Performance Support Systems I read Chapter 7, which is titled “Showcase Hybridity: a Role for Blogfolios.” Middlebrook and Sun are essentially claiming that it would be good to combine blogging practices with ePortfolio practices in classroom settings and professional settings. They say that ePortfolios are static mediums, and that blogs are, in contrast, dynamic and could allow for customization of the “evidence showroom,” or the ePortfolio. They say blogs can also help express digital identity, and can facilitate web communication between students and higher-ups in the field. Specifically, then, blogs could potentially help students gain a sense of self and grounding in the field they choose. A blogfolio could be (and so far, has been) a great professional tactic.
Middlebrook and Sun lay out some obstacles for using a blogfolio. First, it would be difficult to train teachers in how to use them. Second, we would need to decide how to assess them. Third, we would need to invent assignments. Fourth, the whole process could lead to information overload for both student and teacher. Fifth, blogs are met with skepticism in academia. Sixth, there is, to a certain extent, a permanency of web entries; so, if a student writes something that is factually incorrect, irrational, too emotionally-driven, or unintelligent, there is a slight danger that it will haunt them in ways paper couldn’t. But Middlebrook and Sun communicate that the pros here outweigh the cons easily, especially since this kind of online composing could allow for professional contact in new and exciting ways.
Lam’s article “Two portfolio systems: EFL students’ perceptions of writing ability, text improvement, and feedback” seeks to find out how a select group of Chinese students responded to two different types of portfolio assessments. Lam mentions that the assessment environment in China is centered around examinations and products. Students are primarily interested in finding out what errors they make, though they do also want to know about how they are doing in terms of craft, organization, etc. The research for this article was conducted in a second semester writing class; the portfolios that were in question were composed of twelve written drafts (three for each of the four genres), and there were two types of portfolios: working and showcase. The students had to revise their drafts using self, peer, and instructor feedback.
This was a qualitative experiment seeking out detailed results; the information was gathered through interviews, examinations of reflection journals created by the students participating in the study, classroom observations, and analysis of the students’ texts. Lam’s findings section indicates that the students in Group A (the group trying the working ePortfolio) preferred (and really valued) instructor feedback more than peer feedback in earlier drafts, especially in terms of teaching them how to perform revisions. They said that, since peer feedback was less error-finding-oriented and more big-picture/clarity/paragraph development-oriented, it would be best saved for later drafts.
The students in Group B (the group trying showcase ePortfolios) were concerned that there was too much focus on the finished product rather than the process of writing itself. They also did not seem to trust, as easily, self and peer review, and they were more prone to procrastinate. Lam has several very helpful graphics that show the numbers involved with this study (and through them, he gives you many specific examples that illustrate his generalized claims). I encourage you to have a look at them if this study interests you.
In their chapter, “What are you going to do with that major? An Eportfolio as bridge from the university to the world,” Karen Johnson and Susan Kahn share their attempts to use e-portfolios as a transition from writing in the university to writing in the professional world. Beginning from the common recognition of fear, anxiety, and potential homelessness that many English majors face as they near graduation, they design an e-portfolio curriculum for IUPUI’s Senior Capstone Course that utilizes e-portfolios as a means of presenting a professional self. Designed with the needs of their mostly non-traditional students in mind, their curriculum engages students in a structured presentation of their work as English majors along with sustained reflection which relates their experience within the major to the career paths individual students hope to pursue as well their role as global citizens in a highly networked world. They found that this program relieved many of the anxieties of students and helped them to develop confidence that their work as an English major had indeed developed skills with which they could make a living. Moreover, they demonstrated an increased ability to reflect not only on their experiences, but on their writing and intellectual processes.
I read chapter 3 in the eportfolio book. This chapter focused on the social nature of writing in digital, networked spaces, arguing that students can take advantage of these social affordances. Klein argues that eportfolios allow for “connection, communication, and collaboration” (p.52-53). As a result, eportfolios allow students to showcase their “intellectual leadership, analytical ability, and personal creativity” (p. 53). Klein ties the affordances of Facebook, Twitter, and social bookmarking to eportfolios. She argues that by making portfolios social, the writing they contain can cross the boundary between contexts.
Klein focuses on the eportfolio collection in the honors college of the City University of New York. Students in the honors college create eportfolios using WordPress. The college believed that WordPress provided a platform where students could display their work while also being a social space where students could connect by commenting on one another’s work. Students were able to do just that. Klein reports that “Through written reflections, digital photos, and—in some cases—short films, students demonstrated critical multimedia literacy, the ultimate learning objective of many college-level courses” (p.60).
Klein also believes that by being social, eportfolios allow students to showcase their knowledge and skills for potential employers. This allows work done in academia to cross contexts into the world of business. Klein focuses on one student who showcased his design and marketing skills.
The Hamp-Lyons and Condon piece serves as a nice precursor to Yancey’s piece because they are working only with print portfolios (the piece was published in 1993). They conduct a study on the assumptions that readers bring to portfolio assessment because they are “committed” to portfolio assessment and want to understand more about “all we do, even when it is successful” (315). They write about the portfolio assessment program at Washington State University, and structure their article on the exploration of five assumptions, summarized briefly below.
1. “Because a portfolio contains more texts than a timed essay examination, it provides more evidence and therefore a broader basis for judgment, making decisions easier” (319). They dispute this claim by saying that readers cannot actually read portfolios holistically because of their containing multiple texts; these multiple texts “force readers to consider one text in light of the other” (319). Thus, decisions are more difficult because multiple texts and numerous characteristics of these texts must be considered.
2. “A portfolio will contain texts of more than one genre, and multiple genres also lead to a broader basis for judgments, making decisions easier” (319). They identify two underlying assumptions here–that quality of text will vary from genre to genre, and that a portfolio will have multiple genres. If quality does not differ, they say, then there is no viable reason to include multiple genres. If, on the other hand, it does differ, that makes readers’ decisions harder because they must move back and forth between texts, considering earlier texts in light of later texts. Additionally, they say that assumptions one and two seem to also assume that readers will attend to the entire portfolio, which they have not found to be the case. Typically readers move toward a decision while reading the first text and use the rest of their reading to support that decision. They also found that “readers tend to reduce the cognitive–and time–load in portfolio reading by finding short cuts to decisions” (322). Because of this, they claim that students who place their “best” text at the end of the portfolio might be doing themselves a disservice.
3. “Portfolios will make process easier to see in a student’s writing and enable instructors to reward evidence of the ability to bring one’s own text significantly forward in quality” (323). Again, they found this to not be the case because their portfolios did not contain evidence of process–meaning multiple drafts, notes on revisions, etc. If we want to see evidence of process, they advise requiring multiple drafts of a text rather than several “polished” pieces (showcase portfolio).
4. “Portfolio assessment allows pedagogical and curricular values to be taken into account” (324). They claim that this is the case in their assessment because the “connection between curriculum and portfolio is carefully and consistently built” (324). In other words, they create many opportunities for faculty to come together and discuss the portfolios, their pedagogies, the curriculum, etc. Without this kind of working environment, the portfolio may not represent pedagogical and curricular values.
5. “Portfolio assessment aids in building consensus in assessment and instruction” (325). Again, they claim that this is not necessarily always they case. The process of faculty working to find points of agreement and places of compromise is important for the success of this type of assessment.
Their ultimate conclusion is that portfolio assessment should be recursively revised based on faculty conversations, data from the assessment itself, knowledge of what does and does not seem to be working. They also conclude by debating the uses and limitations of external criteria.