RSS

Author Archives: profkelp

“Large-scale assessment, locally-developed measures, and automated scoring of essays: Fishing for red herrings?” by William Condon (from Assessing Writing 18 (2013); pages 100–108)

Rank: 0 (“Because these tests underrepresent the construct as it is understood by the writing community, such tests should not be used in writing assessment, whether for admissions, placement, formative, or achievement testing” (Condon 100).

Argument: The real problem is that what AES allows us to do (conveniently assess masses using one standard) is “too constraining” and “severely under-represent[s] the construct, writing, yet purport[s] to measure that construct effectively” (101). Basically, a 25 minute (or slightly more or less) assessment (scored by either human or machine) is not enough to produce a good assessment (101 and 103).

Assumptions: Machines can offer superficial feedback on things like grammar and syntax because they can count, but they are of no use until they can understand and assess content (102). Using them pervasively is unwise (102)

Points of interest:

  • We’re in danger of compromising what is a rich construct for the sake of profit. What is that going to do to our students?
  • Condon’s great validity table/chart is on page 104.
  • He suggests that placement should be non-vertical; that is, instead of sorting students into places on a curricular totem pole-like spectrum (with remedial at the bottom, traditional in the middle, and honors at the top), Condon advocates a system where we adhere more to a traditional curriculum, and give remedial students supplemental work and help (106).
Advertisements
 
Leave a comment

Posted by on November 14, 2013 in Uncategorized

 

“Writeplacer Plus in Place: An Exploratory Case Study” by Anne Herrington and Charles Moran (from Machine Scoring of Student Essays: Truth and Consequences, pages 114-127)

Rank: 2-3 (Herrington and Moran are suspicious enough to be opposed to the decision to implement Writeplacer, but they do see its upsides (cost/efficiency).

Argument: The researchers are convinced of Writeplacer’s cost-effectiveness as a placement exam scorer (or, at least, they see how it is a cheaper and more efficient method than hand-scoring), but they are troubled by the removal of human assessors for multiple reasons.

Assumptions: This is an interview-based case study of the implementation of Writeplacer Plus (as an entrance-level sorting device for newly-registered students) at Valley College). The researchers interviewed administrators, teachers, and students about the use of Writeplacer. Writeplacer, in this situation, was used as a one-time test (meant to act as a filter), and so, the administrators see no problem using it as a filter (since it’s not functioning within the classroom itself).

Points of Interest: 

  • The core (and fascinating) problem that the researchers isolate is that the students are writing not to humans, but to computers (a problem that extends beyond this study to the greater realm of AES) (114). They wonder how knowledge of this different audience could affect students’ rhetorical practice.
  • Administrators and faculty have distinctly opposed opinions about Writeplacer (the former think of Writeplacer as a great “filter” (124); the latter are suspicious and don’t want to be blindsided (because when they were hand-scoring exams, they could get an idea of what kind of writing they would have to work with). Students were more likely to shift between those positive and negative poles of opinion. Overall, they were shocked (specifically, some approved, some disapproved, but all wanted actual human teachers in their courses).
  • No matter what, everyone involved (essentially) was in favor of keeping AES out of the actual classroom (even though Writeplacer Plus was justified by administrators as a one-time “filter”).
 
Leave a comment

Posted by on November 14, 2013 in Uncategorized

 

21st century digital literacies (in the context of general ed)

21st century digital literacies (in the context of general ed): what are they?

21st century digital literacies include composing in word/Google docs/or other digital spaces that allow for fairly instantaneous formatting (of documents and images, digitally); part of 21st century digital literacies is the ability of a student to make a decision about which mode of composition fits whichever project the student has been assigned.

To grow this ability, teachers should offer more options to students in terms of composition modes. Are we preparing students for academia? Not really, but we should be preparing them for life/the other classes they will take in college. Also, We should try to acknowledge, or even legitimize, everyday writing in digital contexts, especially writing that happens outside the classroom, like Dr. Neal says on page 78.

When/where are they learned and /or assessed?

Right now multimodal assignments are a very small part FYC curriculum; the students have the digital studio here, but in the classroom multimodal assignments are only given a few weeks’ attention at the end of the semester. Do we need to change this? And if so, how would we be able to assess multimodal assignments?

If we think that multimodality is so important to 21st century digital literacies, we have to figure out the practical application for it outside the composition classroom.

Should we expect FYC teachers to know software that allows students to compose in multimodal ways? What kinds of things are we asking our students to do? Why? What about WAC? Are we teaching students to be able to write to meet requirements in other classes? We think so. And if so, then we need to give them the tools to be able to do that, and multimodality is included in this.

Reflective work would be an ideal way to assess rhetorical skills and decisions in this kind of context. Reflections can measure not only how well they approached the assignment but also how they feel about their own performance.

We also think a good starting point, in terms of creating a short rubric, is on page 96 of Dr. Neal’s book- we would want to look at depth, persuasiveness, relationships between modes, media, and texts, relationships to traditional conventions organization, arrangement, progression, attention to audience and context.

-Josh, Erin, Andrew, and Kendall

 
Leave a comment

Posted by on November 5, 2013 in Uncategorized

 

Chapter 7: “Showcase Hybridity: a Role for Blogfolios” and Lam’s article “Two portfolio systems: EFL students’ perceptions of writing ability, text improvement, and feedback”

From ePortfolio Performance Support Systems I read Chapter 7, which is titled “Showcase Hybridity: a Role for Blogfolios.” Middlebrook and Sun are essentially claiming that it would be good to combine blogging practices with ePortfolio practices in classroom settings and professional settings.  They say that ePortfolios are static mediums, and that blogs are, in contrast, dynamic and could allow for customization of the “evidence showroom,” or the ePortfolio. They say blogs can also help express digital identity, and can facilitate web communication between students and higher-ups in the field. Specifically, then, blogs could potentially help students gain a sense of self and grounding in the field they choose. A blogfolio could be (and so far, has been) a great professional tactic.

Middlebrook and Sun lay out some obstacles for using a blogfolio. First, it would be difficult to train teachers in how to use them. Second, we would need to decide how to assess them. Third, we would need to invent assignments. Fourth, the whole process could lead to information overload for both student and teacher. Fifth, blogs are met with skepticism in academia.  Sixth, there is, to a certain extent, a permanency of web entries; so, if a student writes something that is factually incorrect, irrational, too emotionally-driven, or unintelligent, there is a slight danger that it will haunt them in ways paper couldn’t. But Middlebrook and Sun communicate that the pros here outweigh the cons easily, especially since this kind of online composing could allow for professional contact in new and exciting ways.

 

Lam’s article “Two portfolio systems: EFL students’ perceptions of writing ability, text improvement, and feedback” seeks to find out how a select group of Chinese students responded to two different types of portfolio assessments. Lam mentions that the assessment environment in China is centered around examinations and products. Students are primarily interested in finding out what errors they make, though they do also want to know about how they are doing in terms of craft, organization, etc. The research for this article was conducted in a second semester writing class; the portfolios that were in question were composed of twelve written drafts (three for each of the four genres), and there were two types of portfolios: working and showcase. The students had to revise their drafts using self, peer, and instructor feedback.

This was a qualitative experiment seeking out detailed results; the information was gathered through interviews, examinations of reflection journals created by the students participating in the study, classroom observations, and analysis of the students’ texts. Lam’s findings section indicates that the students in Group A (the group trying the working ePortfolio) preferred (and really valued) instructor feedback more than peer feedback in earlier drafts, especially in terms of teaching them how to perform revisions. They said that, since peer feedback was less error-finding-oriented and more big-picture/clarity/paragraph development-oriented, it would be best saved for later drafts.

The students in Group B (the group trying showcase ePortfolios) were concerned that there was too much focus on the finished product rather than the process of writing itself. They also did not seem to trust, as easily, self and peer review, and they were more prone to procrastinate. Lam has several very helpful graphics that show the numbers involved with this study (and through them, he gives you many specific examples that illustrate his generalized claims). I encourage you to have a look at them if this study interests you.

 
1 Comment

Posted by on October 29, 2013 in Uncategorized

 

Introverts and Extroverts in Composition Learning Environments

Intro

Like Jeff, I’d also like to think like Penrod, who wonders whether students feel welcome in learning environments and whether or not that affects their performance or involvement as composers in the learning environment. But instead of studying whether or not students feel welcome, I would like to study which pedagogical methods appeal to extroverts and which ones appeal to introverts (that is, I want to know which methods get them to be interested in and engage with the materials as well as their classmates).

Hypothesis

Extroverted students will be more engaged by gregarious activities like group work, collaboration, and discussion, etc. Introverted students will be more engaged by more independent activities, like free-writing, individual brainstorming, and conferencing.

Methods

We would need notably diverse activities, two groups of students (one introverted, one extroverted; possibly categorized through Myers-Briggs or something equivalent), and a survey system that we could use to collect data and debrief.

Data Collection

Each student would participate in each activity, give feedback as to whether they liked it, felt engaged/inspired by it, etc., using a measurable scale, and then I would look for correlations in the data that would tell me which student type preferred which activity.

Discussion

This study (which would assume that learning environments favor extroverted students) would be geared toward a hybrid pedagogy that would consist of mixed methods that could be used (in a “happy medium” sort of way) to engage both personality types evenly, and exclude none (also, this study proposal is inspired by the (scientifically-sound) ideas of Susan Cain: http://www.ted.com/talks/susan_cain_the_power_of_introverts.html).

 
2 Comments

Posted by on October 14, 2013 in Uncategorized

 

In-class thoughts on education, writing, and assessment in the future

What is changing about education and where is it going?

  • We could be going in Raschke, Brown & Duguid route where everything goes online.
  • There will probably be a more diverse demographic (not 80% white anymore) (depending on which part of the country- more low income, English language learning students
  • Oregon state has asked the question, “how can we diversify?” Assessment might begin to reflect the same question. How do you make sure that the school serves the entire population?
  • There might be a greater move toward standardization, which is a move of fairness (birthed from No Child Left Behind; test-based accountably- initiation of a core curriculum).
  • The economy might improve, then we’ll have more money for humanities.
  • We’ll trend toward super-specialization; broad disciplines like sociology will split into much smaller, more specific disciplines. Professors will ask themselves what makes them distinct. More interdisciplinarity because of this ^
  • Composition might be delivered in different ways; that might coalesce into a situation where a student might be enrolled at FSU and enrolled via FSU online (the distinction will still be there though)
  • Online learning (for general education courses) will lose some of its stigma- because it will be more prevalent (with the exception of institutions like Phoenix and DeVry); hybrid courses will become more commonplace. (But there are certain degrees you just cant get online, like a med degree.
  • Instead of undergrad prep, high school students will become so accelerated that they’ll already have undergrad focuses and maybe even be preparing for grad school. They won’t enter college with an undeclared major. A lot of general education will be done in high school and by the time they enter college they’ll at least be ready to start work in their major.
  • Ivy leagues will become reserved for graduate students because everyone needs a college degree and the graduate degree is the new undergraduate degree.

Assessment predictions:

  • The SATs might become obsolete within the next 50 years.
  • Writing assessments, instead of being used to judge just writing, will be used to assess other things, like experiences with medicine, communities, leadership, their character traits.
  • If we move online, like Raschke suggests, how will that change our assessment practices? We’ll probably implement progress tracks. This might be tied back to the instructors.
  • We might trend toward using contracted module assessments (when instructors/administrators use a program like Blackboard as a portal to models where students are guided through graded online courses) in an effort to help students progress.
  • There might be a possible implementation of exit assessment to test learning and performance in students’ college tracks.

-Joe, Sarah, Jacob, Jeff, Amy, and Kendall

 
Leave a comment

Posted by on September 24, 2013 in Uncategorized