Monthly Archives: September 2013
NOTE: Because of the size of the file, we’re linking to it here instead of embedding it! Enjoy!
Also: web-based full-screen version
Assessment Timeline Process Memo
Jason Custer and Kendall Parris
For our timeline project, we wanted to discuss some of the unique features that belonged to certain machine scoring systems. These different features can, arguably, constitute a kind of technological identity that is unique to each system; thus, we spent a good while trying to figure out the best way to formulate a visual that would represent both a time spectrum and the technologies’ different “identities” simultaneously. After some brainstorming, and a brief recollection of a similar historical parody one of us saw on the internet some months before (you can see it here: http://www.collegehumor.com/facebook-history), we decided that creating a spoof Facebook wall (including a series of posts and discussions) could provide us with the ability to visualize both the advent and individuality of each of the ten machine scoring technologies we chose to represent.
In an effort to create believable Facebook posts, we incorporated Facebook vernacular, emoticons, carefully-selected profile pictures, hashtags (because for some reason Facebook has integrated them…), et cetera into our project. We also tried to isolate the core concept or ability that individualized each technology so that we could use it in a post to define that particular system as a kind of Facebook “personality.” Sometimes we put the systems into conversations with each other in the comments box to help elucidate a particular system’s function. (and sometimes we did this just for fun). Also, instead of “likes.” we made it so that that section of the post relayed validity percentages.
Additionally, to represent the relationships between software (“better,” “worse,” “more/less effective,” etc.) we found traditional means of representation (proximity, size, color) lacking, since these measures do not immediately visually indicate the benefits of each piece of software. As a result of choosing the Facebook theme for our work, we were able to use a somewhat discrete method to signify “performance” by using Facebook’s built-in, already somewhat vague “like” system. By equating Likes to Performance, we were able to visually represent an otherwise abstract concept for which traditional visual arrangement seemed lacking. In this way, like the measures of performance and validity themselves, the numbers are presented and left for the reader to determine the value of beyond the simple numbers presented here.
Bibliography and Works Cited
Burstein, Jill, Claudia Leacock, and Richard Swartz. “Automated Evaluation of Essays and Short Answers .” Loughborough University (2001): 1-13.
Hearst, Mari A. “The Debate on Automated Essay Grading.” IEEE Intelligent Systems & Their Applications 15.5 (2000): 22.
Herrington, Anne, and Charles Moran. “What Happens when Machines Read our Students’ Writing?” College English 63.4 (2001): 480.
Jordan, Sally. “E-Assessment: Past, Present and Future .” The Open University: Pedagogic Directions: 1-20.
Leacock, Claudia, and Martin Chodorow. “C-Rater: Automated Scoring of Short-Answer Questions.” Computers & the Humanities 37.4 (2003): 389-405.
Mitchell, Tom, et al. “Towards Robust Computerised Marking of Free-Text Responses.” Loughborough University (2002): 233-249.
Valenti, Salvatore, Francesca Neri, and Alessandro Cuchiarelli. “An Overview of Current Research on Automated Essay Grading.” Journal of Information Technology Education 2 (2003).
Werner, Gergory J. “A Complete Approach to Automated Essay Grading .” George Washington University: Department of Computer Science: 1-6.
Zhang, Mo. “Contrasting Automated and Human Scoring of Essays.” Educational Testing Service: R & D Connections 21 (2013).
I made my timeline on the website Dipity: http://www.dipity.com/amypiotrowski/Assessment-for-Freshman-Comp/
For my assessment history timeline, I looked at the history of Advanced Placement English. The most interesting trend I saw was the shift from AP being seen as elitist to being seen as a way to get ahead in college. AP started as the idea of elite colleges and private secondary schools. Many students at elite colleges reported being bored by their freshmen coursework because these courses covered material they learned in high school. The solution school administrators came to was to design advanced courses and exams so students could enter college with advanced standing. For its first 20 years, AP was seen as an elitist program, not for most students. After 1983’s Nation at Risk, several reports and commissions praised AP. The program grew as students began to see AP as a way for anyone to get college credits. Laws passed by state legislatures in support of AP continued that growth. By the 1990’s, AP courses became almost a necessity on the transcript of anyone hoping to get into college. In 2007, the synthesis question was added to the AP English Language and Composition Exam to make the tasks on the test more like the tasks students do in a freshman composition course. This past spring, 476,277 students took the AP English Language and Composition exam, a huge increase over the 377 students who took the first AP English Composition exam in 1955.
I agree with Dr. Neal’s argument that freshman composition will increasingly be taught in high schools, not in colleges. I wonder if high school students will take AP English Language and Composition courses or take dual enrollment courses. Which way of teaching freshman composition in the high school will be dominant in 10 or 20 years? Or will school districts try to combine AP and dual enrollment in one course, as the Texas district I taught in was beginning to do when I left for FSU?
AP Central – The English Language and Composition Exam. Retrieved from http://apcentral.collegeboard.com/apc/members/exam/exam_information/2001.html
AP Course Audit – Online Learning. Retrieved from http://www.collegeboard.com/html/apcourseaudit/district_approach2.html
AP Program Participation and Performance Data 2013. Retrieved from http://research.collegeboard.org/programs/ap/data/participation/2013
Jones, J. (2001). Recomposing the AP English exam. English Journal, 91(1), 51-56.
Rothschild, E. (1999). Four decades of the Advanced Placement program. The History Teacher, 32(2), 175-206.
Schwartz, J. (2004). Confessions of an AP reader. English Journal, 93(4), 53-57.