Possible Assessment Study – Teacher Experience and Assessment

03 Oct

We’ve talked about gender and race, but I’m curious as to whether whether teachers’ assessment of student writing is affected by their age and teaching experience.  Do younger teachers assess writing differently from older teachers?  Do preservice teachers, early career, and veteran teachers assess writing differently?  I’d define a preservice teacher as a teacher candidate in an certification program, an early career teacher as a teacher with less than five years experience, and a veteran teacher as a teacher with more than five years experience.

Hypothesis: My sense is that veteran teachers, having seen a lot more student writing, may be more tolerant of grammatical features that they know are typical of the grade level they are teaching.  But veteran teachers may also be more intolerant of certain errors related that don’t bother younger teachers as much, for example text speak.

Methods: I’d modify Ball’s methodology.  I’d give the same group of essays to preservice, early career, and veteran teachers.  Each group would be trained on the same rubric to assess the student essays, similar the Ball’s.  Then, the study could compare holistic ratings and ratings on organization, coherence, and mechanics.  Are the mean score for the holistic rating, organization, coherence, and mechanics different depending on whether the piece was scored by a preservice teacher, an early career teacher, or a veteran teacher?


Posted by on October 3, 2013 in Uncategorized


7 responses to “Possible Assessment Study – Teacher Experience and Assessment

  1. brucebowlesjr

    October 8, 2013 at 6:32 pm


    I think there might be a lot of truth to your hypothesis–it seems rather logical.

    What about other facets of writing such as content and organization? How do you think age difference might influence assessment of writing in regard to more global criteria?

    I’d contemplate examining multiple facets of writing unless you desire to isolate the grammatical variable.

  2. jc12t

    October 9, 2013 at 3:41 pm

    You know, I think this is a pretty important question–and I’m not just saying that. I think we often assume that older generations are focusing on the wrong stuff–like they’re stuck in the days of formalism. But how true are those assumptions? Jacob and I were talking about a similar type of project before we chose the direction we did. We were talking about whether older generations of teachers will assign multimodal projects in an attempt to “get with the times,” but that they would grade more forgivingly because they, themselves, are probably impressed by any digital project they get. On the other hand, a younger instructor–more familiar with how digital projects are made–may be a more critical grader. I don’t know if that can be applied to your project, but I think it’s worth considering if you wanted to do something with digital composing.

  3. jasonecuster

    October 10, 2013 at 12:43 pm

    Comment: As someone slowly moving my way through having more experience with teaching and questioning my own assessment practices now more than ever, I would certainly be interested in knowing the results of this.

    Question: Do you think it would be more helpful to follow instructors along their own careers, or take samples of newer/older teachers? I see both having some obvious downsides for the results, but I wonder which would be more helpful.

    Suggestion: That said, I would recommend considering this as a… what are they called… longitudinal (…?) study. I think it would be interesting to see how an instructor evaluates the same assignment with the same rubric in perhaps the first semester of teaching in a PhD program like we have here in the English department, and maybe again in the final year or final semester, to see what has shifted, perhaps (and to have a stable variable: the same person evaluating, maybe even have them evaluate the same paper again later, once they forget what it looked like). Or first year teachers at a new university/school and seeing how their approach changes, say, 5 years later.

  4. sarahm1320

    October 11, 2013 at 7:04 pm

    I definitely think this could be a great study, especially with all of the conversations lately about fairness and equality of education. Are students being assessed and taught with similar criteria, regardless of the experience level of their teacher? This problem becomes even more relevant when we consider the high levels of turn-over for schools in low-income neighborhoods, causing them to have a disproportionate number of new teachers.

    Have you thought about the effects of different training backgrounds? i.e., teacher training programs have different emphases, thereby introducing another variable into your study.

    Something that I think might be interesting is to not train the teachers to use the same rubric – rather, they would simply be instructed to assess the composition based on the criteria that they normally use. That way, the results would be a more organic expression of any differences that they have in evaluating student writing. Training them to use a rubric would still most likely reveal these differences, since different teachers interpret rubrics differently, but it would be to a lesser degree. In summary, think you might find more interesting, descriptive, qualitative results without a standardized rubric.

  5. E Workman

    October 14, 2013 at 5:55 am

    I also agree that this study would be really interesting, and your hypothesis seems plausible, especially given what Ball writes about newer teachers having more liberal leanings than more experienced teachers. I’m also reminded of Sarah McCarthy’s study on the effects of NCLB on teachers’ practices because she also looked at teachers with a range of experience. (Article here:

    Sarah’s question and suggestion both make a lot of sense to me. Looking at teacher training programs could be really interesting, though I imagine that it could also complicate the methods. One question that I’ve wondered about in relation to post-secondary education is the effects of teacher preparation on part-time faculty’s teaching practices. Do those who are trained in year x continue using the same practices 10 years later, for instance? I’m not certain if this question would work in the K-12 context, though.

    Sarah’s idea to have teachers assess the texts using the criteria they normally do also seems like a useful idea, though I imagine that this could make comparing responses somewhat difficult as they are presumably going to focus on different aspects of the text. I wonder if you could get at this in interviews with teachers–rather than having them use their “normal” criteria, you could still have them use rubrics and then later reflect on how that rubric deviated from their usual assessment practices.

  6. andrewdavidburgess

    October 15, 2013 at 2:58 pm

    Comment: I would actually pose the opposite hypothesis. I know my mom is a first-year comp teacher and she and I always get in debates because she gives her students terrible grades based on grammar.

    Question: How old is old?

    Suggestion: My only suggestion would be to solidify your categories of teachers. Are graduate TAs preservice? How long does it take to become a veteran?

  7. jacobwcraig

    October 20, 2013 at 1:43 am

    Sort of along Andrew’s lines, I think that assessment isn’t necessarily an age thing. It feels more like a question of whether or not the teacher has been mentored in comp studies, whether the teacher identifies with a specific comp studies theory, or maybe what generation of CCCCs the instructor belongs to (assuming the teacher does comp research). In other words, I feel like this is more about how they understand writing intellectually and the ways that their understanding influences their concept of writing.

    Jason has a smart idea about this study: doing a longitudinal study of a teacher from their first semester til you have to discontinue the study. There’s something like that in a book called *Misunderstanding the Assignment*, but it is a different study with a different question. The methods could be valuable to you.


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: