Validity and Reliability

According to Samuel Messick, validity is

an overall evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of interpretations and actions based on test scores or other modes of assessment (Validity 1989, italics in original).

Lee Cronbach says of test validity

One validates, not a test, but an interpretation of data arising from a specified procedure (Test Validation 1971).

Lorrie Shepard  (Evaluating Test Validity 1992) explains the usefulness of the many types of validity being merged into unified construct validity. She also points out that since the 1940s, “validity is used synonymously to mean predictive correlation coefficient” or “a test is valid for anything with which it correlates” (410). The primary types of validity she unpacks is her article are

• Content validity: test’s describing an individual’s performance on a defined universe of tasks
• Construct validity (not unified): needed when making inferences about unseen traits such as intelligence or anxiety.
• Predictive validity: test used to predict future performance, which required collecting criterion data later than the test
• Concurrent validity: test’s ability to substitute for a less convenient measure that was already accepted
• The last two (both related to outside criteria) were combined to form criterion-related validity

Pamela Moss (1994) importantly challenges the stranglehold reliability has on validity:

I now return to my title, “Can there be validity without reliability?” When reliability is defined as consistency among independent measures intended as interchangeable, the answer is, yes. Should there be? Here, the answer is, it depends on the context and purposes for assessment (10).

Let’s look at these (re)conceptions of validity and reliability in light of our readings for this week.