Standards for Demonstrating Content Validity Evidence
Standards for
Demonstrating
Content
Validity
Evidence
What is Content Validity Evidence?
Evidence of content validity generally “consists of a demonstration of a strong linkage between the content of the selection procedure and important work behaviors, activities, worker requirements, or outcomes of the job” (Principles, 2003). In other words, a test is content valid to the degree that it “looks” like important aspects of the job. Of course, the process of demonstrating that a test looks like the job is more complicated than making a simple arm’s-length judgment. Inferences of job-relatedness are made based on rational judgments established by a set of best practices that seek to systematically link components of a job to components of a test.
When is it Appropriate?
A test can be supported by content validity evidence to the extent that the construct that is being measured is a representative sample of the content of the job or is a direct job behavior. Tests that assess job knowledge, supervisory skills and communication skills would be appropriate to validate with content validity evidence; however, tests that assess aptitude, personality, or more nebulous and multifaceted constructs like these should not be validated using content evidence.
Method of Establishing Content Evidence
In order to establish evidence of content validity, one needs to demonstrate “what important work behaviors, activities, and worker KSAOs are included in the (job) domain, describe how the content of the work domain is linked to the selection procedure, and explain why certain parts of the domain were or were not included in the selection procedure” (Principles, 2003). Further, it must be demonstrated that the selection procedure that measures a skill or ability should closely approximate an observable work behavior, or its product should closely approximate an observable work product (Uniform Guidelines, 1978). The method used to accomplish this goal involves a number of steps:
1. conduct a job-task analysis to identify essential job tasks, knowledge areas, skills and abilities;
2. link job tasks, knowledge areas or skills to the associated test construct or component that it is intended to assess;
3. use subject-matter experts internal to the department (where possible) to affirm the knowledge or skills that will be assessed in the test and the appropriateness and fidelity of the questions or scenarios that will be used (these can be accomplished in a number of ways, including the use of content-validity ratios [CVR] – systematic assessments of job-relatedness made by subject-matter experts);
4.document that the most essential knowledge areas and skills were assessed and explain why less essential knowledge and skills were excluded.
“The documented methods used in developing the selection procedure constitute the primary evidence for the inference that scores from the selection procedure can be generalized to the work behaviors and can be interpreted in terms of predicted work performance” (Principles, 2003). Therefore, the technical report that is used to document the methodology employed to develop the test is sufficient to serve as the evidence of content validity.
Rank-Ordering Candidates based on a Content-Valid Selection Procedure
In order to use rank-ordered selection, a test user must demonstrate that a higher score on the selection procedure is likely to result in better job performance. “Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance” (Uniform Guidelines, 1978). To the extent that the scoring system awards points based on the demonstration of knowledge or behaviors that distinguish between minimal and maximal performance, the selection procedure is likely to predict job performance.
Questions? Contact Us!
