The Comprehensive and Compensatory System Testing Philosophy
The Comprehensive
and Compensatory
SYSTEM Testing
Philosophy
The Problem with Cognitive Ability Testing
When selecting a test that will enable you to identify the individuals who are most likely to succeed on a particular job, you have a number of types of tests from which to choose. Cognitive ability tests are commonly used because they are generally considered to be the single best predictor of job performance (Schmidt & Hunter, 1998). But this high level of validity comes at a cost—cognitive ability tests also tend to demonstrate a high level of disparate impact. Disparate impact exists when the selection rates between two protected classes of applicants (e.g., males versus females; African Americans versus whites) differ significantly. Cognitive ability tests commonly result in very large performance differences between certain protected classes. For example, performance differences of one standard deviation or more are typical between African American and white applicants on cognitive ability tests (Sackett & Wilk, 1994).
Disparate impact drastically reduces our ability to select a diverse group of applicants. Especially in the public safety industry, where is it important for the police and fire department workforce to reflect the racial composition of the community it serves, it is necessary to identify testing strategies that will minimize disparate impact but will still allow for the selection of qualified applicants. Indeed, it is the burden of employers to use tests that are highly valid but result in the slightest disparity possible. For this reason, it is the goal of IO Solutions to develop entry-level selection tests that maximize validity levels while minimizing disparate impact outcomes. This was IO Solutions’ central mission when the company was started over a decade ago, and this remains a primary focus today.
The Solution for Reducing Disparate Impact
The solution to the high-validity/high-disparate impact problem that is presented by cognitive ability tests is to create a test battery that will capitalize on the validity of the cognitive predictor while suppressing disparate impact. This can be accomplished by introducing a testing solution that is valid but displays little to no disparate impact. Personality tests are well suited for this purpose. Research has shown that personality tests generally do not display disparate impact, and many personality constructs, such as conscientiousness and integrity, are excellent predictors of job performance (Hogan, Hogan & Roberts, 1996).
In addition to combining a less disparate test with the cognitive ability test, we also have to combine the scores of the two tests in a manner that will create the benefit we are seeking to achieve (Ryan, Polyhart, & Friedel, 1998; Sackett & Ellingson, 1997). A compensatory scoring system combines scores on two testing tools in such a way that high performance on one tool will balance lesser performance on the other tool and vice-versa. The two scores are averaged, or combined, using an appropriate weighted average that can be based on the results of a job analysis study. This compensatory system allows us to maximize the ability of the personality test to balance the disparate impact of the cognitive test. In contrast, it is not ideal to establish an individual cut-off score for the cognitive test prior to combining scores on the cognitive and personality components or to administer the personality test only to those who pass the cognitive ability test. These solutions defeat the goal of reducing disparity.
The outcome of combining cognitive and personality predictors is a comprehensive and compensatory system that maintains high levels of validity while minimizing disparate impact. The process is comprehensive because it assesses a wider array of predictors than just cognitive ability. The process is compensatory because it allows candidates with better-suited personalities to compensate for lesser cognitive ability scores.
The Benefit of Compensatory Testing Systems
The clear advantage of compensatory testing models is that they maintain high validity levels while reducing disparate impact, but the advantage is even greater. The Equal Employment Opportunity Commission (EEOC) suggests a rule of thumb for gauging disparate impact called the 4/5th Rule. If the ratio that describes the passing rate of the minority group in comparison with the passing rate of the majority group is less than .80, or 4/5th, the testing process is said to display adverse impact. Generally, cognitive ability tests will result in adverse impact, but the larger problem is that a disproportionate number of racial minorities will achieve scores at the lower end of the score distribution. This means that even if these candidates pass the test, they will not likely be selected for employment. Besides balancing the passing rates of protected classes, compensatory testing systems also create a more balanced distribution of test scores, resulting in a greater proportion of racial minorities scoring towards the upper end of the distribution. Practically speaking, more racial minorities will be selected. This system not only has a greater chance of satisfying the EEOC’s 4/5th Rule, it also accomplishes the important agency objective of hiring a diverse group of qualified employees.
Questions? Contact Us!
