Recruitment has never been more challenging; we think most people would agree with that statement. Once we finally have our shortlist, how do we ensure the selection process we are utilising is going to be predictive of performance? Organisations often rely on various predictors to assess a candidate’s potential for job performance; but are these the right predictors?
Our expert Omni Assessment team has explored findings from new research conducted by Sackett et al. (2022; 2023), which delves into the validity of these predictors in candidate selection. This research builds on the seminal work of Schmidt and Hunter (1998) and provides valuable insights into the effectiveness of different predictors, including personality traits, emotional intelligence, situational judgement tests (SJTs), and cognitive ability, in predicting overall job performance.
Understanding the Study
We’re going to get into the details here; it’s important!
Schmidt and Hunter’s work on the relationship between predictors in the candidate selection process and job performance has been highly influential in the field of talent acquisition for over two decades. Most HR professionals have been quoting this study for years based on the immense attention it received, with over 6,500 citations. Building on this foundation, Sackett and his team conducted a critical analysis of existing research studies of candidate selection tools.
In their research, Sackett and colleagues found certain flaws with the statistical adjustments used in the earlier study, and as a result, conducted their own analysis. When doing so, they expanded the range of predictors studied beyond the original research work, including the Big 5 personality traits, emotional intelligence, and SJTs. Additionally, Sackett and his team addressed concerns for diversity and equitable outcomes in selection processes by including data on White-Black mean differences.
Key Findings
The central observation from the study is that commonly used predictors, on average, exhibit lower operational criterion-related validity for overall job performance than previously acknowledged. This implies that the factors we typically evaluate in candidates seem to have less accuracy in predicting job performance than we had previously assumed. Notably, even within their latest analysis, most rankings in terms of predictive capability remained consistent, with one of the most substantial shifts occurring in the ascendancy of structured interviews as the foremost individual predictor.
As a result of this, cognitive ability, which was previously considered a standout predictor, no longer holds the same prominence. Amazing results to think about, considering 75% of Fortune 500 organisations reportedly use cognitive ability tests as part of their selection process.
Key Implications for Talent Acquisition
The research has important implications for talent acquisition practices:
- Tailored Selection Systems: The findings underscore the value of selection systems tailored to the specific requirements of each role, rather than relying on generic, off-the-shelf solutions. By tailoring the selection approach, organisations can enhance the validity of their candidate selection process, leading to better hiring decisions and improved job performance.
- Job-Specific Predictors: Even within a tailored process, the results highlight the importance of using job-specific predictors, such as structured interviews, job knowledge tests, and work sample tests, in terms of predictive validity. These assessment tools appear to be more effective in predicting job performance compared to more generalised psychological constructs like abilities and personality traits, as they take into consideration the unique competencies and demands of the position.
- Addressing Diversity: Notably, the study’s incorporation of data on White-Black mean differences in selection predictors underscores the crucial consideration of diversity in talent acquisition. According to the analysis done by Sackett and colleagues, even amongst the most potent predictors, some predictors display large group mean differences, e.g. work samples, job knowledge tests, and cognitive ability tests. In contrast, there are other strong predictors, including structured interviews that show much smaller mean differences. These findings reinforce the value of employing multiple predictors to mitigate group differences and promote fair outcomes in their selection processes.
Cognitive Ability: A Revised Perspective
The revised validity of cognitive ability tests has come as quite a shock for many in the industry, challenging the commonly held view that cognitive ability has high predictive capabilities. However, as pointed out by recent studies, the analysis done by Schmidt and Hunter relied on much older data (Griebe et al., 2022). In their own meta-analytical study, Griebe and colleagues found that cognitive ability, or the ‘g factor’ presents a significantly lower degree of accuracy in predicting job performance (r= .23 rather than .51).
At this point, many may be thinking, “So, what is the point of having cognitive tests?”, but when deciding on the effectiveness of an assessment tool, we also need to consider the wider context.
- Importance in High-Volume Screening: Firstly, many of the strong predictors, i.e. structured interviews or work sample tests are less adaptable for high-volume screening scenarios, e.g. graduate schemes. In such situations, cognitive ability measures, along with other general assessments, such as personality and emotional intelligence assessments, can prove to be extremely useful tools.
- Predicting Learning Performance: Most of the predictors that yielded strong correlations are viable only when candidates have prior experience and/or training in specific job-related skills, such as work sample tests or job knowledge assessments. However, cognitive ability continues to excel in predicting performance in learning contexts, such as higher education or settings where high levels of training are needed. When significant investments in training are involved, cognitive tests can be the best available predictor, based on current literature.
- Considering Situational Relevance: Sackett and colleagues emphasised the importance of distinguishing between typical and maximum performance. Maximum performance characterises the highest achievable level of output under optimal effort conditions, while typical performance encapsulates the average output sustained over an extended period. Since typical performance is influenced by factors such as effort, procedural intricacies, cultural dynamics, and external influences, the correlation between cognitive ability and performance is expected to be less pronounced in comparison to the association seen for maximum performance. This underscores the critical need to align the selection system with the precise performance outcome being targeted. Depending on whether the selection system aims to measure maximum performance or typical performance, this decision dictates the weightage attributed to cognitive ability.
Conclusion
The comprehensive analysis conducted by Sackett and his team challenges prior assumptions about predictor validity in candidate selection. Although some predictors may not be as strong as once believed, the paper underscores the significance of considering the wider context when selecting a tool (e.g. with cognitive tests) and using the contextual information to tailor the selection process, aligning it with performance objectives. It’s also crucial to recognise that these findings pertain to individual predictors in isolation, and so the study emphasises the value of integrating multiple predictors, which would also, to an extent, address certain diversity concerns. Implementing these insights can lead to more effective talent acquisition and improved organisational performance.
Contact us for an informal chat if you’d like more information on this research, to find out more about the best tools for candidate selection and/or creating your own bespoke assessment programmes.
T: 0161 929 4343
E: insights@omnirms.com
