Nearly every business, regardless of size, industry, or geography, must overcome the same universal challenges.
First, they must be able to work efficiently, minimizing waste and needless overhead that eat into profits. They also need to maximize productivity, implementing processes, tools, and equipment that allow them to get the most out of the resources they have available. Most importantly, they must be able to hire and keep the right people.
The difference between category leaders and struggling competitors often boils down to a disparity in talent. Without highly qualified, productive employees who can work creatively, independently, and efficiently, a company’s long-term prospects are bleak. As global economic activity rebounds in 2022 and firms in nearly every sector ramp up hiring activity, this is one of the challenges in hiring that looms largest. Today, many employers find themselves with all the sophisticated tools and proven processes they need to be successful — but without the people best-equipped to create the highest-quality products and most impactful customer experiences.
At Biddle Consulting Group, our goal is to help them find this key missing ingredient. We work with employers to develop and administer valid and reliable pre-employment assessments that dramatically improve their odds of identifying and hiring the best candidates for every role. In our experience, many employers view talent acquisition as one of the easiest aspects of the human resource cycle. They trust their intuition when it comes to people and prefer to allocate resources to HR functions that they believe are more vital to business operations.
The implications of this approach to hiring are twofold. First, it breeds inefficiencies that translate into direct costs. Prior to the global health crisis, American employers spent more than $4,000 per job, on average, by the time a candidate was selected. Secondly, it leads to numerous (and higher) indirect costs, as employers miss the opportunity to reduce expenses associated with training, turnover, and inadequate performance.
Every company faces resource constraints, and employers often ask us how to account for these when making hiring decisions. This blog post will answer that question and help you make improvements to your own hiring process.
Developing an Effective Testing Framework
A growing number of employers are using various screening tests and pre-employment assessments to measure candidates’ suitability for a role. Oftentimes, however, the results of these tests provide little indication of a candidate’s real potential.
To achieve the goal of predicting job performance, test questions must be deliberately and thoughtfully developed with that aim in mind, and then evaluated as part of a test validation process. In this context, “validity” refers to whether a test is measuring what it is designed or claims to measure and does so in a manner that is fair to all test takers.
Many employers are unsure how to measure test validity and skip this step altogether. But this could expose them to regulator scrutiny or legal action. Others simply assume that all information is equal and design a uniform employee examination that relies on job duty statements or class specifications. Unless they take time to gather the evidence required to document test validity, they’ll continue administering tests that are neither reliable indicators of performance nor fair to test takers and later wonder why new hires aren’t panning out.
There are three basic approaches to establishing test validity outlined in the Uniform Guidelines on Employee Selection Procedures, which all organizations must adhere to when making hiring decisions (more on this later).
- Content Validity: The content of a pre-employment assessment should represent the critical content of the job in question. To establish content validity, an employer should first develop a complete understanding of the requirements of the job by conducting a job analysis. This analysis must rely on more than vague and unsubstantiated employee statements about the role and go beyond a simple review of job descriptions or classification specifications (which represent the ideal of the job as conceived by the employer, rather than objective reality). A valid job analysis identifies the critical tasks performed in the course of the job as well as the knowledge, skills, and abilities required to perform them successfully. A valid test will then assess an applicant’s possession of the particular knowledge, skills, and abilities required upon entry to the position. Thus, linking test content to objective data on job requirements is the basis of content validation.
- Criterion Validity: Unlike content validation which is more of an administrative and procedural approach to establishing validity, criterion validation is highly statistical. Ideally, the employer will base the initial test development on a job analysis, but the key to criterion validation is demonstrating a statistically significant relationship between performance on a test and performance on the job. The statistics involved in establishing criterion validity are beyond the scope of this article. However, the general approach goes like this. First, an employer would administer the test to either current employees (called concurrent validation) or new applicants (called predictive validation). Next, they’d either collect job performance data on current employees or on new employees. They’d then analyze the data using a correlation coefficient. If the correlation between test performance and job performance is statistically significant and if the test explains a sufficient amount of the variance in job performance, the test is “criterion valid.”
- Construct Validity: This is the most statistically complex approach to validity. It’s not used very frequently for employment testing, as it requires large data sets and significant statistical and methodological knowledge. Basically, construct validity is used to measure what are called “constructs.” A construct would be something like dependability, ethical behavior, intelligence, etc. To determine that a test is valid, a large set of items that are believed to measure the construct in question, (dependability, for example) are developed. These may be similar to those of another known to measure the construct in question. Next, the items are administered to a sample size that is large and representative of the population. Once the data is collected, we move to a statistical process known as factor analysis, which determines which of the items are highly related to each other and which aren’t. These highly related items form a “factor.” The test developer carefully examines the items in the factor and determines whether that factor measures what it intends to measure. If so, the test is considered “construct valid.”
Accuracy vs. Validity
There is an important distinction between test validity and test accuracy. A test can be perfectly accurate without being valid. For example, a carefully designed test of basic arithmetic might be both a highly accurate and valid assessment for the job of bank teller. But if you administer that same test as a selection device for the job of groundskeeper — which would, in most cases, require little math — it wouldn’t be valid.
Even if the ability to accurately answer the questions on an employment test is important to a role, basing a selection solely on that can lead to difficulties in defending the hiring decision. Perhaps other more important knowledge, skills, and abilities should have been included, which might compensate for slightly lower scores. Again, the importance of conducting a job analysis prior to designing a test can’t be overstated. Employers must base their selection instruments on the demonstrated critical knowledge skills and abilities that are required for job success.
Ensuring Hiring Compliance
The Uniform Guidelines on Employee Selection Procedures were developed in 1978 through a collaboration between four federal agencies: the U.S. Department of Justice, the Department of Labor, the Equal Employment Opportunity Commission (EEOC), and the Civil Service Commission (now the Office of Management and Budget). Their goal was to ensure that every person receives an equal opportunity to compete for and gets jobs for which they’re the most qualified. As such, the Uniform Guidelines provide a single set of principles that employers, labor organizations, employment agencies, and licensing and certification boards can follow to avoid violating the federal law prohibiting employment practices that discriminate by race, color, religion, sex, and national origin.
These principles aren’t to be confused with affirmative action, which is designed to address issues where a court has determined that an employer has engaged in a pattern or practice that has, intentionally or unintentionally, systematically discriminated against one or more groups of people based on gender or race (predominantly). In such cases, a court may require an employer to take “affirmative action” to address the discrimination that they have engaged in, like by hiring qualified individuals from the gender or racial group that was previously excluded.
Affirmative action is simply ensuring that specific wrongs are righted. Once they have been and procedures are in place to ensure fairness moving forward, affirmative action stops and the ongoing effort toward equal employment opportunity continues. To further that effort and avoid ethical issues in recruitment, you can do a few things:
1. Read the EEOC guidelines. You don’t have to be a technical expert to understand the guidance or apply the recommended procedures outlined in the Uniform Guidelines. Despite having been developed decades ago, this important document contains goals and recommendations that are as sound today as they ever were. Moreover, in cases involving the legality or fairness of an employment test, courts give high regard to what the Uniform Guidelines require. Many employers who have flouted the requirements of the Uniform Guidelines have had to write substantial checks to those who have successfully challenged their testing practices.
2. Question your personal biases. Hiring is one of the most critical decisions a business can make. Everyone tends to be attracted to people who meet their expectations and fit (or avoid) certain stereotypes — many of which have nothing to do with whether the applicant would be successful on the job. Thus, a commitment to using only valid and reliable assessments to support hiring decisions is not only the most effective way to hire the most qualified person, but also the most ethical because it provides each applicant with an equal opportunity to demonstrate their abilities and qualifications for the job.
3. Don’t be afraid to innovate. The Uniform Guidelines offer a clear and practical framework for determining the proper methods for validation and use of employment tests and other selection procedures. Moreover, they aren’t exclusionary. Rather than specifying exactly what an employer must do, the Uniform Guidelines identify the specific characteristics of valid and legally defensible employment tests. They leave the door open for employers to innovate and adapt a variety of techniques to their unique situations.
The Role of Hiring Technology in Ethical Hiring Practices
Over the past few years, we’ve seen a robust emergence of artificial intelligence. We’ve also seen the emergence of AI-enabled employment selection. The underlying assumption is that when it comes to hiring, artificial intelligence is more effective than human intelligence.
Using AI in the hiring process can seem particularly attractive because it promises to save employer resources. A marketing slogan might say, “Rather than waste the time of a dedicated recruiting staff, let our technology screen applications, and we will match the most likely applicants to your job.”
While this sounds great, an AI-driven approach to hiring involves serious ethical questions and risks. For starters, a computer doesn’t know who completed a given application. Savvy applicants will understand the “correct” terms and information to include on their résumés (or find someone else to do it for them), essentially allowing them to “trick” the algorithm, which assesses applications, not applicants.
Furthermore, the algorithms behind these tools are trained only on the data available to their developers. No matter how large of a data set a system is trained on, they won’t have all the data necessary to make their models perfectly representative. Therefore, the parameters used to select and refer applicants will always be missing information.
It’s also critical to ask, “Who developed the algorithm?” Like everyone else, the humans who build AI tools have biases, which is why hiring technology is now being heavily scrutinized and even banned in some jurisdictions. Often, AI is a black box. Employers relying on automated technology can’t see how decisions are made or how parameters and criteria are weighted. They must simply trust the algorithm (and its flawed designers). Unfortunately, when it comes to defending employment decisions, it’s not the designer of the selection instrument (including AI) that faces the weight of liability but rather the user. That said, using a selection instrument that you can’t understand equates to using one you can’t defend.
Finally, by its statistical nature, AI is designed to make predictions based on what’s typical in order to provide your organization with a consistent level of quality. It won’t be able to spot the rare outlier who is atypical — but who might revolutionize your business.
None of this is to say that there is no role for algorithms in supporting human decisions, but employers are on the safest — and most ethical — grounds if they ensure that all their selection and screening procedures are related to the requirements of the job and consistent with business necessity.
Biddle Consulting Group has a passion for helping organizations do the right thing in hiring. We want to help organizations become the best they can be through the application of science in the design, validation, and administration of their pre-employment assessments. Get in touch to start working together today.