Online tests are a crucial step in the hiring process, serving as an initial filter to qualify candidates for face-to-face interviews. To ensure these tests are effective, it's essential to track specific metrics and adhere to best practices. This article provides a detailed guide on the key metrics to monitor and the best practices to follow, along with a template for organizing your data.
Key Metrics to Track
Completion Rate
Definition: The percentage of candidates who complete the test out of those who started it.
Formula: (Number of Candidates Who Completed the Test / Number of Candidates Who Started the Test) * 100
Importance: Indicates engagement and helps identify potential issues with the test's length or difficulty.
Pass Rate
Definition: The percentage of candidates who pass the test out of those who completed it.
Formula: (Number of Candidates Who Passed the Test / Number of Candidates Who Completed the Test) * 100
Importance: Measures the overall difficulty of the test and helps in setting the right benchmark for qualifying.
Average Time to Complete
Definition: The average time taken by candidates to complete the test.
Formula: Total Time Taken by All Candidates / Number of Candidates Who Completed the Test
Importance: Ensures the test is not too long and maintains candidates’ interest and focus.
Score Distribution
Definition: The range and distribution of scores achieved by candidates.
Importance: Helps in understanding the overall performance and effectiveness of the test questions in differentiating candidates.
Item Analysis
Definition: Performance analysis of individual test questions.
Importance: Identifies questions that are too easy, too difficult, or potentially ambiguous, ensuring test quality and fairness.
Drop-off Rate
Definition: The percentage of candidates who start but do not complete the test.
Formula: (Number of Candidates Who Did Not Complete the Test / Number of Candidates Who Started the Test) * 100
Importance: Highlights potential issues in the test structure or user experience that need to be addressed.
Candidate Feedback
Definition: Feedback from candidates regarding the test experience.
Importance: Provides qualitative insights to improve test content, format, and user experience.
Technical Issues Reported
Definition: The number and type of technical problems encountered by candidates.
Importance: Ensures a smooth testing process and identifies areas for technical improvements.
Engagement Metrics
Definition: Metrics such as the number of clicks, time spent per question, and navigation patterns.
Importance: Offers insights into candidate behavior and engagement with the test.
Correlation with Interview Performance
Definition: The relationship between online test scores and face-to-face interview performance.
Importance: Validates the predictive accuracy of the online test in identifying suitable candidates.
Test Metrics: Good vs. Bad Metrics
Metric | Good Metric | Bad Metric |
Completion Rate | > 80% | < 60% |
Pass Rate | 50% - 70% (dependent on difficulty) | < 30% or > 90% |
Average Time to Complete | Within 10% of expected time | > 20% longer or shorter than expected |
Highest Score | Close to 100% | < 90% |
Lowest Score | Above 30% | < 10% |
Mean Score | 60% - 70% | < 40% or > 80% |
Median Score | 60% - 70% | < 40% or > 80% |
Drop-off Rate | < 20% | > 40% |
Clicks Per Question | Consistent with expected interaction | Significantly higher or lower |
Time Spent Per Question | Consistent with question difficulty | Significantly higher or lower |
Technical Issues Reported | < 5% of candidates reporting issues | > 15% of candidates reporting issues |
Correlation with Interview Performance | r > 0.6 | r < 0.3 |
Detailed Breakdown
Completion Rate
Good: > 80%
Indicates high engagement and manageable test length/difficulty.
Bad: < 60%
Suggests issues with test engagement, length, or difficulty.
Pass Rate
Good: 50% - 70%
Balanced difficulty, suitable for filtering candidates.
Bad: < 30% or > 90%
Too difficult or too easy, respectively.
Average Time to Complete
Good: Within 10% of expected time
Shows that the test length is appropriate.
Bad: > 20% longer or shorter than expected
Indicates issues with test pacing or question clarity.
Highest Score
Good: Close to 100%
Top performers are fully demonstrating their abilities.
Bad: < 90%
Indicates the test might be too hard or there are ambiguous questions.
Lowest Score
Good: Above 30%
Even lower performers are able to answer some questions correctly.
Bad: < 10%
Test may be too difficult or poorly designed.
Mean Score
Good: 60% - 70%
Indicates a balanced test with a good spread of scores.
Bad: < 40% or > 80%
Suggests the test is either too hard or too easy.
Median Score
Good: 60% - 70%
Consistent with a balanced difficulty level.
Bad: < 40% or > 80%
Indicates skewed difficulty.
Drop-off Rate
Good: < 20%
Most candidates are completing the test.
Bad: > 40%
High drop-off suggests issues with test design or engagement.
Clicks Per Question
Good: Consistent with expected interaction
Indicates questions are clear and straightforward.
Bad: Significantly higher or lower
May indicate confusing questions or overly simple ones.
Time Spent Per Question
Good: Consistent with question difficulty
Shows candidates are spending appropriate time per question.
Bad: Significantly higher or lower
Indicates potential issues with question clarity or difficulty.
Technical Issues Reported
Good: < 5% of candidates reporting issues
Indicates a stable testing platform.
Bad: > 15% of candidates reporting issues
Suggests significant technical problems.
Correlation with Interview Performance
Good: r > 0.6
Strong correlation, indicating the test is a good predictor of interview performance.
Bad: r < 0.3
Weak correlation, suggesting the test is not effectively predicting interview success.
By monitoring these metrics and aiming for the "good" benchmarks, you can ensure that your online tests are effective tools for qualifying candidates for face-to-face interviews.