This feature is currently in its beta and requires a manual trigger for evaluation. Turn around time for the same is about 30 mins - 2 hours. We're expecting the feature to be live by Jan 30, 2025
WeCP makes it simple to evaluate file upload questions by using AI-driven evaluation. Instead of manually reviewing every response, you can define evaluation criteria in the form of prompts, and WeCP AI will handle the rest. Follow this guide to set up and auto-evaluate file upload questions seamlessly.
Step 1: Create a File Upload Question
Log in to your WeCP Admin Dashboard.
Navigate to the Test Designer or Question Library.
Select the option to add a new question.
Choose the File Upload question type from the list.
Provide a clear question description to guide candidates on what to upload. For example:
"Upload a code file demonstrating your solution."
"Submit your design in a PDF format."
"Attach a spreadsheet with your analysis."
Step 2: Define an Evaluation Prompt
To enable AI evaluation, you need to provide an evaluation prompt. This prompt should define:
What to evaluate: Specify what the AI should look for in the uploaded file.
Evaluation criteria: Include key points such as correctness, completeness, creativity, or adherence to specific standards.
Examples of Evaluation Prompts:
For a coding file:
"Evaluate the correctness, efficiency, and readability of the code provided. Check for adherence to Python conventions and optimize the solution for a sorting algorithm."
For a design file:
"Assess the uploaded design for creativity, alignment with branding guidelines, and the effective use of colors and typography."
For an analysis file:
"Review the spreadsheet for accurate calculations, clear formatting, and the logical structure of data presentation."
Step 3: Configure Auto-Evaluation Settings
After defining the question, scroll to the Evaluation Settings section.
Enable the AI Evaluation toggle.
Enter your evaluation prompt into the designated field.
(Optional) Add weightage to specific criteria if necessary (e.g., correctness: 50%, creativity: 30%, formatting: 20%).
Step 4: Test Your Evaluation Prompt
Before assigning the test to candidates:
Use WeCP’s Preview or Test Prompt feature.
Upload a sample file and check if the AI evaluation aligns with your expectations.
Fine-tune the prompt if needed for better accuracy.
Step 5: Assign the Test
Once your question is ready, assign the test to candidates as usual.
Candidates will upload their files in response to the question.
Step 6: Review AI Evaluation Results
After the test is completed, navigate to the Results tab.
Locate the responses for your file upload question.
View the AI-generated evaluation scores and feedback for each candidate.
Detailed evaluation will include comments or scores based on your prompt.
(Optional) Override or adjust AI evaluations if needed for specific cases.
Tips for Effective Auto-Evaluation
Be specific in your prompt: The clearer your instructions, the more accurate the AI evaluation.
Test and refine: Use multiple sample files to fine-tune the evaluation process.
Combine with manual review (if needed): For high-stakes assessments, consider a hybrid approach where AI evaluation is followed by a manual review of top candidates.