All Collections
Hiring Managers
Understanding Question types, Languages, and Frameworks
How and across what parameters does WeCP AI judge a candidate's English Communication Skill
How and across what parameters does WeCP AI judge a candidate's English Communication Skill
Priyanka Khandagale avatar
Written by Priyanka Khandagale
Updated over a week ago

Traditional English-speaking assessment tools like Versant etc primarily used pattern recognition technology to evaluate the English proficiency of non-native speakers. These tools primarily focus on understanding the linguistic features of a test taker's responses such as grammar, vocabulary, pronunciation, fluency, and comprehension using a rule-based system.

These traditional tools evaluate candidates using standards like CERF etcetera; while CERF has many strengths, it also has a number of limitations:

  1. Subjectivity: The CEFR descriptors are broad and can be subject to different interpretations. This can lead to inconsistencies in the assessment of language proficiency. For instance, what one examiner may deem as B2 (upper intermediate) proficiency, another may deem as C1 (advanced).

  2. Cultural Variations: The CEFR doesn't adequately account for variations in language use across different cultural contexts. This means that a speaker might be rated lower or higher depending on the cultural assumptions of the test designer or evaluator.

  3. Lack of Detail: While the CEFR provides a general guideline for language proficiency, it does not provide a detailed syllabus or curriculum. This can make it difficult for educators to design specific learning objectives and assessment tasks based on the CEFR levels.

  4. Neglect of Certain Language Aspects: The CEFR's focus is primarily on communication and comprehension, and it may not fully assess certain aspects of language proficiency, such as cultural knowledge, translation skills, or the ability to understand and produce academic or professional language.

  5. Not Comprehensive for All Languages: Although CEFR has been designed to be applicable to all European languages, there can be specific features or challenges in certain languages that are not adequately addressed by the framework.

  6. Scalability: Traditional CEFR assessments often involve human evaluation, which can be time-consuming and hard to scale up for large numbers of learners.

WeCP's AI leverages large language models (LLMs) as its core technology to evaluate English-commuication skill, on the other hand, can offer a more comprehensive analysis of a speaker's proficiency. Here are some core benefits of using LLMs over traditional rule-based system:

  1. Understanding of Context: While traditional tools can detect the correctness of a given statement in terms of grammar, vocabulary, etc., WeCP AI can also understand the semantic and pragmatic aspects of the language, allowing them to evaluate the context and coherence of the speaker's responses.

  2. Bias Mitigation: WeCP AI automatically reduces the bias in test scoring by eliminating human error or unconscious bias.

  3. Custom Paramters: WeCP AI can be customised with organisation specific parameters to judge someone's english skills. For example, if your organization values certain industry-specific vocabulary, the WeCP AI could be customised to recognize and evaluate the use of such vocabulary. If the organization values certain styles of communication, the WeCP AI could potentially be trained to recognize and evaluate these styles as well.

  4. Dynamic Interactions: Traditional tools often involve one-way interactions where the test taker responds to a set of predefined questions or prompts. WeCP AI, with its capacity to generate context-aware responses, can simulate a two-way interaction similar to a conversation with a human evaluator.

  5. Personalized Feedback: WeCP AI can provide more personalized and detailed feedback. They can point out specific areas of strength and weakness, suggest ways to improve, and even generate examples or exercises tailored to the learner's needs.

  6. Continuous Learning: WeCP AI can learn from each interaction, continually updating their understanding and capabilities. This allows them to offer more accurate assessments and recommendations over time.

Below are the parameters currently WeCP AI is trainned on to judge Eglish Communication Skill:

English (Speaking)

  1. Pronunciation: This is the ability to articulate words clearly and understandably. It also encompasses elements such as stress, intonation, and rhythm.

  2. Fluency: Fluency refers to the smoothness of speech, including natural pacing, proper use of pauses, and avoiding excessive hesitation or self-correction.

  3. Grammar: This includes correct usage of tense, noun-verb agreement, sentence structure, word order, and the appropriate use of grammatical constructions.

  4. Vocabulary: This refers to the range and accuracy of words used in speaking. It includes the usage of appropriate words for the context and situation, and the ability to use synonyms and paraphrases to explain unfamiliar words.

  5. Coherence: Coherence involves organizing thoughts and ideas logically and consistently, using cohesive devices and markers to connect sentences and ideas together.

  6. Interactive Communication: The ability to initiate, maintain, and end a conversation appropriately. It also involves reacting to and prompting further conversation from others, and adjusting one's language according to the context.

  7. Task Achievement: This is about achieving the specific goals of the speech or conversation, such as giving an opinion, suggesting a solution, or describing a situation.

  8. Pragmatic Competence: This involves understanding and appropriately using the social norms of language, including politeness and formality levels.

  9. Discourse Management: The ability to construct longer turns at talk, managing interaction and using a variety of discourse markers.

  10. Accuracy: The degree to which the speech is free from errors in grammar, vocabulary, pronunciation, etc.

  11. Intelligibility: The ability to make oneself understood, even if there are some errors or non-standard pronunciation.

  12. Contextual Understanding: This relates to how well the speaker understands and responds appropriately to the context of the conversation.

  13. Idiomatic language: The ability to understand and use idiomatic expressions, slang, or colloquialisms where appropriate.

  14. Non-verbal communication: This includes aspects like body language, eye contact, and other non-verbal cues that can affect communication.

  15. Confidence: This relates to the speaker's confidence in their ability to communicate effectively in English, which can influence many of the other parameters listed above.

  16. Accent: While everyone has an accent and it's not necessarily a negative attribute, a heavy accent that affects comprehensibility might be assessed.

  17. Listening comprehension: The ability to accurately understand, process, and respond to what is being said by others.

English (Writing)

  1. Grammar: Correct usage of syntax, punctuation, verb tenses, noun-verb agreement, etc.

  2. Spelling: Accurate spelling of words, including homophones and commonly misspelled words.

  3. Vocabulary: Appropriate and varied word choice, use of synonyms, and avoiding repetition of the same words or phrases.

  4. Coherence and Cohesion: Logical organization of ideas, effective paragraphing, and the use of connectors, pronouns, and conjunctions to link sentences and ideas together.

  5. Task Achievement/Response: Fulfillment of the task requirements, such as answering the question fully, staying on topic, and meeting the word limit.

  6. Argumentation: The ability to formulate, present, and defend an argument effectively, including providing supporting evidence or examples.

  7. Style and Tone: Appropriate use of formal or informal language depending on the context, and maintaining a consistent tone throughout the text.

  8. Clarity: Clear expression of ideas, avoiding ambiguity or confusion.

  9. Punctuation: Correct and effective use of punctuation marks to aid the readability and comprehension of the text.

  10. Sentence Structure: Varied use of simple, compound, and complex sentences, and correct word order.

  11. Precision: Use of precise and specific language, avoiding vagueness or generalities.

  12. Originality: Ability to present unique ideas or perspectives, and to paraphrase or summarize effectively when using others' ideas.

  13. Formatting: Correct and consistent use of formatting conventions, such as indentation, margins, and spacing.

  14. Use of Citations and References: When required, the accurate citation of sources, and inclusion of a correctly formatted reference list or bibliography.

  15. Mechanics: Correct use of capitalization, abbreviations, numbers, etc.

  16. Use of Idiomatic Language: Understanding and correct usage of idiomatic expressions, phrasal verbs, or colloquialisms where appropriate.

  17. Concision: Ability to express ideas concisely, avoiding redundancy or unnecessary words.

  18. Flow: The rhythm and pace of the writing, achieved through the use of varied sentence lengths and structures.

  19. Persuasiveness: If applicable, the effectiveness of the writing in persuading the reader of a particular point of view.

  20. Relevance: This pertains to how well the writer addresses the topic or task at hand. The content should be directly related to the given prompt or purpose of the writing. Irrelevant information or digressions can detract from the overall effectiveness of the writing. In an argumentative context, all arguments, examples, or evidence provided should directly support the thesis statement. In an informational context, the content should provide the necessary details the audience needs to understand the subject matter.

English (Listening)

  1. Comprehension: The ability to understand the main ideas, details, and implicit information in what is being said.

  2. Vocabulary Recognition: The ability to understand a wide range of vocabulary, including idiomatic expressions, colloquial language, and technical jargon where applicable.

  3. Distinguishing Sounds: The ability to distinguish different sounds, accents, speech rates, and intonations in English.

  4. Understanding Context: The ability to understand the context or situation in which the language is being used, including the social, cultural, or historical background.

  5. Interpretation of Paralinguistic Elements: Understanding the speaker's emotions, attitudes, or intentions through tone of voice, stress, intonation, etc.

  6. Inference: The ability to make educated guesses or infer information that is not explicitly stated based on the available information.

  7. Understanding Implicit Meaning: This involves understanding sarcasm, irony, or indirect suggestions that are common in spoken English.

  8. Following Directions: If the listening involves instructions or directions, the listener's ability to understand and potentially follow these instructions is important.

  9. Note-taking Ability: In academic or professional settings, the ability to take notes while listening can be an important part of comprehending and remembering information.

  10. Predictive Listening: The ability to anticipate what a speaker is going to say based on context and prior information.

  11. Discrimination Skills: This refers to the ability to recognize differences in sounds, word boundaries, and where one word ends and another begins, especially in rapid speech.

  12. Summarization Skills: The ability to summarize or paraphrase the main points or specific details of a listening text.

WeCP Grading Vs CEFR Levels

CEFR Grading Rubric

Grade

Interpretation

A1

Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce themselves and others and can ask and answer questions about personal details such as where someone lives, people they know and things they have. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.

A2

Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of their background, immediate environment and matters in areas of immediate need.

B1

Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can produce simple connected text on topics, which are familiar, or of personal interest. Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.

B2

Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in their field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.

C1

Can understand a wide range of demanding, longer texts, and recognise implicit meaning. Can express themself fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices.

C2

Can understand with ease virtually everything heard or read. Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation. Can express themself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in more complex situations.

WeCP AI Grading Rubric

Grade

Interpretation

Beginner

(40% - 50%)

Overview: At this level, the individual has a limited vocabulary and struggles with basic sentence formation. They have difficulty understanding and participating in even simple conversations.

Reading: Able to recognize and understand only basic words and phrases. Struggles with more complex texts.

Writing: Limited to writing simple sentences, often with grammatical errors. Complex ideas are hard to convey.

Speaking: Vocabulary is restricted to everyday, familiar phrases. Conversations beyond basic introductions and greetings are difficult.

Listening: Can understand only slow, carefully articulated speech on familiar matters.

Use Cases: Suitable for very rudimentary interactions, like asking for directions or ordering food.

Intermediate

(50% - 60%)

Overview: At this stage, the speaker can handle common situations but will struggle with technical or abstract topics.

Reading: Can understand texts related to daily life and work but struggles with more complex or academic material.

Writing: Capable of writing texts for personal or work-related purposes but with limited vocabulary and varying grammatical accuracy.

Speaking: Can engage in conversations on familiar subjects but often pauses to find the right words.

Listening: Understands general conversations but may need some repetition and clarification.

Use Cases: Suited for basic customer service roles, simple social interactions, and straightforward workplace communication.

Regular

(60% - 70%)

Overview: The speaker can manage most everyday and work situations but will have limitations in vocabulary and may make occasional mistakes.

Reading: Comfortable with general texts, newspapers, and straightforward technical material.

Writing: Can write with reasonable accuracy but may struggle to be concise or nuanced.

Speaking: Good conversational skills but may have a noticeable accent or slight hesitations.

Listening: Understands normal speech but may struggle with complex language or accents.

Use Cases: Suitable for standard office roles and social situations where advanced technical language is not required.

Proficient

(70% - 80%)

Overview: Proficient speakers can discuss a range of topics with ease and have a good grasp of idiomatic expressions.

Reading: Able to understand complex texts, including technical, academic, and professional materials.

Writing: Capable of writing detailed, well-structured documents with few errors.

Speaking: Fluent with minimal accent and excellent vocabulary.

Listening: Understands almost all forms of spoken English, including various accents and rapid speech.

Use Cases: Well-suited for roles requiring specialized knowledge, public speaking, or complex negotiations.

Expert

(80% - 90%)

Overview: Expert-level speakers are near-native and can understand and produce complex and nuanced language.

Reading: Can understand virtually all forms of written English, including highly abstract, idiomatic, or technical material.

Writing: Produces clear, smooth-flowing text in a variety of styles.

Speaking: Articulate and fluent, capable of expressing complex thoughts effortlessly.

Listening: Can understand everything from slang and idioms to complex debates and discussions.

Use Cases: Suitable for high-level roles in academia, business, or government where advanced language skills are essential.

Master

(90% - 100%)

Overview: At the Master level, the individual is indistinguishable from a well-educated native speaker in all forms of communication.

Reading: Fully comprehends all texts, including theoretical and highly abstract writing.

Writing: Produces sophisticated, well-structured writing suitable for publication.

Speaking: Fluent, eloquent, and capable of rhetorical sophistication.

Listening: Completely understands all dialects, levels of formality, and specialized language.

Use Cases: Suited for any role requiring advanced language skills, from diplomacy to academia to executive leadership.

Limitations of CEFR grading

While the Common European Framework of Reference for Languages (CEFR), which has six levels ranging from A1 (beginner) to C2 (proficient) is widely respected and provides a standardized way to gauge language ability, it has some limitations:

1. Too Generalized:

  • Issue: The CEFR levels are designed to be broad categories that cover a range of skills.

  • Impact: This could mean that someone who excels in reading but struggles in speaking might end up with the same overall level as someone with the opposite strengths and weaknesses.

2. Too Academic:

  • Issue: Standardized tests often measure academic forms of language.

  • Impact: This might not reflect someone's ability to use English in practical, everyday situations or specialized business contexts, which could be especially relevant for BPO roles

3. Doesn't Adapt with Change in Industry:

  • Issue: Language proficiency can change over time due to various factors like lack of practice or immersion.

  • Impact: A CEFR level obtained at one point may not accurately represent a person's current skill level.

4. Europe-centric:

  • Issue: The CEFR system is European-centric.

  • Impact: It might not fully capture the nuances of English as it is used in other regions, such as American, Indian or Australian English.

5. Lack of Sub-Divisions:

  • Issue: Each CEFR level covers a range of abilities.

  • Impact: Someone at the lower end of a level (e.g., C1) might struggle to keep up in a setting that expects proficiency at the higher end of that same level.

6. No Skill-Specific Ratings:

  • Issue: CEFR gives an overall level but doesn't necessarily break down the rating into listening, speaking, reading, and writing.

  • Impact: Employers or educational institutions looking for strengths in specific areas may find the overall levels insufficient.

7. Stress and Test Anxiety:

  • Issue: Standardized tests can be stressful, which might impact performance.

  • Impact: Test scores may not fully reflect a candidate's language abilities in less stressful, real-world scenarios.

Advantage of WeCP over CEFR levels

The WeCP way of evaluating English communication skills, as described, offers several potential advantages over the Common European Framework of Reference for Languages (CEFR) grading system:

Flexibility:

  1. Customization: WeCP's grading system can be tailored to fit the specific needs of the industry or market you are targeting. This can be especially helpful if you are serving specialized sectors like tech, healthcare, or finance.

  2. Localized Understanding: Given that your grading system can be adjusted to suit local conditions, it can be adapted to consider local idioms, accents, and cultural nuances in communication.

Comprehensive:

  1. Skill Segmentation: Unlike CEFR, which broadly categorizes skills, your system provides a more granular look at different facets of language use: reading, writing, speaking, and listening. This can provide a more complete picture of a candidate's abilities.

  2. Real-world Context: By defining 'Use Cases' for each level, you provide a practical, real-world application for each proficiency level, making it easier for employers to match a candidate's grade to a specific job requirement.

Business-focused:

  1. Work-Related Metrics: Your categories include considerations for workplace communication, making it easier for employers to understand how a candidate's language skills will apply in a business context.

  2. Growth-Driven: Your system can also be oriented to reflect the growth paths within a corporate structure, from entry-level to executive roles, which is beneficial for long-term talent development and management.

Ease of Understanding:

  1. Simplicity: Your grade descriptors are straightforward and easy to understand, reducing the likelihood of misinterpretation.

  2. Clear Milestones: With descriptors for each skill at each level, it's easy for candidates to identify areas for improvement and for employers to set training goals.

Adaptability for Tech:

  1. Automation-friendly: Your detailed parameters can be more easily translated into an automated assessment tool, which would fit well with WeCP's focus on skill assessment and remote interviewing software.

Candidate Selection Framework (Recommended by WeCP)

Market/Country

Most Suitable Grade

Considerations

United States

Proficient to Master

American idioms, multiple accents, high customer expectations

United Kingdom

Proficient to Expert

British idioms and accents, high customer service standards

Canada

Proficient to Expert

French-English bilingualism may be beneficial, diverse accents

Australia

Proficient to Expert

Australian idioms and accents, high customer service standards

India

Intermediate to Proficient

Diversity in accents, cost-sensitive market, varying fluency

Philippines

Regular to Proficient

High English proficiency but may have local accents

Middle East

Regular to Proficient

Multiple languages often spoken, varying degrees of English fluency

European Union

Proficient to Expert

Multi-lingual settings, CEFR standardization, high expectations

Latin America

Regular to Proficient

Spanish-English bilingualism beneficial, varying English fluency

South-East Asia

Regular to Proficient

Varying English proficiency, multiple local languages

Africa

Regular to Proficient

Diverse linguistic landscape, varying degrees of English fluency

Did this answer your question?