Creating Criteria

To create a new criterion, navigate to Tools > Criteria and click New Criterion in the top right. You can also edit an existing criterion by clicking on it from the table.

Start by filling in the basic details:

• Name — Give the criterion a clear, descriptive name (e.g. "Empathy and Tone of Voice", "Compliance and Conduct").

• Group — Select or create a group to categorize related criteria (e.g. "Communication", "Processes"). Groups are used for reporting purposes, and criteria in the same group will appear together on scorecards and in reports.

• Instructions for Evaluators — Provide clear guidance for human QAs on how to assess this criterion. These instructions are optional if you plan to use this criterion exclusively with AutoQA. You can use the rich text editor to format instructions with bold text, links, lists, and headings.

Rating Scale & Scores

Choose how this criterion will be scored. There are four options: Binary, 3-point, 4-point, or 5-point scale.

Each rating has an emoji, a name, and a score. Names and scores are fully customizable — for example, on a 3-point scale you might use "Yes" (1), "Partial" (0.5), and "No" (0). You can use both whole numbers and decimals, but each score must be unique across all ratings.

Allow Criterion to Be Skipped

Enable this if you want both human evaluators and AutoQA to have the option to skip this criterion. Skipped criteria won't count towards the final evaluation score. If disabled, a rating must always be provided.

Mode (Scoring Impact)

This determines how the criterion's score affects the overall evaluation:

• Critical — If the lowest rating is selected, the entire evaluation automatically fails and the score goes to 0%.

• Standard (recommended) — The criterion's score contributes proportionally to the overall evaluation. For example, if a criterion is scored 0.5 out of 1, it contributes 50% of its weight to the total.

• Soft — The criterion's rating has no impact on the final evaluation score. Use this to collect data and feedback without penalising agent performance.

AutoQA Configuration

If you want to automate this criterion using AI, enable AutoQA and add a prompt. This prompt tells the AI what checks to perform and how to assign ratings.

You can write your prompt from scratch or use the built-in tools:

• Template — Inserts a pre-built structure with sections for checks, additional notes, scoring methodology, and optional examples. Fill in each section to match your requirements.

• Refine — Paste preliminary or basic instructions and let the AI convert them into a more detailed, structured prompt. You can choose between simplistic or detailed output. Review the result and apply, reject, or edit before saving.

For best practices on writing effective AutoQA instructions, go to AutoQA Best practices.

AutoQA Options

• AutoQA Tools — Configure external endpoints that AutoQA can call with parameters to fetch additional data during evaluation.

• Related Knowledge Base Articles — Link specific articles to provide context for this criterion. You can select individual articles or enable Always Sync to keep it linked to all articles in your knowledge base. This is particularly useful for criteria around product knowledge or process adherence.

• Use Helpdesk Macros — For supported help desks (e.g. Zendesk, Intercom), AutoQA can ingest macros and compare them against agent messages to identify if a macro was used.

• Message Types — Choose which message types AutoQA should include as input (e.g. customer messages, agent responses, internal notes).

• Ticket Field Events — Select which ticket field events to include (e.g. status changes, priority updates).

• Macro Events — Optionally include macro events in the evaluation input (Zendesk-specific).

Root Causes

Root causes are tags or labels that provide more detail when a criterion is marked down. They capture the specific reason a criterion failed, enabling targeted coaching and process improvements.

Typically, root causes align with the individual checks defined in your AutoQA prompt. For example, if you have four checks, you'd expect roughly four corresponding root causes.

Adding Root Causes

You can add root causes manually one by one, or click Generate to let the AI suggest root causes based on your AutoQA instructions. Review, edit, add, or remove suggestions as needed.

Root Cause Settings

• Require root cause selection — Make it mandatory for evaluators or AutoQA to select at least one root cause when a criterion receives less than a full score.

• Allow multiple root causes — Let evaluators select more than one root cause per markdown, for cases where multiple issues occur simultaneously.

Any edits made to a criterion that is already in use on scorecards will take effect immediately.