AI scoring best practices

AI Scoring evaluates interactions only from the conversation transcript and the evaluation form you provide.

It does not use metadata such as:

• Queue or routing path
• Timestamps or duration
• Platform or system metrics

AI Scoring focuses purely on what was said (or written) — not on tone, attitude, or inferred meaning.

Do

• Write objective, transcript-driven questions.
• Provide clear help text with measurable criteria.
• Use consistent terminology (for example, “agent” and “customer”).
• Write in complete sentences.
• Refine questions continuously based on AI scoring results.

Don’t

• Ask subjective questions (for example, “Was the agent polite?”).
• Use vague shorthand (for example, “Greeting protocol”).
• Assume context not visible in the transcript.
• Ask about tone or emotion that AI cannot infer.

Each question should include short, measurable help text (≤500 characters) that defines what counts as Yes or No, includes examples, and clarifies edge cases.

Example: well-written question and help text

Intent: Verify the customer’s identity before providing account-specific details.

Question: Did the agent verify the customer’s identity before providing account-specific details?

Help text: The agent must confirm at least one credential (for example, account number, date of birth, phone number, or order ID).

• Yes: Agent verified a credential before proceeding.
• No: Agent skipped or asked unrelated questions.

Transcript examples

• Review AI scoring reports monthly to identify low-confidence questions.
• Add examples reflecting your industry’s phrasing (for example, healthcare vs. retail).
• Align human evaluators with the same definitions used by AI.
• Refine questions and help text when AI misclassifies results.

Question:
Did the agent greet the customer at the beginning of the conversation using a standard greeting such as “hi,” “hello,” or “good morning”?

Answer Options:
Yes / No

Help Text:
The agent must begin with a polite greeting. Acceptable greetings include “hi,” “hello,” “good morning,” or equivalent.

Transcript Example:
Customer: “I need help with my account.”
Agent: “Good morning! Thanks for calling ABC Support.”
→ AI marks Yes

Question:
Did the agent verify the customer’s identity before addressing account-specific concerns?

Answer Options:
Yes / No

Help Text:
The agent must verify at least one customer credential, such as date of birth, account ID, or phone number, before proceeding.

Transcript Example:
Customer: “I want to update my billing address.”
Agent: “Can I confirm the last four digits of your account number?”
→ AI marks Yes

Question:
Did the agent allow the customer to finish speaking without interruptions?

Answer Options:
Yes / No

Help Text:
An interruption occurs when the agent cuts the customer off mid-sentence. Clarifications after the customer finishes are acceptable.

Transcript Example:
Customer: “The issue started when—”
Agent: “Let me stop you there…”
→ AI marks No

Question:
Did the agent use the customer’s name at least once during the conversation?

Answer Options:
Yes / No

Help Text:
Using the customer’s name personalizes the interaction. The AI checks whether the name, as provided in the record or conversation, was mentioned.

Transcript Example:
Customer: “I’m Maria, and I need help with my order.”
Agent: “Thank you, Maria. Let’s take a look at your order.”
→ AI marks Yes

Question:
Did the agent acknowledge the customer’s issue before moving to resolution?

Answer Options:
Yes / No

Help Text:
Empathy is demonstrated when the agent acknowledges the issue with phrases such as “I understand,” “I’m sorry that happened,” or “I know this must be frustrating.” The AI looks for acknowledgment before resolution.

Transcript Example:
Customer: “I’ve been overcharged.”
Agent: “I’m sorry to hear that. Let’s review your bill together.”
→ AI marks Yes

Question:
Did the agent provide a clear resolution step, such as a refund, replacement, or troubleshooting instruction?

Answer Options:
Yes / No

Help Text:
A resolution must include a specific action (refund, replacement, or technical steps). General apologies without clear next steps do not count.

Transcript Example:
Agent: “I’ve processed a replacement order that will arrive within two days.”
→ AI marks Yes

Question:
Did the agent explain the escalation process clearly, including who to contact, what information is needed, and expected response times?

Answer Options:
Yes / No

Help Text:
A complete escalation includes:

1. Next contact point

2. Required information

3. Expected timeframe for response

Transcript Example:
Agent: “If this issue persists, I’ll escalate to Level 2. They’ll need your case ID and will respond within 24 hours.”
→ AI marks Yes

Question:
Did the agent comply with mandatory disclosure or compliance statements (for example, terms, disclaimers, or legal requirements)?

Answer Options:
Yes / No

Help Text:
Compliance statements vary by industry and may include required disclosures, disclaimers, or legal notices. AI checks for these specific phrases in the transcript.

Transcript Example:
Agent: “For security purposes, I cannot provide account details without verifying your identity first.”
→ AI marks Yes

Question:
Did the agent avoid unnecessary dead air or long silences without explanation?

Answer Options:
Yes / No

Help Text:
Silences longer than 15 seconds are acceptable only when explained to the customer (for example, “I’ll place you on a brief hold while I check this”).

Transcript Example:
Agent: “Give me a moment while I pull up your account details.” [10 seconds of silence]
→ AI marks Yes

Question:
Did the agent confirm customer satisfaction or summarize next steps before closing the conversation?

Answer Options:
Yes / No

Help Text:
Before closing, the agent should confirm resolution or clearly summarize next steps. This demonstrates completion and helps ensure customer understanding.

Transcript Example:
Agent: “I’ve reset your password. Can you confirm you’re able to log in now?”
→ AI marks Yes