score a sample of agent replies against the team's quality checklist

• Open a spreadsheet containing the team's quality assurance rubric and scoring criteria.
• Select a random sample of completed customer support tickets from the help desk.
• Read through the agent's responses and internal notes for each selected ticket.
• Compare the agent's actions against each line item on the QA scorecard, such as tone, accuracy, and policy adherence.
• Calculate the final score and note specific feedback or coaching points in the tracking sheet.
• Copy the scores and notes into an email or direct message to send to the agent.

Spending hours cross-referencing long ticket threads with a rubric, which leads to reviewing only a small, unrepresentative fraction of what the support team actually writes.

1. You

   1. Prepare the review set

   Choose a representative sample of completed tickets and provide the relevant ticket conversations, internal notes, QA rubric, scoring criteria, and applicable support policies. Remove information that the reviewer should not share outside the support team.

2. AI

   2. Map each ticket to the rubric

   For every sampled ticket, compare the agent's replies and internal notes with each rubric line item, separating evidence from assumptions. Flag missing context or policy areas it cannot verify reliably.

3. AI

   3. Draft scores and coaching notes

   Create a draft score for each criterion, cite the relevant part of the ticket, and summarize strengths, gaps, and possible coaching points. Mark uncertain judgments for human review rather than presenting them as confirmed findings.

4. You

   4. Make the final QA decision

   Check the evidence behind flagged or consequential judgments, correct scores where context or policy interpretation requires human expertise, and decide whether each case needs coaching, escalation, or no action. The human owns the final score and any performance consequence.

5. AI

   5. Format the approved results

   Turn the human-approved scores and notes into a consistent QA record, including the ticket identifier, criterion results, final score, evidence, and agreed coaching or escalation action.

6. You

   6. Deliver and record feedback

   Enter the approved results in the team's QA tracker and send the relevant coaching notes to the agent through the normal support-management channel. Retain the record according to the team's access and retention rules.

• Review last week's support tickets and find the three macros that need rewriting
• Write the escalation summary before handing a stuck ticket to engineering