How to configure custom coaching scorecards and playbooks in Demodesk AI Coach

What and why

Demodesk AI Coach scores every recorded sales call against a custom scorecard you define — quantitative ratings (the “flame” score per criterion), qualitative feedback with timestamp evidence, and methodology alignment (SPICE, MEDDIC, BANT, or your own framework). This guide shows you how to build a scorecard from scratch, tie it to a specific call type, and start scoring calls within minutes.

Most sales managers spend less than 5% of their time coaching. A configured AI Coach scorecard scales feedback to every rep on every call — no 1:1s required.

Who this is for

Sales managers, Heads of Sales, and RevOps leaders who want consistent, methodology-aligned coaching across the whole team — particularly teams running multiple call types (discovery, demo, closing) that each need different evaluation criteria.

Prerequisites

• A Demodesk Coaching & AI seat (admin access to configure scorecards)
• At least one recorded call in your library to test the scorecard against
• A defined sales methodology or framework (SPICE, MEDDIC, BANT, Challenger, or your own playbook)
• CRM connected (Salesforce, HubSpot, or Pipedrive) — optional but recommended so scorecards can be triggered automatically by deal stage

Steps

1. Open the Scorecards configuration

Go to the Agents tab in the top navigation, then click the Scorecards sub-tab. This is where every scorecard your team uses is created, edited, and assigned to call types.

2. Create a new scorecard and name it by call type

Click New scorecardand give it a name that maps to a specific call type — “Discovery — SPICE” or “Demo — Closing Readiness”, for example. Naming by call type matters because the criteria for a first discovery call are different from those for a closing meeting.

3. Add evaluation criteria from your framework

Add each criterion you want the AI to score. For a SPICE-based discovery scorecard:

• Situation— Did the rep uncover the prospect's current state?
• Pain— Was the pain point quantified?
• Impact— Did the rep explore business impact?
• Critical event— Is there a compelling reason to act now?
• Decision process— Are stakeholders and timeline clear?

For each criterion, write a short description of what “good” looks like. The AI uses this to decide the score. Be specific. “Rep asks at least two open-ended questions about current process” scores more reliably than “rep does discovery well.”

4. Set the rating scale (flame ratings)

Each criterion is scored on a visual scale — Demodesk uses flame icons to show how strongly the criterion was met. Choose your scale (typically 1–5 flames) and define what each level means in plain language:

• 1 flame — Not addressed
• 3 flames — Partially addressed, missing key element
• 5 flames — Fully addressed with clear evidence

The clearer your level definitions, the more consistent the scoring across calls.

5. Enable qualitative feedback with timestamp evidence

Turn on qualitative feedback so AI Coach generates written commentary alongside the numeric score — and links each comment to the exact moment in the call where the evidence appears. A rep can click the timestamp, hear themselves say (or not say) the thing, and learn faster than from any 1:1.

6. Assign the scorecard to a call type or deal stage

Link the scorecard to a specific call type so it only runs on the right calls:

• Meeting title contains (e.g., “Discovery”, “Demo”)
• CRM deal stage (e.g., “Qualification”, “Proposal”)
• Manual selection per call

A discovery scorecard scoring a closing call produces noise. Assignment keeps the feedback relevant.

7. Test the scorecard against a past call

Before rolling out, run the scorecard against a recent call you know well. Go to the call recording, apply the scorecard, and check whether the scores and feedback match your own judgment. Adjust criterion descriptions if the AI scores too lenient or too harsh.

8. Roll out to the team and review weekly

Once the scorecard matches your judgment, enable it for the team. New calls of the matching type are scored automatically. Review the Performance dashboard weekly to spot patterns — which criteria are consistently weak, which reps are excelling, and where the team needs group coaching.

Tips

• Start with one call type, not five. Build a great discovery scorecard before adding demo and closing. Each scorecard takes iteration to calibrate.
• Use your own language, not generic framework language.If your team calls it “critical event” but the framework calls it “compelling event”, use your wording. Reps recognize their own playbook faster.
• Calibrate against your top 20%.Score five calls from your best reps. Whatever criteria they consistently nail at 5 flames is what “good” looks like for the rest of the team.
• Five to seven criteria per scorecard is the sweet spot. More and the signal gets diluted. Fewer and you miss what matters.
• Read the qualitative feedback before the scores. The timestamp comments tell you whya rep scored low. That's the coachable moment. The score alone is just a number.

Related skills and agents

• AI Coach product page — full capability overview
• AI Sales Assistant — pairs with AI Coach for post-call summaries and follow-ups
• Marketplace: Discovery & Qualification skills — pre-built skills for SPICE, MEDDIC, BANT
• Solutions: Sales Managers — how managers scale coaching from 1:10 to 1:50