Business value, safety, integration and data governance carry 61 of the 100 points. Voice quality and presentation matter only where they improve customer handling and operational outcomes.
| Criterion | Weight | Poor: 0–1 | Pilot-ready: 3 | Strong: 5 |
|---|---|---|---|---|
| 1. Incremental business value and workflow fit | 18 | Counts every answered call as recovered revenue; ignores delivery capacity and existing callbacks | Buyer-approved baseline, eligible-enquiry definition, contribution calculation, capacity constraint and alternative options | Repeatable evidence of additional completed-job contribution or realised cost reduction, after supervision and rework; finance can reconcile the result |
| 2. Safety, escalation and decision boundaries | 16 | Improvised diagnosis; unsupported prices; “human fallback” without an available destination | Approved prohibited actions and emergency routes; buyer-specific safety and escalation tests passed | Repeated variations pass; live exceptions are reviewed; permission controls constrain actions; changes trigger retesting |
| 3. Integration and booking integrity | 15 | A logo on a slide; email capture sold as confirmed scheduling; duplicate or wrong-site updates | Exact read/write actions demonstrated in the buyer’s environment; mapping, permissions, retries and reconciliation documented | Live records reconcile; concurrent bookings, stale availability, cancellation and timeout recovery work without unauthorised commitments |
| 4. Privacy, security and data governance | 12 | “GDPR compliant” without a data map, terms or access evidence | Processing roles, lawful-basis analysis, DPIA decision, contracts, retention, access and transfer arrangements reviewed | Controls are verified in configuration; access revocation and deletion/export exercises succeed; supplier changes and incidents have accountable handling |
| 5. Reliability, fallback and support | 10 | “24/7 AI” supported only during unspecified hours; outages silently lose enquiries | Named support responsibilities, monitoring, tested diversion and a workable manual process | Peak-load and dependency-failure drills pass; detection and restoration evidence is available; the office can operate during disruption |
| 6. Measurement and pilot design | 9 | Supplier dashboard is the only evidence; bookings presented as completed jobs | Buyer-owned baseline, comparison method, outcome definitions, raw exports and stop rules | Cohort-level results reconcile to jobs and costs; attribution limitations are disclosed; results remain useful after adjusting for channel mix and seasonality |
| 7. Full-year cost and commercial clarity | 7 | Low headline fee with unknown minutes, support, changes or cancellation costs | Itemised setup and recurring charges; usage scenarios; internal effort; renewal and exit terms | Actual invoices reconcile to the model; peak usage and supplier price changes are modelled; costs remain within approved controls |
| 8. Staff usability and customer experience | 5 | Office staff rekey everything; customers cannot reach a person; exceptions have no owner | Service manager approves scripts and queues; staff can correct, pause and escalate; accessibility alternatives are tested | Net workload reduction is observed; staff can operate the process without agency intervention; complaints and abandonment are monitored |
| 9. Ownership, portability and exit | 5 | Supplier controls critical accounts or numbers without a workable handover; export promises are vague | Account access, number arrangements, configuration rights, data formats and exit assistance are written down | Export and handover rehearsal succeeds; another operator can understand the workflow; residual access and data deletion are verified |
| 10. Delivery competence and accountability | 3 | Anonymous case studies, unclear subcontractors and no accountable implementation lead | Named delivery/support leads, relevant reference checks, milestone plan and clear responsibility split | Comparable delivery evidence, candid failure history, documented change management and referenceable support performance |
| Total | 100 |
How should different reviewers score it?
Have the owner or finance approver lead criteria 1 and 7; the service manager lead 2, 3 and 8; the technical/privacy reviewers lead 4 and 5; and the project owner lead 6, 9 and 10.
Score independently first. Reconcile disagreements by examining evidence—not by averaging a privacy reviewer’s failed control with a salesperson’s confidence.
Keep the weights fixed across the shortlist. For a back-office-only project, interpret safety as protection against unauthorised technical content and actions; do not award free points merely because there is no telephone agent.
Put the criteria to work
Score three supplier plans →PUT THE RESEARCH TO WORK