Most MSP annual reviews are a status update dressed up as a strategy meeting. The MSP walks through a slide deck, everyone nods, the contract renews. Chris McGlasson, fractional CIO and founder of ClearStack Advisory, has sat on both sides of this conversation, and he built a framework for companies in the $10M-$20M revenue range to run a review that actually surfaces problems before they become expensive: SCORE. Security, Cost, Outcomes, Roadmap, Escalation.
At this revenue size, IT spend is significant enough to matter but rarely has a dedicated executive scrutinizing it line by line. That's exactly the gap an MSP can quietly grow into, and exactly why an annual review needs real structure.
Why the standard annual review doesn't work
A typical review covers uptime percentages and a list of tickets closed. Those numbers are usually fine, because they're the easiest metrics for an MSP to control and present favorably. They rarely answer the questions that actually matter: is this relationship still the right fit, is the cost structure still reasonable, and is the roadmap keeping pace with where the business is going. Chris McGlasson built SCORE around those gaps.
S: Security
- What's our current patch compliance rate across all endpoints, and where are the gaps?
- When was our last third-party security assessment, and what did it find?
- Walk me through what happens in the first hour of a ransomware incident. Who gets called, and in what order?
- What's our actual backup recovery time, tested, not theoretical?
Security answers should come with specifics. A vague "we've got it covered" is the answer that should end the meeting and start a search for a new provider.
C: Cost
- What percentage of our monthly spend is fixed versus billed as extra?
- Show me the last three months of overage or out-of-scope billing, itemized.
- If we cut headcount or added a location, how would that change the contract?
- Are we paying for licenses or seats nobody's using?
Cost transparency is one of the clearest signals of the relationship's health. An MSP that hesitates on this question is protecting margin at the client's expense.
O: Outcomes
- What were the three biggest IT-related business risks a year ago, and where do they stand now?
- Give me one initiative from the roadmap we discussed last year that didn't happen. What changed?
- How does our downtime this year compare to last year?
- What's one thing you'd have done differently if you were fully accountable to our P&L?
Outcomes questions separate a vendor that manages tickets from a partner that manages risk. Chris McGlasson notes that the last question is the one that tends to get the most honest answer, since it asks the MSP to think like an owner instead of a contractor.
R: Roadmap
- What's changed in our environment or industry that should shift our IT priorities next year?
- Where are we behind peer companies our size on infrastructure, security, or tooling?
- What's the plan for AI-related tools and policy, and who's driving it?
- If our headcount doubled in 18 months, what would break first?
A roadmap that looks identical to last year's is a sign the relationship has stalled into maintenance mode.
E: Escalation
- Who is our actual point of contact when something goes seriously wrong, by name?
- What's the average time from a critical ticket being opened to a senior engineer engaging?
- What happened the last time something did go seriously wrong, and what changed afterward?
- If we needed to add capacity fast, what's the real turnaround time?
Escalation questions matter more than any SLA on paper. The contract says four hours. The real answer is whoever picks up the phone at 11pm on a Saturday.
What to do with the answers
Vague or defensive answers on any of the five categories are worth a follow-up conversation, not necessarily an immediate exit. But a pattern across two or more categories, especially security and cost, is a strong signal the relationship needs either a hard reset or a replacement. Companies at $10M-$20M in revenue often don't have someone whose job it is to run this review with the rigor it needs. That's usually where a fractional CIO earns their retainer in a single meeting.
Not sure where your IT really stands? Start with a Clarity Audit: a focused assessment of your backups, security, vendors, and AI readiness. It's the first step in every engagement I take on.
About the author: Chris McGlasson is a fractional CIO and the founder of ClearStack Advisory, where he helps mid-market companies build AI governance, IT strategy, and technology roadmaps without the overhead of a full-time CIO. He previously built and sold LANPRO Systems, an IT services firm serving 350+ enterprise clients.