Six Weeks of Discovery

In August I joined a mid-sized B2B SaaS company for a six-week discovery. The brief was open: find where AI could cut cost or grow revenue, prove it with numbers, and put a plan in front of the CEO and CTO that they could fund one stage at a time.

What I found on day one was not a blank page. It was the opposite. Customer Success had an AI feature inside their platform scoring calls, and nobody trusted the scores. Product had built a transcript analysis tool that had quietly been switched off. Marketing was routing leads through a Google Sheet by hand. Engineering was spending around $3,000 a month on AI coding tools with no measure of what it bought. Someone had run a single automated job that burned 700,000 of a 1,000,000 credit allowance in one go. And across all of it sat a question nobody had answered: were any of these tools processing customer data somewhere the contracts did not allow?

The company did not have an AI problem. It had a visibility problem. Every team had bought or built something, each for a sensible local reason, and there was no shared picture of what those things were, what they cost, what they were supposed to fix, or whether they were working. Approval for anything new had stalled because the leadership team, quite reasonably, wanted to see value before spending more, and nobody could show them any.

This post is about how I turned that into a register of 51 evidenced problems, a set of costed business cases, and a staged roadmap in six weeks. It is also about why I think this kind of work sits more naturally with a delivery lead who understands AI than with a business analyst or an outside consultant. I will try to be concrete about both.

Why the usual routes struggle here

The obvious move is to hire a business analyst or bring in a consultancy. Both are good at parts of this. Both, in my experience, run into the same three walls.

The first wall is that the problems do not arrive as requirements. They arrive as a Customer Success manager saying "I don't trust the scores" and a product lead saying "we turned it off because the answers were wrong." To get from there to something you can act on, you need to know why a retrieval-based tool gives wrong answers when asked to summarise across thousands of documents, or why a scoring model that looks fine on eight calls falls apart on fifty. A BA documents what the user says. An AI-literate delivery lead can hear the symptom and name the mechanism, which changes the question you ask next.

The second wall is money. Leadership did not want a list of ideas. They wanted to know, for each one, which line in the P&L moves, by how much, and how we would know. That means building a baseline from system data, pricing a run cost before anything is built, and being honest about which numbers are facts and which are the analyst's guess. Consultancies tend to produce the ROI slide at the end. The work here was to build the ROI from the evidence as you go, so that every figure traces back to a named person, a date, and a source.

The third wall is that the answer has to be delivered by the people in the building, with the engineers they actually have, next quarter. A consultant's roadmap is often a target operating model that assumes capacity which does not exist. A delivery lead sizes every stage against the real team, and designs the first stage so that it can be killed cheaply if the assumption behind it turns out to be wrong.

None of this is a criticism of analysts or consultants. It is a statement about where the work sits. Discovery for AI is a delivery problem from the first interview, because the only questions worth asking are the ones whose answers change what gets built, in what order, and how success will be measured.

How the six weeks ran

Weeks one and two: listen, record, write it down as they said it. I interviewed people across Customer Success, Product, Marketing, Engineering and Support, eleven sessions in all, every one recorded and transcribed. Before the first interview I built two reference lists: who owns what, and what each system is, who runs it and what it costs. That meant interview time went on problems rather than decoding acronyms.

Each transcript became problem statements in a register. The rules were strict. Every statement is one business sentence, in the words of the person who said it, with the verbatim quote, the speaker and the date attached. Each claim is marked as stated, implied or inferred by me. Where evidence ran out, the entry says "Unknown, ask [name]" rather than a plausible guess. Where two people gave different numbers for the same thing, both numbers stay visible with a note on who resolves it. And the register carries no solutions, no costs and no ranking. That neutrality is what made it possible for six departments to agree it was accurate. By the end of week two there were 47 entries; it finished at 51.

Week three: confirm, then prioritise in the room. Each department got its statements back to review. Where someone reworded a statement, their wording won and the change was logged. In the same conversation I asked what would be true if the problem were gone, which gave every statement a testable outcome. Then I mapped what the problems had in common underneath. Eight mechanisms turned up again and again, and seven of them had fixes of under twenty hours. I proposed a top ten with the criteria written down so the leadership team could argue with the criteria rather than with me. The decision stayed with them.

Weeks four and five: questions, business cases, money. For each prioritised problem I walked an empty business case section by section. Every field I could not fill became a numbered question that named the field its answer would populate, with an owner and a priority of blocking, shaping or optional. Those question lists, grouped by owner, were the workshop agendas. Answers were written into the case while the person was still in the room to check them. Five business cases came out of this, each with a baseline, a run cost, a hard spend ceiling and a measurement plan, grouped into roadmaps where each stage has to earn the next.

Week six: test the biggest claim against real data. Before the review with the CEO and CTO, we put the most contested assumption to a direct test, which I describe below. The review itself went line by line through the numbers, which is exactly what the previous five weeks had been built to survive.

Where AI knowledge changed the answer

Two moments in the six weeks would, I think, have gone differently without hands-on AI experience.

The tool that was switched off. Product had built an internal tool to answer questions across the company's call transcripts, between 2,000 and 4,000 new calls a month. It gave wrong answers and was turned off. The register entry could have stopped at "transcript tool unreliable, rebuild it." Reading the design, the cause was clear: it was a retrieval system, built to find and summarise a handful of relevant passages, and it was being asked questions like "what are customers saying about pricing this quarter," which need every transcript counted, not five passages retrieved. That is a category error, not a quality problem, and no amount of prompt tuning fixes it. The replacement business case was therefore not "build a better chatbot" but "build a structured extraction pipeline over a copy of the transcript table, prove the cost per call on three months of Customer Success calls, then decide." The pilot's acceptance test was 25 to 30 hand-checked calls, built within 8 to 12 person-hours by correcting the pipeline's first pass. That is a different project, with a different cost, and a different answer to "why did the last one fail."

The scores nobody trusted. Customer Success had AI call scoring inside their platform, and the team leaders had opposite complaints: one said it was too harsh, one said it was too nice. The usual move is to collect more opinions. Instead the CEO asked for an A/B test on eight calls, and we ran it on twenty-one. Every transcript was read blind, against the team's own scoring rubric, with the platform's numbers masked, and written to file before any number was revealed. Then we compared. Across 176 section scores the platform ran generous by about three quarters of a point, in one direction only, for all four CSMs in both countries. The risk score was within one point of the blind read on every single call, and its narrative had caught the one cancellation notice in the data. But the scoring caps the rubric demanded were applied on some calls and not others, three sections defaulted to top marks regardless of the transcript, and the automation had silently never scored the highest-risk call in the set.

The conclusion mattered commercially. The model was fine. Every fault was configuration: prompts, caps, the trigger that chose which calls to score. That is a fix the company could make itself, inside the platform it already paid for, in days. Without the test, the most likely outcome was a six-figure replacement project for a tool that mostly worked.

Building the ROI from the evidence, not for the slide

The CEO's test for every item was the same: can this saving be measured? So the money was built in layers, with the same measure carried through each layer and written down once.

At the bottom, every one of the 51 problem statements got a single line called "where the money shows": the metric, its baseline and its source, tagged to one of six categories. Spend controlled. Cost avoided. Capacity released. Revenue protected or grown. Capability built. Risk and compliance. Two rules kept this honest. Capacity only counts as money if it changes a hiring, allocation or purchase decision; an hour saved that nobody redeploys is not a saving. And value that cannot be traced to a P&L line is recorded as unscored upside, never invented.

Take the Customer Success example. CSMs were spending around 30 minutes preparing for each call, running three calls a day, and about a third of booked meetings did not show up against a target of 20 percent. Across a team of roughly 44 CSMs that is about five hours a week each of prepared-but-unused time. The CS director had already said the aim was a fourth call a day. So the measure was not "time saved"; it was no-show share multiplied by prep time multiplied by staff cost, with a baseline captured before anything changed, and the test for success was whether the fourth call actually happened. The leadership team's challenge was fair: validate that with a CSM's real day, not arithmetic. That became a prerequisite task on the roadmap rather than a number on a slide.

One layer up, problems that shared a mechanism were grouped into clusters, and each cluster carried a P&L test: if this fix lands, these lines move, with each problem's measure counted once. At the top, each roadmap tagged every measure to the delivery stage whose outcome would move it, with baselines taken before that stage runs and unmapped measures shown honestly as not yet scheduled.

Run costs were treated with the same suspicion as benefits. One team's AI agent had been estimated at £80 a month and billed at £7.95; nobody knew which was right, so valued work sat switched off. A standing rule was that no case carries an unknown run cost silently. The first stage of the biggest roadmap was designed to produce a measured cost per call against forecast, because that one number priced every stage behind it.

All of this lived in a small register app I built with a no-code tool, with a commercials page that generated a multi-sheet calculator workbook on demand, so the finance conversation could happen in a spreadsheet the CFO could change. The CS figures were walked through and agreed with the CS leads before they went anywhere near the CEO. Numbers the function heads had not validated were labelled as mine.

What the delivery background actually did

Looking back, the AI knowledge got the diagnosis right. The delivery background is what made the output fundable.

It showed up in the shape of the roadmap. Each roadmap had exactly one funded stage at a time. Everything behind it was a draft with a one-line promotion condition: "promotes when stage one returns a measured cost per call under ceiling." That is governance between the stages and agile inside them, and it exists so that a bad assumption dies at stage-one prices. A list of recommendations cannot do that. A sequence of gated bets can.

It showed up in what counted as a stage. Every stage had to end in something falsifiable: either a learning verdict ("assumption X confirmed or killed, deciding Y") or a value verdict ("adopted by N users, measured against baseline"). A stage that could not name the decision it unblocked was curiosity, and it was cut. The same test applied to the discovery questions: a question that could not name the business case field it would fill did not get asked.

It showed up in the separation of concerns. Problems lived in the register with no solutions attached. Solutions and money lived in business cases. Prioritisation lived in the meeting. Mixing those three is what makes most opportunity registers unusable within a month, because every entry becomes somebody's pet project. Keeping them apart is a delivery discipline, not an analysis technique.

And it showed up in sizing against the real team. The CTO's framing late in the work was the right one: split everything into what the AI lead can deliver alone with prompts and configuration, what needs one light dependency such as someone standing up a data store, and what is an engineering build. Quick wins from the first lane go first as proof. Builds follow once the loop has been proven once, end to end. Non-AI fixes, such as a no-show problem that may be a booking-practice problem, get called out separately rather than smuggled into the AI plan.

My standing line to businesses is "data, process, then AI." Six weeks of this work confirmed it in the other direction too: most of what looked like AI problems were data problems and process problems wearing an AI badge. You need someone who can tell the three apart, price each one, and sequence the fix so the business can afford to be wrong.

What I would do again, and what I would change

I would keep the evidence discipline exactly as it was. Every claim quoted, every gap declared, every contradiction left visible until someone with a name resolved it. It slowed the first two weeks and paid for itself in week six, when the numbers were challenged line by line and held.

I would keep the blind test. Reading twenty-one transcripts cold against the team's own rubric before looking at the machine's answer was the single most persuasive thing in the whole engagement, and it cost two days. Whenever a business is about to replace an AI tool because "it doesn't work," run that test first.

What I would change is when I sized the plan against the engineering the company actually had. I learned in week five that there was no spare Python capacity and nobody free to run models against the data stores. Had I asked that question in week one, the no-engineering lane would have led the roadmap from the start, with the builds as a visibly conditional second phase, and the conversation with the CEO would have been about proof already in hand rather than proof to come. Ask about capacity before you ask about ambition.

I would also budget a day earlier for a time-and-motion study with one real CSM. Arithmetic on averages is where every leadership team, rightly, pushes back hardest. A measured day beats a calculated week.

What I am looking for next. I want to do this again, inside a business that wants an AI lead to find where AI cuts cost or grows revenue, prove it with numbers, and deliver it into teams so it sticks. The six weeks above are the first forty percent of that job. The rest is shipping two or three improvements with before-and-after figures, proving the loop once end to end, and having every live item co-signed by its function head. I am equally happy to run the discovery stage on its own as a fixed engagement for a company that wants the register, the business cases and the calculator without a permanent hire.

If that sounds like a problem you have, I would like to hear about it.