We wrote a technical deep-dive on building a cohort pLTV model with AI assistance, and a reader made a point we can’t argue with: it was a senior data scientist’s guide to managing AI juniors. Great if you speak survival models. Not so great if you’re the CEO or PM who wants to point Claude at the problem and mostly needs to know what to demand from it.
So this is the same playbook with the chemistry taken out and the recipe kept. The whole promise of vibe-coding is that the AI handles the mechanics. Your job is the logic and the guardrails, and that job matters more than it sounds, because the expensive failures in this kind of project are not coding errors. They’re business logic errors that run without complaint.
Everything below is built for one concrete configuration, because abstract LTV advice is easy to nod along to and impossible to use.
Say you’re the CEO of a consumer subscription app. Seven-day free trial, 60% of trials convert to paid in a normal week. Conversion swings hard with the calendar: around 70% in Q5, the post-holiday “fifth quarter” when resolution energy peaks and ads go cheap, and closer to 40% in the dead season. At conversion, 70% of payers take the monthly plan at $9.99 and 30% take the yearly at $69.99. And about 5% of new subscribers back out right away, refunding or canceling within days, almost always on the yearly plan, because $69.99 landing on a card statement triggers second thoughts that $9.99 never does.
If you run a subscription app, none of this will feel made up.
Before anyone writes code, push the configuration through by hand. It’s fifteen minutes of arithmetic, and it settles most of the argument.
Take 100 trial starts at 60% conversion. That’s 42 monthly payers and 18 yearly payers, less the retraction on yearly. In week one, monthly brings in about $420 and yearly about $1,197. So yearly is 74% of the cash you can actually see. Play the same cohort forward, though, and it flips: monthly subscribers stick around for roughly seven payments on average, so that side ends up worth about $2,937, while the yearly side lands around $1,556 after roughly 30% of them renew.
Yearly is three quarters of the money you can see and a third of the money that actually exists. Monthly is the exact mirror image. Sit with that for a second, because everything that goes wrong later goes wrong here.
We’ve sat in enough meetings where a promo campaign was crowned the winner off early returns and budget had already moved, and nobody had clocked that the promo also pushed everyone toward the yearly plan. Run that promo through this app: plan mix goes 50/50, week-one cash jumps about 42%, and the true value of the cohort moves about 4%. The campaign didn’t get better. The cash just showed up earlier.
Seasonality plays the same trick. A model that has quietly settled on “conversion is about 60%” will undervalue your January cohorts by about 14% and overvalue your dead-season cohorts by about 50%, and your budget will follow it in both directions.
If you take one thing from this post: plan mix and seasonality are the problem. Which algorithm the AI picks is a rounding error on top.
The single most important instruction you will give the AI is about shape. Left alone, it will train one big model on a revenue column and hope the features sort it out. They won’t, and worse, a single big model hides its reasoning, so when it’s wrong you find out in the ad budget.
Think of it like building a car. You don’t build the whole thing in one lump; you build the transmission, the fuel system, and the wheels separately, and each one gets tested on its own bench before assembly. Here, the benches are five plain-language questions:
Each question becomes a small model the AI can build in isolation, and, more importantly, each one produces a number you can sanity-check against figures your finance team already tracks: conversion rate, refund rate, month-two retention. The AI then combines the five pieces by simulating around ten thousand possible futures for each monthly batch of customers, which is what turns five small answers into a forecast with an honest range around it, instead of one number with false confidence.
One flag to keep waving: question five. For the first year of this model’s life, none of your yearly subscribers has reached their renewal date yet, so that number is an educated guess, not a measurement. Insist that it’s labeled as a guess everywhere it appears, because someone downstream will otherwise treat it as fact.
You don’t need to understand these libraries. You do need to hand the AI the right list, because left to its own devices it will pick the wrong one (more on that in a moment). Paste this into your first prompt:
Two traps are worth knowing about even at this altitude. First, the famous-library trap: the most popular LTV library out there, lifetimes, was built for businesses like retail, where you never actually observe a customer leaving. A subscription business watches every renewal happen. AI assistants recommend lifetimes constantly, because it has the most GitHub stars, and if you don’t stop them, your results will be fundamentally broken even though the code looks clean and runs fine.
Second, the payment-retry trap: when a card fails, billing systems retry it for a few days, and to a naive model that window looks like a wave of cancellations right after every billing date. It isn’t churn, it’s plumbing. Retention gets counted after the retry window closes.
We’ve watched this exact build happen enough times to know the pattern. The AI gets the pipeline running in hours instead of weeks, which still feels a little ridiculous. It also makes the same six mistakes nearly every time, and all six are silent: the code runs, the dashboard fills up, the numbers are wrong.
It grabs the famous library. Covered above. Wrong tool for a subscription business, chosen for its popularity.
It lets the model peek at the future. The default way to test a model splits the data randomly, which quietly mixes next quarter’s outcomes into the training data. That’s like grading a forecaster after showing them the answers. Every test has to split by calendar time: train on the past, test on what came after.
It grades unfinished customers as failures. Someone who signed up 20 days ago hasn’t had time to generate six months of revenue, and the lazy version records them as a zero. The model then learns that recent customers are worthless, which in practice means it tells you to cut your newest campaigns. This one has real teeth: it systematically punishes whatever you launched last.
It uses information that didn’t exist yet. Predicting a customer’s value “as of day one” while accidentally using data from day ninety. Impressive test scores, useless in production.
It fumbles the calendar. Hardcoded 30-day months, and timezone slips at the trial boundary. A trial that starts at 23:50 in Copenhagen converts on “day 8” in UTC, and yes, that one has burned us.
It reports the flattering number. There are accuracy metrics that average away exactly the errors that cost money. The AI gravitates toward them. What you want is accuracy reported at the level where you make decisions: per plan, per season, per channel.
The fix, we’ve found, is not better prompting in the moment. It’s writing the rules before the code. The AI gets the five-piece structure, the shopping list, and a list of forbidden moves up front. It has to prove itself on a dress rehearsal first (next section), and no predictions get shown before the validation report exists. Think of it as a brilliant new hire with no memory of your last incident review: what you get back is determined by the acceptance tests you set, not by the code it types.
Six gates, in order, and a failed gate stops the line. Skipping one because the board meeting is on Thursday is how these things end up allocating money badly.
Gate 0, the dress rehearsal. Have the AI generate fake data where we planted the answers: the 60% conversion, the seasonal swing, the 70/30 plan mix, the 5% retraction. Then make the pipeline find them. A model that can’t recover numbers we hid on purpose has no business estimating ones nobody knows. Keep the fake-data generator forever; it becomes the regression test for every future change.
Gate 1, beat the dumb baseline. The model has to outperform the simplest possible alternative: last quarter’s realized multiplier, applied naively. If the fancy model can’t beat that, something is leaking or the machine learning is adding noise, and either way you ship the simple version. Nobody ever got fired for shipping the baseline, though a few people probably should have gotten promoted for it.
Gate 2, accurate where the money moves. Within 5 to 10% per plan, per season, and per top channel. A model can rank your campaigns in the right order and still be 30% off on the totals, and a budget needs both.
Gate 3, honest about uncertainty. The model’s 80% confidence ranges should be wrong about 20% of the time. If they’re never wrong, it’s not accurate, it’s overconfident, and the usual culprit is that yearly-renewal guess from earlier being dressed up as a fact.
Gate 4, stress tests. Rerun the promo scenario (plan mix shifted hard toward yearly), hide Q5 from the model entirely and make it predict one cold, drop a random month of history. You’re looking for graceful degradation, not a cliff.
Gate 5, shadow mode. One full budget cycle where the model runs next to your real decisions without touching them. Boring, and unskippable.
The last test is the simplest to describe: pretend it’s last January, let the model predict with only the data that existed then, and score it against what actually happened. Repeat for every month you have. The catch is that this replay has to cover a full seasonal cycle, including a Q5 and a dead season; with conversion swinging between 40 and 70%, a replay that misses either extreme tells you nothing.
Two habits make the replay honest. Keep every old version of the model on file, so the test uses the model that would have existed at the time, not today’s improved code pretending it was always this smart. And score the decision, not just the prediction: rerun your actual budget rule against what happened and ask whether it would have made you money. A model can get more accurate on average while getting worse exactly where the budget moves.
This playbook ships alongside the post as a Claude skill, cohort-pltv-builder.skill, so the discipline travels with the model instead of living in someone’s head. Once saved, it kicks in whenever someone asks Claude to build, forecast, or debug an LTV model, and it enforces the sequence above: interview, spec, dress rehearsal, five small models, guardrails, historical replay. The forbidden moves from this post are baked in as a checklist Claude runs on itself before showing results.
Claude can write this code without the skill. What it can’t do without the skill is refuse to show anyone predictions before the validation report exists, and that refusal is worth more than the code.
If you want the statistical mechanics behind all of this, the technical companion piece has the full detail, along with the research it stands on: Fader and Hardie’s retention work, Google’s lifetime value paper, Kuaishou’s industrial-scale system, and the honest limits on how predictable a single customer can ever be.
Our technical deep-dive on building a cohort pLTV model with AI assistance was, as one reader put it, a senior data scientist’s guide to managing AI juniors. This is the same playbook for the CEO or PM who just needs to know what to demand from the AI, and where it will quietly get the business logic wrong.
