Guide
Business outcomes as the reward.
How to fine-tune a support agent on what its replies caused: renewals, repeat orders and churn, not what a judge thought.
To fine-tune a support agent on business outcomes, join each conversation to what the customer did afterwards, predict that outcome on the day of the conversation, and use the prediction as the reward in reinforcement fine-tuning. An LLM judge stays as a guardrail for policy and tone, not as the reward.
Updated November 2026. Written by the team that builds Churney’s reward layer.
Why not just use a judge
A judge scores how a reply looked. Train against it and the agent learns the replies the judge likes: more apologies, more discounts, more escalations. The outcome you care about, a renewal, a repeat order, a customer who did not churn, arrives 30 to 120 days later. That is too late to train on directly, so you predict it.
Step by step
Nine steps, from logs to a live agent.
- Pick one outcome and a horizon. For example net revenue at day 90, retained at day 90, or a second order within 60 days. Write down how finance would measure it.
- Join conversations to outcomes. In your warehouse, key every conversation to a customer ID and join orders, subscriptions, refunds and tickets. Keep only customers whose outcome has matured past the horizon for training.
- Predict the outcome on day 0. Train a model that predicts the outcome from the customer’s history plus the conversation, and check its calibration on held-out customers whose outcomes have landed.
- Correct for how past decisions were made. Your logs reflect the old agent’s choices, such as who got a refund or a pause. Without a correction the reward learns those habits instead of their effect.
- Turn the prediction into a reward. Score each reply by the predicted outcome with that reply, relative to what you would expect for that customer anyway. Scale it so a typical reply scores near zero.
- Run reinforcement fine-tuning. Use the platform you already have. We have run the same RFT tasks on Vertex AI, Together, Fireworks, Tinker and SageMaker.
- Gate before traffic. Every candidate passes your evals, a policy judge for tone and rules, and a human approval before it sees customers.
- Test on live traffic and wait for outcomes. Keep a holdout on the current model and compare realized value per customer once the horizon has passed.
- Recalibrate as outcomes land. Compare each predicted reward with the outcome it predicted and retrain the reward model, so the agent cannot drift towards replies that only fool the prediction.
What you need
- Conversation logs with a customer ID on every conversation.
- Orders, subscriptions or refunds in a warehouse, with timestamps.
- Enough history that outcomes have matured for the horizon you picked.
- A fine-tuning platform for the model behind the agent.
FAQ
Questions we get.
Can the reward be the real outcome instead of a prediction?
Only if you can wait for it. A day-90 outcome means a 90-day delay on every training example. A calibrated prediction gives a reward on day 0 and is checked when the outcome arrives.
Does raw customer data go into training?
It does not need to. Predictions can run in your own tenant or warehouse, and training sees prompts and scores, never raw customer records.
Where does Churney fit?
Churney supplies the reward: it connects to your warehouse, predicts the outcome of each interaction and scores it, on the training platform you already use. Since 2022 the same kind of predicted value has been the bidding signal on Meta, Google and TikTok for 30+ brands. Continual calibration is available to design partners.
Bring your agent. We bring the reward.
A 30-minute call with a research engineer about your agent, your outcomes and your training setup.
Book a callMore on the idea: the reward layer for AI and the post that introduced it.