Every ranking pipeline ships with a target metric. The choice of metric is the most consequential architectural decision you will make, and it is almost always the one made first, fastest, and with the least debate.
Click-through is easy to instrument and easy to defend in a planning meeting. It is also a metric that converges on a local maximum that looks great on a dashboard and erodes the product over six months.
The week-three problem
We watch week-three retention because week-one is contaminated by curiosity and week-two by sunk cost. Week three is the first measurement that reflects what the product actually does for the user.
Matching pipelines optimized for click-through reliably degrade week-three retention. The signal is mechanical: the model learns to surface candidates that produce engagement in the immediate term, and those candidates are disproportionately the ones who do not lead to longer interactions.
What we do instead
Optimize for the longest signal you can afford to wait for. If your engagement loop is weekly, optimize for week-three. If it is monthly, optimize for month-two. The lag is a feature, not a bug. It forces the model to learn the actual behavior you want to reinforce, not its shadow.
Pair the long signal with a small, fast feedback loop that catches obvious failures within hours. Long signal for quality, fast signal for safety. Neither replaces the other.