Recommender Systems and Personalization for PMs

A recommender system trained to maximize immediate engagement will, by design, learn that showing a user more of what they've already responded to positively is a reliable way to generate the next positive response. This is not a bug; it is the system correctly optimizing its stated objective. The trap is that "more of what worked before" is a strategy with a hidden long-term cost: it can narrow a user's exposure over time, reducing the surface area of the catalog they ever encounter, which can degrade the sense of discovery and serendipity that made the product valuable in the first place — a cost that is invisible to any metric measuring only the immediate response to what's currently being shown.

This is the recommender-system-specific version of Goodhart's Law from Lesson 41: optimizing directly and exclusively for a short-term proxy (immediate engagement) can actively work against the long-term outcome (sustained, satisfied usage) it was meant to serve as a stand-in for.

This lesson introduces the Discovery Frontier, a model dividing the space of everything a recommender could show a user into three zones:

A recommender that only ever serves the Known Preference Zone will maximize short-term click-through but risks the filter-bubble narrowing described above. A recommender that pushes too aggressively into the Irrelevant Zone in the name of "diversity" will simply frustrate users with content that has no plausible connection to anything they've shown interest in. The Discovery Frontier — the zone of content that is adjacent to known interest, plausibly relevant, but not yet explored by this particular user — is where a well-designed recommender should deliberately spend a meaningful fraction of its recommendation slots, since this is the zone where genuine discovery, and the resulting durable increase in a user's sense of the platform's value, actually happens.

The discipline of the Discovery Frontier model is treating the balance between these zones as an explicit, tunable product decision — what fraction of recommendations should come from the Known Preference Zone versus the Discovery Frontier — rather than an emergent, unexamined side effect of whatever the engagement-maximizing model happens to converge on by default.

Traditional recommender evaluation metrics (precision at k, recall at k, click-through rate) measure only whether the model correctly predicted what a user would click — they are, by construction, Known Preference Zone metrics, and a model can score extremely well on them while still producing a narrowing filter bubble. Diversity metrics measure how varied the recommended set is, both within a single list and across a user's recommendations over time. Novelty metrics measure how much of what's recommended is genuinely new to that specific user, as opposed to a repeat of previously-shown content reframed. A responsible recommender evaluation reports accuracy-style metrics alongside diversity and novelty metrics, since optimizing for the former alone can directly and measurably degrade the latter two.

Recommender systems face a specific structural challenge called the cold-start problem: the system has little or no interaction history for a new user (or a new item newly added to the catalog), making it difficult to generate personalized, relevant recommendations using the same collaborative or interest-based signals that work well for established users and items. Common mitigation strategies include using onboarding preference surveys to gather explicit initial signal, falling back to popularity-based or editorially curated recommendations until sufficient behavioral data accumulates, and using content-based features (an item's inherent attributes, rather than other users' behavior) to make reasonable initial matches even with no interaction history at all.