Recommendation System Specification Template

Build a recommendation system when you have more than 1,000 items in your catalog and more than 10,000 monthly active users with interaction data. Below those thresholds, manual curation or simple popularity-based lists perform comparably with less engineering investment. The [AI PM Handbook](/ai-guide) covers the build vs. buy decision for AI features in detail.

Use a combination of offline and online metrics. Offline metrics (precision@k, NDCG) measure algorithmic accuracy against held-out data. Online metrics (CTR, items per session) measure real user behavior in A/B tests. Business metrics (revenue, retention) measure actual impact. No single metric captures recommendation quality. The [AI Eval Scorecard](/tools/ai-eval-scorecard) helps structure a multi-dimensional evaluation.

Filter bubbles occur when recommendations reinforce a narrow set of user preferences, reducing exposure to new topics. Prevent this by adding diversity constraints (maximum items from same category), exploration slots (reserve 1-2 positions for serendipitous recommendations), and monitoring catalog coverage over time. If coverage drops below 20% of your catalog in a 30-day window, your system has a filter bubble problem.

Use a tiered approach. For brand new users: popularity-based or onboarding questionnaire. For users with 1-5 interactions: content-based filtering on the items they engaged with. For users with 5+ interactions: full collaborative filtering. Define the thresholds for each tier in the Cold Start Strategy section and test them in an A/B experiment.

Daily batch retraining is sufficient for most products. Real-time training is only justified if your item catalog changes rapidly (news, social media) or if user preferences shift within a single session (e-commerce browsing). More frequent retraining increases infrastructure cost and complexity. Start with daily and increase frequency only if offline metrics show staleness between retraining cycles. ---