AI Data Requirements Template for AI Products

It depends on the task complexity and model type. Fine-tuning an LLM for classification might need 500-2,000 labeled examples. Training a custom ML model from scratch might need 50,000+. Start with the minimum viable dataset, evaluate model performance, and add more data where the model struggles. Quality matters more than quantity.

Synthetic data can supplement real data but should not replace it entirely. Use synthetic data for edge cases and minority classes where real examples are scarce. Always validate that synthetic data does not introduce [biases or hallucination patterns](/glossary/hallucination) that do not exist in real data. Label synthetic examples separately so you can measure their impact.

The PM owns the document. Data engineers own the pipeline specification sections. ML engineers own the quality standards and labeling sections. Legal owns the governance sections. The PM's job is to ensure all sections are complete and consistent, not to write every section alone.

Define a refresh cadence for each data source based on how quickly the underlying data changes. Customer behavior data might need daily refreshes. Industry benchmarks might be quarterly. Build monitoring that alerts when data distribution shifts significantly from the training distribution. This is called [data drift](/glossary/feature-flag) and it degrades model performance silently.

Define a priority order for data sources and document conflict resolution rules. For example: verified user-submitted data overrides inferred data, which overrides default values. Log conflicts for analysis. If conflict rates exceed 5%, investigate the root cause rather than relying on resolution rules.