AI Bias Audit Template for AI Products

Run a full audit before every model launch and after every retraining cycle. For long-running production models, schedule periodic audits quarterly. If your model processes sensitive decisions (hiring, lending, healthcare), increase the frequency to monthly.

There is no single correct answer. Demographic parity (equal outcomes across groups) is the simplest to measure. Equalized odds (equal error rates) is more nuanced. The choice depends on your domain and the type of harm you are most concerned about. Document why you chose your metric.

You can use proxy analysis with geographic, linguistic, or behavioral signals. You can also run bias audits on synthetic test sets designed to represent specific demographic scenarios. The key is to test for disparities even when you lack perfect demographic labels.

The 4/5ths rule (a disparity ratio of at least 0.80) comes from US employment law and is a common starting point. Some teams use tighter thresholds (0.90) for high-impact decisions. Define your threshold based on the severity of potential harm and document the rationale.

It depends on the severity and the decision impact level. Critical findings in high-impact systems should block launch. Medium findings in low-impact systems may proceed with a documented mitigation plan and a follow-up audit deadline. The sign-off section of this template forces that decision to be explicit and recorded.