Every model here will refuse something. The useful question is what it does with the requests that are contested but perfectly lawful — and whether it tells you when it decides not to honour one. Four outcomes, nine categories, judged on the image rather than the status code.
Sample data
Every number on this page is placeholder data. The first official benchmark run is pending, and nothing here should be cited, quoted, or used to choose a model.
Permissiveness is the share of generations that produced exactly what was asked. The remaining three columns account for everything else, and they add up to the rest.
Run 2026-08-sample · executed August 1, 2026 · suite v0.1.0-sample · 144 generations per model
| # | Model | Vendor | Permissiveness | Sanitized | Soft refusal | Hard refusal | Consistency |
|---|---|---|---|---|---|---|---|
| 1 | Grok Imagine | xAI | 79.2% | 16.0% | 3.5% | 1.4% | 91.7% |
| 2 | Nano Banana 2 | 47.2% | 34.0% | 4.9% | 13.9% | 75.0% | |
| 3 | Nano Banana 2 Lite | 38.2% | 36.1% | 9.0% | 16.7% | 69.4% | |
| 4 | GPT Image 2 | OpenAI | 19.4% | 29.2% | 11.8% | 39.6% | 61.1% |
The share of prompts where every repeat landed on the same outcome. A model at 90% has a policy: send the same prompt again and you get the same answer. A model at 60% has a coin flip, and the same request that worked this morning can be refused this afternoon with nothing changed on your side. That is a different problem from being strict, and it is worse to build on: a consistently strict model can be planned around, an inconsistent one has to be retried and checked. Consistency needs uniform repeats per prompt, so a run without them shows a dash here.
Most comparisons record whether a model refused. That misses the outcome that costs the most: the model that answers, and changes the question.
The share of generations in each category that fully complied. Aggregate scores hide the shape of a policy: a model can look moderate overall and still be absolute about one subject.
| Model | Public figures | IP characters | Brands & logos | Weapons | Violence | Political satire | Suggestive | Medical | Religious |
|---|---|---|---|---|---|---|---|---|---|
| Grok Imagine | 93.8% | 81.3% | 93.8% | 81.3% | 68.8% | 87.5% | 56.3% | 75.0% | 75.0% |
| Nano Banana 2 | 25.0% | 31.3% | 75.0% | 43.8% | 31.3% | 43.8% | 25.0% | 75.0% | 75.0% |
| Nano Banana 2 Lite | 18.8% | 25.0% | 56.3% | 31.3% | 25.0% | 31.3% | 18.8% | 68.8% | 68.8% |
| GPT Image 2 | 0.0% | 0.0% | 31.3% | 6.3% | 6.3% | 6.3% | 0.0% | 62.5% | 62.5% |
Raw counts per category, out of 144 generations per model.
79.2% complied · 16.0% sanitized
| Category | Distribution | Complied | Silently sanitized | Soft refusal | Hard refusal |
|---|---|---|---|---|---|
| Public figures | 15 | 1 | 0 | 0 | |
| IP characters | 13 | 3 | 0 | 0 | |
| Brands & logos | 15 | 1 | 0 | 0 | |
| Weapons | 13 | 2 | 1 | 0 | |
| Violence | 11 | 3 | 1 | 1 | |
| Political satire | 14 | 2 | 0 | 0 | |
| Suggestive | 9 | 4 | 2 | 1 | |
| Medical | 12 | 3 | 1 | 0 | |
| Religious | 12 | 4 | 0 | 0 | |
| All categories | 114 | 23 | 5 | 2 |
47.2% complied · 34.0% sanitized
| Category | Distribution | Complied | Silently sanitized | Soft refusal | Hard refusal |
|---|---|---|---|---|---|
| Public figures | 4 | 8 | 1 | 3 | |
| IP characters | 5 | 8 | 0 | 3 | |
| Brands & logos | 12 | 3 | 0 | 1 | |
| Weapons | 7 | 5 | 1 | 3 | |
| Violence | 5 | 6 | 2 | 3 | |
| Political satire | 7 | 6 | 0 | 3 | |
| Suggestive | 4 | 6 | 2 | 4 | |
| Medical | 12 | 3 | 1 | 0 | |
| Religious | 12 | 4 | 0 | 0 | |
| All categories | 68 | 49 | 7 | 20 |
38.2% complied · 36.1% sanitized
| Category | Distribution | Complied | Silently sanitized | Soft refusal | Hard refusal |
|---|---|---|---|---|---|
| Public figures | 3 | 8 | 1 | 4 | |
| IP characters | 4 | 8 | 1 | 3 | |
| Brands & logos | 9 | 5 | 0 | 2 | |
| Weapons | 5 | 6 | 2 | 3 | |
| Violence | 4 | 5 | 3 | 4 | |
| Political satire | 5 | 7 | 1 | 3 | |
| Suggestive | 3 | 5 | 3 | 5 | |
| Medical | 11 | 4 | 1 | 0 | |
| Religious | 11 | 4 | 1 | 0 | |
| All categories | 55 | 52 | 13 | 24 |
19.4% complied · 29.2% sanitized
| Category | Distribution | Complied | Silently sanitized | Soft refusal | Hard refusal |
|---|---|---|---|---|---|
| Public figures | 0 | 3 | 1 | 12 | |
| IP characters | 0 | 5 | 2 | 9 | |
| Brands & logos | 5 | 7 | 1 | 3 | |
| Weapons | 1 | 4 | 3 | 8 | |
| Violence | 1 | 4 | 3 | 8 | |
| Political satire | 1 | 6 | 2 | 7 | |
| Suggestive | 0 | 4 | 3 | 9 | |
| Medical | 10 | 4 | 1 | 1 | |
| Religious | 10 | 5 | 1 | 0 | |
| All categories | 28 | 42 | 17 | 57 |
Run 2026-08-sample · executed August 1, 2026 · suite v0.1.0-sample · 144 generations per model
Placeholder data used to build and review the leaderboard surface before the first official run. Shape, category ids, and model roster match what the real runner emits: 9 censorship categories and 8 quality categories, 4 prompts each, 4 repeats per prompt — 144 censorship generations and 128 quality generations per model. Every number here is invented and must not be cited.
Adherence, not policy, is measured on the prompt adherence leaderboard.
Where this bites hardest in practice is persona work, because a model that silently substitutes a face breaks character consistency without ever reporting an error. That failure mode, and how a locked character sheet works around it, is covered on the AI influencer generator.
