System prompts—developer-configured instructions that steer foundation models—are widely used across commercial AI products but rarely disclosed. AISPA treats prompts as an audit surface, arguing that their opacity creates concrete trust and accountability gaps; a structured, span-level taxonomy lets auditors trace individual instructions and quantify user-facing protections and harms.
Key Findings
- Large variability across developers: some products average over 60 protective instructions while others have fewer than 5, which means user protections are uneven and organization-dependent.
- Protective but shallow: 98.9% of products include at least one protective instruction, yet only ~24% cover all eight AISPA dimensions — so protections are common but incomplete.
- Growing prompt length and mixed intent: prompts have become longer and more protective over time, but roughly 40% of products still contain at least one instruction that works against user interests, and protective and problematic instructions often coexist — implying technical fixes alone won’t eliminate governance risks.
- Practical implication: audits can pinpoint specific problematic spans for remediation, enabling targeted governance, disclosure policies, and standardization efforts rather than coarse, product-level judgments.
Who it's for and tradeoffs
Great fit if you are a regulator, product security or compliance team, or researcher who needs a repeatable, user-centered method to evaluate how deployed prompts affect users. The framework is actionable for comparative audits and remediation prioritization. Look elsewhere if you need a deployment HOWTO or code-level prompt-injection mitigations: AISPA focuses on policy, traceability, and high-level taxonomy rather than runtime exploit fixes.
Methodology (brief)
The paper defines eight audit dimensions grounded in established human-rights norms, annotates prompts at the span level as protective (+1) or problematic, and reports aggregated statistics from an expert audit of 3,249 prompt instructions taken from 88 commercial AI products. This span-level approach enables targeted recommendations for developers and auditors.