LLM AI Security & Governance Checklist

The rapid rise of generative AI (GenAI) is set to impact users and businesses significantly. Alongside its potential for discovery, efficiency and growth, it introduces new security and privacy challenges — so organisations need a clear plan for managing their use of AI.

To help, OWASP has published a governance checklist. The summary below highlights the key areas for leaders in technology, security, privacy, compliance and legal roles.

Adversarial risk

• Review how competitors are investing in AI, and the market implications.
• Threat-model how attackers might use GenAI against your organisation, staff and customers.
• Update your incident response plan and playbooks to cover LLM incidents.

AI asset inventory

• Catalogue existing AI services, tools and owners, and add AI components to your Software Bill of Materials (SBOM).
• Record AI data sources and their sensitivity.
• Determine whether penetration testing or red teaming of deployed AI is required.

Training and awareness

• Train users on ethics, responsibility and legal issues such as licensing and copyright.
• Update security awareness to include GenAI threats such as voice and image cloning and enhanced spear phishing.

Governance

• Establish an AI RACI chart and assign responsibility for AI risk and governance.
• Set data-management policies, including classification and usage limits, so models only use appropriately classified data.
• Create an AI policy and publish an acceptable-use matrix for GenAI tools.

Legal and regulatory

• Review warranties, terms and conditions, and end-user licence agreements for GenAI considerations.
• Assess intellectual-property, copyright and indemnification risks, and confirm insurance coverage.
• Check compliance requirements around hiring tools, electronic monitoring and facial recognition.

Implementing LLM solutions

• Threat-model components and trust boundaries; enforce least-privilege access and defence in depth.
• Secure the training pipeline, and validate input and output handling.
• Include application testing, source-code review, vulnerability assessment and red teaming before release.
• Require third-party audits and penetration testing for external providers, and rehearse LLM incidents in tabletop exercises.