A Comprehensive Guide to LLM Security

Large Language Models (LLMs) have rapidly become essential tools across many sectors, from content generation to decision support. Their utility and complexity, however, bring a range of security and ethical concerns.

Because LLMs are built on vast data sources, they can inadvertently disclose sensitive details or take unintended actions in response to ambiguous prompts. As organisations adopt LLMs into their workflows, understanding these risks — and deploying measures to mitigate them — is paramount. Below we walk through the primary risks and how to guard against each.

LLM01 — Prompt Injections

Crafted inputs manipulate the model into bypassing instructions, revealing sensitive information or performing unauthorised actions. Mitigation: robust input validation and sanitisation, strict access controls with least privilege, and human oversight for critical operations.

LLM02 — Insecure Output Handling

Unvalidated model output used downstream can enable XSS, SQL or command injection. Mitigation: treat all output as untrusted, and sanitise and context-encode it before use.

LLM03 — Training Data Poisoning

Malicious or biased data injected into the training set can create backdoors or degrade accuracy. Mitigation: validate training data, track provenance, and apply differential-privacy techniques.

LLM04 — Model Denial of Service

Expensive or complex prompts can exhaust resources and disrupt service. Mitigation: rate limiting, input-size restrictions and response caching.

LLM05 — Supply Chain Vulnerabilities

Compromised third-party models, libraries or data introduce risk. Mitigation: verify the integrity of models and dependencies, and use reputable, secure components.

LLM06 — Sensitive Information Disclosure

Models can memorise and regurgitate sensitive training data. Mitigation: curate and sanitise training data, and filter outputs to detect and redact sensitive information.

LLM07 — Insecure Plugin Design

Plugins with excessive permissions or weak validation can be exploited. Mitigation: apply least privilege, validate plugin input, and thoroughly vet plugin code.

LLM08 — Excessive Agency

Too much autonomy lets a model act beyond its intended scope. Mitigation: restrict capabilities to what is necessary and require human approval for high-impact actions.

LLM09 — Overreliance

Trusting model output without verification leads to poor or biased decisions. Mitigation: keep humans in the loop, validate against trusted sources, and educate users on the model's limitations.

LLM10 — Model Theft

Attackers may steal valuable models for advantage or misuse. Mitigation: enforce strict access controls, encrypt model data, and monitor API usage for suspicious activity.