The surge in popularity of Large Language Models (LLMs) after the widespread introduction of pre-trained chatbots in late 2022 has been astonishing. Companies, keen to leverage the capabilities of LLMs, are swiftly incorporating them into their systems and customer-oriented services. However, the rapid rate of LLM adoption has exceeded the development of robust security measures, making many of these implementations susceptible to significant vulnerabilities.

There was a clear gap in consolidated resources addressing security issues in LLMs. Developers, not fully aware of the unique risks tied to LLMs, were navigating fragmented information. Given this landscape, OWASP’s mandate appeared ideally suited to promote a more secure integration of such technology.

The table that follows provides an overview of the OWASP Top 10 concerning LLMs. In our subsequent article, we’ll delve into each item, examining potential misuse and recommended countermeasures.

The following information has been summarised from a recently published OWASP Top 10 for LLMs.

LLM01: Prompt Injection: This occurs when malicious user input (prompts) directly or indirectly manipulates the LLM to alter its intended behavior, bypass security controls, reveal sensitive information, or execute unintended actions.

LLM02: Sensitive Information Disclosure: Vulnerabilities where the LLM or its associated application unintentionally or maliciously reveals sensitive data, such as personal information, proprietary code, or confidential business details, potentially impacting user privacy or system security.

LLM03: Supply Chain Vulnerabilities: Risks arising from compromises or weaknesses in the LLM’s development, training, deployment, or operational components and dependencies. This includes insecure third-party models, compromised training data sources, or vulnerable infrastructure.

LLM04: Data and Model Poisoning: Malicious actors intentionally corrupt or manipulate the pre-training, fine-tuning, or embedding data used to develop the LLM. This can lead to biased, compromised, or exploitable model behavior, affecting its accuracy, safety, or integrity.

LLM05: Improper Output Handling: Insufficient validation, sanitization, or filtering of LLM-generated outputs before they are processed by downstream applications or displayed to users. This can lead to various vulnerabilities like Cross-Site Scripting (XSS), server-side injection, or remote code execution.

LLM06: Excessive Agency: Granting an LLM-based system too much autonomy or control over external systems, actions, or sensitive operations without sufficient human oversight or robust safety mechanisms. This can lead to unintended, unauthorized, or harmful actions in the real world.

LLM07: System Prompt Leakage: The LLM inadvertently or maliciously reveals its hidden system-level instructions, configurations, or contextual information (the “system prompt”) to end-users or attackers. This could expose sensitive operational details, intellectual property, or aid further prompt injection attacks.

LLM08: Vector and Embedding Weaknesses: Vulnerabilities arising from the manipulation or misinterpretation of vector representations and embeddings used by LLMs. This can impact data retrieval, semantic search, or lead to unintended model behavior, potentially allowing for data exfiltration or denial-of-service.

LLM09: Misinformation: The LLM generates or propagates incorrect, biased, or misleading information (often referred to as “hallucinations”) as a core vulnerability. This poses significant risks to the reliability, trustworthiness, and safety of applications that depend on its factual accuracy.

LLM10: Unbounded Consumption: Insufficient controls on resource usage by the LLM, leading to excessive consumption of computational resources (CPU, GPU), memory, API calls, or external service interactions. This can result in denial-of-service, financial exploitation, performance degradation, or system instability.

References

OWASP Top 10
for LLM – https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v1_0.pdf