OWASP Top 10 For LLM (Large Language Model)

by | Sep 5, 2023 | AI Security | 0 comments

Background

The surge in popularity of Large Language Models (LLMs) after the widespread introduction of pre-trained chatbots in late 2022 has been astonishing. Companies, keen to leverage the capabilities of LLMs, are swiftly incorporating them into their systems and customer-oriented services. However, the rapid rate of LLM adoption has exceeded the development of robust security measures, making many of these implementations susceptible to significant vulnerabilities.

There was a clear gap in consolidated resources addressing security issues in LLMs. Developers, not fully aware of the unique risks tied to LLMs, were navigating fragmented information. Given this landscape, OWASP’s mandate appeared ideally suited to promote a more secure integration of such technology.

The table that follows provides an overview of the OWASP Top 10 concerning LLMs. In our subsequent article, we’ll delve into each item, examining potential misuse and recommended countermeasures.

The following information has been summarised from a recently published OWASP Top 10 for LLMs.

OWASP Top 10 For LLMs

OWASP Top 10 Entry
LLM01: Prompt Injection
LLM02: Insecure Output Handling
LLM03: Training Data Poisoning
LLM04: Model Denial of Service
LLM05: Supply Chain Vulnerabilities
LLM06: Sensitive Information Disclosure
LLM07: Insecure Plugin Design
LLM08: Excessive Agency
LLM09: Overreliance
LLM10: Model Theft
Description
Manipulates a large language model (LLM) through crafty inputs, causing unintended actions by the LLM. Direct injections overwrite system prompts, while indirect ones manipulate inputs from external sources.
Vulnerability occurs when an LLM output is accepted without scrutiny, exposing backend systems. Misuse may lead to severe consequences like XSS, CSRF, SSRF, privilege escalation, or remote code execution.
Occurs when LLM training data is tampered, introducing vulnerabilities or biases that compromise security, effectiveness, or ethical behavior. Sources include Common Crawl, WebText, OpenWebText, & books.
Attackers cause resource-heavy operations on LLMs, leading to service degradation or high costs. The vulnerability is magnified due to the resource-intensive nature of LLMs and unpredictability of user inputs.
LLM application lifecycle can be compromised by vulnerable components or services, leading to security attacks. Using third-party datasets, pre-trained models, and plugins can add vulnerabilities.
LLMs may inadvertently reveal confidential data in its responses, leading to unauthorized data access, privacy violations, and security breaches. It’s crucial to implement data sanitization and strict user policies.
LLM plugins can have insecure inputs and insufficient access control. This lack of application control makes them easier to exploit and can result in consequences like remote code execution.
LLM-based systems may undertake actions leading to unintended consequences. The issue arises from excessive functionality, permissions, or autonomy granted to the LLM-based systems.
Systems or people overly depending on LLMs without oversight may face misinformation, miscommunication, legal issues, and security vulnerabilities due to incorrect or inappropriate content generated by LLMs.
Involves unauthorized access, copying, or exfiltration of proprietary LLM models. The impact includes economic losses, compromised competitive advantage, and potential access to sensitive information.

References

OWASP Top 10
for LLM – https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v1_0.pdf