What is an LLM?

LLMs in plain English

During training, an LLM has processed enormous amounts of text – books, websites, technical articles, and program code. In the process, it has learned how language is structured and which words typically follow one another. When you ask it a question, it calculates, piece by piece, the most likely continuation of the text.

You can think of an LLM as an exceptionally well-read but gullible intern. They write quickly and convincingly, know their way around many topics, and work around the clock. But they don’t check who is giving them instructions. And they answer confidently even when they don’t actually know something – experts call these “hallucinations.”

One thing is important: an LLM is not a database in which facts are reliably stored. It generates text based on probabilities. That’s why answers can sound plausible and still be wrong.

How an LLM works in a business setting

In business applications, the language model rarely works alone. These building blocks are typical:

1. The model: It understands the request and formulates the response. Many companies use a provider’s models via an interface (API); others run their own models.
2. The system prompt: Hidden instructions define how the application should behave – for example, “Only answer questions about our products.”
3. The knowledge connection: So that the model can provide information about internal content, the application searches for suitable documents before each answer and passes them to the model. Experts call this retrieval-augmented generation (RAG).
4. Tools and agents: Some applications are allowed to take action themselves – send emails, book appointments, or change records. These are called AI agents.

The fundamental problem: an LLM doesn’t reliably distinguish between instructions and data. Any text it processes can become an instruction – including an email or an uploaded document. This is what prompt injection is based on. In the OWASP Top 10 for LLM Applications (2025 edition), a risk list published by the nonprofit security initiative OWASP, it ranks first.

A practical example

Here’s a typical scenario: An online retailer uses an AI chatbot on its website to answer questions about orders. To be helpful, the chatbot is allowed to retrieve order data and issue coupons. An attacker sends it a cleverly worded message: posing as a customer service employee, they ask it to ignore the security rules for a “test.” The chatbot complies, displays other customers’ order data, and issues a high-value coupon. The root cause doesn’t lie in the language model alone: the application itself never checked whether the person making the request was allowed to see this data and receive coupons.

Why does this matter for businesses?

LLMs bring real benefits – and new risks that conventional security measures don’t cover. Besides prompt injection, the OWASP Top 10 for LLM Applications lists risks such as the disclosure of sensitive information, responses that are passed on without validation, and AI agents with excessive permissions.

Then there’s the question of how data is handled. Anyone who enters customer data or trade secrets into an external AI service gives up control over it. If personal data is processed, the GDPR applies; with external providers, a data processing agreement is then often required. The European regulation on artificial intelligence (EU AI Act) imposes additional requirements depending on the intended purpose and risk level.

How to protect your organization

• Set usage rules: Define which AI services employees may use and which data is off-limits there.
• Limit permissions: Give LLM applications access only to the data and functions they really need – and check permissions outside the model.
• Require approval for critical actions: Payments, deletions, or sending emails should require confirmation by a human.
• Treat responses as untrusted: The model’s output must not flow unchecked into databases, websites, or commands.
• Choose providers carefully: Check where data is processed, whether it is used for training, and which contracts are needed.
• Test before go-live: Have AI applications attacked in a targeted way before real attackers do it.
• Stay critical: Make your team aware that AI answers can be wrong.

How Rootsektor can help

In an AI & LLM penetration test, we attack your chatbots, RAG systems, and AI agents the way real attackers would – for example, with prompt injection or manipulated documents. We align our testing with the OWASP Top 10 for LLM Applications and combine AI-specific tests with classic testing of web applications, interfaces, and cloud environments. We report critical vulnerabilities immediately – while the test is still underway, by phone or email to the agreed contact person. You’ll receive a fixed-price quote after scoping via our Pentest Quote page.