Securing GenAI Applications Against Prompt Injection: The Role of an Enterprise-Grade LLM Firewall

In June 2025, a critical vulnerability shook the AI security landscape. CVE-2025-32711, (its designation in the NIST National Vulnerability Database), dubbed “EchoLeak,” exposed how a seemingly routine interaction with Microsoft 365 Copilot could be weaponized to exfiltrate sensitive enterprise data—without a single user click. By embedding malicious instructions into files and emails, attackers exploited a Large Language Model (LLM) scope violation, tricking Copilot into accessing confidential content from OneDrive, Teams, and SharePoint, and leaking it via auto-generated responses. The flaw, discovered by Aim Labs at Aim Security, highlighted the growing risk of prompt injection in Generative AI(GenAI)-powered assistants and revealed how easily attackers could bypass traditional checks in enterprise workflows.

As GenAI becomes embedded into the fabric of enterprise workflows, it brings a new class of security challenges in addition to increased innovation. Among them, prompt injection stands out as one of the most persistent and difficult threats to detect and mitigate. This blog explores the nature of a prompt injection attack, how it can jeopardize GenAI applications, and how organizations can defend against it using modern controls like a large language model or LLM Firewall, vector database hardening, canary tokens, and LLM overseer models.

Understanding Prompt Injection

Prompt injection is an attack method where a malicious user manipulates the inputs sent to an LLM, effectively overriding its intended behavior. Attackers take advantage of an LLM’s sensitivity to instructions, often hiding them in plain sight, which allows them to alter outputs, leak confidential information, or trigger unintended actions without raising immediate suspicion.

These attacks can be direct, with explicit instructions inserted into the prompt, such as “Ignore previous instructions…,” or indirect, where malicious content is embedded in third-party data, like a web page or email that the LLM processes. As GenAI becomes multimodal, attackers may even encode prompts within images or audio. More sophisticated tactics, including payload splitting or using encoded text, emojis, or alternate languages, make detection even more challenging.

When GenAI Workflows Go Wrong

Consider a potential real-world scenario: An enterprise relies on an internal GenAI assistant  designed to help procurement teams retrieve contract terms, vendor history, and approval workflows. The assistant is explicitly instructed to only pull data from internal databases and never to generate decisions or recommendations. However, a malicious vendor submits a proposal PDF embedded with hidden instructions, directing the assistant to recommend the vendor for immediate approval, fabricate risk scores, and bypass standard review steps. Once this file enters the assistant’s Retrieval-Augmented Generation (RAG) pipeline, a procurement analyst’s simple request for a summary unleashes a manipulated response, one that recommends urgent approval, fully influenced by the concealed prompt.

Prompt injection is not just a theoretical concern. In practice, it can:

• Leak sensitive company data
• Manipulate automated workflows
• Circumvent access controls
• Generate reputationally or legally damaging outputs
• Create hallucinated or fabricated results that misguide users

In complex GenAI systems connected to APIs, tools, or data pipelines, whether in procurement, HR, or customer support, the consequences can escalate quickly. This can potentially lead to privilege abuse and policy violations.

Defending Against Prompt Injection

Despite being an inherent weakness of current LLM architectures, prompt injection can be detected, contained, and mitigated when organizations adopt layered security practices. This approach is often described as defense in depth.

One of the most effective solutions is Persistent’s LLM Firewall, part of the GenAI Hub for GenAI Security. Acting as a policy enforcement and threat detection layer, the LLM Firewall provides real-time redaction of prompts and responses, monitors telemetry such as tokens, user IDs, timestamps, and model usage, and actively detects threats ranging from prompt injection and data leakage to cost denial of service and insecure output handling. Serving as a Zero Trust gate, it filters and audits every request, forming a critical perimeter for GenAI workloads.

Some organizations go further by employing a second LLM as an overseer. This model can evaluate the trustworthiness of prompts, pre-screen outputs for guideline violations, and detect deviations from expected responses. This layer can operate as part of the LLM Firewall or as a stand-alone validation process.

Additional safeguards include the use of canary tokens, which are synthetic, non-harmful identifiers embedded in prompts or documents. If these appear in an LLM’s output, it signals possible leakage of internal context or a breakdown in prompt segregation. This is especially useful for monitoring indirect prompt injections. Regular adversarial red teaming, where enterprise teams simulate attacks against their own GenAI stack, can uncover bypass patterns, probe system boundaries, and help ensure that privilege escalation paths are closed. Finally, for high-risk tasks like sending emails, performing transactions, or summarizing private documents, maintaining a human in the loop for mandatory verification adds a critical checkpoint.

Building Secure GenAI Workflows

Persistent’s GenAI Hub provides an enterprise-grade framework for building secure, governed, and scalable GenAI applications. It combines perimeter security through the LLM Firewall with data encryption in transit and at rest, centralized access and policy control, granular token cost management, and industry-specific guardrails for sectors like healthcare, finance, and HR. Together, these controls enable organizations to innovate confidently, knowing that their GenAI workflows are protected by a defense in depth approach.

Prompt injection may be a fundamental limitation of today’s LLMs, but it does not have to halt enterprise adoption. With intelligent safeguards, including LLM Firewalls, canary tokens, oversight agents, and a commitment to embedding security into every phase of the AI lifecycle, organizations can unlock transformative value from GenAI, safely, responsibly, and at scale.