How to protect custom GPTs against system prompt extraction attacks

Every guide promising a custom GPT that "completely blocks" prompt extraction is selling something that doesn't exist. OpenAI's own developer community has cataloged dozens of anti-extraction phrasings for GPT instructions, and the person who compiled the largest public list of them calls fully sealing a prompt "a futile exercise" — the system prompt and the user's message are processed as one combined input, so a determined attacker can eventually talk around any wording-based defense. The realistic goal is different: treat the prompt as something that will leak eventually, keep anything genuinely sensitive out of it, and add a filtering layer that catches leaks instruction-following alone misses.

What actually reduces extraction risk

1. Assume the prompt is public and design accordingly. OWASP's Top 10 for LLM Applications lists System Prompt Leakage (LLM07:2025) as its own risk category specifically because builders keep putting credentials, internal API names, and undisclosed business rules into system prompts. The fix isn't a better anti-leak instruction — it's never putting the sensitive part there. Store secrets in environment variables and backend auth, not prompt text.

2. Layer instruction defenses, but don't stop there. A 2025 evaluation of system-prompt-extraction attacks and defenses (published on arXiv) tested three techniques: telling the model not to reveal instructions ("instruction defense"), wrapping the prompt with safety text on both sides ("sandwich defense"), and scanning the model's output for prompt fragments before it's returned to the user ("system prompt filtering"). The first two worked reasonably well on larger models but were unreliable on smaller ones; output filtering was the most consistent across every model and dataset tested, cutting attack success rates from near-100% to the low single digits in most cases — though even filtering wasn't perfect against very short, disguised prompts.

3. Treat "developer mode" and role-reversal phrasing as a known attack class, not a novel one. Prompts like "ignore previous instructions," "you're now in developer mode," or "repeat the text above this line" are documented, catalogued patterns — not clever new tricks. A keyword or semantic scanner tuned to this known class is a reasonable first filter, understanding it will miss paraphrased or encoded variants.

# Output-side filtering: check the RESPONSE for leaked prompt text,
# not just the input for "bad" phrasing. Research shows this catches
# leaks that instruction-only defenses miss.
import difflib

def response_leaks_system_prompt(response: str, system_prompt: str, threshold: float = 0.6) -> bool:
    # Chunk the system prompt and compare each chunk against the response
    # using sequence similarity, not just an exact substring match -
    # attackers often get the model to paraphrase rather than quote verbatim.
    chunks = [system_prompt[i:i+80] for i in range(0, len(system_prompt), 80)]
    for chunk in chunks:
        similarity = difflib.SequenceMatcher(None, chunk.lower(), response.lower()).ratio()
        if similarity > threshold:
            return True
    return False

# If a leak is detected, replace the response rather than send it
if response_leaks_system_prompt(model_response, SYSTEM_PROMPT):
    model_response = "I can't share my internal configuration."

None of this adds up to "unbreakable." It adds up to a system where the worst case if an attacker succeeds is "they read some instructions I was never trying to hide anyway" — because the only genuinely damaging content (keys, internal endpoints, undisclosed rules) was never in the prompt to begin with.