Your Anthropic Prompt Library Audit Report

Opportunity #1 — Static text block (3842 chars) missing cache_control $626/mo

{
            "content": [
                {
                    "type": "text",
                    "text": "[Current URL: https://github.com/ryanhoangt]\n[Focused element bid: 119]\n\n[Action executed successfully.]\n============== BEGIN accessibility tree ==============\nRootWebArea 'ryanhoangt (Ryan H. Tran) · GitHub', focused\n\t[119] generic\n\t\t[120] generic\n\t\t\t[121] generic\n\t\t\t\t[122] link 'Skip to content', clickable\n\t\t\t\t[123] generic\n\t\t\t\t\t[124] generic\n\t\t\t\t[135] generic\n\t\t\t\t\t[137] generic, clickable\n\t\t\t\t[142] banner ''\n\t\t\t\t\t[143] heading 'Nav

{
    "type": "text",
    "text": LARGE_STATIC_TEXT,
    "cache_control": {"type": "ephemeral"}
}

Opportunity #2 — Static text block (1804 chars) missing cache_control $321/mo

"content": [
                {"type": "text", "text": "go to github ryanhoangt by browser"},
                {
                    "type": "text",
                    "text": '<extra_info>\nThe following information has been included based on a keyword match for "github". It may or may not be relevant to the user\'s request.\n\nYou have access to an environment variable, `GITHUB_TOKEN`, which allows you to interact with\nthe GitHub API.\n\nYou can use `curl` with the `GITHUB_TOKEN` to interact with GitHub\'s API.\nALWAYS use the GitHub API for operations instead of a web browser.\n\nHere are s

{
    "type": "text",
    "text": LARGE_STATIC_TEXT,
    "cache_control": {"type": "ephemeral"}
}

Opportunity #3 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

) == kwargs.get("system"):
                        return messages
                    messages = [
                        {"role": "system", "content": kwargs.get("system")}
                    ] + messages
                elif isinstance(messages, str):
                    messages = [

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #4 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

fallback_messages = messages or []
    if system and fallback_messages:
        fallback_messages = [{"role": "system", "content": system}] + fallback_messages
    local_count = litellm.token_counter(
        model=model,
        messages=fallback_messages,
        tools=tools,  # type: ignore[arg-type]

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #5 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

res = litellm.completion(
                **base_completion_call_args,
                messages=[
                    {
                        "role": "system",
                        "content": "response user question with JSON object",
                    },

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #6 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

def test_gemini_context_caching_separate_messages():
    messages = [
        # System Message
        {
            "role": "system",
            "content": [

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #7 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

response = await litellm.acompletion(
        model="anthropic/claude-sonnet-4-5-20250929",
        messages=[
            # System Message
            {
                "role": "system",
                "content": [

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #8 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

litellm.set_verbose = True

    messages = [
        {
            "role": "system",
            "content": [{"type": "text", "text": "You are 2twNLGfqk4GMOn3ffp4p."}],
        },

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #9 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

response = await litellm.acompletion(
                model="bedrock/anthropic.claude-3-5-haiku-20241022-v1:0",
                messages=[
                    {
                        "role": "system",
                        "content": [
                            {

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

Opportunity #10 — `{role: 'system'}` entry inside messages[] (should use top-level `system` param) $40/mo

# Test empty string content - should not produce any anthropic system message content
    messages = [
        {"role": "system", "content": ""},
        {"role": "user", "content": "Hello"},
    ]

# Anthropic-native shape (cacheable prefix):
client.messages.create(
    system=SYSTEM_PROMPT,            # top-level
    messages=[{"role": "user",  # NO system entries here
               "content": user_msg}],
    ...
)

How Anthropic prompt caching works

Anthropic's prompt cache lets you mark static portions of your prompt (system instructions, retrieval context, few-shot examples) with cache_control: {"type": "ephemeral"}. The first call writes the cache (1.25x base input rate); subsequent calls within ~5 minutes read from the cache at 0.1x base rate (a 90% discount). For Sonnet, that's $0.30/M cached tokens vs $3/M base.

The math gets dramatic at moderate scale: a 4K-token system prompt called 100K times/month costs $1,200 uncached vs $150 cached ($120 reads + ~$30 amortized writes). That's $1,050/month saved on a single block — and most production workloads have 3-5 such blocks.

Cache scope: the cache key is the entire prefix up to (and including) the last cache_control marker. So order matters: put the most-static content first, then less-static, then the per-call variable content last. Anthropic supports up to 4 cache breakpoints per request.

30-day re-audit voucher

Why this matters: Anthropic cache savings only materialize once the code change ships. The re-audit voucher creates an accountability loop — we can't claim "issue resolved" unless the v1 ruleset agrees on re-scan. Same deterministic engine, same file paths, same line numbers. No moving goalposts.