Close Menu
MyAppsPlus

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    How to Watch Wayne Rooney in the Jungle — Stream I’m a Celeb for free

    November 16, 2026

    Jordan Belfort’s new AI voice startup, Tone, wants to sell like the ‘Wolf of Wall Street’

    November 16, 2026

    Early Black Samsung now live – Galaxy Fold 8 $350 off, S26 Ultra $450 off, Tabs from $200, more

    November 16, 2026
    Facebook X (Twitter) Instagram
    Facebook X (Twitter) Instagram
    MyAppsPlusMyAppsPlus
    Monday, November 16
    • Home
    • Breaking Tech
    • Apps & Software
    • AI & Automation
    • Android
    • iPhone & iOS
    • More
      • Reviews
      • How-To Guides
      • Deals & Discounts
      • Shop
    MyAppsPlus
    Home»AI & Automation»Use prompt caching with open-weight models on Amazon Bedrock
    AI & Automation

    Use prompt caching with open-weight models on Amazon Bedrock

    myappsplusBy myappsplusNovember 16, 20260015 Mins Read
    Share Facebook Twitter Pinterest Copy Link LinkedIn Tumblr Email Telegram WhatsApp
    Follow Us
    Google News Flipboard
    Use prompt caching with open-weight models on Amazon Bedrock
    Share
    Facebook Twitter LinkedIn Pinterest Email Copy Link

    Many generative AI applications send the same long context with every request, and prompt caching avoids processing it again each time. That context can be a system prompt, reference documents, tool definitions, or the conversation so far. Open-weight models on Amazon Bedrock can reuse it across requests, which helps reduce input costs and response latency.

    Prompt caching reuses the computed state of a matching prompt prefix, so repeated context is read from cache instead of being processed again. Amazon Bedrock manages the cache for you. Depending on the model, you can use implicit caching, which works automatically, explicit caching, where you mark the reusable prefix, or both. The more context you reuse, and the more often you reuse it, the more you can save.

    As with other models on Amazon Bedrock, your prompts and outputs aren’t shared with the model provider or used to train models. For more information, see Data protection in the Amazon Bedrock User Guide.

    Open-weight models on Amazon Bedrock now support explicit prompt caching. In this post, we use both modes with an open-weight model through the Chat Completions API and show how to read cache usage from each response. We then apply implicit caching to an OpenCode coding agent session on the same API and close with best practices.

    Prompt caching for open-weight models on Amazon Bedrock

    When a request starts with the same content as an earlier one, Amazon Bedrock reads that prefix from cache and processes only the rest. Cache reads are billed at a lower rate than standard input, so the savings can grow each time a prefix is reused. Depending on the model, cache writes can be billed at a higher rate than standard input. See Amazon Bedrock pricing for the rates of each model.

    Supported modes depend on the model and the API. For each model’s caching modes, minimum prefix length, and how long entries stay cached, see Models at a glance. The two modes compare as follows:

    Feature Implicit caching Explicit caching
    Request changes None On the Chat Completions API, prompt_cache_options with mode set to explicit, plus one or more prompt_cache_breakpoint markers
    What is cached Eligible prefixes, selected automatically on a best-effort basis Content up to and including each breakpoint
    Good fit Prompts that already start with a stable prefix, such as conversations and agent loops A fixed shared prefix with a changing question, or layered prefixes that you want to control

    Implicit caching: Amazon Bedrock reuses eligible prefixes automatically, on a best-effort basis. In a conversation, each request resends the full context, so the previous request becomes the cached prefix of the next one (Figure 1). As the prompt caching guide recommends, put static content first and dynamic content last. Keeping the start of the prompt stable lets every later request reuse it. If an early part changes, as in turn 4 of Figure 1, the cache is rebuilt from that point.

    Figure 1: Implicit prompt caching across the turns of a conversation

    Explicit caching: You mark the end of each reusable prefix with a cache breakpoint. You can add several breakpoints, up to the maximum that the model supports, to cache layers that change at different rates, as Figure 2 shows. Content up to each breakpoint is written to cache once and then read on later requests, and content after the last breakpoint is processed on each request. Check the model card for the minimum token count before a breakpoint, the maximum number of breakpoints, and the time to live (TTL), which resets with each cache hit.

    Figure 2: Explicit prompt caching with a breakpoint after the shared prefix

    With implicit caching, Amazon Bedrock can also write content to cache that later requests never reuse, such as a question that’s different each time. When the content after a shared prefix changes on every request, use explicit caching to limit cache writes to the shared prefix. Content after the last breakpoint is billed at the standard input rate. On models that charge more for cache writes, you don’t pay the higher write rate for content that’s never read back.

    Solution overview

    The following diagram shows how both examples in this post reach Amazon Bedrock through the same Chat Completions API endpoint.

    Figure 3: How the two examples in this post reach Amazon Bedrock

    1. The Python application and the OpenCode coding agent send requests to the Amazon Bedrock Runtime endpoint in your Region, authenticated with a short-term Amazon Bedrock API key.
    2. Amazon Bedrock checks the prompt cache for a matching prefix. It reads a matching prefix from cache, and writes new cacheable content to cache.
    3. Amazon Bedrock sends the request to the selected open-weight model and returns the response, including the cache usage fields that the following sections show you how to read.

    Prerequisites

    To run the samples, you need the following:

    • An AWS account with access to your selected model on Amazon Bedrock. Model availability and prompt caching support vary by AWS Region, so check the model card for the Regions that your model supports.
    • AWS credentials for an AWS Identity and Access Management (IAM) identity that has permission to invoke your selected model on Amazon Bedrock.
    • Python 3.12 with the openai (3.26.0 or later) and aws-bedrock-token-generator (1.1.0 or later) packages.
    • For the coding agent example, npm to install OpenCode 1.18.35.

    Before you use a model, review its license and service terms, linked from its model card. Your use of the model is also subject to the AWS Responsible AI Policy.

    Cache a product manual through the Chat Completions API

    The example is a support assistant for a fictional water filter, the AnyCompany AquaPure 500. The system prompt holds a one-line instruction and the owner’s manual, which is the reusable prefix. Each request asks a different customer question.

    Set up the client

    The samples use the OpenAI Python SDK with the Amazon Bedrock Chat Completions endpoint. For authentication, the provide_token() function from the aws-bedrock-token-generator package turns your AWS credentials into a short-term Amazon Bedrock API key, valid for up to 12 hours. You don’t need to create or store a long-term key.

    1. Create a virtual environment and install the packages:

    python3.12 -m venv .venv
    source .venv/bin/activate
    pip install "openai>=3.26.0" "aws-bedrock-token-generator>=1.1.0"

    2. Create the manual. Ours is about 4,000 tokens of Markdown. The following excerpt shows its format:

    # AnyCompany AquaPure 500 Owner's Manual
    
    ### 7.1 Replacement schedule
    | Stage | Cartridge | Part number | Replace every |
    | ----- | ------------------- | ----------- | ------------- |
    | 1 | Sediment pre-filter | AP-F1 | 6 months |
    | 4 | Carbon post-filter | AP-F4 | 12 months |
    
    ### 9. Error codes
    | Code | Description | What to do |
    | ---- | ----------------------- | -------------------------------------------- |
    | E01 | Low feed water pressure | Check that the feed water valve is open. |
    | E07 | Drain line restriction | Check the black concentrate tube for kinks. |
    
    ### 11. Warranty
    - Faucet: 3 years from the date of purchase. The 3 V cell is not covered.

    Save it as manual.md and add sections until it’s longer than the minimum prefix length on your model card. Placeholder text works, because the cache only needs an identical prefix. Keep the three sections shown, which answer the sample questions.

    3. Save the following setup code as common.py in the same directory. The samples that you create next import from it.

    from pathlib import Path
    
    from aws_bedrock_token_generator import provide_token
    from openai import OpenAI
    
    REGION = "<AWS_REGION>"
    MODEL_ID = "<MODEL_ID_OR_INFERENCE_PROFILE_ID>"
    
    MANUAL = Path(__file__).with_name("manual.md").read_text()
    
    SYSTEM_PROMPT = (
        "You are the AnyCompany AquaPure support assistant. Answer the customer's question in at "
        "most three sentences, using only the owner's manual below.nn" + MANUAL
    )
    
    QUESTIONS = [
        "My filter shows error code E07. What does it mean and how do I fix it?",
        "How often do I need to replace the AP-F4 post-filter?",
        "Is the 3 V battery in the smart faucet covered by the warranty?",
    ]
    
    # Chat Completions API client: the OpenAI SDK pointed at the Amazon Bedrock OpenAI-compatible endpoint
    client = OpenAI(
        api_key=provide_token(region=REGION),
        base_url=f"https://bedrock-runtime.{REGION}.amazonaws.com/openai/v1",
    )
    
    def report(call_number, response):
        """Print the cache fields from the response usage."""
        usage = response.usage
        read = usage.prompt_tokens_details.cached_tokens or 0
        written = usage.prompt_tokens_details.cache_write_tokens or 0
        print(
            f"Call {call_number}: prompt_tokens={usage.prompt_tokens} cached_tokens={read} "
            f"cache_write_tokens={written} uncached={usage.prompt_tokens - read - written}"
        )

    If your model returns reasoning tokens, they count toward max_tokens. The samples set max_tokens=1024 so that short answers have room to finish.

    Implicit caching

    Implicit caching takes no cache controls. Put the manual in the system message and the question in the user message. Sending both as parts of one user message got no implicit cache hits. Save the following as implicit.py and run it with python implicit.py:

    from common import MODEL_ID, QUESTIONS, SYSTEM_PROMPT, client, report
    
    for i, question in enumerate(QUESTIONS, start=1):
        response = client.chat.completions.create(
            model=MODEL_ID,
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT}, # static, same every call
                {"role": "user", "content": question}, # changes every call
            ],
            max_tokens=1024,
        )
        report(i, response)

    Verify implicit caching

    Each response reports cache activity in usage.prompt_tokens_details. Figure 4 shows how to read these fields and the pattern to expect across the three calls of each sample.

    Figure 4: How to read the cache usage fields (illustrative)

    In implicit mode, call 1 writes the prompt to cache. Calls 2 and 3 read the system message, with the instruction and the manual, back from cache. They also write a few tokens each, because implicit mode extends the cached prefix to cover each new question. A few tokens after the last message stay uncached on every call. This is the pattern for the first run. If you run the sample again within the TTL, every call reads from cache, and none writes.

    Explicit caching

    Choose explicit caching to control exactly what is cached. As Figure 4 shows, calls 2 and 3 of the implicit run each write a few tokens to cache, whereas the same calls with explicit caching write nothing. With several breakpoints, you can cache layered prefixes, such as shared instructions and tools followed by a per-customer document (Figure 2).

    For explicit caching, add the following to the request:

    • prompt_cache_options with mode set to explicit selects explicit mode. It’s an Amazon Bedrock request field, and the OpenAI SDK sends it through extra_body.
    • prompt_cache_breakpoint on a text content part marks the end of the reusable prefix. Content up to and including that part is cached, and content after it’s processed again on every request.

    Save the explicit version of the same loop as explicit.py:

    from common import MODEL_ID, QUESTIONS, SYSTEM_PROMPT, client, report
    
    for i, question in enumerate(QUESTIONS, start=1):
        response = client.chat.completions.create(
            model=MODEL_ID,
            messages=[
                {
                    "role": "system",
                    "content": [
                        {
                            "type": "text",
                            "text": SYSTEM_PROMPT,
                            # Cache everything up to and including this content part.
                            "prompt_cache_breakpoint": {"mode": "explicit"},
                        }
                    ],
                },
                {"role": "user", "content": question}, # after the breakpoint, not cached
            ],
            max_tokens=1024,
            extra_body={"prompt_cache_options": {"mode": "explicit"}},
        )
        report(i, response)

    Set both parts together. Explicit mode without a breakpoint caches nothing, and a breakpoint without explicit mode behaves like implicit caching. For the optional ttl field of prompt_cache_options, check the model card for supported values.

    Verify explicit caching

    If you run explicit.py right after implicit.py, its first call is already a cache hit. For a cold first call, add a unique first line to the system prompt, for example SYSTEM_PROMPT = f"Run ID: {uuid.uuid4()}n" + SYSTEM_PROMPT (with import uuid at the top of explicit.py).

    As the explicit bars in Figure 4 show, only the first call writes. Each repeat reads the whole prefix up to the breakpoint and writes nothing, and the question after the breakpoint makes up the uncached remainder.

    Use prompt caching with the OpenCode coding agent

    Coding agents resend a large, stable prefix on every step: the system prompt, tool definitions, and the conversation so far. That makes them a natural fit for implicit caching. OpenCode is an openugh a custom OpenAI-compatible provider, so it can use the same Amazon Bedrock endpoint as the Python samples. Figure 5 shows the requests that OpenCode sends during one turn and which parts Amazon Bedrock reads from cache

    Figure 5: Requests that OpenCode sends to Amazon Bedrock during one turn

    In one turn, OpenCode sends request 1 with its system prompt, tool definitions, and your prompt, and Amazon Bedrock writes it to cache. The model responds with a tool call, which OpenCode runs on your machine. OpenCode then sends request 2, which contains request 1 plus the tool call and its result. Amazon Bedrock reads request 1 from cache, writes only the new tail, and returns the response.

    Configure OpenCode

    Install OpenCode and check the version:

    npm install --prefix ~/.local/opt/opencode opencode-ai@1.18.35
    export PATH="$HOME/.local/opt/opencode/node_modules/.bin:$PATH"
    opencode --version

    Next, add the following opencode.json to the root of your repository, or save it as ~/.config/opencode/opencode.json to apply it to every project. It defines a provider named bedrock that uses the OpenAI-compatible package to call the Amazon Bedrock Chat Completions endpoint:

    {
      "$schema": "https://opencode.ai/config.json",
      "model": "bedrock/<MODEL_ID_OR_INFERENCE_PROFILE_ID>",
      "small_model": "bedrock/<MODEL_ID_OR_INFERENCE_PROFILE_ID>",
      "provider": {
        "bedrock": {
          "npm": "@ai-sdk/openai-compatible",
          "name": "Amazon Bedrock (Chat Completions)",
          "options": {
            "baseURL": "https://bedrock-runtime.<AWS_REGION>.amazonaws.com/openai/v1",
            "apiKey": "{env:BEDROCK_API_KEY}"
          },
          "models": {
            "<MODEL_ID_OR_INFERENCE_PROFILE_ID>": { "name": "Open-weight model on Amazon Bedrock" }
          }
        }
      }
    }

    OpenCode uses small_model for session titles, so pointing it at the same model keeps every request on a model that you’ve enabled. The provider reads its key from BEDROCK_API_KEY. Set it to a short-term Amazon Bedrock API key from provide_token(), in the virtual environment from the first example. If the key expires, run the command again to get a new one:

    export BEDROCK_API_KEY="$(python -c '
    from aws_bedrock_token_generator import provide_token
    print(provide_token(region="<AWS_REGION>"))')"

    Confirm that OpenCode lists the model:

    opencode models bedrock

    Verify implicit caching in a coding session

    We ran a three-turn session against a small repository that holds one short Python file. Turn 1 asks for a bug review, turn 2 for a fix, and turn 3 for a short follow-up:

    opencode run --format json --title "cache-demo" 
    "Read src/inventory.py and list any bugs in Inventory.remove. Do not edit files."
    opencode run --format json -s <sessionID> 
    "Now propose a fixed version of Inventory.remove as a code block. Do not edit files."
    opencode run --format json -s <sessionID> 
    "In one sentence, what does low_stock return?"

    Replace <sessionID> with the sessionID value from the JSON output of turn 1. The opencode export <sessionID> command lists each request’s token usage: cache.read holds the cached_tokens value from Amazon Bedrock, and input holds the rest. OpenCode 1.18.35 doesn’t record cache_write_tokens for an OpenAI-compatible provider, so cache.write stays at 0.

    Figure 6 shows how much of each request in the session is read from cache.

    Figure 6: Cache reads across an OpenCode session (illustrative)

    The first request processes the whole prompt, most of it the OpenCode system prompt and tool definitions, and Amazon Bedrock writes it to cache. Each later request reads everything up to the end of the previous request from cache, so only the new tail of the conversation is processed. This works with no caching configuration.

    Best practices

    These practices apply to any model that supports prompt caching:

    • Put static content first, in both modes. Place the system prompt, tool definitions, and reference documents before content that changes, and keep them identical between requests. In explicit mode, place each breakpoint after the static content that it covers. Timestamps and request IDs belong at the end of the prompt.
    • Make the prefix long enough. Prefixes that meet the minimum token count on the model card are eligible for caching.
    • Confirm hits with the usage fields. Check the cache reads and writes in each response, in either mode. Across a workload, track the CacheReadInputTokenCount and CacheWriteInputTokenCountruntime metrics.
    • Keep the cache warm. Steady traffic keeps a prefix cached, because the TTL resets with each hit. After an idle gap longer than the TTL, the next request writes the prefix again.
    • Cache what you reuse. Savings come from cache reads, so cache content that later requests read again, especially where cache writes cost more than standard input. With cross-Region inference, expect some extra cache writes at times of high demand.

    Clean up

    The examples make inference calls only and create no resources in your AWS account. Cache entries expire after they go unread for the TTL. The short-term API key from provide_token() also expires on its own, and the samples never write it to disk. To clear it from your shell sooner, run unset BEDROCK_API_KEY.

    1. Delete the sample directory and its Python virtual environment.

    2. Delete opencode.json from your repository, or ~/.config/opencode/opencode.json if you saved it globally.

    3. If you no longer need OpenCode, delete ~/.local/opt/opencode.

    Conclusion

    Prompt caching lets open-weight models on Amazon Bedrock reuse repeated context with little or no change to your code. In this post, implicit caching covered the support assistant and the OpenCode session with no cache settings at all. Explicit caching gave precise control with one request option and a breakpoint for each reusable layer. In every case, the usage fields on each response show what came from cache, so you can measure the benefit on your own traffic.

    To get started, choose an open-weight model on Amazon Bedrock that supports prompt caching. Run the samples from this post with your own reference content, and compare the cache reads with your uncached input. The prompt caching guide covers the limits and options for each model, and the Amazon Bedrock runtime metrics let you track cache reads and writes across your applications. For questions, reach out through AWS Support or the AWS re:Post community.

    Amazon caching models Openweight Prompt
    Follow on Google News Follow on Flipboard
    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email Copy Link
    myappsplus
    • Website

    Related Posts

    Nokia and friends plan ‘Northern’ AI gigafactory

    November 16, 2026

    A-I now predicting heart attack recovery

    November 16, 2026

    After Nvidia scandal, Commerce Department moves to curb remote access to AI chips

    November 16, 2026
    Add A Comment
    Leave A Reply Cancel Reply

    Top Posts

    Google’s new AI shopping cart tracks deals across stores

    October 25, 202630 Views

    Save on your groceries, gas, and holiday shopping with this $15 BJ’s membership (Reg. $60) 

    November 4, 202629 Views

    Get Up to 38% Off Acer, Alienware, HP and More During Woot’s Laptop Sale

    October 23, 202627 Views
    Latest Reviews

    iPhone Ultra is coming, here’s what the final rumors say

    myappsplusSeptember 9, 2026

    LG Smart TVs Caught Recording Audio in Standby and Scanning Your Network

    myappsplusSeptember 9, 2026

    Vikings & Legends Global Announce U.S. Bank Stadium Enhancements for 2026

    myappsplusSeptember 9, 2026
    Stay In Touch
    • Facebook
    • YouTube
    • TikTok
    • WhatsApp
    • Twitter
    • Instagram

    Subscribe to Updates

    Get the latest tech news from FooBar about tech, design and biz.

    Most Popular

    Vikings & Legends Global Announce U.S. Bank Stadium Enhancements for 2026

    September 9, 20260 Views

    IKEA just launched a Matter-enabled driver that makes your integrated cabinet lights part of your smart home

    September 9, 20260 Views

    Latest Dell Technologies Research Reveals that AI Ambition Outpaces Readiness Across Singapore

    September 9, 20260 Views
    Our Picks

    How to Watch Wayne Rooney in the Jungle — Stream I’m a Celeb for free

    November 16, 2026

    Jordan Belfort’s new AI voice startup, Tone, wants to sell like the ‘Wolf of Wall Street’

    November 16, 2026

    Early Black Samsung now live – Galaxy Fold 8 $350 off, S26 Ultra $450 off, Tabs from $200, more

    November 16, 2026

    Subscribe to Updates

    Subscribe to our newsletter and get the latest tech news, app updates, AI trends, smartphone reviews, and exclusive deals delivered straight to your inbox.

    Facebook X (Twitter) Instagram Pinterest
    • About Us
    • Get In Touch
    • Disclaimer
    • Privacy Policy
    • Terms & Conditions
    © 2026 MyAppsPlus. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.