# Semantic／Prompt Cache

CURRENT_STATUS: UNKNOWN
OBSERVATION: SPEC_ONLY
FRESHNESS: UNKNOWN
PAGE_TYPE: reference
CANONICAL_URL: /tools/ea-083-semantic-prompt-cache
CLAIM_IDS: none
EVIDENCE_IDS: EVD-EA-083-URL-C86BEEF864F4, EVD-EA-083-URL-9A3BBC4F637B, EVD-EA-083-URL-F05F01EE79E9, EVD-EA-083-URL-BE38DC244073, EVD-EA-083-URL-D094AD1847C5, EVD-EA-083-URL-483A81E492FB, EVD-EA-083-URL-2E0FC8579315, EA-083-R11-RESEARCH, EA-083-R11-WEB-READBACK
UNKNOWN_IDS: EA-083-UNKNOWN-01, EA-083-UNKNOWN-CURRENTNESS
OPS_IDS: none

## Summary
“Prompt/context caching can reduce repeated-prefix processing cost and latency without reusing an old answer, while semantic response caching may skip inference by reusing an answer for a similar request. The latter therefore requires explicit equivalence, freshness, authorization, invalidation, and audit controls.”

## Current judgment
Provider prompt/context caching is primarily reuse of model-side computation for matching prompt prefixes/context; it does not by itself reuse an old generated answer. Application semantic caching may return a previously generated answer for a merely similar request and therefore creates an additional correctness and authorization boundary. Cache keys must include all answer-affecting dimensions that are not safely encoded in the matched content: tenant/workspace/user security scope as needed, model/revision, system/developer prompt version, tool schema/version, retrieval corpus snapshot or freshness epoch, policy/safety version, locale, output schema, and material generation parameters. Semantic cache should default to bypass for high-stakes, authorization-sensitive, freshness-sensitive, personalized, or action-producing requests unless a bounded equivalence contract is proven.

## Scope
Enterprise AI Core / provider mapping / cost / architecture / security / privacy / audit / implementation

## Confirmed
- Provider prompt/context caching is primarily reuse of model-side computation for matching prompt prefixes/context; it does not by itself reuse an old generated answer. Application semantic caching may return a previously generated answer for a merely similar request and therefore creates an additional correctness and authorization boundary. Cache keys must include all answer-affecting dimensions that are not safely encoded in the matched content: tenant/workspace/user security scope as needed, model/revision, system/developer prompt version, tool schema/version, retrieval corpus snapshot or freshness epoch, policy/safety version, locale, output schema, and material generation parameters. Semantic cache should default to bypass for high-stakes, authorization-sensitive, freshness-sensitive, personalized, or action-producing requests unless a bounded equivalence contract is proven.

## UNKNOWN
- EA-083-UNKNOWN-01: exact current GPTCache release/tag/commit and maintenance status; provider-specific data-retention interaction for every enterprise plan/region; measured false-hit envelope on enterprise corpora. STOP_CONDITION: packet sufficient for architecture inclusion; product-volatility matrix must be refreshed at update time.

---
- EA-083-UNKNOWN-CURRENTNESS: 外部根拠の再確認日がcanonical body/control-planeで確定していないため、currentnessはUNKNOWNとして保持する。日付を推測補完しない。

## Evidence
- EVD-EA-083-URL-C86BEEF864F4 (RESEARCH): 1. source class: official product documentation; exact title: Prompt caching; author/org: OpenAI; canonical URL: — https://developers.openai.com/api/docs/guides/prompt-caching
- EVD-EA-083-URL-9A3BBC4F637B (RESEARCH): 2. source class: official product documentation; exact title: Prompt caching; author/org: Anthropic; canonical URL: — https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- EVD-EA-083-URL-F05F01EE79E9 (RESEARCH): 3. source class: official product documentation; exact title: Context caching; author/org: Google; canonical URL: — https://ai.google.dev/gemini-api/docs/caching
- EVD-EA-083-URL-BE38DC244073 (RESEARCH): 4. source class: author-maintained OSS; exact title: GPTCache: A Library for Creating Semantic Cache for LLM Queries; author/org: ZillizTech; canonical URL: — https://github.com/zilliztech/gptcache
- EVD-EA-083-URL-D094AD1847C5 (RESEARCH): 5. source class: author issue/failure report; exact title: Semantic Cache is not working #651; author/org: GPTCache issue reporter; canonical URL: — https://github.com/zilliztech/GPTCache/issues/651
- EVD-EA-083-URL-483A81E492FB (RESEARCH): 6. source class: author issue/security report; exact title: Possible Leakage of Private Info in Semantic Cache Requests #652; author/org: GPTCache issue reporter/security research  — https://github.com/zilliztech/GPTCache/issues/652
- EVD-EA-083-URL-2E0FC8579315 (RESEARCH): 7. source class: primary research; exact title: SCALM: Towards Semantic Caching for Automated Chat Services with Large Language Models; author/org: Jiaxing Li et al.; canonical URL — https://arxiv.org/abs/2406.00025
- EA-083-R11-RESEARCH (RESEARCH): EA-083 監査済みresearch / canonical revision binding
- EA-083-R11-WEB-READBACK (IMPL): EA-083 Web body identity / canonical readback

## Failure / limits
False semantic hit; stale answer after source/policy/model change; cross-tenant or cross-user leakage; cache-hit timing/probing; cached answer containing revoked data; low hit rate from variable prefixes; cache-write cost exceeding savings; cache stampede; inability to manually purge provider-managed ephemeral cache; semantic similarity that ignores negation, numbers, time, identity, or authorization context. Provider cache isolation claims do not prove application-layer semantic cache isolation.
## Practice
Provider/model/API endpoint; minimum cacheable tokens; implicit vs explicit mode; TTL/retention; region/data-residency path; batch interaction; account/organization boundary; application cache store/vector index; eviction/invalidation support.
## Practice
Provider prompt/context caching is production product capability in OpenAI, Anthropic, and Gemini APIs as of 2026-09-16, with materially different semantics. Semantic response caching is implementable via OSS/application infrastructure but requires application-owned equivalence, ACL, invalidation, and audit logic.
## Practice
fixed workload with exact-repeat, prefix-repeat, paraphrase, negation flip, numeric/date change, user/tenant change, ACL change, corpus revision, policy revision, model revision, and no-cache controls. Measure token cache-hit rate, semantic-cache precision/recall, false-hit severity, stale-hit rate, unauthorized-hit rate, p50/p95 latency, provider input/write/read cost, cache-store cost, and purge/invalidation propagation time. Never optimize hit ratio without false-hit and stale-hit constraints.
## Practice
Audit cache layer, cache key namespace, matched source entry, creation/expiry time, model/prompt/corpus/policy versions, tenant/ACL scope, hit/miss/bypass reason, and invalidation event. Do not place raw secrets/PII in application cache keys; use scoped opaque identifiers/hashes where appropriate. Cost model must include cache writes, misses, storage/vector search, invalidation, and false-hit remediation, not only read-token discount.
## Practice
Build one Core cache contract, but expose two modes: PROVIDER_PREFIX_CACHE and APPLICATION_RESPONSE_CACHE. Require cache_namespace + model_ref + prompt/tool/policy/corpus version + security_scope + TTL/freshness + hit provenance. Re-pin provider model/TTL/pricing/retention behavior at publication time.
## Practice
GPT-5.6+ prompt-cache semantics, pricing, TTL and breakpoint behavior are model/version-pinned and not frozen across releases; cache hit does not imply deterministic output.
## WHY_THIS_MATTERS_TO_UPDATE
Repeated long system prompts, tool definitions, reference context, and semantically equivalent user requests can dominate enterprise inference cost and latency. However, provider prompt/context caches and application semantic caches are different mechanisms with different correctness, privacy, invalidation, and observability properties. Treating them as one feature can create stale or unauthorized answer reuse.

## SCOPE_ROUTING
Enterprise AI Core / provider mapping / cost / architecture / security / privacy / audit / implementation

## CLAIM_OR_RESEARCH_QUESTION
Which cache layer can safely reduce repeated inference, under what cache-key/TTL/version/ACL conditions, and how should false semantic hits, stale results, tenant isolation, and cache invalidation be governed?

## EVIDENCE TABLE
1. source class: official product documentation; exact title: Prompt caching; author/org: OpenAI; canonical URL: https://developers.openai.com/api/docs/guides/prompt-caching ; access date: 2026-09-16; revision/model: current docs, GPT-5.6+ behavior documented; locator: How caching works / Cache location / Summary of model differences; evidence label: VERIFIED_SPEC / PRODUCT_VOLATILE; relation: support/context. Key evidence: GPT-5.6+ minimum cacheable visible prefix 1,024 tokens; cache writes 1.25x uncached input rate and reads 0.1x; cache reuse is prefix/breakpoint based; caches are not shared across organizations or regional processing boundaries; prompt_cache_key can separate accounting and reduce cache-hit probing across users.
2. source class: official product documentation; exact title: Prompt caching; author/org: Anthropic; canonical URL: https://platform.claude.com/docs/en/build-with-claude/prompt-caching ; access date: 2026-09-16; revision: current docs; locator: TTL support / pricing / cache storage and sharing; evidence label: VERIFIED_SPEC / PRODUCT_VOLATILE; relation: support/limit/context. Key evidence: default 5-minute and optional 1-hour TTL; 5-minute write 1.25x, 1-hour write 2x, cache read generally 0.1x base input; cache entries isolated between organizations and, on listed platforms, workspaces; no manual clear, expiry-based lifecycle.
3. source class: official product documentation; exact title: Context caching; author/org: Google; canonical URL: https://ai.google.dev/gemini-api/docs/caching ; updated: 2026-09-02 UTC; access date: 2026-09-16; model/API: Gemini 2.5+ / current Gemini API; locator: Implicit caching; evidence label: VERIFIED_SPEC / PRODUCT_VOLATILE; relation: support/availability. Key evidence: implicit caching enabled by default for Gemini 2.5+; minimum token thresholds are model-specific; explicit cache objects require generateContent rather than Interactions API.
4. source class: author-maintained OSS; exact title: GPTCache: A Library for Creating Semantic Cache for LLM Queries; author/org: ZillizTech; canonical URL: https://github.com/zilliztech/gptcache ; access date: 2026-09-16; version: exact current release/tag not pinned in this run; locator: README / How does it work?; evidence label: OSS_PRACTICE; relation: implementation/context. Key evidence: semantic cache uses embeddings/vector similarity and explicitly acknowledges false-positive hits and false-negative misses.
5. source class: author issue/failure report; exact title: Semantic Cache is not working #651; author/org: GPTCache issue reporter; canonical URL: https://github.com/zilliztech/GPTCache/issues/651 ; publication: 2024-09-16; access date: 2026-09-16; evidence label: SOURCE_REPORT; relation: counter/limit. Reported failure: same cached response returned for different questions in one semantic-cache setup. This is not generalized to all versions/deployments.
6. source class: author issue/security report; exact title: Possible Leakage of Private Info in Semantic Cache Requests #652; author/org: GPTCache issue reporter/security research group; canonical URL: https://github.com/zilliztech/GPTCache/issues/652 ; publication: 2024-09-18; access date: 2026-09-16; evidence label: SOURCE_REPORT; relation: counter/security. Report alleges timing differences may expose private attributes; unverified beyond issue scope and not generalized.
7. source class: primary research; exact title: SCALM: Towards Semantic Caching for Automated Chat Services with Large Language Models; author/org: Jiaxing Li et al.; canonical URL: https://arxiv.org/abs/2406.00025 ; publication: 2024-05-24; access date: 2026-09-16; evidence label: RESEARCH; relation: context/benchmark. Research reports semantic-aware caching can improve hit ratio/token savings over compared GPTCache configurations; benchmark does not establish enterprise correctness/privacy.

## FINDINGS
Provider prompt/context caching is primarily reuse of model-side computation for matching prompt prefixes/context; it does not by itself reuse an old generated answer. Application semantic caching may return a previously generated answer for a merely similar request and therefore creates an additional correctness and authorization boundary. Cache keys must include all answer-affecting dimensions that are not safely encoded in the matched content: tenant/workspace/user security scope as needed, model/revision, system/developer prompt version, tool schema/version, retrieval corpus snapshot or freshness epoch, policy/safety version, locale, output schema, and material generation parameters. Semantic cache should default to bypass for high-stakes, authorization-sensitive, freshness-sensitive, personalized, or action-producing requests unless a bounded equivalence contract is proven.

## LIMITS / COUNTEREVIDENCE / FAILURE ENVELOPE
False semantic hit; stale answer after source/policy/model change; cross-tenant or cross-user leakage; cache-hit timing/probing; cached answer containing revoked data; low hit rate from variable prefixes; cache-write cost exceeding savings; cache stampede; inability to manually purge provider-managed ephemeral cache; semantic similarity that ignores negation, numbers, time, identity, or authorization context. Provider cache isolation claims do not prove application-layer semantic cache isolation.

## AVAILABILITY_GATES
Provider/model/API endpoint; minimum cacheable tokens; implicit vs explicit mode; TTL/retention; region/data-residency path; batch interaction; account/organization boundary; application cache store/vector index; eviction/invalidation support.

## IMPLEMENTATION_STATE
Provider prompt/context caching is production product capability in OpenAI, Anthropic, and Gemini APIs as of 2026-09-16, with materially different semantics. Semantic response caching is implementable via OSS/application infrastructure but requires application-owned equivalence, ACL, invalidation, and audit logic.

## TEST_PROFILE
fixed workload with exact-repeat, prefix-repeat, paraphrase, negation flip, numeric/date change, user/tenant change, ACL change, corpus revision, policy revision, model revision, and no-cache controls. Measure token cache-hit rate, semantic-cache precision/recall, false-hit severity, stale-hit rate, unauthorized-hit rate, p50/p95 latency, provider input/write/read cost, cache-store cost, and purge/invalidation propagation time. Never optimize hit ratio without false-hit and stale-hit constraints.

## SECURITY / PRIVACY / AUDIT / COST / OPERABILITY
Audit cache layer, cache key namespace, matched source entry, creation/expiry time, model/prompt/corpus/policy versions, tenant/ACL scope, hit/miss/bypass reason, and invalidation event. Do not place raw secrets/PII in application cache keys; use scoped opaque identifiers/hashes where appropriate. Cost model must include cache writes, misses, storage/vector search, invalidation, and false-hit remediation, not only read-token discount.

## SAFE_WORDING_FOR_FUTURE_BOOK
“Prompt/context caching can reduce repeated-prefix processing cost and latency without reusing an old answer, while semantic response caching may skip inference by reusing an answer for a similar request. The latter therefore requires explicit equivalence, freshness, authorization, invalidation, and audit controls.”

## UPDATE_WORK_HANDOFF
Build one Core cache contract, but expose two modes: PROVIDER_PREFIX_CACHE and APPLICATION_RESPONSE_CACHE. Require cache_namespace + model_ref + prompt/tool/policy/corpus version + security_scope + TTL/freshness + hit provenance. Re-pin provider model/TTL/pricing/retention behavior at publication time.

## OPEN_UNKNOWN
exact current GPTCache release/tag/commit and maintenance status; provider-specific data-retention interaction for every enterprise plan/region; measured false-hit envelope on enterprise corpora. STOP_CONDITION: packet sufficient for architecture inclusion; product-volatility matrix must be refreshed at update time.

---

## AUDITED_CURRENTNESS_BOUNDARY
GPT-5.6+ prompt-cache semantics, pricing, TTL and breakpoint behavior are model/version-pinned and not frozen across releases; cache hit does not imply deterministic output.

NEXT: /agent/start.json
STOP: This is a knowledge projection, not an execution order.