{
  "format": "agent-article-v0",
  "article_id": "PM-CLAUDE-CORE-07",
  "canonical_url": "/providers/pm-claude-core-07",
  "page_type": "reference",
  "claim_ids": [],
  "evidence_ids": [
    "EVD-PM-CLAUDE-CORE-07-URL-6727386D515F",
    "EVD-PM-CLAUDE-CORE-07-URL-C49DB9436032",
    "EVD-PM-CLAUDE-CORE-07-URL-DA45DE5646CB",
    "EVD-PM-CLAUDE-CORE-07-URL-D11C2860A404",
    "EVD-PM-CLAUDE-CORE-07-URL-DBCB362197DB",
    "EVD-PM-CLAUDE-CORE-07-URL-1BA226019836",
    "EVD-PM-CLAUDE-CORE-07-URL-1F49D0119CED",
    "EVD-PM-CLAUDE-CORE-07-URL-8E2E37051F9B",
    "EVD-PM-CLAUDE-CORE-07-URL-827ACF165DA2",
    "EVD-PM-CLAUDE-CORE-07-URL-16D2741870AD",
    "PM-CLAUDE-CORE-07-DOCUMENT-READBACK"
  ],
  "unknown_ids": [
    "PM-CLAUDE-CORE-07-CURRENTNESS"
  ],
  "operational_object_ids": [],
  "provider": [
    "CLAUDE"
  ],
  "models": [],
  "plans": [],
  "surfaces": [],
  "regions": [],
  "current_status": "UNKNOWN",
  "observation_status": "UNOBSERVED",
  "freshness": "UNKNOWN",
  "last_verified": null,
  "supersedes": [],
  "superseded_by": [],
  "provenance": [
    "DOC-083cde4caa627644-PM-CLAUDE-CORE-07",
    "SRC-PM-CLAUDE-CORE-07-URL-6727386D515F",
    "SRC-PM-CLAUDE-CORE-07-URL-C49DB9436032",
    "SRC-PM-CLAUDE-CORE-07-URL-DA45DE5646CB",
    "SRC-PM-CLAUDE-CORE-07-URL-D11C2860A404",
    "SRC-PM-CLAUDE-CORE-07-URL-DBCB362197DB",
    "SRC-PM-CLAUDE-CORE-07-URL-1BA226019836",
    "SRC-PM-CLAUDE-CORE-07-URL-1F49D0119CED",
    "SRC-PM-CLAUDE-CORE-07-URL-8E2E37051F9B",
    "SRC-PM-CLAUDE-CORE-07-URL-827ACF165DA2",
    "SRC-PM-CLAUDE-CORE-07-URL-16D2741870AD"
  ],
  "article_sources": [
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-6727386D515F",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, Define success criteria and build evaluations: <[",
      "href": "https://platform.claude.com/docs/en/test-and-evaluate/develop-tests",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-6727386D515F"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, Define success criteria and build evaluations: <[https://platform.claude.com/docs/en/test-and-evaluate/develop-tests"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-C49DB9436032",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, Prompt engineering overview / prompting best practices: <[",
      "href": "https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-C49DB9436032"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, Prompt engineering overview / prompting best practices: <[https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-DA45DE5646CB",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, Prompt engineering overview / prompting best practices: <[https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview](https://platform.claude.com/",
      "href": "https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-DA45DE5646CB"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, Prompt engineering overview / prompting best practices: <[https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview)>; <[https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-D11C2860A404",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, Improve your prompts in the developer console: <[",
      "href": "https://claude.com/blog/prompt-improver",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-D11C2860A404"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, Improve your prompts in the developer console: <[https://claude.com/blog/prompt-improver"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-DBCB362197DB",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, Batch processing / Structured outputs: <[",
      "href": "https://platform.claude.com/docs/en/build-with-claude/batch-processing",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-DBCB362197DB"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, Batch processing / Structured outputs: <[https://platform.claude.com/docs/en/build-with-claude/batch-processing"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-1BA226019836",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, Batch processing / Structured outputs: <[https://platform.claude.com/docs/en/build-with-claude/batch-processing](https://platform.claude.com/docs/en/build-with-claude/ba",
      "href": "https://platform.claude.com/docs/en/build-with-claude/structured-outputs",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-1BA226019836"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, Batch processing / Structured outputs: <[https://platform.claude.com/docs/en/build-with-claude/batch-processing](https://platform.claude.com/docs/en/build-with-claude/batch-processing)>; <[https://platform.claude.com/docs/en/build-with-claude/structured-outputs"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-1F49D0119CED",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, claude-cookbooks: <[",
      "href": "https://github.com/anthropics/claude-cookbooks",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-1F49D0119CED"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, claude-cookbooks: <[https://github.com/anthropics/claude-cookbooks"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-8E2E37051F9B",
      "evidence_class": "RESEARCH",
      "title": "Anthropic, claude-cookbooks: <[https://github.com/anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks)>; issue #761: <[",
      "href": "https://github.com/anthropics/claude-cookbooks/issues/761",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-8E2E37051F9B"
      ],
      "supporting_locators": [
        "Provenance",
        "- Anthropic, claude-cookbooks: <[https://github.com/anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks)>; issue #761: <[https://github.com/anthropics/claude-cookbooks/issues/761"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-827ACF165DA2",
      "evidence_class": "RESEARCH",
      "title": "Shi et al., Judging the Judges: <[",
      "href": "https://aclanthology.org/2025.ijcnlp-long.18/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-827ACF165DA2"
      ],
      "supporting_locators": [
        "Provenance",
        "- Shi et al., Judging the Judges: <[https://aclanthology.org/2025.ijcnlp-long.18/"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-PM-CLAUDE-CORE-07-URL-16D2741870AD",
      "evidence_class": "RESEARCH",
      "title": "Dekoninck et al., ConStat: <[",
      "href": "https://arxiv.org/abs/2405.16281",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "APPROVED_DOCUMENT_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-PM-CLAUDE-CORE-07-URL-16D2741870AD"
      ],
      "supporting_locators": [
        "Provenance",
        "- Dekoninck et al., ConStat: <[https://arxiv.org/abs/2405.16281"
      ],
      "scope": "Provenance",
      "provenance_status": "APPROVED_DOCUMENT_REFERENCE_PROJECTED"
    },
    {
      "id": "PM-CLAUDE-CORE-07-DOCUMENT-READBACK",
      "evidence_class": "IMPL",
      "title": "原資料との収録照合",
      "supports": [
        "資料単位の本文ハッシュ、段落数、表数を照合しています。"
      ],
      "limit": "収録照合は主張の真偽や外部根拠の最新性を追加保証しません。",
      "origin_class": "DOCUMENT_INGESTION_READBACK",
      "source_or_experiment_ids": [
        "DOC-083cde4caa627644-PM-CLAUDE-CORE-07"
      ],
      "supporting_locators": [
        "18WPldGemks2px0ukhKYINeBMrBwAmlAi",
        "083cde4caa627644036a2225f1f6ff29b8015bd902614fe1f7a00a997b4eb59c",
        "provider_mappings/PM-CLAUDE-CORE-07.md",
        "6f3ea62ffe35d7f186dcfffdac5f2dbc13eb5419afca09dafeeb24405127f841"
      ],
      "scope": "公開指定を受けた資料単位の全文・表・出典。",
      "provenance_status": "SOURCE_IDENTITY_VERIFIED"
    }
  ],
  "research_sections": [
    {
      "heading": "資料の識別情報",
      "text": "Source SHA-256 87d70f34057257f0c115f9ec6b6456f8c459c75d6068ad3dc28aaa5523f391f3"
    },
    {
      "heading": "Product Mapping — Claude / CORE-07",
      "text": "PRODUCT_MAPPING_ID: PM-CLAUDE-CORE-07-v0.1\n\nPROVIDER: CLAUDE / ANTHROPIC\n\nCORE_PACKET: PKT-GEMINI-A-CORE-07-v0.1\n\nAS_OF: 2026-09-10 JST\n\nSTATUS: PUBLIC_MAPPING_BOUNDED_COMPLETE / CONSOLE_TEMPORAL_BOUNDARY / LIVE_ORG_PENDING\n\nLAYER: PRODUCT_MAPPING (not Enterprise AI Core; not Organization Overlay)"
    },
    {
      "heading": "Scope",
      "text": "CORE-07のdataset、scorer/judge、holdout、slice gate、release evidenceを、Anthropicの評価設計guidance、Console prompt improver/evaluator、Message Batches、structured outputs、Agent SDK/Claude Codeおよびauthor-maintained cookbookへ対応づける。公開仕様・固定履歴・限定issueだけを対象とし、実organization、非公開dataset、Claude/model judgeの実性能、組織閾値はOrganization Overlayへ残す。"
    },
    {
      "heading": "Surface and lifecycle separation",
      "text": "| SURFACE | CURRENT EVIDENCE | DECISION | LIMIT |\n| --- | --- | --- | --- |\n| evaluation design guidance | success criteriaをspecific/measurableにし、task-specific・実分布・edge caseを含む評価、code/human/LLM gradingを使い分ける | provider guidanceをevaluation contract設計へ利用 | hosted registry、artifact immutability、retention、release authorityの保証ではない |\n| Console prompt improver / evaluator | 2024-10-14発表ではprompt/example改善、ideal output、5段階評価、反復feedbackを提供 | historical/current-accessed Console surfaceとして記録し、candidate作成面と評価面を分離 | 発表時のprefill手法はClaude 4.6以降のcurrent prompting guidanceと互換でない。現行のversion/export/retention保証は公開証拠から確定しない |\n| Message Batches | independent requestsをcustom_idで結合し、結果は順不同・部分成功、24時間expiry、29日availability | bulk eval実行carrierとして利用可能 | hosted eval registry・判定器・release decisionではない。batch完了を評価受入としない |\n| structured outputs | JSON schemaに沿う出力を強制し、schema cacheを持つ | grader/result envelopeの構文検証に利用 | schema conformanceはrubric妥当性、意味品質、judge calibrationを証明しない |\n| Agent SDK / Claude Code | tool、permission、hooks、session、result、OTel等のtrajectory carrier | agent evalではconfig、tool receipt、artifact/readback、外部最終状態を別join | ResultMessage/traceだけでは業務成果・side effectの完全性を証明しない |\n| anthropics/claude-cookbooks | automated evaluation、RAG等のauthor-maintained recipes | external/self-managed harness例として扱う | sample notebookはhosted product、SLA、current-model互換性の保証ではない |"
    },
    {
      "heading": "Core → Claude mapping rows",
      "text": "| ROW | CORE CONTRACT | CLAUDE / ANTHROPIC FINDING | DECISION / UNKNOWN |\n| --- | --- | --- | --- |\n| PM-C07-CLAUDE-001 | evaluation artifact separation | current guidanceはsuccess criteria、test cases、graderを説明するが、candidate/dataset/scorer/judge/run/releaseを一つのimmutable objectとして保証しない | Core contract IDsで外部結合し、Console/UI名を正本identityにしない |\n| PM-C07-CLAUDE-002 | representative and edge distribution | task-specific、real-world distribution、edge cases、held-out testsを推奨 | representative/failure/challenge/sealed holdoutを分離し、各sliceのprovenanceとaccessを記録 |\n| PM-C07-CLAUDE-003 | deterministic-first scoring | code-based gradingは高速・信頼性、人手は柔軟だが高コスト、LLM gradingはscalableだが信頼性確認が必要とされる | exact/schema/rule/tool receiptを先行し、LLM judgeはversioned補助判定に限定 |\n| PM-C07-CLAUDE-004 | judge registry and calibration | LLM graderには明確なrubricと指定出力が必要。独立研究はposition biasがjudge/taskごとに変動することを示す | judge model/config/prompt/rubric/order/sampling/abstention、人手不一致率を保存し、swap/permutationを行う |\n| PM-C07-CLAUDE-005 | candidate optimization / holdout split | Console prompt improverはprompt・examples・feedbackを反復変更し得る | improver/evaluatorへ提示したcaseをoptimization corpusへ分類し、sealed holdoutから失効。発表時機能の現行永続性はUNKNOWN |\n| PM-C07-CLAUDE-006 | temporal compatibility | 2024発表のprompt improverはassistant prefillを手法に含むが、current guidanceはClaude 4.6以降のlast-assistant prefill非対応を示す | candidate、model、API behavior、improver生成時点を固定し、生成promptをcurrent modelで回帰評価。古い機能説明を現行互換保証へ昇格しない |\n| PM-C07-CLAUDE-007 | evaluator infrastructure failure | cookbook issue #761は、Sonnet 4.6で拒否されたprefill例外をbroad catchがis_correct=Falseへ変換し、全件0%に見せ得ると報告 | API/harness/parser/timeoutをINFRA_ERRORとしてcandidate failureから分離し、評価器異常時はfail closed。issueは対象script/version限定 |\n| PM-C07-CLAUDE-008 | batch/output carrier boundary | Batchは順不同・部分状態、structured outputsは構文契約を提供 | custom_id、missing/errored/canceled/expired、retry、schema/parser errorを別記録し、valid N不足を平均へ混ぜない |\n| PM-C07-CLAUDE-009 | bounded contamination and release gate | ConStatはpublic benchmarkと同分布fresh/rephrased casesのgapを測る限定反証を提供 | named Claude modelの汚染断定には使わず、critical slice AND gate、fresh challenge、human adjudication、artifact readbackでreleaseを判定 |"
    },
    {
      "heading": "Safe verification profile",
      "text": "非機密synthetic fixtureだけを用い、今回はAnthropic API、Console、organization、model、Batch、SDK/CLI、tool、dataset、judgeを操作していない。\n\n- candidateにmodel ID、effort/thinking、system prompt、tools、permissions、SDK/CLI版、settings/source snapshotを含める。\n\n- dev/regression/challenge/failure corpus/sealed holdoutを分離し、prompt improver・example generation・人手feedbackへの露出をleak ledgerへ記録する。\n\n- deterministic checksを先行し、LLM judgeはorder swap、反復、abstention、人手calibrationを伴う補助判定にする。\n\n- prefillを含む旧prompt/harnessをcurrent modelへ移すnegative compatibility testを置き、API例外を0点へ変換しない。\n\n- Batchではcustom_idと各statusをread backし、missing/expired/erroredをINFRA_ERRORとして再実行可否と分離する。\n\n- agent taskはtrajectory、tool receipt、artifact digest、external readback、acceptanceを別証拠として結合する。\n\n- overall平均がpassでもcritical slice、known regression、Japanese、safety/over-refusal、tail、variance、valid Nのいずれかがfailならreleaseをblockする。\n\n既存EXP-C07-001の12/12はCOMPILER_EXPERIMENTであり、Claude、Console、Batch、judge、modelのlive試験へ再分類しない。"
    },
    {
      "heading": "Organization Overlay handoff",
      "text": "- adopted surface、organization/workspace、dataset owner/access、retention/export、holdout custodian\n\n- candidate model/effort/prompt/tool/config、grader/judge allowlist、人手sampleとdisagreement threshold\n\n- improver/training/eval corpusの分離、leak response、freshness/failure corpus更新周期\n\n- Batch quota/retry/deadline、infra-error policy、evidence retention\n\n- release threshold、critical slice、variance、zero-tolerance、exception/rollback authority"
    },
    {
      "heading": "Provenance",
      "text": "- Anthropic, Define success criteria and build evaluations: <[https://platform.claude.com/docs/en/test-and-evaluate/develop-tests](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests)> (accessed 2026-09-09)\n\n- Anthropic, Prompt engineering overview / prompting best practices: <[https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview)>; <[https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices)> (accessed 2026-09-09)\n\n- Anthropic, Improve your prompts in the developer console: <[https://claude.com/blog/prompt-improver](https://claude.com/blog/prompt-improver)> (published 2024-10-14; accessed 2026-09-09)\n\n- Anthropic, Batch processing / Structured outputs: <[https://platform.claude.com/docs/en/build-with-claude/batch-processing](https://platform.claude.com/docs/en/build-with-claude/batch-processing)>; <[https://platform.claude.com/docs/en/build-with-claude/structured-outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs)> (accessed 2026-09-09)\n\n- Anthropic, claude-cookbooks: <[https://github.com/anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks)>; issue #761: <[https://github.com/anthropics/claude-cookbooks/issues/761](https://github.com/anthropics/claude-cookbooks/issues/761)> (opened 2026-07-07; accessed 2026-09-09)\n\n- Shi et al., Judging the Judges: <[https://aclanthology.org/2025.ijcnlp-long.18/](https://aclanthology.org/2025.ijcnlp-long.18/)>\n\n- Dekoninck et al., ConStat: <[https://arxiv.org/abs/2405.16281](https://arxiv.org/abs/2405.16281)>"
    },
    {
      "heading": "Exit condition",
      "text": "Anthropicの評価guidance、Console improvement/evaluation、Batch、structured outputs、agent trajectory、cookbookを別surfaceとしてCORE-07へ対応づけ、temporal compatibilityとevaluator-infrastructure failureを境界付きで追加した。公開情報から確認できないConsole artifactのversion/export/retention、live judge性能、organization閾値はUNKNOWNのまま残す。Core claimは変更しないためGemini/GPT遡及監査は不要である。"
    }
  ],
  "human_url": "/providers/pm-claude-core-07",
  "markdown_url": "/agent/article.md?path=%2Fproviders%2Fpm-claude-core-07",
  "json_url": "/agent/article.json?path=%2Fproviders%2Fpm-claude-core-07"
}