# GROK：評価データと公開前の評価契約

CURRENT_STATUS: UNKNOWN
OBSERVATION: UNOBSERVED
FRESHNESS: UNKNOWN
PAGE_TYPE: reference
CANONICAL_URL: /providers/pm-grok-core-07
CLAIM_IDS: none
EVIDENCE_IDS: EVD-PM-GROK-CORE-07-URL-66321E57AE7A, EVD-PM-GROK-CORE-07-URL-C547143F6623, EVD-PM-GROK-CORE-07-URL-CC7D258F82FC, EVD-PM-GROK-CORE-07-URL-0F8FE06795BB, EVD-PM-GROK-CORE-07-URL-0878B9C4BDC7, EVD-PM-GROK-CORE-07-URL-E3CAAEA65CC5, EVD-PM-GROK-CORE-07-URL-2DC3FAB17BB9, EVD-PM-GROK-CORE-07-URL-086CFC997988, EVD-PM-GROK-CORE-07-URL-905D5D7A5FB4, PM-GROK-CORE-07-DOCUMENT-READBACK
UNKNOWN_IDS: PM-GROK-CORE-07-CURRENTNESS
OPS_IDS: none

## Summary
CORE-07のgolden dataset、scorer/judge、holdout、slice/tail gate、release evidenceを、Grok 4.6 announcement/model card、xAI APIのBatch/structured-output/model identity、author-maintained Grok Build、独立benchmark methodologyへ対応づける。公開benchmark値はその評価条件に限定し、xAI organizationのrelease acceptance、実tenant性能、非公開dataset、汚染有無、judge妥当性へ外挿しない。

## Current judgment
CORE-07のgolden dataset、scorer/judge、holdout、slice/tail gate、release evidenceを、Grok 4.6 announcement/model card、xAI APIのBatch/structured-output/model identity、author-maintained Grok Build、独立benchmark methodologyへ対応づける。公開benchmark値はその評価条件に限定し、xAI organizationのrelease acceptance、実tenant性能、非公開dataset、汚染有無、judge妥当性へ外挿しない。

## Scope
CORE-07のgolden dataset、scorer/judge、holdout、slice/tail gate、release evidenceを、Grok 4.6 announcement/model card、xAI APIのBatch/structured-output/model identity、author-maintained Grok Build、独立benchmark methodologyへ対応づける。公開benchmark値はその評価条件に限定し、xAI organizationのrelease acceptance、実tenant性能、非公開dataset、汚染有無、judge妥当性へ外挿しない。

## Confirmed

## UNKNOWN
- PM-GROK-CORE-07-CURRENTNESS: 原資料の確定版から収録しています。外部根拠の最新性を今回の取込日で確認済みと扱いません。本文の未解決事項・残余制約を保持します。

## Evidence
- EVD-PM-GROK-CORE-07-URL-66321E57AE7A (RESEARCH): xAI, Introducing Grok 4.6: <[ — https://x.ai/news/grok-4-6
- EVD-PM-GROK-CORE-07-URL-C547143F6623 (RESEARCH): xAI, Grok 4.6 Model Card: <[ — https://media.x.ai/v1/website/card-4p6-4cd2dc57.pdf
- EVD-PM-GROK-CORE-07-URL-CC7D258F82FC (RESEARCH): xAI, xai-org/grok-build: <[ — https://github.com/xai-org/grok-build
- EVD-PM-GROK-CORE-07-URL-0F8FE06795BB (RESEARCH): xAI, Grok Build changelog: <[ — https://x.ai/build/changelog
- EVD-PM-GROK-CORE-07-URL-0878B9C4BDC7 (RESEARCH): Artificial Analysis, Benchmarking Methodology / Intelligence Index: <[ — https://artificialanalysis.ai/methodology
- EVD-PM-GROK-CORE-07-URL-E3CAAEA65CC5 (RESEARCH): Artificial Analysis, Benchmarking Methodology / Intelligence Index: <[ — https://artificialanalysis.ai/
- EVD-PM-GROK-CORE-07-URL-2DC3FAB17BB9 (RESEARCH): xAI Batch API / Structured Outputs / Models: <[ — https://docs.x.ai/developers/advanced-api-usage/batch-api
- EVD-PM-GROK-CORE-07-URL-086CFC997988 (RESEARCH): xAI Batch API / Structured Outputs / Models: <[https://docs.x.ai/developers/advanced-api-usage/batch-api](https://docs.x.ai/developers/advanced-api-usage/batch-api)>; <[ — https://docs.x.ai/developers/model-capabilities/text/structured-outputs
- EVD-PM-GROK-CORE-07-URL-905D5D7A5FB4 (RESEARCH): xAI Batch API / Structured Outputs / Models: <[https://docs.x.ai/developers/advanced-api-usage/batch-api](https://docs.x.ai/developers/advanced-api-usage/batch-api)>; <[https://doc — https://docs.x.ai/developers/models
- PM-GROK-CORE-07-DOCUMENT-READBACK (IMPL): 原資料との収録照合

## Practice
CORE-07のgolden dataset、scorer/judge、holdout、slice/tail gate、release evidenceを、Grok 4.6 announcement/model card、xAI APIのBatch/structured-output/model identity、author-maintained Grok Build、独立benchmark methodologyへ対応づける。公開benchmark値はその評価条件に限定し、xAI organizationのrelease acceptance、実tenant性能、非公開dataset、汚染有無、judge妥当性へ外挿しない。
## 資料の識別情報
Source SHA-256 e2ec3734d887823cc889549b45a5c24b8b98bf0fb9ca247ff10ff40461134b20

## Product Mapping — Grok / CORE-07
PRODUCT_MAPPING_ID: PM-GROK-CORE-07-v0.1

PROVIDER: GROK / xAI

CORE_PACKET: PKT-GEMINI-A-CORE-07-v0.1

AS_OF: 2026-09-13 JST

STATUS: PUBLIC_MAPPING_BOUNDED_COMPLETE / BENCHMARK_PROVENANCE_SPLIT / LIVE_ORG_PENDING

LAYER: PRODUCT_MAPPING (not Enterprise AI Core; not Organization Overlay)

## Scope
CORE-07のgolden dataset、scorer/judge、holdout、slice/tail gate、release evidenceを、Grok 4.6 announcement/model card、xAI APIのBatch/structured-output/model identity、author-maintained Grok Build、独立benchmark methodologyへ対応づける。公開benchmark値はその評価条件に限定し、xAI organizationのrelease acceptance、実tenant性能、非公開dataset、汚染有無、judge妥当性へ外挿しない。

## Surface and lifecycle separation
| SURFACE | CURRENT EVIDENCE | DECISION | LIMIT |
| --- | --- | --- | --- |
| release announcement | 2026-08-12のGrok 4.6発表はAA Intelligence、GDPVal-AA、DeepSWE 1.1、CursorBench 3.2、FrontierCode 1.1等を掲示 | model/release時点のcapability evidenceとして保存 | competitor値は各社system cardまたはleaderboard由来で、単一共通runではない |
| model card | final deployed checkpointを原則としつつ、Cursor、Mercor、Cognition、Datacurve、Abundant AI、Harbor、Artificial Analysis等の実施結果と複数harness/effortを併記 | benchmark名/version、runner、harness、effort、sample、metric、reported-byを行単位で保存 | public scoreはorganization固有task、holdout、acceptance thresholdの証拠ではない |
| Grok Build | xAIのcoding-agent harness/TUI。公開repoはmonorepoからperiodic syncされ、SOURCE_REVを持つ | coding/agent評価ではbinary versionとsource revision、config、tools、workspaceをcandidate environmentへ含める | current repo/sourceは過去model-card runのexact harness artifactを自動再構成しない |
| Batch API | request単位IDとpending/success/error/cancel/expiryを持つ非同期carrier | bulk evaluationのtransport/statusとして利用可能 | batch completionはvalid result、semantic pass、release acceptanceではない |
| structured outputs | JSON Schema subsetでresult envelopeを制約可能 | parser/schemaのdeterministic checkへ利用 | schema conformanceはoracle、rubric、judge calibration、意味品質を保証しない |
| independent benchmark | Artificial Analysisはmodel/API/systemの独立benchmark methodologyとversioned composite indexを公開 | first-party model cardと独立測定を別provenanceで比較 | index version変更により構成評価が変わるため、異版scoreを同一系列として直結しない |

## Core → Grok mapping rows
| ROW | CORE CONTRACT | XAI / GROK FINDING | DECISION / UNKNOWN |
| --- | --- | --- | --- |
| PM-C07-GROK-001 | candidate/evaluation artifact separation | announcement、model card、API response、Batch、Grok Buildは別artifact/surface | candidate config、dataset、scorer/judge、run、result、release decisionを外部ledgerで結合 |
| PM-C07-GROK-002 | versioned benchmark provenance | model cardはbenchmarkごとにversion、harness、thinking effort、実施者が異なる | scoreごとにexact benchmark/harness/effort/reported-by/as-ofを保存し、混合bar chartを共通runへ昇格しない |
| PM-C07-GROK-003 | representative slice and acceptance | public coding/knowledge/safety benchmarksは広いが、組織固有のJapanese業務、RAG、tool、副作用、artifact受入を代表するとは限らない | organization task distribution、failure corpus、freshness、sealed holdoutを別設計 |
| PM-C07-GROK-004 | deterministic-first scoring | structured outputとcoding harnessはschema、test、artifactの機械判定を支援できる | schema/test/receipt/readbackを先行し、model/judge評価だけでcritical correctnessを閉じない |
| PM-C07-GROK-005 | judge registry/calibration | model cardにはmodel-based gradingや外部pairwise/rubric評価があるが、全judge config・order・human disagreementが一様に公開されるわけではない | judge model/config/prompt/rubric/order/sampling/abstention、人手校正をrun artifactへ固定 |
| PM-C07-GROK-006 | harness identity | Grok Build repoはSOURCE_REVを持ち、2026-09-09のcurrent releaseはv1.0.25 | binary/version/source revision、model、tools、permissions、workspace image、networkを固定。current sourceを過去結果のexact harnessとみなさない |
| PM-C07-GROK-007 | infra/candidate failure split | Batchはpartial/error/cancel/expiryを持ち、agent harnessもtool/terminal/network失敗を含み得る | INFRA_ERROR、invalid、missing、candidate failureを分離し、valid N不足を0点や合格へ変換しない |
| PM-C07-GROK-008 | contamination/holdout boundary | public benchmark scoreと「contamination-resistant」という選定理由は、当該modelの非汚染を単独証明しない | leak ledger、fresh/rephrased challenge、sealed holdout、ConStat型gapを限定反証として使用し、汚染断定にも無汚染証明にも使わない |
| PM-C07-GROK-009 | slice/tail/repetition gate | benchmark aggregateやpass@1はcritical tail、組織固有regression、反復varianceを隠し得る | slice AND gate、zero-tolerance、repeat distribution、worst case、unsafe/over-refusal対を要求 |
| PM-C07-GROK-010 | Organization boundary | 採用model/effort/surface/harness、dataset owner、threshold、holdout custodian、judge allowlist、release authorityは公開資料から確定しない | Organization Overlayへ移送し、UNKNOWNを維持 |

## Safe verification profile
非機密synthetic fixtureと許可されたtest teamだけを使う。今回はxAI API、team、Console、model、Batch、Grok Build binary、tool、dataset、judgeを操作していない。

- candidateにmodel/date pin、reasoning effort、system prompt、tool/source/schema、API/SDK、Grok Build binary/version/SOURCE_REV、workspace imageを固定する。

- dev、representative、known-failure、freshness、challenge、sealed holdoutを分離し、benchmark/public example/prompt tuningへの露出をleak ledgerへ記録する。

- exact/schema/calculation/test/receipt/artifact/readbackをdeterministic-firstで判定し、LLM judgeはorder swap、反復、abstention、人手sampleを伴う補助判定にする。

- Batchはrequest IDごとにstatusをread backし、error/cancel/expire/missing/parser failureをINFRA_ERRORとしてcandidate errorから分離する。

- agent/coding taskはtrajectory、tool receipt、git/workspace diff、tests、artifact digest、external readback、human acceptanceを別証拠としてjoinする。

- 公開benchmark再現laneとorganization golden set laneを分け、異なるbenchmark version/harness/effortのscoreを同一aggregateへ混ぜない。

- overall平均がpassでもcritical slice、known regression、Japanese、safety/over-refusal、tail、variance、valid Nのいずれかがfailならreleaseをblockする。

既存EXP-C07-001の12/12はCOMPILER_EXPERIMENTであり、Grok/xAI、Grok Build、Batch、judge、modelのlive試験へ再分類しない。

## Organization Overlay handoff
- adopted API/consumer/Build surface、team/region、model/date pin/effort、SDK/Build version/source revision

- dataset owner/access、holdout custodian、leak ledger、freshness/failure corpus更新周期

- scorer/judge registry、人手sample、disagreement/abstention/tie-break policy

- Batch quota/deadline/retry、infra-error policy、valid-N floor、evidence retention

- slice threshold、critical failure、variance、cost ceiling、release/exception/rollback authority

## Provenance
- xAI, Introducing Grok 4.6: <[https://x.ai/news/grok-4-6](https://x.ai/news/grok-4-6)> (published 2026-08-12; read 2026-09-13)

- xAI, Grok 4.6 Model Card: <[https://media.x.ai/v1/website/card-4p6-4cd2dc57.pdf](https://media.x.ai/v1/website/card-4p6-4cd2dc57.pdf)> (release artifact read 2026-09-13)

- xAI, xai-org/grok-build: <[https://github.com/xai-org/grok-build](https://github.com/xai-org/grok-build)> (SOURCE_REV c4ea71cfdbcdb21e32e41bc25a0043d7d4836714; read 2026-09-13)

- xAI, Grok Build changelog: <[https://x.ai/build/changelog](https://x.ai/build/changelog)> (v1.0.25 dated 2026-09-09; read 2026-09-13)

- Artificial Analysis, Benchmarking Methodology / Intelligence Index: <[https://artificialanalysis.ai/methodology](https://artificialanalysis.ai/methodology)>; <[https://artificialanalysis.ai/](https://artificialanalysis.ai/)> (current index v4.3 observed 2026-09-13)

- xAI Batch API / Structured Outputs / Models: <[https://docs.x.ai/developers/advanced-api-usage/batch-api](https://docs.x.ai/developers/advanced-api-usage/batch-api)>; <[https://docs.x.ai/developers/model-capabilities/text/structured-outputs](https://docs.x.ai/developers/model-capabilities/text/structured-outputs)>; <[https://docs.x.ai/developers/models](https://docs.x.ai/developers/models)> (reused)

## Exit condition
Grok 4.6のrelease/model-card benchmark、外部実施者、harness/effort、Grok Build source/release、Batch、structured output、独立benchmark methodologyを別surfaceとしてCORE-07へ対応づけた。公開scoreをorganization release acceptance、汚染有無、judge妥当性、live tenant性能へ昇格しない。Core claimは変更しないためGemini/GPT/Claude遡及監査は不要である。

NEXT: /agent/start.json
STOP: This is a knowledge projection, not an execution order.