{
  "format": "agent-article-v0",
  "article_id": "EA-088",
  "canonical_url": "/tools/ea-088-gpu-npu-capacity-planning",
  "page_type": "reference",
  "claim_ids": [],
  "evidence_ids": [
    "EVD-EA-088-URL-4F7AA3D7A047",
    "EVD-EA-088-URL-D055A8F098F0",
    "EVD-EA-088-URL-403B57FB4806",
    "EVD-EA-088-URL-EF7AEAF571C5",
    "EVD-EA-088-URL-97A585987B60",
    "EVD-EA-088-URL-46A1F6B8FEF5",
    "EVD-EA-088-URL-C2FDA96ED6A6",
    "EVD-EA-088-URL-B5CAABDFCC04",
    "EVD-EA-088-URL-5818D9FA73EA",
    "EVD-EA-088-URL-E56B6E7FC5BA",
    "EVD-EA-088-URL-0E241A2F5382",
    "EVD-EA-088-URL-AB3C73CD6AEC",
    "EVD-EA-088-URL-8CB7063F9404",
    "EA-088-R11-RESEARCH",
    "EA-088-R11-WEB-READBACK"
  ],
  "unknown_ids": [
    "EA-088-UNKNOWN-01",
    "EA-088-UNKNOWN-CURRENTNESS"
  ],
  "operational_object_ids": [],
  "provider": [
    "Google"
  ],
  "models": [],
  "plans": [],
  "surfaces": [],
  "regions": [],
  "current_status": "UNKNOWN",
  "observation_status": "SPEC_ONLY",
  "freshness": "UNKNOWN",
  "last_verified": null,
  "supersedes": [],
  "superseded_by": [],
  "provenance": [
    "COMM-080-EA0088",
    "SRC-EA-088-URL-4F7AA3D7A047",
    "SRC-EA-088-URL-D055A8F098F0",
    "SRC-EA-088-URL-403B57FB4806",
    "SRC-EA-088-URL-EF7AEAF571C5",
    "SRC-EA-088-URL-97A585987B60",
    "SRC-EA-088-URL-46A1F6B8FEF5",
    "SRC-EA-088-URL-C2FDA96ED6A6",
    "SRC-EA-088-URL-B5CAABDFCC04",
    "SRC-EA-088-URL-5818D9FA73EA",
    "SRC-EA-088-URL-E56B6E7FC5BA",
    "SRC-EA-088-URL-0E241A2F5382",
    "SRC-EA-088-URL-AB3C73CD6AEC",
    "SRC-EA-088-URL-8CB7063F9404",
    "WEBBODY-EA0088-R8-7148721F"
  ],
  "article_sources": [
    {
      "id": "EVD-EA-088-URL-4F7AA3D7A047",
      "evidence_class": "RESEARCH",
      "title": "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA;",
      "href": "https://www.nvidia.com/en-us/data-center/h100/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-4F7AA3D7A047"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA; https://www.nvidia.com/en-us/data-center/h100/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-D055A8F098F0",
      "evidence_class": "RESEARCH",
      "title": "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA; https://www.nvidia.com/en-us/data-center/h100/ ;",
      "href": "https://www.nvidia.com/en-us/data-center/h200/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-D055A8F098F0"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA; https://www.nvidia.com/en-us/data-center/h100/ ; https://www.nvidia.com/en-us/data-center/h200/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-403B57FB4806",
      "evidence_class": "RESEARCH",
      "title": "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA; https://www.nvidia.com/en-us/data-center/h100/ ; https://w",
      "href": "https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-403B57FB4806"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA; https://www.nvidia.com/en-us/data-center/h100/ ; https://www.nvidia.com/en-us/data-center/h200/ ; https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-EF7AEAF571C5",
      "evidence_class": "RESEARCH",
      "title": "2) official accelerator orchestration documentation — NVIDIA GPU Operator / MIG documentation; NVIDIA;",
      "href": "https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-EF7AEAF571C5"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "2) official accelerator orchestration documentation — NVIDIA GPU Operator / MIG documentation; NVIDIA; https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-97A585987B60",
      "evidence_class": "RESEARCH",
      "title": "2) official accelerator orchestration documentation — NVIDIA GPU Operator / MIG documentation; NVIDIA; https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/ ;",
      "href": "https://docs.nvidia.com/datacenter/tesla/mig-user-guide/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-97A585987B60"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "2) official accelerator orchestration documentation — NVIDIA GPU Operator / MIG documentation; NVIDIA; https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/ ; https://docs.nvidia.com/datacenter/tesla/mig-user-guide/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-46A1F6B8FEF5",
      "evidence_class": "RESEARCH",
      "title": "3) official cloud accelerator documentation — Cloud TPU v6e documentation; Google Cloud;",
      "href": "https://cloud.google.com/tpu/docs/v6e",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-46A1F6B8FEF5"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "3) official cloud accelerator documentation — Cloud TPU v6e documentation; Google Cloud; https://cloud.google.com/tpu/docs/v6e"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-C2FDA96ED6A6",
      "evidence_class": "RESEARCH",
      "title": "4) official accelerator documentation — AWS Inferentia2 / EC2 Inf2 instances; Amazon Web Services;",
      "href": "https://aws.amazon.com/machine-learning/inferentia/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-C2FDA96ED6A6"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "4) official accelerator documentation — AWS Inferentia2 / EC2 Inf2 instances; Amazon Web Services; https://aws.amazon.com/machine-learning/inferentia/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-B5CAABDFCC04",
      "evidence_class": "RESEARCH",
      "title": "4) official accelerator documentation — AWS Inferentia2 / EC2 Inf2 instances; Amazon Web Services; https://aws.amazon.com/machine-learning/inferentia/ ;",
      "href": "https://aws.amazon.com/ec2/instance-types/inf2/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-B5CAABDFCC04"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "4) official accelerator documentation — AWS Inferentia2 / EC2 Inf2 instances; Amazon Web Services; https://aws.amazon.com/machine-learning/inferentia/ ; https://aws.amazon.com/ec2/instance-types/inf2/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-5818D9FA73EA",
      "evidence_class": "RESEARCH",
      "title": "5) author-maintained implementation documentation — Distributed Inference and Serving / Optimization and Tuning; vLLM project;",
      "href": "https://docs.vllm.ai/en/latest/serving/distributed_serving.html",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-5818D9FA73EA"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "5) author-maintained implementation documentation — Distributed Inference and Serving / Optimization and Tuning; vLLM project; https://docs.vllm.ai/en/latest/serving/distributed_serving.html"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-E56B6E7FC5BA",
      "evidence_class": "RESEARCH",
      "title": "5) author-maintained implementation documentation — Distributed Inference and Serving / Optimization and Tuning; vLLM project; https://docs.vllm.ai/en/latest/serving/distributed_se",
      "href": "https://docs.vllm.ai/en/latest/configuration/optimization.html",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-E56B6E7FC5BA"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "5) author-maintained implementation documentation — Distributed Inference and Serving / Optimization and Tuning; vLLM project; https://docs.vllm.ai/en/latest/serving/distributed_serving.html ; https://docs.vllm.ai/en/latest/configuration/optimization.html"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-0E241A2F5382",
      "evidence_class": "RESEARCH",
      "title": "6) independent/open industry benchmark — MLPerf Inference v6.0 results; MLCommons;",
      "href": "https://mlcommons.org/2026/04/mlperf-inference-v6-0-results/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-0E241A2F5382"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "6) independent/open industry benchmark — MLPerf Inference v6.0 results; MLCommons; https://mlcommons.org/2026/04/mlperf-inference-v6-0-results/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-AB3C73CD6AEC",
      "evidence_class": "RESEARCH",
      "title": "7) independent/open benchmark methodology — A new GPT-OSS benchmark and DeepSeek R1 updates for latency-optimized reasoning; MLCommons;",
      "href": "https://mlcommons.org/2026/03/mlperf-inference-gpt-oss/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-AB3C73CD6AEC"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "7) independent/open benchmark methodology — A new GPT-OSS benchmark and DeepSeek R1 updates for latency-optimized reasoning; MLCommons; https://mlcommons.org/2026/03/mlperf-inference-gpt-oss/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-088-URL-8CB7063F9404",
      "evidence_class": "RESEARCH",
      "title": "8) independent/open mobile benchmark — MLPerf Mobile v6.0: New GenAI Benchmarks for On-Device LLMs; MLCommons;",
      "href": "https://mlcommons.org/2026/06/mlperf-mobile-v6/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-088-URL-8CB7063F9404"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "8) independent/open mobile benchmark — MLPerf Mobile v6.0: New GenAI Benchmarks for On-Device LLMs; MLCommons; https://mlcommons.org/2026/06/mlperf-mobile-v6/"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EA-088-R11-RESEARCH",
      "evidence_class": "RESEARCH",
      "title": "EA-088 監査済みresearch / canonical revision binding",
      "supports": [
        "1. Capacity planningの最小単位は accelerator SKU ではなく `MODEL × RUNTIME × WORKLOAD × SLA × TOPOLOGY` である。同じacceleratorでもmodel precision、quantization、context、batch/concurrency、KV-cache policy、runtime/kernel、tensor/pipeline parallelismでfitとperformanceは変わる。\n2. memory fitをweightsだけで計算してはならない。少なくとも model weights + KV cache + runtime/activation/workspace + communication/fragmentation headroom を分離する。autoregressive LLMではcontext lengthとconcurrent sequenceがKV-cache pressureを増やす。OOMしないこととSLAを満たすことは別判定である。\n3. HBM容量とbandwidthは別軸。大容量memoryはmodel/context fitを改善し得るが、decode/prefill、memory bandwidth、compute、interconnect、scheduler bottleneckのどれが支配的かはworkload依存。公称FLOPS/TOPSだけでtoken throughputを予測しない。\n4. parallelismは容量拡張であると同時に通信コストを導入する。vLLM等のdistributed servingではtensor/pipeline parallelismを利用できるが、accelerator間・node間interconnect、collective communication、load balanceがtail latencyとthroughputを変える。GPU枚数を倍にして性能が線形に倍増するとは扱わない。\n5. partitioningはcapacity isolationとperformance isolationを同一視しない。MIG等でaccelerator資源をpartitionできても、選択profile、memory/compute slice、workload shape、scheduler、I/Oを含めてtenant fixtureで検証する必要がある。\n6. NPU/TPU/ASICは「GPUより速い/安い」という一般claimにしない。Inferentia/TPU/on-device NPU等は対応compiler/runtime/operator/model、topology、region/device availabilityがgateになる。supported pathでのprice/performanceが良くてもunsupported model/operatorやmigration costを含む総運用適合性は別問題である。\n7. 公開benchmarkは比較入口でありcapacity guaranteeではない。MLPerf Inference v6.0はarchitecture-neutral/reproducibleな比較基盤を提供し、GPT-OSS 120BやDeepSeek-R1でlatency-constrained scenariosも持つ。だがsubmissionは特定model、precision、software stack、system topology、scenarioで成立した結果であり、tenant prompt length、output length、concurrency、RAG/tool overhead、SLAへそのまま外挿しない。\n8. MLPerf v6.0ではmulti-node submissionが増え、large-scale capacityが単体accelerator性能ではなくnetwork/interconnect/storage/software orchestrationを含むsystem problemであることを裏付ける。\n9. on-device/NPUについてもMLPerf Mobile v6.0がLLM benchmarkを導入し、CPU executionと特定supported NPU accelerationを区別している。「NPU搭載端末」だけでは任意LLMがNPU実行可能とは言えない。"
      ],
      "limit": "canonical監査・revision bindingの存在を示す。外部ソースcurrentnessが本文またはcontrol-planeで確定していない場合は、currentnessをUNKNOWNのまま保持する。",
      "origin_class": "AUDITED_CANONICAL_RESEARCH_PACKET",
      "source_or_experiment_ids": [
        "COMM-080-EA0088"
      ],
      "supporting_locators": [
        "COMM-080-EA0088",
        "EA-088-R5-LEGACY-MIGRATION-20261008-R1",
        "content/staging/EA-088.md"
      ],
      "scope": "監査済みEA本文・canonical revision・既存Source Graph relationの範囲。欠損currentnessは推測で閉じない。",
      "provenance_status": "AUDIT_PASS_CANONICAL_BINDING"
    },
    {
      "id": "EA-088-R11-WEB-READBACK",
      "evidence_class": "IMPL",
      "title": "EA-088 Web body identity / canonical readback",
      "supports": [
        "canonical staging bodyのSHA-256一致とR9 canonical ledger readbackをProduction projectionの入力として検証する。"
      ],
      "limit": "body identity/readbackは本文claim自体の真偽や外部ソースの最新性を追加保証しない。",
      "checked_at": null,
      "origin_class": "IMPLEMENTATION_READBACK",
      "source_or_experiment_ids": [
        "WEBBODY-EA0088-R8-7148721F"
      ],
      "supporting_locators": [
        "WEBBODY-EA0088-R8-7148721F",
        "content/staging/EA-088.md",
        "EA-088-R5-LEGACY-MIGRATION-20261008-R1"
      ],
      "scope": "R9/R10 canonical readback後のProduction生成時body identity。",
      "provenance_status": "CANONICAL_READBACK_BODY_HASH_MATCH"
    }
  ],
  "research_sections": [
    {
      "heading": "WHY_THIS_MATTERS_TO_UPDATE",
      "text": "Private AI、self-hosted inference、edge/on-device、fine-tuning、batch/streamingを実装段階へ進めると、モデル名やparameter countだけでは必要accelerator数を決められない。weights、KV cache、activation/workspace、quantization、context length、concurrency、prefill/decode比、parallelism、interconnect、runtime/kernel、SLA、power/costがcapacityを決める。GPU/NPU/TPU等のaccelerator class間でもmemory architecture・software stack・supported model/operatorが異なるため、単純な「TOPS」「VRAM」「GPU枚数」比較を辞典のcapacity claimへ持ち込まないための独立contractが必要である。"
    },
    {
      "heading": "SCOPE_ROUTING",
      "text": "Enterprise AI Core: capacity planning / deployment acceptance / operability / cost governance\nprovider mapping: NVIDIA GPU、Google TPU、AWS Inferentia/Trainium等のavailability/spec mapping\norganization overlay: quota、region、procurement、power/cooling、SRE ownership\nbenchmark: MLPerf + tenant workload replay\nimplementation: model server/runtime、parallelism、KV cache、quantization、scheduler\nnon-LLM component: accelerator、HBM、interconnect、host memory/network/storage"
    },
    {
      "heading": "CLAIM_OR_RESEARCH_QUESTION",
      "text": "1. accelerator capacityはparameter countやnominal VRAMだけでなく、実際のserving/training workload envelopeで決めるべきか。\n2. GPU/NPU/TPUの公称specや公開benchmarkを、tenant固有SLA・model/runtime構成へどこまで外挿できるか。\n3. model fit、steady-state throughput、tail latency、burst/concurrency、failure/recovery、cost/powerを同一capacity contractで追跡できるか。"
    },
    {
      "heading": "EVIDENCE TABLE",
      "text": "1) official hardware specification — NVIDIA H100 Tensor Core GPU / H200 Tensor Core GPU / Blackwell architecture; NVIDIA; https://www.nvidia.com/en-us/data-center/h100/ ; https://www.nvidia.com/en-us/data-center/h200/ ; https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/ ; accessed 2026-09-16; exact board/SXM/PCIe SKU must be pinned per deployment; locator: memory capacity/bandwidth and platform specifications; VERIFIED_SPEC + PRODUCT_VOLATILE; support/context.\n2) official accelerator orchestration documentation — NVIDIA GPU Operator / MIG documentation; NVIDIA; https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/ ; https://docs.nvidia.com/datacenter/tesla/mig-user-guide/ ; accessed 2026-09-16; deployment-specific Operator/driver/CUDA/device-plugin/MIG profile must be pinned; locator: component lifecycle and MIG partition/profile semantics; VERIFIED_SPEC + OSS_PRACTICE + PRODUCT_VOLATILE; support/limit.\n3) official cloud accelerator documentation — Cloud TPU v6e documentation; Google Cloud; https://cloud.google.com/tpu/docs/v6e ; accessed 2026-09-16; TPU v6e; locator: architecture/specification/topology; VERIFIED_SPEC + PRODUCT_VOLATILE; support/context.\n4) official accelerator documentation — AWS Inferentia2 / EC2 Inf2 instances; Amazon Web Services; https://aws.amazon.com/machine-learning/inferentia/ ; https://aws.amazon.com/ec2/instance-types/inf2/ ; accessed 2026-09-16; exact instance type and Neuron release must be pinned; locator: accelerator memory, instance topology, Neuron software; VERIFIED_SPEC + PRODUCT_VOLATILE; support/context.\n5) author-maintained implementation documentation — Distributed Inference and Serving / Optimization and Tuning; vLLM project; https://docs.vllm.ai/en/latest/serving/distributed_serving.html ; https://docs.vllm.ai/en/latest/configuration/optimization.html ; accessed 2026-09-16; exact installed vLLM release/commit UNKNOWN in this packet; locator: tensor/pipeline parallel deployment and GPU memory/KV-cache tuning; OSS_PRACTICE; support/limit.\n6) independent/open industry benchmark — MLPerf Inference v6.0 results; MLCommons; https://mlcommons.org/2026/04/mlperf-inference-v6-0-results/ ; published 2026-04-01 / accessed 2026-09-16; MLPerf Inference v6.0; locator: GPT-OSS 120B, DeepSeek-R1 interactive, multi-node submissions, LoadGen++; RESEARCH + VERIFIED_SPEC; support/context/limit.\n7) independent/open benchmark methodology — A new GPT-OSS benchmark and DeepSeek R1 updates for latency-optimized reasoning; MLCommons; https://mlcommons.org/2026/03/mlperf-inference-gpt-oss/ ; published 2026-03-24 / accessed 2026-09-16; MLPerf Inference v6.0; locator: GPT-OSS 120B performance mode, Interactive TTFT p99 <=2.0s / TPOT <=15ms, Server TTFT p99 <=3.0s / TPOT <=80ms, accuracy/compliance separation; RESEARCH + VERIFIED_SPEC; support/limit.\n8) independent/open mobile benchmark — MLPerf Mobile v6.0: New GenAI Benchmarks for On-Device LLMs; MLCommons; https://mlcommons.org/2026/06/mlperf-mobile-v6/ ; published 2026-06-15 / accessed 2026-09-16; MLPerf Mobile v6.0; locator: Llama 3.2 1B/3B and Llama 3.1 8B on-device tests, CPU fallback and supported NPU acceleration; RESEARCH + VERIFIED_SPEC; support/context/limit."
    },
    {
      "heading": "FINDINGS",
      "text": "1. Capacity planningの最小単位は accelerator SKU ではなく `MODEL × RUNTIME × WORKLOAD × SLA × TOPOLOGY` である。同じacceleratorでもmodel precision、quantization、context、batch/concurrency、KV-cache policy、runtime/kernel、tensor/pipeline parallelismでfitとperformanceは変わる。\n2. memory fitをweightsだけで計算してはならない。少なくとも model weights + KV cache + runtime/activation/workspace + communication/fragmentation headroom を分離する。autoregressive LLMではcontext lengthとconcurrent sequenceがKV-cache pressureを増やす。OOMしないこととSLAを満たすことは別判定である。\n3. HBM容量とbandwidthは別軸。大容量memoryはmodel/context fitを改善し得るが、decode/prefill、memory bandwidth、compute、interconnect、scheduler bottleneckのどれが支配的かはworkload依存。公称FLOPS/TOPSだけでtoken throughputを予測しない。\n4. parallelismは容量拡張であると同時に通信コストを導入する。vLLM等のdistributed servingではtensor/pipeline parallelismを利用できるが、accelerator間・node間interconnect、collective communication、load balanceがtail latencyとthroughputを変える。GPU枚数を倍にして性能が線形に倍増するとは扱わない。\n5. partitioningはcapacity isolationとperformance isolationを同一視しない。MIG等でaccelerator資源をpartitionできても、選択profile、memory/compute slice、workload shape、scheduler、I/Oを含めてtenant fixtureで検証する必要がある。\n6. NPU/TPU/ASICは「GPUより速い/安い」という一般claimにしない。Inferentia/TPU/on-device NPU等は対応compiler/runtime/operator/model、topology、region/device availabilityがgateになる。supported pathでのprice/performanceが良くてもunsupported model/operatorやmigration costを含む総運用適合性は別問題である。\n7. 公開benchmarkは比較入口でありcapacity guaranteeではない。MLPerf Inference v6.0はarchitecture-neutral/reproducibleな比較基盤を提供し、GPT-OSS 120BやDeepSeek-R1でlatency-constrained scenariosも持つ。だがsubmissionは特定model、precision、software stack、system topology、scenarioで成立した結果であり、tenant prompt length、output length、concurrency、RAG/tool overhead、SLAへそのまま外挿しない。\n8. MLPerf v6.0ではmulti-node submissionが増え、large-scale capacityが単体accelerator性能ではなくnetwork/interconnect/storage/software orchestrationを含むsystem problemであることを裏付ける。\n9. on-device/NPUについてもMLPerf Mobile v6.0がLLM benchmarkを導入し、CPU executionと特定supported NPU accelerationを区別している。「NPU搭載端末」だけでは任意LLMがNPU実行可能とは言えない。"
    },
    {
      "heading": "LIMITS / COUNTEREVIDENCE / FAILURE ENVELOPE",
      "text": "- vendor theoretical peak FLOPS/TOPSをapplication throughputへ変換しない。\n- nominal HBM合計がmodel artifactより大きくても、KV cache/workspace/fragmentation/replicationでOOMし得る。\n- modelがfitしてもTTFT/TPOT/p95-p99 latency、throughput、queue waitがSLAを満たすとは限らない。\n- quantizationでmemoryを削減しても、quality regression、unsupported kernels、conversion time、runtime compatibilityが生じ得る。\n- tensor/pipeline parallelismはcommunication overheadとfailure domainを増やす。\n- multi-node scale-outはnetwork topology、collectives、host memory、storage/model loadingを新たなbottleneckにする。\n- benchmark winnerを別model、別precision、別software revision、別prompt/output distributionへ一般化しない。\n- GPU/NPU/TPU availabilityはregion、quota、instance/device generation、cloud capacity、driver/compiler/runtime supportに依存する。\n- thermal/power/cooling constraintsはon-prem/edgeでcapacityを制約する。TDPの単純合算をfacility designの代替にしない。\n- accelerator partitioning/virtualizationがあることと、security/performance isolationが要件を満たすことは別claim。"
    },
    {
      "heading": "AVAILABILITY_GATES",
      "text": "exact accelerator SKU/form factor and HBM capacity; cloud region/zone/quota/reservation/stock; driver+firmware+CUDA/ROCm/Neuron/TPU runtime/compiler revision; framework/model-server version and supported kernels/operators; model license/weights format/precision/quantization path; interconnect topology and multi-node network; Kubernetes device plugin/operator/DRA/MIG profile; on-device NPUのSoC+OS+SDK/runtime+supported model/operator; owned infrastructureのpower/cooling/rack/network constraints."
    },
    {
      "heading": "IMPLEMENTATION_STATE",
      "text": "Core候補 `ACCELERATOR CAPACITY CONTRACT`: workload_id/version; model_id + weights hash + tokenizer/processor; runtime/server version/commit + container digest; precision/quantization + conversion artifact hash; accelerator vendor/SKU/count + HBM per device; driver/firmware/compiler/runtime; topology/interconnect/network; tensor/pipeline/data/expert parallelism + replicas; max context/input/output distribution; concurrency/batch/scheduler/KV-cache policy; measured idle/model-loaded/steady/peak memory; TTFT/TPOT/tokens-s/requests-s/p50-p95-p99 latency; OOM/rejection/queue rate; power/energy where measurable; hourly/capital cost + utilization; failover/restart/model-load time; benchmark fixture/hash + acceptance threshold; capacity headroom + rollback configuration."
    },
    {
      "heading": "BENCHMARK_OR_EVAL",
      "text": "後続Work用SAFE TEST_PROFILE（未実施）: A) fixed weights/precisionでcold load・steady memory・max safe context/concurrency。B) concurrency 1→NでTTFT/TPOT/p50/p95/p99/queue time。C) offline/batchとinteractiveを分離しtokens/s・requests/s。D) short/median/long input × short/long outputでKV-cache pressureとOOM/rejection。E) 1 accelerator、tensor-parallel、multi-nodeでscaling efficiency。F) FP/BF16 baseline対quantized artifactでmemory・latency・throughput・task quality。G) full device対MIG/virtualized profileでlatency variance/isolation。H) synthetic worker/node lossでrecovery time/request loss。I) accepted-SLA構成だけでcost/request、cost/1M tokens、energy/request。J) MLPerf等と同一hardware familyでもtenant fixtureとの差を記録。"
    },
    {
      "heading": "SECURITY / PRIVACY / AUDIT / COST / OPERABILITY",
      "text": "Security: shared accelerator/partitioning、device plugin、driver/firmware、container/model artifact supply chainを監査対象とする。accelerator memory isolationを名称だけで保証扱いしない。\nPrivacy: model/input/KV/cache/telemetryがどのmemory/storage/logへ残るかをruntimeごとに確認する。\nAudit: hardware SKUだけでなくdriver/runtime/container/model/quantization/topologyまでversion pinしないとcapacity resultを再現できない。\nCost: purchase/instance単価だけでなくutilization、idle、reservation、rebuild/re-quantization、engineering、power/cooling、networkを含める。SLA未達の安価構成を「低コスト」と評価しない。\nOperability: quota exhaustion、GPU failure、driver upgrade、kernel/runtime regression、model reload時間、capacity fragmentation、autoscaling warm-upを運用試験へ含める。"
    },
    {
      "heading": "SAFE_WORDING_FOR_FUTURE_BOOK",
      "text": "「GPU/NPU capacityは、acceleratorの公称性能やVRAMだけでは決まらない。モデルartifact、precision/quantization、contextとconcurrency、KV cache、runtime、parallelism、interconnect、SLAを固定した実測単位で設計する。公開benchmarkはhardware/software構成を比較する重要な入口だが、別workloadの性能保証ではない。GPU・TPU・NPU・専用ASICは、対応software stackとavailability gateを含むdeployment artifactとして比較する。」"
    },
    {
      "heading": "UPDATE_WORK_HANDOFF",
      "text": "1. EA-087 `PRIVATE_AI_DEPLOYMENT_CONTRACT` のaccelerator–driver–runtime–model compatibility matrixを本EAの `ACCELERATOR CAPACITY CONTRACT` へ接続する。\n2. 公開後更新時にNVIDIA/AMD/Intel GPU、Google TPU、AWS Inferentia/Trainium、主要on-device NPUのcurrent SKU/spec/runtime/region matrixを再pinする。\n3. candidate hardwareをparameter countで選ばず、対象model/workloadのmemory estimator→single-device smoke→load curve→parallel scaling→failure/cost testの順で採否判断する。\n4. MLPerf Inferenceは公開後時点の最新roundを再確認し、同一model/scenarioに近いsubmissionだけを比較入口にする。\n5. 後続のFinOps/energy/edge隣接項目には本Packetのmeasured utilization・power・cost artifactを再利用し、claim relationは別に保持する。"
    },
    {
      "heading": "OPEN_UNKNOWN",
      "text": "2026-09-16時点の各cloud/provider全accelerator SKUのregion/quota/on-demand/reservation availability matrix; NVIDIA GPU Operator、vLLM、Neuron SDK、TPU runtime等の本番採用時exact release/tag/commit compatibility matrix; AMD/Intelを含むcurrent cross-vendor同一tenant fixture比較; specific enterprise workloadでのKV-cache/parallelism/quantization最適点; on-device NPUで任意model/operatorを実行できる範囲。これらは実測または公開後再確認なしに閉じない。"
    },
    {
      "heading": "STOP_CONDITION",
      "text": "capacity planningのCore contract、公開一次仕様、author-maintained implementation、独立benchmark、failure envelope、安全な後続TEST_PROFILEまで揃った。製品横断SKU価格表を無限採集せず、current availability/pricing/runtime compatibilityとtenant workload実測は公開後Workの再pin/実機段階へ残す。\n\n---"
    }
  ],
  "human_url": "/tools/ea-088-gpu-npu-capacity-planning",
  "markdown_url": "/agent/article.md?path=%2Ftools%2Fea-088-gpu-npu-capacity-planning",
  "json_url": "/agent/article.json?path=%2Ftools%2Fea-088-gpu-npu-capacity-planning"
}