{
  "format": "agent-article-v0",
  "article_id": "EA-009",
  "canonical_url": "/tools/ea-009-llm",
  "page_type": "reference",
  "claim_ids": [],
  "evidence_ids": [
    "EVD-EA-009-URL-E5AA42A7B612",
    "EVD-EA-009-URL-332F06032935",
    "EVD-EA-009-URL-B9174393A70D",
    "EVD-EA-009-URL-D831938B547E",
    "EVD-EA-009-URL-959000DB1D53",
    "EVD-EA-009-URL-8415671B7E0F",
    "EVD-EA-009-URL-CB4C66EB61D6",
    "EVD-EA-009-URL-913CBB13B2C9",
    "EVD-EA-009-URL-721409A86EF4",
    "EVD-EA-009-URL-640632B09475",
    "EA-009-CURRENT-APPROVAL",
    "EA-009-CURRENT-BODY-READBACK"
  ],
  "unknown_ids": [
    "EA-009-UNKNOWN-01",
    "EA-009-UNKNOWN-02"
  ],
  "operational_object_ids": [],
  "provider": [],
  "models": [],
  "plans": [],
  "surfaces": [],
  "regions": [],
  "current_status": "CURRENT",
  "observation_status": "SPEC_ONLY",
  "freshness": "CURRENT",
  "last_verified": "2026-10-11",
  "supersedes": [],
  "superseded_by": [],
  "provenance": [
    "V21-CANDIDATE-EA009-BASE-20261011-C2",
    "SRC-EA-009-URL-E5AA42A7B612",
    "SRC-EA-009-URL-332F06032935",
    "SRC-EA-009-URL-B9174393A70D",
    "SRC-EA-009-URL-D831938B547E",
    "SRC-EA-009-URL-959000DB1D53",
    "SRC-EA-009-URL-8415671B7E0F",
    "SRC-EA-009-URL-CB4C66EB61D6",
    "SRC-EA-009-URL-913CBB13B2C9",
    "SRC-EA-009-URL-721409A86EF4",
    "SRC-EA-009-URL-640632B09475",
    "BODY-SHA256:c8588f479fe73e6ee22d8addad49d6f173243371855e81b45d0937600c68b7c8"
  ],
  "article_sources": [
    {
      "id": "EVD-EA-009-URL-E5AA42A7B612",
      "evidence_class": "RESEARCH",
      "title": "| E09-01 | author-maintained OSS | llama.cpp README / ggml-org |",
      "href": "https://github.com/ggml-org/llama.cpp/blob/master/README.md",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-E5AA42A7B612"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-01 | author-maintained OSS | llama.cpp README / ggml-org | https://github.com/ggml-org/llama.cpp/blob/master/README.md"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-332F06032935",
      "evidence_class": "RESEARCH",
      "title": "| E09-02 | author-maintained release | llama.cpp Releases / ggml-org |",
      "href": "https://github.com/ggml-org/llama.cpp/releases",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-332F06032935"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-02 | author-maintained release | llama.cpp Releases / ggml-org | https://github.com/ggml-org/llama.cpp/releases"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-B9174393A70D",
      "evidence_class": "RESEARCH",
      "title": "| E09-03 | author-maintained runtime docs | vLLM documentation / vLLM project |",
      "href": "https://docs.vllm.ai/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-B9174393A70D"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-03 | author-maintained runtime docs | vLLM documentation / vLLM project | https://docs.vllm.ai/"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-D831938B547E",
      "evidence_class": "RESEARCH",
      "title": "| E09-04 | author-maintained OSS | SGLang / sgl-project |",
      "href": "https://github.com/sgl-project/sglang",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-D831938B547E"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-04 | author-maintained OSS | SGLang / sgl-project | https://github.com/sgl-project/sglang"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-959000DB1D53",
      "evidence_class": "RESEARCH",
      "title": "| E09-05 | author-maintained release | SGLang Releases / sgl-project |",
      "href": "https://github.com/sgl-project/sglang/releases",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-959000DB1D53"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-05 | author-maintained release | SGLang Releases / sgl-project | https://github.com/sgl-project/sglang/releases"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-8415671B7E0F",
      "evidence_class": "RESEARCH",
      "title": "| E09-06 | official product/runtime docs | Text Generation Inference (TGI) / Hugging Face |",
      "href": "https://huggingface.co/docs/inference-endpoints/engines/tgi",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-8415671B7E0F"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-06 | official product/runtime docs | Text Generation Inference (TGI) / Hugging Face | https://huggingface.co/docs/inference-endpoints/engines/tgi"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-CB4C66EB61D6",
      "evidence_class": "RESEARCH",
      "title": "| E09-07 | open standard definition | The Open Source AI Definition 1.0 / Open Source Initiative |",
      "href": "https://opensource.org/ai/open-source-ai-definition",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-CB4C66EB61D6"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-07 | open standard definition | The Open Source AI Definition 1.0 / Open Source Initiative | https://opensource.org/ai/open-source-ai-definition"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-913CBB13B2C9",
      "evidence_class": "RESEARCH",
      "title": "| E09-08 | official framework security docs | Loading models / Hugging Face Transformers |",
      "href": "https://huggingface.co/docs/transformers/en/models",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-913CBB13B2C9"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-08 | official framework security docs | Loading models / Hugging Face Transformers | https://huggingface.co/docs/transformers/en/models"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-721409A86EF4",
      "evidence_class": "RESEARCH",
      "title": "| E09-09 | benchmark specification | Edge Agentic Inference Benchmark for MLPerf Inference v6.1 / MLCommons |",
      "href": "https://mlcommons.org/2026/07/mlperf-inference-v61-edge-agentic/",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-721409A86EF4"
      ],
      "supporting_locators": [
        "EA-009 ローカルLLM／セルフホスト編",
        "| E09-09 | benchmark specification | Edge Agentic Inference Benchmark for MLPerf Inference v6.1 / MLCommons | https://mlcommons.org/2026/07/mlperf-inference-v61-edge-agentic/"
      ],
      "scope": "EA-009 ローカルLLM／セルフホスト編",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-009-URL-640632B09475",
      "evidence_class": "RESEARCH",
      "title": "https://huggingface.co/docs/text-generation-inference/index",
      "href": "https://huggingface.co/docs/text-generation-inference/index",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-009-URL-640632B09475"
      ],
      "supporting_locators": [
        "Current primary evidence",
        "- https://huggingface.co/docs/text-generation-inference/index"
      ],
      "scope": "Current primary evidence",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EA-009-CURRENT-APPROVAL",
      "evidence_class": "RESEARCH",
      "title": "現行本文の監査・revision対応",
      "supports": [
        "現行台帳のAUDIT_PASSと候補・revision・本文ハッシュの対応を確認しています。"
      ],
      "limit": "監査状態と来歴の記録です。外部根拠の最新性や本文の全主張を追加保証しません。",
      "origin_class": "AUDITED_CANONICAL_RESEARCH_PACKET",
      "source_or_experiment_ids": [
        "V21-CANDIDATE-EA009-BASE-20261011-C2"
      ],
      "supporting_locators": [
        "1KqhJtiSb2S84rWmeYVwKcT_76dBwZ_B0ZZuAWQqH0rQ",
        "EA_REVISION_LEDGER!A391:U391",
        "EA-009-V21-LEGACY-MIGRATION-20261011-R1"
      ],
      "provenance_status": "CURRENT_CANONICAL_APPROVAL_READBACK"
    },
    {
      "id": "EA-009-CURRENT-BODY-READBACK",
      "evidence_class": "IMPL",
      "title": "現行本文の収録照合",
      "supports": [
        "公開生成入力の本文SHA-256が、監査済み候補に結び付いたハッシュと一致しています。"
      ],
      "limit": "本文の同一性の検査です。外部システムの挙動や全主張の真偽を実証したものではありません。",
      "checked_at": "2026-10-12",
      "origin_class": "IMPLEMENTATION_READBACK",
      "source_or_experiment_ids": [
        "BODY-SHA256:c8588f479fe73e6ee22d8addad49d6f173243371855e81b45d0937600c68b7c8"
      ],
      "supporting_locators": [
        "content/staging/EA-009.md",
        "c8588f479fe73e6ee22d8addad49d6f173243371855e81b45d0937600c68b7c8",
        "EA-009-V21-LEGACY-MIGRATION-20261011-R1"
      ],
      "provenance_status": "CURRENT_APPROVED_BODY_HASH_MATCH"
    }
  ],
  "research_sections": [
    {
      "heading": "EA-009 ローカルLLM／セルフホスト編",
      "text": "### WHY_THIS_MATTERS_TO_UPDATE\nローカルLLM／セルフホストは「モデルを自社GPUで動かす」一語では運用境界を表現できない。実際にはmodel artifactとlicense、tokenizer/chat template、quantization/packaging、inference runtime、serving API、hardware/driver、network/egress、認証・ログ、upgrade/rollbackが独立して変化する。2026年にはllama.cpp、vLLM、SGLang等が並行して発展する一方、Hugging Face TGIはmaintenance modeへ移行しており、runtime選択そのものがPRODUCT_VOLATILEである。公開後版では「self-hosted = private/offline/open source」という誤った短絡を解き、再現可能なdeployment manifestと受入試験を中心に整理する必要がある。\n\n\n### SCOPE_ROUTING\nEnterprise AI Core / implementation / runtime & serving / non-LLM infrastructure / security & privacy / supply chain / benchmark / cost & operability / Edge AI cross-reference\n\n\n### CLAIM_OR_RESEARCH_QUESTION\n1. self-hosted、local、offline、air-gapped、private、open sourceは同義か。\n2. 同じmodel weightsでもquantization、tokenizer/chat template、runtime、driver/hardwareが変われば実効能力・品質・性能は変わるか。\n3. OpenAI-compatible APIを提供するruntime同士をsemantic-equivalentと扱ってよいか。\n4. model artifactのprovenance/licenseとruntime/container/dependencyのsupply chainを別々に管理すべきか。\n5. 量子化や高速化を採用する際、latency/throughputだけでなくtask qualityをacceptance gateにすべきか。\n6. self-hostの費用をAPI token単価との比較だけで評価できるか。\n\n\n### EVIDENCE TABLE\n| ID | SOURCE_CLASS | exact title / author-org | canonical URL | publication/update/access | version / locator | LABEL | RELATION |\n|---|---|---|---|---|---|---|---|\n| E09-01 | author-maintained OSS | llama.cpp README / ggml-org | https://github.com/ggml-org/llama.cpp/blob/master/README.md | accessed 2026-09-11 | local/cloud inference; `llama-server`; broad hardware/backend support | OSS_PRACTICE / PRODUCT_VOLATILE | support |\n| E09-02 | author-maintained release | llama.cpp Releases / ggml-org | https://github.com/ggml-org/llama.cpp/releases | accessed 2026-09-11 | current release surface changes rapidly; publication-time pin required | OSS_PRACTICE / PRODUCT_VOLATILE | support, temporal limit |\n| E09-03 | author-maintained runtime docs | vLLM documentation / vLLM project | https://docs.vllm.ai/ | accessed 2026-09-11 | OpenAI-compatible serving, batching/caching/parallelism/quantization features; exact current stable release to re-pin at publication | OSS_PRACTICE / PRODUCT_VOLATILE | support, limit |\n| E09-04 | author-maintained OSS | SGLang / sgl-project | https://github.com/sgl-project/sglang | accessed 2026-09-11 | high-performance LLM/VLM serving; RadixAttention, batching, parallelism, quantization, structured output | OSS_PRACTICE / PRODUCT_VOLATILE | support |\n| E09-05 | author-maintained release | SGLang Releases / sgl-project | https://github.com/sgl-project/sglang/releases | 2026-09-05 release / accessed 2026-09-11 | v0.5.19, commit 0bcd822 | OSS_PRACTICE / PRODUCT_VOLATILE | support, temporal |\n| E09-06 | official product/runtime docs | Text Generation Inference (TGI) / Hugging Face | https://huggingface.co/docs/inference-endpoints/engines/tgi | maintenance mode since 2025-12-11 / accessed 2026-09-11 | recommends vLLM/SGLang alternatives; migration requires new endpoint | VERIFIED_SPEC / PRODUCT_VOLATILE | counter, lifecycle |\n| E09-07 | open standard definition | The Open Source AI Definition 1.0 / Open Source Initiative | https://opensource.org/ai/open-source-ai-definition | stable 1.0 / accessed 2026-09-11 | use/study/modify/share plus preferred form including training-data information, parameters, model/inference code | VERIFIED_SPEC | boundary, counter |\n| E09-08 | official framework security docs | Loading models / Hugging Face Transformers | https://huggingface.co/docs/transformers/en/models | accessed 2026-09-11 | `trust_remote_code=True`; pin specific revision/commit | VERIFIED_SPEC / OSS_PRACTICE | security, supply-chain limit |\n| E09-09 | benchmark specification | Edge Agentic Inference Benchmark for MLPerf Inference v6.1 / MLCommons | https://mlcommons.org/2026/07/mlperf-inference-v61-edge-agentic/ | 2026-07 / accessed 2026-09-11 | Qwen3.6-27B Q4_K_M GGUF reference; alternate quantization allowed only if accuracy threshold met | RESEARCH / VERIFIED_SPEC | benchmark, counter |\n\n\n### FINDINGS\n1. **self-hostedは単一属性ではない。** 少なくとも次の8面へ分解する。\n   - A MODEL ARTIFACT / LEGAL: weights、config、tokenizer、chat template、model card、license、revision/hash、provenance。\n   - B TRANSFORMATION / PACKAGING: quantization、GGUF等のartifact format、converter/version/settings、adapter/LoRA。\n   - C INFERENCE RUNTIME: llama.cpp、vLLM、SGLang等。runtime version/commit、supported architecture/operator、backendを固定する。\n   - D SERVING / API: native/OpenAI-compatible endpoint、streaming、tool calling、structured output、usage/error semantics。\n   - E COMPUTE / TOPOLOGY: CPU/GPU/NPU、CUDA/ROCm/Metal/Vulkan/SYCL等、driver、VRAM/RAM、interconnect、parallelism。\n   - F DATA / NETWORK / SECURITY: model pull、egress、remote code、telemetry、TLS/auth、logs/secrets、multi-tenancy。\n   - G OPERATIONS / LIFECYCLE: rollout、canary、rollback、cold start、queue/backpressure、health、cache/OOM、upgrade。\n   - H EVALUATION / CAPACITY: qualityとlatency/throughput/resource envelopeを別測定する。\n2. **self-host ≠ private ≠ offline ≠ air-gapped。** model download、license check、container/package pull、telemetry、external tool/API、logging先等が外向き通信を持てばself-hostでも完全offlineではない。反対に、組織管理infra上へ閉じた構成は可能だが、構成と実測で証明する必要がある。\n3. **weights available ≠ Open Source AI。** OSI OSAID 1.0は、use/study/modify/shareの自由に加えて、修正のpreferred formとしてtraining-data information、parameters、model/inference code等を要求する。「重みがダウンロード可能」を自動的にopen sourceと表現しない。model licenseとruntime licenseも別契約として確認する。\n4. **再現性にはmodel名だけでは足りない。** 最低manifestは `model repo/revision/hash | tokenizer | chat template | quantization format + converter/version/params | runtime version/commit | driver/CUDA/ROCm等 | hardware SKU/count/interconnect | launch flags | context length | batching/KV-cache/speculative settings` を記録する。\n5. **量子化は単なる容量削減ではなく受入条件付き変換。** MLPerf Edge Agentic v6.1はQ4_K_M GGUFをreferenceにし、submitterの別quantizationもaccuracy thresholdを満たす場合に認める。したがってQ4/Q8等を「元modelと同じ品質」と仮定せず、業務golden setで品質と性能を同時測定する。\n6. **OpenAI-compatibleはtransport/interface compatibilityであってsemantic equivalenceの保証ではない。** runtimeごとにtool calling、structured output/JSON schema、streaming chunk、usage、error/retry、seed/logprobs、stop、tokenization/chat-template handlingが異なり得る。client移行前にcontract testが必要。\n7. **runtime lifecycleは高頻度に変わる。** Hugging FaceはTGIを2025-12-11からmaintenance modeとし、vLLM/SGLang等への移行を推奨する。辞典では「代表runtime」を恒久推奨として焼き込まず、support stateとcurrent releaseをas-ofで管理する。\n8. **model repositoryもsoftware supply chain。** Transformersはcustom modelで`trust_remote_code=True`を使う場合、repository codeを読み込むためspecific revision/commit pinningを追加security layerとして推奨する。weightsだけでなくcustom code、container image、Python/native dependencies、converterもprovenance対象にする。\n\n\n### LIMITS / COUNTEREVIDENCE / FAILURE ENVELOPE\n- 「ローカルなら機密情報が外へ出ない」は構成確認なしには成立しない。egress、telemetry、remote model/code pull、external tools、central loggingを確認する。\n- 「self-hostedならcompliance済み」ではない。access control、audit、patching、backup、retention、incident responseを自組織が負う範囲が増える。\n- 「open weights = open source」は不可。license/materialsをmodelごとに確認する。\n- 同じmodel family名でもrevision、tokenizer、chat template、quantization、runtime kernelで結果が変わり得る。\n- API互換endpointでもtool/schema/error semanticsまで同一とは限らない。\n- 量子化はVRAM/throughput改善とquality degradationのtradeoffを持ち得る。benchmark値を自社taskへ外挿しない。\n- runtimeがmodel architecture/operatorを未対応ならload failure、wrong output、fallback/degraded pathが起こり得る。\n- VRAM/RAM不足、KV-cache exhaustion、fragmentation、long-context、concurrency surgeでOOM/timeout/preemption/backpressureが発生する。\n- distributed servingはinterconnect/NCCL等の障害、node loss、driver/backend mismatchを追加する。\n- cold start/model loadとsteady-state throughputを混同しない。\n- unauthenticated OpenAI-compatible endpoint、広いnetwork bind、prompt/result log、secret envはself-host特有の安全性を自動的に持たない。\n- upgradeでruntime/model/template behaviorが変わり得るためrollback artifactを保持する。\n\n\n### AVAILABILITY_GATES\n`MODEL_LICENSE / MODEL_REVISION / ARTIFACT_FORMAT / QUANTIZATION / RUNTIME_MODEL_SUPPORT / RUNTIME_VERSION / OS_KERNEL / CPU_GPU_NPU / VRAM_RAM / DRIVER_BACKEND / STORAGE / MODEL_PULL_NETWORK / CONTAINER_PACKAGE_REGISTRY / INTERCONNECT / API_FEATURE_SUPPORT / AUTH_TLS / EGRESS_POLICY / ORG_SECURITY_POLICY` を別gateとして持つ。\n\n\n### IMPLEMENTATION_STATE\n- llama.cpp: active author-maintained OSS。local inferenceとOpenAI-compatible serverを含む。release cadenceが速いためcurrent release/commitは公開直前に再取得する。\n- vLLM: active maintained serving runtime。OpenAI-compatible serving、continuous batching、prefix caching、parallelism、quantization等を持つ。exact current stable versionは本packetでは固定せず公開時再取得。\n- SGLang: active maintained serving runtime。2026-09-05時点でv0.5.19 / commit 0bcd822をrelease evidenceとして確認。\n- Hugging Face TGI: production serving runtimeとして歴史的に重要だが2025-12-11以降maintenance mode。current recommendationとして固定せずmigration/lifecycle例として扱う。\n- Model artifacts: per-model license/provenanceは個別確認が必要。runtimeがOSSでもmodel自体がOSI Open Source AIを満たすとは限らない。\n\n\n### BENCHMARK_OR_EVAL\n単一のtokens/secランキングを採用しない。最低限、同じmodel artifactと同じworkload定義で次を分ける。\n1. QUALITY: fixed golden set task success、structured output/tool-call validity、domain accuracy、quantized vs baseline regression。\n2. TTFT: time to first token。\n3. INTER-TOKEN / TPOT: steady decode latency。\n4. E2E: p50/p95/p99 request latency。\n5. THROUGHPUT: output tokens/s、requests/s。\n6. LOAD SHAPE: input/output length distribution、context、concurrency、arrival pattern、batch policy。\n7. RESOURCE: VRAM/RAM、KV-cache utilization、GPU/CPU utilization、energy/power where relevant。\n8. FAILURE: OOM、timeout、queue wait、429/backpressure、load failure、worker/node failure。\n9. COLD/WARM: model load/cold startとwarm steady stateを分離。\n10. COST: accepted task当たりhardware/lease/power/ops cost。\nMLPerf等は再現性のある評価設計例として使うが、特定benchmark勝者を自社workloadの最良runtime/modelへ外挿しない。\n\n\n### SECURITY / PRIVACY / AUDIT / COST / OPERABILITY\n- SECURITY: model repo、custom code、converter、runtime、container、native libraryをsupply-chain inventory化。hash/signature/revision/SBOMを可能な範囲で固定し、remote codeは必要性とrevisionをreviewする。endpointは認証/TLS/network bind/egressを明示する。\n- PRIVACY: self-hostをprivacy labelにせずdata-flowとして検証。prompt、retrieved context、output、logs、metrics、crash dump、model/tool外部通信を別途確認する。\n- AUDIT: raw sensitive promptを無制限保存するのではなく、request/trace ID、model artifact hash、runtime version、quantization、config/policy、tool/action、terminal statusを再構成可能にする。必要なpayloadはmasking/retention policy下で扱う。\n- COST: API token priceとの単純比較を避け、hardware購入/lease、power/cooling、idle utilization、storage/network、redundancy、operator labor、upgrade/testing、failure capacityをTCOに含める。\n- OPERABILITY: immutable-ish deployment manifest、golden image/container、health check、capacity alert、queue/backpressure、canary、rollback、artifact cacheを本番受入条件にする。\n\n\n### SAFE_WORDING_FOR_FUTURE_BOOK\n「ローカルLLM／セルフホストは、『モデルを自社GPUで動かす』だけではない。実際の運用境界は、モデル重みとライセンス、tokenizer/chat template、量子化artifact、推論runtime、API server、hardware/driver、network/egress、ログ、認証、更新手順へ分かれる。重みを入手できることはOpen Source AIやoffline動作を自動的に意味しない。またOpenAI互換APIを提供するruntime同士でもtool callingやstructured output等の挙動まで同一とは限らない。自社運用ではartifactとruntimeのversionを固定し、量子化後の品質、latency/throughput、OOM/overload、外向き通信、upgrade/rollbackを実測する必要がある。」\n\n\n### UPDATE_WORK_HANDOFF\n1. EA-009は8面（model/legal, transformation/packaging, runtime, serving/API, compute, data/network/security, operations, evaluation）を章骨格候補にする。\n2. 公開直前にllama.cpp / vLLM / SGLang / TGI等のcurrent release/support stateを再取得する。runtime名を恒久推奨へしない。\n3. `DEPLOYMENT_MANIFEST`を定義する: model repo/revision/hash、license、tokenizer/chat template、quant artifact+converter、runtime version/commit、container/image hash、driver/backend、hardware、launch flags、context/batching/cache/speculation。\n4. `API_CONTRACT_MATRIX`を作る: chat/completions、streaming、tool calling、structured output/JSON schema、stop、usage、errors、seed/logprobs、model list、tokenization/template semantics。\n5. per-model license tableではOpen Source AI / open weights / source-available / proprietary licenseを分離し、OSI OSAID 1.0を定義参照に使う。\n6. TEST_PROFILE-LOCAL-001（未実施）: 必要artifactをpreload後、all egressを遮断してboot→load→inferenceを実施し、DNS/download/telemetry/license/network依存を観測する。\n7. TEST_PROFILE-LOCAL-002（未実施）: 同一fixtureをllama.cpp/vLLM/SGLangのOpenAI-compatible endpointへ投げ、chat/stream/tool/structured output/stop/error/usage/seed/logprobsの差を記録する。\n8. TEST_PROFILE-LOCAL-003（未実施）: base/high-precision artifactと各quantizationを同じgolden setで比較し、quality regression、TTFT/TPOT/throughput、VRAM/RAMを同時計測する。\n9. TEST_PROFILE-LOCAL-004（未実施）: context/concurrencyを段階的に増やしKV-cache/OOM/queue saturationまで負荷を上げ、backpressure、timeout、health、recoveryを確認する。\n10. TEST_PROFILE-LOCAL-005（未実施）: model/runtimeの片方だけをupgradeしregressionを測定、rollbackできることを確認する。\n11. TEST_PROFILE-LOCAL-006（未実施）: model/runtime/container/dependencyのhash/revision/signature/SBOM、remote-code無効またはreview済み状態を確認する。\n12. EA-010「Edge AI／完全オフライン運用編」、EA-011 Enterprise AIベンチマーク実験室、EA-018 Edge AI系統、EA-046 SLM、EA-085 hardware-aware AI等へcross-referenceし、EA-009はserver/local inferenceの運用境界、EA-010は完全offline acceptanceへ編集分離する。\n\n\n### OPEN_UNKNOWN / STOP_CONDITION"
    },
    {
      "heading": "OPEN_UNKNOWN",
      "text": "- 2026-09-11時点のvLLM exact latest stable releaseと全model/backend compatibility matrix。\n- llama.cpp / SGLang / vLLM各serverのcurrent built-in auth/mTLS機能と推奨production hardeningの完全比較。\n- 主要open-weight modelのlicense/training-data disclosureをOSI OSAID 1.0基準で判定したcurrent matrix。\n- independent production-scaleで同一model/workloadをruntime横断比較した2026 evidence。\n- multi-tenant isolation/security guaranteeのruntime別比較。\n- preload済みair-gapped構成でのtelemetry/egress完全実測。\n- hardware amortization、power、operator laborまで含めたorganization別TCO分岐点。"
    },
    {
      "heading": "STOP_CONDITION",
      "text": "self-host/local/offline/open-sourceの非同一性、8面の運用taxonomy、runtime lifecycle変動、量子化acceptance、API semantic compatibility、supply-chain risk、capacity/evaluation、deployment manifest/test profileまで一次資料・author-maintained OSS・benchmark specificationから確立した。全model×runtime×hardware compatibilityやTCOを今固定するとPRODUCT_VOLATILEかつEA-010/011/018/046/085と重複するため、EA-009は`BOUNDED_CLOSE`。"
    },
    {
      "heading": "RUN STATUS",
      "text": "EA-009: BOUNDED_CLOSE"
    },
    {
      "heading": "Currentness findings",
      "text": "- Runtime choice remains strongly PRODUCT_VOLATILE.\n- Hugging Face continues to mark Text Generation Inference (TGI) as maintenance mode and recommends engines such as vLLM or SGLang for Inference Endpoints.\n- llama.cpp continues high-frequency releases; the current release stream includes v0.6.0 and rapid backend/server changes, reinforcing the need to pin exact release/commit and hardware/backend when reproducing behavior.\n- The canonical packet's separation of model artifact/license, tokenizer/chat template, quantization/packaging, inference runtime, serving API, hardware/driver, network/egress, auth/logging, and upgrade/rollback remains current.\n- “Self-hosted” still does not imply offline, private, open-source, or reproducible without those contracts."
    },
    {
      "heading": "Current primary evidence",
      "text": "- https://huggingface.co/docs/text-generation-inference/index\n- https://huggingface.co/docs/inference-endpoints/engines/tgi\n- https://github.com/ggml-org/llama.cpp/releases"
    },
    {
      "heading": "Counterevidence / failure envelope retained",
      "text": "- Runtime benchmarks do not transfer across model, quantization, backend, driver, hardware, batching, context length, or concurrency without reproduction.\n- Open-weight/model-download availability does not establish license compatibility or artifact integrity.\n- A locally served HTTP endpoint may still have external model, telemetry, package, identity, update, or tool dependencies.\n- Maintenance mode is not the same as immediate unusability; existing supported deployments need migration testing, not blind replacement.\n- No live GPU benchmark or production migration was performed in this rescue."
    },
    {
      "heading": "SAFE_WORDING",
      "text": "For self-hosted LLMs, pin the model artifact and license, tokenizer/template, quantization, runtime release/commit, serving contract, accelerator/driver, network/egress, authentication/logging, and rollback artifact. Treat benchmark and compatibility claims as configuration-specific."
    },
    {
      "heading": "OPEN_UNKNOWN",
      "text": "- workload-specific vLLM/SGLang/llama.cpp performance and correctness on target hardware;\n- current backend regressions for specific accelerators and model architectures;\n- organization-specific packaging/signature and offline mirror controls;\n- migration behavior from existing TGI deployments."
    },
    {
      "heading": "Research Gate decision",
      "text": "`PASS_FOR_BOUNDED_BASE_REVISION`."
    },
    {
      "heading": "Candidate implementation overlay",
      "text": "- target: `content/staging/EA-009.md`\n- base repository commit: `97d3d104ae0b16a95e7dffddc3a8cb327795a796`\n- path precondition: `ABSENT`\n- candidate: `V21-CANDIDATE-EA009-BASE-20261011-C1`\n- READY: NOT_CREATED\n- AUDIT_PASS: NOT_CLAIMED\n- GIT_WRITE: NOT_PERFORMED"
    }
  ],
  "human_url": "/tools/ea-009-llm",
  "markdown_url": "/agent/article.md?path=%2Ftools%2Fea-009-llm",
  "json_url": "/agent/article.json?path=%2Ftools%2Fea-009-llm"
}