{
  "format": "agent-article-v0",
  "article_id": "EA-089",
  "canonical_url": "/tools/ea-089-local-ai-cluster",
  "page_type": "reference",
  "claim_ids": [],
  "evidence_ids": [
    "EVD-EA-089-URL-9E6EB9E3A549",
    "EVD-EA-089-URL-4C46467F6778",
    "EVD-EA-089-URL-3B76C103B39D",
    "EVD-EA-089-URL-9001F26B6D5C",
    "EVD-EA-089-URL-A14700A9414B",
    "EVD-EA-089-URL-7762AA271822",
    "EA-089-R11-RESEARCH",
    "EA-089-R11-WEB-READBACK"
  ],
  "unknown_ids": [
    "EA-089-UNKNOWN-01",
    "EA-089-UNKNOWN-CURRENTNESS"
  ],
  "operational_object_ids": [],
  "provider": [],
  "models": [],
  "plans": [],
  "surfaces": [],
  "regions": [],
  "current_status": "UNKNOWN",
  "observation_status": "SPEC_ONLY",
  "freshness": "UNKNOWN",
  "last_verified": null,
  "supersedes": [],
  "superseded_by": [],
  "provenance": [
    "COMM-081-EA0089",
    "SRC-EA-089-URL-9E6EB9E3A549",
    "SRC-EA-089-URL-4C46467F6778",
    "SRC-EA-089-URL-3B76C103B39D",
    "SRC-EA-089-URL-9001F26B6D5C",
    "SRC-EA-089-URL-A14700A9414B",
    "SRC-EA-089-URL-7762AA271822",
    "WEBBODY-EA0089-R8-69A06A5C"
  ],
  "article_sources": [
    {
      "id": "EVD-EA-089-URL-9E6EB9E3A549",
      "evidence_class": "RESEARCH",
      "title": "author/org: exo labs / exo-explore",
      "href": "https://github.com/exo-explore/exo",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-089-URL-9E6EB9E3A549"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "canonical URL: https://github.com/exo-explore/exo"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-089-URL-4C46467F6778",
      "evidence_class": "RESEARCH",
      "title": "author/org: ggml-org",
      "href": "https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-089-URL-4C46467F6778"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "canonical URL: https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-089-URL-3B76C103B39D",
      "evidence_class": "RESEARCH",
      "title": "author/org: ggml-org/llama.cpp issue reporter",
      "href": "https://github.com/ggml-org/llama.cpp/issues/8832",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-089-URL-3B76C103B39D"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "canonical URL: https://github.com/ggml-org/llama.cpp/issues/8832"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-089-URL-9001F26B6D5C",
      "evidence_class": "RESEARCH",
      "title": "author/org: ggml-org/llama.cpp issue reporter",
      "href": "https://github.com/ggml-org/llama.cpp/issues/15055",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-089-URL-9001F26B6D5C"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "canonical URL: https://github.com/ggml-org/llama.cpp/issues/15055"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-089-URL-A14700A9414B",
      "evidence_class": "RESEARCH",
      "title": "author/org: SchedMD",
      "href": "https://slurm.schedmd.com/gres.html",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-089-URL-A14700A9414B"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "canonical URL: https://slurm.schedmd.com/gres.html"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EVD-EA-089-URL-7762AA271822",
      "evidence_class": "RESEARCH",
      "title": "author/org: Ray Project / Anyscale community",
      "href": "https://github.com/ray-project/ray/blob/master/doc/source/cluster/key-concepts.rst",
      "supports": [],
      "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。",
      "origin_class": "SOURCE_PACKET_REFERENCE",
      "source_or_experiment_ids": [
        "SRC-EA-089-URL-7762AA271822"
      ],
      "supporting_locators": [
        "EVIDENCE TABLE",
        "canonical URL: https://github.com/ray-project/ray/blob/master/doc/source/cluster/key-concepts.rst"
      ],
      "scope": "EVIDENCE TABLE",
      "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED"
    },
    {
      "id": "EA-089-R11-RESEARCH",
      "evidence_class": "RESEARCH",
      "title": "EA-089 監査済みresearch / canonical revision binding",
      "supports": [
        "- 「束ねる」には少なくとも二種類ある。(A) 1 inferenceを複数device/nodeへshardするmodel-parallel/distributed inference、(B) independent replicas/jobsを複数nodeへ配置するthroughput/queue cluster。両者を同じcapacity claimにしない。\n- exoはconsumer/Apple Siliconを含むlocal devicesを自動発見し、topology-aware parallelism、MLX distributed、複数API互換、offline modeを実装している。Local AI Clusterが研究玩具だけでなく実装可能なclassであることは確立。\n- Slurm GRESはGPUを明示的resourceとしてschedule/accountできる。したがって小規模clusterでも「空いているGPUを何となく使う」から、allocation/accountingを伴うshared resourceへ移行できる。\n- llama.cpp RPCはremote device offloadを実装するが、README自身がproof-of-concept・fragile・insecureと明記。local clusterという概念の存在証拠にはなるが、production securityの根拠にはしない。\n- network/interconnectがcritical pathになるmodel parallelismでは、総memory増加とlatency/throughput改善は別。heterogeneous slow nodeを足すとmodel fitは改善してもsingle-request latencyを悪化させ得る。"
      ],
      "limit": "canonical監査・revision bindingの存在を示す。外部ソースcurrentnessが本文またはcontrol-planeで確定していない場合は、currentnessをUNKNOWNのまま保持する。",
      "origin_class": "AUDITED_CANONICAL_RESEARCH_PACKET",
      "source_or_experiment_ids": [
        "COMM-081-EA0089"
      ],
      "supporting_locators": [
        "COMM-081-EA0089",
        "EA-089-R5-LEGACY-MIGRATION-20261008-R1",
        "content/staging/EA-089.md"
      ],
      "scope": "監査済みEA本文・canonical revision・既存Source Graph relationの範囲。欠損currentnessは推測で閉じない。",
      "provenance_status": "AUDIT_PASS_CANONICAL_BINDING"
    },
    {
      "id": "EA-089-R11-WEB-READBACK",
      "evidence_class": "IMPL",
      "title": "EA-089 Web body identity / canonical readback",
      "supports": [
        "canonical staging bodyのSHA-256一致とR9 canonical ledger readbackをProduction projectionの入力として検証する。"
      ],
      "limit": "body identity/readbackは本文claim自体の真偽や外部ソースの最新性を追加保証しない。",
      "checked_at": null,
      "origin_class": "IMPLEMENTATION_READBACK",
      "source_or_experiment_ids": [
        "WEBBODY-EA0089-R8-69A06A5C"
      ],
      "supporting_locators": [
        "WEBBODY-EA0089-R8-69A06A5C",
        "content/staging/EA-089.md",
        "EA-089-R5-LEGACY-MIGRATION-20261008-R1"
      ],
      "scope": "R9/R10 canonical readback後のProduction生成時body identity。",
      "provenance_status": "CANONICAL_READBACK_BODY_HASH_MATCH"
    }
  ],
  "research_sections": [
    {
      "heading": "WHY_THIS_MATTERS_TO_UPDATE",
      "text": "単体端末では収まらないモデルや複数利用者の推論を、既存のPC/Mac/GPU資産を束ねて成立させられる可能性がある。一方、総RAM/VRAMの単純加算は性能・可用性・安全性を保証せず、network topology、runtime、scheduler、heterogeneous device、failure domainまで含む運用設計が必要。"
    },
    {
      "heading": "SCOPE_ROUTING",
      "text": "Enterprise AI Core / implementation / non-LLM component / infrastructure / interoperability / benchmark / operability"
    },
    {
      "heading": "CLAIM_OR_RESEARCH_QUESTION",
      "text": "Local AI Clusterはどの層で成立し、単一node推論と比べて何が増え、どの条件で「複数台を束ねる」ことが実益になるか。consumer/office hardwareのheterogeneous clusterをproduction-grade shared inferenceへ外挿できるか。"
    },
    {
      "heading": "EVIDENCE TABLE",
      "text": "1. source class: author-maintained OSS / implementation\nexact title: exo: Run frontier AI locally.\nauthor/org: exo labs / exo-explore\ncanonical URL: https://github.com/exo-explore/exo\npublication/update/access: accessed 2026-09-16\nversion/release/tag/commit: exact current tag/commit not pinned in this run\nsupporting locator: README Features / Quick Start; automatic discovery, topology-aware auto parallel, tensor parallelism, MLX distributed, offline mode, read-only pre-downloaded model directories\nevidence label: OSS_PRACTICE / PRODUCT_VOLATILE\nrelation: support/context\n\n\n2. source class: author-maintained OSS / implementation\nexact title: llama.cpp RPC backend README\nauthor/org: ggml-org\ncanonical URL: https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md\npublication/update/access: accessed 2026-09-16\nversion/release/tag/commit: master; exact commit not pinned\nsupporting locator: Overview / Important; remote ggml devices, one-or-several RPC servers, proof-of-concept, fragile and insecure, do not expose to open network or sensitive environment\nevidence label: OSS_PRACTICE\nrelation: support/limit\n\n\n3. source class: author issue / failure report\nexact title: Bug: RPC inference is drastically slower even on localhost #8832\nauthor/org: ggml-org/llama.cpp issue reporter\ncanonical URL: https://github.com/ggml-org/llama.cpp/issues/8832\npublication/update/access: opened 2024-08-02; accessed 2026-09-16\nversion/release/tag/commit: issue-specific environment\nsupporting locator: reported llama3.1-8B CUDA RPC 1.9 tok/s vs local 25 tok/s on same host\nevidence label: SOURCE_REPORT\nrelation: counter/limit\n\n\n4. source class: author issue / failure report\nexact title: Eval bug: Crash when offloading large models via RPC if model size exceeds ~75% of server RAM #15055\nauthor/org: ggml-org/llama.cpp issue reporter\ncanonical URL: https://github.com/ggml-org/llama.cpp/issues/15055\npublication/update/access: accessed 2026-09-16\nversion/release/tag/commit: issue-specific Apple M3 Ultra environment\nsupporting locator: Problem description; large-model remote offload failure under high memory pressure\nevidence label: SOURCE_REPORT\nrelation: counter/limit\n\n\n5. source class: official scheduler documentation\nexact title: Slurm Workload Manager — Generic Resource (GRES) Scheduling\nauthor/org: SchedMD\ncanonical URL: https://slurm.schedmd.com/gres.html\npublication/update/access: accessed 2026-09-16\nversion/release/tag/commit: current docs; AutoDetect=nvidia noted as added in Slurm 24.11\nsupporting locator: Overview / AutoDetect / Accounting; GPU GRES, explicit requests, GPU memory/utilization accounting, heterogeneous GPU types\nevidence label: VERIFIED_SPEC\nrelation: support/context\n\n\n6. source class: author-maintained framework documentation\nexact title: Ray Cluster Key Concepts\nauthor/org: Ray Project / Anyscale community\ncanonical URL: https://github.com/ray-project/ray/blob/master/doc/source/cluster/key-concepts.rst\npublication/update/access: accessed 2026-09-16\nversion/release/tag/commit: master; exact commit not pinned\nsupporting locator: Head Node / Worker Node; head carries GCS/autoscaler/driver management, workers execute tasks/actors\nevidence label: OSS_PRACTICE\nrelation: context"
    },
    {
      "heading": "FINDINGS",
      "text": "- 「束ねる」には少なくとも二種類ある。(A) 1 inferenceを複数device/nodeへshardするmodel-parallel/distributed inference、(B) independent replicas/jobsを複数nodeへ配置するthroughput/queue cluster。両者を同じcapacity claimにしない。\n- exoはconsumer/Apple Siliconを含むlocal devicesを自動発見し、topology-aware parallelism、MLX distributed、複数API互換、offline modeを実装している。Local AI Clusterが研究玩具だけでなく実装可能なclassであることは確立。\n- Slurm GRESはGPUを明示的resourceとしてschedule/accountできる。したがって小規模clusterでも「空いているGPUを何となく使う」から、allocation/accountingを伴うshared resourceへ移行できる。\n- llama.cpp RPCはremote device offloadを実装するが、README自身がproof-of-concept・fragile・insecureと明記。local clusterという概念の存在証拠にはなるが、production securityの根拠にはしない。\n- network/interconnectがcritical pathになるmodel parallelismでは、総memory増加とlatency/throughput改善は別。heterogeneous slow nodeを足すとmodel fitは改善してもsingle-request latencyを悪化させ得る。"
    },
    {
      "heading": "LIMITS / COUNTEREVIDENCE / FAILURE ENVELOPE",
      "text": "- aggregate RAM/VRAM >= model size は必要条件になり得るが十分条件ではない。runtimeがそのdevice combinationをsupportすること、tensor placement、network bandwidth/latency、workspace/KV headroomが必要。\n- llama.cpp RPC issue #8832の13x slowdownは単一issue/environmentであり全version一般化禁止。ただし「remote offload overheadを無視できない」反証として保持。\n- issue #15055も全deployment一般化禁止。高memory pressure時のremote offload failure envelopeを示すSOURCE_REPORTとしてのみ使用。\n- consumer auto-discoveryは便利だが、production networkでのidentity/authentication/authorizationと同義ではない。\n- node loss時にmodel-parallel request全体が失敗する構成と、replica clusterで別nodeへretry可能な構成はfailure domainが異なる。"
    },
    {
      "heading": "AVAILABILITY_GATES",
      "text": "OS/architecture; accelerator/runtime compatibility; model format; RAM/VRAM per node; interconnect bandwidth/latency; switch/NIC/Thunderbolt/RDMA availability; scheduler/runtime version; model license; LAN segmentation; authentication; storage/NFS/model distribution; power/thermal envelope."
    },
    {
      "heading": "IMPLEMENTATION_STATE",
      "text": "OBSERVED externally only through primary docs/OSS; no tenant live test performed. Practical patterns: exo/MLX distributed for Apple Silicon-oriented cluster; llama.cpp RPC for experimental remote ggml offload; Slurm for explicit GPU job scheduling/accounting; Ray for task/actor cluster orchestration. These are not interchangeable layers."
    },
    {
      "heading": "BENCHMARK_OR_EVAL",
      "text": "Future TEST_PROFILE: fixed model artifact and prompt set; compare 1 node vs 2/4 nodes; model-parallel vs replica/queue mode; homogeneous vs heterogeneous nodes; Ethernet/Wi-Fi/Thunderbolt/RDMA where applicable. Measure model-load time, TTFT, TPOT, tokens/s, requests/s, p50/p95/p99, inter-node traffic, peak memory/node, power, node-loss recovery, cold/warm restart, scheduler wait, output equivalence. Never report only aggregate memory."
    },
    {
      "heading": "SECURITY / PRIVACY / AUDIT / COST / OPERABILITY",
      "text": "Security: RPC/device-discovery plane must not be presumed trusted; isolate management/data planes and authenticate endpoints. Privacy: local execution reduces external data transfer but does not by itself enforce user/tenant isolation. Audit: node/model/runtime/topology/job/request mapping required. Cost: reuse of owned hardware can reduce acquisition cost but power, networking, admin time, low utilization and heterogeneous tail latency must be included. Operability: patch/driver/runtime skew and failed node replacement are primary burdens."
    },
    {
      "heading": "SAFE_WORDING_FOR_FUTURE_BOOK",
      "text": "「複数のPC/Mac/GPUを束ねるLocal AI Clusterは、単体では収まらないモデルの分散推論または複数requestの共有処理を可能にし得る。ただし総RAM/VRAMの加算だけでは性能を予測できず、runtime対応、network topology、scheduler、failure domainを実測する必要がある。consumer向けdistributed runtimeにはexperimental/security制約を持つ実装もあるため、production採用は別途評価する。」"
    },
    {
      "heading": "UPDATE_WORK_HANDOFF",
      "text": "Core候補 `LOCAL_AI_CLUSTER_CONTRACT`: cluster_id; node_id; hardware/accelerator; OS; driver/runtime; model artifact/hash; partition/replica strategy; topology/interconnect; scheduler/runtime version; per-node memory; request placement; benchmark fixture; TTFT/TPOT/throughput; network traffic; failure/recovery; authn/authz; energy/cost; deployment/rollback. EA-088 ACCELERATOR CAPACITY CONTRACTと接続する。"
    },
    {
      "heading": "OPEN_UNKNOWN",
      "text": "exo exact current release/tag/commit and production hardening state; llama.cpp RPC current exact commit/security evolution; cross-platform heterogeneous runtime matrix; small-office RDMA/Thunderbolt operational evidence; independent long-duration production case studies. Stop condition for this run: architecture/implementation/counterevidence/test profileまでboundedに確立。\n\n\n---\n\n---"
    }
  ],
  "human_url": "/tools/ea-089-local-ai-cluster",
  "markdown_url": "/agent/article.md?path=%2Ftools%2Fea-089-local-ai-cluster",
  "json_url": "/agent/article.json?path=%2Ftools%2Fea-089-local-ai-cluster"
}