{ "format": "agent-article-v0", "article_id": "EA-004", "canonical_url": "/tools/ea-004-llm-ai-ocr-reranker-ai-llm-provider-neutral-taxonomy", "page_type": "reference", "claim_ids": [], "evidence_ids": [ "EVD-EA-004-URL-D9A675813802", "EVD-EA-004-URL-657721D9A4F4", "EVD-EA-004-URL-B84B4DEDAA63", "EVD-EA-004-URL-22EE7B973A03", "EVD-EA-004-URL-F4556A8EA396", "EVD-EA-004-URL-246E7181EC16", "EVD-EA-004-URL-8A4426B6E7B6", "EVD-EA-004-URL-8BCFDBBFE900", "EVD-EA-004-URL-24E5B13D37B5", "EVD-EA-004-URL-7B8EC5719183", "EVD-EA-004-URL-898604380DC5", "EVD-EA-004-URL-F843C036A46F", "EVD-EA-004-URL-036F41D19CA8", "EVD-EA-004-URL-A6832C2814E6", "EVD-EA-004-URL-D96CD7BAEC7D", "EVD-EA-004-URL-9DB454907994", "EVD-EA-004-URL-08DC05778074", "EVD-EA-004-URL-3BDEE34C7D3F", "EA-004-CURRENT-APPROVAL", "EA-004-CURRENT-BODY-READBACK" ], "unknown_ids": [ "EA-004-UNKNOWN-01", "EA-004-UNKNOWN-02", "EA-004-UNKNOWN-03" ], "operational_object_ids": [], "provider": [ "Google", "Gemini" ], "models": [], "plans": [], "surfaces": [], "regions": [], "current_status": "CURRENT", "observation_status": "SPEC_ONLY", "freshness": "CURRENT", "last_verified": "2026-10-11", "supersedes": [], "superseded_by": [], "provenance": [ "V21-CANDIDATE-EA004-BASE-20261011-C2", "SRC-EA-004-URL-D9A675813802", "SRC-EA-004-URL-657721D9A4F4", "SRC-EA-004-URL-B84B4DEDAA63", "SRC-EA-004-URL-22EE7B973A03", "SRC-EA-004-URL-F4556A8EA396", "SRC-EA-004-URL-246E7181EC16", "SRC-EA-004-URL-8A4426B6E7B6", "SRC-EA-004-URL-8BCFDBBFE900", "SRC-EA-004-URL-24E5B13D37B5", "SRC-EA-004-URL-7B8EC5719183", "SRC-EA-004-URL-898604380DC5", "SRC-EA-004-URL-F843C036A46F", "SRC-EA-004-URL-036F41D19CA8", "SRC-EA-004-URL-A6832C2814E6", "SRC-EA-004-URL-D96CD7BAEC7D", "SRC-EA-004-URL-9DB454907994", "SRC-EA-004-URL-08DC05778074", "SRC-EA-004-URL-3BDEE34C7D3F", "BODY-SHA256:87446226c7586845406a87532e4324795b32f9eb2bde3b1ebca2648a99be1ee2" ], "article_sources": [ { "id": "EVD-EA-004-URL-D9A675813802", "evidence_class": "RESEARCH", "title": "org: scikit-learn project", "href": "https://scikit-learn.org/stable/user_guide.html", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-D9A675813802" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://scikit-learn.org/stable/user_guide.html" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-657721D9A4F4", "evidence_class": "RESEARCH", "title": "org: tesseract-ocr project", "href": "https://tesseract-ocr.github.io/tessdoc/", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-657721D9A4F4" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://tesseract-ocr.github.io/tessdoc/" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-B84B4DEDAA63", "evidence_class": "RESEARCH", "title": "canonical URLs:", "href": "https://cloud.google.com/document-ai", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-B84B4DEDAA63" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URLs: https://cloud.google.com/document-ai" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-22EE7B973A03", "evidence_class": "RESEARCH", "title": "canonical URLs: https://cloud.google.com/document-ai ;", "href": "https://docs.cloud.google.com/document-ai/docs/layout-parse-chunk", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-22EE7B973A03" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URLs: https://cloud.google.com/document-ai ; https://docs.cloud.google.com/document-ai/docs/layout-parse-chunk" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-F4556A8EA396", "evidence_class": "RESEARCH", "title": "org: Google Cloud", "href": "https://docs.cloud.google.com/vision/docs/labels", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-F4556A8EA396" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.cloud.google.com/vision/docs/labels" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-246E7181EC16", "evidence_class": "RESEARCH", "title": "org: Google Cloud", "href": "https://docs.cloud.google.com/speech-to-text/docs", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-246E7181EC16" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.cloud.google.com/speech-to-text/docs" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-8A4426B6E7B6", "evidence_class": "RESEARCH", "title": "authors: Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever", "href": "https://arxiv.org/abs/2212.04356", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-8A4426B6E7B6" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://arxiv.org/abs/2212.04356" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-8BCFDBBFE900", "evidence_class": "RESEARCH", "title": "org: Google Cloud", "href": "https://docs.cloud.google.com/text-to-speech/docs", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-8BCFDBBFE900" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.cloud.google.com/text-to-speech/docs" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-24E5B13D37B5", "evidence_class": "RESEARCH", "title": "org: Google Cloud", "href": "https://docs.cloud.google.com/translate/docs/translate-text", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-24E5B13D37B5" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.cloud.google.com/translate/docs/translate-text" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-7B8EC5719183", "evidence_class": "RESEARCH", "title": "canonical URLs:", "href": "https://sbert.net/", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-7B8EC5719183" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URLs: https://sbert.net/" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-898604380DC5", "evidence_class": "RESEARCH", "title": "canonical URLs: https://sbert.net/ ;", "href": "https://www.sbert.net/docs/package_reference/cross_encoder/model.html", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-898604380DC5" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URLs: https://sbert.net/ ; https://www.sbert.net/docs/package_reference/cross_encoder/model.html" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-F843C036A46F", "evidence_class": "RESEARCH", "title": "authors: Niklas Muennighoff, Nouamane Tazi, Loïc Magne, Nils Reimers", "href": "https://arxiv.org/abs/2210.07316", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-F843C036A46F" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://arxiv.org/abs/2210.07316" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-036F41D19CA8", "evidence_class": "RESEARCH", "title": "canonical URLs:", "href": "https://onnxruntime.ai/docs/", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-036F41D19CA8" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URLs: https://onnxruntime.ai/docs/" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-A6832C2814E6", "evidence_class": "RESEARCH", "title": "canonical URLs: https://onnxruntime.ai/docs/ ;", "href": "https://onnxruntime.ai/inference", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-A6832C2814E6" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URLs: https://onnxruntime.ai/docs/ ; https://onnxruntime.ai/inference" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-D96CD7BAEC7D", "evidence_class": "RESEARCH", "title": "- scikit-learn 1.9.1 stable documentation, accessed 2026-10-11: maintained production-oriented families still include classification, regression, clustering, dimensionality reducti", "href": "https://scikit-learn.org/stable/", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-D96CD7BAEC7D" ], "supporting_locators": [ "CURRENTNESS_DECISION", "URL: https://scikit-learn.org/stable/" ], "scope": "CURRENTNESS_DECISION", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-9DB454907994", "evidence_class": "RESEARCH", "title": "- Sentence Transformers current documentation, accessed 2026-10-11: the maintained library explicitly distinguishes dense SentenceTransformer, CrossEncoder, SparseEncoder and Multi", "href": "https://sbert.net/docs/migration_guide.html", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-9DB454907994" ], "supporting_locators": [ "CURRENTNESS_DECISION", "URL: https://sbert.net/docs/migration_guide.html" ], "scope": "CURRENTNESS_DECISION", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-08DC05778074", "evidence_class": "RESEARCH", "title": "- ONNX Runtime Execution Providers documentation, accessed 2026-10-11: a common runtime API spans hardware-specific execution providers with different support/maturity. Runtime/dep", "href": "https://onnxruntime.ai/docs/execution-providers/", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-08DC05778074" ], "supporting_locators": [ "CURRENTNESS_DECISION", "URL: https://onnxruntime.ai/docs/execution-providers/" ], "scope": "CURRENTNESS_DECISION", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-004-URL-3BDEE34C7D3F", "evidence_class": "RESEARCH", "title": "- Google Cloud Document AI current release notes / processor docs, accessed 2026-10-11: specialized document/layout processors include both specialized OCR behavior and Gemini-powe", "href": "https://docs.cloud.google.com/document-ai/docs/release-notes", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-004-URL-3BDEE34C7D3F" ], "supporting_locators": [ "CURRENTNESS_DECISION", "URL: https://docs.cloud.google.com/document-ai/docs/release-notes" ], "scope": "CURRENTNESS_DECISION", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EA-004-CURRENT-APPROVAL", "evidence_class": "RESEARCH", "title": "現行本文の監査・revision対応", "supports": [ "現行台帳のAUDIT_PASSと候補・revision・本文ハッシュの対応を確認しています。" ], "limit": "監査状態と来歴の記録です。外部根拠の最新性や本文の全主張を追加保証しません。", "origin_class": "AUDITED_CANONICAL_RESEARCH_PACKET", "source_or_experiment_ids": [ "V21-CANDIDATE-EA004-BASE-20261011-C2" ], "supporting_locators": [ "1KqhJtiSb2S84rWmeYVwKcT_76dBwZ_B0ZZuAWQqH0rQ", "EA_REVISION_LEDGER!A386:U386", "EA-004-V21-LEGACY-MIGRATION-20261011-R1" ], "provenance_status": "CURRENT_CANONICAL_APPROVAL_READBACK" }, { "id": "EA-004-CURRENT-BODY-READBACK", "evidence_class": "IMPL", "title": "現行本文の収録照合", "supports": [ "公開生成入力の本文SHA-256が、監査済み候補に結び付いたハッシュと一致しています。" ], "limit": "本文の同一性の検査です。外部システムの挙動や全主張の真偽を実証したものではありません。", "checked_at": "2026-10-12", "origin_class": "IMPLEMENTATION_READBACK", "source_or_experiment_ids": [ "BODY-SHA256:87446226c7586845406a87532e4324795b32f9eb2bde3b1ebca2648a99be1ee2" ], "supporting_locators": [ "content/staging/EA-004.md", "87446226c7586845406a87532e4324795b32f9eb2bde3b1ebca2648a99be1ee2", "EA-004-V21-LEGACY-MIGRATION-20261011-R1" ], "provenance_status": "CURRENT_APPROVED_BODY_HASH_MATCH" } ], "research_sections": [ { "heading": "WHY_THIS_MATTERS_TO_UPDATE", "text": "台帳EA-004はOCR、画像認識、音声認識・合成、翻訳、埋め込み、reranker、画像・動画生成、分類器等を体系化する構想である。しかし2026年の実装では「非LLM」をmodel architectureの否定として定義すると破綻する。Sentence Transformersのembedding/rerankerはTransformer系を含み、Google Cloud Translationは同一API系統でstandard NMTとTranslation LLMを提供し、Document AI layout parserにはspecialized OCRとGemini-powered parserが併存する。専用AI workloadとLLM architectureは排他的ではない。公開後辞典では「自由文を中心に生成する汎用LLMと異なる、task-specific operational contractを持つAI component」という運用分類としてEA-004を定義し、各componentのinput/output schema、quality metric、failure mode、latency/cost/privacy/deploymentを比較する方が安全である。" }, { "heading": "SCOPE_ROUTING", "text": "Enterprise AI Core: component selection、evaluation contract、fallback、observability、data boundary、cost/latency、hardware/runtime。\nNon-LLM component: OCR/document extraction、vision recognition、ASR、TTS、translation、embedding、reranker、classification/regression/clustering/anomaly detection、media generation。\nImplementation: scikit-learn、Tesseract、Sentence Transformers、ONNX Runtime等のOSS/runtime、およびmanaged specialized AI APIs。\nProvider mapping: Google Cloud Document AI/Vision/Speech/Translation等はcurrent managed-service sampleとして使用。特定providerをEA-004の標準実装としない。\nBenchmark: task-specific evaluatorをcomponentごとに分離。MTEB等のbenchmarkを他taskへ外挿しない。\nInteroperability: dedicated componentをRAG/agent/workflowの前処理・retrieval・ranking・postprocessingへ接続する。" }, { "heading": "CLAIM_OR_RESEARCH_QUESTION", "text": "Q1. EA-004の「非LLM」はmodel architecture分類ではなく、task-specific operational contract分類として定義すべきか。\nQ2. OCR/ASR/vision/embedding/reranker/classifier等は、汎用LLMで代替可能な場合でも独立componentとして残す運用価値があるか。\nQ3. 各componentの品質は共通の「AI精度」ではなく、task固有metricとfailure envelopeで評価すべきか。\nQ4. local/OSS runtimeとmanaged APIを、精度だけでなくdata movement、hardware、latency、cost、audit、version controlで比較すべきか。\nQ5. 専用AI service内部でLLM/Transformer等を使っている場合、EA-004から除外すべきか。" }, { "heading": "EVIDENCE TABLE", "text": "1) source class: author-maintained ML framework documentation\nexact title: User Guide — scikit-learn 1.9.1 documentation\norg: scikit-learn project\ncanonical URL: https://scikit-learn.org/stable/user_guide.html\npublication/update/access: current stable documentation / accessed 2026-09-11\nversion/release: scikit-learn 1.9.1 documentation\nsupporting locator: Supervised learning; Unsupervised learning; Model selection and evaluation; Metrics and scoring\nlabel: OSS_PRACTICE\nrelation: support / taxonomy / evaluation\nestablished: production AI/ML workloadにはLLM以外にもclassification、regression、clustering、outlier detection、dimensionality reduction、feature extraction等の成熟したtask familiesが存在し、classification/regression/clusteringで評価metricも分かれる。EA-004を「生成AI以外の残り物」ではなく独立したML operational planeとして扱う根拠になる。\n\n\n2) source class: author-maintained OCR OSS documentation\nexact title: Tesseract User Manual\norg: tesseract-ocr project\ncanonical URL: https://tesseract-ocr.github.io/tessdoc/\npublication/update/access: accessed 2026-09-11\nversion/release: Tesseract 5.x current stable major line\nsupporting locator: Introduction; Releases and Changelog; Tesseract with LSTM; API/Testing\nlabel: OSS_PRACTICE\nrelation: support / implementation / counter\nestablished: TesseractはApache-2.0のOCR engineで、CLI/APIから画像中のprinted textを抽出し、多言語traineddataを利用する。5.xがcurrent stable majorで、LSTM recognizerを含む。OCRというtask-specific componentはLLM chat layerなしでも独立実装・local deploymentできる。\nlimit: 「非LLM=古典的アルゴリズム」ではない。Tesseract自体もLSTM neural networkを含む。\n\n\n3) source class: official managed document-AI documentation\nexact title: Document AI / Process documents with Gemini layout parser\norg: Google Cloud\ncanonical URLs: https://cloud.google.com/document-ai ; https://docs.cloud.google.com/document-ai/docs/layout-parse-chunk\npublication/update/access: current pages; layout parser page published/crawled 2026-09 / accessed 2026-09-11\nversion/release: Enterprise Document OCR + layout parser versions including stable pretrained-layout-parser-v1.0-2024-06-03 and Gemini-powered preview generations through 2026\nsupporting locator: Enterprise Document OCR; primary use cases; processor versions; data residency note\nlabel: VERIFIED_SPEC + PRODUCT_VOLATILE\nrelation: support / counter / availability / privacy-limit\nestablished: managed document processingではdedicated Enterprise Document OCRが独立して提供される一方、layout parserはspecialized OCRとGemini generative capabilityを組み合わせる。2026 preview processorにはGemini 3.0系を使うものもあり、一部versionはglobal endpoint利用によりData Residency要件を満たさないと明記される。\ncounter: 「OCR/document intelligence=必ず非LLM内部実装」は成立しない。分類単位は内部model種ではなくprocessor contract/data boundary/versionで持つべき。\n\n\n4) source class: official managed vision API documentation\nexact title: Detect Labels — Cloud Vision API\norg: Google Cloud\ncanonical URL: https://docs.cloud.google.com/vision/docs/labels\npublication/update/access: updated/crawled 2026-08/09 surface / accessed 2026-09-11\nversion/release: current Cloud Vision API; builtin/stable / builtin/latest selectable in related request docs\nsupporting locator: label detection output; score; async batch; API setup\nlabel: VERIFIED_SPEC + PRODUCT_VOLATILE\nrelation: support / operational contract\nestablished: image label detectionはimage inputからlabel descriptionとscore等のstructured responseを返すtask APIとして存在し、LLMの自由文回答と異なるcontractを持つ。batch processingやmodel selectorも運用条件になる。\n\n\n5) source class: official managed ASR documentation\nexact title: Cloud Speech-to-Text documentation\norg: Google Cloud\ncanonical URL: https://docs.cloud.google.com/speech-to-text/docs\npublication/update/access: last updated 2026-08-26 UTC / accessed 2026-09-11\nversion/release: Speech-to-Text V2 current documentation\nsupporting locator: send audio and receive text transcription; streaming/long/short audio; confidence; pricing/quota resources\nlabel: VERIFIED_SPEC + PRODUCT_VOLATILE\nrelation: support / operational contract / availability\nestablished: ASRはaudio→transcriptという専用contractを持ち、streaming/batch/long-form、language/model、confidence、quota等が運用軸となる。product pageではV2にdata residency、audit logging、CMEK等のenterprise controlsが記載される。\n\n\n6) source class: primary research\nexact title: Robust Speech Recognition via Large-Scale Weak Supervision\nauthors: Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever\ncanonical URL: https://arxiv.org/abs/2212.04356\npublication/update/access: 2022-12-06 / accessed 2026-09-11\nversion/release: arXiv 2212.04356\nsupporting locator: abstract; multilingual/multitask speech recognition evaluation\nlabel: RESEARCH\nrelation: support / counter / evaluation\nestablished: Whisperは680,000 hoursのmultilingual/multitask supervisionでASR等を行うTransformer modelで、speech taskがlarge pretrained neural modelでも独立taskとして評価される例である。\ncounter/limit: ASRを「small/classical/non-transformer model」と同義にできない。model architectureとoperational taskを分ける。\n\n\n7) source class: official managed TTS documentation\nexact title: Cloud Text-to-Speech documentation\norg: Google Cloud\ncanonical URL: https://docs.cloud.google.com/text-to-speech/docs\npublication/update/access: last updated 2026-08-26 UTC / accessed 2026-09-11\nversion/release: current Cloud TTS\nsupporting locator: text or SSML input → audio data; voices/languages; regional endpoints; quotas/pricing\nlabel: VERIFIED_SPEC + PRODUCT_VOLATILE\nrelation: support / operational contract\nestablished: TTSはtext/SSML→audioという明確なoutput contractを持つ。voice/language/region/endpoint/quotaがLLM text generationとは別の可用性・品質軸になる。\n\n\n8) source class: official managed translation documentation\nexact title: Translate text with Cloud Translation\norg: Google Cloud\ncanonical URL: https://docs.cloud.google.com/translate/docs/translate-text\npublication/update/access: current 2026 documentation / accessed 2026-09-11\nversion/release: Cloud Translation Basic/Advanced current; standard NMT and standard Translation LLM coexist\nsupporting locator: Basic API model choice; Advanced text translation; glossary/customization boundaries\nlabel: VERIFIED_SPEC + PRODUCT_VOLATILE\nrelation: support / counter / topology\nestablished: 同じtranslation serviceでstandard NMTとTranslation LLMをmodel parameterで選べる。Advancedではdocument translationやglossary/custom model等の別機能がある。\ncounter: 「translation=非LLM」「LLM=translationとは別製品」というarchitecture-based dichotomyはcurrent product realityと一致しない。翻訳というtask contractと実装modelを二軸にする必要がある。\n\n\n9) source class: author-maintained embedding/reranker framework documentation\nexact title: SentenceTransformers Documentation / CrossEncoder\norg: UKP Lab origin; maintained by Hugging Face\ncanonical URLs: https://sbert.net/ ; https://www.sbert.net/docs/package_reference/cross_encoder/model.html\npublication/update/access: current docs / accessed 2026-09-11\nversion/release: Sentence Transformers v6.0 current documentation surface\nsupporting locator: Sentence Transformer; Cross Encoder; Sparse Encoder; Multi-Vector Encoder; CrossEncoder score/class probabilities\nlabel: OSS_PRACTICE + PRODUCT_VOLATILE\nrelation: support / topology / counter\nestablished: current Sentence Transformersはdense embedding、CrossEncoder reranking、sparse embeddings、ColBERT-style multi-vector retrievalを別model familyとして提供する。CrossEncoderはsentence embeddingを生成せずpairをjointly処理してscore/class probabilityを返し、embeddingはprecompute可能だがrerankerは候補pairごとの計算になる。\ncounter: embedding/rerankerを「LLMではない単一vector API」と一括できない。retrieval architecture、latency、indexability、evaluationが異なる。\n\n\n10) source class: primary benchmark research\nexact title: MTEB: Massive Text Embedding Benchmark\nauthors: Niklas Muennighoff, Nouamane Tazi, Loïc Magne, Nils Reimers\ncanonical URL: https://arxiv.org/abs/2210.07316\npublication/update/access: 2022-10-13 / accessed 2026-09-11\nversion/release: arXiv 2210.07316\nsupporting locator: abstract; 8 embedding tasks / 58 datasets / 112 languages; no method dominates all tasks\nlabel: RESEARCH\nrelation: support / counter / benchmark\nestablished: embedding qualityはSTSだけでなくclassification、clustering、pair classification、reranking、retrieval等で評価すべきで、原論文では単一embedding methodが全taskを支配しない。\ncounter: 一つのembedding leaderboard scoreから企業内検索/RAG/分類等すべての品質を保証できない。\n\n\n11) source class: author-maintained inference runtime documentation\nexact title: ONNX Runtime / ONNX Runtime for Inferencing\norg: ONNX Runtime project / Microsoft\ncanonical URLs: https://onnxruntime.ai/docs/ ; https://onnxruntime.ai/inference\npublication/update/access: current docs crawled 2026-08/09 / accessed 2026-09-11\nversion/release: current ONNX Runtime documentation; Execution Provider architecture\nsupporting locator: cross-platform model accelerator; framework interoperability; Execution Providers; cloud/edge/mobile/browser; model validation warning\nlabel: OSS_PRACTICE + VERIFIED_SPEC\nrelation: support / deployment / security-limit\nestablished: models trained in PyTorch/TensorFlow/scikit-learn等を共通runtimeでcloud/edge/mobile/webへ展開し、CPU/GPU/NPU等のExecution Providerへpartitionできる。AI componentの運用ではmodel能力だけでなくruntime/hardware/packagingが独立軸になる。\nsecurity limit: ONNX RuntimeはONNX spec conformityを検証してもmodel accuracy/performance/suitabilityは利用者責任で、untrusted modelは過剰memory/compute等の危険があり安全環境でのinspection/testを推奨する。" }, { "heading": "FINDINGS", "text": "EA-004は採用価値が高いが、章名「非LLM AI」は説明上の便宜的umbrellaと明記すべきである。安全な分類は少なくとも次の六群である。\nA. Perception / extraction: OCR、document layout/entity extraction、image/object/label recognition、ASR、barcode/handwriting等。raw modality→structured observationを返す。\nB. Representation / retrieval: dense/sparse/multi-vector embeddings、vector similarity、candidate retrieval。入力を検索可能なrepresentationへ変換する。\nC. Ranking / decision: reranker/CrossEncoder、classifier、regressor、anomaly detector、forecast等。candidate/record→score/class/valueを返す。\nD. Constrained transformation: machine translation、normalization、transliteration等。入力と出力のsemantic relationが強く規定される。\nE. Synthesis / rendering: TTS、image/video/audio generation等。生成AIを含むが必ずしもlanguage modelではなく、media-specific safety/quality/latency contractを持つ。\nF. Runtime / deployment plane: ONNX Runtime、Core ML等。model taskとは別にhardware、execution provider、edge/cloud、version、resource envelopeを決める。\n\n\n「LLMか非LLMか」と「専用componentか汎用componentか」は別軸である。2026年のDocument AIやTranslationのように、専用OCR/translation contractの内部でLLMを選択・併用する製品が存在する。したがってEA-004の主軸はarchitectureではなく、input→typed output、side effects、metric、failure mode、deployment/data boundaryである。\n\n\n専用componentを汎用LLMへ置き換えられるかどうかも二値ではない。LLM/multimodal modelでOCR、classification、translation、vision説明を行える場合でも、dedicated APIはscore/confidence、schema、batch/stream、region、model selector、latency/cost meter等の明示的contractを持つことがある。一方、専用processor側がGemini等を内部利用する場合もある。選定は「AI世代」ではなく、必要なoutput contractとoperational envelopeで行う。" }, { "heading": "LIMITS / COUNTEREVIDENCE / FAILURE ENVELOPE", "text": "- 「非LLM=古典ML」は誤り。OCRはLSTM、ASRはTransformer、embedding/rerankerもTransformer、専用Document AI/TranslationにもLLM実装が入り得る。\n- 「専用AI=常にLLMより正確/安い/速い」は確立できない。task、model version、data distribution、hardware、batching、quotaで変わるため実測が必要。\n- OCR accuracyをASR/translation/retrieval/classificationへ横展開できない。metricの意味が異なる。\n- embedding benchmarkの上位modelが自社RAGで最適とは限らない。MTEB自体がtask間で単一methodが支配しないことを示す。\n- CrossEncoder rerankerは一般にcandidate pairをjoint processingするためprecomputed embeddingと同じlatency/index特性を持たない。\n- managed APIのconfidence/scoreはprovider間で同じ校正尺度とは限らない。score閾値を横断コピーしない。\n- LLM-powered specialized processorを「LLMではないからdata residency上安全」と推測しない。Google Document AIの例ではprocessor versionによりglobal Gemini endpoint使用等のresidency差が明示される。\n- local/OSSはdataを外部APIへ送らない構成を可能にするが、自動的にsecure/compliantになるわけではない。model provenance、malicious model、patching、runtime/hardware、loggingを自組織が負担する。\n- generative image/video/audioは自由生成性・safety評価を持つため、classifier/OCR等のdeterministic-looking componentと同じacceptance criterionにしない。" }, { "heading": "AVAILABILITY_GATES", "text": "Managed AI: cloud project/account、API enablement、IAM/auth、region/endpoint、processor/model version、quota、billing、data-residency/CMEK/audit support、file/audio/image limits。\nLocal/OSS: model weights/traineddata license/provenance、runtime version、CPU/GPU/NPU backend、memory/storage、operator support、language pack、device OS、packaging。\nOCR/document: input format/page limits、native PDF vs raster、language/script、layout/table/handwriting capability、processor version。\nASR/TTS: language/voice/model、streaming vs batch、audio codec/rate/channel、region、latency、speaker/diarization capability。\nTranslation: language pair、NMT vs LLM model、glossary/custom model/document support、region/API edition。\nEmbedding/reranker: model family、dimension/index schema、context/input limit、dense/sparse/multi-vector compatibility、candidate set size、hardware/latency。\nClassical ML: feature schema、training data/version、threshold/calibration、drift/retraining pipeline。" }, { "heading": "IMPLEMENTATION_STATE", "text": "Classical ML: mature maintained ecosystems such as scikit-learn 1.9.1。\nOCR: mature local OSS Tesseract 5.x and production managed OCR/document services coexist。\nVision/ASR/TTS/Translation: production managed APIs exist; internal model generation and enterprise controls are PRODUCT_VOLATILE。\nEmbedding/reranking: actively evolving maintained ecosystem; Sentence Transformers v6.0 now spans dense/sparse/CrossEncoder/multi-vector families, so fixed two-stage “embedding + reranker” taxonomy alone is already incomplete。\nRuntime: ONNX Runtime provides mature cross-framework/cross-hardware deployment plane; exact EP/backend support is version/hardware dependent。\nGenerative media: EA-004のumbrellaに含めるが、本RUNでは画像/動画生成model/providerのcurrent catalogを深掘りしない。別EA-072等へ詳細をrouteする。" }, { "heading": "BENCHMARK_OR_EVAL", "text": "共通の一個の「AI精度」は置かない。最低限、component-specific evaluatorを要求する。\n- OCR/document: character/word error、field precision/recall/F1、layout/table structure accuracy、page failure rate。用途によりhuman correction timeも測る。\n- ASR: WER/CER、language/domain/noise別slice、streaming latency、timestamp/diarization error。\n- TTS: intelligibility、pronunciation error、speaker/voice consistency、latency、human preference/MOS等。主観metricは評価設計を明示。\n- Translation: BLEU/COMET等を使う場合もhuman adequacy/terminology/glossary complianceと分離し、language/domain別に評価。\n- Embedding/retrieval: Recall@k、nDCG/MRR等をtask datasetで評価。MTEB順位を自社corpusへ直接外挿しない。\n- Reranker: candidate-stage固定でnDCG/MRR/Recall変化、added latency、throughputをembedding retrievalと別計測。\n- Classification/regression/anomaly: precision/recall/F1/AUROC/calibration、MAE/RMSE等をbusiness loss/thresholdと接続。class imbalanceやdriftをslice評価。\n- Generative media: prompt adherence/qualityだけでなくsafety、copyright/provenance、latency/cost、人手審査を別軸化。\n- Runtime: accuracy parityに加えてp50/p95 latency、throughput、memory、energy、binary size、cold start、hardware fallback。" }, { "heading": "SECURITY / PRIVACY / AUDIT / COST / OPERABILITY", "text": "Security/Privacy: dedicated componentごとにdata typeとmovementを記録する。画像、音声、文書、embeddingは機密性が異なり、embeddingを「匿名化済み」とみなさない。managed endpointのregion/residency/CMEK/audit条件とlocal processingを別比較する。local modelはprovenance/signature/dependency/runtime脆弱性を管理する。\nAudit: typed request/responseはloggingしやすい場合があるが、confidence scoreだけでdecision traceabilityが完成するわけではない。model/processor version、threshold、pre/postprocessing、human overrideまで残す。\nCost: page/minute/character/request/vector query/GPU-hour等、meterの単位がcomponentで異なる。LLM token単価へ無理に換算せず、end-to-end task costとhuman correction costを併記する。\nOperability: model/API versionを固定・記録し、processor upgrade時にgolden dataset regressionを再実行する。batch/stream、retry/idempotency、quota、hardware fallbackをcomponent別に設計する。" }, { "heading": "SAFE_WORDING_FOR_FUTURE_BOOK", "text": "「『非LLM AI』は、古いAIや小さなモデルだけを指す言葉ではない。企業システムではOCR、音声認識、音声合成、画像認識、翻訳、埋め込み、reranker、分類・回帰・異常検知など、特定の入力を特定の型の出力へ変換する専用AIが今も独立した運用部品として使われる。これらの内部実装はLSTM、Transformer、NMT、場合によってはLLMを含み得るため、『LLMかどうか』と『専用部品かどうか』は別に確認する。選定時に重要なのは、入力と出力の契約、評価指標、失敗条件、データ移動、実行場所、遅延、費用、監査、モデルversionである。汎用LLMで同じ仕事ができる場合でも、専用APIのschema、confidence、batch/stream処理、region、hardware、quota等が運用上の価値を持つことがある。」" }, { "heading": "UPDATE_WORK_HANDOFF", "text": "1. EA-004を「非LLMというmodel taxonomy」ではなく「task-specific AI component taxonomy」として冒頭定義する。architecture軸（classical/neural/Transformer/foundation/LLM）とoperational contract軸を二次元化する。\n2. 六群 taxonomy（perception/extraction、representation/retrieval、ranking/decision、constrained transformation、synthesis/rendering、runtime/deployment）を章骨格候補とする。\n3. 各component共通表に Input / Output schema / Side effect / Primary metric / Confidence/calibration / Batch-stream / Deployment / Data movement / Version / Failure / Cost meter / Human fallback を置く。\n4. EA-070 Document Intelligence、EA-071 Speech AI、EA-072 Vision/Video AI、EA-073 Forms/Handwriting/Barcode、EA-074 Translation/Localization、EA-077 Classical ML × Generative AI、EA-082 Embedding/Rerankerへ詳細を分岐し、EA-004ではumbrellaと選択原則に留める。\n5. Document AIを反例として残す。専用OCR/layout APIでもGemini-powered versionが存在するため、service categoryから内部architecture/data residencyを推測しない。\n6. Translationもstandard NMTとTranslation LLMを同一task service内で分離できる例として使用し、task taxonomyとmodel taxonomyの独立性を示す。\n7. Sentence Transformers v6.0はdense/sparse/CrossEncoder/multi-vectorへ拡張しているため、embedding/reranker節はvector DBだけで閉じずretrieval architecture familyとして再編する。\n8. MTEBはbenchmark設計例として使うが、2022原論文の数値をcurrent model rankingとして扱わない。公開時はcurrent MTEB version/leaderboard governanceを再確認する。\n9. ONNX Runtime等をdeployment planeとして別節化し、「モデル選定」と「どこでどう実行するか」を分離する。公開時current runtime/version/EP matrixを再確認する。\n10. managed API比較を追加する場合はGoogleだけに偏らずAzure AI Services/AWS AI services等の同task APIを採集し、provider mappingへ分離する。\n11. TEST_PROFILE（未実施）: 機密でない固定golden setを用い、OCR/ASR/translation/embedding/reranker/classifierについて dedicated OSS/local vs managed specialized API vs multimodal/general LLM の三方式を、task metric・latency・cost・data movement・schema validity・failure rateで比較する。認証/課金が必要なAPI実測は本RUNでは行わない。\n12. 公開直前にscikit-learn/Tesseract/Sentence Transformers/ONNX Runtimeのcurrent release、Google managed processor/model version、regional/data-residency条件を再確認する。" }, { "heading": "OPEN_UNKNOWN", "text": "/ STOP_CONDITION" }, { "heading": "OPEN_UNKNOWN", "text": "- 2026年時点の主要provider横断で、同一task（OCR/ASR/TTS/translation/vision）を専用APIと汎用multimodal/LLM APIで比較した独立production evidence。\n- dedicated APIのconfidence scoreが実運用でどの程度calibratedかというprovider/task別証拠。\n- OCR/ASR/translation等で、汎用LLM代替が品質・cost・latency・privacyのどこで勝敗するかの企業実測。\n- current MTEB leaderboard/version governanceと、企業domainへの相関。\n- Sentence Transformers v6.0 multi-vector/sparse/dense/CrossEncoder各familyのproduction-scale独立比較。\n- image/video/audio generationをEA-004 umbrellaでどこまで扱い、EA-072等へどこから分岐するかの編集境界。\n- 各local runtime/modelのSBOM/model provenance/signing/secure-loading実務をどのCore章へ接続するか。" }, { "heading": "STOP_CONDITION", "text": "EA-004の最重要境界、すなわち「非LLMをarchitectureの否定として定義しない」「専用task contractと内部model種を分離する」「componentごとに評価器・data boundary・runtime/cost/failureを持つ」は、mature OSS、current managed APIs、primary research、embedding benchmarkから確立した。画像/動画生成や各provider全製品比較をここで深掘りすると後続専門EAと重複するためbounded closeする。次の未調査最小番号はEA-005 MCP実践辞典" }, { "heading": "CURRENTNESS_DECISION", "text": "The canonical EA-004 packet remains valid as a provider-neutral taxonomy. Current sources continue to support its central rule: “non-LLM AI” is not a reliable model-architecture boundary; operationally useful categories should be defined by task/input-output contract, evaluator, failure envelope, data boundary and runtime/deployment characteristics.\n\nCURRENT_PRIMARY_EVIDENCE:\n- scikit-learn 1.9.1 stable documentation, accessed 2026-10-11: maintained production-oriented families still include classification, regression, clustering, dimensionality reduction, preprocessing, model selection and metrics. This supports keeping classical/specialized ML as an independent operational plane rather than treating it as obsolete residual AI.\n URL: https://scikit-learn.org/stable/\n- Sentence Transformers current documentation, accessed 2026-10-11: the maintained library explicitly distinguishes dense SentenceTransformer, CrossEncoder, SparseEncoder and MultiVectorEncoder model types; the v6 migration guide introduces MultiVectorEncoder and changed dependency/output contracts. This reinforces separating retrieval/ranking operational contract from a simplistic “embedding model” bucket.\n URL: https://sbert.net/docs/migration_guide.html\n- ONNX Runtime Execution Providers documentation, accessed 2026-10-11: a common runtime API spans hardware-specific execution providers with different support/maturity. Runtime/deployment selection is therefore a separate operational plane from model task taxonomy.\n URL: https://onnxruntime.ai/docs/execution-providers/\n- Google Cloud Document AI current release notes / processor docs, accessed 2026-10-11: specialized document/layout processors include both specialized OCR behavior and Gemini-powered parser/extractor versions; some Gemini-backed versions use a global endpoint and have explicit data-residency limitations. Service category therefore does not reveal internal model architecture or data boundary.\n URL: https://docs.cloud.google.com/document-ai/docs/release-notes\n URL: https://docs.cloud.google.com/document-ai/docs/layout-parse-chunk" }, { "heading": "CURRENTNESS_FINDINGS", "text": "1. ESTABLISHED: classification/regression/clustering and other non-generative task families remain maintained first-class ML workloads.\n2. ESTABLISHED: retrieval/ranking libraries now span dense, sparse, cross-encoder and multi-vector families under one ecosystem; “embedding + reranker” is not a complete architecture taxonomy.\n3. ESTABLISHED: deployment/runtime is an independent dimension; ONNX Runtime’s EP model explicitly separates common inference API from hardware/backend selection and maturity.\n4. ESTABLISHED: specialized managed services can internally use LLM/foundation-model components while preserving a specialized task contract. Google Document AI is a current example.\n5. ESTABLISHED: managed-service product category must not be used to infer data-residency guarantees; specific processor/model/version/endpoint rules matter.\n6. LIMIT: no universal claim is made that specialized components outperform, cost less, or are safer than general LLM/multimodal models. Those comparisons remain workload- and version-specific." }, { "heading": "COUNTEREVIDENCE_AND_FAILURE_ENVELOPE", "text": "The existing packet’s counterevidence remains controlling: specialized component ≠ classical architecture; local/OSS ≠ automatically secure/compliant; confidence scores are not portable across providers; benchmark leadership does not establish enterprise-domain superiority; generative media needs a different acceptance envelope from deterministic-looking extractors/classifiers." }, { "heading": "SAFE_WORDING_UPDATE", "text": "“『非LLM AI』はmodel architectureの否定語ではなく、企業運用ではtask-specific AI componentの便宜的なumbrellaとして扱う。OCR、ASR、TTS、翻訳、embedding、reranker、分類・回帰・異常検知などは、内部にTransformerやfoundation modelを含む場合でも、入力・出力schema、評価指標、failure mode、data movement、runtime、latency/costという独自の運用契約を持ち得る。製品カテゴリから内部model種やdata residencyを推定しない。”" }, { "heading": "OPEN_UNKNOWN_RETAINED", "text": "- provider横断の専用API vs general LLM/multimodal APIの独立production comparison\n- confidence calibrationのtask/provider別実運用 evidence\n- enterprise domainとcurrent retrieval benchmarkの相関\n- local model/runtime provenance, signing, secure loadingの実務標準化\n- image/video/audio generationをEA-004 umbrellaと専門EAへ分割する編集境界\n\nRESEARCH_GATE:\nPASS_FOR_BOUNDED_BASE_REVISION — canonical packet already contains exact questions, claim-level evidence, task-specific metrics, counterevidence, scope/data/runtime boundaries, failure envelope, availability gates, security/privacy/audit/cost/operability, SAFE_WORDING and OPEN_UNKNOWN. Currentness was rechecked on 2026-10-11 against maintained OSS/runtime docs and current Document AI processor behavior.\n\nCANDIDATE_IMPLEMENTATION_OVERLAY:" } ], "human_url": "/tools/ea-004-llm-ai-ocr-reranker-ai-llm-provider-neutral-taxonomy", "markdown_url": "/agent/article.md?path=%2Ftools%2Fea-004-llm-ai-ocr-reranker-ai-llm-provider-neutral-taxonomy", "json_url": "/agent/article.json?path=%2Ftools%2Fea-004-llm-ai-ocr-reranker-ai-llm-provider-neutral-taxonomy" }