{ "format": "agent-article-v0", "article_id": "EA-070", "canonical_url": "/tools/ea-070-document-intelligence", "page_type": "reference", "claim_ids": [], "evidence_ids": [ "EVD-EA-070-URL-E19DD0B74880", "EVD-EA-070-URL-08A0DBA4343E", "EVD-EA-070-URL-4A7D14F99EFD", "EVD-EA-070-URL-93BBCEAD3AA9", "EVD-EA-070-URL-B5AF494A227B", "EVD-EA-070-URL-C0B92483BA50", "EVD-EA-070-URL-94365E0A85D6", "EA-070-R11-RESEARCH", "EA-070-R11-WEB-READBACK" ], "unknown_ids": [ "EA-070-UNKNOWN-01", "EA-070-UNKNOWN-02", "EA-070-UNKNOWN-CURRENTNESS" ], "operational_object_ids": [], "provider": [], "models": [], "plans": [], "surfaces": [], "regions": [], "current_status": "UNKNOWN", "observation_status": "SPEC_ONLY", "freshness": "UNKNOWN", "last_verified": null, "supersedes": [], "superseded_by": [], "provenance": [ "COMM-066-EA0070", "SRC-EA-070-URL-E19DD0B74880", "SRC-EA-070-URL-08A0DBA4343E", "SRC-EA-070-URL-4A7D14F99EFD", "SRC-EA-070-URL-93BBCEAD3AA9", "SRC-EA-070-URL-B5AF494A227B", "SRC-EA-070-URL-C0B92483BA50", "SRC-EA-070-URL-94365E0A85D6", "WEBBODY-EA0070-R8-305CFC26" ], "article_sources": [ { "id": "EVD-EA-070-URL-E19DD0B74880", "evidence_class": "RESEARCH", "title": "author/org: Microsoft", "href": "https://learn.microsoft.com/azure/ai-services/document-intelligence/concept-general-document", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-E19DD0B74880" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://learn.microsoft.com/azure/ai-services/document-intelligence/concept-general-document" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-070-URL-08A0DBA4343E", "evidence_class": "RESEARCH", "title": "author/org: Google Cloud", "href": "https://docs.cloud.google.com/document-ai/docs/output", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-08A0DBA4343E" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.cloud.google.com/document-ai/docs/output" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-070-URL-4A7D14F99EFD", "evidence_class": "RESEARCH", "title": "author/org: Google Cloud", "href": "https://docs.cloud.google.com/document-ai/docs/handle-response", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-4A7D14F99EFD" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.cloud.google.com/document-ai/docs/handle-response" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-070-URL-93BBCEAD3AA9", "evidence_class": "RESEARCH", "title": "author/org: Amazon Web Services", "href": "https://docs.aws.amazon.com/textract/latest/APIReference/API_AnalyzeDocument.html", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-93BBCEAD3AA9" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.aws.amazon.com/textract/latest/APIReference/API_AnalyzeDocument.html" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-070-URL-B5AF494A227B", "evidence_class": "RESEARCH", "title": "author/org: Amazon Web Services", "href": "https://docs.aws.amazon.com/textract/latest/dg/how-it-works-tables.html", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-B5AF494A227B" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.aws.amazon.com/textract/latest/dg/how-it-works-tables.html" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-070-URL-C0B92483BA50", "evidence_class": "RESEARCH", "title": "author/org: Amazon Web Services", "href": "https://docs.aws.amazon.com/textract/latest/dg/bestqueries.html", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-C0B92483BA50" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://docs.aws.amazon.com/textract/latest/dg/bestqueries.html" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EVD-EA-070-URL-94365E0A85D6", "evidence_class": "RESEARCH", "title": "author/org: Google Cloud", "href": "https://cloud.google.com/products/document-ai/pricing", "supports": [], "limit": "元の監査済み資料に記載された参照先です。URLの復元は、資料の最新性や全主張の支持を新たに検証したことを意味しません。", "origin_class": "SOURCE_PACKET_REFERENCE", "source_or_experiment_ids": [ "SRC-EA-070-URL-94365E0A85D6" ], "supporting_locators": [ "EVIDENCE TABLE", "canonical URL: https://cloud.google.com/products/document-ai/pricing" ], "scope": "EVIDENCE TABLE", "provenance_status": "SOURCE_PACKET_REFERENCE_PROJECTED" }, { "id": "EA-070-R11-RESEARCH", "evidence_class": "RESEARCH", "title": "EA-070 監査済みresearch / canonical revision binding", "supports": [ "1. Document Intelligenceは単一OCR機能ではない。最低でも text OCR / reading order / layout / table structure / key-value & forms / selection marks / signatures / domain schema extraction / normalization / confidence & geometry / chunking を分離する。\n2. 「OCR accuracy」と「structured extraction accuracy」は別品質軸。文字列が正しくてもrow/column association、key-value binding、reading order、nested line item、page anchorが誤ればdownstream AIは誤った事実構造を受け取る。\n3. current provider実装はversioned processor/model/APIである。provider名だけでcapabilityを固定せず、processor/model ID/API revisionをprovenanceに保持する。\n4. document processor outputはderived artifactであり原文そのものではない。LLM/RAGへ渡す場合、原文locator/page/geometryと抽出fieldを結び、後から原文へ戻れる設計が必要。\n5. specialized parserのschemaは強力だがdocument class、language、layout、field coverageに依存する。invoice parserの高confidence fieldを契約書・自由帳票へ一般化しない。\n6. direct multimodal LLM parsingとspecialized Document Intelligenceは代替関係と決め打ちしない。前者は柔軟性、後者はschema/geometry/confidence/versioned outputに強みがあるため、fixtureで比較してroutingする。" ], "limit": "canonical監査・revision bindingの存在を示す。外部ソースcurrentnessが本文またはcontrol-planeで確定していない場合は、currentnessをUNKNOWNのまま保持する。", "origin_class": "AUDITED_CANONICAL_RESEARCH_PACKET", "source_or_experiment_ids": [ "COMM-066-EA0070" ], "supporting_locators": [ "COMM-066-EA0070", "EA-070-R5-LEGACY-MIGRATION-20261008-R1", "content/staging/EA-070.md" ], "scope": "監査済みEA本文・canonical revision・既存Source Graph relationの範囲。欠損currentnessは推測で閉じない。", "provenance_status": "AUDIT_PASS_CANONICAL_BINDING" }, { "id": "EA-070-R11-WEB-READBACK", "evidence_class": "IMPL", "title": "EA-070 Web body identity / canonical readback", "supports": [ "canonical staging bodyのSHA-256一致とR9 canonical ledger readbackをProduction projectionの入力として検証する。" ], "limit": "body identity/readbackは本文claim自体の真偽や外部ソースの最新性を追加保証しない。", "checked_at": null, "origin_class": "IMPLEMENTATION_READBACK", "source_or_experiment_ids": [ "WEBBODY-EA0070-R8-305CFC26" ], "supporting_locators": [ "WEBBODY-EA0070-R8-305CFC26", "content/staging/EA-070.md", "EA-070-R5-LEGACY-MIGRATION-20261008-R1" ], "scope": "R9/R10 canonical readback後のProduction生成時body identity。", "provenance_status": "CANONICAL_READBACK_BODY_HASH_MATCH" } ], "research_sections": [ { "heading": "WHY_THIS_MATTERS_TO_UPDATE", "text": "企業AIの入力は生テキストだけではなく、PDF・スキャン・請求書・契約書・帳票・表・フォーム等を含む。LLMへ画像/PDFを直接渡す方式だけでは、OCR、reading order、layout、table topology、key-value、selection mark、signature、domain field、normalized value、confidence、source geometry等の構造を監査可能な形で固定できるとは限らない。Document Intelligenceを独立した非LLM部品として扱い、抽出結果をLLM/RAG/agentへ渡す境界を明示する必要がある。" }, { "heading": "SCOPE_ROUTING", "text": "non-LLM component / implementation / interoperability / Enterprise AI Core / provider mapping / benchmark / governance" }, { "heading": "CLAIM_OR_RESEARCH_QUESTION", "text": "OCR・layout・table・form・invoice等のdocument understandingはLLM本体と分離可能な専門処理層であり、企業運用では「文字が読めた」ことと「文書構造・業務fieldが正しく抽出された」ことを分離評価すべきか。current managed servicesのcapability/version/availabilityと、LLM downstream利用時のfailure envelopeは何か。" }, { "heading": "EVIDENCE TABLE", "text": "1) source class: official product documentation\nexact title: Document Intelligence general document model\nauthor/org: Microsoft\ncanonical URL: https://learn.microsoft.com/azure/ai-services/document-intelligence/concept-general-document\npublication/update/access: updated 2026-08-15 / accessed 2026-09-15\nversion: v4 GA API 2024-11-30; v3.1 GA 2023-07-31\nsupporting locator: general document features / key-value pair extraction / input requirements\nevidence label: VERIFIED_SPEC / PRODUCT_VOLATILE\nrelation: support / limit\nfinding: v4以降prebuilt-documentはdeprecatedで、key-value pairs等はLayout model + features=keyValuePairsへ移行。v3.0 APIは2029-03-30 support end予定。PDF/image/Office対応やpage/file limitsもmodel/tier依存。\n\n\n2) source class: official product documentation\nexact title: Sample processor output\nauthor/org: Google Cloud\ncanonical URL: https://docs.cloud.google.com/document-ai/docs/output\npublication/update/access: accessed 2026-09-15\nversion/model IDs: Enterprise OCR pretrained-ocr-v2.1.1-2025-01-31; Custom Extractor pretrained-foundation-model-v3.5-2026-05-26; Form Parser v2.1-2023-06-26; Layout Parser v1.0-2024-06-03\nsupporting locator: Digitize text / Extract documents processor tables\nevidence label: VERIFIED_SPEC / PRODUCT_VOLATILE\nrelation: support / context\nfinding: OCR、quality analysis、entity extraction、form parsing、layout parsing、document chunkingがprocessor別に分離され、processor/model versionが明示される。\n\n\n3) source class: official API documentation\nexact title: Handle processing response\nauthor/org: Google Cloud\ncanonical URL: https://docs.cloud.google.com/document-ai/docs/handle-response\npublication/update/access: accessed 2026-09-15\nversion: current docs\nsupporting locator: Entities, nested entities, and normalized values\nevidence label: VERIFIED_SPEC\nrelation: support / limit\nfinding: Invoice parser等はschema-grounded entities、confidence、text/page anchors、normalized values、nested line itemsを返す。ただしchild naming conventionは全processorで一様ではない。\n\n\n4) source class: official product/API documentation\nexact title: AnalyzeDocumentRequest / Analyze Document\nauthor/org: Amazon Web Services\ncanonical URL: https://docs.aws.amazon.com/textract/latest/APIReference/API_AnalyzeDocument.html\npublication/update/access: API reference published 2026-09-11 / accessed 2026-09-15\nversion: current Textract API\nsupporting locator: FeatureTypes; response BlockType\n evidence label: VERIFIED_SPEC / PRODUCT_VOLATILE\nrelation: support / context\nfinding: TABLES、FORMS、SIGNATURES、LAYOUTをfeature typeとして選択でき、QUERY/QUERY_RESULT、merged cell、table title/footer、layout header/footer/list/figure等をBlockとして返す。\n\n\n5) source class: official product documentation\nexact title: Tables\nauthor/org: Amazon Web Services\ncanonical URL: https://docs.aws.amazon.com/textract/latest/dg/how-it-works-tables.html\npublication/update/access: accessed 2026-09-15\nversion: current docs\nsupporting locator: table extraction description\nevidence label: VERIFIED_SPEC\nrelation: support\nfinding: cell、merged cell、column header、title、section title、footer、table type、summary cell等を構造として抽出する。\n\n\n6) source class: official product documentation\nexact title: Best Practices for Queries\nauthor/org: Amazon Web Services\ncanonical URL: https://docs.aws.amazon.com/textract/latest/dg/bestqueries.html\npublication/update/access: accessed 2026-09-15\nversion: current docs\nsupporting locator: Extracting Tables using Queries / Long Answers\nevidence label: VERIFIED_SPEC\nrelation: counter / limit\nfinding: Queriesはwhole table/whole row/column抽出をsupportせず、長いanswerはlatency/timeoutを増やす。query featureの存在を一般的structured extraction能力へ外挿できない。\n\n\n7) source class: official pricing\nexact title: Document AI pricing\nauthor/org: Google Cloud\ncanonical URL: https://cloud.google.com/products/document-ai/pricing\npublication/update/access: accessed 2026-09-15\nversion: current pricing\nsupporting locator: Form Parser / Layout Parser pricing tables\nevidence label: VERIFIED_SPEC / PRODUCT_VOLATILE\nrelation: context / limit\nfinding: processor別page/count課金があり、Document Intelligence層はLLM token costとは別meterとして管理すべき。" }, { "heading": "FINDINGS", "text": "1. Document Intelligenceは単一OCR機能ではない。最低でも text OCR / reading order / layout / table structure / key-value & forms / selection marks / signatures / domain schema extraction / normalization / confidence & geometry / chunking を分離する。\n2. 「OCR accuracy」と「structured extraction accuracy」は別品質軸。文字列が正しくてもrow/column association、key-value binding、reading order、nested line item、page anchorが誤ればdownstream AIは誤った事実構造を受け取る。\n3. current provider実装はversioned processor/model/APIである。provider名だけでcapabilityを固定せず、processor/model ID/API revisionをprovenanceに保持する。\n4. document processor outputはderived artifactであり原文そのものではない。LLM/RAGへ渡す場合、原文locator/page/geometryと抽出fieldを結び、後から原文へ戻れる設計が必要。\n5. specialized parserのschemaは強力だがdocument class、language、layout、field coverageに依存する。invoice parserの高confidence fieldを契約書・自由帳票へ一般化しない。\n6. direct multimodal LLM parsingとspecialized Document Intelligenceは代替関係と決め打ちしない。前者は柔軟性、後者はschema/geometry/confidence/versioned outputに強みがあるため、fixtureで比較してroutingする。" }, { "heading": "LIMITS / COUNTEREVIDENCE / FAILURE ENVELOPE", "text": "- scan quality、rotation、small text、handwriting、language/script、complex merged tables、multi-column reading order、footnote/header/footer、checkbox、stamp/signature、nested tablesでerror classが異なる。\n- confidence scoreはcalibration済みbusiness correctnessを自動保証しない。fieldごとのthreshold/human review policyが必要。\n- table cell textが正しくてもrow/column spanやheader associationが誤れば計算・RAG chunkは破損する。\n- domain parserは未定義fieldを落とす可能性があり、schema coverageとextraction precision/recallを分離する。\n- downstream LLMが抽出JSONを再解釈して誤る可能性があるため、processor accuracyとanswer accuracyを混同しない。\n- benchmark/public sampleは特定tenant・特定document distributionの保証ではない。" }, { "heading": "AVAILABILITY_GATES", "text": "provider / cloud region / processor type / API revision / model version / language / input MIME / page and file-size limit / sync-vs-async path / pricing tier / custom-training requirement / preview-vs-GAを個別確認する。Microsoft v4へのmigration、Google processor version、AWS FeatureTypesをcurrent claimとして再確認する。" }, { "heading": "IMPLEMENTATION_STATE", "text": "Microsoft Azure Document Intelligence、Google Cloud Document AI、Amazon Textractはいずれもmanaged document-processing層を現行提供。実装方式・schema・versioningは相互に同一ではない。共通Coreではvendor-neutralなDocument Extraction Contractを置き、provider mappingで各output schemaへ対応させるのが妥当。" }, { "heading": "BENCHMARK_OR_EVAL", "text": "公開benchmarkの単一scoreを製品保証へ使わない。後続Workでは自前fixtureを優先し、少なくとも OCR CER/WER、reading-order error、table cell accuracy、row/column topology、key-value F1、domain-field exact/normalized match、page-anchor accuracy、abstention/empty-field behavior、latency、cost/pageを別々に測る。document class別・language別・scan quality別にsliceする。" }, { "heading": "SECURITY / PRIVACY / AUDIT / COST / OPERABILITY", "text": "- 文書はPII、契約情報、財務情報を含み得るため、processing region、retention/logging、encryption、access control、human review経路をproviderごとに確認する。\n- extraction output自体もsensitive derived dataとしてACL/retention対象にする。\n- auditには source document hash/version、processor/model/API version、config/features、timestamp、output artifact ID、confidence、human correctionを保持する。\n- costはpage/processor課金、custom training、LLM downstream token、human reviewを別meter化する。\n- migration/deprecationを監視し、同一fixtureでprocessor version regressionを行う。" }, { "heading": "SAFE_WORDING_FOR_FUTURE_BOOK", "text": "「企業AIの文書入力は、LLMへPDFを渡す処理だけで完結しない。OCR、レイアウト、表、フォーム、業務field抽出は独立したDocument Intelligence層として実装でき、主要cloudはversioned processor/APIを提供している。ただし文字認識・構造抽出・業務field抽出・最終回答は別々の品質段階であり、一段の成功を後段の正確性へ外挿してはならない。」" }, { "heading": "UPDATE_WORK_HANDOFF", "text": "Core候補: DOCUMENT EXTRACTION CONTRACT\nfields: source_document_id/hash/version; page; bounding geometry; raw text; reading order; block type; table/cell coordinates and spans; key/value linkage; entity/schema field; normalized value; confidence; processor/provider/model/API version; config/features; language; extraction timestamp; source locator; human correction; ACL; retention.\nProvider mappingではAzure Document Intelligence / Google Document AI / Amazon Textractをこのcontractへmappingする。将来Microsoft/Google/AWS以外を追加しても同じcontractで比較する。" }, { "heading": "TEST_PROFILE", "text": "synthetic + redistributable non-sensitive documentsを用意し、clean PDF / noisy scan / rotated / multi-column / merged-table / form / invoice / contract-like / multilingualを含める。ground truthをtext・reading order・table topology・field schema・page locatorで別々に持ち、processor単体→LLM downstreamの二段評価を実施する。認証済みtenant live testは未実施。" }, { "heading": "OPEN_UNKNOWN", "text": "/ STOP_CONDITION" }, { "heading": "OPEN_UNKNOWN", "text": "Azure/Google/AWS各serviceの全region・全language・全processorのcurrent availability、各confidence calibration、custom model training/data governance詳細、direct multimodal LLMとの同一fixture比較、independent reproducible benchmarkでのcurrent managed-service横断比較。" }, { "heading": "STOP_CONDITION", "text": "current official capability/version/limit、structured-output failure envelope、vendor-neutral contract、safe comparative TEST_PROFILEが揃ったためEA-070の先行Evidence Packetはboundedに閉じる。公開後更新時はproduct volatile項目を再確認し、必要なら実測を追加する。\n\n---" } ], "human_url": "/tools/ea-070-document-intelligence", "markdown_url": "/agent/article.md?path=%2Ftools%2Fea-070-document-intelligence", "json_url": "/agent/article.json?path=%2Ftools%2Fea-070-document-intelligence" }