埋め込みモデル・検索結果の再順位付けモデルの比較

Enterprise retrieval quality is not determined by the LLM alone. Candidate generation, embedding space, chunk representation, query/document task semantics, metadata/ACL filtering, reranking, score calibration and index/model migration create independent failure surfaces. The update should therefore treat embedding and reranking as versioned retrieval infrastructure rather than an invisible implementation detail of RAG.

Enterprise AI Core / retrieval / implementation / interoperability / benchmark / governance / provider mapping.

What must be pinned, evaluated and audited when an enterprise system uses embedding models and rerankers, and what claims remain unsafe when benchmark or provider documentation is the only evidence?

• Embedding and reranking are different stages. Dense embedding normally provides efficient candidate generation; a cross-encoder/reranker can score query-document pairs with richer interaction at greater per-candidate cost. The Sentence Transformers retrieve-and-rerank pattern is an implementation example, not a universal requirement.
• Embedding vectors are model artifacts, not timeless data. Model ID/revision, task mode, dimensionality, normalization and preprocessing must be pinned. Replacing an embedding model can change the vector space even when dimensionality remains compatible; old and new vectors must not be assumed mutually comparable without migration testing.
• Provider task semantics matter. Vertex embedding documentation exposes task-oriented request semantics; therefore query/document representation should preserve which task mode produced each vector. A generic embedding_model=... field is insufficient where provider APIs distinguish retrieval roles or task types.
• Reranker scores are model-relative relevance signals. They are not probabilities of truth, authorization, safety or business correctness. A higher rerank score cannot override ACL/policy gates.
• Benchmark rank is not deployment proof. BEIR and MTEB demonstrate that retrieval/embedding quality is multi-dataset and multi-task. A leaderboard aggregate cannot establish performance for a tenant corpus, language mix, chunking policy, ACL topology, freshness requirement or domain-specific relevance definition.
• Retrieval evaluation must be staged. Candidate recall@K should be measured before reranking; reranking metrics should then be measured on the candidate set. Otherwise a strong reranker can hide candidate-generation misses in aggregate reporting, while a reranker cannot recover documents absent from the candidate pool.
• Index migration is an operational event. A model/dimension/preprocessing change requires re-embedding, index rebuild or dual-index migration, shadow evaluation and rollback planning. It should not be treated as a prompt-only change.

• A high MTEB/BEIR score does not prove tenant-corpus quality; benchmark-to-product extrapolation is prohibited.
• Dense retrieval can miss exact identifiers, rare names, numbers or lexical constraints; hybrid lexical+dense retrieval remains a workload-dependent alternative rather than a guaranteed improvement.
• Reranking only reorders candidates it receives; candidate-generation false negatives are unrecoverable downstream unless another retrieval path contributes them.
• ANN index settings can trade recall for latency/cost independently of embedding-model quality.
• Chunking, document deduplication, metadata filters and ACL filtering can dominate end-to-end retrieval quality even when the embedding model itself is unchanged.
• Score distributions can shift across model revisions, languages and corpora; fixed score thresholds require recalibration.
• Query/document role inversion or incorrect provider task_type can create silent quality degradation without API failure.
• Multilingual benchmark strength does not prove parity for a specific low-resource language/domain.

• Provider/account/region/model availability and retirement status.
• API/model revision and allowed input types/lengths.
• Embedding dimension and any supported dimension-reduction parameter.
• Task/query/document mode semantics.
• Reranker candidate/document limits and request-size limits.
• Vector-store support for dimension, metric, filtering and index type.
• Data residency/retention/training terms for externally hosted embedding/reranking APIs.
• Cost/throughput quotas for bulk re-embedding and online reranking.

PROPOSAL — `RETRIEVAL MODEL ARTIFACT CONTRACT` Persist at minimum: corpus snapshot/hash; document/chunk ID; source ACL/security label; chunker/version/config; normalization/preprocessing; embedding provider/model/revision; task role/type; dimension; vector normalization; distance metric; vector-store/index type and ANN parameters; index build/version; query embedding config; candidate retriever(s); top-K; metadata/ACL filter stage; reranker provider/model/revision; reranker top-N; raw retrieval/rerank scores; final selected evidence IDs; latency/cost; evaluation suite/version; deployment/rollback refs.

Recommended execution order: authorization-aware corpus selection/filtering -> candidate retrieval (dense/lexical/hybrid as designed) -> candidate-set logging -> reranking -> evidence selection -> generation. Whether ACL filtering must occur pre-retrieval, during retrieval or post-retrieval is system-specific, but unauthorized content must not become an LLM-visible evidence path.

TEST_PROFILE (not executed in this run) Create a tenant-safe fixture with: exact-ID lookup; paraphrase; acronym; rare proper noun; numeric constraint; multilingual query; long document; near duplicate; conflicting versions; ACL-excluded relevant document; freshness-sensitive item; hard negative; no-answer query.

Compare at least: lexical baseline; dense-only; hybrid if supported; dense + reranker; current production stack vs candidate migration stack. Measure separately: candidate recall@K, MRR/nDCG@K, reranker delta, end-to-end answer evidence recall, unauthorized-retrieval rate, stale-evidence rate, duplicate rate, no-answer abstention, latency p50/p95, embedding/index build cost, online query/rerank cost. Keep model-level and pipeline-level metrics separate.

For migrations, dual-index the same frozen corpus, run shadow queries, compare overlap and relevance judgments, then test rollback. Do not mix old/new vectors in one logical similarity space unless the provider/implementation explicitly guarantees compatibility and this is independently validated.

• Security: retrieval relevance never supersedes authorization. Preserve ACL/filter provenance and test an ACL-excluded-but-highly-relevant adversarial fixture.
• Privacy: externally hosted embedding/rerank APIs may receive source text or query text; provider data-use/residency/retention terms require current verification.
• Audit: log corpus/index/model/chunker/reranker versions with selected evidence IDs so an answer can be replayed against the retrieval stack that produced it.
• Cost: bulk re-embedding, index duplication during migration, storage and reranking per query are distinct cost centers.
• Operability: support shadow index, dual-read comparison, threshold recalibration, drift monitoring and rollback.

"Embedding models and rerankers are versioned retrieval components, not invisible properties of the LLM. Their quality depends on corpus, task, chunking, filters, index configuration and evaluation target. Public benchmark results are useful for candidate selection but do not establish tenant-specific retrieval quality. Model or index changes should therefore be handled as auditable migrations with corpus-specific evaluation and rollback."

• Re-pin every commercial provider model ID, revision, region, limits, pricing and data-control terms immediately before publication; mark as PRODUCT_VOLATILE.
• Decide whether this material extends the existing permission-aware retrieval Core or becomes a dedicated retrieval-model subsection; do not duplicate ACL material already established elsewhere.
• Add at least one current independent multilingual/domain retrieval evaluation beyond BEIR/MTEB if the update makes cross-language claims.
• For any named vector database, collect official index/filter semantics and an independent failure/operations source before making product-specific reliability claims.
• If the book recommends a reranker threshold, require corpus-specific calibration evidence; no universal threshold is established here.

exact 2026-09-16 release/tag/commit for MTEB and Sentence Transformers; current exact commercial reranker model IDs/regions/pricing/limits across providers; provider-specific embedding backward-compatibility guarantees; tenant-specific relevance/ACL performance. These remain unclosed rather than inferred.

STOP_CONDITION for this packet: satisfied when the update Work has (a) re-pinned current commercial model/API availability, (b) attached tenant-safe candidate-recall + rerank evaluation, (c) documented index migration/rollback, and (d) verified authorization placement. Until then, only the architecture/evaluation claims above should be treated as stable Core guidance.

---

Current embedding model IDs/dimensions/pricing/compatibility remain PRODUCT_VOLATILE and benchmark results remain workload dependent.