Fine-tuning／Adapter／Distillation大全

企業AIでは「基盤モデルをそのまま使う」以外に、教師付きfine-tuning、LoRA等のparameter-efficient fine-tuning (PEFT)、量子化併用PEFT、knowledge distillationなど複数の適応経路が実用化されている。これらは同じ“カスタマイズ”ではなく、更新対象パラメータ、学習資源、serving artifact、teacher依存、再学習条件、rollback可能性、データ境界が異なる。公開後版ではprompt/RAGとの選択境界、provider managed customizationとOSS/self-hosted customizationの差、評価・監査・コストを独立して扱う必要がある。

Enterprise AI Core / provider mapping / organization overlay / implementation / governance / benchmark-evaluation / security-privacy / cost-operability

• Fine-tuning、adapter/LoRA/QLoRA、distillationをどのartifact/lifecycle境界で分離すべきか。
• “学習できた”を“本番で安全・有効”へ外挿しないため、何を固定・比較・監査すべきか。
• provider managed tuningの存在と、特定model/region/plan/APIでのavailabilityをどう分離するか。
• teacherの品質をstudentが自動的に継承するという誤解をどう防ぐか。

[E1] source_class=primary paper | title=LoRA: Low-Rank Adaptation of Large Language Models | author/org=Edward J. Hu et al. | canonical=https://arxiv.org/abs/2106.09685 | published=2021-06-17 | accessed=2026-09-16 | version=arXiv:2106.09685 | locator=Abstract/Method | label=RESEARCH | relation=support/context | establishes freezing pretrained weights while injecting trainable low-rank matrices; parameter-efficient adaptation is structurally distinct from full fine-tuning. [E2] source_class=primary paper | title=QLoRA: Efficient Finetuning of Quantized LLMs | author/org=Tim Dettmers et al. | canonical=https://arxiv.org/abs/2305.14314 | published=2023-05-23 | accessed=2026-09-16 | version=arXiv:2305.14314 | locator=Abstract/Methods | label=RESEARCH | relation=support/limit | establishes backpropagation through a frozen quantized pretrained model into LoRA adapters; quantization+adapter is a distinct implementation path, not equivalent to full-precision full-model tuning. [E3] source_class=primary paper | title=Distilling the Knowledge in a Neural Network | author/org=Geoffrey Hinton, Oriol Vinyals, Jeff Dean | canonical=https://arxiv.org/abs/1503.02531 | published=2015-03-09 | accessed=2026-09-16 | version=arXiv:1503.02531 | locator=Abstract/Distillation section | label=RESEARCH | relation=support/context | establishes teacher/ensemble knowledge transfer to a smaller deployable model through training targets; distillation is a training relationship, not model identity equivalence. [E4] source_class=author-maintained implementation/docs | title=PEFT documentation — LoRA | author/org=Hugging Face | canonical=https://huggingface.co/docs/peft/main/en/conceptual_guides/lora | accessed=2026-09-16 | version=current web docs; exact package release not pinned in this run | locator=LoRA conceptual guide | label=OSS_PRACTICE / PRODUCT_VOLATILE | relation=support/context | current PEFT implementation exposes LoRA as adapter-based parameter-efficient tuning; exact package/version must be pinned by update Work before reproducibility claims. [E5] source_class=official product docs | title=Fine-tuning guide | author/org=OpenAI | canonical=https://platform.openai.com/docs/guides/fine-tuning | accessed=2026-09-16 | version=current docs; supported model IDs are PRODUCT_VOLATILE | locator=Fine-tuning methods/workflow/evaluation guidance | label=VERIFIED_SPEC / PRODUCT_VOLATILE | relation=support/limit | managed fine-tuning exists as provider workflow; current supported models, method availability and account eligibility must be re-read at adoption time. [E6] source_class=official product docs | title=Model Distillation | author/org=OpenAI | canonical=https://platform.openai.com/docs/guides/distillation | accessed=2026-09-16 | version=current docs; model IDs PRODUCT_VOLATILE | locator=Distillation workflow | label=VERIFIED_SPEC / PRODUCT_VOLATILE | relation=support/context | provider workflow can use stronger-model outputs/evals as part of a smaller-model optimization loop; this does not establish teacher-equivalent quality across arbitrary workloads. [E7] source_class=official product docs | title=Tune Gemini models by using supervised fine-tuning | author/org=Google Cloud | canonical=https://cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models | accessed=2026-09-16 | version=current Vertex AI docs; model/region support PRODUCT_VOLATILE | locator=Supported models / tuning workflow | label=VERIFIED_SPEC / PRODUCT_VOLATILE | relation=support/limit | managed supervised tuning is provider/model-gated; availability must be evaluated by model, region, project permissions and service state rather than provider-level yes/no. [E8] source_class=official product docs | title=Customize your model to improve its performance for your use case | author/org=Amazon Web Services | canonical=https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html | accessed=2026-09-16 | version=current Bedrock docs; model/region support PRODUCT_VOLATILE | locator=Model customization methods | label=VERIFIED_SPEC / PRODUCT_VOLATILE | relation=support/limit | Bedrock exposes managed model customization paths, but supported customization method varies by model/provider/region and must be pinned at update time.

F1. “Fine-tuning”を一語で扱わない。最低でも (a) full/managed supervised fine-tuning、(b) adapter/LoRA PEFT、(c) quantized PEFT/QLoRA、(d) distillation を別artifact classとする。E1-E3。 F2. LoRA/adapterはbase modelとadapter artifactの組合せとして扱う必要がある。base model revisionが変わればadapterの意味・互換性・性能は再検証対象になる。これはE1の構造から導くIMPLEMENTATION INFERENCEであり、全製品への互換保証/非保証を断定しない。 F3. QLoRAは“LoRAの安価版”というだけではない。quantization configuration、base weights、adapter、training stackが再現性contractに入る。E2。 F4. Distillationはteacher output/behaviorをstudent trainingへ利用する関係であり、studentがteacherの能力、安全性、知識鮮度、tool behaviorを完全継承することを意味しない。E3/E6。 F5. managed customizationはprovider-level capabilityではなく、model ID × method × region × account/project permission × quota × API/service revisionのavailability gateで評価する。E5-E8。 F6. prompt/RAG/tuningは代替関係だけではない。頻繁に変わる知識・ACL付き企業情報は外部retrieval側、安定した出力様式・task behaviorはtuning候補になり得るが、採用は同一holdoutでbaseline比較して決める。PROPOSAL。 F7. training lossの改善はdeployment acceptanceではない。untuned baseline、prompt/RAG baseline、tuned artifactを同じhidden holdoutで比較し、critical regressionsを平均scoreで相殺しない。

• catastrophic forgetting / capability regression: 狭いtaskへの適応で一般能力・安全挙動が落ち得る。特定手法の必然的failure rateは本Packetでは確定しない。
• overfitting / leakage: train/eval重複、near-duplicate、template leakageで見かけ上の改善が生じる。
• adapter/base drift: base revision変更後に旧adapter評価をそのまま継承しない。
• distillation ceiling/shift: teacher error/biasをstudentへ転写し得る一方、studentの容量・training objectiveにより別のfailureも生じる。teacher pass ≠ student pass。
• safety regression: task quality改善とpolicy/safety behaviorは別評価軸。
• data governance: tuning corpusへの秘密情報・PII投入可否、retention、training data handlingはprovider/tenant契約・設定依存。public docsだけで特定tenant状態を証明しない。
• benchmark transfer: public benchmark improvementをtenant workloadの保証へ外挿しない。

provider; base model exact ID/revision; customization method; region; account/project/org eligibility; API/service revision; quota; minimum/maximum dataset constraints; supported input/output modality; adapter serving support; dedicated/shared endpoint requirement; encryption/data residency controls; retention; pricing; deprecation state. 2026-09-16時点のcurrent product matrixはPRODUCT_VOLATILEであり、公開後Workで再pinする。

• Research/OSS: LoRAとQLoRAは確立した公開手法で、PEFT等に実装経路がある。RESEARCH / OSS_PRACTICE。
• Managed provider: OpenAI/Vertex AI/Bedrockにmodel customization経路が存在する。VERIFIED_SPEC / PRODUCT_VOLATILE。ただし“全modelでfine-tuning可能”とはしない。
• Enterprise Core proposal: customization artifactを base_model_ref + method + training_data_manifest + hyperparameters + code/service revision + resulting_artifact_id/hash + eval_suite + approval として登録する。

後続TEST_PROFILE: A. fixed hidden holdoutを、untuned base / prompt-only / RAG where applicable / tuned or adapter / distilled studentで比較。 B. task quality: exact/task success, schema adherence, domain terminology, calibration where meaningful。 C. regression: general capability probes, refusal/safety, tool/function-call correctness, multilingual, long-context subset。 D. memorization/privacy: canary/near-verbatim extraction、train-neighbor test、PII/secret scan。 E. robustness: paraphrase、OOD、adversarial/ambiguous inputs。 F. operability: training wall time、training cost、artifact storage、serving latency p50/p95、per-request cost、rollback time。 G. distillation: teacher vs student disagreement matrixを保持し、teacher-correct/student-wrong、teacher-wrong/student-correct、both-wrongを分離。 Acceptanceは平均scoreだけでなくcritical-regression gateを持つ。

training corpusのorigin、classification、consent/licensing、PII/secrets、provider upload path、retention/deletion evidenceを保持。公開資料をtenant実設定の証明にしない。 AUDIT: dataset manifest/hash、split seed、base model ID、method、hyperparameters、service/package version、artifact ID/hash、eval suite/version、approval、deployment endpoint/version、rollback targetを記録。

training費用だけでなく、data curation、eval、artifact storage、dedicated endpoint、inference token/compute、再学習頻度をTCO化。LoRA/QLoRAの研究上の効率性を特定managed productの料金優位へ外挿しない。

base retirement、adapter compatibility、provider model deprecation、dataset refresh、rollback、multi-region availabilityをlifecycleに含める。

「モデル適応には、教師付きfine-tuning、LoRA等のparameter-efficient tuning、量子化併用PEFT、distillationなど異なる経路がある。これらは学習・配備artifactとfailure surfaceが異なるため、“カスタマイズ済み”を一括して扱わない。採用時はbase model、method、training data、artifact、evaluation、availabilityをversion付きで固定し、未調整baselineと同一holdoutで品質・安全性・privacy・costを再評価する。Distillationはteacherとstudentの同一性を保証しない。」

• 公開後更新時にOpenAI / Google Vertex AI / AWS Bedrockのcurrent supported model × customization method × region × pricing matrixを公式docsから再pin。
• OSS採用候補ではPEFT/Transformers/bitsandbytes等のexact release/tag/commitと互換matrixを固定。
• tuning corpusのgovernance contractとeval split contamination checkを先に作る。
• MODEL CUSTOMIZATION ARTIFACT CONTRACT をCore候補として検討: base_model_ref, tokenizer/processor_ref, method, adapter/quantization config, dataset manifest/hash, training code/service revision, hyperparameters, resulting artifact ID/hash, eval suite, safety/privacy results, approval, deployment/rollback refs。
• prompt/RAG/tuning/distillationを同一task economicsで比較し、training costだけで選ばない。

/ STOP_CONDITION

2026-09-16時点の各providerのexact supported model IDs、region、pricing、quota、retention/data-residency matrix; PEFT current exact release/tag/commit; managed provider内部がLoRA等どのparameterizationを採るか（providerが明示しない限りUNKNOWN）。

本Packetでは手法境界、主要一次研究、managed provider existence、failure envelope、evaluation/audit contractまで確立。次回同じEA-081を再採集するのは、上記OPEN_UNKNOWNをcurrent provider mappingとして閉じる場合、またはsource version/claim relationに実質差分が生じた場合のみ。

---

OpenAI fine-tuning is a shrinking PRODUCT_VOLATILE path; provider-managed fine-tuning must not be represented as stable future default or as exposing LoRA/adapter internals.