The organizing decision of AI engineering: given a foundation model you did not train, which adaptation technique fixes the failure you actually have? The ladder runs from cheapest to most expensive — prompting (no weight change), then RAG, then fine-tuning, then combining — and you climb it only when the rung below is exhausted. The rung is chosen by diagnosing the type of failure: if the model fails because it lacks information (private company data, recent events), RAG gives it access to that information; if the model misbehaves — outputs that are factually correct but irrelevant or wrongly formatted — fine-tuning helps more. When both are true, start with RAG (it's easier), begin with a simple term-based retriever, and add fine-tuning after; combining the two often yields the biggest gain. The whole ladder is gated by an evaluation pipeline built first, so you can tell which failure you're looking at.
Claims
- Exhaust prompting before heavier techniques; extract maximum value from prompts before spending on RAG or fine-tuning. principle
- RAG addresses information failures; fine-tuning addresses behavior failures. principle
- When a model has both failure types, start with RAG (easier) and combine with fine-tuning; the combination often gives the biggest boost. (best practice — context: adapting a model that both lacks information and misbehaves)
- Design the evaluation criteria and pipeline before adapting, so failure type is observable. (best practice — context: any model-adaptation project)
- Avoid fine-tuning if you need a general-purpose model — it can improve target tasks while degrading others. (best practice — context: fine-tuning trades generality for on-task performance)
Related
- The Augmented LLM — the composed system (RAG + tools + memory) that adaptation builds toward.
- Parameter-Efficient Fine-Tuning (PEFT / LoRA) — the cheap way to take the fine-tuning rung.
- Post-Training Alignment — the provider-side fine-tuning that precedes your own adaptation.
- Data-Centric AI — the data that every rung above prompting depends on.
- Semantic Retrieval, Hybrid Retrieval — the retrieval mechanisms the RAG rung uses.
- Distillate: AI Engineering in 76 Minutes — Chip Huyen's Book, Speedrun