Fine-Tuning

Fine-Tuning is the process of further training a pre-trained Large Language Model with custom data to optimize it for specific tasks or domains. Instead of training a model from scratch, the existing weights of a foundation model are adjusted — saving enormous compute costs and time.

There are various approaches: full fine-tuning adjusts all parameters (expensive but powerful), LoRA and QLoRA modify only a small portion of weights (efficient and often sufficient), and instruction tuning trains on question-answer pairs for better instruction following. RLHF (Reinforcement Learning from Human Feedback) is a specialized form that incorporates human preferences.

When to use fine-tuning vs. RAG: Fine-tuning is worthwhile when the model needs to learn a specific style, tone, or specialized behavior — such as a company's brand voice, medical terminology, or a specific response format. For purely factual knowledge, RAG is usually the better choice as it's more current and affordable.

Costs vary widely: OpenAI fine-tuning starts at a few dollars for small datasets, while training a complete open-source model like LLaMA on your own GPU cluster can cost thousands. LoRA-based approaches offer a good middle ground.