AI Training Foundations
Lesson 1 of 4Beginner 9 min

What it means to train an AI model

Pretraining, fine-tuning, preference training and prompting are four different levers. Knowing which one you're pulling saves weeks.

In this lesson you'll

  • Describe what happens during pretraining
  • Tell adaptation by prompting apart from adaptation by changing weights
  • Know which lever to reach for first

Today's language models are Transformers [1], neural networks trained to predict the next token of text. During pretraining, a model reads a very large corpus and adjusts billions of weights to get better at that one prediction task. Its loss falls predictably as model size, data and compute grow, following power laws [2]. Pretraining is where general knowledge and language ability come from, and it costs far more than almost any team will spend on its own task.

Everything after pretraining is adaptation: making a general model good at your particular job. There are two families of adaptation, and the distinction matters more than any single technique.

Adapting without touching weights

GPT-3 showed that large models can pick up a task from instructions and a handful of examples placed in the prompt, with no gradient updates at all. The authors called this in-context or few-shot learning [3]. Prompt engineering, system prompts and retrieval all work this way. They're fast to iterate on, cheap and fully reversible.

Adapting by changing weights

  • Supervised fine-tuning (SFT): continue training on input → ideal-output pairs for your task.
  • Parameter-efficient fine-tuning: methods such as LoRA train a small number of added weights and leave the base model frozen [4].
  • Preference training (RLHF and relatives): teach the model which of two outputs people prefer. InstructGPT used this to make a 1.3B-parameter model whose outputs labelers preferred over those of the 175B GPT-3 [5].

Rule of thumb

Reach for the cheapest lever that could work. Start with a strong prompt and a measured baseline, and only change weights when the evaluation tells you prompting has hit its ceiling.

In Kojen

The Forge runs prompt tuning and model search for real today. LoRA/QLoRA, full fine-tuning and RLHF for text models are previews with simulated metrics, so no text model weights are trained yet.

Key takeaways

  • Pretraining gives general ability; adaptation makes it useful for your task.
  • Prompting adapts behaviour without changing weights, and it's the fastest loop you have.
  • Fine-tuning and preference training change weights, so they need more data, more care and more evaluation.

Practise in the dojo

Run a prompt-tuning job on any project and look at how much a better system prompt moves the score before you consider fine-tuning.

Open the Forge

Check your understanding

Score 70 or more to complete the lesson. Progress is saved in this browser.

1. What did the GPT-3 paper show about few-shot learning?
2. Which of these changes the model's weights?

Further reading

Sources

Checked against the original sources in November 2026.

  1. [1]Vaswani, A. et al. (2017). Attention Is All You Need. NeurIPS 2017 · arXiv:1706.03762.
  2. [2]Kaplan, J. et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
  3. [3]Brown, T. et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020 · arXiv:2005.14165.
  4. [4]Hu, E. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. ICLR 2022 · arXiv:2106.09685.
  5. [5]Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022 · arXiv:2203.02155.