46. Text Generation

After training, a language model generates text autoregressively.

context
 ↓
predict next token
 ↓
append token
 ↓
predict next token
 ↓
repeat

Common decoding strategies include:

• greedy decoding
• temperature sampling
• top-k sampling
• top-p sampling