Fine-tuning vs in-context learning: what is the difference?

I would start by separating what changes. With in-context learning, I put the instructions and any examples directly into the prompt. The model parameters do not change, so I can iterate immediately by editing the prompt. The tradeoff is that every request carries and processes the extra prompt tokens, and the behavior I can obtain is constrained by the context window. It is a good fit when the task is quick to prototype, the instructions change often, or I want temporary behavior without a training run.

Fine-tuning is different because I train the model on examples and update model parameters. That can make the desired format, tone or narrow task behavior more consistent and can make a smaller model perform as well as a larger model that only receives examples in its prompt. These benefits require evaluation. Learned behavior can reduce how much instruction I need to supply on every call.

I would also be explicit about the cost. Fine-tuning requires curating data, running training and evaluating whether the result improved. After deployment, a changed model normally means redeployment. I would not use fine-tuning as my default way to teach frequently changing facts; retrieval is better because it lets me update the source of truth without retraining. So I choose based on whether I need durable behavior changes or current information, and whether the per-request prompt cost matters.