Fine-tuning makes sense when you have thousands of labeled examples, a very specific domain, and critical latency. For everything else, structured prompting with continuous evaluation wins.
The most common mistake: fine-tuning before having a solid baseline with the base model.