What Makes a Good Fine-Tuning Dataset (and Why Most Aren't)

When a fine-tuned model disappoints, the first instinct is to blame the model or the training settings. Try a bigger base model, more epochs, a different learning rate. Occasionally that's the problem. Much more often the model did exactly what it was trained to do, and the dataset taught it the wrong thing.

Here's what we look at when a dataset needs to produce a model someone will actually rely on.

Build the evaluation set before anything else

It sounds backwards, but it's the single habit that saves the most money. Before collecting or labeling training data, set aside a few hundred examples that represent real use, including the hard cases, and decide exactly how they'll be scored. Exact match, a rubric, a human review checklist. Whatever fits the task, as long as it's the same every time.

This set never gets trained on. Its whole job is to tell you whether version three is actually better than version two, and to catch the moment when a model starts looking good on paper while getting worse at the thing you care about.

Consistency beats volume

Two annotators labeling the same kind of example two different ways is poison for a fine-tune. The model can't learn a rule that the data itself doesn't follow, so it learns something in between, and that something tends to show up in production as randomness.

Write labeling guidelines with real examples, including the borderline ones. Have a second person review a sample. Measure how often reviewers disagree, and when they do, fix the guideline instead of just picking a winner. A few thousand examples labeled to one clear standard will beat fifty thousand labeled to several.

Cover the cases that actually break things

Real data is lopsided. Most support tickets are the same five questions. Most product images are clean, well-lit shots of the common item. A dataset sampled straight from reality will teach the model the easy cases thoroughly and the hard ones barely at all, which is backwards from where the value is.

Deliberately oversample the edge cases: the ambiguous request, the rare defect, the input in a different format, the question the model should decline. If you already have a model in production, its failures are the best source of training examples you'll ever get.

Clean up the boring problems

Duplicates and near-duplicates make a model overconfident about whatever they repeat, and if they leak into the evaluation set they make your scores look better than they are. Remove them, and check that no evaluation example has a close twin in the training data.

Strip out personal data you don't need. Fix encoding problems, broken formatting, truncated text. None of this is interesting work, and all of it shows up in the final model if it's skipped.

Match the format to the job

Instruction-tuning data wants clear prompt–response pairs in the exact shape the model will see in production, including any system prompt. Preference data for DPO or RLHF-style training needs pairs of responses where one is genuinely better than the other for a reason a reviewer can state. Vision tasks need boxes, masks or labels drawn to a written standard, down to how tightly a box should hug an object.

Getting the format wrong doesn't usually cause an error. It causes a model that works fine in the notebook and strangely in production, because production looks different from what it was trained on.

Languages the base model barely knows

For low-resource languages, including Bangla, the base model's weakness is often the whole reason for fine-tuning, and it makes data quality even more important. Machine-translated training data is tempting because it's fast, but it teaches the model translation artifacts, stiff phrasing and vocabulary no native speaker would use. Native-speaker written or reviewed data costs more and is worth it, at least for the evaluation set and a solid core of the training set.

How big does it need to be?

Smaller than most people expect, for narrow tasks. A few hundred to a few thousand high-quality examples is often enough to change format and behavior on a focused task with a modern open-weight model. Broad capability changes need far more. The honest answer is to train on a small, clean set first, score it on the evaluation set, and let the curve tell you whether more data is buying improvement.

The short version

Define how you'll measure success before training. Label to one clear standard. Oversample the hard cases. Remove duplicates and leaks. Match the format to production. And spend the effort on quality before quantity.

If you'd rather hand this off, we build custom and domain-specific datasets and fine-tune models on them, under NDA, with the data and weights delivered to you. If you're still deciding whether fine-tuning is even the right tool, start with fine-tuning vs RAG vs prompting.