Fine-Tuning Llama 3 for Domain-Specific Enterprise Tasks

The release of open models like Meta’s Llama 3 has made advanced AI available to everyone. However, out of the box, these models are generalists. They know a little bit about everything, but they don’t know the deep details of your specific business. For enterprise tasks, like translating complex legal documents, summarising medical notes, or writing code in your company’s private language, you need a specialist, not a generalist.

Why Fine-Tune?

Fine-tuning is basically sending the model to university. We take a pre-trained model and teach it using your specific data.

1. Style: Improve the model so it writes in your brand’s voice.
2. Knowledge: Teach the model industry acronyms and concepts it wouldn’t know from the internet.
3. Behaviour: Force the model to output data in exact formats (like “always output strict JSON”).

The Workflow

We use a strong pipeline for this process:

1. Formatting: We convert your documents into a chat format (User asks, Assistant answers).
2. Packing: We organise the data to make training faster and cheaper.
3. Training: We use standard libraries to update the model.

   # A simple example of starting a training run
   trainer = SFTTrainer(
       model=model,
       train_dataset=dataset,
       max_seq_length=2048,
   )
   trainer.train()

4. Evaluation: We test the new model against a “control group” of questions to make sure it hasn’t forgotten how to speak normal English while learning the new topic.

Case Study: Legal Tech

We helped a UK law firm fine-tune Llama 3 on 50,000 anonymised contracts. The result was a model that could extract legal clauses with 94% accuracy, beating the much larger (and more expensive) GPT-4 model which only managed 78%.

Want your own private AI? Chat with our AI Engineers about building custom models for your business.