11. Custom Model Fine-tuning

Why Fine-tune?

Base LLMs are general-purpose. Fine-tuning adapts a model to your domain — your terminology, tone, data formats, and business logic. This leads to higher accuracy, fewer hallucinations, and more consistent outputs for domain-specific tasks.

When to Fine-tune vs. Prompt Engineer

Fine-tuning Methods

• Full Fine-tuning — updates all model weights. Most expressive, most expensive.
• LoRA (Low-Rank Adaptation) — injects small trainable matrices. Fast, cheap, composable.
• QLoRA — quantized LoRA for fine-tuning on consumer GPUs.
• DPO (Direct Preference Optimization) — aligns model to human preferences without RLHF.

Dataset Preparation

Fine-tuning data typically follows a chat format. For instruction tuning, use this structure:

[
  {
    "messages": [
      { "role": "system", "content": "You are a medical coding assistant." },
      { "role": "user", "content": "Diagnosis: Type 2 diabetes, initial encounter" },
      { "role": "assistant", "content": "E11.9" }
    ]
  },
  {
    "messages": [
      { "role": "system", "content": "You are a medical coding assistant." },
      { "role": "user", "content": "Diagnosis: Essential hypertension" },
      { "role": "assistant", "content": "I10" }
    ]
  }
]

Data Quality Checklist

• Minimum 500-1000 high-quality examples
• Ensure label consistency — have 2+ annotators per sample
• Balance classes to avoid bias
• Include edge cases and refusal examples
• Split into train/validation/test (80/10/10)

Fine-tuning with Azure AI Foundry

Azure AI Foundry provides a managed fine-tuning service for models like GPT-4o, Phi-3, and Llama families:

# Azure CLI — start a fine-tuning job
az ml job create --file fine-tuning-job.yml

# fine-tuning-job.yml
$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
type: command
code: .
command: >-
  python finetune.py
  --model gpt-4o
  --training-file azureml://datasets/train.jsonl
  --validation-file azureml://datasets/val.jsonl
environment: azureml:openai-finetune-env:latest
compute: azureml:gpu-cluster
resources:
  instance_count: 1
  instance_type: Standard_ND40rs_v2

Evaluating a Fine-tuned Model

After fine-tuning, evaluate using:

• Hold-out test set — accuracy, F1, exact match on unseen examples
• Side-by-side evaluation — compare base vs. fine-tuned on representative prompts
• Cost & latency — fine-tuned models can use smaller base models, reducing inference cost

Deploying a Fine-tuned Model

Once validated, deploy the fine-tuned model as a serverless endpoint or managed compute:

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://your-endpoint.openai.azure.com/",
    api_version="2024-08-01-preview",
)

response = client.chat.completions.create(
    model="gpt-4o-finetuned",  # your fine-tuned deployment name
    messages=[{"role": "user", "content": "Classify this support ticket: ..."}]
)

Using Fine-tuned Models with MAF Agents

In MAF, you can configure an agent to use any model by specifying the deployment in the model config:

const agent = new Agent({
  name: "MedicalCoder",
  instructions: "Classify diagnoses into ICD-10 codes.",
  model: {
    provider: "azure-openai",
    deployment: "gpt-4o-finetuned",
    endpoint: process.env.AZURE_OPENAI_ENDPOINT
  }
});