Azure OpenAI Fine-Tuning ↗

Documentation Index#

Fetch the complete documentation index at: https://docs.wandb.ai/llms.txt Use this file to discover all available pages before exploring further.

How to Fine-Tune Azure OpenAI models using W&B.

Introduction#

Fine-tuning GPT-3.5 or GPT-4 models on Microsoft Azure using W&B tracks, analyzes, and improves model performance by automatically capturing metrics and facilitating systematic evaluation through W&B’s experiment tracking and evaluation tools.

Prerequisites#

• Set up Azure OpenAI service according to official Azure documentation.
• Configure a W&B account with an API key.

Workflow overview#

1. Fine-tuning setup#

• Prepare training data according to Azure OpenAI requirements.
• Configure the fine-tuning job in Azure OpenAI.
• W&B automatically tracks the fine-tuning process, logging metrics and hyperparameters.

2. Experiment tracking#

During fine-tuning, W&B captures:

• Training and validation metrics
• Model hyperparameters
• Resource utilization
• Training artifacts

3. Model evaluation#

After fine-tuning, use W&B Weave to:

• Evaluate model outputs against reference datasets
• Compare performance across different fine-tuning runs
• Analyze model behavior on specific test cases
• Make data-driven decisions for model selection

Real-world example#

• Explore the medical note generation demo to see how this integration facilitates:
  • Systematic tracking of fine-tuning experiments
  • Model evaluation using domain-specific metrics
• Go through an interactive demo of fine-tuning a notebook

Additional resources#

• Azure OpenAI W&B Integration Guide
• Azure OpenAI Fine-tuning Documentation