Fine-tuning - OpenAI API
English
OpenAI offers comprehensive documentation and guides for fine-tuning its models, allowing developers to customize pre-trained models for specific tasks and domains. This process can lead to higher quality results, more consistent output, and reduced costs compared to prompt engineering alone.
Here's a general overview of the OpenAI fine-tuning process:
- Understand Fine-tuning Fine-tuning involves taking a pre-trained OpenAI model (like GPT-3.5 or GPT-4) and further training it on a smaller, specialized dataset. This adjusts the model's internal parameters to better align with the patterns and requirements of your target application.
- Build Your Training Dataset The most crucial and time-consuming step is preparing a high-quality dataset of examples. This dataset should contain example prompts and their desired model outputs. OpenAI recommends having at least 200 training examples for good performance.
- Upload Training Data Once your dataset is ready, you upload it to OpenAI, typically in JSONL format.
- Create a Fine-tuning Job You then create a fine-tuning job, specifying a base model (e.g.,
gpt-4o-mini) and your uploaded training file. You can create fine-tuned models either through the OpenAI dashboard or via the API. - Evaluate Your Results After the training process, which can take minutes to hours depending on the dataset size, you can evaluate your fine-tuned model. It's recommended to split your dataset into training and test portions to evaluate the model's performance against a benchmark.
- Iterate and Optimize If the results are not satisfactory, you can iterate on data quality and quantity, and adjust hyperparameters.
Key benefits of fine-tuning include:
- Achieving higher accuracy on niche tasks.
- Enabling the model to consistently format responses in a specific way.
- Reducing the length and cost of prompts.
- Handling more examples than can fit within a single prompt's context window.
- Incorporating domain-specific data, avoiding the need for expensive Retrieval-Augmented Generation (RAG) workflows.
OpenAI supports different fine-tuning methods, including supervised fine-tuning, and has also introduced vision fine-tuning on models like GPT-4o, allowing fine-tuning with images in addition to text.
For more detailed information, you can refer to the official OpenAI documentation:
- Supervised fine-tuning overview
- Model optimization
- Fine-tuning best practices
- Fine-tuning guide (comprehensive)
中文
OpenAI 为其模型提供全面的文档和指南，允许开发者针对特定任务和领域定制预训练模型。与仅依赖提示工程相比，这个过程可以带来更高质量的结果、更一致的输出和更低的成本。
以下是 OpenAI 微调过程的概述：
- 了解微调 微调涉及使用预训练的 OpenAI 模型（如 GPT-3.5 或 GPT-4）并在更小、更专业的数据集上进行进一步训练。这会调整模型的内部参数，使其更好地与目标应用程序的模式和要求保持一致。
- 构建训练数据集 最关键且耗时的步骤是准备高质量的示例数据集。该数据集应包含示例提示及其所需的模型输出。OpenAI 建议至少有 200 个训练示例以获得良好性能。
- 上传训练数据 数据集准备就绪后，将其上传到 OpenAI，通常使用 JSONL 格式。
- 创建微调作业 然后创建微调作业，指定基础模型（例如
gpt-4o-mini）和上传的训练文件。可以通过 OpenAI 仪表板或 API 创建微调模型。 - 评估结果 训练过程可能需要几分钟到几小时，具体取决于数据集大小，之后可以评估微调后的模型。建议将数据集分为训练和测试部分，以根据基准评估模型的性能。
- 迭代和优化 如果结果不令人满意，可以迭代数据的质量和数量，并调整超参数。
微调的主要优势包括：
- 在特定任务上实现更高的准确性。
- 使模型能够以特定方式一致地格式化响应。
- 减少提示的长度和成本。
- 处理超出单个提示上下文窗口限制的更多示例。
- 整合领域特定数据，避免昂贵的信息检索增强生成（RAG）工作流程。
OpenAI 支持不同的微调方法，包括监督微调，并在 GPT-4o 等模型上引入了视觉微调，允许除了文本外还可以使用图像进行微调。
有关更详细的信息，可以参考官方 OpenAI 文档：
- 监督微调概述
- 模型优化
- 微调最佳实践
- 微调指南（综合版）