First Impressions of Early-Access GPT-4 Fine-Tuning

Supersimple has been using fine-tuned OpenAI models since the availability of GPT-3 Davinci for domain-specific use cases in natural language data questioning. The company recently gained access to the GPT-4 fine-tuning API and found that a fine-tuned GPT-4 outperforms fine-tuned GPT-3.5 by more than 50% in their specific use case. They have been using these models for answering users' natural language questions about data, aiming to provide an effective starting point for further deep dives into the data. The performance comparison shows that fine-tuned GPT-4 is slower and costlier than fine-tuned GPT-3.5 but offers significant improvements in accuracy. Despite these enhancements, models still struggle with broad and open-ended queries. To address this issue, Supersimple employs a mix of various specialized models, prompts, and heuristics to improve both accuracy and response time.