GPT-3.5-turbo-instruct

The OG GPT3 based instruct model from OpenAI.

Model

Details

Developer

OpenAI

License

Private

Model parameters

175B

Pretraining tokens

Release date

Nov 2022

Supported context length

Price for prompt tokens*

$1.5/Million tokens

Price for response tokens*

$2/Million tokens

*Note: Data based on 11/14/2023

Model Performance Across Task-Types

Here's how GPT-3.5-turbo-instruct performed across all three task types

Metric

ChainPoll Score

QA without RAG

0.70

QA with RAG

0.68

Long form text generation

0.74

Model Info Across Task-Types

Digging deeper, here’s a look how GPT-3.5-turbo-instruct performed across specific datasets

Tasks	Insights	Dataset Name	Dataset Performance
QA without RAG	The model performs well which show less bias and good factual knowledge.	Truthful QA	0.56
QA without RAG		Trivia QA	0.84
QA with RAG	The model peforms decently well which demonstrates good reasoning and comprehension skills. It struggles on mathematical skills as it scores relatively low on DROP compared to other dataset. It performs almost as good as Llama-2-70b-chat.	MS Marco	0.83
		Hotpot QA	0.64
		Drop	0.52
		Narrative QA	0.75
Long form text generation	The model performs satisfactory at this task which demonstrates good factual knowledge and ability to generate long text without errors.	Open Assistant	0.74

💰 Cost insights

The model offers a decent balance of cost and performance. It is 30x cheaper compared to GPT4 and 2x costlier compared to Llama 70b variant. OpenAI provides generous rate limits for production application. But we would suggest to use GPT3.5 instead of this.

LLMHALLUCINATIONINDEXLLM