Llama-2-70b-chat

The big brother in Llama 2 family of LLMs developed and publicly released by Meta. This model was pretrained on 2 trillion tokens of data from publicly available sources and fine-tuned on over one million human-annotated instruction datasets.

Model

Llama-2-70b-chat

Details

Developer

Model Performance Across Task-Types

Here's how Llama-2-70b-chat performed across all three task types

Metric

ChainPoll Score

QA without RAG

0.65

QA with RAG

0.68

Long form text generation

0.82

Model Info Across Task-Types

Digging deeper, here’s a look how Llama-2-70b-chat performed across specific datasets

Tasks	Insights	Dataset Name	Dataset Performance
QA without RAG	The model performs worse compared to GPT variants but shows best performance amongst open source models. It shows signs of bias and errors in its factual knowledge. The large number of parameters helps model be good at facts.	Truthful QA	0.53
QA without RAG		Trivia QA	0.77
QA with RAG	The model exhibits satisfactory performance which demonstrates good reasoning and comprehension skills. It struggles on mathematical skills as it scores relatively low on DROP compared to other dataset. It performs almost as good as 13b variant and GPT-3.5-turbo-instruct.	MS Marco	0.84
		Hotpot QA	0.61
		Drop	0.52
		Narrative QA	0.74
Long form text generation	The model excels at this task which demonstrates great ability to generate long text without factual errors.	Open Assistant	0.82

💰 Cost insights

The model offers a decent balance of cost and performance. It is 2x cheaper compared to GPT3.5 and 4x costlier compared to Llama 13b variant.

LLMHALLUCINATIONINDEXLLM