Model Insights

meta-llama-3.1-8b-instruct

Details

Developer

Model Performance Across Task-Types

Chainpoll Score

Short Context

0.89

Medium Context

Model Insights Across Task-Types

Digging deeper, here’s a look how meta-llama-3.1-8b-instruct performed across specific datasets

Short Context RAG

Medium Context RAG

This heatmap indicates the model's success in recalling information at different locations in the context. Green signifies success, while red indicates failure.

Performance Summary

Tasks	Dataset	Context adherence	Avg response length
Short context RAG	Drop	0.87	292.4
	Hotpot	0.88	188.7
	MS Marco	0.93	408.6
	ConvFinQA	0.88	435.3
Medium context RAG	Medium context RAG	1.00	NA
Long context RAG	Long context RAG	NaN	NA