LLM Hallucination Index RAG SpecialModel Insights
snowflake-arctic-instruct
Details
Developer
Snowflake
License
Apache 2.0
Model parameters
480B
Supported context length
4k
Price for prompt token
$2.4/Million tokens
Price for response token
$2.4/Million tokens
Model Performance Across Task-Types
Chainpoll Score
Short Context
0.85
Model Insights Across Task-Types
Digging deeper, here’s a look how snowflake-arctic-instruct performed across specific datasets
Short Context RAG
Performance Summary
| Tasks | Task insight | Cost insight | Dataset | Context adherence | Avg response length |
|---|---|---|---|---|---|
| Short context RAG | The model demonstrates good reasoning and comprehension skills, excelling at short context RAG. It also decent mathematical proficiency, as evidenced by its performance on DROP and ConvFinQA benchmarks. It could have performed better given its size and MoE architecture. | It is costly model for the performance it offers. We recomend using 25x cheaper Gemini Flash or 20x cheaper Haiku. You can also go ahead with 12x cheaper Llama-3-8b which offers a better performance at low cost. | Drop | 0.84 | 250 |
| Hotpot | 0.81 | 250 | |||
| MS Marco | 0.91 | 250 | |||
| ConvFinQA | 0.83 | 250 |