LLM Hallucination Index RAG SpecialModel Insights
dbrx-instruct
Details
Developer
Databricks
License
Databricks Open Model License
Model parameters
132B
Supported context length
32k
Price for prompt token
$1.2/Million tokens
Price for response token
$1.2/Million tokens
Model Performance Across Task-Types
Chainpoll Score
Short Context
0.88
Medium Context
0.95
Model Insights Across Task-Types
Digging deeper, here’s a look how dbrx-instruct performed across specific datasets
Short Context RAG
Medium Context RAG
This heatmap indicates the model's success in recalling information at different locations in the context. Green signifies success, while red indicates failure.
Performance Summary
| Tasks | Task insight | Cost insight | Dataset | Context adherence | Avg response length |
|---|---|---|---|---|---|
| Short context RAG | The model demonstrates good reasoning and comprehension skills, excelling at short context RAG. It also decent mathematical proficiency, as evidenced by its performance on DROP and ConvFinQA benchmarks. It could have performed better for its size. | It is costly model for the performance it offers. We recomend using 12x cheaper Gemini Flash or 5x cheaper Haiku. | Drop | 0.86 | 408 |
| Hotpot | 0.85 | 408 | |||
| MS Marco | 0.90 | 408 | |||
| ConvFinQA | 0.89 | 408 | |||
| Medium context RAG | Great powerformance overall with minor problems for context more than 5000 tokens. | Good performance but we recommed using 12x cheaper Gemini Flash. | Medium context RAG | 0.95 | 408 |