Databricks' open MoE model aimed at enterprise data and analytics use cases.
Specifications
- Provider
- Databricks
- Type
- Open-source / open-weight
- Modality
- Text (MoE)
- Category
- Language model (MoE)
- Context window
- 32K
- Parameters
- 132B total, ~36B active
- License
- Databricks Open License
- Released
- March 27, 2024
What it was trained for
General-purpose enterprise text and code generation using a fine-grained mixture-of-experts architecture.
Best for
- ▸Enterprise text generation
- ▸Code generation
- ▸Self-hosted open deployments
- ▸RAG and assistant applications
Capabilities
Text generationCode generationInstruction followingLong-context processing
Performance & positioning
Activates only a subset of experts per token for faster inference relative to dense models of comparable size; strong on general language and programming tasks.