zoom_in Click to enlarge

Jamba

▲ ▼

language

description Jamba Overview

Jamba is a large language model released by AI21 Labs in March 2024, notable for its hybrid architecture combining Mamba state-space layers with Transformer attention layers in a mixture-of-experts configuration. The model has 52 billion total parameters across eight experts, of which approximately 12 billion are active per token. The hybrid design is intended to provide long-context processing with lower memory consumption than standard attention-only architectures. Released under an open-weight license for research, Jamba was the first production-scale model to use the Mamba architecture in a widely distributed form.

help Jamba FAQ

What makes AI21 Labs' Jamba model architecturally unique?

Jamba is the first production-scale model to hybridize Mamba (State Space Model or SSM) and traditional Transformer architectures. This design combines the two structures to allow for a much larger context window.

When was the Jamba language model released?

The model was released by AI21 Labs in March 2024. It marked a significant, commercial shift away from relying purely on standard Transformer architectures.

How many parameters does the Jamba model feature?

The model features a 52-billion-parameter architecture. Because of its hybrid SSM and Transformer design, it operates highly efficiently compared to standard models of that size.

Who created the Jamba AI model?

Jamba was created by AI21 Labs, a company well-known for its foundational work in natural language processing. They released the model as an open-weights project for the AI developer community.

swap_horiz

Looking for Jamba alternatives? Compare top competitors ranked & scored

arrow_forward