Mixtral 8x7B (via local runner)

Mixtral 8x7B has roughly 47 billion total parameters, so running it at full precision would require around 90 GB of VRAM. With 4-bit quantization, you can run it on approximately 24–28 GB of VRAM, which puts it within reach of a single RTX 3090 or 4090, or a dual-GPU setup with smaller cards.

Mistral AI designed Mixtral 8x7B with 8 expert subnetworks, but only 2 experts are activated per token during inference. This means the effective compute per token is similar to a 13B model, while the total knowledge capacity approaches a much larger model.

Yes, you can run Mixtral 8x7B through a local inference server like llama.cpp, Ollama, or LM Studio and connect it to JetBrains IDEs via plugins that support OpenAI-compatible API endpoints. This gives you local code completion and chat without sending proprietary code to a cloud service.

Llama 3 70B generally outperforms Mixtral 8x7B on coding benchmarks because it's a dense model with more active parameters per token. Mixtral's advantage is that it runs faster and requires less VRAM due to its sparse activation, making it a practical trade-off if your hardware can't handle a full 70B dense model.