Best LLM Runner

LocalAI is a powerful and versatile local LLM runner built around the idea of seamless model management. It excels in its intuitive interface, offering granular control over model parameters like temp...

The Ollama Web UI offers an interactive web interface to run and experiment with local large language models. It’s notable for its ease of use and direct integration with the Ollama LLM runner. Develo...

LlamaFile is a compact software package designed to run large language models locally. It presents a simple, interactive interface within a single executable file across various platforms. This makes...

ExLlamaV2 is a specialized machine learning engine designed to accelerate the processing of Large Language Models like LLaMA. It’s notable for its speed and efficiency, particularly when utilizing GPU...

The Candle project offers a lightweight software solution built in Rust designed to execute Large Language Models (LLMs). It’s notable for its minimalist design and suitability for resource-constraine...

Koboldcpp is a minimalist C++ application designed for interactive fiction and roleplaying experiences. It provides an offline LLM runner based on llama.cpp, offering a streamlined interface suitable...

TabbyAPI is an open-source Python application that provides a locally hosted API server compatible with OpenAI’s interface. It facilitates development using LLMs from LM Studio, enabling offline exper...

The Aphrodite Engine is a machine-learning tool designed for local, offline deep learning experimentation. It’s notable for its support of tensor parallelism and PagedAttention, enabling the execution...

The RWKV Runner is a software tool designed to efficiently run RWKV large language models offline. It facilitates inference using both CPU and GPU hardware, providing a cross-platform solution for use...