Open Source Alternative
Home / AI & LLM

llama.cpp

C/C++ inference for LLMs on ordinary hardware

llama.cpp runs large language models in C and C++ with quantization to cut memory use. It works on CPUs and many GPUs, includes an OpenAI-compatible server, and underpins many local AI apps.

inferencequantizationc-plus-pluslocal-llm

More in AI & LLM

Featured

Ollama

AI & LLM

Run large language models on your own machine

vs OpenAI API, ChatGPTOpen sourcelocal-llminference

AnythingLLM

AI & LLM

All-in-one app for chatting with your documents

vs ChatGPT, Notion AIOpen sourceragdocuments

ComfyUI

AI & LLM

Node-based interface for image and video generation

vs Midjourney, DALL-EOpen sourceimage-generationdiffusion

GPT4All

AI & LLM

Local chat app for open language models

vs ChatGPTOpen sourcedesktoplocal-llm

Jan

AI & LLM

Offline ChatGPT-style desktop app

vs ChatGPTOpen sourcedesktoplocal-llm

Langfuse

AI & LLM

Tracing, evaluation and prompt management for LLM apps

vs LangSmith, HeliconeOpen sourceobservabilitytracing