Open Source Alternative
Home / AI & LLM

vLLM

High-throughput LLM serving engine

vLLM is an inference and serving library for large language models. It uses paged attention and continuous batching to serve many requests on GPUs, and offers an OpenAI-compatible API server.

inferenceservinggpullm

More in AI & LLM

Featured

Ollama

AI & LLM

Run large language models on your own machine

vs OpenAI API, ChatGPTOpen sourcelocal-llminference

AnythingLLM

AI & LLM

All-in-one app for chatting with your documents

vs ChatGPT, Notion AIOpen sourceragdocuments

ComfyUI

AI & LLM

Node-based interface for image and video generation

vs Midjourney, DALL-EOpen sourceimage-generationdiffusion

GPT4All

AI & LLM

Local chat app for open language models

vs ChatGPTOpen sourcedesktoplocal-llm

Jan

AI & LLM

Offline ChatGPT-style desktop app

vs ChatGPTOpen sourcedesktoplocal-llm

Langfuse

AI & LLM

Tracing, evaluation and prompt management for LLM apps

vs LangSmith, HeliconeOpen sourceobservabilitytracing