AGNT/Articles/Guide
Guide

The 50 Best Local AI Agents in 2026

50 local and self-hosted AI agents compared by execution, model, storage, telemetry, platform, license, and practical limits.

Contents

"Local AI agent" gets used to mean three different things, and the difference matters more than any feature list.

The first meaning is local inference: the model weights sit on your disk and the tokens are generated by your CPU or GPU. The second is self-hosting: the application runs on infrastructure you control, but the model it calls may live anywhere. The third is local orchestration: the agent loop, the tool calls, and the file access happen on your machine, while the reasoning is done by a remote API.

Most tools marketed as "local" are the second or third kind. That is not a scandal — a self-hosted agent with a remote model is still a meaningful privacy and control improvement over a fully managed SaaS agent. But it is not offline, and a compliance reviewer who is told otherwise will find out during the audit rather than during procurement.

This list covers 50 projects that support genuine local execution or self-hosting. Every entry labels four boundaries explicitly:

  • Execution boundary — where the agent loop, tool calls, and file access run.
  • Model boundary — where the weights live and where the forward pass happens.
  • Storage boundary — where conversations, embeddings, and artifacts are written.
  • Telemetry boundary — what leaves the machine that is not a model call.

How to read this list

On telemetry. The telemetry lines below describe documented and observable network behavior, not independent audits. Nobody in this list has been packet-captured by us under controlled conditions and given a clean bill of health. Where a project's telemetry posture is not documented somewhere I can point at, the line says so instead of guessing. Use the privacy verification checklist below to confirm anything that matters to you.

On licenses. Licenses were checked against each project's stated license at the time of writing. Several projects in this list use source-available licenses that people routinely miscall "open source" — n8n's Sustainable Use License, Open WebUI's branding-restricted BSD variant, AutoGPT's PolyForm Shield split. Those distinctions are called out where they exist. Re-check the LICENSE file at the source URL before you redistribute, resell, or build a commercial product on top of any of them.

On hardware. No throughput numbers appear in this article. Tokens per second depends on your quantization, context length, batch size, backend, thermal envelope, and driver version, and a number measured on someone else's laptop is worse than no number. The hardware guidance section gives memory ranges derived from weight arithmetic, which is the part that actually generalizes.

If you want the broader open-source field rather than the local-execution slice specifically, read the best open-source AI agents. If you are deciding how to structure the agent itself rather than which one to install, read the AI agent architectures guide.

Part 1 — Local model runtimes

These are the substrate. Almost every other entry in this article is either built on one of them or can be pointed at one. If you get this layer right, "local" becomes a configuration choice everywhere else.

1. Ollama

A single binary that pulls quantized models from a registry, manages them like container images, and exposes both a native REST API on localhost:11434 and an OpenAI-compatible endpoint. Ships with a launch subcommand that wires coding agents and assistants directly to a local model, and now includes its own agent package in-tree.

  • Platforms: macOS, Windows, Linux, Docker
  • Local components: model daemon, registry client, REST server, CLI, first-party Python and JS SDKs
  • Model options: GGUF via llama.cpp; MLX on Apple Silicon; imports from Hugging Face and local Modelfile definitions
  • Execution boundary: entirely local process
  • Model boundary: weights stored and executed locally after download
  • Storage boundary: models and manifests on local disk; the daemon holds no conversation history
  • Telemetry boundary: model pulls contact Ollama's registry over the network. Once a model is on disk, the inference path does not require connectivity. Version-check behavior and any additional reporting should be confirmed against the current source.
  • Best fit: the default local backend for everything else on this list
  • Limitation: the registry is a centralized dependency for discovery, and multi-GPU and long-context serving are less tunable than a dedicated inference server
  • License: MIT
  • Source: https://github.com/ollama/ollama

2. llama.cpp

The C/C++ inference engine that most of this category is built on, including Ollama's default backend. Runs GGUF-quantized models on CPU, CUDA, ROCm, Metal, Vulkan, and SYCL, and ships llama-server with an OpenAI-compatible API plus a built-in web UI.

  • Platforms: macOS, Windows, Linux, BSD, Android (via termux), plus embedded targets
  • Local components: inference library, llama-cli, llama-server, quantization and conversion tooling, GBNF grammar-constrained decoding
  • Model options: any GGUF-convertible architecture the project supports; conversion scripts included
  • Execution boundary: local process, no daemon required
  • Model boundary: fully local
  • Storage boundary: none beyond the model files and whatever your client writes
  • Telemetry boundary: the library and server do not require network access at inference time. Model acquisition is a separate step you perform yourself.
  • Best fit: teams that want the smallest possible dependency surface and direct control over sampling, grammars, and offload splits
  • Limitation: it is infrastructure, not a product. Model management, chat history, and tool orchestration are your problem.
  • License: MIT
  • Source: https://github.com/ggml-org/llama.cpp

3. LM Studio

A desktop application for downloading, configuring, and serving local models, with an OpenAI-compatible server, an MCP client, Python and TypeScript SDKs, a lms CLI, and a headless mode (llmster) for servers and CI. Its agent layer, Bionic, runs tool-using sessions against local models.

  • Platforms: macOS (Apple Silicon), Windows x64/ARM64, Linux x64
  • Local components: GUI, headless server, model manager, MCP client, RAG over attached documents, SDKs
  • Model options: GGUF via llama.cpp on all platforms; MLX additionally on Apple Silicon; discovery and download through Hugging Face
  • Execution boundary: local application and local server; MCP servers you connect may reach out on their own
  • Model boundary: fully local once weights are downloaded
  • Storage boundary: models, chats, and configurations on local disk
  • Telemetry boundary: the application documents that it can operate entirely offline once you have model files. It is closed source, so offline capability is a documented property rather than an auditable one. Any MCP server you install has its own independent network behavior.
  • Best fit: the fastest path from "no local models" to "OpenAI-compatible endpoint on my laptop," especially on Apple Silicon
  • Limitation: proprietary and unauditable; you are trusting the vendor's offline claim rather than verifying the code
  • License: Custom source-available license; not OSI-approved open source. Free to download; commercial-use terms are set by the vendor and should be read before deployment.
  • Source: https://lmstudio.ai/docs/app

4. Jan

An open-source desktop assistant that runs models locally and can also be pointed at remote providers. Positioned as an offline-first alternative to hosted chat apps, with a local API server for other applications to consume.

  • Platforms: macOS, Windows, Linux
  • Local components: desktop app, local inference engine, local API server, extension system
  • Model options: GGUF models downloaded in-app; optional remote provider connections you configure explicitly
  • Execution boundary: local application
  • Model boundary: local by default; becomes remote the moment you add a cloud provider key
  • Storage boundary: threads and models in a local application data folder
  • Telemetry boundary: the project is open source, so the analytics path is inspectable. Confirm the current opt-out mechanism in the repository rather than assuming a default.
  • Best fit: a general-purpose local chat assistant for a non-technical user on a single machine
  • Limitation: thinner agentic tooling than the coding-agent category; treat it as an assistant with a local API rather than a task-completing agent
  • License: Apache 2.0
  • Source: https://github.com/menloresearch/jan

5. LocalAI

A drop-in OpenAI API replacement that runs on consumer hardware and covers more than text: chat completions, embeddings, image generation, audio transcription, and text-to-speech, all behind familiar endpoints. Designed to be deployed as a container in your own infrastructure.

  • Platforms: Linux, macOS, Windows via Docker; Kubernetes via community charts
  • Local components: API server, multiple inference backends, model gallery, image and audio pipelines
  • Model options: GGUF, transformers, diffusion models, Whisper-family speech models, various TTS backends
  • Execution boundary: self-hosted server, typically containerized
  • Model boundary: local; models pulled once from a gallery or supplied directly
  • Storage boundary: models and generated artifacts in mounted volumes you control
  • Telemetry boundary: no external inference dependency by design. Gallery installs fetch from remote sources at install time.
  • Best fit: replacing a cloud OpenAI dependency across an entire internal stack without rewriting client code
  • Limitation: broad surface means more moving parts; backend selection and GPU acceleration take real configuration effort
  • License: MIT
  • Source: https://github.com/mudler/LocalAI

6. vLLM

A high-throughput inference and serving engine built around PagedAttention and continuous batching. This is what you run when a local agent stops being one person's laptop and becomes a service several agents hit concurrently.

  • Platforms: Linux with NVIDIA CUDA primarily; AMD ROCm, Intel, and other backends at varying maturity
  • Local components: OpenAI-compatible API server, offline batch inference API, tensor and pipeline parallel serving
  • Model options: unquantized and quantized Hugging Face transformer checkpoints across a wide supported-architecture list
  • Execution boundary: self-hosted server in your own datacenter, VPC, or workstation
  • Model boundary: local to the serving host
  • Storage boundary: model cache on the serving host; no conversation persistence
  • Telemetry boundary: vLLM has shipped usage statistics collection with a documented opt-out environment variable. Set it explicitly in regulated deployments rather than relying on defaults.
  • Best fit: multi-user or multi-agent workloads where request concurrency, not single-stream latency, is the constraint
  • Limitation: GPU-class hardware assumed; not a laptop tool, and not a chat application
  • License: Apache 2.0
  • Source: https://github.com/vllm-project/vllm

7. SGLang

A serving framework built for structured generation and complex prompting patterns — multi-turn agent loops, constrained JSON output, and heavy prefix reuse across parallel calls. Its RadixAttention prefix cache is directly relevant to agents that re-send a large fixed system prompt on every step.

  • Platforms: Linux with NVIDIA CUDA primarily; additional backends in progress
  • Local components: runtime, OpenAI-compatible server, structured-output front-end language
  • Model options: Hugging Face transformer checkpoints; quantization support varies by architecture
  • Execution boundary: self-hosted server
  • Model boundary: local to the serving host
  • Storage boundary: model cache and KV cache on the serving host
  • Telemetry boundary: open source; verify current instrumentation defaults in the repository
  • Best fit: agent systems where every step reuses a long shared prefix and output must conform to a schema
  • Limitation: narrower hardware and model coverage than vLLM; you are optimizing for a specific shape of workload
  • License: Apache 2.0
  • Source: https://github.com/sgl-project/sglang

8. llamafile

Packages model weights and the llama.cpp inference engine into a single executable file that runs on multiple operating systems and CPU architectures without an installer, using the Cosmopolitan Libc build approach. The most portable form a local model takes.

  • Platforms: macOS, Windows, Linux, FreeBSD, OpenBSD, NetBSD on x86-64 and ARM64
  • Local components: the executable itself, which contains the weights, the runtime, and a built-in server and web UI
  • Model options: GGUF weights, either embedded in the file or supplied externally at runtime
  • Execution boundary: a single local process
  • Model boundary: fully local; the weights are inside the artifact
  • Storage boundary: none by default beyond the file
  • Telemetry boundary: no network dependency for inference. This is the easiest entry on the list to verify, because you can run it with networking disabled entirely.
  • Best fit: air-gapped machines, USB-stick deployment, reproducible archival of a specific model plus a specific runtime
  • Limitation: Windows caps individual executable size, which affects how large a fully embedded llamafile can be; large models are usually run with external weights
  • License: Apache 2.0 (with LLVM exceptions applying to portions of the build tooling)
  • Source: https://github.com/Mozilla-Ocho/llamafile

9. KoboldCpp

A single-file llama.cpp derivative focused on long-context, persistent-character, and story workloads, with an OpenAI-compatible API, context shifting, and a bundled web UI. Heavily used as a backend for local roleplay and simulation front-ends.

  • Platforms: Windows, Linux, macOS; CUDA, ROCm, Vulkan, and CPU builds
  • Local components: inference server, Kobold Lite web UI, image and speech features in some builds
  • Model options: GGUF text models; several builds bundle support for image generation and speech backends
  • Execution boundary: local process
  • Model boundary: fully local
  • Storage boundary: local save files for stories, characters, and settings
  • Telemetry boundary: no external inference dependency. Optional tunneling features exist and are opt-in — do not enable them on a machine you are treating as isolated.
  • Best fit: long-running conversational agents with persistent state and very long contexts
  • Limitation: AGPL-3.0 has real consequences if you expose a modified version as a network service
  • License: AGPL-3.0
  • Source: https://github.com/LostRuins/koboldcpp

10. GPT4All

A desktop application from Nomic for running local models on ordinary laptops and desktops, with a LocalDocs feature for retrieval over your own files and a Python client wrapping llama.cpp.

  • Platforms: Windows (x64 and ARM), macOS, Ubuntu/Linux x86-64
  • Local components: desktop chat client, LocalDocs retrieval, model downloader, Python bindings, optional Docker API server
  • Model options: GGUF models from a curated in-app catalog or supplied manually
  • Execution boundary: local application
  • Model boundary: fully local; the project's stated position is no API calls and no GPU required
  • Storage boundary: models, chats, and LocalDocs indexes on local disk
  • Telemetry boundary: the application has historically included an opt-in analytics and datalake prompt on first run. Read that dialog carefully rather than clicking through it.
  • Best fit: non-technical users on modest hardware who want private document chat with no terminal
  • Limitation: the public release cadence has slowed considerably, with v3.10.0 as the most recent tagged release shown on the repository. Treat it as stable rather than actively evolving, and check activity before adopting it for something long-lived.
  • License: MIT
  • Source: https://github.com/nomic-ai/gpt4all

Part 2 — Local coding agents

This is where "local" gets slippery. Almost every agent below can run against a local model, and almost every one of them defaults to a cloud API. The distinction is called out per entry, because a coding agent that reads your entire repository is exactly where the difference has consequences.

11. Aider

A terminal coding agent that works directly on a Git repository, builds a repository map to give the model structural context, and commits each change as a discrete commit with a generated message. The Git-native design makes every action reviewable and revertible.

  • Platforms: macOS, Windows, Linux (Python package)
  • Local components: CLI, repository mapping, diff application, Git integration, local model configuration
  • Model options: any provider through a unified LLM interface, including Ollama and any OpenAI-compatible local server
  • Execution boundary: local — the agent loop, file edits, and commits all happen on your machine
  • Model boundary: configurable and remote by default. Point it at a local endpoint or your source goes to whichever provider you configured.
  • Storage boundary: chat history and cached maps inside the repository working directory
  • Telemetry boundary: the project has included an analytics mechanism with a documented disable flag. Set it explicitly.
  • Best fit: developers who want every AI edit to arrive as a reviewable commit rather than an opaque workspace mutation
  • Limitation: local models with weak instruction-following produce malformed diffs that fail to apply; the experience degrades sharply below a certain model capability
  • License: Apache 2.0
  • Source: https://github.com/Aider-AI/aider

12. Cline

A VS Code extension that runs a plan-then-act agent across multiple files, executes terminal commands with approval, and supports MCP servers for tool extension. Approval gates are per-action by default, which makes it usable with models you do not fully trust.

  • Platforms: anywhere VS Code runs — macOS, Windows, Linux
  • Local components: extension host process, file editing, terminal execution, MCP client, checkpointing
  • Model options: many providers, including Ollama, LM Studio, and any OpenAI-compatible local endpoint
  • Execution boundary: local — commands run on your machine with your permissions
  • Model boundary: provider-dependent. Cloud by default configuration; genuinely local when pointed at a local server.
  • Storage boundary: task history and checkpoints in VS Code extension storage
  • Telemetry boundary: open source with a telemetry setting exposed in the extension configuration; set it before first use
  • Best fit: multi-file refactors inside an editor where you want to approve each shell command
  • Limitation: agentic coding is context-hungry, and small local models with short effective context struggle on repositories of any real size
  • License: Apache 2.0
  • Source: https://github.com/cline/cline

13. Continue

An open-source AI code assistant for VS Code and JetBrains that separates chat, autocomplete, and edit into distinct configurable roles, each of which can be bound to a different model. Local autocomplete with cloud chat is a common and sensible split.

  • Platforms: VS Code and JetBrains IDEs on macOS, Windows, Linux
  • Local components: IDE extension, local indexing, context providers, per-role model configuration
  • Model options: Ollama, llama.cpp, LM Studio, vLLM, and any OpenAI-compatible endpoint, alongside commercial providers
  • Execution boundary: local IDE process
  • Model boundary: fully configurable per role. A local-only configuration is straightforward and explicitly supported.
  • Storage boundary: local index and session data in the extension data directory
  • Telemetry boundary: anonymous telemetry has been on by default historically with a documented opt-out in the configuration file. Set it in the config, not in your head.
  • Best fit: teams that want a small fast local model for completions and a stronger model only where it earns its keep
  • Limitation: the configuration surface is large, and a partially configured setup can silently route to a cloud provider you did not intend
  • License: Apache 2.0
  • Source: https://github.com/continuedev/continue

14. OpenHands

A platform for agents that do the things a developer does — write code, run commands, browse the web, call APIs — with execution isolated inside a sandboxed runtime container. The sandbox is the point: agent-executed code does not touch the host directly.

  • Platforms: Linux, macOS, Windows via Docker
  • Local components: agent server, sandboxed runtime container, browser tool, event stream, web UI
  • Model options: any provider through a unified LLM layer, including local Ollama and OpenAI-compatible servers
  • Execution boundary: local, and additionally sandboxed — a meaningful safety property, not a marketing line
  • Model boundary: provider-dependent. The docs are candid that weaker local models degrade agent success rates.
  • Storage boundary: workspace volumes and event logs on the host you designate
  • Telemetry boundary: self-hosted deployment has no mandatory external reporting; the browser tool reaches the open internet whenever the agent uses it
  • Best fit: letting an agent run arbitrary commands without giving it your actual shell
  • Limitation: Docker is required, and long autonomous runs consume a lot of tokens regardless of where the model lives
  • License: MIT
  • Source: https://github.com/All-Hands-AI/OpenHands

15. Goose

Block's open-source agent, available as a CLI and desktop app, built around extensions that are MCP servers. It installs, executes, edits, and tests on your machine rather than in a hosted workspace.

  • Platforms: macOS, Linux, Windows
  • Local components: CLI, desktop application, MCP extension system, session persistence, recipes
  • Model options: many providers including Ollama and other local OpenAI-compatible endpoints
  • Execution boundary: local machine, with your permissions
  • Model boundary: configurable. Local model support is documented; tool-calling reliability is the limiting factor.
  • Storage boundary: sessions and configuration in a local dotfile directory
  • Telemetry boundary: open source; review the current instrumentation and any opt-out in the repository before enterprise rollout
  • Best fit: general-purpose desktop automation where MCP is already your tool protocol
  • Limitation: it takes real actions on a real machine with real permissions. Scope what it can reach.
  • License: Apache 2.0
  • Source: https://github.com/block/goose

16. Void

An open-source fork of VS Code positioned as an auditable alternative to closed AI editors, with agent mode, checkpoints, and direct local model connection. Because it is a fork rather than an extension, the AI integration reaches deeper into the editor.

  • Platforms: macOS, Windows, Linux
  • Local components: full editor, agent mode, indexing, direct connections to local model servers
  • Model options: Ollama, vLLM, LM Studio, llama.cpp, plus commercial providers
  • Execution boundary: local editor process
  • Model boundary: direct local connection supported without an intermediate broker — the specific property that distinguishes it from closed editors
  • Storage boundary: editor workspace and local settings
  • Telemetry boundary: the stated design goal is that requests go directly to your chosen provider rather than through a vendor backend. Being open source, this is verifiable in the code.
  • Best fit: developers who want a Cursor-style experience with no vendor in the request path
  • Limitation: a VS Code fork lags upstream and has a smaller extension compatibility guarantee than the real thing
  • License: Apache 2.0
  • Source: https://github.com/voideditor/void

17. Tabby

A self-hosted coding assistant server providing completion and chat to IDE clients, with a Rust core, built-in code indexing, and OpenAPI endpoints. Designed to be the one AI service a team runs on-premises for the whole engineering org.

  • Platforms: Linux, macOS, Windows; Docker and bare metal; NVIDIA and Apple Silicon acceleration
  • Local components: inference server, code index over your repositories, admin UI, IDE extensions for VS Code, JetBrains, Vim/Neovim
  • Model options: curated code and chat models served locally; can also proxy to external endpoints if configured
  • Execution boundary: self-hosted server plus thin IDE clients
  • Model boundary: local to your server in the standard deployment
  • Storage boundary: index, users, and completion events in server-side storage you control
  • Telemetry boundary: the project has documented anonymous usage collection with a documented disable flag for self-hosted instances. Set it at deployment time.
  • Best fit: an engineering org that needs completions for everyone without source code leaving the building
  • Limitation: you are now operating an inference service — capacity planning, GPU allocation, and upgrades are yours
  • License: Apache 2.0
  • Source: https://github.com/TabbyML/tabby

18. OpenCode

A terminal-native coding agent with a client/server split, so the agent can run on one machine and be driven from another — including from a phone. Provider-agnostic by design rather than tied to a single model vendor.

  • Platforms: macOS, Linux, Windows; terminal-first
  • Local components: TUI client, agent server, LSP integration, session management
  • Model options: many providers through a unified interface; Ollama integration is documented on the Ollama side as a supported launch target
  • Execution boundary: local, with the server optionally on a different local machine you own
  • Model boundary: provider-dependent. Local when configured against a local endpoint.
  • Storage boundary: sessions on the machine running the server
  • Telemetry boundary: open source; confirm current defaults in the repository
  • Best fit: developers who live in a terminal and want to detach and reattach agent sessions
  • Limitation: the client/server design means an exposed server port is a real attack surface — bind it to loopback unless you have a reason not to
  • License: MIT
  • Source: https://github.com/sst/opencode

19. Refact.ai

A self-hostable coding assistant covering autocomplete, chat, and an agent mode, with a server you deploy yourself and IDE plugins for VS Code and JetBrains. Fine-tuning on your own codebase is a first-class feature rather than an afterthought.

  • Platforms: Linux server with GPU acceleration; Docker; IDE plugins cross-platform
  • Local components: self-hosted inference and orchestration server, IDE plugins, fine-tuning pipeline
  • Model options: a set of supported open-weight code models served locally; external providers configurable
  • Execution boundary: self-hosted server plus local IDE plugins
  • Model boundary: local when self-hosted; the hosted cloud offering is a separate product with a different boundary
  • Storage boundary: code index, telemetry, and fine-tuning data on your server
  • Telemetry boundary: the self-hosted deployment keeps usage data on your instance. The cloud offering does not. Be certain which one you installed.
  • Best fit: organizations that want a code model adapted to their own repositories without shipping those repositories anywhere
  • Limitation: self-hosted fine-tuning needs GPU capacity and an operator who wants the job
  • License: BSD-3-Clause
  • Source: https://github.com/smallcloudai/refact

20. twinny

A VS Code extension built specifically for local inference, doing fill-in-the-middle completion and chat against Ollama or llama.cpp with no account, no signup, and no telemetry gateway between the editor and the model.

  • Platforms: VS Code on macOS, Windows, Linux
  • Local components: extension, FIM completion, chat panel, symmetry peer-to-peer inference sharing
  • Model options: any model served by Ollama or llama.cpp, including small FIM-capable code models
  • Execution boundary: local editor process
  • Model boundary: local by default — this is the design premise rather than a supported option
  • Storage boundary: local extension storage
  • Telemetry boundary: the project positions itself as private and free with no data collection. Being open source and small, it is one of the easier extensions to actually read.
  • Best fit: the minimal viable private Copilot replacement on a single developer machine
  • Limitation: a smaller feature set than Continue or Cline, and the peer-to-peer sharing feature changes the privacy boundary entirely if you enable it
  • License: MIT
  • Source: https://github.com/twinnydotdev/twinny

21. Open Interpreter

Runs model-generated code on your computer — Python, JavaScript, shell — inside a local session with a confirmation step, giving a natural-language interface to the machine itself rather than to a repository.

  • Platforms: macOS, Windows, Linux
  • Local components: CLI, code execution across multiple languages, filesystem access, optional local model configuration
  • Model options: any provider through a unified interface; documented setup for Ollama and other local models
  • Execution boundary: local, unsandboxed by default, running with your user's permissions
  • Model boundary: provider-dependent. Local-model instructions are maintained in the documentation.
  • Storage boundary: conversations in a local application directory
  • Telemetry boundary: open source; check the current defaults before enterprise use
  • Best fit: ad-hoc data wrangling, file manipulation, and system tasks described in plain language
  • Limitation: unsandboxed execution is a genuine risk. Approve every command or run it in a VM.
  • License: AGPL-3.0
  • Source: https://github.com/openinterpreter/open-interpreter

22. gptel

An LLM client for Emacs that works in any buffer, supports tool calling and multiple backends including local servers, and keeps the entire interaction in plain text inside your editor. Async by default, so a slow local model never blocks the editor.

  • Platforms: anywhere Emacs runs
  • Local components: Emacs package, buffer-based interaction, tool definitions in Elisp, per-buffer backend selection
  • Model options: Ollama, llama.cpp, GPT4All, and other local backends alongside commercial APIs
  • Execution boundary: local Emacs process
  • Model boundary: configurable per buffer — you can genuinely run one buffer local and another remote
  • Storage boundary: whatever files you save; conversations are just buffers
  • Telemetry boundary: none beyond the model requests you configure
  • Best fit: Emacs users who want local model access woven into existing workflows rather than bolted on
  • Limitation: Emacs-only, and tool calling depends on the backend and model supporting it properly
  • License: GPL-3.0
  • Source: https://github.com/karthink/gptel

Part 3 — Desktop assistants and chat frontends

Chat interfaces have become agent hosts. Most of these support tool calling, MCP servers, and document retrieval, which puts them in scope even though "chat UI" undersells what they do.

23. AGNT

Disclosure: AGNT is our product. It is included because its runtime, tools, workspace, and memory run on the user’s machine; its source-available licensing boundary is stated below.

A desktop agent platform where agents, workflows, skills, and tools run as processes on your own machine against a local workspace directory, with the model provider configurable per agent. The orchestration layer, file access, and execution all stay on the device; the reasoning goes wherever you point it.

  • Platforms: Windows, macOS, Linux, Docker, and headless server — download AGNT
  • Local components: agent runtime, workflow engine, tool and plugin system, local workspace directory, persistent memory, local API on loopback
  • Model options: provider-configurable, including local backends such as Ollama and LM Studio alongside commercial APIs
  • Execution boundary: local desktop application; tools, shell commands, and file operations execute on your machine
  • Model boundary: determined entirely by the provider you configure. A local provider keeps inference on-device; a cloud key does not.
  • Storage boundary: projects, conversations, agent memory, and generated artifacts under a local application data directory
  • Telemetry boundary: source is public, but the license is custom rather than OSI-approved. Provider routing is configurable; verify egress with the checklist below if it matters for your use case.
  • Best fit: running persistent agents, visual workflows, goals, skills, plugins, MCP tools, and evaluations against local files under one inspectable runtime
  • Limitation: designed for a trusted local workspace or small team rather than public multi-tenant SaaS isolation; agent capability still tracks the model you select
  • License: Proprietary
  • Source: https://github.com/agnt-gg/agnt · https://agnt.gg

24. Open WebUI

The most widely deployed self-hosted chat interface for local models. Covers RAG over uploaded documents, web search integration, multi-user access control, model management, and a pipelines system for custom logic.

  • Platforms: Docker, Kubernetes, Python package; any host that runs those
  • Local components: web application, user and permission management, document RAG, vector store, pipelines framework
  • Model options: Ollama natively, plus any OpenAI-compatible endpoint including local servers
  • Execution boundary: self-hosted server
  • Model boundary: local when backed by Ollama or a local OpenAI-compatible server
  • Storage boundary: users, chats, documents, and embeddings in the instance database and volumes
  • Telemetry boundary: the deployment documents environment variables to disable outbound features. Note that default configurations may attempt to reach external services for model lists or update checks — audit your compose file before treating an instance as isolated.
  • Best fit: giving a whole team a shared private ChatGPT-equivalent over local models
  • Limitation: read the license before deploying. It is a BSD-3-derived license with a branding clause: you may not alter or remove Open WebUI branding above 50 end users in a rolling 30-day period without written permission or an enterprise license. Calling this "MIT-like" in a procurement document is a mistake.
  • License: Open WebUI License — BSD-3-Clause terms plus a binding branding-preservation condition
  • Source: https://github.com/open-webui/open-webui

25. AnythingLLM

An all-in-one desktop and Docker application that turns documents into a queryable workspace, with agent capabilities, MCP support, and a built-in embedder and vector database so a working install needs no external services.

  • Platforms: macOS, Windows, Linux desktop; Docker for multi-user server deployment
  • Local components: application, built-in LLM provider, built-in embedding model, built-in vector database, document processor, agent framework, MCP client
  • Model options: a bundled local provider, plus Ollama, LM Studio, llama.cpp, and commercial APIs
  • Execution boundary: local desktop or self-hosted server
  • Model boundary: fully local out of the box — this is the notable property. It works with zero external configuration.
  • Storage boundary: documents, vectors, and chats in a local application directory or mounted volume
  • Telemetry boundary: the project documents anonymous telemetry with a documented environment-variable disable. Set it if the deployment is sensitive.
  • Best fit: the shortest path to private document chat for someone who does not want to assemble a stack
  • Limitation: the bundled defaults are convenient rather than optimal; heavier workloads want a dedicated embedder and vector store
  • License: MIT
  • Source: https://github.com/Mintplex-Labs/anything-llm

26. LibreChat

A self-hosted multi-provider chat platform with conversation branching, artifacts, code interpreter integration, agent definitions, MCP support, and per-user model access control. Built for organizations rather than individuals.

  • Platforms: Docker, Kubernetes, Node.js
  • Local components: web app, API server, MongoDB, optional Meilisearch, RAG API, agent definitions
  • Model options: Ollama and any OpenAI-compatible local endpoint, plus every major commercial provider
  • Execution boundary: self-hosted server
  • Model boundary: local when configured against local endpoints; the point of the product is that it is provider-agnostic
  • Storage boundary: conversations, files, and search index in your own database and volumes
  • Telemetry boundary: self-hosted with no mandatory external reporting; some optional integrations are external services by definition
  • Best fit: an organization that needs one governed chat surface across a mix of local and commercial models
  • Limitation: several moving services to operate — this is a real deployment, not a single container
  • License: MIT
  • Source: https://github.com/danny-avila/LibreChat

27. Lobe Chat

A modern chat framework with a plugin ecosystem, function calling, multimodal support, knowledge base features, and one-click self-hosted deployment. Strong design quality relative to the category.

  • Platforms: Docker, Node.js, Vercel, desktop builds
  • Local components: web application, plugin runtime, knowledge base, local database mode
  • Model options: Ollama and OpenAI-compatible local endpoints alongside many commercial providers
  • Execution boundary: self-hosted server or desktop
  • Model boundary: local when pointed at a local endpoint
  • Storage boundary: browser-local or server database depending on deployment mode; choose deliberately, because the two have different privacy characteristics
  • Telemetry boundary: self-hosted deployments have no mandatory external dependency, but individual plugins are third-party network calls
  • Best fit: teams that care about interface quality and want a plugin ecosystem
  • Limitation: the license carries supplemental terms beyond stock Apache 2.0 covering branding and commercial use — read them before shipping a rebranded product
  • License: Apache 2.0 with supplemental terms in the repository
  • Source: https://github.com/lobehub/lobe-chat

28. Khoj

A self-hostable personal assistant that indexes your notes, documents, and images and answers questions across them, with agent capabilities, scheduled automations, and clients for desktop, browser, Obsidian, and Emacs.

  • Platforms: Docker, Python package; clients across desktop, browser, Obsidian, Emacs, WhatsApp
  • Local components: server, search index, document ingestion, automation scheduler, local model support
  • Model options: local models via llama.cpp and Ollama, or commercial APIs
  • Execution boundary: self-hosted server plus thin clients
  • Model boundary: local when configured with local models — explicitly supported, not a workaround
  • Storage boundary: indexed content and embeddings in your instance's database
  • Telemetry boundary: self-hosted; the messaging integrations route through third-party platforms by definition, so enabling WhatsApp changes the boundary entirely
  • Best fit: a persistent second brain over years of personal notes that never leaves your server
  • Limitation: AGPL-3.0 obligations apply if you offer a modified version as a network service
  • License: AGPL-3.0
  • Source: https://github.com/khoj-ai/khoj

29. NextChat

A lightweight cross-platform chat client — web, desktop, and PWA — with a small footprint, prompt templates, and MCP support. Notable for how little it needs to run.

  • Platforms: Windows, macOS, Linux desktop; web; PWA
  • Local components: client application, local conversation storage, prompt library
  • Model options: Ollama and OpenAI-compatible endpoints alongside commercial providers
  • Execution boundary: local client, or a self-hosted web deployment
  • Model boundary: local when configured against a local endpoint
  • Storage boundary: conversations stored client-side by default
  • Telemetry boundary: the desktop client talks to the endpoints you configure. Verify the specific build you install, since community forks and hosted variants exist.
  • Best fit: a fast, small, no-server chat client for a local Ollama instance
  • Limitation: shallower agentic and RAG features than the heavier options here
  • License: MIT
  • Source: https://github.com/ChatGPTNextWeb/NextChat

30. big-AGI

A feature-dense AI suite with personas, multi-model comparison, beam search across responses, function calling, and generative UI. More experimental than the mainstream frontends, and more interesting for it.

  • Platforms: Docker, Node.js, Vercel, browser
  • Local components: web application, persona system, multi-model orchestration
  • Model options: Ollama, LM Studio, LocalAI, and OpenAI-compatible endpoints, plus commercial providers
  • Execution boundary: self-hosted or local
  • Model boundary: local when configured against local endpoints
  • Storage boundary: browser-local by default in typical deployments
  • Telemetry boundary: self-hosted instances depend on the providers you configure; verify defaults for any deployment target you did not build yourself
  • Best fit: comparing several local models side by side on the same prompt before standardizing
  • Limitation: the feature surface moves quickly, which is a cost as well as a benefit
  • License: MIT
  • Source: https://github.com/enricoros/big-AGI

31. Msty

A polished desktop client for local and remote models with split-chat comparison, knowledge stacks over your files, and no required Docker or terminal setup. The friendliest install in the category.

  • Platforms: macOS, Windows, Linux
  • Local components: desktop application, bundled local inference, knowledge stacks with local embedding, model manager
  • Model options: local GGUF models managed in-app, plus commercial API connections
  • Execution boundary: local application
  • Model boundary: local when using bundled models; remote the moment you add an API key
  • Storage boundary: models, chats, and knowledge stacks in a local application directory
  • Telemetry boundary: proprietary. The vendor's privacy documentation is the only source, and it cannot be independently verified from source.
  • Best fit: non-technical users who need local model access without installing a runtime first
  • Limitation: closed source, and the free tier's feature boundary is set by the vendor and can move
  • License: Proprietary, with a free tier
  • Source: https://msty.app

32. Page Assist

A browser extension that gives you a sidebar and web UI for your local models, with the ability to chat about the current page, do retrieval over documents, and use local web search — all without the page content leaving your machine.

  • Platforms: Chrome, Firefox, and other Chromium-based browsers
  • Local components: extension, sidebar UI, page content extraction, local RAG, local search integration
  • Model options: Ollama primarily; OpenAI-compatible endpoints supported
  • Execution boundary: browser extension on your machine
  • Model boundary: local — the extension is built specifically for local runtimes
  • Storage boundary: browser extension storage
  • Telemetry boundary: requests go to the local endpoint you configure. Extension permissions are broad by necessity (page content access), so review them.
  • Best fit: summarizing and questioning the page in front of you without sending it anywhere
  • Limitation: a browser extension with page-read permission is a serious trust decision regardless of how good its privacy story is
  • License: MIT
  • Source: https://github.com/n4ze3m/page-assist

Part 4 — Retrieval, memory, and knowledge

An agent without memory restarts from zero every session. These projects supply the retrieval and persistence layer, and all of them can run entirely on infrastructure you control.

33. RAGFlow

A retrieval engine built on deep document parsing — it does layout analysis on PDFs, tables, and scanned documents rather than naively splitting text, and shows you the grounded citations behind every answer.

  • Platforms: Docker Compose on Linux, macOS, Windows
  • Local components: document parsing service, chunking and layout models, vector store, retrieval API, web UI, agent workflows
  • Model options: Ollama and other local endpoints for both generation and embedding, plus commercial providers
  • Execution boundary: self-hosted stack
  • Model boundary: local when configured against local model services
  • Storage boundary: parsed documents, chunks, and embeddings in your own volumes
  • Telemetry boundary: self-hosted; the parsing models are downloaded at setup and then run locally
  • Best fit: complex real-world documents — contracts, scanned reports, financial filings — where naive chunking loses the structure
  • Limitation: heavyweight. The document intelligence models want meaningful RAM and disk, and this is a multi-container deployment.
  • License: Apache 2.0
  • Source: https://github.com/infiniflow/ragflow

34. R2R

A retrieval system with an API-first design covering ingestion, hybrid search, knowledge graph construction, and agentic retrieval where the system reasons about what to fetch rather than doing one-shot similarity search.

  • Platforms: Docker, Python package
  • Local components: ingestion pipeline, Postgres with pgvector, graph construction, retrieval API, orchestration
  • Model options: local providers including Ollama, plus commercial APIs
  • Execution boundary: self-hosted
  • Model boundary: local when configured with local providers
  • Storage boundary: documents, chunks, and graphs in your own Postgres instance
  • Telemetry boundary: self-hosted deployment; verify current instrumentation defaults in the repository
  • Best fit: building retrieval into a product where you need graph relationships, not just nearest neighbors
  • Limitation: graph construction is expensive in model calls, which is a real cost even when the model is local — it is your GPU time instead of someone's invoice
  • License: MIT
  • Source: https://github.com/SciPhi-AI/R2R

35. Perplexica

A self-hosted answer engine that searches the web, reads results, and synthesizes cited answers — the open equivalent of a commercial AI search product, running on your own hardware.

  • Platforms: Docker, Node.js
  • Local components: application, search orchestration, reranking and embedding, source citation
  • Model options: Ollama and OpenAI-compatible endpoints; local embedding models supported
  • Execution boundary: self-hosted
  • Model boundary: local when configured against a local endpoint
  • Storage boundary: local chat history in your deployment
  • Telemetry boundary: this one is explicitly not offline. It queries a search backend, typically a self-hosted SearXNG instance, which in turn queries public search engines. Your query text reaches the open internet by design. Local inference means the synthesis is private, not the search.
  • Best fit: research agents that need current web information without a commercial AI search subscription
  • Limitation: answer quality is bounded by the search backend, and the privacy boundary is fundamentally different from every other entry in this section
  • License: MIT
  • Source: https://github.com/ItzCrazyKns/Perplexica

36. Onyx

A self-hosted AI workspace that connects to internal systems — documents, wikis, chat, ticketing — and provides search and agents over them, with permission-aware retrieval that respects the source system's access controls.

  • Platforms: Docker Compose, Kubernetes
  • Local components: connector framework, indexing pipeline, vector store, permission sync, chat and agent interface
  • Model options: local models via Ollama and OpenAI-compatible endpoints, plus commercial providers
  • Execution boundary: self-hosted stack
  • Model boundary: local when configured with local endpoints
  • Storage boundary: index and permissions in your own infrastructure; connectors read from the third-party systems you authorize
  • Telemetry boundary: self-hosted core, but every connector is an outbound integration to another service by definition
  • Best fit: enterprise internal search where the retrieval layer must not leak documents across permission boundaries
  • Limitation: portions of the repository are enterprise-licensed rather than MIT. Check which directories you are depending on before building on it.
  • License: MIT for the core, with separately licensed enterprise directories in the same repository
  • Source: https://github.com/onyx-dot-app/onyx

37. Mem0

A memory layer that extracts durable facts from conversations, stores them, and retrieves them later, so an agent accumulates knowledge about a user across sessions instead of restarting each time.

  • Platforms: Python and TypeScript packages; self-hosted or managed
  • Local components: memory extraction, vector store integration, graph memory option, local database backends
  • Model options: any provider including local Ollama; embedding model likewise configurable
  • Execution boundary: a library inside your application, or a self-hosted server
  • Model boundary: fully local is achievable — configure both the extraction LLM and the embedder to local endpoints
  • Storage boundary: your chosen vector store and database; a fully local configuration is documented
  • Telemetry boundary: the open-source package has included anonymous telemetry with a documented disable flag. The managed platform is a separate product with an entirely different boundary — do not conflate them.
  • Best fit: giving a local agent memory that survives restarts without writing your own extraction logic
  • Limitation: memory extraction is itself an LLM call per turn, which is meaningful overhead on constrained local hardware
  • License: Apache 2.0
  • Source: https://github.com/mem0ai/mem0

38. Letta

A framework for stateful agents descended from the MemGPT research, treating context as a managed memory hierarchy with the agent editing its own memory blocks and paging older information in and out.

  • Platforms: Docker, Python package; server plus SDK model
  • Local components: agent server, memory management, tool execution, persistence to your own database, developer UI
  • Model options: local endpoints including Ollama and vLLM, plus commercial providers
  • Execution boundary: self-hosted server
  • Model boundary: local when configured against local endpoints
  • Storage boundary: agent state and memory in your own Postgres or SQLite instance
  • Telemetry boundary: self-hosted server; the cloud offering is a distinct product with a distinct boundary
  • Best fit: agents intended to run for months and accumulate state, rather than task-scoped agents
  • Limitation: self-editing memory demands solid instruction-following and function calling. Small local models handle it inconsistently.
  • License: Apache 2.0
  • Source: https://github.com/letta-ai/letta

39. Chroma

An embedding database designed to be embedded — it runs in-process inside a Python or JavaScript application with no server, and scales up to a client/server deployment when you need one.

  • Platforms: Python, JavaScript, Rust core; in-process or server
  • Local components: the entire database, persisting to local disk
  • Model options: bundled local embedding functions, or any embedder you supply including local ones
  • Execution boundary: in-process in your application, or a self-hosted server
  • Model boundary: embeddings computed locally if you choose a local embedding function
  • Storage boundary: a local directory
  • Telemetry boundary: anonymized product telemetry has been enabled by default with a documented opt-out setting. Turn it off explicitly in privacy-sensitive deployments.
  • Best fit: the lowest-friction vector store for a local agent prototype that may become production
  • Limitation: in-process mode is single-application; concurrent multi-process access needs the server mode
  • License: Apache 2.0
  • Source: https://github.com/chroma-core/chroma

40. Qdrant

A vector database written in Rust with payload filtering, quantization, hybrid search, and a well-documented single-binary or single-container deployment. The choice when the collection outgrows what an embedded store handles comfortably.

  • Platforms: Docker, Kubernetes, binary; Linux, macOS, Windows
  • Local components: the database, HTTP and gRPC APIs, web dashboard, on-disk and quantized storage modes
  • Model options: storage only — bring your own embeddings from any local model
  • Execution boundary: self-hosted server
  • Model boundary: not applicable; it stores vectors, it does not produce them
  • Storage boundary: local volumes with configurable on-disk and in-memory tiering
  • Telemetry boundary: telemetry has been present with a documented configuration option to disable it. Set it in your config file at deployment.
  • Best fit: the retrieval backend for a local agent stack with millions of vectors and real filtering requirements
  • Limitation: another service to run, monitor, and back up
  • License: Apache 2.0
  • Source: https://github.com/qdrant/qdrant

Part 5 — Workflow and orchestration platforms

Visual and programmatic builders for multi-step agent workflows. Every one of these self-hosts; the licenses vary far more than the marketing pages suggest.

41. n8n

A workflow automation platform with hundreds of integrations and a native AI agent node, self-hostable as a container with all execution and credentials on your own infrastructure.

  • Platforms: Docker, Kubernetes, npm; Linux, macOS, Windows
  • Local components: workflow editor, execution engine, credential store, scheduler, AI agent and vector store nodes, code nodes
  • Model options: Ollama and OpenAI-compatible local endpoints via dedicated nodes, plus commercial providers
  • Execution boundary: self-hosted; workflows run on your instance with your credentials
  • Model boundary: local when using local model nodes
  • Storage boundary: workflows, executions, and encrypted credentials in your own database
  • Telemetry boundary: the self-hosted distribution has documented environment variables to disable diagnostics and version notifications. Set them at deployment.
  • Best fit: agents that need to touch dozens of business systems, where the integration library is the actual value
  • Limitation: not open source. The Sustainable Use License restricts you to internal business, personal, or non-commercial use, and forbids distributing it commercially. Files with .ee. in the filename or .ee in the directory name are excluded entirely and require an enterprise license. This is a common and expensive misreading.
  • License: Sustainable Use License 1.0 (source-available, not OSI open source), with separately licensed enterprise files
  • Source: https://github.com/n8n-io/n8n

42. Flowise

A drag-and-drop builder for LLM chains, agents, and assistants, with a node graph that maps closely onto LangChain and LlamaIndex primitives and a deployable API for every flow you build.

  • Platforms: Docker, npm; Linux, macOS, Windows
  • Local components: visual editor, flow execution engine, document stores, embedded vector stores, API and embeddable widget
  • Model options: Ollama, LocalAI, llama.cpp, and OpenAI-compatible endpoints, plus commercial providers
  • Execution boundary: self-hosted
  • Model boundary: local when using local model nodes
  • Storage boundary: flows, credentials, and document stores in your own database
  • Telemetry boundary: a documented environment variable disables telemetry in self-hosted deployments. Set it.
  • Best fit: prototyping a retrieval or agent pipeline visually before committing it to code
  • Limitation: visual graphs stop scaling as complexity rises; expect to port successful prototypes into real code
  • License: Apache 2.0
  • Source: https://github.com/FlowiseAI/Flowise

43. Langflow

A visual builder for agents and retrieval flows where every component maps to Python you can read and override, and every flow is exposed as an API endpoint.

  • Platforms: Python package, Docker, desktop builds
  • Local components: visual editor, Python component runtime, flow API server, local database
  • Model options: Ollama, LM Studio, and OpenAI-compatible local endpoints, plus commercial providers
  • Execution boundary: self-hosted or local desktop
  • Model boundary: local when configured against local endpoints
  • Storage boundary: flows and variables in a local SQLite or your own Postgres
  • Telemetry boundary: a documented environment variable disables analytics. Set it at first run.
  • Best fit: teams that want a visual surface but refuse to be trapped in one — the Python escape hatch is real
  • Limitation: the component API has changed substantially across major versions; pin your version
  • License: MIT
  • Source: https://github.com/langflow-ai/langflow

44. Dify

An LLM application platform combining a visual workflow builder, a prompt IDE, RAG pipelines, agent definitions, and observability, self-hostable as a full stack.

  • Platforms: Docker Compose, Kubernetes
  • Local components: API server, worker, web app, vector database, document processing, agent runtime, observability
  • Model options: Ollama, LocalAI, Xinference, OpenLLM, and OpenAI-compatible endpoints, plus commercial providers
  • Execution boundary: self-hosted stack
  • Model boundary: local when configured against local model providers
  • Storage boundary: applications, datasets, and logs in your own database and object storage
  • Telemetry boundary: self-hosted; check the current configuration options in the deployment documentation
  • Best fit: building and operating several internal LLM applications with one governed platform
  • Limitation: the license is Apache 2.0 with additional conditions — restrictions on multi-tenant hosting and on removing branding. If your plan is to resell it as a service, read the license first.
  • License: Apache 2.0 with additional conditions specified in the repository
  • Source: https://github.com/langgenius/dify

45. Activepieces

An automation platform with an AI agent capability, MCP support, and a TypeScript piece framework, positioned as a genuinely open-source alternative in a category dominated by source-available licenses.

  • Platforms: Docker, Kubernetes, npm
  • Local components: flow editor, execution engine, piece framework, credential store, MCP server capability
  • Model options: configurable providers including local endpoints
  • Execution boundary: self-hosted
  • Model boundary: local when configured against a local endpoint
  • Storage boundary: flows and credentials in your own database
  • Telemetry boundary: self-hosted; review the deployment configuration for analytics options
  • Best fit: teams that need automation plus agents and have a hard requirement for a permissive license
  • Limitation: a smaller integration catalog than n8n, and enterprise-licensed directories exist alongside the MIT core
  • License: MIT for the core, with separately licensed enterprise directories
  • Source: https://github.com/activepieces/activepieces

46. Windmill

A developer platform that turns scripts in Python, TypeScript, Go, Bash, and SQL into workflows and internal UIs, with a Rust execution engine built for throughput. The right tool when your agent's "tools" are really just functions you already wrote.

  • Platforms: Docker, Kubernetes, binary
  • Local components: script editor, workflow engine, worker pool, app builder, secret management, schedules
  • Model options: any model you call from your own code, including local endpoints
  • Execution boundary: self-hosted, with workers you place and size yourself
  • Model boundary: entirely up to your code
  • Storage boundary: scripts, flows, and secrets in your own Postgres
  • Telemetry boundary: self-hosted; verify current defaults in the deployment documentation
  • Best fit: engineering teams who want agent orchestration expressed as code with a UI on top, not a node graph
  • Limitation: AGPL-3.0 for the core with a separate enterprise edition — the network-service obligations are real if you modify it
  • License: AGPL-3.0, with a separately licensed enterprise edition
  • Source: https://github.com/windmill-labs/windmill

Part 6 — Agent frameworks and ambient agents

47. CrewAI

A Python framework for multi-agent systems built on role-based delegation: you define agents with goals and backstories, assemble them into a crew, and give the crew a task with either sequential or hierarchical process control.

  • Platforms: Python package; runs anywhere Python does
  • Local components: the entire framework, agent definitions, task orchestration, tool integrations, memory
  • Model options: Ollama and any OpenAI-compatible local endpoint through its provider layer, plus commercial APIs
  • Execution boundary: your process, on your machine
  • Model boundary: local is a supported and documented configuration, not a hack
  • Storage boundary: local memory files and whatever vector store you configure
  • Telemetry boundary: the framework has included anonymous telemetry with a documented opt-out environment variable. Set it before running anything sensitive.
  • Best fit: decomposing a workflow into specialists that hand off to each other
  • Limitation: role-play framing does not by itself make output better, and multi-agent delegation multiplies token consumption — which on local hardware means multiplying wall-clock time
  • License: MIT
  • Source: https://github.com/crewAIInc/crewAI

48. LangGraph

A framework for agents expressed as explicit state graphs with nodes, edges, and conditional transitions, including durable checkpointing and human-in-the-loop interrupts. The right abstraction when you need to know exactly what the agent can do next.

  • Platforms: Python and JavaScript packages
  • Local components: graph runtime, checkpointers backed by local SQLite or Postgres, streaming, interrupt handling
  • Model options: any provider through the LangChain integration layer, including Ollama and local OpenAI-compatible servers
  • Execution boundary: your process
  • Model boundary: configurable; fully local is achievable
  • Storage boundary: checkpoints in a local database of your choosing
  • Telemetry boundary: the framework itself does not require external services, but tracing integration is a separate hosted product that must be explicitly disabled or left unconfigured. Check your environment variables — a stray tracing key sends your agent's full execution traces to a third party.
  • Best fit: production agents where control flow must be inspectable and resumable rather than emergent
  • Limitation: more upfront design work than a role-based framework; the graph is the point and you have to draw it. The agent architectures guide covers when that cost is worth paying.
  • License: MIT
  • Source: https://github.com/langchain-ai/langgraph

49. AutoGPT

The project that popularized autonomous agent loops, now split into two very different things in one repository: the original standalone agent and Forge, and a newer low-code platform for building and deploying agents.

  • Platforms: Docker for the platform; Python for the classic agent
  • Local components: platform server, agent builder, block library, execution engine; the classic agent runs as a local Python process
  • Model options: Ollama is documented as a supported local provider for the platform, alongside commercial APIs
  • Execution boundary: self-hosted
  • Model boundary: local when configured with Ollama
  • Storage boundary: agent definitions and execution history in your own deployment
  • Telemetry boundary: self-hosted; review current configuration in the repository
  • Best fit: studying how autonomous loops were built, and the platform for teams that want a hosted-style builder on their own hardware
  • Limitation: the license is split and this trips people up constantly. Everything inside autogpt_platform/ is under the PolyForm Shield License, which prohibits using it to provide any product that competes with AutoGPT. Everything outside that folder — the classic agent, Forge, the benchmark, the classic GUI — is MIT. If you are building a commercial agent platform, the platform folder is off limits.
  • License: MIT outside autogpt_platform/; PolyForm Shield 1.0.0 inside it
  • Source: https://github.com/Significant-Gravitas/AutoGPT

50. Home Assistant

The largest deployed base of genuinely local ambient agents. Its conversation and assist pipeline lets a local LLM control real devices, with local wake-word detection, local speech-to-text, and local text-to-speech, so a voice command can complete without any packet leaving the house.

  • Platforms: Home Assistant OS, Docker, Python; runs well on small single-board computers, with the LLM typically hosted separately
  • Local components: automation engine, assist pipeline, wake word detection, local speech-to-text and text-to-speech add-ons, device integrations, LLM conversation agents with tool exposure
  • Model options: Ollama integration and OpenAI-compatible endpoints; the model normally runs on a separate machine on the LAN
  • Execution boundary: local, on your own hardware, controlling your own devices
  • Model boundary: local when using the Ollama integration against a LAN host
  • Storage boundary: state history and recordings in your own instance database
  • Telemetry boundary: analytics are opt-in with a clear prompt. Individual device integrations are cloud services in their own right — a local voice pipeline that ends in a cloud-dependent smart plug is not a local system end to end.
  • Best fit: voice and automation agents where a network outage must not disable the lights
  • Limitation: device tool-calling reliability depends heavily on the model; small models confuse similarly named entities, and a wrong tool call has physical consequences
  • License: Apache 2.0
  • Source: https://github.com/home-assistant/core

Comparison matrix

"Local inference" means the project itself runs model weights on your hardware. "Configurable" means it can be pointed at a local endpoint but does not run inference itself. "Cloud required" flags whether the core function needs an external service.

# Project Category Local inference Self-host License class Cloud required for core function
1 Ollama Runtime Yes Yes Permissive (MIT) No (after model download)
2 llama.cpp Runtime Yes Yes Permissive (MIT) No
3 LM Studio Runtime Yes Yes Proprietary No (documented offline mode)
4 Jan Runtime Yes Yes Permissive (Apache 2.0) No
5 LocalAI Runtime Yes Yes Permissive (MIT) No
6 vLLM Runtime Yes Yes Permissive (Apache 2.0) No
7 SGLang Runtime Yes Yes Permissive (Apache 2.0) No
8 llamafile Runtime Yes Yes Permissive (Apache 2.0) No
9 KoboldCpp Runtime Yes Yes Copyleft (AGPL-3.0) No
10 GPT4All Runtime Yes Yes Permissive (MIT) No
11 Aider Coding agent No Yes Permissive (Apache 2.0) Configurable
12 Cline Coding agent No Yes Permissive (Apache 2.0) Configurable
13 Continue Coding agent No Yes Permissive (Apache 2.0) Configurable
14 OpenHands Coding agent No Yes Permissive (MIT) Configurable
15 Goose Coding agent No Yes Permissive (Apache 2.0) Configurable
16 Void Coding agent No Yes Permissive (Apache 2.0) Configurable
17 Tabby Coding agent Yes Yes Permissive (Apache 2.0) No
18 OpenCode Coding agent No Yes Permissive (MIT) Configurable
19 Refact.ai Coding agent Yes Yes Permissive (BSD-3) No (self-hosted edition)
20 twinny Coding agent No Yes Permissive (MIT) No
21 Open Interpreter Coding agent No Yes Copyleft (AGPL-3.0) Configurable
22 gptel Coding agent No Yes Copyleft (GPL-3.0) Configurable
23 AGNT Assistant No Yes (desktop) Proprietary Configurable
24 Open WebUI Assistant No Yes Source-available (branding clause) Configurable
25 AnythingLLM Assistant Yes Yes Permissive (MIT) No
26 LibreChat Assistant No Yes Permissive (MIT) Configurable
27 Lobe Chat Assistant No Yes Apache 2.0 + supplemental Configurable
28 Khoj Assistant Yes Yes Copyleft (AGPL-3.0) Configurable
29 NextChat Assistant No Yes Permissive (MIT) Configurable
30 big-AGI Assistant No Yes Permissive (MIT) Configurable
31 Msty Assistant Yes Yes (desktop) Proprietary No
32 Page Assist Assistant No N/A (extension) Permissive (MIT) No
33 RAGFlow Retrieval Partial (parsing models) Yes Permissive (Apache 2.0) Configurable
34 R2R Retrieval No Yes Permissive (MIT) Configurable
35 Perplexica Retrieval No Yes Permissive (MIT) Yes — web search
36 Onyx Retrieval No Yes MIT core + enterprise dirs Configurable
37 Mem0 Memory No Yes Permissive (Apache 2.0) Configurable
38 Letta Memory No Yes Permissive (Apache 2.0) Configurable
39 Chroma Vector store Yes (embeddings) Yes Permissive (Apache 2.0) No
40 Qdrant Vector store N/A Yes Permissive (Apache 2.0) No
41 n8n Orchestration No Yes Source-available (SUL 1.0) Configurable
42 Flowise Orchestration No Yes Permissive (Apache 2.0) Configurable
43 Langflow Orchestration No Yes Permissive (MIT) Configurable
44 Dify Orchestration No Yes Apache 2.0 + conditions Configurable
45 Activepieces Orchestration No Yes MIT core + enterprise dirs Configurable
46 Windmill Orchestration No Yes Copyleft (AGPL-3.0) + EE Configurable
47 CrewAI Framework No Yes Permissive (MIT) Configurable
48 LangGraph Framework No Yes Permissive (MIT) Configurable
49 AutoGPT Framework No Yes MIT / PolyForm Shield split Configurable
50 Home Assistant Ambient No Yes Permissive (Apache 2.0) Configurable

Hardware guidance

No throughput numbers here. What follows is memory arithmetic, which is the part that transfers between machines.

The one calculation that matters. A model's weights occupy roughly parameters × bits-per-parameter ÷ 8 bytes. At 4-bit quantization that is about 0.5 GB per billion parameters; at 8-bit, about 1 GB per billion; at 16-bit, about 2 GB per billion. Real quantization formats carry per-block metadata, so add roughly 5-15 percent over the naive figure depending on the format.

Then add KV cache. The cache scales with context length, layer count, and attention head configuration, and at long contexts it can rival or exceed the weights themselves. Budget headroom, do not budget exactly.

Practical ranges, stated as ranges because they are:

  • 8-16 GB unified memory or system RAM. Models in the 3-8B parameter range at 4-bit quantization fit with room for a modest context. This is enough for autocomplete, summarization, classification, extraction, and simple tool calls. It is generally not enough for reliable multi-step agent loops over a large codebase.
  • 16-32 GB. The 7-14B range at 4-bit, or 7-8B at 8-bit with a longer context. This is where local coding assistance stops feeling like a demo. Multi-file agent work is possible and inconsistent.
  • 24-48 GB VRAM. Models in the roughly 20-32B range at 4-bit, or smaller models at higher precision with long contexts. This is the practical floor where agentic loops — plan, call tool, read result, revise — hold together across many steps without derailing.
  • 48-96 GB and above, or multi-GPU. Larger dense models and mixture-of-experts architectures at usable quantization, with the context headroom that agents actually consume. This is also where a serving engine like vLLM or SGLang earns its complexity, because you are now sharing one model across several concurrent agents.

Four things that are true regardless of your hardware tier:

  1. Context length costs memory continuously, not once. An agent that accumulates tool outputs over twenty steps has a KV cache that grows the whole time. Sizing for the first turn and being surprised at the twentieth is the most common local-agent failure.
  2. CPU-only inference works and is slow. llama.cpp and llamafile run acceptably on CPU for short interactive use. Agent loops that make dozens of sequential calls turn that slowness into minutes per task.
  3. Quantization below 4-bit degrades instruction-following before it degrades prose. A model that still writes fluent paragraphs at aggressive quantization may have already lost reliable JSON and function-call formatting — which is precisely what an agent depends on. Test tool calling specifically, not chat quality.
  4. Apple Silicon's unified memory is a real architectural advantage here, because the entire memory pool is addressable by the GPU. A 64 GB Mac can hold models that would need a much more expensive discrete-GPU configuration. Throughput characteristics differ from a high-end NVIDIA card; capacity is the axis where it wins.

Privacy verification checklist

Do not take any project's privacy claim on faith, including the ones in this article. Run these checks against your actual installation.

Before installing

  1. Read the LICENSE file at the source URL. Confirm the license class matches what the marketing page implies. Check for separately licensed directories (.ee, ee/, enterprise/, platform/).
  2. Search the repository for telemetry, analytics, posthog, sentry, segment, and mixpanel. Note the environment variables that disable each hit.
  3. Read the docker-compose.yml in full. Identify every external image and every service that expects internet access.
  4. Check the last commit date and the release cadence. An unmaintained security-relevant service is a liability regardless of its license.

During installation

  1. Set every telemetry opt-out environment variable explicitly in your compose file or config. Do not rely on the default being what you want, and do not rely on it staying that way across upgrades.
  2. Verify where model weights are downloaded from and confirm the checksum if the project publishes one.
  3. Bind every service to 127.0.0.1 unless you specifically need LAN access. A local agent with an inference server on 0.0.0.0 and no auth is an open API to anyone on the network.
  4. Check for stray provider credentials in your environment. OPENAI_API_KEY, ANTHROPIC_API_KEY, and tracing keys like LANGCHAIN_TRACING_V2 get picked up automatically by many frameworks and silently redirect both inference and full execution traces off the machine.

After installation

  1. Run the real test: disconnect the network and complete a full task. Not one message — a whole task with tool calls and file writes. Anything that requires connectivity will fail loudly and you will know exactly what it was.
  2. With the network back on, watch outbound connections during a normal session. netstat -b on Windows, lsof -i on macOS and Linux, Little Snitch or a firewall log for a continuous view.
  3. For containers, put the service on an internal Docker network with no gateway and see what breaks.
  4. Find where conversations, embeddings, and uploaded documents are actually written on disk. Confirm the directory is covered by your disk encryption and your backup policy, and that it is not in a cloud-synced folder.
  5. Audit every MCP server and plugin separately. A local agent hosting a remote MCP server is not local, and the agent's own privacy documentation says nothing about what that server does.
  6. Re-run steps 5 and 9 after every major version upgrade. Defaults change, and telemetry that was opt-in in one release has been opt-out in the next.

FAQ

What actually counts as a local AI agent?

Three properties, and you should specify which ones you need. Local execution means the agent loop and its tool calls run on your hardware. Local inference means the model weights are on your disk and the forward pass happens on your processor. Local storage means conversations, embeddings, and artifacts are written to disk you control. Many products deliver the first and third and market themselves on the second.

Can a local model actually complete real agentic work, or is this only good for chat?

It depends almost entirely on tool-calling reliability, not on general fluency. Agent loops fail when the model emits malformed function calls, hallucinates parameters, or loses track of what it already tried. Recent open-weight models in the 20-30B range handle bounded, well-specified tasks with a small number of clearly documented tools. Long-horizon work across a large codebase with a dozen available tools remains where frontier hosted models hold a real advantage. Test with your actual tool schemas before deciding.

Which single project should I start with?

Install Ollama, pull one model, and connect one client. If you want a chat interface, use Open WebUI or AnythingLLM. If you want coding assistance, use Continue with a local model bound to the autocomplete role. Getting one end-to-end local loop working teaches you more about your hardware's actual ceiling than any amount of comparison reading.

Is self-hosted the same as offline?

No, and conflating them causes real compliance failures. A self-hosted n8n instance calling a commercial API sends your data to that vendor exactly as a SaaS product would. Self-hosting relocates the orchestration layer and the credential store. It does not relocate inference unless you also change the model provider.

Are AGPL and source-available licenses a problem for my company?

They are a problem if you misidentify them. AGPL-3.0 obligations trigger when you offer a modified version as a network service — running an unmodified instance internally is generally fine, and the specifics are a question for your counsel, not for an article. Source-available licenses are different in kind: n8n's Sustainable Use License restricts you to internal business, personal, and non-commercial use outright, and AutoGPT's PolyForm Shield terms prohibit building anything that competes. Neither is open source. Get the classification right before the deployment, not during the review.

How much does telemetry actually matter if no prompt content is sent?

Metadata is not nothing. Usage telemetry typically reveals which models you run, how often, from which network, with which integrations enabled — a usable picture of what your organization is building. Some tools have historically sent more than intended through error reporting that captured surrounding context. The practical position: disable it where you can, verify with a packet-level check where it matters, and accept it where the tool is genuinely worth the exposure.

What is the real hardware floor for something useful?

A machine with 16 GB of unified memory or a GPU with 8-12 GB of VRAM runs 7-8B models at 4-bit quantization well enough for autocomplete, summarization, extraction, and simple tool calls. Below that, CPU inference works but agent loops making dozens of sequential calls become impractically slow. The floor for reliably useful multi-step agent work sits considerably higher — see the hardware guidance above.

Should I run one large model or several small specialized ones?

Route by role rather than picking one answer. A small fast model handles autocomplete, classification, and routing at low latency. A larger model handles planning and complex tool use. Continue's per-role model configuration and AGNT's per-agent provider selection both express this directly, and it usually produces better results than forcing one model to do everything.

Where do I go next?

For the broader open-source agent ecosystem beyond the local-execution slice, read the best open-source AI agents. For the design decisions behind agent loops, read the AI agent architectures guide. To run persistent agents, workflows, goals, memory, plugins, and MCP tools on your own machine, download AGNT.

Run a local agent in AGNT

AGNT is built for this exact job: agents, workflows, goals, tools, skills, memory, plugins, evaluations, and traces running from a local workspace with the model provider you choose. Download AGNT for Windows, macOS, or Linux, or self-host it from the repository.

Last reviewed 11 August 2026. Licenses, telemetry defaults, and maintenance status change without notice. Every claim in this article should be re-verified against the linked source before it goes into a procurement decision or a compliance document.

Continue reading

Put it into practice

Marketplace

Browse reusable assets

Agents · Tools · Workflows · Skills · Plugins · WidgetsDocs

Read the documentation

Install, connect a model provider, run a verified job.Download

Run AGNT locally

Free forever on your own machine.