Hermes Agent for Beginners: Run Your First Local AI Agent

Install Hermes Agent with its one-line installer, run hermes model, and choose Local models. Since v0.22.0 Hermes installs llama.cpp for your hardware, rates catalog models against your memory, and recommends one that fits fully in GPU memory at 20 or more tokens per second with at least 64K context. No account or API key is needed.

What changed, and why local setup is now simple

Hermes Agent is Nous Research's open source, MIT-licensed agent: a terminal and desktop app with tools, persistent memory, skills, scheduled jobs, and messaging gateways. Until recently, running it on a local model meant installing a separate inference server and wiring it up by hand.

Release v0.22.0, published in mid-October, 2026, added what the release notes call local models in one click. Hermes installs and maintains llama.cpp with the right engine for your hardware (CPU, CUDA, Metal, Vulkan or ROCm), rates every catalog model against your GPU and RAM, and picks the highest-quality build that fits. No account and no API key are needed for the local path. The steps below follow the current documentation.

Before you start: what your machine can run

The Hermes local models documentation gives rough sizing: a GPU with 8 GB or more of memory runs the small catalog models comfortably, and 16 GB or more runs the 27B to 35B models at high quality. On a Mac, unified memory plays that role. Supported engines are Metal or CPU on macOS, CUDA, Vulkan, or CPU on Windows and Linux, with HIP/ROCm as an explicit choice on supported x64 systems.

Two rules from the docs shape what you will be offered. Hermes recommends a model only when it fits entirely in GPU or unified memory at a predicted 20 or more tokens per second, and it never offers builds below 4-bit quantisation. Every recommended model gets at least a 64K context window, because agent prompts are long.

Step by step

Four steps take a fresh machine to a working local agent.

1. Install Hermes

On macOS, Linux, or WSL2, the README's installer is curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash. On native Windows, run iex (irm https://hermes-agent.nousresearch.com/install.ps1) in PowerShell. Reload your shell, then run hermes to confirm it starts. If anything misbehaves, hermes doctor diagnoses common problems.

2. Choose the local model path

In a terminal, run hermes model and choose Local models. It lists catalog models priced for your machine, installs the engine, downloads the model, starts the server, and sets it as default. In the desktop app, open Settings, Providers, Local Models, click Install runtime, pick a model, click Download, then Use. Ctrl+C pauses a terminal download, and choosing the model again resumes it.

3. Read the fit labels

Each model carries a label: green runs fully in GPU memory, amber spills into system RAM and runs slower, red is too big. Start with the recommended green model. The v0.22.0 catalog includes Qwen3.8 27B, Qwen3.8 Flash Next, Qwen3.6 35B-A3B, and DeepSeek V4 Flash, plus any GGUF from Hugging Face, though community models lack the validated badge.

4. Run a first task

Start with something read-only, such as summarising a folder of notes or explaining a codebase. Watch which tools it calls and leave command approval on. Once that works, try a scheduled job through the built-in cron feature, such as a morning summary.

If you already run Ollama or LM Studio

Hermes works with existing local servers, but the context window is the usual trap. The provider docs set a 64,000-token minimum and note that Ollama's default context depends on VRAM, 4,096 tokens below 24 GB. Raise it with OLLAMA_CONTEXT_LENGTH=64000 ollama serve. For LM Studio, load with lms load <model> --context-length 64000. For llama-server, pass --jinja or tool calling is ignored, and set -c explicitly. Then run hermes model, choose the custom endpoint option, and point it at the server's OpenAI-compatible URL.

Beginner mistakes worth avoiding

Assuming onboarding is local. On a fresh desktop install, the guided setup chat runs on a free trial model, nous/welcome, per the release notes. If nothing may leave the machine, skip setup and go straight to Local Models.

Forcing a red or amber model. It will load, slowly. A smaller green model that responds quickly is more useful than a large one that stalls every turn.

Blaming the model for slow first replies. Long agent prompts take time to process before the first token appears, especially on laptops.

Stuffing context. Pasting whole documents fills the context and slows every turn. Give the agent a way to search your notes instead, so it reads only the relevant passages. RDK is one option: it indexes local vaults as encrypted private chunks that agents search before calling a model.

Turning off approvals early. A local model is private, not infallible. Keep approvals on for commands and file writes until you trust what it does.

Where to go next

Connect a messaging gateway so you can reach the agent from your phone, add one MCP server for a tool you use daily, and let the memory and skills features accumulate over a week of real use. If a task exceeds what your local model can do, /model switches provider mid-session, so hard tasks can go to a hosted model while routine work stays local.

Frequently asked questions

Is Hermes Agent free to run locally?
Yes. Hermes Agent is open source under the MIT licence, and its local models feature runs open models entirely on your machine with no account or API key, according to the v0.22.0 release notes. Your costs are the hardware and electricity. Hosted providers and the Nous Portal subscription are optional for when you want larger models or hosted tools.
What hardware do I need to run Hermes with a local model?
The Hermes documentation says a GPU with 8 GB or more of memory runs the small catalog models comfortably, and 16 GB or more runs 27B to 35B models at high quality. Macs use unified memory. Hermes only recommends models that fit fully in GPU or unified memory at a predicted 20 or more tokens per second.
Why does Hermes need a 64K context window?
Agent prompts carry system instructions, tool definitions, memory, and conversation history, so they are long before you type anything. The Hermes provider documentation sets 64,000 tokens as the minimum and rejects smaller windows at startup. If you use your own Ollama or LM Studio server, raise its context setting, because defaults are often much lower.
Can I use Ollama with Hermes Agent instead of the built-in runtime?
Yes. Start Ollama with a context length of at least 64,000 tokens, for example OLLAMA_CONTEXT_LENGTH=64000 ollama serve, then run hermes model and choose the custom endpoint option with Ollama's OpenAI-compatible URL. The built-in runtime is simpler for beginners because it handles engine choice, model fit, and context sizing automatically.