How to install Hermes Agent and connect it to Ollama, LM Studio, LiteLLM and Apple's model

So far in this series you've used artificial intelligence models to chat. An agent goes one step further: besides answering, it can do things on your computer to complete a task. Search for information, read and write files, run commands, or chain several steps until it finishes what you asked.

Hermes Agent, from Nous Research, is a free, open-source agent. It runs in the terminal (and also has a desktop app), remembers what it learns between sessions and can use any OpenAI-compatible model: yours from Ollama, LM Studio or LiteLLM.

In this guide you'll install it, connect it to your models and see what role the Apple model we turned on in the previous guide can play.

What you should know before you start

• An agent acts, it doesn't just talk. Hermes can create, change or delete files and run commands. Start by trying it in a test folder, read what it proposes before accepting it, and don't give it access to important documents until you understand how it works.
• It needs models with a lot of context memory. Hermes requires the model to have a context memory of at least 64,000 tokens; if it's smaller, it refuses it at startup. That means adjusting Ollama and LM Studio, and it rules out Apple's model as the main model (explained below).
• Small models fall short. An agent needs models that can use tools and follow long instructions. Locally, the bigger the model your computer can run, the better it works. For serious tasks, a paid model through LiteLLM and OpenRouter usually gives better results.

Requirements

• Mac: the desktop app needs a Mac with an Apple chip. The terminal version also works on other Macs.
• Windows 10 or 11: Hermes has a PowerShell installer. It can also be used inside WSL 2 (the Linux built into Windows).
• Linux: any recent distribution.
• Git installed (see step 1).
• At least one of these services installed and running (there are guides on this blog): Ollama, LM Studio or LiteLLM.

Step 1. Open the terminal and install Git

• Mac: press ⌘ + Space, type "Terminal" and press Enter. Type git --version. If it isn't installed, macOS offers to install the "command line developer tools": accept and wait for it to finish.
• Windows: download Git from git-scm.com, install it with the default options and then open PowerShell from the Start menu.
• Linux (Ubuntu or Debian): press Ctrl + Alt + T and type:

sudo apt install git curl xz-utils

Step 2. Install Hermes Agent

Mac and Linux

Copy this command into the terminal and press Enter:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

Windows

Copy this command into PowerShell and press Enter:

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

The installer takes care of everything Hermes needs (its own Python version and other tools), so it takes a few minutes.

When it finishes, close the terminal and open a new one. Then check that everything is in order:

hermes doctor

This command reviews the installation and tells you if anything is missing.

Step 3. Prepare your models for 64,000 tokens

Before connecting Hermes, raise the context memory of the model you'll use.

If you use Ollama

• Mac and Windows: open Ollama's settings and set Context length to 64k or more.
• Linux (or if you prefer the terminal): quit Ollama and start it with:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Pick a model your computer can run with that context. As a guide: qwen3.5 on 16 GB computers, and gpt-oss:20b or qwen3.8 on computers with 32 GB or more. Check the context with ollama ps after using it.

If you use LM Studio

Load the model with a 64,000 context. In the app, when loading it, raise Context Length to 64000. Or from the terminal:

lms load qwen/qwen3.5-9b --context-length 64000

Remember to have LM Studio's local server on (Developer tab → Start server).

If you use LiteLLM

LiteLLM doesn't have its own context: it depends on the model it sends the request to. OpenRouter models usually have very large contexts; Ollama or LM Studio models need the adjustment above.

Step 4. Connect Hermes to your model

The easiest way is the setup wizard. In the terminal, type:

hermes model

A menu opens. Use the arrow keys to move and Enter to choose.

Option A: Ollama

1. Choose Custom endpoint (self-hosted / VLLM / etc.).
2. For the address (base URL), type http://localhost:11434/v1.
3. Leave the key (API key) empty.
4. Type the model name as it appears in ollama ls, for example qwen3.5.

Option B: LM Studio

1. Choose LM Studio from the list (Hermes has it as its own option).
2. Check that the address is http://localhost:1234/v1.
3. Choose the loaded model.

Option C: LiteLLM

1. Choose Custom endpoint (self-hosted / VLLM / etc.).
2. For the address, type http://localhost:4000/v1.
3. For the key, paste your LiteLLM master key (the one starting with sk-).
4. Type the model name as you defined it in LiteLLM's config.yaml, for example free-qwen or or-sonnet-5.

LiteLLM is the most flexible option: from a single point, Hermes can use both your local models and OpenRouter's.

Step 5. First test

Run a quick one-question test:

hermes chat --oneshot -q "Say hello and tell me which model you are"

If you get an answer, Hermes is connected to your model.

To open the agent's full interface in the terminal, type:

hermes --tui

You'll see a header with the provider and model you're using. Type a simple task to begin, for example: "Create a file called notes.txt in this folder with a list of three ideas to organise my week".

Step 6. Keep several models at hand

Hermes keeps its configuration in the config.yaml file in the .hermes folder inside your user folder, and the keys in the .env file in that same folder. You can repeat hermes model to add more providers; the wizard saves them for you.

If you'd rather write it by hand, the Hermes documentation describes a providers section with one entry per provider. For example:

providers:
  ollama:
    api: http://localhost:11434/v1
    default_model: qwen3.5
    context_length: 64000
  lmstudio:
    api: http://localhost:1234/v1
    default_model: qwen/qwen3.5-9b
    context_length: 64000
  litellm:
    api: http://localhost:4000/v1
    key_env: LITELLM_API_KEY
    default_model: free-qwen
    context_length: 64000

And in the .env file, the LiteLLM key:

LITELLM_API_KEY=sk-my-switchboard-8f3k29dq7w

Inside a Hermes session, you switch model with the /model command. For example, /model custom:litellm:or-sonnet-5 uses the or-sonnet-5 model from the litellm provider.

What about Apple's model?

In the previous guide we turned on the Apple Intelligence model with fm serve. Technically, Hermes could connect to it like any other OpenAI-compatible service (address http://localhost:1976/v1, model system). But it doesn't work as Hermes's main model: Apple's model has a context memory of 4,096 tokens, and Hermes requires at least 64,000. At startup, Hermes will reject it.

So the recommended split is:

• Hermes with a large model: local (Ollama or LM Studio with a 64,000 context) or in the cloud through LiteLLM.
• Apple's model for short tasks from Open WebUI: summarising, extracting data or rewriting text.

If Apple increases its model's memory in the future, you'll just need to add it as one more provider with the same steps.

Keeping Hermes up to date

Hermes is updated often. To install the latest version:

hermes update

And if you ever want to uninstall it:

hermes uninstall

Common problems

• "hermes: command not found". Close the terminal and open a new one. On Mac, if it persists, run source ~/.zshrc; on Linux, source ~/.bashrc.
• Hermes rejects the model at startup for lack of context. The model has less than 64,000 tokens of context. Go back to step 3 and confirm the real context with ollama ps or in LM Studio.
• It can't connect to Ollama or LM Studio. Check that they're open: in your browser, http://localhost:11434 should say "Ollama is running" and http://localhost:1234/v1/models should list your LM Studio models.
• Authentication error with LiteLLM. The key must be exactly your LiteLLM master key, which starts with sk-.
• The agent is very slow or often gets things wrong. The model is too small for the job or your computer is short on memory. Try a larger model, or a paid one through LiteLLM.
• Something doesn't work and you don't know why. Run hermes doctor: it reviews the installation and configuration and tells you what's wrong.

What you've built

With this guide you complete the artificial intelligence part of the series: you have local models with LM Studio and Ollama, a switchboard with LiteLLM, a web interface with Open WebUI, an AI editor in Cursor, Apple's model built into your Mac, and an agent, Hermes, able to work with all of them. All under your control and, for the most part, inside your own computer.

In the last guide you'll set up a server at home, always on and reachable from anywhere, to bring all of this together.

Next guide: How to set up your own home server with Ubuntu Server and reach it from anywhere with Tailscale