How to install and set up LiteLLM to bring all your AI models together

Once you start using artificial intelligence models, you soon end up with several different places: some models in Ollama, others in LM Studio, and maybe one in the cloud, such as OpenRouter. Each has its own address and its own way of connecting.

LiteLLM fixes that mess. It's a free program that works like a switchboard: your applications connect only to LiteLLM, with one address and one key, and LiteLLM routes each request to the right model, wherever it is.

In this guide you will install LiteLLM, connect it to Ollama, to LM Studio and to a free OpenRouter model, and check that everything answers. You don't need to know how to program: we tell you exactly what to type.

What you will achieve

By the end you will have:

• LiteLLM running on your computer at http://localhost:4000.
• A master key that protects access.
• A list of models with simple names you choose, for example gemma-local or qwen-free, that any compatible application can use.

This switchboard is the foundation for the next guides in the series: Open WebUI (a web interface to chat with all your models) and Cursor (an AI code editor).

Before you start

• A computer with Mac, Windows 10/11 or Linux.
• An Internet connection for the installation.
• At least one of these services, depending on the models you want to use:
• Ollama installed with a model downloaded (there's a guide on this blog).
• LM Studio with its local server turned on (it has its own guide too).
• A free OpenRouter account, if you want to use cloud models.

You don't need all three: one is enough to follow the guide.

Step 1. Open the terminal

You will type a few commands in the terminal. You only have to copy and paste.

• Mac: press ⌘ + Space, type "Terminal" and press Enter.
• Windows: open the Start menu, type "PowerShell" and press Enter.
• Linux: press Ctrl + Alt + T.

To paste into the terminal: ⌘ + V on Mac, right-click or Ctrl + V in PowerShell, and Ctrl + Shift + V on Linux.

Step 2. Install uv, the installer

LiteLLM is written in Python. Instead of installing Python by hand, we'll use uv, a tool that takes care of everything and avoids version problems.

Mac and Linux

Copy this command into the terminal and press Enter:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows

Copy this command into PowerShell and press Enter:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

When it finishes, close the terminal and open a new one so it recognises the newly installed tool. Check that it works:

uv --version

You should see a version number.

Step 3. Install LiteLLM

Type this command (it's the same on Mac, Windows and Linux):

uv tool install 'litellm[proxy]'

The installation takes a minute or two. When it finishes, close the terminal, open a new one and check that the program answers:

litellm --help

If you see a list of options, LiteLLM is installed.

Step 4. Create the LiteLLM folder

We'll keep the configuration in its own folder called litellm, inside your user folder.

Mac and Linux

mkdir -p ~/litellm
cd ~/litellm

Windows

mkdir $HOME\litellm
cd $HOME\litellm

From now on, every time you open a terminal to use LiteLLM, first go into this folder with cd ~/litellm (Mac and Linux) or cd $HOME\litellm (Windows).

Step 5. Get your keys ready

The master key

LiteLLM asks for a master key: a password that every application connecting to it must use. It must start with sk-. Make up a long one, for example:

sk-my-switchboard-8f3k29dq7w

Keep it somewhere safe: you'll need it in the next guides.

The OpenRouter key (optional)

If you want to use free cloud models:

1. Go to openrouter.ai and create an account.
2. Go to openrouter.ai/settings/keys and click Create Key.
3. Give it a name (for example, "LiteLLM") and, if offered, set a credit limit: that way you'll never spend more than planned.
4. Copy the key that appears (it starts with sk-or-). It's shown only once.

Free OpenRouter models end in :free. They have limits: 20 requests per minute and 50 per day (1,000 per day if you have ever bought at least 10 credits). You can see which ones exist at openrouter.ai/models, filtering by free price. The list changes often.

Step 6. Write the configuration file

LiteLLM's configuration is a text file called config.yaml. Let's create it with a simple editor.

Mac and Linux

Inside the litellm folder, type:

nano config.yaml

The nano editor opens inside the terminal.

Windows

Inside the litellm folder, type:

notepad config.yaml

Notepad asks whether you want to create the file: click Yes.

The content

Copy this text into the editor. Then change what's explained below.

model_list:
  - model_name: gemma-local
    litellm_params:
      model: ollama_chat/gemma3
      api_base: http://localhost:11434

  - model_name: lmstudio-gemma
    litellm_params:
      model: lm_studio/google/gemma-3-4b
      api_base: http://localhost:1234/v1

  - model_name: qwen-free
    litellm_params:
      model: openrouter/qwen/qwen3.8-27b:free
      api_key: sk-or-PASTE-YOUR-OPENROUTER-KEY-HERE

general_settings:
  master_key: sk-my-switchboard-8f3k29dq7w

What each part means:

• model_list is the list of models your switchboard offers. Each model starts with a dash.
• model_name is the name your applications will see. Use whatever you like, without spaces.
• model says where the real model lives: ollama_chat/ for Ollama, lm_studio/ for LM Studio and openrouter/ for OpenRouter, followed by the model's exact name on that service.
• api_base is the service's address: http://localhost:11434 for Ollama and http://localhost:1234/v1 for LM Studio.
• api_key is the service's key, when it needs one.
• master_key is your master key.

Now adapt it:

1. Replace sk-my-switchboard-8f3k29dq7w with your master key.
2. In the OpenRouter model, replace sk-or-PASTE-YOUR-OPENROUTER-KEY-HERE with your OpenRouter key. If you won't use OpenRouter, delete the whole qwen-free block (its four lines).
3. In the Ollama model, gemma3 must be a model you have downloaded. You can list them with ollama ls. If you don't use Ollama, delete the gemma-local block.
4. In the LM Studio model, google/gemma-3-4b must match the identifier LM Studio gives. To see it, with its server on, open http://localhost:1234/v1/models in your browser and copy the "id" value. If you don't use LM Studio, delete the lmstudio-gemma block.

Save the file:

• nano (Mac and Linux): press Ctrl + O, then Enter, and exit with Ctrl + X.
• Notepad (Windows): File → Save and close the window.

Step 7. Start LiteLLM

Before starting, make sure the services in your configuration are running: Ollama open and, if you use it, the LM Studio server on.

In the terminal, inside the litellm folder, type:

litellm --config config.yaml

After a few seconds you'll see several lines of text. These two are the important ones: the first confirms it read your configuration, the second that it's accepting requests.

LiteLLM: Proxy initialized with Config, Set models:
Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)

LiteLLM is now running. Don't close this window: if you close it, LiteLLM stops. To stop it on purpose, press Ctrl + C in it.

Step 8. Check that it works

Open a second terminal window (leave the first one open). Replace sk-my-switchboard-8f3k29dq7w with your master key in the commands below.

List the models

On Mac and Linux:

curl http://localhost:4000/v1/models -H "Authorization: Bearer sk-my-switchboard-8f3k29dq7w"

On Windows (note that you type curl.exe):

curl.exe http://localhost:4000/v1/models -H "Authorization: Bearer sk-my-switchboard-8f3k29dq7w"

You should see text that includes your model names: gemma-local, lmstudio-gemma, qwen-free.

Ask a question

On Mac and Linux:

curl http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-my-switchboard-8f3k29dq7w" -H "Content-Type: application/json" -d '{"model": "gemma-local", "messages": [{"role": "user", "content": "Hi, introduce yourself in one sentence"}]}'

On Windows:

Invoke-RestMethod -Uri http://localhost:4000/v1/chat/completions -Method Post -Headers @{Authorization = "Bearer sk-my-switchboard-8f3k29dq7w"} -ContentType "application/json" -Body '{"model": "gemma-local", "messages": [{"role": "user", "content": "Hi, introduce yourself in one sentence"}]}'

If you get an answer with the model's text, your switchboard works. Try replacing gemma-local with qwen-free or lmstudio-gemma to check the others.

How your applications will use it

Any OpenAI-compatible application can use all your models with these two details:

• Address (base URL): http://localhost:4000/v1
• Key (API key): your master key.

In the browser, http://localhost:4000 also shows the API's technical documentation.

Adding more models

To add a model, open config.yaml again, copy one of the blocks that start with - model_name:, paste it at the end of the list (before general_settings) and change the name and the model. Then stop LiteLLM with Ctrl + C and start it again with litellm --config config.yaml.

Optional: web dashboard and per-application keys

LiteLLM has an admin dashboard at http://localhost:4000/ui and lets you create virtual keys: a different key for each application or person, with spending limits. Both need a PostgreSQL database, so they're outside this beginner guide. If you need them later, the easiest way is the Docker installation described in the official LiteLLM documentation.

Keeping it running

With this installation, LiteLLM runs while the terminal window is open. For daily use that's usually enough: open the terminal, go into the folder and start it. If you want it to start with your computer, the usual options are running it with Docker or registering it as a system service, something more advanced we'll cover in another guide.

Common problems

• "litellm: command not found" or "not recognized". Close the terminal and open a new one after installing. If it persists, run uv tool update-shell, then close and reopen.
• Error reading config.yaml. It's almost always the indentation: check that each line has the same spaces as the example and that there are no tabs.
• "No connected db", "401" or "Authentication Error". The key you're using doesn't match the file's master_key: LiteLLM treats it as a virtual key, and those need a database. Copy the master key again; remember it must start with sk-.
• The Ollama or LM Studio model doesn't answer. Check that the program is open, that the model is downloaded and that the name in config.yaml is exactly the same.
• The OpenRouter model returns a limit error. You've exceeded the free requests for the minute or the day. Wait, or try another :free model.
• "Address already in use". Another program is using port 4000. Start LiteLLM on another port with litellm --config config.yaml --port 4001 and use that address in your applications.

Next steps

Your switchboard is ready. In the next guide you will install Open WebUI, a ChatGPT-style web interface, and connect it to LiteLLM to chat with all your models from the browser.

Next guide: How to install Open WebUI and use your LiteLLM models from the browser