Deployment GuideUpdated November 1, 2026

Practical VPS tutorial

How to Install Ollama on a VPS and Run a Private AI Model

Install Ollama on an Ubuntu VPS without a GPU, run a small AI model, test response speed, and connect from your laptop through a private SSH tunnel.

Start with a CPU VPS if

you want to find out whether a small model is useful before paying for acceleration.

Rent a GPU if

CPU inference is too slow for your task.

You can run Ollama on an Ubuntu VPS without a GPU. This guide installs a small model, tests its API and connects it to your laptop through SSH. You don't need Docker or a domain.

How to Install Ollama on a VPS and Run a Private AI Model — overview

I wouldn't rent a GPU just to find out whether a personal summarizer is useful. Start with a modest CPU server and let your actual prompts tell you whether it needs more power.

We'll use qwen3:4b as the example. The model runs on the VPS even when your laptop is off; server maintenance and access are still your responsibility.

Before you start: Ollama VPS requirements

Use Ubuntu 24.04 LTS with systemd, SSH access and a sudo-enabled account. Run the server commands in the VPS's Bash shell. Step 5 marks the commands to run on your own computer.

For this small-model exercise, 4 vCPU, 8 GB RAM and 60 GB SSD is a reasonable test budget. These are planning figures, not official minimums or a speed guarantee.

ResourceWhat matters
CPUSustained CPU time; check the provider's fair-use policy.
RAMRoom for weights, context, the runtime, Ubuntu and other apps.
StorageSpace for the OS, model files and future downloads.
GPUOptional; consider one when CPU inference is too slow for your task.

The qwen3:4b model entry lists a roughly 2.5 GB Q4_K_M download. That isn't its total running-memory requirement. Our AI VPS RAM guide explains the extra memory used by context and the runtime.

4 vCPU, 8 GB RAM, 60 GB SSDreasonable test budget for this small-model exercise
2.5 GBthe qwen3:4b Q4_K_M download, not its total running-memory requirement
4,096 tokenscontext length set in the service override
How to Install Ollama on a VPS and Run a Private AI Model — detail

1. Install Ollama on Ubuntu

From your computer, connect to the VPS. Replace deploy with your existing sudo-enabled username and SERVER_IP with its address:

ssh deploy@SERVER_IP

On the VPS, install the tools used below:

sudo apt update
sudo apt install -y curl ca-certificates jq zstd less

Download the official installer, inspect it, then run it:

curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
less ollama-install.sh
sh ollama-install.sh

Press q to exit less. The installer may request your sudo password and configures a service on this systemd setup.

ollama --version
sudo systemctl enable --now ollama
sudo systemctl is-active ollama

The last command should print active. If it doesn't, read the error before continuing:

sudo journalctl -u ollama -n 50 --no-pager

2. Keep the API private and set a small context

Ollama's local API requires no authentication. We'll bind it to loopback and reach it through SSH, with no public opening for port 11434.

On the VPS:

sudo systemctl edit ollama.service

Add these lines in the editable area above the commented service contents, then save and close the editor:

[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_NO_CLOUD=1"
Environment="OLLAMA_CONTEXT_LENGTH=4096"

OLLAMA_NO_CLOUD=1 disables Ollama's cloud models and web search; see its local-only configuration. The 4,096-token context limits how much text the model can consider at once. Increasing it requires more memory.

Apply the settings and check the listener:

sudo systemctl daemon-reload
sudo systemctl restart ollama
sudo ss -ltnp 'sport = :11434'
curl --fail --silent --show-error http://127.0.0.1:11434/api/tags

Look for 127.0.0.1:11434 in the listener output and a model list from the API. An empty list is normal. If you see 0.0.0.0:11434 or [::]:11434, check the override: those addresses aren't restricted to loopback.

3. Download a model and try terminal chat

On the VPS:

ollama pull qwen3:4b
ollama list
ollama run qwen3:4b

Ask “Explain an SSH key to a beginner in three sentences.” Type /bye to leave the chat.

This specific small model keeps the example consistent. Change models later if its answers don't suit your task; a successful installation doesn't establish answer quality.

Two useful CLI commands:

ollama ps
ollama stop qwen3:4b

ps shows loaded models and CPU/GPU placement. stop unloads a model while keeping its downloaded files. The next request loads it again.

4. Test the Ollama API and response speed

An application needs a working HTTP response. Run this on the VPS:

curl --fail --silent --show-error \
  http://127.0.0.1:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3:4b",
    "prompt": "Explain an SSH key to a beginner in three sentences.",
    "stream": false,
    "think": false,
    "options": {"num_ctx": 4096, "num_predict": 128}
  }' | jq '{
    response,
    done,
    total_seconds: (.total_duration / 1000000000),
    load_seconds: (.load_duration / 1000000000),
    generation_tokens_per_second:
      (if .eval_duration > 0
       then .eval_count / (.eval_duration / 1000000000)
       else null end)
  }'

Look for an answer and done: true. stream: false returns one JSON response after generation finishes. think: false disables the Qwen3 model's thinking mode for this short test; controls can differ with other models.

Repeat the request. You just unloaded the model, so the first call includes loading time; the repeat shows its warm performance. The Generate API reports durations in nanoseconds, hence the divisions by one billion.

5. Access Ollama remotely through an SSH tunnel

On your own computer, open a separate terminal:

ssh -N -o ExitOnForwardFailure=yes -L 127.0.0.1:11435:127.0.0.1:11434 deploy@SERVER_IP

Leave it open. A quiet terminal is normal: -N opens the tunnel without a remote shell. The explicit local bind keeps the forwarded port on your computer's loopback address. See OpenSSH's forwarding documentation.

In another terminal on your computer:

curl --fail http://127.0.0.1:11435/api/tags

In Windows PowerShell, use curl.exe in place of curl for that command. The response should list your downloaded model.

Where the client runsOllama server address
Directly on the VPShttp://127.0.0.1:11434
On your computer, with the tunnel openhttp://127.0.0.1:11435

Port 11435 avoids colliding with a local Ollama installation using 11434. If it's occupied too, change the local port in both the tunnel and client address. Ctrl+C closes the tunnel; Ollama keeps running on the VPS.

Add a browser interface when you need one

Open WebUI is an optional chat interface with persistent accounts and history. Follow its deployment guide after the API works.

A container's 127.0.0.1 normally refers to the container. Connecting it to the host's loopback-only Ollama needs an appropriate network setup. Keep the UI private, allow extra RAM and retain its data volume and secret key when updating.

Restart, back up and update

On the VPS:

sudo systemctl restart ollama
ollama list

Repeat the API request. Your downloaded model should still be available; the first call after restarting may take longer because it reloads into memory.

Save the service override and custom model definitions. Back up anything you can't download again, including a chat interface's database and secrets if you add one. Our self-hosted app backup guide covers persistent data and restore checks.

For updates, record ollama --version, review the Linux update instructions and rerun a known prompt afterward. Those instructions also explain selecting a specific installer version for repeatable deployments.

Common Ollama VPS problems

ProblemCheck this first
API connection refused on the VPSService status and journalctl -u ollama.
Laptop cannot connectTunnel still open; client using port 11435.
SSH reports the local address is in useChoose another local port and update the client address.
Model download failsModel tag, outbound connectivity and free disk space with df -h.
Generation freezes or the model process disappearsAvailable RAM, swap activity and kernel out-of-memory messages.
Replies are consistently too slowTry a smaller model and shorter prompt; check provider CPU limits.

For memory problems, run these on the VPS during a request:

free -h
vmstat 1 10
ollama ps
sudo journalctl -k --since '15 minutes ago' | grep -Ei 'oom|out of memory|killed process'

Sustained swapping can make inference painfully slow. A CPU/GPU split in ollama ps describes model placement, not automatic offloading to disk. If memory is healthy but replies remain too slow, compare a GPU VPS with the same workload.

FAQ

Can I run Ollama on an 8 GB VPS without a GPU?

Yes, small quantized models are candidates for CPU inference on an 8 GB VPS. This guide uses qwen3:4b with a 4,096-token context as a starting test. Other services, longer context and simultaneous requests increase memory demand.

Does Ollama need Docker on Ubuntu?

No. This guide installs Ollama directly and manages it with systemd. Docker is a separate deployment choice, useful for some optional interfaces and apps.

Is a local model on a VPS completely private?

Local inference keeps model computation on that VPS, and this setup disables Ollama's cloud features. The hosting provider still operates the underlying infrastructure. Review connected tools, logging, access and backups before using sensitive prompts.

Should I open port 11434 for remote Ollama access?

Use the SSH tunnel for this setup. Public access would expose a local API without authentication. Shared access needs an authenticated access layer, plus HTTPS or a suitable private network.

  1. Plan CPU, RAM and storage for an AI VPS
  2. Work out model and context memory requirements
  3. Decide whether your local model needs a GPU VPS

Next step

Keep the deployment moving

Sources and further reading

Commands assume a fresh Ubuntu server. Replace domains, users, database names and provider-specific values before running them. VPS sizing is a practical starting point, not a guarantee: monitor the real workload and resize when CPU, memory, disk or network headroom becomes tight.

Where to run this

Affiliate links: you pay the same price as buying direct.