A Local Chat Interface for Ollama

Ollama makes it easy to run large language models locally. Its API is useful for scripts and integrations, but a web interface is more convenient for everyday conversations. Open WebUI provides that interface and can connect to an Ollama service running in another container.

This article shows how to run both services with Podman and podman-compose. The compose file keeps Ollama private inside a container network and exposes Open WebUI only on the local machine.

Installing Podman and podman-compose

On Debian, Ubuntu, Fedora, RHEL, or another distribution using apt or dnf, install Podman and podman-compose:

sudo apt update && sudo apt install podman podman-compose || sudo dnf install podman podman-compose

Verify the installation:

podman --version
podman-compose --version

The Compose File is using an AMD GPU

Save the following configuration as ollama.yaml: It is tested on an AMD Radeon integrated GPU on my laptop (works with Vulkan).

version: "3.8"

# Use command:
# podman-compose -f ollama.yaml up -d
#
# Pull model using ollama directly inside podman container:
# podman exec -it ollama ollama pull llama3.1:8b
# podman exec -it ollama ollama pull llama3.2:1b
# podman exec -it ollama ollama pull gpt-oss:20b

services:
  ollama:
    image: docker.io/ollama/ollama:latest
    container_name: ollama
    devices:
      - /dev/dri:/dev/dri
      - /dev/kfd:/dev/kfd
    group_add:
      - keep-groups
    environment:
      - OLLAMA_IGPU_ENABLE=1
      - OLLAMA_HOST=0.0.0.0
    volumes:
      - ollama:/root/.ollama
    networks:
      - ollama-net
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_AUTH=False
    volumes:
      - open-webui:/app/backend/data
    ports:
      - "127.0.0.1:10000:8080"
    networks:
      - ollama-net
    restart: unless-stopped

networks:
  ollama-net:
    driver: bridge

volumes:
  ollama:
  open-webui:

The OLLAMA_BASE_URL value uses the service name ollama, so Open WebUI can reach Ollama through the internal network. Ollama does not need a host port. The 127.0.0.1 binding also prevents Open WebUI from being directly accessible from other machines.

The device mappings enable GPU access on systems that provide /dev/dri and /dev/kfd, such as systems with an AMD integrated GPU. If those devices are not present on the host, remove the devices and group_add entries and run the containers using the CPU.

Using an NVIDIA GPU

For an NVIDIA GPU, install the NVIDIA driver and NVIDIA Container Toolkit according to the official NVIDIA instructions. After configuring the NVIDIA repository, install the toolkit with:

sudo apt install nvidia-container-toolkit || sudo dnf install nvidia-container-toolkit

Generate the Container Device Interface (CDI) specification and verify that the GPU is available:

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
nvidia-ctk cdi list
nvidia-smi

Use the following ollama service instead of the AMD configuration above:

  ollama:
    image: docker.io/ollama/ollama:latest
    container_name: ollama
    devices:
      - nvidia.com/gpu=all
    environment:
      - OLLAMA_HOST=0.0.0.0
    volumes:
      - ollama:/root/.ollama
    networks:
      - ollama-net
    restart: unless-stopped

Remove the AMD-specific /dev/dri, /dev/kfd, group_add, and OLLAMA_IGPU_ENABLE settings. Use either the AMD or NVIDIA GPU configuration, not both at the same time. The nvidia-smi command must work on the host before starting the containers.

The named volumes keep downloaded models and the Open WebUI database when containers are recreated.

Starting the Services

Change to the directory containing ollama.yaml and start the stack:

podman-compose -f ollama.yaml up -d

Check that both containers are running:

podman ps
podman-compose -f ollama.yaml logs -f

Open http://localhost:10000 in a browser. The compose file disables authentication for a simple local setup, so the interface is available immediately.

Downloading a Model

Models are downloaded into the persistent ollama volume. For example:

podman exec -it ollama ollama pull llama3.1:8b

Other possible models include:

podman exec -it ollama ollama pull llama3.2:1b
podman exec -it ollama ollama pull gpt-oss:20b

Note that you should select a model that fits into the VRAM available on your GPU.

After downloading a model, select it in Open WebUI and start a conversation. The model size should match the available system memory and GPU resources. Smaller models are a better starting point on machines with limited resources.

Managing the Stack

Useful commands for daily operation:

# Stop the containers and keep the volumes
podman-compose -f ollama.yaml down

# Start existing containers again
podman-compose -f ollama.yaml start

# Show the Ollama container logs
podman logs -f ollama

# List models stored by Ollama
podman exec -it ollama ollama list

Do not remove the named volumes unless the downloaded models and Open WebUI data are no longer needed. To update the images, pull them and recreate the services:

podman-compose -f ollama.yaml pull
podman-compose -f ollama.yaml up -d

Conclusion

Podman Compose provides a small and repeatable way to run Ollama with Open WebUI. Ollama remains isolated from the host network, Open WebUI stores its data in a persistent volume, and the whole setup can be started with one command. This makes it convenient to experiment with local models without installing the application stack directly on the host system.