A Local Chat Interface for Ollama

Ollama makes it easy to run large language models locally. Its API is useful for scripts and integrations, but a web interface is more convenient for everyday conversations. Open WebUI provides that interface and can connect to an Ollama service running in another container.

This article shows how to run Ollama, Open WebUI, and Open Terminal with Podman and podman-compose. Open WebUI provides the chat interface and connects to Ollama; Open Terminal adds a terminal integration for supported tools. Ollama and Open Terminal stay private inside the container network, while Open WebUI is exposed only on the local machine.

Installing Podman and podman-compose

On Debian, Ubuntu, Fedora, RHEL, or another distribution using apt or dnf, install Podman and podman-compose:

sudo apt update && sudo apt install podman podman-compose || sudo dnf install podman podman-compose

Verify the installation:

podman --version
podman-compose --version

The Compose File is using an AMD GPU

Save the following configuration as ollama.yaml: It is tested on an AMD Radeon integrated GPU on my laptop (works with Vulkan).

version: "3.8"

# Use command:
# podman-compose -f ollama.yaml up -d
#
# Pull models with podman:
# podman exec -it ollama ollama pull llama3.1:8b
# podman exec -it ollama ollama pull llama3.2:1b
# podman exec -it ollama ollama pull ministral-3:14b
# podman exec -it ollama ollama pull gpt-oss:20b
# podman exec -it ollama ollama pull gemma4:e4b
#
# Update the images:
# podman pull docker.io/ollama/ollama:latest
# podman pull ghcr.io/open-webui/open-webui:latest
# podman pull ghcr.io/open-webui/open-terminal:latest
#
# Stop the services:
# podman-compose -f ollama.yaml down
#
# Open Terminal API key:
# Set OPEN_TERMINAL_API_KEY in a .env file next to this file.
# If it is not set, the fallback "change-me" below is used; replace it.
services:
  ollama:
    image: docker.io/ollama/ollama:latest
    container_name: ollama
    devices:
      - /dev/dri:/dev/dri
      - /dev/kfd:/dev/kfd
    group_add:
      - keep-groups
    environment:
      - OLLAMA_IGPU_ENABLE=1
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_CONTEXT_LENGTH=65536
      - OLLAMA_NUM_PARALLEL=1
      - OLLAMA_FLASH_ATTENTION=1
      - OLLAMA_KV_CACHE_TYPE=q8_0
      - OLLAMA_KEEP_ALIVE=-1
    volumes:
      - ./.ollama:/root/.ollama:Z
    ports:
      - "127.0.0.1:11434:11434"
    networks:
      - ollama-net
    restart: unless-stopped
  open-terminal:
    image: ghcr.io/open-webui/open-terminal:latest
    container_name: open-terminal
    userns_mode: "keep-id:uid=1000,gid=1000"
    environment:
      - OPEN_TERMINAL_API_KEY=${OPEN_TERMINAL_API_KEY:-change-me}
    volumes:
      - ./open-terminal:/home/user:Z,U
    networks:
      - ollama-net
    restart: unless-stopped
  open-webui:
    image: ghcr.io/open-webui/open-webui:latest
    container_name: open-webui
    depends_on:
      - ollama
      - open-terminal
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_AUTH=False
      # Pre-configures the Open Terminal integration in Open WebUI.
      - 'TERMINAL_SERVER_CONNECTIONS=[{"id":"open-terminal","name":"Open Terminal","enabled":true,"url":"http://open-terminal:8000","key":"${OPEN_TERMINAL_API_KEY:-change-me}","auth_type":"bearer","config":{"access_grants":[{"principal_type":"user","principal_id":"*","permission":"read"}]}}]'
    volumes:
      - ./.open-webui:/app/backend/data:Z
    ports:
      - "127.0.0.1:10000:8080"
    networks:
      - ollama-net
    restart: unless-stopped
networks:
  ollama-net:
    driver: bridge

The OLLAMA_BASE_URL value uses the service name ollama, so Open WebUI can reach Ollama through the internal network. The terminal integration similarly uses the service name open-terminal; it does not need a host port. Ollama's API and Open WebUI are bound to 127.0.0.1, preventing direct access from other machines.

The open-terminal service uses a dedicated ./open-terminal host directory and is not published on a host port. Open WebUI is preconfigured to connect to it with bearer authentication. Set a private API key in a .env file in the same directory as ollama.yaml before starting the stack:

OPEN_TERMINAL_API_KEY=replace-this-with-a-long-random-secret

Do not share this key or leave the fallback value change-me in use. Open WebUI authentication is disabled in this example, so keep the interface bound to localhost and use the terminal integration only with trusted local users and models.

The device mappings enable GPU access on systems that provide /dev/dri and /dev/kfd, such as systems with an AMD integrated GPU. If those devices are not present on the host, remove the devices and group_add entries and run the containers using the CPU.

Using an NVIDIA GPU

For an NVIDIA GPU, install the NVIDIA driver and NVIDIA Container Toolkit according to the official NVIDIA instructions. After configuring the NVIDIA repository, install the toolkit with:

sudo apt install nvidia-container-toolkit || sudo dnf install nvidia-container-toolkit

Generate the Container Device Interface (CDI) specification and verify that the GPU is available:

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
nvidia-ctk cdi list
nvidia-smi

Use the following ollama service instead of the AMD configuration above:

  ollama:
    image: docker.io/ollama/ollama:latest
    container_name: ollama
    devices:
      - nvidia.com/gpu=all
    environment:
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_CONTEXT_LENGTH=65536
      - OLLAMA_NUM_PARALLEL=1
      - OLLAMA_FLASH_ATTENTION=1
      - OLLAMA_KV_CACHE_TYPE=q8_0
      - OLLAMA_KEEP_ALIVE=-1
    volumes:
      - ./.ollama:/root/.ollama:Z
    networks:
      - ollama-net
    restart: unless-stopped

Remove the AMD-specific /dev/dri, /dev/kfd, group_add, and OLLAMA_IGPU_ENABLE settings. Use either the AMD or NVIDIA GPU configuration, not both at the same time. The nvidia-smi command must work on the host before starting the containers.

The bind-mounted ./.ollama and ./.open-webui directories keep downloaded models and Open WebUI data on the host when containers are recreated.

Hardware Performance Testing

The configurations above were tested on a laptop equipped with an AMD Ryzen 7 5700U with integrated Radeon Graphics. When running Ollama via Podman with the AMD GPU configuration, the inference performance was satisfactory, and the GPU utilization remained within acceptable limits.

Additionally, the same stack was tested on a system with an integrated NVIDIA A1000 GPU. With the NVIDIA Container Toolkit installed and the devices: - nvidia.com/gpu=all setting in the ollama service, the model inference performed exceptionally well, demonstrating that Podman can effectively leverage the NVIDIA GPU for accelerated workloads.

Starting the Services

Change to the directory containing ollama.yaml and start the stack:

podman-compose -f ollama.yaml up -d

Check that all three services are running:

podman ps
podman-compose -f ollama.yaml logs -f

Open http://localhost:10000 in a browser. The compose file disables authentication for a simple local setup, so the interface is available immediately.

Downloading a Model

Models are downloaded into the persistent ./.ollama directory on the host. For example:

podman exec -it ollama ollama pull gemma4:e4b

Other possible models include:

podman exec -it ollama ollama pull llama3.2:1b
podman exec -it ollama ollama pull gpt-oss:20b

Note that you should select a model that fits into the VRAM available on your GPU.

After downloading a model, select it in Open WebUI and start a conversation. The model size should match the available system memory and GPU resources. Smaller models are a better starting point on machines with limited resources.

Using Open Terminal with Ollama

The Compose configuration connects Open WebUI to Open Terminal automatically, so there is no separate CLI installation or manual integration setup. Start the services, open http://localhost:10000, and choose a model that supports the tools you want to use. Open Terminal is available through Open WebUI's terminal integration.

The terminal service runs in its own container and mounts ./open-terminal at /home/user. Files created there persist on the host in that directory. Keep any files you do not want a model or terminal session to access outside this mounted directory.

Managing the Stack

Useful commands for daily operation:

# Stop the containers and keep the host data directories
podman-compose -f ollama.yaml down

# Start existing containers again
podman-compose -f ollama.yaml start

# Show the Ollama container logs
podman logs -f ollama

# List models stored by Ollama
podman exec -it ollama ollama list

Do not remove the ./.ollama and ./.open-webui host directories unless the downloaded models and Open WebUI data are no longer needed. To update the images, pull them and recreate the services:

podman-compose -f ollama.yaml pull
podman-compose -f ollama.yaml up -d

Conclusion

Podman Compose provides a repeatable way to run Ollama, Open WebUI, and Open Terminal together. The services communicate over a private container network, persistent data stays in host directories, and the stack can be started with one command. This makes it convenient to experiment with local models and terminal tools without installing the application stack directly on the host system.