LOCAL AI
Default Port: 11434Minimum 12GB VRAMBest Cloud GPU Providers for Ollama & Open-WebUI
Ollama wraps GGUF quantization, model management, and local OpenAI-compatible APIs into a lightweight runner. Combined with Open-WebUI, it creates an instant private ChatGPT clone.
Verified 1-Click Cloud Launchers for Ollama & Open-WebUI
Click to launch directly with pre-installed CUDA drivers, PyTorch, and template weights
Direct Docker Run Command
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run deepseek-r1:32bDeployment Best Practices & Gotchas:
- 32B Q4 quantized models fit comfortably inside a single RTX 4090 (24GB VRAM).
- 70B models run best on 2x RTX 3090/4090 with layer offloading.
- Mount a persistent volume to /root/.ollama so downloaded model weights persist across restarts.
Recommended GPU Hardware for Ollama & Open-WebUI
To achieve reliable throughput without running into CUDA Out of Memory (OOM) errors, we recommend deploying on one of the following verified GPU models:
RTX 3090 (24GB)RTX 4090 (24GB)L40S (48GB)