1-Click Launch Recipes for Modal Labs

Docker & SSH

Copy-paste deployment recipes for ComfyUI, vLLM, Ollama, and PyTorch

Live Rates Cost Simulator Launch on Modal Labs

OpenAI-Compatible SDK Recipes

Developer API SDK & Code Recipes for Modal Labs

Tested OpenAI-standard drop-in code snippets for Python, TypeScript, and cURL.

OpenAI API Compatible

# 1. Install official OpenAI client:
# pip install openai

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.modal.com/v1",
    api_key=os.environ.get("MODAL_TOKEN_ID", "your_api_key_here")
)

response = client.chat.completions.create(
    model="vllm-llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful AI assistant."},
        {"role": "user", "content": "Explain KV cache compression in 3 concise bullet points."}
    ],
    temperature=0.6,
)

print(response.choices[0].message.content)

💡 Tip: Set MODAL_TOKEN_ID in your environment variables. Zero client SDK changes required.Open Modal Labs Console

1. ComfyUI + FLUX.1 Persistent Cache

Port 8188

Always mount your models directory to a persistent volume (e.g., /workspace/ComfyUI/models). This ensures that heavy 24GB checkpoint weights (FLUX.1 [dev], SDXL, SD 1.5) remain saved when you stop your instance, preventing hours of repetitive downloads.

# Mount models volume into container:
-v /workspace/ComfyUI/models:/root/ComfyUI/models \
-v /workspace/ComfyUI/output:/root/ComfyUI/output

2. High-Throughput vLLM Server

Port 8000

Deploy Meta-Llama-3.3-70B or DeepSeek-R1 with OpenAI-compatible API drop-in replacement. Set --tensor-parallel-size equal to your allocated GPU count (e.g., 2 or 4 GPUs for 70B models).

vllm/vllm-openai:latest \
  --model meta-llama/Llama-3.3-70B-Instruct \
  --tensor-parallel-size 2 \
  --gpu-memory-utilization 0.95

3. Port Forwarding & Networking

SSH Tunneling

If Modal Labs uses NAT without direct public IP mapping, use an SSH local port forward to access web UIs securely on your local browser:

# Forward remote ComfyUI (8188) to local localhost:8188:
ssh -L 8188:localhost:8188 root@<host_ip> -p <ssh_port>

4. Zero-Egress Cloud Backup (R2 / S3)

rclone

Avoid 24/7 idle stopped storage fees on Modal Labs by syncing your fine-tuned weights and model checkpoints directly to Cloudflare R2:

# Fast multi-threaded sync with rclone:
rclone copy /workspace/output r2:my-bucket/checkpoints --transfers 8