Ollama

ollama-setup.md

Option 1: Use with Ollama (easiest for local setup):

First, install and run Ollama:

# Install Ollama (if not already installed)
curl -fsSL https://ollama.com/install.sh | sh

# Start Ollama and pull a model
ollama serve &
ollama pull llama2  # or any other model

Then restart Open WebUI with Ollama configuration:

docker stop open-webui
docker rm open-webui

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Option 2: Use with OpenAI API:

docker stop open-webui
docker rm open-webui

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e OPENAI_API_KEY=your-api-key-here \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main
  
  
  # For linux
docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main
  
# alternative
docker run -d \
  --network host \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Ollam use

ollama list

ollama list  # Should show llama2

Test Ollama directly

ollama run llama2 "Hello, how are you?"
```

4. **Access Open WebUI:**
   Open your browser and go to:
```
   http://localhost:3000

What you can do right now:

1. ✅ Access http://localhost:3000
2. ✅ Create your account (first user = admin)
3. ✅ Explore the interface
4. ❌ Cannot chat yet - because llama2 is still downloading

To check if download is complete:

ollama list
```

When you see something like this, it's ready:
```
NAME            ID              SIZE    MODIFIED
llama2:latest   78e26419b446    3.8 GB  2 minutes ago

If the model doesn't appear in Open WebUI after download:

docker restart open-webui

Step 1: Launch EC2 Instance

Instance specs:

Instance Type: t3.small (or t3.micro if free tier available)
vCPU: 2
RAM: 2 GB (or 1 GB for t3.micro)
Storage: 30 GB GP3
OS: Ubuntu 22.04 LTS

Why t3.small over t3.micro?

• t3.micro (1GB RAM) might struggle with Docker
• t3.small (2GB RAM) is more comfortable
• Only $7-8 more per month

Step 2: Security Group Configuration

Open these ports:

Port 80 (HTTP): 0.0.0.0/0
Port 443 (HTTPS): 0.0.0.0/0  # For SSL later
Port 22 (SSH): Your IP only  # Security!

Step 3: Install Docker on EC2

SSH into your instance:

bash

ssh -i your-key.pem ubuntu@your-ec2-public-ip

Install Docker:

bash

# Update system
sudo apt update && sudo apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Add user to docker group
sudo usermod -aG docker ubuntu

# Start Docker
sudo systemctl enable docker
sudo systemctl start docker

# Log out and back in for group changes
exit
# SSH back in

Step 4: Deploy Open WebUI with OpenAI

bash

# Create .env file for API key (more secure)
echo "OPENAI_API_KEY=sk-your-actual-key-here" > openai.env

# Run Open WebUI
docker run -d \
  -p 80:8080 \
  -v open-webui:/app/backend/data \
  --env-file openai.env \
  -e WEBUI_NAME="Company AI Assistant" \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main
```

### **Step 5: Access Your Setup**
```
http://your-ec2-public-ip
```

Create first admin account when you visit!

## **Cost Breakdown (AWS + OpenAI)**
```
AWS EC2 t3.small:        $15/month
AWS EBS Storage (30GB):  $3/month
Data Transfer (minimal): ~$1/month
OpenAI API Usage:        $10-50/month (depends on usage)
------------------------
TOTAL:                   ~$29-69/month
```

**If using Free Tier (first 12 months):**
```
AWS EC2 t3.micro:        $0/month (free tier)
AWS EBS Storage:         $0/month (free tier)
OpenAI API Usage:        $10-50/month
------------------------
TOTAL:                   ~$10-50/month

AWS-Specific Benefits

✅ IAM Roles: Better security management ✅ CloudWatch: Built-in monitoring ✅ Elastic IP: Static IP address ✅ Backups: EBS snapshots ✅ Scalability: Easy to upgrade later ✅ VPC: Network isolation

Quick Setup Checklist

• Launch EC2 instance (t3.small or t3.micro)
• Configure Security Group (ports 80, 443, 22)
• Allocate Elastic IP (optional but recommended)
• SSH into instance
• Install Docker
• Get OpenAI API key
• Deploy Open WebUI container
• Access via public IP
• Create admin account

Optional: Add Domain & SSL

If you want a proper URL:

1. Point domain to Elastic IP (Route 53 or external DNS)
2. Install Nginx + Let's Encrypt:

bash

sudo apt install nginx certbot python3-certbot-nginx -y

# Configure nginx reverse proxy
sudo nano /etc/nginx/sites-available/openwebui

# Add SSL with certbot
sudo certbot --nginx -d your-domain.com