Skip to content

Instantly share code, notes, and snippets.

@nctiggy

nctiggy/edge-deployment-guide.md

Last active April 16, 2026 18:19
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save nctiggy/2fb1d4a6597058ed79dcc620538124e7 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save nctiggy/2fb1d4a6597058ed79dcc620538124e7 to your computer and use it in GitHub Desktop.
Download ZIP
Edge Deployment Guide: ISO to Running 2-Node Cluster
Raw
edge-deployment-guide.md

Edge Deployment Guide: ISO to Running 2-Node Cluster

Edge Deployment Guide: ISO to Running 2-Node Cluster

Audience: Pavan Jetty Last Updated: 2026-04-16 CanvOS Version: v4.8.20 Palette Version: 4.x SaaS (api.spectrocloud.com)

This guide walks through the complete process: building a Palette Edge ISO, installing it on bare metal, configuring networking, registering hosts with Palette, and creating a 2-node HA cluster.

Table of Contents

  1. Prerequisites
  2. Build the Edge ISO
  3. Create a Bootable USB
  4. Install the ISO on Bare Metal
  5. First Boot: TUI Network Configuration
  6. Alternative: Static IP via Site User Data USB
  7. Verify Edge Host Registration in Palette
  8. Create a 2-Node Cluster in Palette
  9. Attach Addon Profiles
  10. Troubleshooting

1. Prerequisites

Build Machine (for ISO creation)

  • Linux AMD64 machine (Ubuntu 22.04+)
  • 4+ CPU cores, 8GB+ RAM, 150GB free disk
  • Docker, Git, and Earthly installed
  • See docs/build-server-cloud-init.yaml for automated EC2 build server setup

Palette Access

  • Admin access to your Palette tenant
  • A project created for this deployment (e.g., tb-us-dev)
  • A registration token (Settings > Registration Tokens > Add New)

Bare Metal Nodes

  • 2 identical nodes (Taco Bell's Red0 and Red1 equivalent)
  • Each node needs:
    • USB port for installation
    • Network connectivity to the internet (for Palette registration)
    • Minimum: 4 CPU, 16GB RAM, 100GB NVMe/SSD
  • Know the one-time boot menu hotkey for your hardware (commonly F7, F11, or F12)

Tools on Your Laptop

  • Balena Etcher for flashing USBs
  • SSH client
  • Web browser for Palette UI

2. Build the Edge ISO

SSH to your build server and run:

# Clone CanvOS
git clone https://github.com/spectrocloud/CanvOS.git
cd CanvOS
git checkout v4.8.20

# Create .arg file
cat > .arg << 'EOF'
OS_DISTRIBUTION=ubuntu
OS_VERSION=24.04
K8S_DISTRIBUTION=k3s
K8S_VERSION=1.33.5
IMAGE_REGISTRY=ttl.sh
IMAGE_REPO=tb-store-edge
CUSTOM_TAG=tb-store
ARCH=amd64
TWO_NODE=true
UPDATE_KERNEL=true
HTTPS_PROXY=
HTTP_PROXY=
PROXY_CERT_PATH=
CLUSTERCONFIG=spc.tgz
CIS_HARDENING=false
EOF

IMPORTANT: TWO_NODE=true configures Postgres + Kine instead of etcd (required for 2-node HA). UPDATE_KERNEL=true includes kernel headers (required for Piraeus/DRBD storage).

Create the user-data file that gets baked into the ISO:

# user-data
#cloud-config
install:
  poweroff: true
  bind_mounts:
    - /etc/lvm
    - /var/lib/drbd
    - /var/lib/linstor.d
    - /var/lib/linstor-pools

stylus:
  site:
    paletteEndpoint: api.spectrocloud.com
    edgeHostToken: <YOUR_REGISTRATION_TOKEN>
    projectName: <YOUR_PROJECT_NAME>

users:
  - name: kairos
    passwd: kairos
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    groups:
      - admin

Replace <YOUR_REGISTRATION_TOKEN> and <YOUR_PROJECT_NAME> with your actual values from Palette.

The bind_mounts entries are required for Piraeus/LINSTOR DRBD storage to persist across reboots.

install.poweroff: true means the node will power off after installation (not reboot). This is intentional — you remove the USB, then power on.

Build the provider image and ISO:

# Build the installer ISO
earthly +iso

# Build and push the provider image (used by cluster profile later)
earthly --push +build-provider-images

The ISO will be at build/palette-edge-installer.iso (or similar name). Copy it to your laptop:

scp ubuntu@<BUILD_SERVER>:~/CanvOS/build/*.iso .

3. Create a Bootable USB

Using Balena Etcher (Recommended)

  1. Download and install Balena Etcher
  2. Insert a USB drive (8GB+ recommended)
  3. Open Etcher
  4. Click Flash from file and select the .iso file
  5. Select your USB drive
  6. Click Flash!

SCREENSHOT NEEDED: screenshot-etcher-flash.png — Balena Etcher with the ISO selected, ready to flash

Using dd (Linux/macOS)

# Find your USB device (BE CAREFUL — wrong device = data loss)
# macOS: diskutil list
# Linux: lsblk

# macOS example (replace diskN with your USB):
sudo dd if=palette-edge-installer.iso of=/dev/rdiskN bs=4m status=progress
sync

# Linux example (replace sdX with your USB):
sudo dd if=palette-edge-installer.iso of=/dev/sdX bs=4M status=progress
sync

4. Install the ISO on Bare Metal

Repeat this for both nodes (Red0 and Red1):

  1. Insert the bootable USB into the node
  2. Power on and hit the one-time boot menu hotkey during POST:
    • F7 — common on Intel NUC, MS-01, many UEFI boards
    • F11 — Dell, many server boards
    • F12 — Lenovo, HP, many laptops
  3. Select the USB drive from the boot menu

IMPORTANT: Do NOT change the permanent BIOS boot order. Leave hard disk as the first boot device. Use the one-time boot menu instead. This way, after installation, the node boots from the internal disk automatically — no need to go back into BIOS.

SCREENSHOT NEEDED: screenshot-boot-menu.png — One-time boot menu showing USB drive selected (hardware-specific)

  1. The Palette Edge installer runs automatically. You'll see:
    • Partitioning the disk
    • Copying the OS image
    • Installing the bootloader
    • Progress messages scrolling

SCREENSHOT NEEDED: screenshot-installer-progress.png — Console showing the installer running (partitioning/copying)

  1. When complete, the node powers off (because install.poweroff: true)
  2. Remove the USB drive
  3. Power the node back on — it boots from the internal disk automatically (boot order was never changed)

5. First Boot: TUI Network Configuration

On first boot from the internal disk, the Palette Edge TUI (Text User Interface) appears automatically.

TUI Landing Page

The landing page shows system information:

  • Host UID (e.g., edge-9df46853e4ef4d7c893c33397c6f1b56)
  • IP address (DHCP by default)
  • Palette endpoint
  • Agent version

SCREENSHOT NEEDED: screenshot-tui-landing.png — TUI landing page showing host UID, IP, and system info

Enter Configuration Mode

Press F2 to enter configuration mode.

Step 1: Login or Create User

If you included users in the user-data (we did — kairos/kairos), you'll see a login prompt. Enter:

  • Username: kairos
  • Password: kairos

SCREENSHOT NEEDED: screenshot-tui-login.png — TUI login screen

Step 2: Hostname

Set a meaningful hostname for this node (e.g., tb-store-red0 or tb-store-red1).

SCREENSHOT NEEDED: screenshot-tui-hostname.png — TUI hostname configuration screen

Step 3: Network Adapter Selection

You'll see a list of available network adapters. Select the one connected to your network.

SCREENSHOT NEEDED: screenshot-tui-nic-list.png — TUI showing available network adapters with DHCP assignments

Step 4: Configure Static IP

Select the adapter and switch from DHCP to Static:

Field Node 1 (Red0) Node 2 (Red1)
IP Address 10.0.12.10 10.0.12.11
Subnet Mask 255.255.255.0 255.255.255.0
Gateway 10.0.12.1 10.0.12.1

Adjust these IPs to match your store's network. Both nodes must be on the same subnet.

SCREENSHOT NEEDED: screenshot-tui-static-ip.png — TUI static IP configuration with IP, mask, and gateway filled in

Step 5: DNS Configuration

Set your DNS servers:

  • Primary: 10.0.11.2 (or your DNS server)
  • Secondary: 8.8.8.8 (Google DNS as fallback)
  • Search domain: (optional)

SCREENSHOT NEEDED: screenshot-tui-dns.png — TUI DNS configuration screen

Step 6: Apply and Logout

Press Enter to apply the configuration, then navigate to Logout.

The node will apply the network settings and the Palette agent will connect to api.spectrocloud.com using your registration token.

6. Alternative: Static IP via Site User Data USB

If you prefer to configure networking without the TUI (useful for automated deployments), you can provide a site-user-data USB alongside the installer USB.

Create the Site User Data File

Create a file called user-data:

#cloud-config
stylus:
  site:
    network:
      interfaces:
        ens18:
          type: static
          ipAddress: "10.0.12.10"
          mask: "255.255.255.0"
          gateway: "10.0.12.1"
          nameserver: "10.0.11.2"
      nameserver: "10.0.11.2"

IMPORTANT: Replace ens18 with your actual NIC name. Common names:

  • ens18 / ens192 — VMware/Proxmox
  • ens5 — AWS EC2 (Nitro)
  • enp1s0 / enp55s0f0np0 — bare metal (varies by hardware)

If unsure, boot with DHCP first and check ip addr output for the active NIC.

Create a node-specific file for each node:

Node 1 (Red0): user-data with ipAddress: "10.0.12.10" Node 2 (Red1): user-data with ipAddress: "10.0.12.11"

Create the Site User Data ISO

# Create empty meta-data file (required)
touch meta-data

# Build the ISO
mkisofs -output site-user-data.iso -volid cidata -joliet -rock user-data meta-data

On macOS, install cdrtools: brew install cdrtools

Flash to USB

Use Balena Etcher to flash site-user-data.iso to a second USB drive. Etcher will warn "this image does not appear to be bootable" — this is expected, proceed anyway.

Deploy

  1. Insert both USBs into the node:
    • USB 1: Edge Installer ISO (bootable)
    • USB 2: Site User Data ISO (configuration only — not bootable)
  2. Power on and use the one-time boot menu (F7/F11/F12) to select USB 1
  3. The installer automatically detects USB 2 and applies the site user data
  4. Node powers off after install — remove both USBs
  5. Power on — boots from internal disk with the static IP pre-configured, no TUI interaction needed

7. Verify Edge Host Registration in Palette

After both nodes boot with network connectivity:

  1. Log in to Palette at https://console.spectrocloud.com
  2. Navigate to your project (e.g., tb-us-dev)
  3. Go to Clusters > Edge Hosts tab
  4. Within 3-5 minutes, both nodes should appear as Registered

SCREENSHOT NEEDED: screenshot-palette-edge-hosts.png — Palette Edge Hosts tab showing both nodes registered with their UIDs and "Registered" state

Each host shows:

  • Host UID: The unique identifier (matches what the TUI displayed)
  • State: Registered (ready for cluster assignment)
  • Health: Healthy
  • IP Address: The static IP you configured

If a node doesn't appear after 5 minutes, SSH in and check:

ssh kairos@10.0.12.10   # password: kairos
systemctl status stylus  # Palette agent status
journalctl -u stylus -f  # Agent logs
curl -s https://api.spectrocloud.com/v1/health  # Connectivity test

8. Create a 2-Node Cluster in Palette

Prerequisites

  • Both edge hosts showing as Registered in Palette
  • An infrastructure cluster profile with:
    • BYOOS (pointing to your provider image from Step 2)
    • Palette Optimized K3s 1.33.5
    • Calico CNI
  • A storage addon profile with Piraeus Operator + CSI

Create the Cluster

  1. Go to Clusters > Add New Cluster > Edge Native

SCREENSHOT NEEDED: screenshot-palette-new-cluster.png — New cluster creation screen

  1. Enter cluster details:

    • Name: tb-store-001 (or your store identifier)
    • Description: Taco Bell Store #001 — 2-Node HA
    • Select your project

  2. Select your infrastructure cluster profile

  3. Configure cluster settings:

    • Toggle "Two-Node" to ON (critical!)
    • VIP: Enter one of the node IPs as a placeholder (e.g., 10.0.12.10)
      • For bare metal with MetalLB, this VIP field is a placeholder — MetalLB handles the real VIP
    • NTP Server: time.google.com
    • SSH Key: Your SSH public key

SCREENSHOT NEEDED: screenshot-palette-two-node-toggle.png — Cluster creation showing the Two-Node toggle ON, VIP field, and NTP config

  1. Configure the Control Plane Pool:
    • Delete the worker pool (two-node uses control-plane-as-worker only)
    • Select your first edge host → assign as Primary
    • Select your second edge host → assign as Secondary

SCREENSHOT NEEDED: screenshot-palette-node-assignment.png — Node pool showing Primary and Secondary edge host assignments

  1. Configure each node (click the Configure button next to each host):
    • Select the network interface (e.g., ens18, ens5)
    • Set IP type to Static
    • The IP, subnet, and gateway should auto-populate from the edge host's current config
    • Verify and save

SCREENSHOT NEEDED: screenshot-palette-node-configure.png — Per-node network configuration showing interface selection and static IP

  1. Review and click Deploy

Watch the Deployment

The cluster will take 10-15 minutes to fully provision:

  • K3s bootstraps on both nodes
  • Kine/Postgres initializes (2-node backend)
  • Calico CNI deploys
  • Nodes join the cluster

In Palette, watch the cluster state progress:

  • Provisioning → K3s installing
  • Running → Cluster is healthy

SCREENSHOT NEEDED: screenshot-palette-cluster-provisioning.png — Cluster in "Provisioning" state with progress indicators

SCREENSHOT NEEDED: screenshot-palette-cluster-running.png — Cluster in "Running" state showing both nodes healthy

9. Attach Addon Profiles

Once the cluster is Running, attach addon profiles to deploy your store applications. The order matters:

Step 1: Piraeus Storage

Attach the Piraeus addon profile first — other profiles need PVCs.

  • Wait for Piraeus satellites to be Running on both nodes (~3 min)

Step 2: MetalLB

Attach the MetalLB addon profile for LoadBalancer VIP support.

  • Wait for MetalLB controller and speakers to be Running (~1 min)

Step 3: Redis

Attach the Redis addon profile.

  • Wait for tb-redis-master-0 pod to be Running (~1 min)

Step 4: MongoDB

Attach the MongoDB addon profile.

  • Wait for tb-mongodb pod to be Running (~2 min)

Step 5: RabbitMQ (optional)

If you're demonstrating device communication via MQTT:

  • Wait for tb-rabbitmq pod to be Running (~2 min)

Step 6: Store Applications

Attach the store apps addon profile last — it needs Redis and MongoDB running.

  • All 8 application pods will start
  • CronJob begins generating orders every minute

Verify

Open the store dashboard in your browser:

http://<METALLB_VIP>:8090

You should see:

  • All services healthy (green dots)
  • Orders accumulating
  • Revenue growing
  • Kitchen queue active
  • Drive-thru cars in line

SCREENSHOT NEEDED: screenshot-store-dashboard.png — Store dashboard showing live activity

10. Troubleshooting

Node Won't Boot from USB

Issue Fix
BIOS doesn't see USB Try a different USB port (USB 2.0 ports more reliable)
Boot menu doesn't show USB Try different hotkey (F7, F11, F12) or check BIOS that USB boot is not disabled
Secure Boot error Disable Secure Boot in BIOS

Node Doesn't Register with Palette

Issue Fix
No network Check cable, verify IP with ip addr
Can't reach Palette curl https://api.spectrocloud.com/v1/health
Wrong token Check cat /oem/userdata for correct token
Wrong project Verify projectName in user-data
Agent not running systemctl restart stylus

Cluster Stuck in Provisioning

Issue Fix
Nodes not joining SSH in, check journalctl -u k3s -f
DRBD failing Verify UPDATE_KERNEL=true was set during ISO build
PVCs stuck Pending Check kubectl get sc — piraeus-fs-storage must exist

Storage Issues (Piraeus/DRBD)

Issue Fix
Satellites CrashLoopBackOff Kernel headers missing — rebuild ISO with UPDATE_KERNEL=true
CSI pods stuck in Init Satellites must be Running first — wait
PVCs Pending Check LINSTOR: kubectl exec -n piraeus-system deploy/linstor-controller -- linstor storage-pool list

Useful Debug Commands (SSH to node)

# Palette agent
systemctl status stylus
journalctl -u stylus -f

# K3s
systemctl status k3s
journalctl -u k3s -f
kubectl get nodes
kubectl get pods -A

# Network
ip addr
ip route
ping <other-node-ip>
curl -s https://api.spectrocloud.com/v1/health

# Storage
cat /proc/drbd    # DRBD status
lsblk             # Block devices

Screenshots Checklist

Please capture these screenshots for the guide:

# Screenshot When to Capture
1 screenshot-etcher-flash.png Balena Etcher with ISO selected, ready to flash
2 screenshot-boot-menu.png One-time boot menu with USB selected (F7/F11/F12)
3 screenshot-installer-progress.png Console during ISO installation (partitioning/copying)
4 screenshot-tui-landing.png TUI landing page showing host UID, IP, system info
5 screenshot-tui-login.png TUI login screen
6 screenshot-tui-hostname.png TUI hostname configuration
7 screenshot-tui-nic-list.png TUI network adapter selection list
8 screenshot-tui-static-ip.png TUI static IP config with IP/mask/gateway
9 screenshot-tui-dns.png TUI DNS configuration
10 screenshot-palette-edge-hosts.png Palette Edge Hosts tab with both nodes registered
11 screenshot-palette-new-cluster.png New Edge Native cluster creation screen
12 screenshot-palette-two-node-toggle.png Two-Node toggle ON with VIP and NTP config
13 screenshot-palette-node-assignment.png Node pool with Primary/Secondary assignments
14 screenshot-palette-node-configure.png Per-node network interface and static IP config
15 screenshot-palette-cluster-provisioning.png Cluster in Provisioning state
16 screenshot-palette-cluster-running.png Cluster Running with both nodes healthy
17 screenshot-store-dashboard.png Store dashboard showing live activity

Tip: For the TUI screenshots, use the physical console or a KVM-over-IP connection. The TUI renders in a terminal — screenshots from a monitor or screen capture tool work fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment