This document describes the original setup to establish an FRR (Free Range Routing) OpenFabric IS-IS based IPv6 routed mesh over Thunderbolt networking between Proxmox nodes, using static /128 loopback addresses in the fc00::/8 ULA space.
This provided:
I learn't an important lesson today, never ever remove ms_bind_ipv4 = false from ceph.conf or the cephFS will be fucked.
This document describes a safe, production-grade method for migrating a Proxmox+Ceph cluster to new /64 IPv6 loopback addresses for monitor daemons (MONs) without downtime, following best practices and using native Proxmox tools (pveceph, GUI).
this gist is part of this series
This assumes you are running Proxmox 8.4 and that the line source /etc/network/interfaces.d/* is at the end of the interfaces file (this is automatically added to both new and upgraded installations of Proxmox 8.2).
This changes the previous file design thanks to @NRGNet and @tisayama to make the system much more reliable in general, more maintainable esp for folks using IPv4 on the private cluster network (i still recommend the use of the IPv6 FC00 network you will see in these docs)
This is currently a work-in-progress documentation - rough notes for me, maybe missing a lot or wrong
The idea is to replace GlusterFS running inside the VM with storage on my cephfs cluster. This is my proxmox cluster and it runs both the storage and is the hypervisor for my docker VMs.
Other possible approaches:
Assumptions:
This container puts a label on each machine based on a config that matches service names. If the service is running the label gets a 1 and if the label get as 0 its not running. this can be used with constraints to either locate serices on a node with another service OR make sure a service doesn't land on a node with another service
I would love to find a better version of this that does this without the need for the manual config file (you can use a file bindmount instead of a config if you prefer)
State is all read-only in a config
this gist is part of this series
This assumes you are running Proxmox 8.2 and that the line source /etc/network/interfaces.d/* is at the end of the interfaces file (this is automatically added to both new and upgraded installations of Proxmox 8.2).
This changes the previous file design thanks to @NRGNet for the suggestions to move thunderbolt settings to a file in /etc/network/interfaces.d it makes the system much more reliable in general, more maintainable esp for folks using IPv4 on the private cluster network (i still recommend the use of the IPv6 FC00 network you will see in these docs)
This one is the one that has to work, even more so the domain controllers. This is what my swarm looks like
you may want to read from the bottom up as later migrations are where i had the process more locked and less experimentation
So the plan is as follows (and is based on my experience with home assistant oddlye enough)
Now that i have nailed the qm disk import command and given all linux kernel have the virtio drivers in them after 5.6 this should be a breeze!
Export VHD from Hyper-V into share proxmox can see (tbh at this point if you don't know how...)
I created a 4GB VM with no disks at all andthe virtio network. Make sure you connect it to a live bridge or hass will hand at starting network manager I added a TPM drive
