Intro

This Quickstart receipe for Qemu assumes a recent FreeBSD release (stable/13 or newer), and provides an example configuration for running arm64 (aarch64) FreeBSD guest on a amd64 FreeBSD Host. Concepts can be applied to other architectures as desired, but syntax and capabilities will vary.

Dependancies

Install qemu pkg install qemu or pkg instal qemu-nox11. Latest release as of this writeup is 8.1.0
Sufficient disk space (50+GB) on a mounted FreeBSD Host disk (e.g.: /qemu-data in this example)
Network environment that allows for multiple mac addresses on one switch port (or vswitch port configuration) for bridging mode
Configure bridge(as desired) in /etc/rc.conf by adding:

cloned_interfaces="bridge0"
ifconfig_bridge0="addm vmx0 edge vmx0 up"

Modify vmx0 to match your existing host network interface name. Wireless (802.11) interfaces may not work properly in bridging mode.

Getting Started

Download desired qcow2 image from FreeBSD site, decompress, and resize (to increase by desired size. Example shows adding 40GB):

mkdir /qemu-data
cd /qemu-data
fetch https://download.freebsd.org/releases/VM-IMAGES/14.0-BETA1/aarch64/Latest/FreeBSD-14.0-BETA1-arm64-aarch64.qcow2.xz
unxz FreeBSD-14.0-BETA1-arm64-aarch64.qcow2.xz
qemu-img resize FreeBSD-14.0-BETA1-arm64-aarch64.qcow2 +40G

Install qemu, novnc, python

pkg install qemu novnc python

Configure networking bridge for use by Qemu guests (replace vmx0 with host network interface name)

ifconfig bridge0 create
ifconfig bridge0 addm vmx0 edge vmx0 up

Make permanant in /etc/rc.conf by adding:

ifconfig_bridge0="addm vmx0 edge vmx0 up"
cloned_interfaces="bridge0"

Create Qemu network ifup/ifdown scripts to handle bridge and tap syntax.
vi /etc/qemu-ifup

#!/bin/sh
ifconfig bridge0 addm $1 up
ifconfig $1 up

vi /etc/qemu-ifdown

#!/bin/sh
ifconfig $1 down
ifconfig bridge0 deletem $1

Enable the scripts to be directly executed by root user from qemu using:

chmod +x /etc/qemu_if*

Boot Qemu with the pre-installed qcow2 image (text mode)

qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt  \
        -bios edk2-aarch64-code.fd -serial telnet:localhost:4444,mux=on,server,wait=off -vnc :0,websocket=on \
        -drive if=none,file=/qemu_data/FreeBSD-14.0-BETA1-arm64-aarch64.qcow2,id=hd0 \
        -device virtio-blk-device,drive=hd0 \
        -device virtio-net-device,netdev=net0 \
        -netdev tap,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown,id=net0

(optional) /qemu/data/start_arm.sh Example:

#!/bin/sh
if [ ! -z ${1} ] ; then
        echo "Attempting to passthrough usb host device based on query string: ${1}"
        usb_map_count=$(usbconfig | grep -cie ${1})
        [ ${usb_map_count} -ne 1 ] && \
                 echo "Total devices matched: ${usb_map_count} is not equal to 1, please refine." && \
                 usbconfig && exit 1
        usb_map=$(usbconfig | grep -ie ${1} | grep -o -e [0-9]\.[0-9])
        usb_map_bus=$(echo ${usb_map} | grep -o -e ^[0-9])
        usb_map_addr=$(echo ${usb_map} | grep -o -e [0-9]$)
        usb_qemu_cli="-device usb-host,hostbus=${usb_map_bus},hostaddr=${usb_map_addr},id=${1}"
        echo "Mapping usb device $(usbconfig | grep -ie ${1}) into the guest."
        echo -n "In qemu monitor, you can inspect with \"info usbhost\" command, "
        echo "or delete the usb device mapping with \"device_del ${1}\""
fi
ifconfig tap0 2>/dev/null | grep -cq -e "Opened by PID" || ifconfig tap0 destroy 2>/dev/null
ps -aux | grep -v grep | grep -cq qemu-system && \
        echo "Qemu is already running. Shutdown the guest(s), then retry." && exit 1
echo "Starting Qemu in background..."
qemu-system-aarch64 -m 4096M -cpu max -smp cpus=4 -M virt \
        -bios edk2-aarch64-code.fd \
        -serial telnet:localhost:4444,mux=on,server,wait=off \
        -monitor telnet:localhost:4445,mux=on,server,wait=off \
        -display none \
        -drive if=none,file=/data/FreeBSD-14.0-BETA3-arm64-aarch64.qcow2,id=hd0 \
        -device virtio-blk-device,drive=hd0 \
        -device virtio-net-device,netdev=net0 \
        -netdev tap,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown,id=net0 \
        -usb \
        -device qemu-xhci,id=xhci \
        ${usb_qemu_cli} \
        -daemonize

Login as root with no password, immediately set one using passwd.
Create users using adduser, start sshd using service sshd enable && service sshd start
Edit configuration file in /etc/rc.conf and give a hostname. verify settings.
Use the new Arm64 (aarch64) version of FreeBSD 14. Since this is now a higher tier variant, there will be pre-built packages for use with pkg utility as well as all of the other normal FreeBSD base things. Package builds for betas may lag ISO releases.

Access Qemu guest console

Connect over telnet protocol from localhost to the guest console. Note: Use Ctrl+] to get a telnet prompt, then quit to exit telnet

telnet localhost 4444

Access Qemu monitor console

Connect over telnet protocol from localhost to the qemu monitor. Note: Use Ctrl+] to get a telnet prompt, then quit to exit telnet

telnet localhost 4445

TODO

Determine why attempting to destroy tapX interface using ifconfig tap0 destroy format hangs instead of erroring out when there is still a pid associated with the interrface.
Write a detection script to check if there is a pid attached, and destroy if not: ifconfig tap0 | grep -cq -e "Opened by PID" || ifconfig tap0 destroy
Document Yubikey passthrough from ESXi->FreeBSD VM(amd64)->FreeBSD QEMU(aarch64)
Document u2f virtual device (emulated and passthrough)
Document Fresh aarch64 install via ISO vs QCOW Image

daemonhorn/FreeBSD Arm64 Qemu.md