Skip to content

Instantly share code, notes, and snippets.

@ConstBur
Created December 26, 2020 23:51
Show Gist options
  • Save ConstBur/c598a4cfc96899c95a93393579118d3b to your computer and use it in GitHub Desktop.
Save ConstBur/c598a4cfc96899c95a93393579118d3b to your computer and use it in GitHub Desktop.
Zoneminder Docker GPU Setup

Prerequisites:

  • Machine with 8+ GB of RAM (at least 4 GB required for Docker to compile libraries)
  • 20-25 GB of minimum free disk space
  • Installed Docker and Nvidia driver on host system
  • Nvidia GPU supporting the same driver as host system (for Docker compatibility, preferably Kepler or later generation to support the latest driver) There can be multiple GPUs of different generations installed at the same time, but they all have to support the same driver. For example, my setup has a Quadro K2000 (Kepler) for display and Quadro M4000 (Maxwell) for compute.

Steps:

  1. Clone dlandon's Zoneminder Docker repository: https://github.com/dlandon/zoneminder
  2. Download dlandon's my_init script in the root of the cloned repository: cd zoneminder; wget https://raw.githubusercontent.com/phusion/baseimage-docker/master/image/bin/my_init; chmod a+x ./my_init
  3. Backup the default Dockerfile and docker-compose.yaml in the repository in case you need them back.
  4. Modify the Dockerfile. The list of changes is:

Here's my sample Dockerfile (455 driver, CUDA 11.1, CuDNN 8.0.5, Ubuntu 20.04):

FROM nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04 as builder

LABEL maintainer="dlandon"

ENV	DEBCONF_NONINTERACTIVE_SEEN="true" \
	DEBIAN_FRONTEND="noninteractive" \
	DISABLE_SSH="true" \
	HOME="/root" \
	LC_ALL="C.UTF-8" \
	LANG="en_US.UTF-8" \
	LANGUAGE="en_US.UTF-8" \
	TZ="Etc/UTC" \
	TERM="xterm" \
	PHP_VERS="7.4" \
	ZM_VERS="1.34" \
  ZMEVENT_VERS="5.7.4" \
	SHMEM="50%" \
	PUID="99" \
	PGID="100"

FROM builder as build1
COPY init/ /etc/my_init.d/
COPY defaults/ /root/
COPY ./my_init  /sbin/

RUN apt-get update && \
    apt-get install -y apt-utils software-properties-common

RUN	add-apt-repository -y ppa:iconnor/zoneminder-$ZM_VERS && \
	add-apt-repository ppa:ondrej/php && \
	apt-get update && \
	apt-get -y upgrade -o Dpkg::Options::="--force-confold" && \
	apt-get -y dist-upgrade -o Dpkg::Options::="--force-confold" && \
	apt-get -y install apache2 mariadb-server && \
	apt-get -y install ssmtp mailutils net-tools wget sudo make && \
	apt-get -y install php$PHP_VERS php$PHP_VERS-fpm libapache2-mod-php$PHP_VERS php$PHP_VERS-mysql php$PHP_VERS-gd && \
	apt-get -y install libcrypt-mysql-perl libyaml-perl libjson-perl libavutil-dev ffmpeg && \
	apt-get -y install --no-install-recommends libvlc-dev libvlccore-dev vlc && \
	apt-get -y install zoneminder
	
FROM build1 as build2
RUN	rm /etc/mysql/my.cnf && \
	cp /etc/mysql/mariadb.conf.d/50-server.cnf /etc/mysql/my.cnf && \
	adduser www-data video && \
	a2enmod php$PHP_VERS proxy_fcgi ssl rewrite expires headers && \
	a2enconf php$PHP_VERS-fpm zoneminder && \
	echo "extension=apcu.so" > /etc/php/$PHP_VERS/mods-available/apcu.ini && \
	echo "extension=mcrypt.so" > /etc/php/$PHP_VERS/mods-available/mcrypt.ini && \
	perl -MCPAN -e "force install Net::WebSocket::Server" && \
	perl -MCPAN -e "force install LWP::Protocol::https" && \
	perl -MCPAN -e "force install Config::IniFiles" && \
	perl -MCPAN -e "force install Net::MQTT::Simple" && \
	perl -MCPAN -e "force install Net::MQTT::Simple::Auth"

FROM build2 as build3
RUN	cd /root && \
	chown -R www-data:www-data /usr/share/zoneminder/ && \
	echo "ServerName localhost" >> /etc/apache2/apache2.conf && \
	sed -i "s|^;date.timezone =.*|date.timezone = ${TZ}|" /etc/php/$PHP_VERS/apache2/php.ini && \
	service mysql start && \
	mysql -uroot < /usr/share/zoneminder/db/zm_create.sql && \
	mysql -uroot -e "grant all on zm.* to 'zmuser'@localhost identified by 'zmpass';" && \
	mysqladmin -uroot reload && \
	mysql -sfu root < "mysql_secure_installation.sql" && \
	rm mysql_secure_installation.sql && \
	mysql -sfu root < "mysql_defaults.sql" && \
	rm mysql_defaults.sql

FROM build3 as build4
RUN	mv /root/zoneminder /etc/init.d/zoneminder && \
	chmod +x /etc/init.d/zoneminder && \
	service mysql restart && \
	sleep 5 && \
	service apache2 restart && \
	service zoneminder start

FROM build4 as build5
RUN	systemd-tmpfiles --create zoneminder.conf && \
	mv /root/default-ssl.conf /etc/apache2/sites-enabled/default-ssl.conf && \
	mkdir /etc/apache2/ssl/ && \
	mkdir -p /var/lib/zmeventnotification/images && \
	chown -R www-data:www-data /var/lib/zmeventnotification/ && \
	chmod -R +x /etc/my_init.d/ && \
	cp -p /etc/zm/zm.conf /root/zm.conf && \
	mkdir -p /etc/cron.weekly && \
	touch /etc/cron.weekly/zmaudit && \
	echo "#!/bin/sh\n\n/usr/bin/zmaudit.pl -f" >> /etc/cron.weekly/zmaudit && \
	chmod +x /etc/cron.weekly/zmaudit && \
	cp /etc/apache2/ports.conf /etc/apache2/ports.conf.default && \
	cp /etc/apache2/sites-enabled/default-ssl.conf /etc/apache2/sites-enabled/default-ssl.conf.default

FROM build5 as build6
RUN	rm -rf /tmp/* /var/tmp/* && \
	chmod +x /etc/my_init.d/*.sh

FROM build6 as build7
VOLUME \
	["/config"] \
	["/var/cache/zoneminder"]

FROM build7 as build8
EXPOSE 80 443 9000

FROM build8
CMD ["/sbin/my_init"]
  1. Modify the docker-compose.yaml file to map your volumes and set necessary flags. Here's a sample docker-compose.yaml where I chose to install ML Hooks, face recognition and YoloV4 models:
version: '3.1'
services:
    zoneminder:
        container_name: zoneminder
        image: dlandon/zoneminder:latest
        restart: unless-stopped
        ports:
            - 8443:443/tcp
            - 9000:9000/tcp
        runtime: nvidia
        network_mode: "bridge"
        privileged: true
        environment:
            - TZ=America/New_York
            - SHMEM=50%
            - PUID=99
            - PGID=100
            - INSTALL_HOOK=1
            - INSTALL_FACE=1
            - INSTALL_TINY_YOLOV3=0
            - INSTALL_YOLOV3=0
            - INSTALL_TINY_YOLOV4=1
            - INSTALL_YOLOV4=1
            - MULTI_PORT_START=0
            - MULTI_PORT_END=0
        volumes:
            - config:/config:rw
            - data:/var/cache/zoneminder:rw
            - cuda:/usr/local/cuda:/usr/local/cuda
volumes:
    config: /mnt/Zoneminder
    data: /mnt/Zoneminder/data
    cuda: /opt/cuda
  1. Modify docker-run.sh to passthrough GPU(s) to the Docker container. Here's a sample docker-run.sh where I passed through the Quadro M4000 GPU (which was device 0 in nvidia-smi output):
#!/bin/bash

docker run -d --name="Zoneminder" --gpus '"device=0"' \
--net="bridge" \
--privileged="true" \
-p 8443:443/tcp \
-p 9000:9000/tcp \
-e TZ="America/New_York" \
-e SHMEM="50%" \
-e PUID="99" \
-e PGID="100" \
-e INSTALL_HOOK="1" \
-e INSTALL_FACE="1" \
-e INSTALL_TINY_YOLOV3="0" \
-e INSTALL_YOLOV3="0" \
-e INSTALL_TINY_YOLOV4="1" \
-e INSTALL_YOLOV4="1" \
-e MULTI_PORT_START="0" \
-e MULTI_PORT_END="0" \
-v "/mnt/Zoneminder":"/config":rw \
-v "/mnt/Zoneminder/data":"/var/cache/zoneminder":rw \
zoneminder
  1. Build the Docker container: docker build -t zoneminder -f Dockerfile .

  2. Run the docker-run.sh script: ./docker-run.sh If there's a "permission denied" error, add execute permission to the script: chmod +x docker-run.sh ; ./docker-run.sh

  3. Look at the logs of the container to see if everything installs fine: docker logs -f Zoneminder The setup will compile OpenCV without CUDA/CuDNN support, I did not find a way to make it compile with CUDA straight away.

  4. When you see "Zoneminder started successfully" or something like that, wait a bit more to see if it will crash. If all is good, stop watching logs and enter into the Bash shell of the Docker container: docker exec -it Zoneminder /bin/bash

  5. Go into the /config/opencv folder and put this script in there (this will compile OpenCV with CUDA support). You will need to change the CUDA_ARCH_BIN parameter to the CUDA compute capability of the GPU you passed through (see link in step 4). For example, Quadro M4000 has CUDA Compute Capability 5.2.

#!/bin/bash

OPENCV_VER=4.5.0
OPENCV_URL=https://github.com/opencv/opencv/archive/$OPENCV_VER.zip
OPENCV_CONTRIB_URL=https://github.com/opencv/opencv_contrib/archive/$OPENCV_VER.zip

apt-get -y install libjpeg-dev libpng-dev libtiff-dev libavcodec-dev libavformat-dev libswscale-dev
apt-get -y install libv4l-dev libxvidcore-dev libx264-dev libgtk-3-dev libatlas-base-dev gfortran

cd /config/opencv
mkdir -p build

wget -q -O opencv.zip $OPENCV_URL
wget -q -O opencv_contrib.zip $OPENCV_CONTRIB_URL
unzip opencv.zip
unzip opencv_contrib.zip
mv $(ls -d opencv-*) opencv
mv opencv_contrib-$OPENCV_VER opencv_contrib
rm *.zip

cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
	-D CMAKE_INSTALL_PREFIX=/usr/local \
	-D INSTALL_PYTHON_EXAMPLES=OFF \
	-D INSTALL_C_EXAMPLES=OFF \
	-D OPENCV_ENABLE_NONFREE=ON \
	-D WITH_CUDA=ON \
	-D WITH_CUDNN=ON \
	-D OPENCV_DNN_CUDA=ON \
	-D CUDA_ARCH_BIN=5.2 \
	-D ENABLE_FAST_MATH=1 \
	-D CUDA_FAST_MATH=1 \
	-D WITH_CUBLAS=1 \
	-D OPENCV_EXTRA_MODULES_PATH=/config/opencv/opencv_contrib/modules/ \
	-D HAVE_opencv_python3=ON \
	-D PYTHON_EXECUTABLE=/usr/bin/python3 \
	-D BUILD_EXAMPLES=OFF ../opencv

make -j$(nproc)
make install
ldconfig
pip3 install face-recognition

cd ..
#rm -r opencv*

echo "Compile is complete."
echo "Now check that the cv2 module in python is working."
echo "Execute the following commands:"
echo "  python3"
echo "  import cv2"
echo "  Ctrl-D to exit"
echo
echo "Verify that the import does not show errors."
echo "If you don't see any errors, then you have successfully compiled opencv."
echo
echo "Once you are satisfied that the compile is working, run the following"
echo "command:"
echo "  echo "yes" > opencv_ok"
echo
echo "The opencv.sh script will run when the Docker is updated so you won't"
echo "have to do it manually."
  1. To use your GPU for event analysis, change /config/hook/objectdetect.ini to enable the proper Yolo algorithm. I installed YoloV4, so I have to find these lines and uncomment/change them:
...some lines

# FOR YoloV4. 
object_framework=opencv
object_processor=gpu
object_config={{base_data_path}}/models/yolov4/yolov4.cfg
object_weights={{base_data_path}}/models/yolov4/yolov4.weights
object_labels={{base_data_path}}/models/yolov4/coco.names

...more lines
  1. Don't forget to also change /config/secrets.ini and /config/zmeventnotification.ini according to your Zoneminder and external services configuration.

  2. Now you just have to configure Zoneminder as you wish as per https://zoneminder.readthedocs.io/en/stable/ and https://zmeventnotification.readthedocs.io/en/latest/index.html.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment