aidin-foroughi

Oct 24 2023

VM Image: Ubuntu 22.04 Make sure secure boot is not enabled.

sudo apt-get install build-essential to install gcc. CUDA toolkit needs it.

At the moment, driver version 535 seems to work fine. The cuda toolkit that is bundled with that version of of the driver is 12.2.2.

https://developer.nvidia.com/cuda-12-2-2-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=runfile_local

aidin-foroughi

opencv API cheatsheet

#include <opencv2/opencv.hpp> #include "opencv2/core.hpp" #include "opencv2/imgproc.hpp"

using namespace cv; using namespace std;

Mat image = imread(Guitar.jpg");

aidin-foroughi

Some tips

strings

iterate const auto& c .push_back .pop_back

aidin-foroughi

After installing docker on linux

You need to install the nvidia drivers. I'll just use the runfile scripts https://developer.nvidia.com/cuda-12-0-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=runfile_local

And just try different versions until it works.

Then install nvidia-container-toolkit, and configure docker and restart the service.

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

aidin-foroughi

Some useful fiftyone tips

Visualize dataset using fifty one

import fiftyone
import fire
import pathlib

aidin-foroughi

Put airflow learnings here

Essentially, you can use it to distribute work on a celery cluster very easily.

You typically want to use of the standard operators BashOperator PythonOperator or the best I think: DockerOperator

aidin-foroughi

Some tips on docker in docker

mount /var/run.docker.sock:/var/run/docker.sock

in windows ///var/run.docker.sock:/var/run/docker.sock

learn about privileged mode

learn about the Docker Socket Proxy

aidin-foroughi

Some docker compose tips and tricks

For environment variables, use poxis parameter expansion to set defaults:

services: myapp: environment: SOME_VALUE: ${SOME_VALUE:-default-value}

You can define a .env file next to docker-compose to provide the variables

aidin-foroughi

Here is what nginx can be used for

Gateway / reverse proxy / load balancer: Instead of exposing a bunch of different backend services separately, have nginx route requests to the appropriate backend.

If you have multiple instances of the same service, you can automatically balance the load using simple round robin or least connection based strategies.

In terms of authentication, basic user/pass based authentication is possible. nginx plus, which is not free, supports more advanced authentication methods, but it's still possible to use nginx too, with a bit more extra work.

The configuration file is /etc/nginx/conf.d/default.conf, which uses a domain specific language looking like:

aidin-foroughi

Some bookmarks

human pose estimation

https://istvansarandi.com/dozens/

Describes how many different pose datasets with their different skeletons can be used to train one single model. Essentially comes up with a common skeleton, using autoencoders.

---