Jason Chalom TRex22

Ollama GPU Selector: Customize GPU Usage for Ollama

If you want to run Ollama on a specific GPU or multiple GPUs, this tutorial is for you. By default, Ollama utilizes all available GPUs, but sometimes you may want to dedicate a specific GPU or a subset of your GPUs for Ollama's use. The idea for this guide originated from the following issue: Run Ollama on dedicated GPU.

Steps:

Create a script let's call it ollama_gpu_selector.sh:
```
nano ollama_gpu_selector.sh
```

Dependencies

Install the following dependencies

brew install pipenv
brew install ffmpeg
pipenv install --python 3.10

Instructions

rvm pkg install openssl

Requirements installation successful.
Fetching openssl-1.0.1i.tar.gz to /Users/user/.rvm/archives
Extracting openssl to /Users/user/.rvm/src/openssl-1.0.1i.....
Configuring openssl in /Users/user/.rvm/src/openssl-1.0.1i....................
Compiling openssl in /Users/user/.rvm/src/openssl-1.0.1i......................

	import torch
	import subprocess
	import time
	import logging

	# Takes about 8GB
	ndim = 25_000
	logging.basicConfig(format='[%(asctime)s] %(filename)s [%(levelname).1s] %(message)s', level=logging.DEBUG)

	def get_gpu_usage():

	##########
	# Win10 Initial Setup Script
	# Author: Disassembler <[email protected]>
	# Version: 1.4, 2016-01-16
	#
	# Modifications and video from CraftComputing : https://www.youtube.com/watch?v=PdKMiFKGQuc&ab_channel=CraftComputing
	#
	##########

	# Ask for elevated permissions if required

	name: Build
	on: [push]
	jobs:
	build_latex:
	runs-on: ubuntu-latest
	steps:
	- name: Set up Git repository
	uses: actions/checkout@v1
	- name: Install TeX Live packages # (Note that you may don't need some of them)
	run: sudo apt install -y texlive-base texlive-xetex texlive-lang-chinese latexmk

	import streamlit as st
	from persist import persist, load_widget_state


	def main():
	if "page" not in st.session_state:
	# Initialize session state.
	st.session_state.update({
	# Default page.
	"page": "home",

	#!/bin/sh
	# https://dl.ubnt.com/unifi/5.11.46/unifi_sh_api
	#username=ubnt
	#password=ubnt
	#baseurl=https://unifi:8443
	#site=default
	#[ -f ./unifi_sh_env ] && . ./unifi_sh_env

	cookie=$(mktemp)

	import sys # import sys
	sys.path.insert(1, "./lib") # add the lib folder to sys so python can find the libraries

	import epd2in7b # import the display drivers
	from PIL import Image,ImageDraw,ImageFont # import the image libraries
	import time
	from gpiozero import Button # import the Button control from gpiozero

	btn1 = Button(5) # assign each button to a variable
	btn2 = Button(6) # by passing in the pin number

	# https://mail.python.org/pipermail/scipy-user/2011-May/029521.html

	import numpy as np

	def KLdivergence(x, y):
	"""Compute the Kullback-Leibler divergence between two multivariate samples.

	Parameters
	----------
	x : 2D array (n,d)