Skip to content

Instantly share code, notes, and snippets.

@nbx3
nbx3 / llama-metal-multi-gpu.patch
Created January 6, 2025 14:02
Patch for llama.cpp multi gpu support
diff --git a/cmake/llama-config.cmake.in b/cmake/llama-config.cmake.in
index 5c55bc6b..69bc027f 100644
--- a/cmake/llama-config.cmake.in
+++ b/cmake/llama-config.cmake.in
@@ -46,6 +46,7 @@ set(GGML_VULKAN_PERF @GGML_VULKAN_PERF@)
set(GGML_VULKAN_VALIDATE @GGML_VULKAN_VALIDATE@)
set(GGML_VULKAN_RUN_TESTS @GGML_VULKAN_RUN_TESTS@)
+set(GGML_METAL_MULTIDEV @GGML_METAL_MULTIDEV@)
set(GGML_METAL_USE_BF16 @GGML_METAL_USE_BF16@)

Keybase proof

I hereby claim:

  • I am nbx3 on github.
  • I am nickjb (https://keybase.io/nickjb) on keybase.
  • I have a public key ASCG-YrHNSEIZqBwyFpY5gaS1tY8_3g7UoB2dyYBeDsrWAo

To claim this, I am signing this object:

#!/usr/bin/env bash
set -euo pipefail
OFFSET=${1:-1d}
SERVICE_FILTER="ccd-|sscs-|probate-|cmc-|div-|finrem-|fees-|bulk-"
TMP=$(mktemp)
az group list | jq -r '.[] | (.name)' | egrep ${SERVICE_FILTER} > ${TMP}