Skip to content

Instantly share code, notes, and snippets.

@james-see

james-see/model-forget-me-not.py

Created November 24, 2023 19:48
Show Gist options
  • Save james-see/1b9e62d6f883ba7416901e0b1ac997b1 to your computer and use it in GitHub Desktop.
Save james-see/1b9e62d6f883ba7416901e0b1ac997b1 to your computer and use it in GitHub Desktop.
Download ZIP
download and convert and quantize a model from huggingface
Raw
model-forget-me-not.py
from huggingface_hub import snapshot_download
model_id="01-ai/Yi-34B"
snapshot_download(repo_id=model_id, local_dir="vicuna-hf",
local_dir_use_symlinks=False, revision="main")
# convert to gguf clone llama.cpp first git clone git clone https://github.com/ggerganov/llama.cpp.git
# python3 convert.py /Users/jc/projects/models/01ai/vicuna-hf --outfile /Users/jc/01ai.gguf --outtype f16
# ./quantize /Users/jc/01ai.gguf /Users/jc/01ai-q5.gguf Q5_K_M
@james-see
Copy link
Author

This is the latest and greatest open source model as of november 2023 from https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment