Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active April 24, 2025 06:41
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@tuansoibk
tuansoibk / cryptography-file-formats.md
Last active May 8, 2025 07:21
Cryptography material conversion and verification commands
  1. Introduction
  2. Standards
  3. Common combinations
  4. Conversion
  5. Verification/Inspection
  6. Tips for recognising

Introduction

It happens that there are many standards for storing cryptography materials (key, certificate, ...) and it isn't always obvious to know which standard is used by just looking at file name extension or file content. There are bunch of questions on stackoverflow asking about how to convert from PEM to PKCS#8 or PKCS#12, while many tried to answer the questions, those answers may not help because the correct answer depends on the content inside the PEM file. That is, a PEM file can contain many different things, such as an X509 certificate, a PKCS#1 or PKCS#8 private key. The worst-case scenario is that someone just store a non-PEM content in "something.pem" file.

@hyqneuron
hyqneuron / pytorch_visualize.py
Created June 7, 2017 07:06
PyTorch graph visualization
import torch
import torch.nn as nn
from torch.nn import Parameter
from torch.autograd import Variable, Function
from collections import defaultdict
import graphviz
"""
This is a rather distorted implementation of graph visualization in PyTorch.
@karpathy
karpathy / min-char-rnn.py
Last active June 3, 2025 20:37
Minimal character-level language model with a Vanilla Recurrent Neural Network, in Python/numpy
"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np
# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)
@karpathy
karpathy / gist:7bae8033dcf5ca2630ba
Created May 5, 2015 07:31
Efficient LSTM cell in Torch
--[[
Efficient LSTM in Torch using nngraph library. This code was optimized
by Justin Johnson (@jcjohnson) based on the trick of batching up the
LSTM GEMMs, as also seen in my efficient Python LSTM gist.
--]]
function LSTM.fast_lstm(input_size, rnn_size)
local x = nn.Identity()()
local prev_c = nn.Identity()()
local prev_h = nn.Identity()()
@karpathy
karpathy / gist:587454dc0146a6ae21fc
Last active May 14, 2025 00:08
An efficient, batched LSTM.
"""
This is a batched LSTM forward and backward pass
"""
import numpy as np
import code
class LSTM:
@staticmethod
def init(input_size, hidden_size, fancy_forget_bias_init = 3):
@davfre
davfre / bamfilter_oneliners.md
Last active January 18, 2025 22:47
SAM and BAM filtering oneliners
@miguelgrinberg
miguelgrinberg / rest-server.py
Last active February 12, 2025 21:09
The code from my article on building RESTful web services with Python and the Flask microframework. See the article here: http://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask
#!flask/bin/python
from flask import Flask, jsonify, abort, request, make_response, url_for
from flask_httpauth import HTTPBasicAuth
app = Flask(__name__, static_url_path = "")
auth = HTTPBasicAuth()
@auth.get_password
def get_password(username):
if username == 'miguel':