Skip to content

Instantly share code, notes, and snippets.

View benthecoder's full-sized avatar
🚀
shipping

Benedict Neo 梁耀恩 benthecoder

🚀
shipping
241 followers · 84 following
View GitHub Profile
@benthecoder
benthecoder / nv-embed.py
Created February 28, 2025 23:58
# NV-Embed-v2 Medical Text Embedding and Visualization
# This script demonstrates how to use NVIDIA's NV-Embed-v2 model for medical text embeddings
# and visualize the results using t-SNE
import numpy as np
import torch
import torch.nn.functional as F
from torch.nn import DataParallel
from tqdm import tqdm
from transformers import AutoModel
@benthecoder
benthecoder / summarize_youtube
Created March 25, 2024 05:01
import argparse
import os
from subprocess import run
import yt_dlp
from openai import OpenAI
client = OpenAI(api_key="") # YOUR KEY
@benthecoder
benthecoder / gist:8212cac40e865368019c4016ec55fc02
Created March 18, 2024 09:39
This file has been truncated, but you can view the full file.
-0.03411770612001419 0.01588732749223709 -0.0009765207068994641 -0.024040192365646362 0.009449911303818226 -0.042031485587358475 0.02132655493915081 -0.013556229881942272 0.010250852443277836 0.01624595746397972 0.008493563160300255 -0.025008494034409523 0.008463677950203419 -0.030340133234858513 -0.06680089235305786 0.035432685166597366 0.021230921149253845 -0.0033980230800807476 0.0328744538128376 0.03344826400279999 0.027662359178066254 -0.019772490486502647 -0.03672375530004501 0.00968302134424448 0.023358793929219246 -0.0030782443936914206 -0.009168984368443489 0.05293384939432144 0.025845298543572426 -0.013855088502168655 -0.04757830128073692 -0.02179277502000332 -0.012061935849487782 -0.02916860766708851 -0.02842743694782257 0.014632120728492737 0.01191848423331976 0.008816330693662167 -0.010346487164497375 0.025630120187997818 0.013460595160722733 0.011523990891873837 -0.03892335295677185 0.040405694395303726 0.008374020457267761 0.05475090816617012 -0.004626332316547632 -0.03980797529220581 0.0033741
@benthecoder
benthecoder / metadata.tsv
Created March 18, 2024 09:21
title date link
compute embeddings for tweets in tweets.json 2024-03-18T08:39:57.724Z https://gist.github.com/gd3kr/c4c0687a5f7e91b1a84bcacea6500011
Eric Zhu "WEBSITE" 2024-03-18T08:39:11.512Z https://ericfzhu.com/?windows=blog%3Binspo%3Bworks
Little guide to building Large Language Models in 2024 - Google Slides 2024-03-18T07:19:41.286Z https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit
Everything I'll forget about prompting LLMs 2024-03-18T07:12:49.834Z https://olickel.com/everything-i-know-about-prompting-llms
Codex - see similar quotes w embeddings 2024-03-18T06:11:43.723Z https://codex.ericfzhu.com/?id=2310
How to Fine-Tune an LLM Part 1: Preparing a Dataset for Instruction Tuning | alpaca_ft – Weights & Biases 2024-03-18T04:07:24.922Z https://wandb.ai/capecape/alpaca_ft/reports/How-to-fine-tune-an-LLM-Part-1-Preparing-a-Dataset-for-Instruction-Tuning--Vmlldzo1NTcxNzE2
Pix2Struct 2024-03-18T03:20:36.578Z https://huggingface.co/docs/transformers/en/model_doc/pix2s
@benthecoder
benthecoder / metadata.tsv
Last active March 18, 2024 09:42
title date link
compute embeddings for tweets in tweets.json 2024-03-18T08:39:57.724Z https://gist.github.com/gd3kr/c4c0687a5f7e91b1a84bcacea6500011
Eric Zhu "WEBSITE" 2024-03-18T08:39:11.512Z https://ericfzhu.com/?windows=blog%3Binspo%3Bworks
Little guide to building Large Language Models in 2024 - Google Slides 2024-03-18T07:19:41.286Z https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit
Everything I'll forget about prompting LLMs 2024-03-18T07:12:49.834Z https://olickel.com/everything-i-know-about-prompting-llms
Codex - see similar quotes w embeddings 2024-03-18T06:11:43.723Z https://codex.ericfzhu.com/?id=2310
How to Fine-Tune an LLM Part 1: Preparing a Dataset for Instruction Tuning | alpaca_ft – Weights & Biases 2024-03-18T04:07:24.922Z https://wandb.ai/capecape/alpaca_ft/reports/How-to-fine-tune-an-LLM-Part-1-Preparing-a-Dataset-for-Instruction-Tuning--Vmlldzo1NTcxNzE2
Pix2Struct 2024-03-18T03:20:36.578Z https://huggingface.co/docs/transformers/en/model_doc/pix2s
@benthecoder
benthecoder / site_metadata.csv
Created October 12, 2022 19:21
source: https://tamudatathon.com/challenges/docs/baker_hughes1
We can make this file beautiful and searchable if this error is corrected: It looks like row 8 should actually have 9 columns, instead of 3 in line 7.
CUSTOMER_NAME,PLANT_NAME,LATITUDE,LONGITUDE,ELEVATION,FUEL_N2_MOL_PCT,FUEL_MW,FUEL_LHV,CO2_FUEL_RATIO
SPIFFY,SPIRITUAL-POLECAT,61.170355655416756,42.87476722769898,112.0,4.44506304774041,16.572225178958153,21514.22236545268,2.6218034758534565
NONCHALANT,NIFTY-ROOK,37.55451549722366,49.90821662683808,-29.0,1.0531449774702362,16.16609726073242,21526.470829320926,2.714869756572029
NONCHALANT,PREHISTORIC-PETREL,29.190865869410757,60.49170182347541,1552.426025390625,10.298848060446083,17.273122273669447,21494.438671718366,2.461189337194369
NONCHALANT,THERAPEUTIC-LIONFISH,13.253365033156228,76.41105642809447,867.591552734375,13.188813728960108,17.61914942302331,21485.25197916242,2.3818954537334576
SOFT,ABORIGINAL-PICULET,-68.63200204949257,66.15530078737422,1253.15283203125,7.581915824443374,16.947813109323903,21503.41743616074,2.5357355888817
SOFT,CARMINE-REINDEER,-37.9973686323425,178.14485465045027,518.8723754882812,14.604307504874159,17.7886321547886,21480.88277080532,2.3430576226374615
SOFT,IDEALISTIC-DODO,-31
@benthecoder
benthecoder / slang_translator.txt
Last active September 25, 2022 03:47
source: https://github.com/rishabhverma17/sms_slang_translator/blob/master/slang.txt
{
"AFAIK": "As Far As I Know",
"AFK": "Away From Keyboard",
"ASAP": "As Soon As Possible",
"ATK": "At The Keyboard",
"ATM": "At The Moment",
"A3": "Anytime, Anywhere, Anyplace",
"BAK": "Back At Keyboard",
"BBL": "Be Back Later",
"BBS": "Be Back Soon",
@benthecoder
benthecoder / bird-solution.csv
Created May 11, 2022 18:44
ID species
2 A
5 C
7 C
8 B
9 C
@benthecoder
benthecoder / train.csv
Created May 11, 2022 18:23
bill_depth bill_length wing_length location mass sex ID
14.3 48.2 210 loc_2 4600 0 284
14.4 48.4 203 loc_2 4625 0 101
18.4 NA 200 loc_3 3400 0 400
14.9821138211382 47.5048780487805 NA NA 4800 0 98
18.9821138211382 38.2593070487805 217.186991869919 loc_3 5200 0 103
@benthecoder
benthecoder / articles.csv
Created April 19, 2022 05:40
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 10 columns, instead of 1 in line 6.
LABEL-1,LABEL-2,LABEL-3,LABEL-4,LABEL-5,LABEL-6,LABEL-7,LABEL-8,LABEL-9,LABEL-10
Predicting Rain with Machine Learning,Will it rain tomorrow? Let’s build an AI model to answer that.,Published on Mar 27,on Mar 27,·,8 min read,·,In Towards Data Science,,
10 Tips to Improve Your Regression Modeling,"From the book ""Regression and Other Stories""",Published on Mar 3,on Mar 3,·,11 min read,·,In bitgrit Data Science Publication,,
Data Analysis in 10 Easy Steps,The key steps in the process of turning data into insights,Published on Feb 21,on Feb 21,·,7 min read,·,In bitgrit Data Science Publication,,
The Missing Library in your Machine Learning Workflow,A quick guide to using Optuna for hyperparameter optimization,Published on Jan 27,on Jan 27,·,7 min read,·,In bitgrit Data Science Publication,·,Unpublished changes
Top Machine Learning Frameworks used by Data Scientists,Here are the Top ML frameworks used by data scientists in 2021.,Published on Jan 4,on Jan 4,·,8 min read,·,In bitgrit Data Science Publication,,
Top