Eric Czech eric-czech

Marin Delphi Scaling Ladder IsoFLOP Extraction

Extract IsoFLOP scaling ladder data (loss values, token counts, FLOPs, params) from the Marin Delphi suite without rerunning analysis jobs.

Create an application comparing Chinchilla Approach 2 token extrapolations against a known, analytical ground truth.

All details below reference Hoffmann et al. 2022, "Training Compute-Optimal Large Language Models" (https://arxiv.org/abs/2203.15556).

Implementation:

Define the Chinchilla loss surface: L(N, D) = E + A/N^α + B/D^β with parameters α=0.34, β=0.28, A=406.4, B=410.7, E=1.69 (Appendix D)

This script verifies that Triton can be built and minimally tested on ARM64 systems, specifically on AWS Graviton2 instances. Instructions:

Create a g5g.xlarge instance using the following AWS CLI command and make sure to add at least 30G storage:

aws ec2 run-instances \
    --instance-type g5g.xlarge \
--image-id ami-02dcfe5d1d39baa4e \

	"""Count rows in bolinas-dna HF datasets (training + validation)."""

	from datasets import load_dataset

	DATASETS = [
	("CDS train", "bolinas-dna/genomes-v5-genome_set-animals-intervals-v5_255_128", "train"),
	("Upstream train", "bolinas-dna/genomes-v5-genome_set-animals-intervals-v1_255_128", "train"),
	("Downstream train", "bolinas-dna/genomes-v5-genome_set-animals-intervals-v15_255_128", "train"),
	("CDS val", "bolinas-dna/genomes-v5-validation-intervals-v5_255_255", "validation"),
	("Upstream val", "bolinas-dna/genomes-v5-validation-intervals-v1_255_255", "validation"),

	import sys, os, tempfile
	from huggingface_hub.utils import http_stream_backoff


	url = "https://cdn.mos.cms.futurecdn.net/v2/t:0,l:160,cw:960,ch:720,q:80,w:960/FaWKMJQnr2PFcYCmEyfiTm.jpg"
	ext = os.path.splitext(url)[1]
	with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as f:
	with http_stream_backoff(
	method="GET",
	url=url,

	#!/usr/bin/env python3

	"""
	Metaflow Example: Simple bedtools sorting with Docker

	Demonstrates using biocontainers/bedtools via Docker SDK to sort genomic intervals.
	"""

	import tempfile
	import os

	(
	pd.read_gbq(
	"SELECT pmid, EXTRACT(YEAR from pub_date) AS pub_year FROM `pmdb-bq.pmdb.article` WHERE pub_date IS NOT NULL",
	use_bqstorage_api=True
	)
	.to_parquet("pub_years.parquet")
	)