Daniel Waltrip dwaltrip

LLM Token Cost Over Time — Key Analyses

Source	Date	Headline Finding	Methodology	Key Caveats
Epoch AI ¹	Mar 2025	Price decline of 9×–900× per year depending on benchmark; median ~50× (or ~200× post-Jan 2024)	Tracked cheapest model to hit specific benchmark milestones across 6 benchmarks; 3:1 input/output weighted average; excluded reasoning models	Benchmarks don't capture all useful capabilities; possible benchmark overfitting; fastest trends are recent and may not persist
a16z ²	Nov 2024	~10× cost reduction per year at equivalent performance	MMLU scores + historical pricing from Internet Archive; limited to OpenAI, Anthropic, and Meta/Llama models	Only one benchmark (MMLU); limited provider coverage; early-stage industry so trendline may shift
MIRI TGT ³	Jan 2025	Massive price dispersion across providers (up to ~10× for the same model); model size partially predicts price	Cross-provider pricing fro

Me: What is 124 * 983? Perform the calculation by breaking it into easier sub-problems, such as (100 * 983) and (20 * 983). Proceed logically and carefully do each step.

GPT-4: Sure, let's break down the multiplication into smaller parts.

124 * 983 can be separated into (100 * 983) + (20 * 983) + (4 * 983)

Let's find the product of 100 * 983:

100 * 983 = 98,300

Movie Maven

On the web server

Node.js app that is the web app (front-end, API, database, etc)
Python script that takes a list of filename text strings and processes it into a list of standardized movie names
Ruby script that uses the cleaned up movie names to fetch associated data from Rotten Tomato, Imdb, etc

On the user's computer

	import sys
	import time
	import subprocess
	import select


	# Modified from: https://stackoverflow.com/a/12523371/111635
	def tail_file(filename, process_line):
	assert callable(process_line), '`process_line` should be a callable'

	import {
	DragDropContext as RealDragDropContext,
	Droppable as RealDroppable,
	Draggable
	} from 'react-beautiful-dnd';
	import React, { Component } from 'react';


	const DroppableManager = {
	_onDragEndLookup: {},

	<html lang="en">
	<head>
	<meta charset="utf-8">
	<title>Mixing keyed and non-keyed Vnodes</title>

	<!-- <script src="https://unpkg.com/mithril@0.2.5/mithril.js"></script> -->
	<script src="https://unpkg.com/mithril@1.1.1/mithril.js"></script>

	<style>
	body { margin: 10px; }

	<html lang="en">
	<head>
	<meta charset="utf-8">
	<title>Memory Leak test</title>
	<script src="mithril-v0.2.5.js"></script>
	<style>
	body { padding: 15px; margin: 0; }
	#widget-container {
	width: 340; height: 340; display: flex; flex-wrap: wrap; margin-top: 10px;
	}

	<html lang="en">
	<head>
	<meta charset="utf-8">
	<title>Memory Leak test</title>
	<script src="mithril-v1.1.1.js"></script>
	<style>
	body { padding: 15px; margin: 0; }
	#widget-container {
	width: 340; height: 340; display: flex; flex-wrap: wrap; margin-top: 10px;
	}

	/****
	Problem:

	You are making some edits, and in one of the computed properties you make
	the correct change to the function body, but forget to update the dependent keys.
	You test the change (the basic case only), and it works great. However, the function now
	has a bug that will need to be caught by QA or it will slip through to production.

	Solution:

	// helper functions

	// includes end points as possible return values
	var randint = function(min, max) {
	return (Math.floor(Math.random() * (max - min + 1)) + min);
	}

	// I was surprised to find out this is necessary to get a numerical sort
	function sort_number(a,b) {
	if (a < b)