ubergarm

Kokoro-82M-v1.0 Performance Benchmark

Introduction

Kokoro is an open-weight TTS model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost-efficient. With Apache-licensed weights, Kokoro can be deployed anywhere from production environments to personal projects.

Source: https://huggingface.co/hexgrad/Kokoro-82M

Why did you do this?

Relax, I only have one Sunday to work on idea, literally my weekend project. So I tried Deepseek to see if it can help. Surprisingly, it works and it saves me another weekend...

What is your setup?

Just chat.deepseek.com (cost = free) with prompts adapted from this gist.

Does it work in one-shot or I have to prompt it multiple times?

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

	import React from "react";
	import { ResponsiveContainer, LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, Legend } from "recharts";

	const showPassRate1 = false;
	const data = [
	{ name: "TQ1_0", unsloth: undefined, aider: 51.6, pass_rate_1: 19.1 },
	{ name: "IQ1_M", unsloth: 79.8, aider: 56.9, pass_rate_1: 24.0 },
	{ name: "TQ1_0-thinking", unsloth: undefined, aider: 60.4, pass_rate_1: 26.2 },
	{ name: "IQ2_XXS", unsloth: 80.3, aider: undefined, pass_rate_1: undefined },
	{ name: "IQ2_M", unsloth: 80.78, aider: 61.3, pass_rate_1: 36.4 },

	Log uploaded on Thursday, December 10, 2020, 12:16:49 AM
	Loaded mods:
	Harmony(brrainz.harmony)[mv:1.0.4.0]: 0Harmony(2.0.2), HarmonyMod(1.0.4)
	Core(Ludeon.RimWorld): (no assemblies)
	SRTS Expanded(smashphil.neceros.srtsexpanded)[mv:1.4.6]: 0Harmony(av:2.0.2,fv:1.2.0.1), SRTS(1.0.0)
	Royalty(Ludeon.RimWorld.Royalty): (no assemblies)
	HugsLib(UnlimitedHugs.HugsLib)[ov:8.0.1]: 0Harmony(av:2.0.2,fv:1.2.0.1), HugsLib(av:1.0.0,fv:8.0.1)
	JecsTools (Unofficial)(jecrell.jecstools)[mv:1.1.2.2]: 0JecsTools(1.1.2.2), AbilityUser(1.1.2.2), AbilityUserAI(1.1.2.2), CompActivatableEffect(1.1.2.2), CompAnimated(1.1.2.2), CompBalloon(1.1.2.2), CompBigBox(1.1.2.2), CompDeflector(1.1.2.2), CompDelayedSpawner(1.1.2.2), CompExtraSounds(1.1.2.2), CompInstalledPart(1.1.2.2), CompLumbering(1.1.2.2), CompOverlays(1.1.2.2), CompOversizedWeapon(1.1.2.2), CompSlotLoadable(1.1.2.2), CompToggleDef(1.1.2.2), CompVehicle(1.1.2.1), PawnShields(1.1.2.2), ThinkNodes(1.1.2.2)
	FSharp.Core(latta.fsharp.core)[mv:4.8.2]: FSharp.Core(av:4.7.0,fv:4.700.2

	Log uploaded on Monday, November 16, 2020, 2:10:08 AM
	Loaded mods:
	Harmony(brrainz.harmony)[mv:1.0.4.0]: 0Harmony(2.0.2), HarmonyMod(1.0.4)
	Core(Ludeon.RimWorld): (no assemblies)
	Royalty(Ludeon.RimWorld.Royalty): (no assemblies)
	HugsLib(UnlimitedHugs.HugsLib)[ov:8.0.1]: 0Harmony(av:2.0.2,fv:1.2.0.1), HugsLib(av:1.0.0,fv:8.0.1)
	KanbanStockpile(ubergarm.kanbanstockpile): 0Harmony(av:2.0.2,fv:2.0.4), 0MultiplayerAPI(av:0.2.0,fv:0.1.0), KanbanStockpile(1.0.7623.32571)

	Active Harmony patches:
	DebugWindowsOpener.DevToolStarterOnGUI: TRANS: HugsLib.Patches.DevToolStarterOnGUI_Patch.ExtendButtonsWindow

	"""
	WARNING: dont use loguru, use structlog
	https://gist.github.com/nkhitrov/38adbb314f0d35371eba4ffb8f27078f


	Configure handlers and formats for application loggers.
	"""
	import logging
	import sys
	from pprint import pformat

	# If you don‘t want to build it youself, you can try `docker pull killercai/postgres`.
	FROM healthcheck/postgres:latest

	# China debian mirror
	RUN sed -i s@/deb.debian.org/@/mirrors.aliyun.com/@g /etc/apt/sources.list
	RUN apt-get clean && apt-get update
	RUN apt-get install -y wget git build-essential libpq-dev python-dev postgresql-server-dev-all
	# SCWS (Simple Chinese Word Segmentation library)
	RUN cd /tmp && wget -q -O - http://www.xunsearch.com/scws/down/scws-1.2.1.tar.bz2 \| tar xjf - && cd scws-1.2.1 && ./configure && make install
	# zhpaser (postgres plugin)

	===========
	; A072257: a(n) = ((6n-17)4^n - 1)/3.
	; -6,-15,-27,21,597,4437,25941,136533,677205,3233109,15029589,68506965,307582293,1364546901,5995058517,26127717717,113100805461,486762960213,2084490794325,8887718991189,37749899220309,159795689903445,674367131702613,2838206015165781,11915774014084437,49914895870022997,208666782734832981,870695927958295893,3626898899909039445,15084056351939581269,62642068416972019029,259791645704742851925,1076060070966390510933,4451814236455238456661,18397552756179659478357,75951394266153460520277,313250310030353132508501,1290780171984369691743573,5314236415389307413812565,21861408571364544242603349,89863485924687435319825749,369125350255666774676952405,1515187027250335232298407253,6215490613912013463556019541,25480932475290743991673640277,104399609979733736516492809557,427501960233217988265164232021,1749621922190004121857428903253,7156944013788545162616803513685,29261601355268295351215565657429,119581706621529640207855669040469,488468031287944396043396301804885,1994436944359

	In addition to a significant decrease in hepatic lipid accumulation in the IOE group, which inhibited energy intake by propionate enrichment, hepatic lipids were also significantly reduced in the mice in the IOP group, which was largely enriched with butyrate. Compared with the IOE group, IOP had a stronger regulatory effect on hepatic metabolism and triglyceride metabolism and higher levels of TCA cycle in the host. In addition, butyrate has the ability to promote browning of white adipose tissue (WAT) to brown adipose tissue (BAT).^[@ref39],[@ref40]^ WAT stores energy, whereas BAT uses energy for heating and consequently host energy expenditure increases.^[@ref41],[@ref42]^ However, adipose tissue weight does not change after WAT browning.^[@ref43]^ Therefore, the weight of adipose tissue of mice in the IOP group dominated by butyrate was greater than that of the mice in the IOE group dominated by propionate.

	In conclusion ([Figure [7](#fig7){ref-type="fig"}](#fig7){ref-type="fig"}C), the improvement of ob