Diego Nieto lesandie

ClickHouse Keeper Tuning for High Ingestion

A practical guide to understanding snapshot_distance, reserved_log_items, rotate_log_storage_interval, and the related settings that actually matter.

1. The mental model

ClickHouse Keeper under the hood uses NuRaft. Every write goes through this pipeline:

How to write a ClickHouse WebAssembly UDF

A short howto for creating, compiling, loading, and using WebAssembly user-defined functions in ClickHouse. Grounded in the upstream documentation (docs/en/sql-reference/functions/wasm_udf.md) and the repo's own test fixtures (tests/queries/0_stateless/wasm/).

WASM UDFs are experimental and not available in ClickHouse Cloud.

ADMIN queries

-------------------
-- Check max conn
-------------------
SELECT  * FROM
(SELECT COUNT(*) used FROM pg_stat_activity) q1,
(SELECT setting::int res_for_super FROM pg_settings WHERE name=$$superuser_reserved_connections$$) q2,
(SELECT setting::int max_conn FROM pg_settings WHERE name=$$max_connections$$) q3;

Keybase proof

I hereby claim:

I am lesandie on github.
I am dnieto (https://keybase.io/dnieto) on keybase.
I have a public key ASAXr56B-V6EOKUoZoqOhWIXfF2GNhdYHELNNEd58QF8IAo

To claim this, I am signing this object:

	<!-- Upgrade from 21/22 to 23.8 and after to 24 -->
	<clickhouse>
	<index_mark_cache_size>5368709120</index_mark_cache_size>
	<merge_tree>
	<ratio_of_defaults_for_sparse_serialization>1</ratio_of_defaults_for_sparse_serialization>
	<compress_marks>0</compress_marks>
	<compress_primary_key>0</compress_primary_key>
	</merge_tree>
	</clickhouse>

	# Simple line buffering example
	# When using the context maganet with,
	# simply use a 1 to open the file in buffering mode
	filename = "input_file.txt"
	with open(filename, 'r', 1) as filehandle:
	filecontent = filehandle.buffer
	for line in filecontent:
	line = line.decode()
	line = line.strip()
	print(line)

	from csv import Reader, Writer

	txt_file = "infile.txt"
	csv_file = "output.txt"
	delimiter = "\|"

	with open(txt_file, 'r') as txt_fh, \
	open(csv_file, 'w'i, newline="", encoding="utf-8") as csv_fh:
	writer(csv_fh).writerows(reader(txt_fh, delimiter=delimiter))