pnagori02

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

pnagori02

ThreadPool Growth: Some Important Details

The CLR ThreadPool has two types of threads - "Worker" and "I/O Completion Port" (aka IOCP) threads.

• Worker threads are used when for things like processing Task.Run(…) or ThreadPool.QueueUserWorkItem(…) methods. These threads are also used by various components in the CLR when work needs to happen on a background thread.
• IOCP threads are used when asynchronous IO happens (e.g. reading from the network).

The thread pool provides new worker threads or I/O completion threads on demand (without any throttling) until it reaches the "Minimum" setting for each type of thread. By default, the minimum number of threads is set to the number of processors on a system.

Once the number of existing (busy) threads hits the "minimum" number of threads, the ThreadPool will throttle the rate at which is injects new threads to one thread per 500 milliseconds. This means that if your system gets a burst of work needing an IOCP thread, it will proces

pnagori02

L1 cache reference                  0.5 ns
Branch mispredict                   5 ns
L2 cache reference                  7 ns                      14x L1 cache
Mutex lock/unlock                   25 ns                     
Main memory reference               100 ns                    4x mutex op, 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy        3,000 ns
Send 2K bytes over 1 Gbps network   20,000 ns
Read 1 MB sequentially from memory  250,000 ns

pnagori02

$ curl --help
Usage: curl [options...] <url>
     --abstract-unix-socket <path> Connect via abstract Unix domain socket
     --alt-svc <file name> Enable alt-svc with this cache file
     --anyauth       Pick any authentication method
 -a, --append        Append to target file when uploading
     --basic         Use HTTP Basic Authentication
     --cacert <file> CA certificate to verify peer against
     --capath <dir>  CA directory to verify peer against
 -E, --cert <certificate[:password]> Client certificate file and password

pnagori02

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?