Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save nmoinvaz/978f248ea7d528c954e3d52fb8dc99c0 to your computer and use it in GitHub Desktop.

Select an option

Save nmoinvaz/978f248ea7d528c954e3d52fb8dc99c0 to your computer and use it in GitHub Desktop.
zlib-ng PR #2281 — apt-cache analysis: skip cache on default-package Ubuntu jobs

zlib-ng PR #2281 — apt-cache analysis and recommendation

PR: zlib-ng/zlib-ng#2281[CI] Cache Ubuntu .deb packages to speed up installing dependencies.

TL;DR

The cache works: it cuts total apt-install time across Ubuntu jobs roughly in half. But cmake.yml enables the cache on every Ubuntu job, including the 23 jobs that only install the default libgtest-dev libbenchmark-dev package set. For those, cache-action overhead exceeds the savings. configure.yml already gates on matrix.packages; mirroring that gate in cmake.yml is a one-line fix.

Methodology

Compared two Orchestrator runs on ubuntu-latest / ubuntu-24.04-arm:

  • develop: run 25509964615 (HEAD 5e4c65ab)
  • PR cache-apt-packages: run 25513495567 (HEAD 620ca91f)

For each Ubuntu job, summed only the dependency-install steps:

  • develop: Install packages (Ubuntu)
  • PR: Get cache key (Ubuntu) + Load package cache (Ubuntu) + Install packages (Ubuntu) + Post Load package cache (Ubuntu)

Whole-job durations are noisy because of QEMU/cross-compile variance, so the apt-step total is the load-bearing signal.

Results

Apt-step time across 140 Ubuntu jobs (the cache effect)

develop PR delta
Total 6,377s 3,427s −46.3% (~49 min)
Mean per job 45.5s 24.5s −46%

Biggest per-job wins (jobs where develop hit slow apt mirrors):

Job develop PR
Ubuntu GCC RISC-V 707s 19s
Ubuntu Clang PPC64LE Power9 572s 21s
Configure / Ubuntu GCC PPC 420s 19s
Configure / Ubuntu GCC PPC64 384s 19s
Ubuntu GCC MIPS 340s 22s
Ubuntu GCC 32-bit 160s 20s

The cache also flattens variance: develop's apt step ranges from ~10s to ~700s; the PR's stays in a tight 15–50s band on cache hits.

Whole-job wall-clock (noisier, includes test execution + emulation)

develop PR delta
All jobs (152) 35,011s 31,047s −11.3%
Ubuntu only (140) 19,017s 16,141s −15.1%

A lot of that delta is runner variance, not the cache. The apt-step number is the trustworthy one.

Where it loses

About 30 small Ubuntu jobs got slightly slower on the apt step. Examples:

Job develop PR
Ubuntu GCC ASAN 15s 30s
Ubuntu GCC SSE4.2 UBSAN 22s 45s
Ubuntu GCC AARCH64 Compat UBSAN 23s 44s
Ubuntu Clang Inflate Strict REDUCED_MEM 10s 18s
Ubuntu Clang Compat Debug 10s 15s

These are all jobs that install only libgtest-dev libbenchmark-dev (a few tiny .debs). The cache-action restore/save round-trip costs ~5–15s, which exceeds the 1–2s of download those packages would have done from a warm mirror.

Recommendation

Gate the cache on matrix.packages being set, matching the pattern already used in configure.yml.

Bucket Count Action
Has explicit packages: (cross-toolchains, large package sets) 33 Keep cache — this is where the wins are
Default-only libgtest-dev libbenchmark-dev 23 Skip cache — overhead exceeds savings

Proposed change to .github/workflows/cmake.yml

    - name: Get cache key (Ubuntu)
-     if: runner.os == 'Linux' && !contains(matrix.os, 'z15')
+     if: runner.os == 'Linux' && !contains(matrix.os, 'z15') && matrix.packages

    - name: Load package cache (Ubuntu)
-     if: runner.os == 'Linux' && !contains(matrix.os, 'z15')
+     if: runner.os == 'Linux' && !contains(matrix.os, 'z15') && matrix.packages

The Install packages (Ubuntu) step's mv/cp blocks are already self-guarded with compgen -G, so they degrade cleanly to a plain apt-get install when nothing was restored. No other changes needed.

Expected savings: ~5–15s × 23 jobs ≈ 2–6 minutes less workflow wall-clock, and the gating logic stays consistent between the two workflow files.

Alternatives considered

  • Per-job cache: true|false matrix flag. Adds verbosity with no benefit over the implicit "no packages → no cache" rule already used in configure.yml.
  • Cache the apt index instead of debs. apt-get update is dominated by index decompression, not download; an index-only cache would not help.
  • Composite action shared by cmake.yml and configure.yml. Worthwhile cleanup eventually but separate from this PR's scope; better as a follow-up that consolidates the gating logic in one place.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment