zsh-bench Integration
Proper benchmarking of interactive zsh using romkatv/zsh-bench. Measures real user-visible latency, NOT time zsh -lic exit (which is meaningless).
Commands
hey zbench # Run + display with threshold indicators (auto-compares if baseline exists)
hey zbench-save # Run + save as baseline + append history
hey zbench-compare # Run + explicit diff against baseline
hey zbench-check # Exit non-zero if over threshold (for git bisect)
hey zbench-baseline # Show saved baseline (no run)
hey zbench-history # Show TSV historyAll commands accept extra zsh-bench args: hey zbench --iters 4 for quick runs.
Metrics & Thresholds
From romkatv's blind perception study — values at or below threshold are indistinguishable from zero:
| Metric | Threshold | What it means |
|---|---|---|
first_prompt_lag_ms | 50ms | Time to see prompt after opening terminal |
first_command_lag_ms | 150ms | Time until first command can execute |
command_lag_ms | 10ms | Delay between Enter and next prompt |
input_lag_ms | 20ms | Keystroke-to-screen latency |
Indicators: 🟢 ≤50% (headroom) · 🟡 ≤100% (imperceptible) · 🟠 ≤200% (noticeable) · 🔴 >200% (sluggish)
exit_time_ms is shown but not used for thresholds — it doesn't measure interactive performance.
Git Bisect Workflow
Find which commit made the shell slow:
git bisect start
git bisect bad HEAD
git bisect good <known-good-commit>
git bisect run hey zbench-checkzbench-check exits non-zero when any metric exceeds its threshold.
File Layout
benchmarks/zsh-bench/
├── <Host>.json # Current baseline per host
└── history/
└── <Host>.tsv # Append-only history (timestamp, git_rev, metrics)
packages/zsh-bench.nix # Nix package (romkatv/zsh-bench with internal/ helpers)
bin/hey.d/zbench.just # Hey recipes
bin/zbench-report # Python — parse, compare, format resultsBaselines are per-host (MacTraitor-Pro.json, Seqeratop.json) because hardware varies.
Typical Workflow
# 1. Establish baseline on a clean build
hey zbench-save
# 2. Make zsh config changes
vim config/zsh/.zshrc
hey rebuild
# 3. Check for regressions
hey zbench # Shows comparison vs baseline
# 4. If satisfied, update baseline
hey zbench-saveDebugging Slow Startup
Phase Timing Script
Don't guess — measure. Paste this into zsh -c '...' to time each phase of startup:
zsh -c '
zmodload zsh/datetime
export XDG_CONFIG_HOME="${XDG_CONFIG_HOME:-$HOME/.config}"
export XDG_CACHE_HOME="${XDG_CACHE_HOME:-$HOME/.cache}"
export ZDOTDIR="${ZDOTDIR:-$XDG_CONFIG_HOME/zsh}"
export ZSH_CACHE="${ZSH_CACHE:-$XDG_CACHE_HOME/zsh}"
function _source { [[ -f "$1" ]] && source "$1"; }
function _cache {
local cache_dir="$XDG_CACHE_HOME/zsh"; local cache_file="$cache_dir/$1.zsh"
if [[ ! -f "$cache_file" ]] || [[ "$commands[$1]" -nt "$cache_file" ]]; then
mkdir -p "$cache_dir"; "$@" > "$cache_file"; fi
source "$cache_file"
}
t0=$EPOCHREALTIME
source $ZDOTDIR/.zshenv 2>/dev/null; t1=$EPOCHREALTIME
source $ZDOTDIR/config.zsh; t2=$EPOCHREALTIME
# ... add phases matching your .zshrc ...
source $ZDOTDIR/completion.zsh 2>/dev/null; t3=$EPOCHREALTIME
_source $ZDOTDIR/extra.zshrc; t4=$EPOCHREALTIME
printf "zshenv: %4.0fms\n" $(( (t1-t0)*1000 ))
printf "config: %4.0fms\n" $(( (t2-t1)*1000 ))
printf "completion: %4.0fms\n" $(( (t3-t2)*1000 ))
printf "extra: %4.0fms\n" $(( (t4-t3)*1000 ))
printf "TOTAL: %4.0fms\n" $(( (t4-t0)*1000 ))
'Adapt phases to match the actual .zshrc. The gap between this total and zsh-bench is overhead from /etc/zshrc (nix-darwin generated) and deferred plugin loading.
To drill into extra.zshrc, time each source line individually — one slow alias file can dominate.
Known Culprits (ranked by typical impact)
| Culprit | Typical cost | Fix |
|---|---|---|
| Redundant compinit | 2000-3000ms | Ensure compinit runs exactly once. Check EOF of .zshrc, /etc/zshrc, and completion.zsh — easy to end up with 2+ calls. Use compinit -C -d "$cache" with 24h staleness check. |
| Nix store globs | 200-400ms | for f in /nix/store/*foo*/*.zsh is slow — thousands of dirs. Cache the resolved path to a file. |
| Shell startup file scanning | 100-500ms | Functions that grep/sed across many files at startup (e.g., fixing session files). Move to on-demand or a cron job. |
Uncached eval "$(tool init)" | 40-100ms each | brew shellenv, direnv hook zsh, fnm env, zoxide init zsh, entire completion zsh. Use _cache pattern to write output to file, re-eval only when binary changes. |
Double brew shellenv | 40-80ms | nix-homebrew adds eval "$(brew shellenv)" to /etc/zshrc. If you handle it in .zshenv, set enableZshIntegration = false in nix-homebrew config. |
| Plugin manager overhead | 10-40ms | Antidote's antidote load does staleness checks. If static file exists, source it directly and skip antidote init entirely. |
| Deferred plugins | 0ms startup | antidote kind:defer is free at startup but zsh-bench won't detect has_syntax_highlighting/has_autosuggestions. This is fine. |
The _cache Pattern
Central to fast startup. Already defined in .zshrc:
function _cache {
local cache_dir="$XDG_CACHE_HOME/zsh"
local cache_file="$cache_dir/$1.zsh"
if [[ ! -f "$cache_file" ]] || [[ "$commands[$1]" -nt "$cache_file" ]]; then
mkdir -p "$cache_dir"
"$@" > "$cache_file"
fi
source "$cache_file"
}
# Usage:
_cache zoxide init zsh # instead of eval "$(zoxide init zsh)"
_cache direnv hook zsh # instead of eval "$(direnv hook zsh)"
_cache entire completion zsh # instead of source <(entire completion zsh)Invalidates when the binary changes ($commands[$1] mtime check). Delete ~/.cache/zsh/*.zsh to force regeneration.
Replay Mode
Use zsh-bench --iters 1 --scratch-dir /tmp/zbench-debug then dbg/replay --scratch-dir /tmp/zbench-debug to watch what zsh-bench actually sees.
Key Design Decisions
- Uses zsh-bench's non-raw output (median of 16 iterations) for stable numbers.
--rawgives per-iteration arrays — useful for variance analysis but not default.- Baselines stored as JSON for easy programmatic comparison.
- History stored as TSV for easy
column -t,awk, or import into spreadsheets. - Regression detection: flags changes > 20% or > 5ms (whichever is larger).