notque/vexjoy-agent / skills/infrastructure/deploy
deploy
FlaggedDeploy and infrastructure: web deploy, cron, package install, shell configuration.
references/concurrency-and-locks.md
View on GitHub# Concurrency and Lock File Reference
> **Scope**: Preventing concurrent cron job execution — flock, PID files, lock file patterns
> **Version range**: Linux (flock via util-linux 2.20+), macOS (flock via brew), all bash versions
> **Generated**: 2026-04-16
---
## Overview
Cron jobs fire on a schedule with no built-in awareness of whether a previous run is still executing. If a job runs longer than its interval (a slow backup, a stalled network call), two instances run simultaneously — both writing the same output files, both modifying the same database rows, both sending duplicate notifications. Lock files are the standard mitigation. `flock` is preferred over manual PID files because the kernel releases the lock automatically on process death, even on crash.
---
## Pattern Table
| Mechanism | Reliability | Use When | Avoid When |
|-----------|-------------|----------|------------|
| `flock -n FD` | High — kernel-managed | bash scripts, Linux | macOS without util-linux |
| `flock -n LOCKFILE CMD` | High — single-shot | Wrapping an external command | Script needs lock for full duration |
| PID file + `kill -0` | Medium — race window | Must be sh-compatible | Can use flock |
| `mkdir` atomic lock | Low — no auto-cleanup | Absolute POSIX sh only | Any other case |
| `lockrun` wrapper | High | Wrapping a command externally | Not installed |
---
## Correct Patterns
### flock with file descriptor (preferred)
```bash
#!/bin/bash
set -euo pipefail
LOCK_FILE="/tmp/$(basename "$0").lock"
exec 200>"$LOCK_FILE"
# -n: non-blocking (fail immediately if locked)
flock -n 200 || { echo "$(date): Another instance is running, exiting." >&2; exit 0; }
trap 'rm -f "$LOCK_FILE"' EXIT
# ... rest of script ...
```
**Why**: File descriptor 200 holds the lock for the script's lifetime. The kernel automatically releases it when the process exits, even on crash or SIGKILL. `exit 0` (not `exit 1`) on conflict — cron sees success, not failure; this prevents alert fatigue from overlapping runs.
---
### flock wrapping a command (single-shot)
```bash
flock -n /tmp/my-job.lock -c '/usr/local/bin/my-job.py'
```
**Why**: Simpler form when the entire job is a single command. No cleanup needed — lock releases when the command exits.
---
### PID file fallback (sh-compatible)
```bash
#!/bin/sh
PID_FILE="/var/run/$(basename "$0").pid"
if [ -f "$PID_FILE" ]; then
OLD_PID=$(cat "$PID_FILE")
if kill -0 "$OLD_PID" 2>/dev/null; then
echo "Already running (PID $OLD_PID)" >&2
exit 0
fi
# Stale PID file — process is gone
rm -f "$PID_FILE"
fi
echo $$ > "$PID_FILE"
trap 'rm -f "$PID_FILE"' EXIT
```
**Why**: `kill -0 PID` checks if the process exists without sending a signal. Stale PID files (from crashes) are cleaned up. Race window: two instances can pass the `[ -f ]` check simultaneously on fast systems — use `flock` when available.
---
<!-- no-pair-required: section heading; individual failure modes below carry Do-instead blocks -->
## Pattern Catalog
### Add Lock Protection to Every Cron Script
**Detection**:
```bash
grep -rL "flock\|\.lock\|\.pid\|lockrun" --include="*.sh" scripts/ cron/ jobs/
rg -L 'flock|\.lock|\.pid' --type sh
```
**Signal**:
```bash
#!/bin/bash
set -euo pipefail
cd /var/data
python3 process_all_records.py # runs for 45 min, cron fires again at minute 0
```
**Why this matters**: If the job takes longer than its interval, two instances run simultaneously. Both read the same input, both write to the same output, both update the same DB rows — producing duplicates, data corruption, or deadlocks.
**Do instead:** Add `flock` at the top of the script (see correct pattern above).
---
### Use Atomic Lock Acquisition with flock
**Detection**:
```bash
grep -rn "if \[ -f.*lock\|test -f.*lock" --include="*.sh" scripts/ cron/
rg 'if \[ -f.*lock' --type sh
```
**Signal**:
```bash
LOCK="/tmp/myjob.lock"
if [ -f "$LOCK" ]; then
echo "Running, exit"
exit 0
fi
touch "$LOCK" # race window between [ -f ] check and touch
trap 'rm -f $LOCK' EXIT
```
**Why this matters**: There is a TOCTOU race: two instances can both pass `[ -f "$LOCK" ]` before either creates the file. On a loaded system or slow filesystem, this happens. Result: both instances run concurrently despite the "protection."
**Do instead:** Use `flock -n` — the kernel makes the lock acquisition atomic.
```bash
exec 200>"/tmp/myjob.lock"
flock -n 200 || exit 0
```
---
### Use flock Instead of Sleep-Retry Loops
**Detection**:
```bash
grep -rn "while.*lock\|sleep.*lock" --include="*.sh" scripts/ cron/
rg 'while.*lock' --type sh
```
**Signal**:
```bash
while [ -f /tmp/myjob.lock ]; do
sleep 5
done
touch /tmp/myjob.lock
# ... work ...
rm /tmp/myjob.lock
```
**Why this matters**: Busy-waits waste CPU, has the same TOCTOU race as file existence checks, and if the script crashes before `rm`, subsequent runs loop forever.
**Do instead:** Use `flock -w TIMEOUT` for waiting with a timeout, or `flock -n` to exit immediately.
```bash
# Wait up to 5 minutes for the lock, then give up
flock -w 300 /tmp/myjob.lock -c 'your_command'
```
---
### Use trap EXIT for Lock Cleanup
**Detection**:
```bash
# Files that mention lock but not trap
grep -rl "lock\|\.pid" --include="*.sh" scripts/ cron/ | xargs grep -L "trap"
rg -l 'lock|\.pid' --type sh | xargs rg -L 'trap'
```
**Signal**:
```bash
touch /tmp/myjob.lock
do_work
rm /tmp/myjob.lock # only runs on success; if set -e fires, lock is never removed
```
**Why this matters**: With `set -e`, any failure exits the script immediately — `rm` never runs. The lock file persists. All future runs think a job is still running and exit immediately, silently. The job never runs again until someone manually removes the lock.
**Do instead:** Always use `trap 'rm -f "$LOCK_FILE"' EXIT` instead of relying on explicit cleanup at the end.
---
## Error-Fix Mappings
| Symptom | Root Cause | Fix |
|---------|------------|-----|
| Job never runs after a crash | Stale lock file from crashed previous run | Use `flock` (auto-released on death) or add PID validation to PID file check |
| Duplicate records / double-processing | No concurrency protection, job ran twice | Add `flock -n` at script start |
| Lock file stays after job fails | Missing `trap EXIT` | Replace end-of-script `rm` with `trap 'rm -f "$LOCK"' EXIT` |
| `flock: command not found` | macOS default shell / minimal container | Install `util-linux` or use PID file fallback |
| Two instances both acquired "lock" | File existence check instead of flock | Replace `[ -f $LOCK ]` pattern with `flock -n FD` |
---
## Detection Commands Reference
```bash
# No lock mechanism at all
grep -rL "flock\|\.lock\|\.pid\|lockrun" --include="*.sh" scripts/ cron/ jobs/
# File existence lock (TOCTOU risk)
grep -rn "if \[ -f.*\.lock\|test -f.*\.lock" --include="*.sh" scripts/
# Lock without trap (stale lock risk)
grep -rl "lock\|\.pid" --include="*.sh" scripts/ | xargs grep -L "trap"
# Sleep-retry lock pattern
grep -rn "while.*lock\|sleep.*lock" --include="*.sh" scripts/
# flock usage (positive check — these are doing it right)
grep -rn "flock" --include="*.sh" scripts/ cron/
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |