Skip to content

hypnguyen1209/offensive-claude / skills/vulnerability-analysis

vulnerability-analysis

Unverified

Use when auditing source code for vulnerabilities — drive CodeQL/Semgrep/Joern to taint untrusted data source-to-sink across injection, memory safety, deserialization/prototype-pollution, secrets/crypto/authz/race, and supply-chain risks

references/taint-engines-static-analysis.md

View on GitHub
# Taint Analysis & Static-Analysis Engines

Cluster: the methodology and tooling backbone — taint theory (source/sink/
sanitizer), and the three production engines you drive over real code: **Semgrep**
(AST + lightweight dataflow), **CodeQL** (relational, deep interprocedural), and
**Joern** (Code Property Graph). CWE-class is per-finding; this file is the engine
that *finds* the class. Backing tools: `scripts/taint_trace.py`,
`scripts/sast_runner.py`, `scripts/joern_taint.sc`.

## Theory / Mechanism

A taint vulnerability exists when **untrusted data (source)** reaches a
**dangerous operation (sink)** along a feasible path without an **adequate
sanitizer**. Three flow models:

- **Source-forward** — start at entry points, trace every path to sinks.
  Complete but slow.
- **Sink-backward** — start at `eval`/`exec`/SQL/`innerHTML`, trace backward to a
  source. Fast, targeted; the default for time-boxed reviews.
- **Hybrid** — sink-backward to rank high-risk sinks, then source-forward on the
  survivors for coverage.

**Taint vs. data flow.** Normal data flow is value-preserving: in `y = x`, `y`
carries `x`. *Taint tracking* adds non-value-preserving steps: in `y = x + 1` or
`y = f"{x}"`, `y` is still influenced by `x`, so taint propagates. Every engine
below distinguishes these; you choose the taint library when the transformation
matters (it almost always does for injection).

**Propagation rules** (apply consistently):

| Operation | Taint result |
|-----------|--------------|
| concat / format / interpolation | propagates (`tainted + clean = tainted`) |
| array/index/field read of tainted container | propagates |
| `len()`, `hash()`, `==` result | clean (bounded/one-way) |
| `int(x)` / parse | bounded value but may still be attacker-chosen → treat as tainted unless range-checked |
| parameterized query bind | **sanitized for SQL** |
| context-correct encoding (`html.escape`, `encodeURIComponent`) | sanitized **for that context only** |
| blacklist `.replace("'", "")` | **NOT** sanitized (bypassable) |
| client-side JS validation / WAF | **NOT** sanitized (attacker-skippable) |

## Engine 1 — Semgrep (fast, PR-loop)

AST pattern matching + a lighter intraprocedural dataflow engine; rules are YAML
and writable in minutes. Taint mode uses four concepts: `pattern-sources`,
`pattern-sinks`, `pattern-sanitizers`, `pattern-propagators`.

```yaml
# command-injection-flask.yml — taint mode
rules:
  - id: flask-cmdi
    mode: taint
    message: User input from Flask request reaches subprocess with shell=True.
    languages: [python]
    severity: ERROR
    metadata: { cwe: "CWE-78", owasp: "A03:2021" }
    pattern-sources:
      - pattern: flask.request.$ANYTHING
      - pattern: request.args.get(...)
      - pattern: request.form[...]
    pattern-sanitizers:
      - pattern: shlex.quote(...)
      - pattern: int(...)
    pattern-sinks:
      - patterns:
          - pattern: subprocess.$FN(...)
          - pattern-inside: subprocess.$FN(..., shell=True, ...)
```

```bash
semgrep scan --config command-injection-flask.yml --metrics off ./src
semgrep scan --config p/owasp-top-ten --sarif --output sg.sarif ./src   # registry pack
semgrep scan --config auto --pro ./src        # Pro: interfile + cross-function dataflow
semgrep scan --config p/default --baseline-commit origin/main ./src     # diff-aware PR scan
```

**Limits to internalize when writing rules** (avoids false confidence): no path
sensitivity (infeasible branches still considered), no pointer/alias or
array-element tracking, limited field sensitivity, and **CE is intraprocedural
only** — cross-file/cross-function taint requires the Pro engine. Independent
testing put CE at ~44–48% and Pro at ~72–75% true-positive recall on test
suites. Strip false positives with `pattern-not` / `pattern-not-inside`.
The **Opengrep** fork (Jan 2025) is a drop-in CE alternative after Semgrep moved
features behind a license.

## Engine 2 — CodeQL (deep, nightly)

Compiles code into a relational database; queries are written in QL. The modern
**modular dataflow API** (CodeQL ≥ 2.13; legacy API removed Dec 2024) uses a
config module implementing `DataFlow::ConfigSig` and applies
`TaintTracking::Global<Cfg>` for non-value-preserving steps.

```ql
/**
 * @name User input to command execution
 * @kind path-problem
 * @problem.severity error
 * @security-severity 9.8
 * @id py/cmdi-custom
 * @tags security external/cwe/cwe-078
 */
import python
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.TaintTracking
import semmle.python.ApiGraphs
import MyFlow::PathGraph

module MyConfig implements DataFlow::ConfigSig {
  predicate isSource(DataFlow::Node s) {
    s = API::moduleImport("flask").getMember("request")
          .getMember(["args","form","values"]).getACall()
  }
  predicate isSink(DataFlow::Node k) {
    exists(API::CallNode c |
      c = API::moduleImport("os").getMember("system").getACall() and
      k = c.getArg(0))
  }
  predicate isBarrier(DataFlow::Node b) {
    b = API::builtin("int").getACall()        // sanitizer: cast to int
  }
}
module MyFlow = TaintTracking::Global<MyConfig>;

from MyFlow::PathNode src, MyFlow::PathNode snk
where MyFlow::flowPath(src, snk)
select snk.getNode(), src, snk, "Untrusted input reaches os.system()."
```

```bash
# Build DB (none-build for interpreted; autobuild/--command for compiled)
codeql database create db --language=python --source-root ./src --overwrite
codeql database create db --language=java   --command "mvn -B clean install -DskipTests"
# Analyze with a shipped suite or your custom .ql
codeql database analyze db codeql/python-queries:codeql-suites/python-security-extended.qls \
  --format=sarifv2.1.0 --output=cq.sarif --threads=0
codeql database analyze db ./my-cmdi.ql --format=sarifv2.1.0 --output=custom.sarif
```

As of Sept 2025 all CodeQL-supported languages support incremental PR analysis
(5–40% faster). Both engines above are driven uniformly via
`scripts/sast_runner.py {codeql,semgrep}`.

## Engine 3 — Joern (Code Property Graph)

Joern merges AST + CFG + PDG into one queryable **CPG** and ships a `reachableBy`
dataflow engine, queried in a Scala DSL (CPGQL). It is the dominant tool in
academic vuln-detection research and excels at C/C++ where source isn't easily
buildable for CodeQL. See `scripts/joern_taint.sc` for a complete multi-CWE
taint script.

```scala
// In the Joern REPL after: joern-parse ./src --output cpg.bin ; joern
importCpg("cpg.bin")
// Backward slice: which user-controlled args reach system()?
val sinks   = cpg.call.name("system|popen|execve").argument
val sources = cpg.method.parameter.name("argv|.*request.*")
sinks.reachableBy(sources).l           // returns flow nodes
// Show full path with intermediate steps:
cpg.call.name("system").argument.reachableByFlows(sources).p
```

```bash
joern-parse ./src --output cpg.bin
joern --script joern_taint.sc --param cpgPath=cpg.bin --param outPath=joern.json
```

## Triage Workflow (the four-step gate)

For every candidate flow, before it becomes a finding:

1. **Reachability** — behind auth? which role? dead/test code?
2. **Controllability** — full raw input (high) vs. format/length-bounded (med)
   vs. second-order via DB (lower but valid).
3. **Impact** — RCE/data-breach (Critical) → privesc (High) → info-disc (Med) →
   DoS (Med).
4. **Sanitizer reality** — does the barrier actually neutralize *for this sink's
   context*? An HTML-encoder upstream of a SQL sink does nothing.

Run all three engines and merge so you triage one ranked list. Findings
corroborated by ≥2 engines jump to the top:

```bash
python3 scripts/sast_runner.py semgrep --src ./src --config p/owasp-top-ten --pro --out sg.sarif
python3 scripts/sast_runner.py codeql  --src ./src --lang python \
    --suite codeql/python-queries:codeql-suites/python-security-extended.qls --out cq.sarif
python3 scripts/sast_runner.py joern   --src ./src --script scripts/joern_taint.sc --out joern.json
python3 scripts/sast_runner.py merge   sg.sarif cq.sarif joern.json --out merged.json --top 50
```

## Modern 2024-2026 Variants

- **CPG + LLM agents** (USENIX Sec 2025: *LLMxCPG*; *codebadger* MCP server) wrap
  Joern's CPG/taint engine behind high-level tools so an LLM can slice and
  trace without hand-writing CPGQL — verified discovery of a libtiff buffer
  overflow and a correct patch for the libxml2 integer overflow **CVE-2025-6021**
  on first attempt. The CPG remains the ground-truth; LLMs hallucinate CPGQL
  when generating multi-hop traversals directly, so tool-mediated access wins.
- **Semgrep Pro interfile dataflow** (2024→) closed much of the cross-function
  gap vs. CodeQL but still trails on the hardest queries — run both.
- **CodeQL incremental PR analysis** (Sept 2025) across all languages.
- **`mode: taint` with `pattern-propagators`** lets Semgrep model custom
  wrapper functions (e.g. an in-house `sanitize()` that *doesn't* sanitize for
  your sink) as taint-passing — critical for real codebases with helper layers.

## Detection

These are *offensive analysis* tools, but the same SARIF feeds blue-team gates.
Detect that an audit/scan is running, and detect the bugs it would miss at runtime.

```yaml
# Sigma: CI/CD SAST scan executed (telemetry for change-control, not malicious)
title: SAST Engine Execution in CI
logsource: { category: process_creation }
detection:
  sel:
    Image|endswith: ['\codeql.exe','\semgrep','\joern']
  condition: sel
level: informational
```
IOCs of a *scan*: `codeql database create`, `semgrep --sarif`, `joern-parse`
process trees; large transient `codeql-db/` / `cpg.bin` artifacts. For the bug
classes themselves, runtime detection lives in each per-class reference (e.g.
SQL keyword regex on the DB tier, `www-data` spawning `/bin/sh`).

## OPSEC

- Static analysis touches only the source tree + builds a local DB/CPG — no
  target interaction, no network (except OSV/registry lookups, which you can run
  offline). It is the quietest phase of an engagement.
- `codeql-db/`, `cpg.bin`, and SARIF can be large and contain full source
  snippets — store inside the engagement workspace, encrypt at rest, purge on
  teardown; never commit them to a shared repo.
- A CodeQL build with `--command` executes the target's build scripts. On a
  hostile/untrusted codebase, build inside a disposable container.
- `taint_trace.py` is regex/heuristic — use it to *rank*, never to clear a file;
  absence of a hit is not absence of a bug.

## References

- CodeQL — *About data flow analysis* & *New dataflow API*:
  https://codeql.github.com/docs/writing-codeql-queries/about-data-flow-analysis/
- Semgrep — *Dataflow analysis engine overview* (sources/sinks/sanitizers/
  propagators): https://semgrep.dev/docs/writing-rules/data-flow/data-flow-overview/
- Joern — *Code Property Graph* docs: https://docs.joern.io/code-property-graph/
- *LLMxCPG: Context-Aware Vulnerability Detection Through CPG-Guided LLMs*,
  USENIX Security 2025.
- Semgrep vs CodeQL recall comparison (CE vs Pro): https://konvu.com/compare/semgrep-vs-codeql
- libxml2 integer overflow CVE-2025-6021: https://www.cvedetails.com/cve/CVE-2025-6021/
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall