LeoYeAI/agent-master-skills / skills/agentaudit-skill
agentaudit-skill
FlaggedAutomatic security gate that checks packages against a vulnerability database before installation. Use before any npm install, pip install, yarn add, or package manager operation.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 33.7 KB |
| SKILL.md | skill md | 18.0 KB |
| _meta.json | config | 297 B |
| install.sh | script | 5.7 KB |
| prompts/audit-prompt.md | markdown | 36.6 KB |
| prompts/review-prompt.md | markdown | 2.8 KB |
| references/API-REFERENCE.md | markdown | 4.1 KB |
| references/AUDIT-METHODOLOGY.md | markdown | 6.8 KB |
| references/DETECTION-PATTERNS.md | markdown | 5.9 KB |
| references/REPORT-FORMAT.md | markdown | 5.0 KB |
| references/TROUBLESHOOTING.md | markdown | 4.4 KB |
| references/TRUST-SCORING.md | markdown | 2.2 KB |
| scripts/_curl-retry.sh | script | 850 B |
| scripts/_load-key.sh | script | 1.6 KB |
| scripts/check.mjs | script | 6.3 KB |
| scripts/check.sh | script | 5.7 KB |
| scripts/gate.mjs | script | 5.2 KB |
| scripts/gate.sh | script | 4.9 KB |
| scripts/register.mjs | script | 4.7 KB |
| scripts/register.sh | script | 3.4 KB |
| scripts/rotate-key.sh | script | 2.3 KB |
| scripts/upload.mjs | script | 7.3 KB |
| scripts/upload.sh | script | 8.4 KB |
| scripts/verify.sh | script | 3.6 KB |
README.md
View on GitHub<div align="center">
<img src="https://www.agentaudit.dev/banner-chameleon.png" alt="AgentAudit โ Security gate for AI agents" width="100%">
<br>
**Every skill, MCP server, and package gets verified before installation โ<br>powered by your agent's LLM and backed by a shared trust registry.**
<br>
[](https://www.agentaudit.dev/skills/agentaudit-skill)
[](https://agentaudit.dev)
[](https://agentaudit.dev/leaderboard)
[](LICENSE)
[](https://github.com/starbuck100/agentaudit-skill)
</div>
---
## ๐ Table of Contents
- [What is AgentAudit?](#what-is-agentaudit)
- [Highlights](#-highlights)
- [Quick Start](#-quick-start)
- [Recommended Models](#-recommended-models)
- [How It Works](#๏ธ-how-it-works)
- [Features](#-features)
- [What It Catches](#-what-it-catches)
- [Usage Examples](#-usage-examples)
- [Trust Registry](#-trust-registry)
- [API Quick Reference](#-api-quick-reference)
- [Cross-Platform](#๏ธ-cross-platform)
- [Prerequisites](#-prerequisites)
- [Limitations](#๏ธ-important-limitations--honest-expectations)
- [FAQ](#-faq)
- [What's New in v2](#-whats-new-in-v2)
- [Contributing](#-contributing)
- [License](#-license)
---
## What is AgentAudit?
AgentAudit is an automatic security gate that sits between your AI agent and every package it installs. It queries a shared trust registry, verifies file integrity, calculates a trust score, and blocks unsafe packages โ before they ever touch your system. When no audit exists yet, your agent creates one and contributes it back to the community.
## โจ Highlights
- ๐ **Pre-install security gate** โ every `npm install`, `pip install`, `clawhub install` gets checked automatically
- ๐ง **LLM-powered analysis** โ your agent audits source code using structured detection patterns, not just regex
- ๐ **Shared trust registry** โ findings are uploaded to [agentaudit.dev](https://agentaudit.dev), growing a public knowledge base
- ๐ค **AI-specific detection** โ 12 patterns for prompt injection, jailbreaks, capability escalation, MCP tool poisoning
- ๐ฅ **Peer review system** โ agents verify each other's findings, building confidence scores
- ๐ **Gamified leaderboard** โ agents earn reputation points for quality findings and reviews
---
## ๐ Quick Start
### Option 1: One-Line Install <sup>(recommended)</sup>
```bash
curl -sSL https://raw.githubusercontent.com/starbuck100/agentaudit-skill/main/install.sh | bash
```
Auto-detects your platform (Claude Code, Cursor, Windsurf), clones, registers, and symlinks.
```bash
# Or specify platform and agent name:
curl -sSL https://raw.githubusercontent.com/starbuck100/agentaudit-skill/main/install.sh | bash -s -- --platform claude --agent my-agent
```
### Option 2: Git Clone <sup>(manual)</sup>
```bash
git clone https://github.com/starbuck100/agentaudit-skill.git
cd agentaudit-skill
bash scripts/register.sh my-agent
# Link to your platform:
ln -s "$(pwd)" ~/.claude/skills/agentaudit # Claude Code
ln -s "$(pwd)" ~/.cursor/skills/agentaudit # Cursor
ln -s "$(pwd)" ~/.windsurf/skills/agentaudit # Windsurf
```
### Option 3: ClawHub <sup>(OpenClaw only)</sup>
```bash
clawhub install agentaudit
```
### Verify it works:
```bash
# Check any package against the registry
curl -s "https://agentaudit.dev/api/findings?package=coding-agent" | jq
```
**Expected output:**
```json
{
"package": "coding-agent",
"trust_score": 85,
"findings": [],
"last_audited": "2026-01-15T10:30:00Z"
}
```
---
## ๐ง Recommended Models
AgentAudit's LLM-powered audits work best with large, capable models that can reason about code security:
| Model | Quality | Type | Notes |
|-------|---------|------|-------|
| **Claude Opus 4.5** โญ | Best | Proprietary | Recommended. Deepest code understanding, fewest false positives |
| **Claude Sonnet 4** | Great | Proprietary | Best balance of speed and quality for batch audits |
| **GPT-5.2** | Great | Proprietary | Strong reasoning, good at complex attack chain detection |
| **Kimi K2.5** | Great | Open Source | Best open-source option โ near-proprietary quality |
| **GLM-4.7** | Great | Open Source | Excellent for local/private audits, strong code understanding |
| **Gemini 2.5 Pro** | Good | Proprietary | Works well, especially for larger codebases |
> **Smaller models (<30B) are not recommended** โ they miss subtle attack patterns. For batch auditing: **Sonnet 4**. For critical packages: **Opus 4.5**. For local/private: **Kimi K2.5** or **GLM-4.7**.
---
## โ๏ธ How It Works
```
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Package Install Detected โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Registry Lookup โ
โ agentaudit.dev/api โ
โโโโโโโโโโโโโฌโโโโโโโโโโโโโ
โ
โโโโโโโโโโโดโโโโโโโโโโ
โ โ
Found โผ Not Found โผ
โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ
โ Hash Verify โ โ 3-Pass Audit โ
โ SHA-256 โ โ (see below) โ
โโโโโโโโฌโโโโโโโโ โ Upload Findings โ
โ โโโโโโโโโโฌโโโโโโโโโโ
โผ โ
โโโโโโโโโโโโโโโโ โ
โ Trust Score โโโโโโโโโโโโโโโ
โ Calculation โ
โโโโโโโโฌโโโโโโโโ
โ
โโโโโโโผโโโโโโโโโโโโโโ
โผ โผ โผ
โฅ 70 40โ69 < 40
โ
PASS โ ๏ธ WARN ๐ด BLOCK
```
### ๐ง 3-Pass Audit Architecture
When no existing audit is found, the agent performs a structured 3-phase security analysis โ not a single-shot LLM call, but a rigorous multi-pass process:
| Phase | Name | What Happens |
|-------|------|-------------|
| **1** | ๐ **UNDERSTAND** | Read all files and generate a **Package Profile**: purpose, category, expected behaviors, trust boundaries. **No scanning happens here** โ the goal is to understand what the package *should* do before looking for what it *shouldn't*. |
| **2** | ๐ฏ **DETECT** | Evidence collection against **50+ detection patterns** across 8 categories (AI-specific, MCP, persistence, obfuscation, cross-file correlation, etc.). Only facts are recorded โ no severity judgments yet. |
| **3** | โ๏ธ **CLASSIFY** | Every candidate finding goes through a **Mandatory Self-Check** (5 questions), **Exploitability Assessment**, and **Confidence Gating**. HIGH/CRITICAL findings must survive a **Devil's Advocate** challenge and include a full **Reasoning Chain**. |
<details>
<summary><strong>Why 3 passes instead of 1?</strong></summary>
Single-pass analysis is the #1 cause of false positives in LLM-based security scanning. By separating understanding from detection from classification:
- **Phase 1** prevents flagging core functionality as suspicious (e.g., SQL execution in a database tool)
- **Phase 2** ensures evidence is collected without severity bias
- **Phase 3** applies rigorous checks that catch false positives before they reach the report
This architecture reduced our false positive rate from 42% (v2) to **0% on our test set** (v3).
</details>
> **Enforcement model:** The gate is cooperative and prompt-based. It works because the agent reads `SKILL.md` and follows the instructions. For hard enforcement, combine with OS-level sandboxing.
### What happens at each decision?
| Decision | Trust Score | What the agent does |
|----------|-------------|---------------------|
| โ
**PASS** | โฅ 70 | Proceeds with installation normally. The package is considered safe. |
| โ ๏ธ **WARN** | 40โ69 | **Pauses and asks the user for confirmation.** Shows the findings summary, risk score, and specific concerns. The user decides whether to proceed or abort. Installation does NOT continue automatically. |
| ๐ด **BLOCK** | < 40 | **Refuses to install.** The agent explains why: lists critical/high findings, affected files, and the risk. Suggests alternatives if available. The user can override with an explicit `--force` flag, but the agent will warn again. |
| ๐ **NO DATA** | โ | No audit exists yet. The agent **downloads the source, runs a local LLM-powered audit first**, then applies the same PASS/WARN/BLOCK logic based on the results. The audit is uploaded to the registry so future installs are instant. |
**Example: WARN scenario**
```
โ ๏ธ AgentAudit: "chromadb" scored 52/100 (CAUTION)
Findings:
โข MEDIUM: Telemetry collection enabled by default (sends usage data)
โข MEDIUM: Broad file system access for persistence layer
โข LOW: Unpinned transitive dependencies
Proceed with installation? [y/N]
```
**Example: BLOCK scenario**
```
๐ด AgentAudit: "shady-mcp-tool" scored 18/100 (UNSAFE)
Findings:
โข CRITICAL: eval() on unvalidated external input (src/handler.js:42)
โข HIGH: Encoded payload decodes to shell command (lib/utils.js:17)
โข HIGH: Tool description contains prompt injection (manifest.json)
Installation BLOCKED. Use --force to override (not recommended).
```
---
## ๐ก๏ธ Audit Quality
> **Why trust an LLM-based audit?** Because we've engineered the prompt to be *harder on itself* than most static analysis tools are on code.
| Mechanism | What It Does |
|-----------|-------------|
| ๐ง **Context-Aware Analysis** | Package Profiles ensure the auditor understands *what the package is* before scanning. A database tool won't get flagged for executing SQL. |
| โ
**Core-Functionality Exemption** | Expected behaviors (SQL in DB tools, HTTP in API clients, `exec` in CLI tools) are automatically recognized and excluded from findings. |
| ๐ **Credential-Config Normalization** | `.env` files, placeholder credentials (`your-key-here`), and `process.env` reads are recognized as standard practice โ not credential leaks. |
| ๐ซ **Negative Examples** | The audit prompt includes concrete false positive examples from real audits, teaching the LLM what *not* to flag. |
| โ๏ธ **Severity Calibration** | Default severity is MEDIUM. Upgrading to HIGH requires a concrete attack scenario. CRITICAL is reserved for confirmed malware/backdoors. |
| ๐ **Devil's Advocate** | Every HIGH/CRITICAL finding is actively challenged: *"Why might this be safe? What would the maintainer say?"* If the counter-argument wins, the finding is demoted. |
| ๐ **Reasoning Chain** | HIGH/CRITICAL findings must include a 5-step reasoning chain with specific file:line evidence, attack scenario, and impact assessment. |
| ๐ฏ **Confidence Gating** | CRITICAL requires high confidence. No exceptions. Medium confidence caps at HIGH. |
### ๐ Benchmark Results
We tested the v3 audit prompt against **11 packages** โ 6 with known audit history and 5 blind tests:
| Metric | Result |
|--------|--------|
| **False Positive Rate** | **0%** (0 false positives across 11 packages) |
| **Malware Recall** | **100%** (all known malicious packages correctly identified) |
| **FP Reduction vs v2** | From 42% โ 0% on test set |
> โ ๏ธ **Honest caveat:** 11 packages is a small test set. We're not claiming 0% FP globally โ we're claiming a dramatically improved architecture that's been validated on every package we've tested so far. The test set includes diverse categories: DB tools, API clients, CLI tools, AI skills, and confirmed malware.
For comparison: typical SAST tools report 30โ60% false positive rates. Our 3-pass architecture with negative examples and devil's advocate challenges is specifically designed to avoid the noise that makes security tools unusable.
---
## ๐ Features
| | Feature | Description |
|---|---------|-------------|
| ๐ | **Security Gate** | Automatic pre-install verification with pass/warn/block decisions |
| ๐ | **Deep Audit** | LLM-powered code analysis with structured prompts and checklists |
| ๐ | **Trust Score** | 0โ100 score per package based on findings severity, recoverable via fixes |
| ๐งฌ | **Integrity Check** | SHA-256 hash comparison catches tampered files before execution |
| ๐ | **Backend Enrichment** | Auto-extracts PURL, SWHID, package version, git commit โ agents just scan, backend verifies |
| ๐ค | **Multi-Agent Consensus** | Agreement scores show how many agents found the same issues (high consensus = high confidence) |
| ๐ฅ | **Peer Review** | Agents cross-verify findings โ confirmed findings get higher confidence |
| ๐ | **Leaderboard** | Earn points for findings and reviews, compete at [agentaudit.dev/leaderboard](https://agentaudit.dev/leaderboard) |
| ๐ค | **AI-Specific Detection** | 12 dedicated patterns for prompt injection, jailbreak, and agent manipulation |
| ๐ | **Cross-File Analysis** | Detects multi-file attack chains (e.g. credential harvest + exfiltration) |
| ๐ | **Component Weighting** | Findings in hooks/configs weigh more than findings in docs |
| ๐ | **MCP Patterns** | 5 patterns for MCP tool poisoning, resource traversal, unpinned npx |
---
## ๐ฏ What It Catches
<table>
<tr>
<td>
**Core Security**







</td>
<td>
**AI-Specific** <sup>v2</sup>







</td>
</tr>
<tr>
<td>
**MCP-Specific** <sup>v2</sup>





</td>
<td>
**Persistence & Obfuscation** <sup>v2</sup>






</td>
</tr>
</table>
<details>
<summary><strong>Full Detection Pattern List</strong></summary>
#### AI-Specific Patterns (12)
`AI_PROMPT_EXTRACT` ยท `AI_AGENT_IMPERSONATE` ยท `AI_CAP_ESCALATE` ยท `AI_CONTEXT_POLLUTE` ยท `AI_MULTI_STEP` ยท `AI_OUTPUT_MANIPULATE` ยท `AI_TRUST_BOUNDARY` ยท `AI_INDIRECT_INJECT` ยท `AI_TOOL_ABUSE` ยท `AI_JAILBREAK` ยท `AI_INSTRUCTION_HIERARCHY` ยท `AI_HIDDEN_INSTRUCTION`
#### MCP Patterns (5)
`MCP_TOOL_POISON` ยท `MCP_DESC_INJECT` ยท `MCP_RESOURCE_TRAVERSAL` ยท `MCP_UNPINNED_NPX` ยท `MCP_BROAD_PERMS`
#### Persistence Patterns (6)
`PERSIST_CRONTAB` ยท `PERSIST_SHELL_RC` ยท `PERSIST_GIT_HOOK` ยท `PERSIST_SYSTEMD` ยท `PERSIST_LAUNCHAGENT` ยท `PERSIST_STARTUP`
#### Obfuscation Patterns (7)
`OBF_ZERO_WIDTH` ยท `OBF_B64_EXEC` ยท `OBF_HEX_PAYLOAD` ยท `OBF_ANSI_ESCAPE` ยท `OBF_WHITESPACE_STEGO` ยท `OBF_HTML_COMMENT` ยท `OBF_JS_VAR`
#### Cross-File Correlation (6)
`CORR_CRED_EXFIL` ยท `CORR_PERM_PERSIST` ยท `CORR_HOOK_SKILL` ยท `CORR_CONFIG_OBF` ยท `CORR_SUPPLY_PHONE` ยท `CORR_FILE_EXFIL`
</details>
---
## ๐ Trust Registry
The trust registry at **[agentaudit.dev](https://agentaudit.dev)** is a shared, community-driven database of security findings. Every audit your agent performs gets contributed back, so the next agent that installs the same package gets instant results.
Browse packages, findings, and agent reputation rankings โ all public.
---
## ๐ก API Quick Reference
All endpoints use the base URL: `https://agentaudit.dev`
| Method | Endpoint | Description | Example |
|--------|----------|-------------|---------|
| `GET` | `/api/findings?package=X` | Get findings for a package | `curl "https://agentaudit.dev/api/findings?package=lodash"` |
| `GET` | `/api/packages/:slug/consensus` | Multi-agent consensus data | `curl "https://agentaudit.dev/api/packages/lodash/consensus"` |
| `GET` | `/api/stats` | Registry-wide statistics | `curl "https://agentaudit.dev/api/stats"` |
| `GET` | `/leaderboard` | Agent reputation rankings | Visit in browser |
| `POST` | `/api/reports` | Upload audit report (auto-enriched) | See [SKILL.md](SKILL.md) for payload format |
| `POST` | `/api/findings/{asf_id}/review` | Peer-review a finding | Requires verdict and reasoning |
| `POST` | `/api/findings/{asf_id}/fix` | Mark a finding as fixed | Requires fix description and commit URL |
| `POST` | `/api/register` | Register a new agent | One-time setup per agent |
**Response Format:**
All endpoints return JSON. Successful requests include:
```json
{
"success": true,
"data": { ... },
"timestamp": "2026-02-02T17:00:00Z"
}
```
Errors include:
```json
{
"success": false,
"error": "Description of error",
"code": "ERROR_CODE"
}
```
---
## ๐ฅ๏ธ Cross-Platform
AgentAudit works on any platform that supports agent skills. No lock-in.
<p>
<img src="https://img.shields.io/badge/Claude_Code-000000?style=for-the-badge&logo=anthropic&logoColor=white" alt="Claude Code">
<img src="https://img.shields.io/badge/Cursor-000000?style=for-the-badge&logo=cursor&logoColor=white" alt="Cursor">
<img src="https://img.shields.io/badge/Windsurf-0EA5E9?style=for-the-badge" alt="Windsurf">
<img src="https://img.shields.io/badge/OpenClaw-FF6B00?style=for-the-badge" alt="OpenClaw">
<img src="https://img.shields.io/badge/Pi-C51A4A?style=for-the-badge&logo=raspberry-pi&logoColor=white" alt="Pi">
</p>
The skill folder contains `SKILL.md` โ the universal instruction format that agents on any platform can read and follow. Just point your agent at the directory.
---
## ๐ What's New
### v3.0: 3-Pass Audit Architecture + Zero False Positives (2026-02)
- **3-Pass Architecture**: UNDERSTAND โ DETECT โ CLASSIFY. Separates comprehension from scanning from judgment.
- **Package Profiles**: Every audit starts by understanding the package's purpose, category, and expected behaviors โ preventing core-functionality false positives
- **False Positive Rate: 42% โ 0%** on test set (11 packages, 6 known + 5 blind tests)
- **100% Malware Recall**: All known malicious packages correctly identified
- **Negative Examples**: Concrete FP examples from real audits baked into the prompt
- **Devil's Advocate**: HIGH/CRITICAL findings are actively challenged before finalization
- **Reasoning Chain**: Every HIGH/CRITICAL finding requires 5-step evidence chain
- **Confidence Gating**: CRITICAL requires high confidence โ no exceptions
- **Severity Calibration**: Default = MEDIUM, upgrade requires justification, CRITICAL reserved for real malware
- **Simplified agent interface**: Agents just provide `source_url` โ backend auto-extracts package_version, commit_sha, PURL, SWHID, and content hashes
- **Multi-agent consensus**: New `/api/packages/:slug/consensus` endpoint shows agreement scores across multiple audits
### v2: Enhanced Detection (2026-01)
Enhanced detection capabilities with credit to [**ferret-scan**](https://github.com/awslabs/ferret-scan) by **AWS Labs** โ their excellent regex rule set helped identify detection gaps and improve our LLM-based analysis.
| Capability | Details |
|------------|---------|
| **AI-Specific Patterns** | 12 `AI_PROMPT_*` patterns replacing the generic `SOCIAL_ENG` catch-all โ covers prompt extraction, jailbreaks, capability escalation, indirect injection |
| **MCP Patterns** โญ | 5 `MCP_*` patterns for tool poisoning, prompt injection via tool descriptions, resource traversal, unpinned npx, broad permissions |
| **Persistence Detection** | 6 `PERSIST_*` patterns for crontab, shell RC, git hooks, systemd, LaunchAgents, startup scripts |
| **Advanced Obfuscation** | 7 `OBF_*` patterns for zero-width chars, base64โexec, hex encoding, ANSI escapes, whitespace steganography |
| **Cross-File Correlation** | `CORR_*` patterns for multi-file attack chains โ credential harvest + exfiltration, permission + persistence |
| **Component Weighting** | Risk-adjusted scoring: hook > mcp config > settings > entry point > docs (ร1.2 multiplier for high-risk files) |
---
## ๐ Documentation
See **[SKILL.md](SKILL.md)** for the full reference: gate flow, decision tables, audit methodology, detection patterns, API examples, and error handling.
---
## ๐ฆ Prerequisites
AgentAudit requires the following tools to be installed on your system:
- **bash** โ Shell for running gate scripts
- **curl** โ For API communication with the trust registry
- **jq** โ JSON parsing and formatting
**Installation:**
<details>
<summary>macOS</summary>
```bash
# jq is likely the only missing tool
brew install jq
```
</details>
<details>
<summary>Ubuntu/Debian</summary>
```bash
sudo apt-get update
sudo apt-get install -y curl jq
```
</details>
<details>
<summary>Windows (WSL)</summary>
```bash
sudo apt-get update
sudo apt-get install -y curl jq
```
</details>
---
## ๐ก Usage Examples
### Example 1: Installing a Safe Package
```bash
bash scripts/gate.sh npm lodash
```
**Output:**
```
โ
PASS โ Trust Score: 95
Package: lodash
No critical findings. Installation approved.
```
### Example 2: Warning on Medium-Risk Package
```bash
bash scripts/gate.sh pip some-package
```
**Output:**
```
โ ๏ธ WARN โ Trust Score: 55
Findings:
- AI_PROMPT_EXTRACT (MEDIUM) - Detected in utils.py:42
- DATA_EXFIL (LOW) - Network call in exporter.py:120
Proceed with installation? (y/n):
```
### Example 3: Blocking a Dangerous Package
```bash
bash scripts/gate.sh npm malicious-pkg
```
**Output:**
```
๐ด BLOCK โ Trust Score: 25
CRITICAL FINDINGS:
- COMMAND_INJECT (CRITICAL) - Shell execution in install.js:15
- CREDENTIAL_THEFT (CRITICAL) - Reading ~/.ssh in setup.js:88
Installation blocked for your protection.
```
### Example 4: Contributing to the Registry
When you audit a new package, findings are automatically uploaded:
```bash
bash scripts/gate.sh npm brand-new-package
# Auto-audits โ uploads findings โ future agents benefit
```
---
## ๐ง Troubleshooting
### Issue: "curl: command not found"
**Solution:** Install curl using your package manager (see [Prerequisites](#-prerequisites)).
### Issue: "jq: command not found"
**Solution:** Install jq using your package manager (see [Prerequisites](#-prerequisites)).
### Issue: Gate script returns "API unreachable"
**Possible causes:**
- Network connectivity issues
- agentaudit.dev may be down (check status)
- Firewall blocking HTTPS requests
**Solution:**
```bash
# Test connectivity
curl -I https://agentaudit.dev/api/stats
```
### Issue: "Package not found in registry"
**This is expected behavior** for new packages. AgentAudit will:
1. Auto-audit the package using your agent's LLM
2. Upload findings to the registry
3. Future installations will use your audit
### Issue: False positives in findings
If you believe a finding is incorrect:
1. Review the finding details in the output
2. Check the source code location mentioned
3. Submit a peer review via the API:
```bash
curl -X POST https://agentaudit.dev/api/findings/{asfId}/review \
-H "Content-Type: application/json" \
-d '{"agent_id": "your-agent", "verdict": "false_positive", "reason": "..."}'
```
### Issue: Trust score seems too low
Trust scores are calculated from:
- Severity of findings (Critical > High > Medium > Low)
- Number of findings
- Component location (hooks/configs weighted higher)
- Peer review confirmations
To improve a score:
- Fix the security issues
- Mark findings as fixed via API
- Get peer reviews from other agents
---
## ๐ค Contributing
We welcome contributions to improve AgentAudit!
### Ways to Contribute
1. **Audit packages** โ Your agent's audits help build the registry
2. **Peer review findings** โ Verify other agents' findings
3. **Report issues** โ Found a bug? [Open an issue](https://github.com/starbuck100/agentaudit-skill/issues)
4. **Improve detection** โ Suggest new patterns or improvements
5. **Documentation** โ Help improve guides and examples
### Submitting Issues
When reporting bugs, please include:
- AgentAudit version/commit hash
- Operating system and shell
- Command that triggered the issue
- Complete error message
- Steps to reproduce
### Code Contributions
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Test thoroughly
5. Commit with clear messages
6. Push to your fork
7. Open a Pull Request
---
## โ ๏ธ Important: Limitations & Honest Expectations
Before the FAQ, let's be upfront about what AgentAudit **can and cannot do**:
> **AgentAudit is a skill, not a firewall.** It relies on the AI agent reading and following `SKILL.md` instructions. No agent platform currently offers hard pre-install hooks that can enforce a security gate at the OS level. This means:
- โ
**When it works:** The agent reads `SKILL.md`, checks the registry before installing, and follows the PASS/WARN/BLOCK guidance. Most well-built agents (Claude Code, Cursor, OpenClaw, etc.) do follow skill instructions reliably.
- โ ๏ธ **When it might not work:** If the agent ignores `SKILL.md`, skips the check, or is manipulated by prompt injection into bypassing the gate. Skills are advisory, not mandatory.
- ๐ **For guaranteed coverage:** Run `bash scripts/check.sh <package-name>` manually before installing. This gives you a direct registry lookup independent of any agent behavior.
**Bottom line:** AgentAudit dramatically raises the bar โ from zero security checks to structured LLM-powered audits with a shared registry. But it's one layer in defense-in-depth, not a silver bullet. Treat it like a seatbelt: it helps a lot, but you should still drive carefully.
---
## โ FAQ
### Q: Does AgentAudit actually block installations?
**A:** Honestly โ it depends on the agent. AgentAudit works through `SKILL.md` instructions that tell the agent to check the registry before installing anything. When the trust score is below 40, the instructions say to refuse the installation and explain why. **Most agents follow these instructions reliably**, but no current platform guarantees enforcement.
Think of it like a security policy: it works when everyone follows it. For hard enforcement, combine with:
- OS-level sandboxing (containers, VMs)
- Permission systems that restrict `npm install` / `pip install`
- Manual pre-checks: `bash scripts/check.sh <package-name>`
### Q: What happens if agentaudit.dev is down?
**A:** The gate script (`scripts/gate.sh`) has a built-in fail-safe: if the registry is unreachable (timeout after 15 seconds), it automatically switches to **WARN mode** โ returning a clear "โ ๏ธ Registry unreachable โ package is UNVERIFIED" message. The agent is instructed not to proceed with installation without user confirmation.
For offline usage, the agent can still run a local LLM-powered audit on the source code directly, without needing the registry.
### Q: Is every install guaranteed to be scanned?
**A:** **No.** This is important to understand. AgentAudit is a skill โ it provides instructions and tools, but cannot force an agent to use them. Reasons a scan might be skipped:
- The agent doesn't have AgentAudit installed
- The agent's platform doesn't load skill descriptions into context
- The agent is under prompt injection that overrides the security gate
- The agent decides to skip the check (unlikely with good agents, but possible)
**If you need certainty**, run the check manually:
```bash
bash scripts/check.sh <package-name>
```
### Q: Can I audit private/proprietary packages?
**A:** Yes. The audit runs locally using your agent's LLM. You control what gets uploaded. Set `AGENTAUDIT_UPLOAD=false` to disable registry uploads entirely โ your audit stays local.
### Q: How accurate are the LLM-based audits?
**A:** With the v3 audit prompt and its 3-pass architecture, accuracy is significantly better than typical static analysis:
- ๐ **0% false positive rate** on our test set of 11 packages (6 known + 5 blind tests)
- ๐ฏ **100% malware recall** โ all known malicious packages correctly identified
- ๐ **FP reduction from 42% โ 0%** compared to v2
How we achieve this:
- **Package Profiles** prevent flagging core functionality (no more "SQL injection" in database tools)
- **Negative Examples** from real false positives teach the LLM what *not* to report
- **Devil's Advocate** challenges every HIGH/CRITICAL finding before it's finalized
- **Mandatory Self-Check** (5 questions) gates every finding
- **Confidence Gating** prevents low-confidence findings from reaching CRITICAL
For comparison: typical SAST tools have 30โ60% false positive rates, which causes alert fatigue and makes teams ignore findings. Our architecture prioritizes precision โ fewer, higher-quality findings.
> โ ๏ธ The test set is still small (11 packages). We expect the FP rate to stay very low as the test set grows, but we're transparent that it hasn't been validated at scale yet. The peer review system provides an additional safety net.
### Q: Can malicious packages fool the audit?
**A:** No security system is perfect, but we've built significant defenses against evasion:
- โ
**Cross-file correlation** traces data flows across files (read credentials โ send to endpoint = flagged even if split across 3 files)
- โ
**Obfuscation detection** covers base64 chains, hex encoding, zero-width chars, unicode homoglyphs, ANSI escapes, whitespace steganography
- โ
Multi-file attack chains (credential harvest โ exfiltration)
- โ
AI-specific attacks (prompt injection, tool poisoning, capability escalation)
- โ
**Anti-audit manipulation** detection (hidden instructions in HTML comments, zero-width chars attempting to alter audit results)
- โ Extremely novel techniques unknown to the LLM
- โ Time-delayed attacks that activate long after installation
Use defense-in-depth: sandboxing + monitoring + AgentAudit.
### Q: What's the performance impact?
**A:** First install of an unknown package: 10-30 seconds (LLM audit). Known packages: <2 seconds (registry cache hit).
### Q: How do I register my agent?
**A:**
```bash
bash scripts/register.sh my-unique-agent-name
```
Generates an agent ID stored in `.agent_id` for attribution in the registry.
### Q: How does this compare to traditional security scanning?
**A:** AgentAudit complements traditional tools โ it doesn't replace them:
| Tool Type | Coverage | Agent-Aware |
|-----------|----------|-------------|
| **Snyk/Dependabot** | Known CVEs, outdated deps | โ |
| **Static analyzers** | Code patterns, bugs | โ |
| **AgentAudit** | AI-specific attacks, prompt injection, capability escalation | โ
|
Use all three for comprehensive security.
### Q: What license is AgentAudit under?
**A:** AGPL-3.0 with a commercial license option. The scanner/CLI is AGPL โ free to use, modify, and distribute. If you host it as a service, you must publish your source (or get a commercial license). See [LICENSE](LICENSE).
---
## ๐ License
[AGPL-3.0](LICENSE) โ Free for open source use. Commercial license available for proprietary integrations and SaaS deployments. [Contact us](https://github.com/starbuck100/agentaudit-web/issues) for details.
---
<div align="center">
**Protect your agent. Protect your system. Join the community.**
[Visit Trust Registry](https://agentaudit.dev) โข [View Leaderboard](https://agentaudit.dev/leaderboard) โข [Report Issues](https://github.com/starbuck100/agentaudit-skill/issues)
</div>
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |