SSZcreate/Pentab_server / .agents/skills/cybersecurity/skills/detecting-shadow-it-cloud-usage
detecting-shadow-it-cloud-usage
VerifiedDetect unauthorized SaaS and cloud service usage (shadow IT) by parsing proxy access logs, DNS query logs, and firewall/netflow data with Python pandas to aggregate traffic by domain, classify domains against known SaaS categories, and score risk by data volume and user count. Use when auditing an organization for unsanctioned cloud/SaaS usage or generating a shadow IT discovery report with remediation recommendations.
| File | Kind | Size |
|---|---|---|
| LICENSE | other | 11.0 KB |
| SKILL.md | skill md | 2.5 KB |
| references/api-reference.md | markdown | 2.3 KB |
| scripts/agent.py | script | 10.8 KB |
references/api-reference.md
View on GitHub# API Reference — Detecting Shadow IT Cloud Usage
## Libraries Used
- **pandas**: DataFrame aggregation for traffic analysis — groupby, agg, nunique
- **tldextract**: Accurate registered domain extraction from URLs/hostnames
- **csv**: CSV log parsing with DictReader
- **re**: Regex parsing for Squid proxy and BIND DNS query log formats
## CLI Interface
```
python agent.py access.log --type proxy parse
python agent.py access.log --type proxy analyze
python agent.py dns-queries.log --type dns full
python agent.py traffic.csv --type csv --approved approved.txt full
```
## Core Functions
### `parse_proxy_log(filepath)` — Parse Squid/common proxy access logs
Regex pattern matches Squid format: `timestamp duration client_ip status bytes method url`.
Falls back to Apache Common Log Format parsing.
### `parse_dns_log(filepath)` — Parse BIND/named DNS query logs
Extracts query name and type from `query: DOMAIN IN TYPE` patterns.
Strips trailing dots from FQDNs.
### `parse_csv_log(filepath)` — Parse generic CSV traffic logs
Expects columns: timestamp, src_ip, dst_domain, bytes_out, bytes_in.
### `analyze_traffic(records)` — Aggregate and classify traffic
Uses pandas groupby on domain: total_bytes (sum), request_count (count),
unique_users (nunique). Falls back to collections.defaultdict if pandas unavailable.
### `classify_domain(domain)` — Categorize against SaaS database
Categories: storage, email, dev_tools, ai_ml, messaging, file_sharing, vpn_proxy.
### `full_audit(log_path, log_type, approved_list)` — Complete shadow IT audit
## Risk Scoring
| Factor | Points |
|--------|--------|
| Unapproved domain | +30 |
| Storage/file-sharing/VPN category | +25 |
| Email category | +15 |
| Data volume (per 10 MB) | +1 (max 20) |
| Unique users (per user) | +3 (max 15) |
## SaaS Category Database
| Category | Example Domains |
|----------|----------------|
| storage | dropbox.com, box.com, mega.nz, wetransfer.com |
| email | protonmail.com, tutanota.com, guerrillamail.com |
| dev_tools | github.com, gitlab.com, replit.com |
| ai_ml | chat.openai.com, claude.ai, huggingface.co |
| messaging | telegram.org, discord.com, signal.org |
| file_sharing | pastebin.com, file.io, gofile.io |
| vpn_proxy | nordvpn.com, expressvpn.com, protonvpn.com |
## Dependencies
- `pandas` >= 1.5.0
- `tldextract` >= 3.4.0 (optional, improves domain extraction accuracy)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 465k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 465k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 465k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 192k | Updated | Claude CodeCodex | Verified | Install |