ilandahan/AID / skills/role-qa-engineer
role-qa-engineer
VerifiedQA Engineer role in AID methodology. Use for test strategy, BDD scenarios, bug reporting, acceptance testing, flaky test prevention.
| File | Kind | Size |
|---|---|---|
| SKILL.extended.md | markdown | 14.9 KB |
| SKILL.md | skill md | 5.9 KB |
| references/bdd-gherkin-guide.md | markdown | 13.9 KB |
| references/bug-investigation-checklist.md | markdown | 5.3 KB |
| references/code-organization-standards.md | markdown | 8.2 KB |
| references/condition-based-waiting.ts | script | 7.9 KB |
| references/security-in-tests.md | markdown | 6.2 KB |
| references/test-data-examples.md | markdown | 8.7 KB |
| references/test-verification-checklist.md | markdown | 7.2 KB |
| scripts/find-polluter.shexecutable | script | 3.9 KB |
references/security-in-tests.md
View on GitHub# Security in Test Code
## Purpose
This document defines security requirements for test code. Test code often has less scrutiny than production code, making it a common source of credential leaks.
---
## IRON RULE
**NO HARDCODED CREDENTIALS ANYWHERE**
This includes:
- Test files (`.test.ts`, `.spec.ts`)
- Fixture files
- README and documentation
- Example code
- Configuration files committed to git
- Comments
---
## What Counts as Credentials?
| Type | Examples | Risk |
|------|----------|------|
| Passwords | `password123`, `P@ssw0rd!` | Account compromise |
| API Keys | `sk_live_xxx`, `AKIA...` | Service abuse/billing |
| Tokens | `ghp_xxx`, `xoxb-xxx` | API access |
| Connection strings | `postgresql://user:pass@host` | Database access |
| Private keys | `-----BEGIN [RSA] PRIVATE KEY-----` | Full compromise |
| Secrets | `JWT_SECRET=<token-value>` | Token forgery |
---
## Patterns: What NOT to Do
### ❌ Hardcoded in Test Files
```typescript
// BAD - Real-looking credentials
const testUser = {
email: 'admin@company.com',
password: '<example-password>'
};
// BAD - API keys that look real
const config = {
stripeKey: 'sk_test_FAKE_KEY_FOR_DOCS_12345',
awsKey: 'AKIAIOSFODNN7EXAMPLE'
};
```
### ❌ Hardcoded in Fixtures
```typescript
// tests/fixtures/users.ts - BAD
export const testUsers = [
{ email: 'john@company.com', password: '<test-password>' },
{ email: 'admin@company.com', password: '<test-password>' }
];
```
### ❌ Hardcoded in README
```markdown
<!-- BAD - README.md -->
## Running Tests
Set up your environment:
```
TEST_DB_PASSWORD=mysecretpassword
STRIPE_KEY=sk_test_abcdef123456
```
```
### ❌ Hardcoded in Config
```javascript
// jest.config.js - BAD
module.exports = {
globals: {
TEST_API_KEY: 'real-api-key-here'
}
};
```
---
## Patterns: What TO Do
### ✅ Environment Variables with Safe Defaults
```typescript
// GOOD - Environment variables with obvious fake defaults
const testConfig = {
apiKey: process.env.TEST_API_KEY || 'test_key_not_real_00000000',
dbUrl: process.env.TEST_DB_URL || 'postgresql://test:test@localhost:5432/test_db',
secret: process.env.TEST_SECRET || 'not-a-real-secret-for-testing-only'
};
// GOOD - In test files
const testUser = {
email: process.env.TEST_USER_EMAIL || 'test@example.com',
password: process.env.TEST_USER_PASSWORD || 'TestOnlyPassword123'
};
```
### ✅ Factory Functions with Fake Data
```typescript
// tests/fixtures/user-factory.ts - GOOD
import { faker } from '@faker-js/faker';
export function createTestUser(overrides = {}) {
return {
id: faker.string.uuid(),
email: faker.internet.email({ provider: 'test-example.com' }),
password: 'pw-' + faker.string.alphanumeric(16),
name: faker.person.fullName(),
...overrides
};
}
// Usage
const user = createTestUser();
const adminUser = createTestUser({ role: 'admin' });
```
### ✅ .env.test File (Gitignored)
```bash
# .env.test (ADD TO .gitignore)
TEST_DB_URL=postgresql://test:testpassword@localhost:5432/test_db
TEST_API_KEY=sk_test_your_sandbox_key_here
TEST_STRIPE_KEY=sk_test_your_stripe_test_key
```
```gitignore
# .gitignore
.env
.env.*
!.env.example
```
### ✅ .env.example with Placeholders
```bash
# .env.example (SAFE TO COMMIT)
TEST_DB_URL=postgresql://user:password@localhost:5432/test_db
TEST_API_KEY=your_test_api_key_here
TEST_STRIPE_KEY=sk_test_your_key_here
```
### ✅ README with Placeholders
```markdown
<!-- GOOD - README.md -->
## Running Tests
1. Copy `.env.example` to `.env.test`
2. Fill in your test credentials
3. Run `npm test`
See `.env.example` for required variables.
```
---
## Credential Patterns by Service
### Database Connections
```typescript
// GOOD
const dbConfig = {
host: process.env.TEST_DB_HOST || 'localhost',
port: parseInt(process.env.TEST_DB_PORT || '5432'),
database: process.env.TEST_DB_NAME || 'test_db',
user: process.env.TEST_DB_USER || 'test_user',
password: process.env.TEST_DB_PASSWORD || 'test_password'
};
```
### API Keys
```typescript
// GOOD - Stripe
const stripeKey = process.env.STRIPE_TEST_KEY || 'sk_test_fake_key_for_testing';
// GOOD - AWS
const awsConfig = {
accessKeyId: process.env.AWS_ACCESS_KEY_ID || 'AKIATEST00000000000',
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY || 'test/secret/key/not/real'
};
// GOOD - GitHub
const githubToken = process.env.GITHUB_TOKEN || 'ghp_testtoken00000000000000000000000000';
```
### JWT Secrets
```typescript
// GOOD
const jwtSecret = process.env.JWT_SECRET || 'test-jwt-secret-not-for-production-use';
```
---
## Security Checklist
### Before Committing Test Code
- [ ] Search for `password` - no real passwords?
- [ ] Search for `secret` - no real secrets?
- [ ] Search for `sk_live`, `pk_live` - no production keys?
- [ ] Search for `AKIA` - no real AWS keys?
- [ ] Search for `ghp_`, `xoxb-` - no real tokens?
- [ ] Check README - no real credentials in examples?
- [ ] Check `.gitignore` - includes `.env*`?
### Automated Checks
Add to CI pipeline:
```yaml
# .github/workflows/security.yml
- name: Check for secrets
run: |
# Fail if we find patterns that look like real credentials
! grep -rE "(sk_live_|pk_live_|AKIA[A-Z0-9]{16})" tests/
! grep -rE "password.*=.*['\"][^'\"]{8,}['\"]" tests/ --include="*.ts"
```
---
## What If I Need Real Credentials for Tests?
### For Local Development
1. Create `.env.test` (gitignored)
2. Add real test/sandbox credentials there
3. Never commit this file
### For CI/CD
1. Use repository secrets (GitHub Secrets, GitLab CI Variables)
2. Inject at runtime:
```yaml
# GitHub Actions
- name: Run tests
env:
TEST_API_KEY: ${{ secrets.TEST_API_KEY }}
TEST_DB_URL: ${{ secrets.TEST_DB_URL }}
run: npm test
```
### For Test/Sandbox Environments
Use sandbox/test API keys provided by services:
- Stripe: `sk_test_...` keys (safe for testing)
- AWS: IAM user with minimal test permissions
- Database: Dedicated test database with test data only
---
## Quick Reference
| Scenario | Solution |
|----------|----------|
| Need user credentials | Factory with fake data |
| Need API key | Env var with fake default |
| Need database URL | Env var with localhost default |
| Need to document setup | .env.example with placeholders |
| Running in CI | Repository secrets |
| Local development | .env.test (gitignored) |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |