itsmostafa/aws-agent-skills / skills/s3
s3
VerifiedAWS S3 object storage for bucket management, object operations, and access control. Use when creating buckets, uploading files, configuring lifecycle policies, setting up static websites, managing permissions, or implementing cross-region replication.
AWS S3
Amazon Simple Storage Service (S3) provides scalable object storage with industry-leading durability (99.999999999%). S3 is fundamental to AWS—used for data lakes, backups, static websites, and as storage for many other AWS services.
Table of Contents
Core Concepts
Buckets
Containers for objects. Bucket names are globally unique across all AWS accounts.
Objects
Files stored in S3, consisting of data, metadata, and a unique key (path). Maximum size: 5 TB.
Storage Classes
| Class | Use Case | Durability | Availability |
|---|---|---|---|
| Standard | Frequently accessed | 99.999999999% | 99.99% |
| Intelligent-Tiering | Unknown access patterns | 99.999999999% | 99.9% |
| Standard-IA | Infrequent access | 99.999999999% | 99.9% |
| Glacier Instant | Archive with instant retrieval | 99.999999999% | 99.9% |
| Glacier Flexible | Archive (minutes to hours) | 99.999999999% | 99.99% |
| Glacier Deep Archive | Long-term archive | 99.999999999% | 99.99% |
Versioning
Keeps multiple versions of an object. Essential for data protection and recovery.
Common Patterns
Create a Bucket with Best Practices
AWS CLI:
# Create bucket (us-east-1 doesn't need LocationConstraint)
aws s3api create-bucket \
--bucket my-secure-bucket-12345 \
--region us-west-2 \
--create-bucket-configuration LocationConstraint=us-west-2
# Enable versioning
aws s3api put-bucket-versioning \
--bucket my-secure-bucket-12345 \
--versioning-configuration Status=Enabled
# Block public access
aws s3api put-public-access-block \
--bucket my-secure-bucket-12345 \
--public-access-block-configuration \
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
# Enable encryption
aws s3api put-bucket-encryption \
--bucket my-secure-bucket-12345 \
--server-side-encryption-configuration '{
"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]
}'
boto3:
import boto3
s3 = boto3.client('s3', region_name='us-west-2')
# Create bucket
s3.create_bucket(
Bucket='my-secure-bucket-12345',
CreateBucketConfiguration={'LocationConstraint': 'us-west-2'}
)
# Enable versioning
s3.put_bucket_versioning(
Bucket='my-secure-bucket-12345',
VersioningConfiguration={'Status': 'Enabled'}
)
# Block public access
s3.put_public_access_block(
Bucket='my-secure-bucket-12345',
PublicAccessBlockConfiguration={
'BlockPublicAcls': True,
'IgnorePublicAcls': True,
'BlockPublicPolicy': True,
'RestrictPublicBuckets': True
}
)
Upload and Download Objects
# Upload a single file
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt
# Upload with metadata
aws s3 cp myfile.txt s3://my-bucket/path/myfile.txt \
--metadata "environment=production,version=1.0"
# Download a file
aws s3 cp s3://my-bucket/path/myfile.txt ./myfile.txt
# Sync a directory
aws s3 sync ./local-folder s3://my-bucket/prefix/ --delete
# Copy between buckets
aws s3 cp s3://source-bucket/file.txt s3://dest-bucket/file.txt
Generate Presigned URL
import boto3
from botocore.config import Config
s3 = boto3.client('s3', config=Config(signature_version='s3v4'))
# Generate presigned URL for download (GET)
url = s3.generate_presigned_url(
'get_object',
Params={'Bucket': 'my-bucket', 'Key': 'path/to/file.txt'},
ExpiresIn=3600 # URL valid for 1 hour
)
# Generate presigned URL for upload (PUT)
upload_url = s3.generate_presigned_url(
'put_object',
Params={
'Bucket': 'my-bucket',
'Key': 'uploads/newfile.txt',
'ContentType': 'text/plain'
},
ExpiresIn=3600
)
Configure Lifecycle Policy
cat > lifecycle.json << 'EOF'
{
"Rules": [
{
"ID": "MoveToGlacierAfter90Days",
"Status": "Enabled",
"Filter": {"Prefix": "logs/"},
"Transitions": [
{"Days": 90, "StorageClass": "GLACIER"}
],
"Expiration": {"Days": 365}
},
{
"ID": "DeleteOldVersions",
"Status": "Enabled",
"Filter": {},
"NoncurrentVersionExpiration": {"NoncurrentDays": 30}
}
]
}
EOF
aws s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration file://lifecycle.json
Event Notifications to Lambda
aws s3api put-bucket-notification-configuration \
--bucket my-bucket \
--notification-configuration '{
"LambdaFunctionConfigurations": [
{
"LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:ProcessS3Upload",
"Events": ["s3:ObjectCreated:*"],
"Filter": {
"Key": {
"FilterRules": [
{"Name": "prefix", "Value": "uploads/"},
{"Name": "suffix", "Value": ".jpg"}
]
}
}
}
]
}'
CLI Reference
High-Level Commands (aws s3)
| Command | Description |
|---|---|
aws s3 ls | List buckets or objects |
aws s3 cp | Copy files |
aws s3 mv | Move files |
aws s3 rm | Delete files |
aws s3 sync | Sync directories |
aws s3 mb | Make bucket |
aws s3 rb | Remove bucket |
Low-Level Commands (aws s3api)
| Command | Description |
|---|---|
aws s3api create-bucket | Create bucket with options |
aws s3api put-object | Upload with full control |
aws s3api get-object | Download with options |
aws s3api delete-object | Delete single object |
aws s3api put-bucket-policy | Set bucket policy |
aws s3api put-bucket-versioning | Enable versioning |
aws s3api list-object-versions | List all versions |
Useful Flags
--recursive: Process all objects in prefix--exclude/--include: Filter objects--dryrun: Preview changes--storage-class: Set storage class--acl: Set access control (prefer policies instead)
Best Practices
Security
- Block public access at account and bucket level
- Enable versioning for data protection
- Use bucket policies over ACLs
- Enable encryption (SSE-S3 or SSE-KMS)
- Enable access logging for audit
- Use VPC endpoints for private access
- Enable MFA Delete for critical buckets
Performance
- Use Transfer Acceleration for distant uploads
- Use multipart upload for files > 100 MB
- Randomize key prefixes for high-throughput (less relevant with 2024 improvements)
- Use byte-range fetches for large file downloads
Cost Optimization
- Use lifecycle policies to transition to cheaper storage
- Enable Intelligent-Tiering for unpredictable access
- Delete incomplete multipart uploads:
{ "Rules": [{ "ID": "AbortIncompleteMultipartUpload", "Status": "Enabled", "Filter": {}, "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7} }] } - Use S3 Storage Lens to analyze storage patterns
Troubleshooting
Access Denied Errors
Causes:
- Bucket policy denies access
- IAM policy missing permissions
- Public access block preventing access
- Object owned by different account
- VPC endpoint policy blocking
Debug steps:
# Check your identity
aws sts get-caller-identity
# Check bucket policy
aws s3api get-bucket-policy --bucket my-bucket
# Check public access block
aws s3api get-public-access-block --bucket my-bucket
# Check object ownership
aws s3api get-object-attributes \
--bucket my-bucket \
--key myfile.txt \
--object-attributes ObjectOwner
CORS Errors
Symptom: Browser blocks cross-origin request
Fix:
aws s3api put-bucket-cors --bucket my-bucket --cors-configuration '{
"CORSRules": [{
"AllowedOrigins": ["https://myapp.com"],
"AllowedMethods": ["GET", "PUT", "POST"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600
}]
}'
Slow Uploads
Solutions:
- Use multipart upload for large files
- Enable Transfer Acceleration
- Use
aws s3 cpwith--expected-sizefor large files - Check network throughput to the region
403 on Presigned URL
Causes:
- URL expired
- Signer lacks permissions
- Bucket policy blocks access
- Region mismatch (v4 signatures are region-specific)
Fix: Ensure signer has permissions and use correct region.
References
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |