jeremylongshore/tons-of-skills-marketplace / plugins/devops/log-aggregation-setup/skills/setting-up-log-aggregation
setting-up-log-aggregation
VerifiedExecute use when setting up log aggregation solutions using ELK, Loki, or Splunk. Trigger with phrases like "setup log aggregation", "deploy ELK stack", "configure Loki", or "install Splunk". Generates production-ready configurations for data ingestion, processing, storage, and visualization with proper security and scalability.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 4.5 KB |
| assets/README.md | markdown | 593 B |
| assets/dashboard_elk.json | config | 4.9 KB |
| assets/dashboard_loki.json | config | 6.8 KB |
| assets/dashboard_splunk.json | config | 3.9 KB |
| assets/example_log_data.json | config | 3.2 KB |
| assets/loki_config_template.yaml | config | 3.7 KB |
| assets/splunk_config_template.conf | config | 4.4 KB |
| references/README.md | markdown | 64 B |
| scripts/README.md | markdown | 720 B |
| scripts/setup_elk.pyexecutable | script | 2.6 KB |
| scripts/setup_loki.pyexecutable | script | 2.5 KB |
| scripts/setup_splunk.pyexecutable | script | 2.5 KB |
assets/loki_config_template.yaml
View on GitHub# Loki Configuration File
# This file configures the Loki log aggregation system.
auth_enabled: false # Disable authentication for simplicity (example-value: Enable authentication in production)
server:
http_listen_port: 3100 # Port Loki listens on for HTTP requests
grpc_listen_port: 9096 # Port Loki listens on for gRPC requests
ingester:
lifecycler:
address: 127.0.0.1 # Address of the ingester
ring:
kvstore:
store: inmemory # Use in-memory store for simplicity (example-value: Use a persistent store like Consul or etcd in production)
replication_factor: 1 # Number of replicas for log data
wal:
enabled: true # Enable Write-Ahead Log for durability
dir: /tmp/loki/wal # Directory for the Write-Ahead Log (example-value: Use a persistent volume in production)
chunk_idle_period: 1h # Time after which an inactive chunk is flushed to storage
chunk_block_size: 262144 # Size of each chunk block (256KB)
chunk_retain_period: 24h # Time after which a chunk is deleted from the ingester
max_transfer_retries: 0 # Maximum number of retries for transferring chunks
schema_config:
configs:
- from: 2020-10-24 # Start date for this schema
store: boltdb-shipper # Use BoltDB shipper for index storage
object_store: filesystem # Use filesystem for chunk storage
schema: v11 # Schema version
index:
prefix: index_ # Prefix for index keys
period: 24h # Index rotation period
storage_config:
boltdb_shipper:
active_index_directory: /tmp/loki/index # Directory for the active index (example-value: Use a persistent volume in production)
shared_dir: /tmp/loki/chunks # Directory for shared chunks (example-value: Use a persistent volume in production)
filesystem:
path: /tmp/loki/chunks # Directory for chunk storage (example-value: Use a persistent volume in production)
limits_config:
enforce_metric_name: false # Disable enforcement of metric names
reject_old_samples: true # Reject samples older than the configured time
reject_old_samples_max_age: 168h # Maximum age of samples (7 days)
max_global_streams_per_user: 0 # 0 means unlimited
max_streams_per_user: 0 # 0 means unlimited
ingestion_rate_mb: 100 # Maximum ingestion rate in MB/s
ingestion_burst_size_mb: 200 # Maximum burst size in MB
max_line_size: 512000 # Maximum line size in bytes (500KB)
max_line_length: 512000 # DEPRECATED: use max_line_size instead
max_query_lookback: 720h # Maximum query lookback (30 days)
split_queries_by_interval: 12h # Split queries by this interval
max_concurrent_queries: 30 # Maximum number of concurrent queries
max_query_series: 1000 # Maximum number of series returned by a query
max_query_parallelism: 16 # Maximum query parallelism
max_query_length: 720h # Maximum query length (30 days)
compactor:
working_directory: /tmp/loki/compactor # Directory for compactor working files (example-value: Use a persistent volume in production)
shared_store: filesystem # Use filesystem for shared storage
compaction_interval: 1h # Interval between compactor runs
retention_enabled: true # Enable retention of old chunks
retention_delete_delay: 24h # Delay before deleting old chunks
retention_max_age: 720h # Maximum age of chunks to retain (30 days)
ruler:
storage:
type: local
local:
directory: /tmp/loki/rules # Directory to store the rules (example-value: Use a persistent volume in production)
rule_path: /tmp/loki/rules # Path where rules are stored (example-value: Use a persistent volume in production)
alertmanager_url: "" # URL of the Alertmanager instance (example-value: YOUR_ALERTMANAGER_URL)
poll_interval: 30s # Interval to poll for rule changes
enable_api: true # Enable the API for managing rulesRelated skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |