fcakyon/claude-codex-settings / plugins/openobserve-skills/skills/openobserve-api
openobserve-api
UnverifiedThis skill should be used when user asks to "query OpenObserve", "create OpenObserve dashboard", "edit OpenObserve panel", "fetch OpenObserve logs", "run OpenObserve search", "list OpenObserve streams", "ingest into OpenObserve", or works with OpenObserve Cloud / self-hosted via REST API. Covers auth, search/SQL, streams, dashboards (CRUD + per-panel ops), the v8 panel JSON schema, and known pitfalls (re-aggregation, hash concurrency, microsecond timestamps).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 16.1 KB |
| references/around.md | markdown | 2.5 KB |
| references/bulk.md | markdown | 7.0 KB |
| references/delete.md | markdown | 10.8 KB |
| references/index.md | markdown | 1.7 KB |
| references/json.md | markdown | 6.8 KB |
| references/list.md | markdown | 2.7 KB |
| references/loki.md | markdown | 2.8 KB |
| references/multi.md | markdown | 5.2 KB |
| references/otlp.md | markdown | 4.4 KB |
| references/recipes/build-dashboard.shexecutable | script | 1.5 KB |
| references/recipes/search-logs.shexecutable | script | 683 B |
| references/recipes/update-panel.shexecutable | script | 884 B |
| references/schema.md | markdown | 2.6 KB |
| references/search.md | markdown | 13.3 KB |
| references/setting.md | markdown | 5.8 KB |
| references/value.md | markdown | 1.9 KB |
references/bulk.md
View on GitHub---
title: Bulk
description: Ingest logs in bulk via POST /api/{org}/_bulk using NDJSON. Compatible with Elasticsearch _bulk API and supports up to 200 fields per record.
---
# Logs Ingestion - Bulk
Endpoint: `POST /api/{organization}/_bulk`
This will upload multiple records in batch with ndjson (newline delimited json). This API is compatible with Elasticsearch _bulk API.
## Request
e.g. POST /api/myorg/_bulk
```json
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus", "kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5", "kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal", "kubernetes.labels.app.kubernetes.io/component": "prometheus", "kubernetes.labels.app.kubernetes.io/instance": "k8s", "kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator", "kubernetes.labels.app.kubernetes.io/name": "prometheus", "kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus", "kubernetes.labels.app.kubernetes.io/version": "2.39.1", "kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c", "kubernetes.labels.operator.prometheus.io/name": "k8s", "kubernetes.labels.operator.prometheus.io/shard": "0", "kubernetes.labels.prometheus": "k8s", "kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1", "kubernetes.namespace_name": "monitoring", "kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64", "kubernetes.pod_name": "prometheus-k8s-1", "log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:108 level=warn component=k8s_client_runtime func=Warningf msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"", "stream": "stderr" }
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus", "kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5", "kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal", "kubernetes.labels.app.kubernetes.io/component": "prometheus", "kubernetes.labels.app.kubernetes.io/instance": "k8s", "kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator", "kubernetes.labels.app.kubernetes.io/name": "prometheus", "kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus", "kubernetes.labels.app.kubernetes.io/version": "2.39.1", "kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c", "kubernetes.labels.operator.prometheus.io/name": "k8s", "kubernetes.labels.operator.prometheus.io/shard": "0", "kubernetes.labels.prometheus": "k8s", "kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1", "kubernetes.namespace_name": "monitoring", "kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64", "kubernetes.pod_name": "prometheus-k8s-1", "log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:116 level=error component=k8s_client_runtime func=ErrorDepth msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"", "stream": "stderr" }
```
> First line is stream action
>
> Second line is record data
### Request action
Create record
```json
{ "index" : { "_index" : "stream1" } }
```
We support `create`, `index`, and `update` actions. `delete` is not supported.
The `_index` is stream name what you want to use.
## Response
The response follows the same shape as the Elasticsearch `_bulk` API, not the simplified `code`/`status` shape shown in older versions of this page:
```json
{
"took": 0,
"errors": false,
"items": [
{
"index": {
"_index": "stream1",
"_id": "5uhtM5kBRQHIfnE6L6mm",
"_version": 1,
"result": "created",
"_shards": { "total": 1, "successful": 1, "failed": 0 },
"_seq_no": 1,
"_primary_term": 1,
"status": 200
}
}
]
}
```
`items` has one entry per submitted record, each keyed by the action it was submitted with (`index`, `create`, or `update`). `errors` is `true` if any item failed. `took` is always `0` — OpenObserve does not currently populate it with elapsed time.
`_version`, `_shards`, `_seq_no`, and `_primary_term` are fixed placeholder values kept for Elasticsearch API compatibility; they don't carry real per-record metadata.
A failed item has `status` >= 400 and carries `error` and `originalRecord` instead of `result`/`_shards`/`_seq_no`/`_primary_term`:
```json
{
"index": {
"_index": "stream1",
"_id": "5uhtM5kBRQHIfnE6L6mm",
"status": 422,
"error": {
"type": "Too old data, only last 5 hours data can be ingested. Data discarded.",
"reason": "Too old data, only last 5 hours data can be ingested. Data discarded.",
"index_uuid": "1",
"shard": "1",
"index": "stream1"
},
"originalRecord": {
"kubernetes.container_name": "prometheus",
"log": "..."
}
}
}
```
## Restriction on number of fields/columns per record
> Applicable to cloud version
Please note only records having 200 or less fields/columns will be considered for ingestion , records having more than 200 fields/columns will be discarded with failed status.
> Applicable to open source version
One can configure ZO_COLS_PER_RECORD_LIMIT to set desired value for allowed number of fields/columns per record.
## Timestamp
By default we add a field `_timestamp` for each record with the value of `NOW` in microseconds (unix epoch value).
we support use of two fields to override the default value.
- _timestamp
- @timestamp
2 data formats are supported for timestamp fields
the value support two data type format:
- microseconds (unix epoch value)
- string value
- RFC 3339 and ISO 8601 date and time string such as `1996-12-19T16:39:57-08:00`
- RFC 2822 date and time string such as `Tue, 1 Jul 2003 10:52:37 +0200`
eg:
use microseconds
```json
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.container_name": "prometheus", "_timestamp": "1674789786006000" }
```
use string datetime
```json
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.container_name": "prometheus", "_timestamp": "2023-01-02T10:01:01Z" }
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| Agent Cost Reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| Agent Cost Reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |