monte-carlo-data/mc-agent-toolkit / skills/push-ingestion
push-ingestion
VerifiedExpert guide for Monte Carlo's push ingestion model. Use this skill whenever a customer or engineer mentions: pushing data to Monte Carlo, the IngestionService, pycarlo push APIs, build me a collection script, push metadata/lineage/query logs, invocation_id tracing, custom lineage nodes or edges, deleting push tables, or any question about why pushed data is not showing up. Also trigger when they ask to generate code that collects metadata, table schema, row counts, freshness, lineage, or query history from any data warehouse or data source and sends it to Monte Carlo. If the user mentions any warehouse, database, or data platform alongside any Monte Carlo topic, this skill is almost certainly relevant.
references/prerequisites.md
View on GitHub# Prerequisites
## Two keys, two purposes
Push ingestion requires **two separate Monte Carlo API keys** — one for pushing data, one
for reading/verifying it. They use identical header names but different endpoints.
| Key | Purpose | Endpoint |
|---|---|---|
| **Ingestion key** (scope=`Ingestion`) | Push metadata, lineage, query logs | `https://integrations.getmontecarlo.com` |
| **GraphQL API key** | Verify pushed data, run management mutations | `https://api.getmontecarlo.com/graphql` |
Both authenticate with:
```
x-mcd-id: <key-id>
x-mcd-token: <key-secret>
```
The secret for both is shown **only once** at creation time — store it securely immediately.
---
## Create the Ingestion key (for pushing)
Use the Monte Carlo CLI:
```bash
montecarlo integrations create-key \
--scope Ingestion \
--description "Push ingestion key"
```
Output:
```
Key id: <id>
Key secret: <secret> ← only shown once
```
Install the CLI if needed:
```bash
pip install montecarlodata
montecarlo configure # enter your API key when prompted
```
**Optional — restrict to a specific warehouse:**
If you want the key to only work for one warehouse UUID, use the GraphQL mutation instead:
```graphql
mutation {
createIntegrationKey(
description: "Push key for warehouse XYZ"
scope: Ingestion
warehouseIds: ["<warehouse-uuid>"]
) {
key { id secret }
}
}
```
---
## Create the GraphQL API key (for verification)
1. Go to **https://getmontecarlo.com/settings/api**
2. Click **Add**
3. Choose key type (personal or account-level — account-level requires Account Owner role)
4. Copy the **Key ID** and **Secret** immediately
The GraphQL endpoint is: `https://api.getmontecarlo.com/graphql`
Test it:
```bash
curl -s -X POST https://api.getmontecarlo.com/graphql \
-H "x-mcd-id: <id>" \
-H "x-mcd-token: <secret>" \
-H "Content-Type: application/json" \
-d '{"query": "{ getUser { email } }"}' | python3 -m json.tool
```
---
## Find your warehouse (resource) UUID
The Ingestion key needs to reference the correct MC resource UUID. To find it:
```graphql
query {
getUser {
account {
warehouses {
uuid
name
connectionType
}
}
}
}
```
Or in the MC UI: **Settings → Integrations** → click the warehouse → copy the UUID from the URL.
---
## Install pycarlo (optional)
The pycarlo SDK simplifies push calls, but is not required. You can also call the push APIs
directly via HTTP/curl — see `references/direct-http-api.md`.
```bash
pip install pycarlo
```
Initialize the ingestion client in your script:
```python
from pycarlo.core import Client, Session
from pycarlo.features.ingestion import IngestionService
client = Client(session=Session(
mcd_id="<ingestion-key-id>",
mcd_token="<ingestion-key-secret>",
scope="Ingestion",
))
service = IngestionService(mc_client=client)
```
Load credentials from environment variables (recommended):
```python
import os
service = IngestionService(mc_client=Client(session=Session(
mcd_id=os.environ["MCD_INGEST_ID"],
mcd_token=os.environ["MCD_INGEST_TOKEN"],
scope="Ingestion",
)))
```
---
## Environment variable conventions
The script templates use these env var names by default:
| Variable | Key type | Used by |
|---|---|---|
| `MCD_INGEST_ID` | Ingestion key ID | push and collect_and_push scripts |
| `MCD_INGEST_TOKEN` | Ingestion key secret | push and collect_and_push scripts |
| `MCD_ID` | GraphQL API key ID | verification scripts, slash commands |
| `MCD_TOKEN` | GraphQL API key secret | verification scripts, slash commands |
| `MCD_RESOURCE_UUID` | Warehouse UUID | all scripts |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |