aktonay/skillz / skills/cloud/bigquery-basics
bigquery-basics
VerifiedManages datasets, tables, and jobs in BigQuery, and integrates with BigQuery ML and Gemini for advanced data analytics and AI-driven insights. Use when you need to interact with BigQuery, run SQL queries, manage BigQuery resources, or leverage BigQuery's built-in ML capabilities. Also use when performing data analysis, ingesting data into BigQuery, or developing AI applications on BigQuery.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 4.6 KB |
| references/cli-usage.md | markdown | 2.0 KB |
| references/client-library-usage.md | markdown | 2.2 KB |
| references/core-concepts.md | markdown | 3.4 KB |
| references/iac-usage.md | markdown | 1.7 KB |
| references/iam-security.md | markdown | 1.9 KB |
| references/mcp-usage.md | markdown | 2.3 KB |
references/core-concepts.md
View on GitHub# BigQuery Core Concepts
BigQuery is a fully managed, AI-ready data platform that helps you manage and
analyze your data with built-in features like machine learning, search,
geospatial analysis, and business intelligence. BigQuery's serverless
architecture lets you use languages like SQL and Python to answer your
organization's biggest questions with zero infrastructure management.
BigQuery provides a uniform way to work with both structured and unstructured
data and supports open table formats like Apache Iceberg. BigQuery streaming
supports continuous data ingestion and analysis while BigQuery's scalable,
distributed analysis engine lets you query terabytes in seconds and petabytes in
minutes.
## Architecture
BigQuery's architecture separates compute and storage, connected by a
petabit-scale network.
- **BigQuery Storage:** A columnar storage format optimized for analytical
queries. It can be replicated across multiple locations for high
availability.
- **BigQuery Analytics:** A scalable, distributed analysis engine that can
process data in BigQuery and in external sources.
## Resource Hierarchy
BigQuery organizes resources in a structured hierarchy:
1. **Organization/Folder/Project:** Standard Google Cloud resource containers.
2. **Dataset:** The top-level container for tables and views.
3. **Table/View:** The basic unit of data storage and logical representation.
## Analytics Workflows
- **Ad Hoc Analysis:** Using GoogleSQL for interactive queries.
- **Geospatial Analysis:** Analyzing and visualizing spatial data using
geography types.
- **Machine Learning (BigQuery ML):** Creating and executing ML models
directly in BigQuery using SQL.
- **Gemini in BigQuery:** AI-powered assistance for data preparation, SQL
generation, and visualization. Refer to the [Gemini
Models](https://ai.google.dev/gemini-api/docs/models) for more information.
- **Stream Processing (BigQuery continuous queries):** Long running SQL
statements that analyze and transform incoming data in near real time as it
arrives in BigQuery. This feature enables unbounded streaming pipelines for
real-time AI inference (using Vertex AI) and Reverse ETL to downstream
systems. Results can be exported to Pub/Sub, Bigtable, Spanner, or other
BigQuery tables. Note that running continuous queries requires a BigQuery
reservation with a `CONTINUOUS` assignment type.
## BigQuery Studio
A unified workspace for data engineering, analysis, and predictive modeling.
- **SQL Editor:** With code completion and generation.
- **Python Notebooks:** Built-in support for Colab Enterprise and BigQuery
DataFrames (BigFrames).
- **Data Discovery:** Integrated with Dataplex for search and profiling.
## Pricing
BigQuery pricing consists of two main components: compute (analysis) costs and
storage costs.
- **Storage:** Storage costs are based on the amount of data stored in
BigQuery tables. Storage is classified as either active storage (any table
or partition modified in the last 90 days) and long-term storage (data that
hasn't been modified for 90 consecutive days, resulting in a price drop of
approximately 50%).
- **Analysis:** Billed based on bytes processed (On-demand) or dedicated slots
(Capacity/Reservations).
For the latest pricing details, visit: [BigQuery
Pricing](https://cloud.google.com/bigquery/pricing).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |