DAAF-Contribution-Community/daaf / .claude/skills/geopandas
geopandas
VerifiedSpatial data: GeoDataFrames, spatial joins, CRS/projections, choropleth/interactive maps, spatial autocorrelation, PySAL. Use for geographic data, spatial files (Shapefile, GeoPackage, GeoParquet), or spatial stats. For charts without GIS use plotly. R equivalent: sf-terra (use when execution language is R).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.6 KB |
| references/crs-projections.md | markdown | 9.4 KB |
| references/data-io.md | markdown | 8.1 KB |
| references/gotchas.md | markdown | 11.4 KB |
| references/pysal-spatial-stats.md | markdown | 19.5 KB |
| references/quickstart.md | markdown | 7.9 KB |
| references/raster-integration.md | markdown | 10.1 KB |
| references/spatial-operations.md | markdown | 11.2 KB |
| references/visualization.md | markdown | 15.2 KB |
references/gotchas.md
View on GitHub# Common Gotchas and Troubleshooting
Frequent issues when working with geopandas and the spatial Python stack — symptoms, causes, and fixes.
---
## CRS Mismatch (a Warning, Not an Error)
### Symptom
```
UserWarning: CRS mismatch between the CRS of left geometries and the CRS of right geometries.
# The join still runs: e.g. EPSG:4326 points vs EPSG:5070 polygons returns 0 rows (or all-NaN with how="left").
# The same warning appears when one input has no CRS. Reprojecting data with no CRS raises:
ValueError: Cannot transform naive geometries. Please set a crs on the object first.
```
### Cause
Spatial operations (join, overlay, clip, distance) require all inputs in the same CRS. When the CRS is missing or doesn't match, geopandas only warns; the operation runs on the raw coordinates and returns empty or wrong results. To make a script stop instead, add `warnings.filterwarnings("error", message="CRS mismatch", category=UserWarning)` or `assert gdf1.crs == gdf2.crs` before the operation.
### Fix
```python
# Check CRS of both inputs
print(gdf1.crs) # e.g., EPSG:4326
print(gdf2.crs) # e.g., EPSG:5070 or None
# If missing CRS, set it (don't reproject — set declares what system the data is already in)
gdf2 = gdf2.set_crs("EPSG:4326")
# If mismatched, reproject one to match the other
gdf2 = gdf2.to_crs(gdf1.crs)
# Now the spatial operation will work
result = gpd.sjoin(gdf1, gdf2)
```
### Prevention
Reproject all inputs to a common CRS at the start of the script:
```python
TARGET_CRS = "EPSG:5070"
gdf1 = gdf1.to_crs(TARGET_CRS)
gdf2 = gdf2.to_crs(TARGET_CRS)
```
---
## Invalid Geometries
### Symptom
- Overlay or spatial join returns empty or incomplete results
- `TopologyException: found non-noded intersection` error
- Unexpected geometry fragments after overlay
### Cause
Invalid geometries violate the Simple Features specification — self-intersections, unclosed rings, duplicate vertices, etc.
### Detection
```python
# Check which geometries are invalid
invalid_mask = ~gdf.geometry.is_valid
print(f"Invalid geometries: {invalid_mask.sum()} / {len(gdf)}")
# Inspect a sample
if invalid_mask.any():
from shapely.validation import explain_validity
for idx in gdf[invalid_mask].index[:5]:
print(f" Row {idx}: {explain_validity(gdf.loc[idx, 'geometry'])}")
```
### Fix
```python
# Option 1: make_valid() — preferred (Shapely 2.0+)
gdf["geometry"] = gdf.geometry.make_valid()
# Option 2: Buffer by zero — classic approach
gdf["geometry"] = gdf.geometry.buffer(0)
# Verify fix
assert gdf.geometry.is_valid.all(), "Some geometries still invalid after repair"
```
### When to Validate
- Before any overlay operation
- Before spatial joins with `contains` or `within` predicates
- After reading Shapefiles (the format is prone to validity issues)
- After any geometry transformation (simplification, projection, dissolve)
---
## Spatial Join Produces Wrong Row Count
### Symptom
After `gpd.sjoin()`, the result has more rows than the left input (unexpected duplicates) or fewer rows (unexpected drops).
### Cause: Many-to-Many Matches
A point on a polygon boundary may match multiple polygons (with `intersects`). Overlapping polygons cause a single feature to match several targets.
### Diagnosis
```python
print(f"Input rows: {len(left_gdf)}")
print(f"Result rows: {len(result)}")
print(f"Duplicated indices: {result.index.duplicated().sum()}")
print(f"Null join columns: {result['right_col'].isna().sum()}")
```
### Fix
```python
# Strategy 1: Use stricter predicate
result = gpd.sjoin(points, polygons, predicate="within") # Instead of "intersects"
# Caution: a point exactly on a shared boundary matches no polygon under "within" and leaves an inner join; compare row counts (or use how="left") after switching.
# Strategy 2: Drop duplicates (keep first match)
result = result[~result.index.duplicated(keep="first")]
# Strategy 3: Aggregate after join
result = result.groupby(result.index).agg({
"county_name": "first",
"population": "sum"
})
```
### Cause: Unmatched Features (Fewer Rows)
Default `how="inner"` drops features with no spatial match.
```python
# Use left join to keep all left features
result = gpd.sjoin(schools, counties, predicate="within", how="left")
# Unmatched schools will have NaN in county columns
```
---
## Memory Issues with Large Shapefiles
### Symptom
Out of memory when reading a large Shapefile, or very slow performance.
### Fixes
```python
# 1. Read only needed columns
gdf = gpd.read_file("huge.shp", columns=["GEOID", "NAME", "geometry"])
# 2. Read only a geographic subset
gdf = gpd.read_file("huge.shp", bbox=(-80, 35, -75, 40))
# 3. Use pyogrio with Arrow backend (lower memory)
gdf = gpd.read_file("huge.shp", engine="pyogrio", use_arrow=True)
# 4. Use GeoParquet instead (much faster reads, column selection)
# Convert once:
gdf_full = gpd.read_file("huge.shp")
gdf_full.to_parquet("huge.parquet")
# Then read efficiently:
gdf = gpd.read_parquet("huge.parquet", columns=["GEOID", "geometry"])
# 5. For visualization of millions of features, use lonboard instead of matplotlib
from lonboard import viz
m = viz(gdf)
```
---
## Shapely 1.x vs 2.x Differences
geopandas 1.x requires **Shapely 2.0+**, which introduced significant changes.
### Key Changes
| Feature | Shapely 1.x | Shapely 2.x |
|---------|-------------|-------------|
| Geometry creation | Objects are mutable | Objects are immutable |
| Operations | Method-based only | Vectorized ufuncs + methods |
| Performance | Slow (Python-level loops) | Fast (C-level vectorized) |
| Array operations | Required PyGEOS separately | Built-in (PyGEOS merged) |
| `numpy` interop | Limited | Full support |
### What Broke
```python
# Shapely 1.x (no longer works)
from shapely.ops import cascaded_union # Removed
cascaded_union(geom_list)
# Shapely 2.x equivalent
from shapely import union_all
union_all(geom_list)
# Or in geopandas:
gdf.geometry.union_all()
# Shapely 1.x
geom.is_empty # Was an attribute in 1.x, still works in 2.x
# The PyGEOS backend option is gone — Shapely 2.x IS the vectorized engine
# Delete any old environment variable:
# unset USE_PYGEOS (no longer needed)
```
### If You See Deprecation Warnings
Some Shapely 1.x functions were removed; others still work but have modern equivalents:
```python
# Removed (will raise ImportError)
from shapely.ops import cascaded_union # Use shapely.union_all() instead
# Legacy names — prefer the modern equivalents
from shapely.ops import unary_union # Still works without a warning in shapely 2.1.2; it calls shapely.union_all()
from shapely import union_all # Preferred vectorized equivalent
# In geopandas: gdf.geometry.union_all() (GeoSeries.unary_union is deprecated: geopandas 1.1.3 emits a DeprecationWarning)
# Repair pattern
geom.buffer(0) # Classic fix — still works
shapely.make_valid(geom) # Preferred in 2.x (more predictable)
```
---
## Coordinate Order Confusion (lon/lat vs lat/lon)
### The Problem
- **Mathematics/GIS convention:** x = longitude, y = latitude → `(lon, lat)` = `(-77.036, 38.901)`
- **Everyday convention:** "latitude and longitude" → people say `(38.901, -77.036)`
- **Google Maps URLs:** `@38.901,-77.036` (lat, lon)
- **GeoJSON spec:** `[longitude, latitude]` (lon, lat)
### Symptoms of Getting It Wrong
- Points plot in the ocean or in the wrong hemisphere
- Spatial joins return zero matches
- Data appears reflected across the equator or prime meridian
### How geopandas Expects Coordinates
```python
# geopandas follows the GIS convention: x = longitude, y = latitude
gpd.points_from_xy(
x=df["longitude"], # x-axis = longitude (horizontal)
y=df["latitude"] # y-axis = latitude (vertical)
)
# Shapely Point: Point(x, y) = Point(longitude, latitude)
from shapely.geometry import Point
capitol = Point(-77.009, 38.890) # (lon, lat)
```
### Quick Diagnostic
```python
# If total_bounds look wrong, coordinates may be swapped
print(gdf.total_bounds)
# Expected for US data: [-125, 24, -66, 50] (minx, miny, maxx, maxy)
# If you see [24, -125, 50, -66], lat/lon are swapped
# Fix: swap x and y
from shapely.ops import transform
gdf["geometry"] = gdf.geometry.map(lambda geom: transform(lambda x, y: (y, x), geom))
```
---
## dtype Warnings with Mixed Geometry Types
### Symptom
```
UserWarning: Geometry column does not contain geometry.
```
Or unexpected behavior when a GeoDataFrame contains mixed geometry types (e.g., both Point and MultiPoint, or Polygon and MultiPolygon).
### Fix
```python
# Check geometry types
print(gdf.geom_type.value_counts())
# Explode MultiGeometries to single parts
gdf = gdf.explode(index_parts=False)
# Or force to Multi type for consistency
from shapely.geometry import MultiPolygon
def to_multi(geom):
if geom.geom_type == "Polygon":
return MultiPolygon([geom])
return geom
gdf["geometry"] = gdf.geometry.map(to_multi)
```
---
## GeoDataFrame Loses CRS After Pandas Operations
### Symptom
After a pandas operation (concat, merge, groupby), `gdf.crs` is None.
### Cause
Some pandas operations return a plain DataFrame, losing the GeoDataFrame type and CRS.
### Fix
```python
# After pd.concat — re-wrap as GeoDataFrame
import pandas as pd
result = pd.concat([gdf1, gdf2])
result = gpd.GeoDataFrame(result, crs=gdf1.crs)
# After merge where geometry might be lost
result = gdf.merge(df, on="key")
if not isinstance(result, gpd.GeoDataFrame):
result = gpd.GeoDataFrame(result, geometry="geometry", crs=gdf.crs)
# After groupby — dissolve preserves geometry; manual groupby does not
# Use dissolve instead of groupby for spatial data:
result = gdf.dissolve(by="state", aggfunc="sum") # Preserves geometry + CRS
```
---
## Plotting Issues
### Map Appears Stretched or Distorted
```python
# Add aspect ratio correction
ax.set_aspect("equal")
# Or use a projection that minimizes distortion for your study area
gdf.to_crs(epsg=5070).plot()
```
### Legend Overlaps Map
```python
gdf.plot(
column="value",
legend=True,
legend_kwds={
"loc": "lower right",
"fontsize": 8,
"shrink": 0.6, # For continuous legends
"pad": 0.02
}
)
```
### Basemap Not Showing
```python
# Ensure data is in Web Mercator for basemap alignment
import contextily as ctx
ax = gdf.to_crs(epsg=3857).plot(alpha=0.5)
ctx.add_basemap(ax)
# Or pass CRS explicitly
ax = gdf.plot(alpha=0.5)
ctx.add_basemap(ax, crs=gdf.crs)
```
---
## geopolars: Not Production-Ready
geopolars (a potential geopandas-like interface backed by Polars) is experimental and not suitable for production use. Stick with geopandas for spatial operations and convert to/from Polars for non-spatial data manipulation:
```python
import polars as pl
import geopandas as gpd
# Polars → GeoDataFrame
df_polars = pl.read_parquet("data.parquet")
df_pandas = df_polars.to_pandas()
gdf = gpd.GeoDataFrame(df_pandas, geometry=gpd.points_from_xy(df_pandas.lon, df_pandas.lat), crs="EPSG:4326")
# GeoDataFrame → Polars (drop geometry, or convert to WKT/WKB)
df_polars = pl.from_pandas(gdf.drop(columns=["geometry"]))
```
---
## References and Further Reading
geopandas FAQ and migration guide. https://geopandas.org/en/stable/docs/user_guide.html
Shapely 2.0 migration guide. https://shapely.readthedocs.io/en/stable/migration.html
Tenkanen, H., Heikinheimo, V., and Whipp, D. (forthcoming). *Introduction to Python for Geographic Data Analysis*. https://pythongis.org/
Dorman, M., Graser, A., Nowosad, J., and Lovelace, R. (2025). *Geocomputation with Python*. https://py.geocompx.org/
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |