nota-america/forgecat-agent-profiles / profiles/anthropics/knowledge-work-plugins/anthropics_knowledge-work-plugins_bio-research/for-claude/.claude/skills/scvi-tools
scvi-tools
VerifiedDeep learning for single-cell analysis using scvi-tools. This skill should be used when users need (1) data integration and batch correction with scVI/scANVI, (2) ATAC-seq analysis with PeakVI, (3) CITE-seq multi-modal analysis with totalVI, (4) multiome RNA+ATAC analysis with MultiVI, (5) spatial transcriptomics deconvolution with DestVI, (6) label transfer and reference mapping with scANVI/scArches, (7) RNA velocity with veloVI, or (8) any deep learning-based single-cell method. Triggers include mentions of scVI, scANVI, totalVI, PeakVI, MultiVI, DestVI, veloVI, sysVI, scArches, variational autoencoder, VAE, batch correction, data integration, multi-modal, CITE-seq, multiome, reference mapping, latent space.
| File | Kind | Size |
|---|---|---|
| LICENSE.txt | markdown | 10.0 KB |
| SKILL.md | skill md | 7.3 KB |
| references/atac_peakvi.md | markdown | 9.3 KB |
| references/batch_correction_sysvi.md | markdown | 10.2 KB |
| references/citeseq_totalvi.md | markdown | 10.1 KB |
| references/data_preparation.md | markdown | 6.2 KB |
| references/environment_setup.md | markdown | 6.3 KB |
| references/label_transfer.md | markdown | 10.2 KB |
| references/multiome_multivi.md | markdown | 8.9 KB |
| references/rna_velocity_velovi.md | markdown | 8.6 KB |
| references/scarches_mapping.md | markdown | 10.8 KB |
| references/scrna_integration.md | markdown | 9.6 KB |
| references/spatial_deconvolution.md | markdown | 11.2 KB |
| references/troubleshooting.md | markdown | 10.8 KB |
| scripts/cluster_embed.pyexecutable | script | 6.3 KB |
| scripts/differential_expression.pyexecutable | script | 6.7 KB |
| scripts/integrate_datasets.pyexecutable | script | 6.9 KB |
| scripts/model_utils.pyexecutable | script | 17.0 KB |
| scripts/prepare_data.pyexecutable | script | 5.1 KB |
| scripts/train_model.pyexecutable | script | 11.3 KB |
| scripts/transfer_labels.pyexecutable | script | 7.2 KB |
| scripts/validate_adata.pyexecutable | script | 11.9 KB |
references/troubleshooting.md
View on GitHub# Troubleshooting Guide for scvi-tools
This reference provides a consolidated guide for diagnosing and resolving common issues across all scvi-tools models.
## Quick Diagnosis
| Symptom | Likely Cause | Quick Fix |
|---------|--------------|-----------|
| "X should contain integers" | Normalized data in X | Use `layer="counts"` in setup |
| CUDA out of memory | GPU memory exhausted | Reduce `batch_size`, use smaller model |
| Training loss is NaN | Bad data or learning rate | Check for all-zero cells/genes |
| Batches not mixing | Too few shared features | Increase HVGs, check gene overlap |
| Over-correction | Too aggressive integration | Use scANVI with labels |
| Import error | Missing dependencies | `pip install scvi-tools[all]` |
## Data Format Issues
### Issue: CITE-seq protein data from Seurat is CLR-normalized
**Cause**: Seurat's `NormalizeData(normalization.method = "CLR")` transforms raw ADT counts. totalVI requires raw integer counts for protein data.
**Symptoms**:
- Protein values are not integers
- Protein values contain negative numbers
- Model training produces poor results
**Solution**:
```python
# Check if protein data is normalized
protein = adata.obsm["protein_expression"]
print(f"Min value: {protein.min()}") # Should be 0 if raw counts
print(f"Contains integers: {np.allclose(protein, protein.astype(int))}")
# If importing from Seurat, use the raw counts assay, not the normalized one
# In R/Seurat, export the RNA assay's counts slot, not the data slot
# GetAssayData(seurat_obj, assay = "ADT", slot = "counts")
```
### Issue: "layer not found" or "X should contain integers"
**Cause**: scvi-tools requires raw integer counts, not normalized data.
**Solution**:
```python
# Check if X contains integers
import numpy as np
print(f"X max: {adata.X.max()}")
print(f"Contains integers: {np.allclose(adata.X.data, adata.X.data.astype(int))}")
# If normalized, recover from raw
if hasattr(adata, 'raw') and adata.raw is not None:
adata = adata.raw.to_adata()
# Or use existing counts layer
adata.layers["counts"] = adata.X.copy()
scvi.model.SCVI.setup_anndata(adata, layer="counts")
```
### Issue: Sparse matrix errors
**Cause**: Incompatible sparse format or dense array expected.
**Solution**:
```python
from scipy.sparse import csr_matrix
# Convert to CSR format (most compatible)
if hasattr(adata.X, 'toarray'):
adata.X = csr_matrix(adata.X)
# Or convert to dense if small enough
if adata.n_obs * adata.n_vars < 1e8:
adata.X = adata.X.toarray()
```
### Issue: NaN or Inf values in data
**Cause**: Missing values or corrupted data.
**Solution**:
```python
import numpy as np
# Check for issues
X = adata.X.toarray() if hasattr(adata.X, 'toarray') else adata.X
print(f"NaN count: {np.isnan(X).sum()}")
print(f"Inf count: {np.isinf(X).sum()}")
print(f"Negative count: {(X < 0).sum()}")
# Replace NaN/Inf with 0
X = np.nan_to_num(X, nan=0, posinf=0, neginf=0)
X = np.clip(X, 0, None) # Ensure non-negative
adata.X = csr_matrix(X)
```
### Issue: batch_key or labels_key not found
**Cause**: Column name mismatch in adata.obs.
**Solution**:
```python
# List available columns
print(adata.obs.columns.tolist())
# Check for similar names
for col in adata.obs.columns:
if 'batch' in col.lower() or 'sample' in col.lower():
print(f"Potential batch column: {col}")
```
## GPU and Memory Issues
### Issue: CUDA out of memory
**Cause**: Model or batch doesn't fit in GPU memory.
**Solutions** (try in order):
```python
# 1. Reduce batch size
model.train(batch_size=64) # Default is 128
# 2. Use smaller model architecture
model = scvi.model.SCVI(
adata,
n_latent=10, # Default is 10-30
n_layers=1 # Default is 1-2
)
# 3. Subset to fewer genes
sc.pp.highly_variable_genes(adata, n_top_genes=1500)
adata = adata[:, adata.var['highly_variable']].copy()
# 4. Clear GPU cache between models
import torch
torch.cuda.empty_cache()
# 5. Use CPU if GPU is too small
model.train(accelerator="cpu")
```
### Issue: No GPU detected
**Cause**: CUDA not installed or version mismatch.
**Diagnosis**:
```python
import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA version: {torch.version.cuda}")
```
**Solution**:
```bash
# Check system CUDA
nvidia-smi
nvcc --version
# Reinstall PyTorch with matching CUDA
pip install torch --index-url https://download.pytorch.org/whl/cu118 # For CUDA 11.8
# Or
pip install torch --index-url https://download.pytorch.org/whl/cu121 # For CUDA 12.1
```
### Issue: Memory error with large datasets
**Cause**: Dataset too large for system RAM.
**Solutions**:
```python
# 1. Process in chunks (for very large data)
# Subsample for initial exploration
adata_sample = adata[np.random.choice(adata.n_obs, 50000, replace=False)].copy()
# 2. Use backed mode for AnnData
adata = sc.read_h5ad("large_data.h5ad", backed='r')
# 3. Reduce gene count aggressively
adata = adata[:, adata.var['highly_variable']].copy()
```
## Training Issues
### Issue: Training loss is NaN
**Cause**: Numerical instability, bad data, or learning rate issues.
**Solutions**:
```python
# 1. Check for problematic cells/genes
sc.pp.filter_cells(adata, min_genes=200)
sc.pp.filter_genes(adata, min_cells=3)
# 2. Remove cells with zero counts
adata = adata[adata.X.sum(axis=1) > 0].copy()
# 3. Use gradient clipping (built into scvi-tools)
model.train(max_epochs=200, early_stopping=True)
```
### Issue: Training doesn't converge
**Cause**: Insufficient epochs, poor hyperparameters, or data issues.
**Solutions**:
```python
# 1. Train longer
model.train(max_epochs=400)
# 2. Check training curves
import matplotlib.pyplot as plt
plt.plot(model.history['elbo_train'])
plt.plot(model.history['elbo_validation'])
plt.xlabel('Epoch')
plt.ylabel('ELBO')
plt.legend(['Train', 'Validation'])
# 3. Adjust model size for data size
# Small data (<10k cells): smaller model
model = scvi.model.SCVI(adata, n_latent=10, n_layers=1, dropout_rate=0.2)
# Large data (>100k cells): can use larger model
model = scvi.model.SCVI(adata, n_latent=30, n_layers=2)
```
### Issue: Overfitting (validation loss increases)
**Cause**: Model too complex or trained too long.
**Solutions**:
```python
# 1. Enable early stopping
model.train(early_stopping=True, early_stopping_patience=10)
# 2. Add regularization
model = scvi.model.SCVI(adata, dropout_rate=0.2)
# 3. Reduce model complexity
model = scvi.model.SCVI(adata, n_layers=1)
```
## Integration Issues
### Issue: Batches don't mix
**Cause**: Too few shared features, strong biological differences, or technical issues.
**Solutions**:
```python
# 1. Check gene overlap between batches
for batch in adata.obs['batch'].unique():
batch_genes = adata[adata.obs['batch'] == batch].var_names
print(f"{batch}: {len(batch_genes)} genes")
# 2. Use more HVGs
sc.pp.highly_variable_genes(adata, n_top_genes=4000, batch_key="batch")
# 3. Train longer
model.train(max_epochs=400)
# 4. Increase latent dimensions
model = scvi.model.SCVI(adata, n_latent=50)
```
### Issue: Over-correction (biological signal lost)
**Cause**: Model removes too much variation.
**Solutions**:
```python
# 1. Use scANVI with cell type labels
scvi.model.SCANVI.from_scvi_model(scvi_model, labels_key="cell_type")
# 2. Reduce model capacity
model = scvi.model.SCVI(adata, n_latent=10)
# 3. Use categorical covariates instead of batch_key
scvi.model.SCVI.setup_anndata(
adata,
layer="counts",
categorical_covariate_keys=["batch"] # Less aggressive than batch_key
)
```
### Issue: One batch dominates clusters
**Cause**: Unbalanced batch sizes or incomplete integration.
**Solutions**:
```python
# 1. Check batch distribution
print(adata.obs['batch'].value_counts())
# 2. Subsample to balance
from sklearn.utils import resample
balanced = []
min_size = adata.obs['batch'].value_counts().min()
for batch in adata.obs['batch'].unique():
batch_data = adata[adata.obs['batch'] == batch]
balanced.append(batch_data[np.random.choice(len(batch_data), min_size, replace=False)])
adata_balanced = sc.concat(balanced)
```
## Model-Specific Issues
### scANVI: Poor label transfer
**Solutions**:
```python
# 1. Check label distribution
print(adata.obs['cell_type'].value_counts())
# 2. Use Unknown for low-confidence cells
adata.obs.loc[adata.obs['prediction_score'] < 0.5, 'cell_type'] = 'Unknown'
# 3. Train scVI longer before scANVI
scvi_model.train(max_epochs=300)
scanvi_model = scvi.model.SCANVI.from_scvi_model(scvi_model, labels_key="cell_type")
scanvi_model.train(max_epochs=100)
```
### totalVI: Noisy protein signal
**Solutions**:
```python
# 1. Use denoised protein values
_, protein_denoised = model.get_normalized_expression(return_mean=True)
# 2. Check isotype controls
# Isotype controls should have low expression
for i, name in enumerate(adata.uns["protein_names"]):
if 'isotype' in name.lower():
print(f"{name}: mean={adata.obsm['protein_expression'][:, i].mean():.1f}")
```
### PeakVI: Poor clustering
**Solutions**:
```python
# 1. Use more variable peaks
from sklearn.feature_selection import VarianceThreshold
selector = VarianceThreshold(threshold=0.05)
adata = adata[:, selector.fit(adata.X).get_support()].copy()
# 2. Binarize data
adata.X = (adata.X > 0).astype(np.float32)
```
### MultiVI: Different cell counts between modalities
**Solutions**:
```python
# Ensure same cells in same order
common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
adata_rna = adata_rna[common_cells].copy()
adata_atac = adata_atac[common_cells].copy()
```
### DestVI: Poor deconvolution
**Solutions**:
```python
# 1. Check gene overlap
common_genes = adata_ref.var_names.intersection(adata_spatial.var_names)
print(f"Common genes: {len(common_genes)}") # Should be >1000
# 2. Use tissue-matched reference
# Reference should contain all cell types expected in spatial data
# 3. Check reference quality
print(adata_ref.obs['cell_type'].value_counts())
```
## Version Compatibility
### scvi-tools 1.x vs 0.x API changes
Key differences:
```python
# 0.x API
scvi.data.setup_anndata(adata, ...)
# 1.x API (current)
scvi.model.SCVI.setup_anndata(adata, ...)
```
### Check versions
```python
import scvi
import scanpy as sc
import anndata
import torch
print(f"scvi-tools: {scvi.__version__}")
print(f"scanpy: {sc.__version__}")
print(f"anndata: {anndata.__version__}")
print(f"torch: {torch.__version__}")
```
### Recommended versions (as of late 2024)
```
scvi-tools>=1.0.0
scanpy>=1.9.0
anndata>=0.9.0
torch>=2.0.0
```
## Getting Help
1. **Check documentation**: https://docs.scvi-tools.org/
2. **GitHub issues**: https://github.com/scverse/scvi-tools/issues
3. **Discourse forum**: https://discourse.scverse.org/
4. **Tutorials**: https://docs.scvi-tools.org/en/stable/tutorials/index.html
When reporting issues, include:
- scvi-tools version (`scvi.__version__`)
- Python version
- Full error traceback
- Minimal reproducible example
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |