Orchestra-Research/AI-Research-SKILLs / 04-mechanistic-interpretability/nnsight
nnsight-remote-interpretability
VerifiedProvides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 12.8 KB |
| references/README.md | markdown | 1.9 KB |
| references/api.md | markdown | 7.0 KB |
| references/tutorials.md | markdown | 7.8 KB |
references/README.md
View on GitHub# nnsight Reference Documentation
This directory contains comprehensive reference materials for nnsight.
## Contents
- [api.md](api.md) - Complete API reference for LanguageModel, tracing, and proxy objects
- [tutorials.md](tutorials.md) - Step-by-step tutorials for local and remote interpretability
## Quick Links
- **Official Documentation**: https://nnsight.net/
- **GitHub Repository**: https://github.com/ndif-team/nnsight
- **NDIF (Remote Execution)**: https://ndif.us/
- **Community Forum**: https://discuss.ndif.us/
- **Paper**: https://arxiv.org/abs/2407.14561 (ICLR 2025)
## Installation
```bash
# Basic installation
pip install nnsight
# For vLLM support
pip install "nnsight[vllm]"
```
## Basic Usage
```python
from nnsight import LanguageModel
# Load model
model = LanguageModel("openai-community/gpt2", device_map="auto")
# Trace and access internals
with model.trace("The Eiffel Tower is in") as tracer:
# Access layer output
hidden = model.transformer.h[5].output[0].save()
# Modify activations
model.transformer.h[8].output[0][:] *= 0.5
# Get final output
logits = model.output.save()
# Access saved values outside context
print(hidden.shape)
```
## Key Concepts
### Tracing
The `trace()` context enables deferred execution - operations are recorded and executed together.
### Proxy Objects
Inside trace, module accesses return Proxies. Call `.save()` to retrieve values after execution.
### Remote Execution (NDIF)
Run the same code on massive models (70B+) without local GPUs:
```python
# Same code, just add remote=True
with model.trace("Hello", remote=True):
hidden = model.model.layers[40].output[0].save()
```
## NDIF Setup
1. Sign up at https://login.ndif.us/
2. Get API key
3. Set environment variable: `export NDIF_API_KEY=your_key`
## Available Remote Models
- Llama-3.1-8B, 70B, 405B
- DeepSeek-R1 models
- More at https://ndif.us/
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |