Skip to content

brycewang-stanford/Auto-Empirical-Research-Skills / skills/33-Galaxy-Dawn-claude-scholar/skills/results-analysis

results-analysis

Verified

This skill should be used when the user asks to "analyze experimental results", "run strict statistical analysis", "compare model performance", "generate scientific figures", "check significance", "do ablation analysis", or mentions interpreting experiment data with rigorous statistics and visualization. It focuses on strict analysis bundles, not Results-section prose.

examples/example-analysis-report.md

View on GitHub
# Strict Analysis Report

## Analysis Question
- Compare Full Model, Frozen Encoder, and Subject Adapter on cross-subject decoding.
- Determine whether the adapter closes the transfer gap without sacrificing stability.

## Data Inventory
- 3 model families
- 5 random seeds per family
- subject-level WER and CER
- training logs for convergence
- ablation outputs for adapter width

## Executive Summary
- Subject Adapter improves mean WER over Frozen Encoder by 3.8 absolute points.
- The improvement is statistically significant under paired testing after multiple-comparison correction.
- Full fine-tuning remains strongest overall, but Subject Adapter offers the best compute-performance tradeoff.
- Performance variance is lower than Frozen Encoder, suggesting a more stable transfer path.

## Main Findings

### 1. Main comparison
- Full fine-tuning: `31.4 ± 1.9` WER
- Subject Adapter: `33.2 ± 1.3` WER
- Frozen Encoder: `37.0 ± 2.1` WER

Observation:
- Subject Adapter consistently outperforms Frozen Encoder across all 5 seeds.

Interpretation:
- The transfer bottleneck is not purely feature reuse; a light adaptation head captures subject-specific alignment that freezing alone misses.

Implication:
- Future work should prioritize adapter variants before investing in heavier full-model tuning.

### 2. Stability
- Subject Adapter has the smallest seed variance among transfer-friendly methods.
- Convergence curves show less oscillation after epoch 8.

Interpretation:
- Adapter tuning is not only better on average, but easier to optimize.

### 3. Ablation
- Reducing adapter width from 256 to 64 hurts WER by 1.6 points.
- Increasing from 256 to 512 yields only marginal improvement.

Interpretation:
- Most of the benefit is captured at moderate width; scale-up is not the current bottleneck.

## Caveats
- Only 5 seeds; inference about tails remains limited.
- Subject count is modest, so subject-level heterogeneity may still be under-estimated.
- Training-time comparison depends on identical hardware assumptions.

## Recommended Next Move
- Promote this finding into a `results-report` note focused on adapter vs freezing.
- Run one robustness pass on held-out low-resource subjects.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall