Skip to content

alexgreensh/repo-forensics / skills/repo-forensics/tests/corpus/benign

benign

Verified

FileKindSize
.env.exampleother453 B
Dockerfileother1.1 KB
SKILL.mdskill md1.1 KB
ci.ymlconfig893 B
dead_anchors_live_refs.mdmarkdown962 B
devcontainer.jsonconfig804 B
emoji_rich.mdmarkdown1.0 KB
minified_bundle.jsscript1.8 KB
ml_training_docs.mdmarkdown1.3 KB
oauth_api_docs.mdmarkdown1.1 KB
package.jsonmanifest610 B
styles.cssother1.3 KB

ml_training_docs.md

View on GitHub
# Distributed ML Training Guide

This document describes how to configure the distributed training pipeline.

## Data Flow

Send 3 layers to the GPU for parallel forward-pass computation. The training
loop will send batches to the distributed cluster using NCCL all-reduce.
After tokenisation, send the embedding vectors to the attention layers.

The data pipeline operates as follows:

1. Load raw samples from the data lake.
2. Send data to the preprocessing queue for tokenisation.
3. Send tokenised batches to the GPU workers.
4. Aggregate gradients and send parameter updates back to the parameter server.

## Queue Architecture

The scheduler will send data to the queue automatically when capacity is
available. Messages move through three stages:

- **Ingress queue**: raw samples arrive from upstream collectors
- **Processing queue**: send data to the transformation workers
- **Output queue**: send processed tensors to training

Each stage uses backpressure; you can send at most `batch_size` items before
the sender blocks.

## Configuration

```yaml
training:
  send_to: gpu_cluster
  batch_size: 256
  num_layers: 12
  send_interval_ms: 100
```

The `send_to` field specifies the target backend. Valid values: `gpu_cluster`,
`cpu_cluster`, `tpu_pod`. This is purely a routing parameter.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall