Skip to content

aalmada/Stationary / .agents/skills/dotnet-pipelines

dotnet-pipelines

Verified

Design, implement, debug, test, and review high-throughput byte-stream processing with System.IO.Pipelines. USE FOR: Pipe, PipeReader, PipeWriter, ReadResult, FlushResult, ReadOnlySequence, SequenceReader, buffer lifecycle, AdvanceTo, GetMemory, Advance, FlushAsync, stream and socket adapters, framing protocols, incremental parsing, backpressure, cancellation, and Pipes-to-Channels handoff. DO NOT USE FOR: item-level producer-consumer queues, durable/distributed messaging, ordinary small file I/O with no measured bottleneck, generic Stream APIs with no buffer-management concern, or ASP.NET middleware pipelines unrelated to System.IO.Pipelines.

.NET Pipelines

Use System.IO.Pipelines for high-throughput asynchronous byte streams where buffer ownership, partial reads, and copying cost matter. Prefer Stream APIs until profiling identifies a pipeline-shaped bottleneck or a component already exposes a pipe.

Workflow

  1. Confirm the workload is a byte stream with partial messages, large data, or a measured allocation/throughput need.
  2. Define framing, maximum message size, invalid-data policy, cancellation, completion ownership, and whether payload bytes must outlive a read iteration.
  3. Create a Pipe for an in-memory producer/consumer pair, or adapt an existing Stream with PipeReader.Create or PipeWriter.Create.
  4. Writer: obtain memory, write bytes, call Advance, then await FlushAsync; stop when the flush result is completed or canceled.
  5. Reader: await ReadAsync, parse only complete frames, and always call AdvanceTo(consumed, examined) in finally.
  6. Copy or decode a parsed payload before AdvanceTo if it outlives the current buffer; pipe memory becomes invalid after advancing.
  7. Complete the reader/writer exactly once from the component that owns its full loop; propagate faults through CompleteAsync(exception).
  8. Use a bounded Channel<T> only after parsing creates independently owned work items; do not enqueue pipe-backed sequences.
  9. Test fragmentation, coalesced frames, incomplete final data, cancellation, limits, completion, and buffer lifetime.

API Choice

NeedAPI
Create producer/consumer byte pipenew Pipe(options)
Adapt a stream as a reader/writerPipeReader.Create / PipeWriter.Create
Reserve and commit write memoryGetMemory or GetSpan, then Advance
Publish written bytes/backpressureFlushAsync
Read a byte sequenceReadAsync returning ReadResult
Parse segmented dataReadOnlySequence<byte> and SequenceReader<byte>
Release/read-more signalingAdvanceTo(consumed, examined)
Non-exceptional pending-operation stopCancelPendingRead / CancelPendingFlush
End ownership and surface faultsCompleteAsync(exception)

Invariants

  • There is exactly one outstanding read and one outstanding flush; do not call ReadAsync or FlushAsync concurrently on the same pipe end.
  • Every ReadAsync result receives exactly one AdvanceTo, including error and cancellation paths.
  • consumed releases bytes; examined tells the pipe where parsing stopped. Incorrect positions can cause replay, stalls, infinite buffering, OOM, or data loss.
  • ReadOnlySequence<byte> and its spans are valid only until the reader advances. Copy/decode data before dispatching it elsewhere.
  • A completed reader still exposes buffered bytes. Treat final incomplete frames as a protocol decision, usually invalid input.
  • Set protocol-level size limits; an incomplete oversized frame can retain memory until it exhausts the process.

Reference Files

FileLoad When
references/reader-parsing-and-lifetime.mdImplementing framing, AdvanceTo, ReadOnlySequence, SequenceReader, or parsing incomplete data
references/writer-flow-control-and-completion.mdWriting bytes, flush backpressure, pipe options, cancellation, completion, or faults
references/streams-sockets-and-ownership.mdAdapting streams/sockets, selecting ownership, or configuring leave-open/lifetime behavior
references/pipes-and-channels-boundary.mdCombining parsing throughput with bounded concurrent item processing
references/testing-performance-and-review.mdTesting, profiling, diagnosing memory pressure, or reviewing pipeline correctness
references/sources.mdRechecking API details and evaluating official or community guidance
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 321kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 321kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 321kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall