Skip to content

Liberty91LTD/cti-skills / skills/feedback-loops

feedback-loops

Verified

Feedback loop implementation for continuous CTI improvement. Consumer feedback, analyst retrospectives, source quality tracking.

Feedback Loops

Feedback is Phase 6 of the CTI Hyperloop. Without it, the cycle is open-ended and quality degrades over time.

Types of Feedback

1. Consumer Feedback

Intelligence consumers (stakeholders) assess the value of products they receive.

Collection points:

  • After every significant product delivery
  • During quarterly PIR reviews
  • During stakeholder briefings

Questions:

QuestionResponse Options
Was this intelligence useful for your decision-making?Very useful / Somewhat useful / Not useful
Was it delivered in time to act on?Yes / Partially / No
Was the format appropriate for your needs?Yes / Too technical / Too high-level / Wrong format
Was the confidence assessment helpful?Yes / Unclear / Not included
What should we prioritise next?Free text

Actions:

  • Useful + timely → Continue current approach, reinforce collection sources
  • Useful + late → Improve detection/triage speed, adjust collection frequency
  • Not useful → Review PIR alignment, stakeholder needs, product format
  • Format wrong → Adjust per stakeholder-management skill

2. Source Quality Tracking

Track the reliability of intelligence sources over time.

For each source used in products:

  • Was the information confirmed, partly confirmed, or disproven?
  • Did the source's Admiralty rating need adjustment?
  • Were there timeliness issues?

Actions:

  • Consistently confirmed → Upgrade reliability rating (e.g., C → B)
  • Inconsistent → Maintain or downgrade rating
  • Repeatedly disproven → Downgrade to D or E
  • Feed updates into source-assessment guidance

3. Analyst Retrospectives

After significant investigations or assessments:

Questions:

  • What analytical techniques were applied? Were they effective?
  • Were key assumptions validated or invalidated?
  • Were there intelligence gaps that could have been filled?
  • Were alternative hypotheses adequately considered?
  • What would you do differently?

Actions:

  • Update SOPs with lessons learned
  • Adjust technique selection guidance
  • Fill identified collection gaps
  • Update knowledge cells with validated/invalidated assumptions

4. Detection Effectiveness

For detection rules produced by the platform:

Metrics:

  • True positive rate (did the rule catch real threats?)
  • False positive rate (did it generate noise?)
  • Coverage (did it miss known instances?)

Actions:

  • High FP → Refine rule, add exclusions
  • Missed detections → Expand rule, add variants
  • No hits → Verify rule logic, check data source availability

Feedback Integration

All feedback feeds back into Phase 1 (Planning & Direction):

Consumer says "not useful" → Review and adjust PIRs
Source proves unreliable → Update Admiralty ratings, adjust collection
Analyst retrospective finds gap → Create new collection tasking
Detection rule has high FP → Refine and redeploy

Tracking

The orchestrator maintains feedback state:

  • Consumer feedback logged in stakeholder register
  • Source quality tracked in knowledge cell Sources & References
  • Retrospective outcomes logged in investigation workspace
  • Detection effectiveness tracked in detection rule metadata
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall