Skip to content

ValentinNikolaev/agent-skills / claude/skills/ai-detector

ai-detector

Verified

Assess AI-like prose (ai-detector, is-AI), including targeted phrase and structure audits, with improvement advice. A direct human request also runs ChatGPT's detector; calls from other workflows stay local. Not proof of authorship.

references/improvement-loop.md

View on GitHub
# Improving the detector and humanizer from feedback

## Contents

- [Choose the owner and authorized scope](#choose-the-owner-and-authorized-scope)
- [Apply the smallest supported correction](#apply-the-smallest-supported-correction)
- [Retain the evidence and test the change](#retain-the-evidence-and-test-the-change)
- [Promote only what the evidence supports](#promote-only-what-the-evidence-supports)

For previous runs of the pair, apply [the prose retrospective](prose-retrospective.md)
to select new evidence, link exact copies, diagnose ownership, and measure cost.
It reuses the shared retrospective helper and consent/reminder policy rather than
creating a second journal or another assessment team. The steps below remain the
prose-maintenance procedure. A service score alone never establishes a lesson.

## Choose the owner and authorized scope

Use current task and standing authorization to decide whether maintaining these
skills is in scope. An analysis request alone is not general permission to
modify installed software or unrelated skill repositories. When skill upkeep is
already authorized, proceed without another approval question.

- `ai-detector` owns invocation mode, local analysis, external submission,
  feedback interpretation, exact-input verification, and actionable findings.
- `humanizer` owns supported prose edits and preservation of the author's
  meaning and voice. Leave author persona and project-specific editorial rules
  in their owning workflow.

Locate the authoritative sources and read their repository instructions before
editing. Preserve dirty overlapping work. Do not patch a generated distribution
or managed installed cache as the source of truth. If the source or authority is
unavailable, return a concrete proposed patch and the limitation. Maintenance
permission is not permission to publish, release, install, or change another
workflow.

## Apply the smallest supported correction

Repair demonstrated workflow defects immediately within authorized scope. For
example, a submitted message that omitted the passage supports a verification
and recovery correction in the detector. It does not support a prose-style rule.

For textual feedback, compare the service's exact recommendation with the source
and voice constraints. Keep a good phrase when the proposed alternative changes
the actor, modality, scope or conversational job. Improve a specific clarity
problem without labelling the whole text AI-written.

Apply a narrow reusable instruction or bounded example when the reasoning is
supported. Record a broader untested pattern as a hypothesis, rather than
silently making it a mandatory rule. Do not add synonym lists, fixed rhythm,
forced informality or a new ban for each isolated service preference.

If the text itself is in the authorized editing scope, let `humanizer` make the
necessary edit. Reuse the owning workflow's existing advisors and exact-copy
assessment where required. Do not create another council just for a local
detector finding. Do not claim that maintaining a skill also revised every
text that previously used it.

## Retain the evidence and test the change

Maintain a reviewable chain for each actual improvement:

> observed defect -> owner and proposed correction -> exact candidate or skill
> patch -> verification -> retained limits and counterevidence

Use the repository's existing evaluation records where available. Keep only
authorized evidence. Record case ID, exact input/version or hash, local
findings, external literal response and session, valid/invalid submission state,
accepted or rejected recommendations, affected skill paths and test outcomes.
Keep historical failures and withdrawals intact. A later successful candidate
must not overwrite its failed baseline.

For skill-level comparison, freeze the cases before tuning and reserve an
independent context when feasible. A registered fixture or valid JSON is not a
successful writing evaluation. Verify the observable routing and preservation
decisions, not just the presence of preferred phrases in instructions.

Run the owning repository's checks and regenerate managed distributions. Reuse
the initial reviewers for any necessary follow-up. When external retesting is
authorized, verify the exact final candidate with a fresh neutral assessment.
Report checks actually run, failures, skipped work and residual uncertainty.

## Promote only what the evidence supports

Use statuses such as proposed, promising, confirmed and rejected for uncertain
editing operations. Preserve meaning/voice constraints, counterexamples and the
next validation step alongside each hypothesis. A local improvement may be
promising while the whole passage still misses the user's criterion.

Promote a broad editorial rule only after independent reuse supports its scope
and the applicable editorial review accepts it. A concrete workflow defect can
be fixed from a reproducible failure without waiting for a large writing corpus.
Do not delay a safe, supported correction solely to collect more scores.

Do not equate convergence among models, a clean in-sample result, or a detector
score with proof of authorship or universal skill effectiveness. Rerun frozen
cases on the final skill when claiming a skill-level result. Stop when the
authorized correction and its necessary verification are complete.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall