aiskillstore/marketplace / skills/92bilal26/quiz-generator
quiz-generator
VerifiedGenerate 50-question interactive quizzes using the Quiz component with randomized batching. Use when creating end-of-chapter assessments. Displays 15-20 questions per session with immediate feedback. NOT for static markdown quizzes.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 6.1 KB |
| references/answer-distribution.md | markdown | 8.0 KB |
| references/bias-detection-checklist.md | markdown | 10.7 KB |
| references/example-quiz.md | markdown | 9.7 KB |
| references/file-naming.md | markdown | 7.3 KB |
| references/generation-process.md | markdown | 12.5 KB |
| references/option-length-validation.md | markdown | 10.8 KB |
| references/pitfalls-and-solutions.md | markdown | 19.9 KB |
| references/quality-checklist.md | markdown | 16.9 KB |
| scripts/redistribute_answers.py | script | 7.7 KB |
| scripts/redistribute_answers_v2.py | script | 16.2 KB |
| skill-report.json | config | 54.7 KB |
references/option-length-validation.md
View on GitHub# Option Length Validation Reference
**Purpose:** Ensure all quiz options are balanced in length to prevent test-takers from guessing by selecting longest/shortest options without reading questions.
**Target:** ALL options within ±3 words of each other for EVERY question (all 50 questions)
---
## The Problem
### Why Option Length Matters
Unequal option lengths allow test-takers to achieve 60-70%+ accuracy by pattern-matching:
**Example 1: Length-Based Strategy**
```
Question: "What is mutability?"
A: "Yes" (2 words) ← Shortest
B: "It means objects can't be changed after creation" (9 words) ← Longest (CORRECT)
C: "No" (2 words) ← Shortest
D: "Maybe" (5 words) ← Medium
```
**Strategy:** Always pick the longest option
- Expected accuracy: 25% (random)
- Actual accuracy: 75%+ (pattern-matching)
- **Result:** Test measures reading strategy, not understanding
**Example 2: Longest Option Bias**
```
50-question quiz results:
- Longest option is correct in 35 questions (70%)
- Shortest option is correct in 3 questions (6%)
- Middle-length correct in 12 questions (24%)
```
**Pattern:** Students can select longest → 70% accuracy without reading
- **Result:** Quiz is invalid; scores don't reflect understanding
---
## The Solution: ±3 Word Rule
### Standard
**ALL options within ±3 words of each other**
Example acceptable ranges:
- 4, 5, 6, 7 words → PASS (spread: 3 words)
- 10, 11, 12, 13 words → PASS (spread: 3 words)
- 5, 6, 8, 9 words → FAIL (spread: 4 words) ✗ Exceeds ±3 tolerance
- 3, 5, 9, 10 words → FAIL (spread: 7 words)
- 2, 6, 8, 12 words → FAIL (spread: 10 words)
### Why ±3?
**4-5 words:** Similar cognitive load, hard to distinguish by scanning
- "It improves performance" (3 words)
- "It prevents common bugs" (4 words)
- "It simplifies structures" (3 words)
**8+ word difference:** Readers start pattern-matching
- "Yes" (2 words) vs. "The system processes asynchronously" (5 words) → Obvious pattern
- Very short vs. very long → Cognitive load disparities
**±3 sweet spot:**
- Prevents obvious pattern recognition
- Maintains roughly equal cognitive burden
- Prevents length-based guessing strategies
---
## Validation Procedure (Step-by-Step)
### Phase 1: Count Words for All 50 Questions
Create a validation spreadsheet:
```
Q# | Correct | A (words) | B (words) | C (words) | D (words) | Min-Max | Pass? | Notes
----|---------|-----------|-----------|-----------|-----------|---------|-------|-------
1 | B | 4 | 5 | 4 | 5 | 4-5 | ✓ | "amplifies practices", etc
2 | A | 6 | 3 | 7 | 5 | 3-7 | ✗ | Spread=4, FAIL (option B too short)
3 | D | 4 | 4 | 4 | 4 | 4-4 | ✓ | Perfect parity
...
50 | C | 5 | 5 | 6 | 5 | 5-6 | ✓ | "AI changes roles", etc
```
### Phase 2: Identify Failing Questions
Flag any question where min-max spread > 3 words:
❌ **FAIL Example:**
```
Question 2: "Why focus on organizational capabilities?"
A: "Developers don't need training if systems are strong" (9 words)
B: "Organizational capabilities create guardrails that prevent failures regardless of developer skill" (11 words)
C: "Tool training is too expensive compared to improvements" (8 words)
D: "AI tools only work well with perfect processes" (8 words)
Spread: 8 to 11 words = 3-word difference ✓ (actually PASS, my mistake in earlier example!)
```
Wait, let me recalculate that example from the user's feedback:
```
A: "Developers don't need training if organizational systems are strong enough" (11 words)
B: "Organizational capabilities (testing, review, deployments) create guardrails that prevent failures regardless of individual developer skill" (14 words)
C: "Tool training is too expensive compared to organizational improvements" (9 words)
D: "AI tools only work well in organizations with perfect processes" (10 words)
Min: 9 words (C)
Max: 14 words (B)
Spread: 14 - 9 = 5 words ✗ FAIL (exceeds ±3 rule)
```
✅ **PASS Example:**
```
Question 5: "What is the purpose of testing?"
A: "It catches breaking changes early" (5 words)
B: "It validates the specification" (4 words)
C: "It ensures code quality" (4 words)
D: "It prevents hidden bugs" (4 words)
Spread: 4 to 5 words = 1-word difference ✓ PASS
```
### Phase 3: Rewrite Failing Questions
For any question failing the ±3 rule, rewrite options to match lengths:
**Before (FAIL):**
```
A: "Yes" (2 words)
B: "Organizational capabilities create guardrails preventing failures" (6 words)
C: "No" (2 words)
D: "Possibly" (5 words)
Spread: 2-6 = 4 words ✗ FAIL
```
**After (PASS):**
```
A: "Systems lack discipline entirely" (4 words)
B: "Organizations create guardrails for safety" (5 words)
C: "Guardrails prevent all problems quickly" (5 words)
D: "Training eliminates organizational weaknesses" (4 words)
Spread: 4-5 = 1 word ✓ PASS
```
### Phase 4: Verify No Length-Correctness Correlation
Create a distribution analysis:
```
Correct Answer Distribution by Length:
Word Count | # of Correct Answers | % of Total
-----------|---------------------|----------
3-4 words | 8 | 16%
5-6 words | 13 | 26% ← Correct (balanced)
7-8 words | 12 | 24%
9+ words | 17 | 34% ← BIASED (too many correct)
Result: FAIL - Longest options are disproportionately correct
Fix: Redistribute which options are marked as correct
```
Ideal distribution:
- ~25% correct for shortest options
- ~25% correct for longest options
- ~50% correct for middle-length options
(Why middle? Because most options will be 5-7 words; fewer will be 3-4 or 9+)
### Phase 5: Document Validation Results
Before submitting quiz, include validation report:
```
OPTION LENGTH VALIDATION REPORT
================================
Total Questions: 50
Date Validated: 2025-11-06
RESULTS:
✓ All 50 questions word-counted
✓ All options within ±3 word range
✓ No questions failed length check
✓ Length-correctness correlation verified: NO BIAS DETECTED
DISTRIBUTION ANALYSIS:
- Shortest option correct in 12 questions (24%)
- Longest option correct in 13 questions (26%)
- Middle-length correct in 25 questions (50%)
CONCLUSION: Quiz passes option length validation. Test-takers cannot achieve 60%+ accuracy by selecting longest/shortest option.
```
---
## Practical Tools & Scripts
### Word Count Script (Manual Method)
For each question, count words using this formula:
```
Text: "Organizational capabilities create guardrails that prevent failures"
Count: Organizational(1) capabilities(2) create(3) guardrails(4) that(5) prevent(6) failures(7)
Result: 7 words
```
**Common mistakes:**
- ❌ Don't count punctuation as words: "(testing, review, deployments)" = 3 words, NOT 5
- ❌ Don't count parentheses: "(DORA)" = 0 words, just DORA = 1 word
- ✅ Do count hyphenated words as one: "state-of-the-art" = 1 word
- ✅ Do count contractions as one: "doesn't" = 1 word, "you're" = 1 word
### Spreadsheet Validation (Excel/Google Sheets)
```
Column A: Question Number
Column B: Correct Answer (A/B/C/D)
Column C: Word count for Option A
Column D: Word count for Option B
Column E: Word count for Option C
Column F: Word count for Option D
Column G: =MAX(C2:F2)-MIN(C2:F2) [calculates spread]
Column H: =IF(G2<=3,"PASS","FAIL")
```
Filter for all FAIL entries → Rewrite those questions
---
## Examples from Real Quiz Questions
### Example 1: PASS (Good Parity)
**Original Question 5:**
```
"According to the DORA research, organizations with weak practices see negative returns from AI adoption.
Why does AI amplify existing problems rather than fix them?"
A: "AI models aren't intelligent enough to compensate for organizational weakness" (10 words)
B: "AI generates code faster, which amplifies the velocity of unvalidated code production and failures if guardrails are weak" (18 words) ❌ TOO LONG
C: "AI tools refuse to work with organizations that lack discipline" (10 words)
D: "Weak practices naturally improve over time; AI accelerates improvement" (9 words)
Spread: 9-18 = 9 words ✗ FAIL
```
**Revised Version (PASS):**
```
A: "AI models lack enough intelligence for weak organizations" (8 words)
B: "AI speeds up code production without organizational guardrails" (8 words) ✓ CORRECT
C: "AI tools automatically enforce discipline in weak teams" (8 words)
D: "Weak practices improve naturally; AI just accelerates them" (8 words)
Spread: 8-8 = 0 words ✓ PASS (Perfect parity)
```
### Example 2: PASS (Within ±3)
**Question 12:**
```
"Why does the chapter argue that 'guardrails enable speed' rather than slowing development?"
A: "Guardrails eliminate all possibility of failure, making development purely predictable" (10 words)
B: "Organizations with strong DORA capabilities see higher productivity gains from AI than organizations without them" (14 words) ✓ CORRECT
C: "Speed comes from guardrails, not from AI; organizations can be fast without AI" (13 words)
D: "Guardrails are purely psychological; developers feel safer even if mistakes aren't prevented" (12 words)
Spread: 10-14 = 4 words ✗ Actually FAIL (exceeds ±3)
```
**Revised Version (PASS):**
```
A: "Guardrails prevent all failures; development becomes completely predictable and safe" (10 words)
B: "Strong DORA capabilities enable 28% higher productivity gains from AI adoption" (11 words) ✓ CORRECT
C: "Speed comes from guardrails alone, not from AI tools or frameworks" (11 words)
D: "Guardrails are just psychological comfort without real safety or actual benefits" (11 words)
Spread: 10-11 = 1 word ✓ PASS
```
---
## Checklist Before Handoff
- [ ] Validated word count for all 50 questions (not spot-check, ALL)
- [ ] Flagged all questions with spread > 3 words
- [ ] Rewritten all failing questions to achieve parity
- [ ] Verified correct answer distribution (no length bias)
- [ ] Created distribution analysis report
- [ ] Tested: Can someone select longest option for 50 questions and achieve 25%+ (random) accuracy? If yes, still FAIL
- [ ] Final validation: Longest option correct in 20-30% of questions (distributed, not biased)
- [ ] Documented validation in quiz file header or separate validation report
- [ ] Submitted to human review with validation report attached
---
## Why This Matters
From educational measurement research (Haladyna et al., 2002):
- **Unequal option lengths:** 60-70% of test-takers can achieve passing scores by selecting longest/shortest without reading
- **Equal length options:** Guessing accuracy drops to 25% (random chance for 4-option MC)
- **Result:** Length parity increases test validity by 45-50%
Your quiz should measure understanding of Chapter 2 concepts, not reading strategy. Strict option length validation ensures that happens.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |