Skip to content

aktonay/skillz / skills/python-testing/run-integration-tests

run-integration-tests

Verified

Run integration tests to verify that extension components work together correctly. Use this after modifying component interactions or event handling.

Run integration tests to verify that multiple components (managers, API, settings) work together correctly.

When to Use This Skill

  • After modifying how components communicate (events, state sharing)
  • After changing the API surface
  • After modifying managers or their interactions
  • When components seem out of sync (UI shows stale data, events not firing)

Quick Reference

ActionCommand
Run all integration testsnpm run compile && npm run compile-tests && npm run integration-test
Run specific testnpm run integration-test -- --grep "manager"
Debug in VS CodeDebug panel → "Integration Tests" → F5

How Integration Tests Work

Integration tests run in a real VS Code instance but focus on component interactions:

  • Does the API reflect manager state?
  • Do events fire when state changes?
  • Do different scopes return appropriate data?

They're faster than E2E (which test full workflows) but more thorough than smoke tests.

Workflow

Step 1: Compile and Run

npm run compile && npm run compile-tests && npm run integration-test

Step 2: Interpret Results

Pass:

  Integration: Environment Manager + API
    ✓ API reflects manager state after refresh
    ✓ Different scopes return appropriate environments
    ✓ Environment objects have consistent structure

  3 passing (25s)

Fail: Check error message and see Debugging section.

Debugging Failures

ErrorCauseFix
API not availableExtension activation failedCheck Debug Console
Event not firedEvent wiring issueCheck event registration
State mismatchComponents out of syncAdd logging, check update paths
TimeoutAsync operation stuckCheck for deadlocks

For detailed debugging: Debug panel → "Integration Tests" → F5

Adding New Integration Tests

Create files in src/test/integration/ with pattern *.integration.test.ts:

import * as assert from 'assert';
import * as vscode from 'vscode';
import { waitForCondition, TestEventHandler } from '../testUtils';
import { ENVS_EXTENSION_ID } from '../constants';

suite('Integration: [Component A] + [Component B]', function () {
    this.timeout(120_000);

    let api: ExtensionApi;

    suiteSetup(async function () {
        const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID);
        assert.ok(extension, 'Extension not found');
        if (!extension.isActive) await extension.activate();
        api = extension.exports;
    });

    test('[Interaction test]', async function () {
        // Test component interaction
    });
});

Test Files

FilePurpose
src/test/integration/envManagerApi.integration.test.tsManager + API tests
src/test/integration/index.tsTest runner entry point
src/test/testUtils.tsUtilities (waitForCondition, TestEventHandler)

Prerequisites

  • CI needs webpack build - Run npm run compile (webpack) before tests, not just npm run compile-tests (tsc)
  • Extension builds - Run npm run compile before tests

Notes

  • Integration tests are faster than E2E (30s-2min vs 1-3min)
  • Focus on testing component boundaries, not full user workflows
  • First run downloads VS Code (~100MB, cached in .vscode-test/)
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 319kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 319kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 319kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 196kUpdated Claude CodeCodexVerifiedInstall