Skip to content

akiiwa-games/AgentCompass / skills/agent-compass

agent-compass

Flagged

Choose owner, model and effort, outbound authorization, advisor use, and continuation for delegated work. Reads local usage and effective account policy, profiles the task with the configured System-One scorer, then allocates a compatible worker. Use for independent worker or advisor work; ordinary local execution needs no routing.

FileKindSize
SKILL.mdskill md5.5 KB
agents/openai.yamlconfig427 B
calibration/README.mdmarkdown55.2 KB
calibration/arena_capture.pyscript14.0 KB
calibration/community.pyscript25.2 KB
calibration/five_axis.pyscript196.4 KB
calibration/measurements.pyscript68.4 KB
calibration/refresh.pyscript48.6 KB
calibration/series.pyscript10.2 KB
calibration/sources.pyscript24.1 KB
calibration/tiers.pyscript14.6 KB
grok-bot/.gitignoreother47 B
grok-bot/dependency_patch.pyscript4.5 KB
grok-bot/install_cli.pyscript3.6 KB
grok-bot/test-recovery.mjsscript8.6 KB
grok-bot/test_install_cli.pyscript4.1 KB
grok-bot/upstream.jsonconfig3.4 KB
references/accounts.jsonconfig155.9 KB
references/classifier-cases.jsonconfig21.5 KB
references/dispatch-lifecycle.mdmarkdown28.9 KB
references/dispatch-rules.mdmarkdown11.3 KB
references/effort-matching.mdmarkdown5.1 KB
references/fast-modes.mdmarkdown3.5 KB
references/five-axis-scores.jsonconfig3.1 MB
references/gemini-decisions-api.mdmarkdown2.9 KB
references/grok-bot.mdmarkdown13.0 KB
references/herdr.mdmarkdown5.1 KB
references/jev-api.mdmarkdown1.6 KB
references/level-3-conversion.jsonconfig2.4 KB
references/model-capabilities.jsonconfig3.6 KB
references/model-capabilities.mdmarkdown14.6 KB
references/parent-address-gate.mdmarkdown10.1 KB
references/plan-templates.jsonconfig13.2 KB
references/provider-discovery.mdmarkdown11.5 KB
references/provider-usage.mdmarkdown4.1 KB
references/public-data.jsonconfig816 B
references/public-measurements.jsonconfig1.3 MB
references/roll-reference.mdmarkdown26.7 KB
references/router-internals.mdmarkdown93.7 KB
references/task-classifier.mdmarkdown6.1 KB
references/user-scores.jsonconfig6.8 KB
references/work-kinds.mdmarkdown4.7 KB
references/worker-communication.mdmarkdown2.1 KB
scripts/account_fleet.pyscript32.9 KB
scripts/account_state.pyscript22.2 KB
scripts/accounts_override.pyscript14.8 KB
scripts/allocator.pyscript19.8 KB
scripts/antigravity_plan.pyscript18.7 KB
scripts/antigravity_telemetry.pyscript5.1 KB
scripts/auto_bundles.pyscript31.3 KB
scripts/axis_scores.pyscript40.3 KB
scripts/caller_input.pyscript3.0 KB
scripts/candidate_eval.pyscript54.0 KB
scripts/candidate_ranking.pyscript9.5 KB
scripts/capability_catalog.pyscript8.0 KB
scripts/check_router_contract.pyscript2.8 KB
scripts/conftest.pyscript2.5 KB
scripts/cwd_policy.pyscript1.5 KB
scripts/data_age.pyscript1.5 KB
scripts/decision_ledger.pyscript18.9 KB
scripts/demo_plan.pyscript6.3 KB
scripts/dispatch_fallback.pyscript40.3 KB
scripts/dispatch_flow.pyscript58.7 KB
scripts/dispatch_ground_truth.pyscript43.5 KB
scripts/dispatch_registry.pyscript113.7 KB
scripts/entry.pyscript1.7 KB
scripts/env_vars.pyscript580 B
scripts/fast_modes.pyscript2.5 KB
scripts/fixtures/grok-api-responses.jsonconfig907 B
scripts/fixtures/reference-accounts.jsonconfig26.9 KB
scripts/grok_bot_paths.pyscript1005 B
scripts/host_herdr.pyscript16.8 KB
scripts/host_placement.pyscript9.4 KB
scripts/host_policy.pyscript10.1 KB
scripts/launch_failures.pyscript5.9 KB
scripts/launch_values.pyscript12.3 KB
scripts/ledger_snapshot.pyscript11.1 KB
scripts/live_models.pyscript21.1 KB
scripts/model_latch.pyscript7.6 KB
scripts/model_resolution.pyscript18.0 KB
scripts/override_store.pyscript2.2 KB
scripts/plan_state_residue.pyscript5.0 KB
scripts/provider_inventory.pyscript128.2 KB
scripts/provider_retry.pyscript34.6 KB
scripts/public_data.pyscript59.2 KB
scripts/quota_economy.pyscript38.0 KB
scripts/quota_exhaustion.pyscript5.9 KB
scripts/quota_overrides.pyscript25.6 KB
scripts/quota_pools.pyscript29.8 KB
scripts/reference_provenance.pyscript32.3 KB
scripts/released_names.pyscript3.8 KB
scripts/roll.pyscript106.3 KB
scripts/route_judge.pyscript83.4 KB
scripts/route_simulation.pyscript26.3 KB
scripts/router_test_support.pyscript6.7 KB
scripts/run_grok_bot.pyscript18.8 KB
scripts/scorer_client.pyscript55.2 KB
scripts/stage_help.pyscript5.8 KB
scripts/state_upgrade.pyscript24.0 KB
scripts/subscription_cli.pyscript3.6 KB
scripts/task_intake.pyscript33.6 KB
scripts/test_account_identity.pyscript7.1 KB
scripts/test_account_state.pyscript63.3 KB
scripts/test_accounts_override.pyscript30.1 KB
scripts/test_allocator.pyscript94.2 KB
scripts/test_antigravity_plan.pyscript13.2 KB
scripts/test_applicable_quota.pyscript9.1 KB
scripts/test_auto_bundles.pyscript10.8 KB
scripts/test_billing_routing_policy.pyscript9.9 KB
scripts/test_cadence_count.pyscript32.2 KB
scripts/test_calibration_units.pyscript27.0 KB
scripts/test_capability_catalog.pyscript5.0 KB
scripts/test_decision_ledger.pyscript8.5 KB
scripts/test_demo_plan.pyscript3.4 KB
scripts/test_derived_state.pyscript21.6 KB
scripts/test_dispatch.pyscript322.3 KB
scripts/test_dispatch_answers.pyscript14.8 KB
scripts/test_dispatch_fallback.pyscript53.2 KB
scripts/test_entry.pyscript1.6 KB
scripts/test_fast_modes.pyscript9.9 KB
scripts/test_five_axis.pyscript290.1 KB
scripts/test_five_axis_economy.pyscript105.0 KB
scripts/test_fleet_parity.pyscript15.6 KB
scripts/test_grok_bot.pyscript47.7 KB
scripts/test_ground_truth_recording.pyscript68.8 KB
scripts/test_herdr.pyscript14.1 KB
scripts/test_host_policy.pyscript24.6 KB
scripts/test_launch_values.pyscript8.0 KB
scripts/test_metered_usd.pyscript60.4 KB
scripts/test_model_resolution.pyscript9.2 KB
scripts/test_named_route_facts.pyscript28.6 KB
scripts/test_native_hosts.pyscript6.0 KB
scripts/test_new_user_fleet.pyscript3.3 KB
scripts/test_no_hardcode_classification.pyscript75.1 KB
scripts/test_no_legacy_scoring.pyscript6.5 KB
scripts/test_override_store.pyscript2.3 KB
scripts/test_plan_says_why.pyscript44.9 KB
scripts/test_provider_inventory.pyscript83.2 KB
scripts/test_provider_names.pyscript126.0 KB
scripts/test_provider_pools.pyscript14.2 KB
scripts/test_public_data.pyscript93.5 KB
scripts/test_quota_exhaustion.pyscript5.6 KB
scripts/test_rate_limit_windows.pyscript6.3 KB
scripts/test_record_says_why.pyscript26.5 KB
scripts/test_reference_provenance.pyscript16.7 KB
scripts/test_refresh_entry.pyscript110.7 KB
scripts/test_released_names.pyscript4.1 KB
scripts/test_reset_priority.pyscript4.8 KB
scripts/test_roll_answers.pyscript44.7 KB
scripts/test_roll_notifications.pyscript10.4 KB
scripts/test_route_judge.pyscript99.8 KB
scripts/test_route_simulation.pyscript7.0 KB
scripts/test_routing_parity.pyscript32.3 KB
scripts/test_routing_review.pyscript25.1 KB
scripts/test_setup_discovery_parity.pyscript9.5 KB
scripts/test_stage_help.pyscript2.0 KB
scripts/test_state_upgrade.pyscript13.8 KB
scripts/test_turn_refusal.pyscript16.7 KB
scripts/test_usage_cache.pyscript4.9 KB
scripts/test_usage_collector.pyscript21.4 KB
scripts/test_usage_core_claude_accounts.pyscript1.8 KB
scripts/test_usage_core_grok_cli_billing.pyscript15.6 KB
scripts/test_usage_core_magpie.pyscript5.4 KB
scripts/test_usage_core_magpie_pool.pyscript10.8 KB
scripts/test_usage_core_safety.pyscript24.4 KB
scripts/test_user_scores.pyscript102.3 KB
scripts/test_venice_usage.pyscript1.3 KB
scripts/test_window_scope.pyscript5.5 KB
scripts/turn_refusal.pyscript10.7 KB
scripts/usage_cache.pyscript4.7 KB
scripts/usage_collector.pyscript7.4 KB
scripts/usage_core.pyscript142.0 KB
scripts/usage_magpie.pyscript10.6 KB
scripts/usage_reading.pyscript17.1 KB
scripts/user_scores.pyscript72.8 KB
scripts/venice_usage.pyscript2.6 KB
scripts/window_scope.pyscript3.2 KB
scripts/worker_context.pyscript8.9 KB
scripts/worker_sessions.pyscript10.9 KB

scripts/test_calibration_units.py

View on GitHub
"""Direct tests of the calibration helpers the five-axis scores rest on: series.py (same-series siblings and the base a
converted model takes), tiers.py (the tier is read off arena.ai Overall and has one use, estimating a missing type score)
and the usd series conversion in five_axis.py."""
from pathlib import Path
import sys
import unittest

sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'calibration'))

import five_axis as fa  # noqa: E402
import measurements  # noqa: E402
import series  # noqa: E402
import sources  # noqa: E402
import tiers  # noqa: E402

# accounts.json's declarations the build reads (sources.effort_readings, five_axis.estimation_order)
READINGS = {'effort_readings': {'labels': {'non-reasoning': 'none', 'reasoning': 'default'}, 'reads': {'default': 'high'}},
            'estimation_order': {'order': list(fa.ESTIMATION_ORDER), 'sibling_first_up_to_distance': 1,
                                 'speed_cross_line_before_fit': True, 'evidence': 'synthetic'}}


# declared family rows (series.identity) for the luna, astra and claude rows of the fixtures
GPT_FAMILIES = {'rows': [{'pattern': '^gpt-(?:6|5\\.6).*luna', 'models': [], 'maker': 'openai', 'product': 'openai', 'line': 'luna', 'size': 'full'},
                         {'pattern': '^gpt', 'models': [], 'maker': 'openai', 'product': 'openai', 'line': 'astra', 'size': 'full'},
                         {'pattern': '^claude', 'models': [], 'maker': 'anthropic', 'product': 'anthropic', 'line': 'claude', 'size': 'full'}]}


class SeriesTests(unittest.TestCase):
    ACC = {'models': {m: {} for m in ('gpt-6-luna', 'gpt-5.6-luna', 'gpt-6-astra', 'dot', 'claude-opus-5-5')},
           'aliases': {}, 'families': GPT_FAMILIES,
           'superseded': {'gpt-5.6-luna': {'by': 'gpt-6-luna'}},
           'same_model': {'dot': {'model': 'gpt-6-astra', 'carries': ['creativity'], 'url': 'https://example.test/dot'}}}

    def test_same_model_returns_only_the_declarations_that_are_objects(self):
        self.assertEqual(set(series.same_model(self.ACC)), {'dot'})
        self.assertEqual(series.same_model({'same_model': {'dot': 'gpt-6-astra'}}), {})
        self.assertEqual(series.same_model({}), {})

    def test_a_same_model_row_takes_its_models_identity_and_lines_split_speed_relatives(self):
        """Rule (series.identity, groups): a `same_model` row is its declared model (its maker, line and size); ability
        relatives span the maker's lines, speed relatives do not; another maker is never a relative."""
        line, ability = series.groups(self.ACC, 'line'), series.groups(self.ACC, 'ability')
        self.assertEqual(line['gpt-6-luna'], ['gpt-5.6-luna'])
        self.assertEqual(line['dot'], ['gpt-6-astra'])
        self.assertEqual(ability['gpt-6-luna'], ['dot', 'gpt-5.6-luna', 'gpt-6-astra'])
        self.assertNotIn('claude-opus-5-5', ability['dot'] + ability['gpt-6-luna'])

    def test_a_same_model_row_to_a_row_that_is_not_scored_joins_nothing(self):
        acc = dict(self.ACC, same_model={'dot': {'model': 'not-a-row'}})
        self.assertNotIn('not-a-row', series.groups(acc)['dot'])

    def test_boards_keep_the_first_numeric_reading_per_source_on_the_component_dimensions(self):
        meas = {'axes': {'creativity': {}, 'coding': {}},
                'models': {'m': {'creativity': [{'source': 'a.x', 'value': 0.5}, {'source': 'a.x', 'value': 0.9},
                                                {'source': 'a.bad', 'value': True}, {'source': 'a.str', 'value': 'high'}],
                                 'coding': [{'source': 'b.y', 'value': 0.3}],
                                 'other_field': [{'source': 'c.z', 'value': 1.0}]},
                           'n': {}}}
        self.assertEqual(series.boards(meas), {'m': {'a.x': 0.5, 'b.y': 0.3}, 'n': {}})
        self.assertEqual(series.boards({}), {})

    def test_nearest_orders_by_the_declared_model_then_version_distance_then_size(self):
        """Rule (series.py, round 5 review D): the declared same_model first, then the declared version distance, then
        size equality; a candidate without a declared version ranks after every declared distance."""
        same = {'t': {'model': 'declared'}}
        versions = {'t': 3.0, 'declared': 1.0, 'v2': 2.0, 'v2-mini': 2.0, 'v1': 1.0, 'v3': 3.0}
        series_rows = ['declared', 'v2', 'v2-mini', 'v1', 'v3', 'blank']
        small = {'v2-mini'}
        order = series.nearest('t', ['v1', 'blank', 'v2-mini', 'v2', 'declared', 'v3'], series_rows, versions, small, same)
        self.assertEqual(order, [(0, ['declared']), (0, ['v3']), (1, ['v2']), (1, ['v2-mini']), (2, ['v1']),
                                 (None, ['blank'])])
        # without the declaration the declared model is one more relative at its version distance
        self.assertEqual(series.nearest('t', ['declared', 'v1'], series_rows, versions, small)[0], (2, ['declared', 'v1']))

    def test_tied_relatives_form_one_group_and_no_name_breaks_the_tie(self):
        """Rule (series.py): relatives at one distance and one size are one group (the caller takes their median);
        their names never order them."""
        versions = {'t': 2.0, 'a': 1.0, 'b': 3.0}
        for candidates in (['a', 'b'], ['b', 'a']):
            self.assertEqual(series.nearest('t', candidates, ['a', 'b'], versions, set()), [(1, ['a', 'b'])])

    def test_a_row_that_is_not_the_nearest_relative_changes_no_estimate(self):
        """Stability (round 5 review D): adding a row that is not the nearest relative of X by the declared distance
        leaves X's nearest group as it was."""
        versions = {'x': 5.0, 'near': 4.0, 'old': 2.0}
        before = series.nearest('x', ['near', 'old'], ['near', 'old'], versions, set())[0]
        for added, version in (('older', 1.0), ('zzz', 3.0), ('aaa', 3.0)):
            v = {**versions, added: version}
            rows = ['near', 'old', added]
            self.assertEqual(series.nearest('x', ['near', 'old', added], rows, v, set())[0], before, added)

    def test_the_version_distance_counts_distinct_declared_versions_between_the_two(self):
        versions = {'t': 3.0, 'a': 3.0, 'b': 2.0, 'c': 2.0, 'd': 1.0}
        rows = ['a', 'b', 'c', 'd', 'e']
        self.assertEqual([series.distance('t', x, rows, versions) for x in rows], [0, 1, 1, 2, None])


class PooledAndInterpolationTests(unittest.TestCase):
    def test_pooled_averages_ties_and_pools_an_inversion(self):
        xs, ys = tiers.pooled([1, 2, 2, 3, 4], [1.0, 2.0, 4.0, 2.0, 5.0])
        self.assertEqual(xs, [1.0, 2.0, 3.0, 4.0])
        self.assertAlmostEqual(ys[1], 8 / 3)            # x=2 averages 3.0 (two readings), x=3 reads 2.0: an inversion, pooled by weight
        self.assertEqual(ys[1], ys[2])
        self.assertEqual((ys[0], ys[3]), (1.0, 5.0))
        self.assertEqual(ys, sorted(ys))

    def test_interpolation_runs_between_points_and_by_the_end_to_end_slope_beyond_them(self):
        f = tiers.Interpolation([10, 20, 40], [2.0, 3.0, 5.0])
        self.assertAlmostEqual(f(15), 2.5)
        self.assertAlmostEqual(f(30), 4.0)
        slope = (5.0 - 2.0) / 30
        self.assertAlmostEqual(f.slope, slope)
        self.assertAlmostEqual(f(50), 5.0 + slope * 10)
        self.assertAlmostEqual(f(5), 2.0 - slope * 5)

    def test_optional_floor_and_one_point_is_no_map(self):
        unfloored = tiers.Interpolation([10, 20], [-0.5, 0.5])
        self.assertAlmostEqual(unfloored(0), -1.5)
        floored = tiers.Interpolation([10, 20], [1.2, 2.0], floor=1.0)
        self.assertEqual(floored(-1000), 1.0)
        with self.assertRaises(ValueError):
            tiers.Interpolation([5, 5], [2.0, 2.0])(5)

    def test_neighbours_name_the_two_points_a_reading_lies_between_or_the_end_segment(self):
        f = tiers.Interpolation([10, 20, 40], [2.0, 3.0, 5.0], ['a', 'b', 'c'])
        low, high = f.neighbours(30)
        self.assertEqual((low['models'], high['models']), (['b'], ['c']))
        low, high = f.neighbours(99)
        self.assertEqual((low['models'], high['models']), (['b'], ['c']))        # beyond the top: the end segment
        low, high = f.neighbours(1)
        self.assertEqual((low['models'], high['models']), (['a'], ['b']))

    def test_spread_is_the_rms_around_the_map_in_sample_and_leaving_each_model_out(self):
        """Rule (tiers.Interpolation.spread): residual_sd is the root mean square of the shared models' tiers around the
        map, loo_rmse the same with each model left out of the map that predicts it."""
        f = tiers.Interpolation([10, 20, 30, 40], [1.0, 3.0, 2.0, 4.0])
        res = [y - f(x) for x, y in f.raw]
        loo = []
        for i, (x, y) in enumerate(f.raw):
            rest = f.raw[:i] + f.raw[i + 1:]
            loo.append(y - tiers.Interpolation([p[0] for p in rest], [p[1] for p in rest])(x))
        rms = lambda v: round(sum(e * e for e in v) / len(v), 12) ** 0.5     # noqa: E731
        got = f.spread()
        self.assertAlmostEqual(got['residual_sd'], rms(res), places=4)
        self.assertAlmostEqual(got['loo_rmse'], rms(loo), places=4)
        self.assertGreater(got['loo_rmse'], got['residual_sd'])
        self.assertEqual(tiers.Interpolation([10, 20], [1.0, 2.0]).spread()['loo_rmse'], None)    # one point left: no map

    def test_norm_keeps_letters_and_digits_only(self):
        self.assertEqual(tiers.norm('Claude Opus-5.5 (high)'), 'claudeopus55high')


class FitterTests(unittest.TestCase):
    @staticmethod
    def board():
        """An arena Overall @High board centered on Claude Opus 5.5, with distinct Elo values."""
        return {
            'Claude Opus 5.5': (1500.0, 1495.0, 1505.0, 'Claude Opus 5.5 (High)', '2026-11-09'),
            'GPT-6.1 Sol': (1480.0, 1475.0, 1485.0, 'GPT-6.1 Sol (High)', '2026-11-09'),
            'Claude Sonnet 5.5': (1470.0, 1465.0, 1475.0, 'Claude Sonnet 5.5 (High)', '2026-11-09'),
            'GPT-6 Astra': (1460.0, 1455.0, 1465.0, 'GPT-6 Astra (High)', '2026-11-09'),
            'Grok 4.7': (1440.0, 1435.0, 1445.0, 'Grok 4.7 (High)', '2026-11-09'),
        }

    def fitter(self):
        f = tiers.Fitter.__new__(tiers.Fitter)
        f.aa_read = '2026-11-09'
        f.overall = self.board()
        f.overall.update({
            'Model A': (1410.0, 1405.0, 1415.0, 'Model A', '2026-11-09'),
            'Model B': (1450.0, 1445.0, 1455.0, 'Model B', '2026-11-09'),
            'Model C': (1490.0, 1485.0, 1495.0, 'Model C', '2026-11-09'),
            'Model D': (1520.0, 1515.0, 1525.0, 'Model D', '2026-11-09'),
        })
        f.anchor_name = 'Claude Opus 5.5'
        f.anchor_raw, f.sd = f.overall_scale(f.overall, f.anchor_name)
        f.aa = {'Model A': 40.0, 'Model B': 50.0, 'Model C': 60.0, 'Model D': 70.0, 'Only AA': 55.0}
        f.maps = {'aa.index': f.board_map(f.aa)}
        return f

    def test_arena_overall_is_centered_on_opus_at_high_with_no_hand_set_anchor_tiers(self):
        self.assertFalse(hasattr(tiers, 'ANCHOR_TIERS'))
        f = self.fitter()
        self.assertEqual(f.arena_overall_tier('Claude Opus 5.5'), 0.0)
        # Distinct Elo rows keep strictly distinct continuous standard-score tiers (never flattened into a PAV plateau)
        ordered = ['Grok 4.7', 'GPT-6 Astra', 'Claude Sonnet 5.5', 'GPT-6.1 Sol', 'Claude Opus 5.5', 'Model D']
        vals = [f.arena_overall_tier(name) for name in ordered]
        self.assertEqual(vals, sorted(vals))
        self.assertEqual(len(set(round(v, 6) for v in vals)), len(vals))
        self.assertIsNone(f.arena_overall_tier('Unlisted'))

    def test_arena_high_rows_rejects_max_and_xhigh_substitutes(self):
        rows = [
            (1510.0, 1500.0, 1520.0, 'GPT-6 Sol (Max)', '2026-11-09'),
            (1495.0, 1490.0, 1500.0, 'Claude Opus 5.5 (High)', '2026-11-09'),
            (1525.0, 1520.0, 1530.0, 'Claude Opus 5.5 (Max)', '2026-11-09'),
            (1430.0, 1425.0, 1435.0, 'Single Config Model', '2026-11-09'),
        ]
        picked = tiers.arena_high_rows(rows, sources.effort_readings(READINGS))
        self.assertNotIn('GPT-6 Sol', picked)
        self.assertEqual(picked['Claude Opus 5.5'][0], 1495.0)
        self.assertEqual(picked['Single Config Model'][0], 1430.0)

    def test_a_board_map_is_the_listed_models_common_with_arena_overall_and_needs_three(self):
        f = self.fitter()
        mapping = f.maps['aa.index']
        self.assertEqual(mapping.n, 4)                          # 'Only AA' is not on arena Overall
        self.assertIsNone(f.board_map({'Model A': 40.0, 'Model B': 50.0}))

    def test_a_model_off_arena_takes_its_reading_from_the_other_overall_board_with_its_neighbours(self):
        f = self.fitter()
        out = f.other_overall({'aa.index': 'Only AA'})
        reading = out['aa.index']
        self.assertEqual((reading['listed_as'], reading['value'], reading['n']), ('Only AA', 55.0, 4))
        self.assertEqual((reading['neighbours']['below']['models'], reading['neighbours']['above']['models']),
                         (['Model B'], ['Model C']))
        self.assertAlmostEqual(reading['tier'], (f.arena_overall_tier('Model B') + f.arena_overall_tier('Model C')) / 2, places=3)
        self.assertEqual(reading['as_of'], '2026-11-09')
        self.assertEqual(f.other_overall({}), {})
        self.assertEqual(f.other_overall({'aa.index': 'Unlisted'}), {})


class UsdConversionTests(unittest.TestCase):
    PRICE = staticmethod(lambda i, o: {'value': {'input': i, 'output': o}})

    def test_list_price_ratio_is_the_ratio_of_the_mix_weighted_costs(self):
        mix = {'input': 0.5, 'output': 0.5}
        self.assertAlmostEqual(fa.list_price_ratio(self.PRICE(2.0, 10.0), self.PRICE(1.0, 5.0), mix), 2.0)
        self.assertAlmostEqual(fa.list_price_ratio(self.PRICE(1.0, 5.0), self.PRICE(2.0, 6.0), mix), 0.75)

    def test_list_price_ratio_is_none_without_a_mix_a_price_or_a_component(self):
        mix = {'input': 0.5, 'output': 0.5}
        self.assertIsNone(fa.list_price_ratio(self.PRICE(2.0, 10.0), self.PRICE(1.0, 5.0), {}))
        self.assertIsNone(fa.list_price_ratio(None, self.PRICE(1.0, 5.0), mix))
        self.assertIsNone(fa.list_price_ratio(self.PRICE(2.0, 10.0), None, mix))
        self.assertIsNone(fa.list_price_ratio({'value': {'input': 2.0}}, self.PRICE(1.0, 5.0), mix))
        self.assertIsNone(fa.list_price_ratio(self.PRICE(2.0, 10.0), self.PRICE(0.0, 0.0), mix))

    def test_a_model_without_a_usd_takes_the_nearest_siblings_times_the_price_ratio_and_cites_it(self):
        def entry(i, o):
            return {'source': 'aa.price', 'kind': 'vendor', 'value': {'input': i, 'output': o}, 'as_of': '2026-11-09',
                    'url': 'https://example.test/p', 'listed_as': 'x'}
        rows = {'gpt-5.6-luna': {'fields': {'price': [entry(1.0, 5.0)]}},
                'gpt-6-luna': {'fields': {'price': [entry(2.0, 10.0)]}},
                'claude-opus-5-5': {'fields': {'price': [entry(9.0, 9.0)]}}}
        usd = {'gpt-5.6-luna': {'value': 4.0, 'status': 'ok', 'kind': 'community'},
               'gpt-6-luna': {'value': None, 'status': 'missing', 'kind': 'none'},
               'claude-opus-5-5': {'value': 7.0, 'status': 'ok', 'kind': 'community'}}
        acc = {**READINGS, 'models': {m: {} for m in rows}, 'aliases': {}, 'families': GPT_FAMILIES, 'superseded': {'gpt-5.6-luna': {'by': 'gpt-6-luna'}}}
        meas = {'capacity': {'token_mix': {'input': 0.5, 'output': 0.5}}, 'models': {m: {} for m in rows}}
        scorer = fa.Scorer(meas, acc)
        fa.convert_usd(usd, rows, scorer, meas, acc)
        got = usd['gpt-6-luna']
        self.assertEqual((got['value'], got['kind'], got['sources']['base']['model'], got['sources']['ratio']),
                         (8.0, 'converted', 'gpt-5.6-luna', 2.0))
        self.assertEqual(usd['gpt-5.6-luna']['value'], 4.0)                          # direct values are not touched
        self.assertEqual(usd['claude-opus-5-5']['value'], 7.0)
        # cost per hour reads the own line only (series.groups 'line', round 6 item 4): the maker's astra line gives nothing
        rows = {'gpt-6-astra': {'fields': {'price': [entry(1.0, 5.0)]}}, 'gpt-6-luna': {'fields': {'price': [entry(2.0, 10.0)]}}}
        usd = {'gpt-6-astra': {'value': 4.0, 'status': 'ok', 'kind': 'community'},
               'gpt-6-luna': {'value': None, 'status': 'missing', 'kind': 'none'}}
        acc = {**acc, 'models': {m: {} for m in rows}}
        meas = {**meas, 'models': {m: {} for m in rows}}
        fa.convert_usd(usd, rows, fa.Scorer(meas, acc), meas, acc)
        self.assertIsNone(usd['gpt-6-luna']['value'])

    def test_no_price_means_no_conversion_and_no_default(self):
        rows = {'gpt-5.6-luna': {'fields': {'price': []}}, 'gpt-6-luna': {'fields': {'price': []}}}
        usd = {'gpt-5.6-luna': {'value': 4.0, 'status': 'ok', 'kind': 'community'},
               'gpt-6-luna': {'value': None, 'status': 'missing', 'kind': 'none'}}
        acc = {**READINGS, 'models': {m: {} for m in rows}, 'aliases': {}, 'families': GPT_FAMILIES, 'superseded': {'gpt-5.6-luna': {'by': 'gpt-6-luna'}}}
        meas = {'capacity': {'token_mix': {'input': 0.5, 'output': 0.5}}, 'models': {m: {} for m in rows}}
        fa.convert_usd(usd, rows, fa.Scorer(meas, acc), meas, acc)
        self.assertIsNone(usd['gpt-6-luna']['value'])

    def test_tier_up_older_gives_an_older_row_its_declared_larger_model_and_nothing_else_does(self):
        """Rule (series.tier_up): a current member of a small row gets the first model of its `tier_up` row; an older row of
        the line only through the row's `tier_up_older` declaration, with its declared version."""
        older = {'fam-0-mini': {'model': 'fam-1', 'version': 'nearest'}}
        row = {'pattern': '^fam.*mini', 'models': ['fam-1-mini'], 'maker': 'acme', 'product': 'acme', 'line': 'fam', 'size': 'small',
               'tier_up': '^fam', 'tier_up_version': 'same'}
        acc = {'models': {m: {} for m in ('fam-1-mini', 'fam-0-mini', 'fam-1')},
               'families': {'rows': [{**row, 'tier_up_older': older},
                                     {'pattern': '^fam', 'models': ['fam-1'], 'maker': 'acme', 'product': 'acme', 'line': 'fam', 'size': 'full'}]}}
        self.assertEqual(series.tier_up(acc), {'fam-0-mini': older['fam-0-mini'],
                                               'fam-1-mini': {'model': 'fam-1', 'version': 'same'}})
        acc['families']['rows'][0] = row
        self.assertEqual(list(series.tier_up(acc)), ['fam-1-mini'])

    def test_a_model_with_no_priced_sibling_takes_its_larger_models_usd_times_the_ratio_on_one_price_list(self):
        """Rule (`convert_usd`, `pair_prices`): with no same-series sibling, the larger model (series.tier_up) is the base,
        `relation: size`; both prices come from the first source that lists both, not one from each."""
        def entry(src, i, o):
            return {'source': src, 'kind': 'vendor', 'value': {'input': i, 'output': o}, 'as_of': '2026-11-09',
                    'url': 'https://example.test/p', 'listed_as': 'x'}
        rows = {'fam-big': {'fields': {'price': [entry('aa.price', 8.0, 8.0), entry('openrouter.price', 4.0, 4.0)]}},
                'fam-big-flash': {'fields': {'price': [entry('openrouter.price', 1.0, 1.0)]}}}
        usd = {'fam-big': {'value': 10.0, 'status': 'ok', 'kind': 'community'},
               'fam-big-flash': {'value': None, 'status': 'missing', 'kind': 'none'}}
        acc = {**READINGS, 'models': {m: {} for m in rows}, 'aliases': {},
               'families': {'rows': [{'pattern': '^fam.*flash', 'models': ['fam-big-flash'], 'maker': 'acme', 'product': 'acme',
                                      'line': 'fam', 'size': 'small',
                                      'tier_up': '^fam', 'tier_up_version': 'same'},
                                     {'pattern': '^fam', 'models': ['fam-big'], 'maker': 'acme', 'product': 'acme', 'line': 'fam',
                                      'size': 'full'}]}}
        meas = {'capacity': {'token_mix': {'input': 0.5, 'output': 0.5}}, 'models': {m: {} for m in rows}}
        fa.convert_usd(usd, rows, fa.Scorer(meas, acc), meas, acc)
        got = usd['fam-big-flash']
        self.assertEqual((got['value'], got['relation'], got['sources']['base']['model'], got['sources']['ratio']),
                         (2.5, 'size', 'fam-big', 0.25))                     # openrouter on both: 1 / 4, not 1 / 8
        self.assertEqual(fa.pair_prices([entry('aa.price', 8.0, 8.0)], [entry('openrouter.price', 1.0, 1.0)])[0]['source'],
                         'aa.price')                                         # no shared list: each its own first


if __name__ == '__main__':
    unittest.main()


class ListingIdentityTests(unittest.TestCase):
    """Round 5 review A: a listing names one snapshot and one model; a row reads another name only on a cited statement."""
    LABELS = sources.effort_readings(READINGS)

    def test_a_date_tag_stays_part_of_the_base_name_and_an_effort_tag_does_not(self):
        self.assertEqual(sources.base_name('Qwen-Max (0919)'), 'Qwen-Max 0919')
        self.assertEqual(sources.base_name('DeepSeek-V4-Pro (High, 20260813)'), 'DeepSeek-V4-Pro 20260813')
        self.assertEqual(sources.base_name('GPT-6 Astra (xhigh)'), 'GPT-6 Astra')
        self.assertEqual(sources.base_name('GPT-6 Astra high'), 'GPT-6 Astra')
        self.assertEqual(sources.base_name('Qwen3 Max'), 'Qwen3 Max')          # "Max" in a Qwen name is the name

    def test_a_reasoning_label_reads_as_declared(self):
        """Rule (accounts.json effort_readings): `non-reasoning` is reasoning off whatever else the tag names; `reasoning`
        is a listing without an effort unless the tag names one; a declaration naming an effort word as a label stops."""
        effort = lambda n: sources.aa_effort(n, self.LABELS)      # noqa: E731
        self.assertEqual([effort('M (Non-reasoning)'), effort('M (Non-reasoning, High)'), effort('M (Reasoning)'),
                          effort('M (Reasoning, Low)'), effort('M')], ['none', 'none', '', 'low', ''])
        with self.assertRaises(ValueError):
            sources.effort_readings({'effort_readings': {'labels': {'high': 'none'}, 'reads': {'default': 'high'}}})
        with self.assertRaises(ValueError):
            sources.effort_readings({'effort_readings': {'labels': {'reasoning': 'max'}, 'reads': {'default': 'high'}}})

    def test_a_fallback_part_is_a_serving_note_not_an_effort(self):
        """Rule (measurements.aa_variants): a tag part ending in `fallback` is AA's note of the model its harness falls back
        to; the listing is read at the effort the rest of the tag names."""
        recs = [{'name': 'M (High, Opus 4.8 Fallback)', 'v': 1}, {'name': 'M (Low)', 'v': 2}, {'name': 'M (Preview build)', 'v': 3}]
        got = measurements.aa_variants(recs, self.LABELS)
        self.assertEqual({e: r['v'] for e, r in got['M'].items()}, {'high': 1, 'low': 2})
        self.assertTrue(measurements.EFFORT_TAG.match('default fallback'))

    def test_every_effort_of_the_one_order_is_read_on_a_board(self):
        """Rule (Builder.select_variants, accounts.json effort_readings, round 6 item 5): Minimal, None and a thinking-off
        label (Instant, Non-Thinking) are read at their effort on every board, never dropped."""
        labels = sources.effort_readings({'effort_readings': {'labels': {'non-reasoning': 'none', 'non-thinking': 'none',
                                                                         'instant': 'none', 'reasoning': 'default'},
                                                              'reads': {'default': 'high'}}})
        items = [('G (Minimal)', 1.0, {}), ('G (High)', 3.0, {}), ('K (Instant)', 2.0, {}), ('K (Thinking)', 4.0, {}),
                 ('M (Non-Thinking)', 1.5, {}), ('M', 2.5, {}), ('N (None)', 0.5, {}), ('N (Low)', 1.0, {})]
        board, _, _, by = measurements.Builder.select_variants(items, labels)
        self.assertEqual(by, {'G': {'minimal': 1.0, 'high': 3.0}, 'K': {'none': 2.0, 'high': 4.0},
                              'M': {'none': 1.5, 'high': 2.5}, 'N': {'none': 0.5, 'low': 1.0}})
        self.assertEqual((board['K'], board['N']), (4.0, 1.0))      # the score is the High row, else nearest High

    def test_edge_pairs_are_read_on_every_listing_of_the_board(self):
        """Rule (Builder.edge_pairs, round 6 item 5): each listing with both efforts of an edge's pair gives one difference
        in panel z, catalog row or not; `none` against its lowest listed reasoning effort, a label-less listing at the
        effort `reads` names."""
        by = {'A': {'none': 10.0, 'low': 14.0, 'medium': 16.0}, 'B': {'none': 9.0, 'default': 11.0},
              'C': {'max': 20.0, 'xhigh': 19.0}, 'D': {'high': 5.0}}
        got = measurements.Builder.edge_pairs(by, 0.0, 2.0, {'default': 'high'})
        self.assertEqual({e: [(q['listed'], q['efforts'], q['difference']) for q in v] for e, v in got.items()},
                         {'none': [('A', ['none', 'low'], -2.0), ('B', ['none', 'high'], -1.0)],
                          'minimal': [('A', ['low', 'medium'], -1.0)], 'ultra': [('C', ['max', 'xhigh'], 0.5)]})

    def test_a_listed_as_under_another_name_needs_a_cited_identity_statement(self):
        """Rule (measurements.listed_as_identity): a row's own spellings need no citation; another name needs
        listed_as_source with kind vendor, community or benchmark, an https url, as_of and statement; a dated maintainer
        declaration is refused, as is a listed_as_source without its listed_as."""
        cite = {'kind': 'vendor', 'url': 'https://example.test/m', 'as_of': '2026-11-18', 'statement': 'X is m-1'}
        measurements.listed_as_identity({'m-1-high': {'listed_as': {'aa': 'M-1 (High)'}}})
        measurements.listed_as_identity({'m-1': {'listed_as': {'aa': 'X'}, 'listed_as_source': {'aa': cite}}})
        for bad in ({'m-1': {'listed_as': {'aa': 'X'}}},
                    {'m-1': {'listed_as': {'aa': 'X'}, 'listed_as_source': {'aa': {**cite, 'kind': 'maintainer'}}}},
                    {'m-1': {'listed_as': {'aa': 'X'}, 'listed_as_source': {'aa': {**cite, 'url': 'http://example.test'}}}},
                    {'m-1': {'listed_as': {'aa': 'X'}, 'listed_as_source': {'aa': {**cite, 'statement': ' '}}}},
                    {'m-1': {'listed_as_source': {'aa': cite}}}):
            with self.subTest(bad=bad), self.assertRaises(ValueError):
                measurements.listed_as_identity(bad)
        with self.assertRaises(ValueError):          # a dated listing is another snapshot, not the row's own spelling
            measurements.listed_as_identity({'m-1': {'listed_as': {'aa': 'M-1 (0919)'}}})


class EstimationOrderTests(unittest.TestCase):
    def test_the_declared_order_must_be_the_codes_order_with_a_distance_and_its_evidence(self):
        """Rule (five_axis.estimation_order): the data declares the order the code applies; another order, a missing
        distance or missing evidence stops the build."""
        doc = READINGS['estimation_order']
        self.assertEqual(fa.estimation_order({'estimation_order': doc})['sibling_first_up_to_distance'], 1)
        for bad in ({**doc, 'order': list(reversed(doc['order']))}, {**doc, 'sibling_first_up_to_distance': '1'},
                    {**doc, 'evidence': ''}, {k: v for k, v in doc.items() if k != 'evidence'}, None,
                    {k: v for k, v in doc.items() if k != 'speed_cross_line_before_fit'},
                    {**doc, 'speed_cross_line_before_fit': 'yes'}):
            with self.subTest(bad=bad), self.assertRaises(ValueError):
                fa.estimation_order({'estimation_order': bad})
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall