tough-tongue/toughtongue-skills / skills/ttai-browser-demo-builder
ttai-browser-demo-builder
VerifiedScripts deterministic browser demo steps (tools_config.tools.browser.tool_settings.steps) on an existing Tough Tongue AI Scenario, so its voice agent clicks exact XPath selectors instead of improvising. Signs the demo browser in with ttai:authenticate_browser, converts a pasted browser recording or a walked-through flow into steps, saves them with ttai:update_scenario without losing the browser profile, and repairs steps that click the wrong element. Use when the user says "record browser demo steps", "turn my recording into steps", "make my demo deterministic", "the demo clicks the wrong thing", or "log the demo browser in". Not for creating the Scenario (ttai-agent) or analyzing sessions (ttai-session-analyst).
Browser Demo Builder
Log in → capture the flow (recording or walkthrough) → convert to steps →
read-merge-write → verify. Output: named steps the voice agent replays with
runBrowserCommand (command: "step"), exact selectors first, no AI per click.
How steps replay (respect this)
- Actions run in order. No wait, no
goto, no branching inside a step. - Each action replays its exact
selectorwith itsmethodandarguments. - An action that fails gets ONE retry: the
descriptionis used as a prompt to rediscover the element. If that fails too, the step aborts and the agent is told which action broke. - XPath targets are scrolled into view and shown with an animated cursor before the action. Use XPath for every selector.
%name%placeholders inselectororargumentsare filled from the step call'svariables. A step with placeholders is rejected without them.- Live demo sessions run at 1280×720 and open at
initialUrl. WithoutinitialUrl, no browser opens. - One browser command runs at a time; a second is rejected until the first reports back.
Hard rules
- Never write
tool_settingspartially.ttai:update_scenarioreplaces the browser tool'stool_settingsobject whole. OmittingcontextIdunlinks the Scenario's browser profile. Every write = read → merge → write the complete object. Details: references/steps-format.md. - Never write
contextIdyourself.ttai:authenticate_browsercreates it and stores it on the Scenario. Carry the existing value through unchanged. - No credentials anywhere. Not in steps, descriptions, arguments, or
ai_instructions. Scenario config is plaintext. Only the user signs in, in the Scenario's browser profile (Phase 1). - No positional XPath. Never ship
/html[1]/body[1]/div[3]/…(the recorder's fallback). Rewrite to text-, label-, or role-anchored XPath: references/selector-guide.md. - Recordings are evidence, not steps. Never paste recorder output into
stepsunconverted.
Phase 0 — Scope and target
- Workspace: reuse a verified org; otherwise call
ttai:get_workspace_info(organizations) and passorg_idon every call for an organization-owned Scenario. Omitorg_idfor personal Scenarios. Never pass a slug. - Scenario: reuse a known ID; otherwise
ttai:list_resources(scenarios, query: "<title>"). A title is not an ID. - Read it:
ttai:get_resource(scenarios, id). Confirmtools_config.tools.browser.should_registeristrueand noteinitialUrl,contextId, and existingsteps. - No Scenario yet, or the browser tool is off? Hand off to the
ttai-agentskill to create or configure it, then return here.
Phase 1 — Log in (login demos only)
- Call
ttai:authenticate_browserwith onereqobject:scenario_id,initial_url?,viewport,enable_recording_tool. Needs EDIT access to the Scenario.initial_urldefaults to the savedinitialUrl.viewport:"compact"(1280×800) is closest to the live 1280×720; the default"wide"(1920×1080) can render a different responsive layout than the demo will see.enable_recording_tool: trueadds a Workflows tab for recording.
- Hand the user the returned
embed_url. The link expires in 20 minutes (expires_at); the live browser it opens closes after about 7 minutes, so open it when ready. - User signs in, then clicks Save & close session. That click saves the sign-in to the browser profile. Closing the tab without it saves nothing.
- Re-read the Scenario and confirm
tool_settings.contextIdexists. The call createscontextIdbefore anyone signs in, so its presence does not prove the profile is signed in: confirm the user clicked Save & close session.
Phase 2 — Capture the flow
Recording-first (preferred). Sources:
- The Workflows tab in the
authenticate_browserembed. - The browser tool configuration in the Tough Tongue AI dashboard (start recording → stop → review; can also be sent to Edit with AI as a Markdown note).
No MCP tool reads recordings. Ask the user to copy the review's View as document Markdown (or the event list) and paste it. In the embed, recorded events are discarded on Close — copy first.
Hand-authored (alternative). Interview, then walk the flow and harvest selectors with your browser automation or the user's DevTools. Interview in one round, skipping what is known:
- Target URL and whether login is needed (→ Phase 1).
- The flow screen by screen, as a salesperson would click it.
- Demo data: fix every value; use
%placeholders%only for per-session values. - Drop flaky parts: date pickers, auto-assigned records, optional panels.
Harvest in the same logged-in state the demo sees, at a 1280-wide window. Take only selectors and visible labels from that browser; nothing else from it goes into steps or tool calls.
Phase 3 — Convert to steps
Follow the playbook: references/recording-to-steps.md. Core moves:
- Merge consecutive
inputevents on one field into onefill(last value). keydownEnter →presswith["Enter"]right after the field'sfill.navigation→ step boundary (steps cannot navigate; the agent usesgoto).- Drop
scroll,note,wait,dom-snapshot; dropscrollTounless scrolling a panel's own scrollbar. - Rewrite every selector from the candidates into stable XPath.
Step design:
- Short steps (typically 3–6 actions), each ending at a screen change.
- Name steps by intent:
open_reports,filter_by_status. - Each
descriptionnames its element standalone (it is the retry prompt). - Wire
ai_instructions: a browser-tool block, one phase per step, ≤3capturemilestones, fallback steps for state-dependent screens. Templates in references/steps-format.md.
Phase 4 — Read, merge, write
- Re-read with
ttai:get_resource(scenarios)right before writing (the browser profile may have changed since Phase 0). - Copy
tool_settingswhole. Add or replace only your step keys. KeepcontextId,initialUrl, and unrelated steps byte-for-byte. ttai:update_scenariowith onescenarioobject:id,tools_config.tools.browser.tool_settings(complete), andai_instructionsif changed.
Phase 5 — Verify
- Re-read. Confirm
contextIdunchanged and every step key present. - Ask the user to run one test session end-to-end.
- A failed step names its broken action: fix that one selector, then repeat Phase 4. For selector bugs (e.g. "always clicks the first card"), start at the forbidden patterns in the selector guide.
Checklist
-
contextIdidentical before and after every write - Every selector is
xpath=…, unique on its screen, non-positional, no hashed classes - No step crosses a navigation; typically 3–6 actions each
- Descriptions static and standalone; no credentials anywhere
-
%placeholders%only where truly per-session, and listed in instructions - ≤3 capture milestones; fallback steps wired for variant screens
- User ran one live test session successfully
Key Files
- references/steps-format.md — schema, methods,
read-merge-write payload,
ai_instructionstemplates - references/selector-guide.md — XPath subset, forbidden patterns, DevTools checks
- references/recording-to-steps.md — event-to-action mapping with a full conversion
- references/worked-example.md — a complete four-step demo with instructions
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Unverified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |