Skip to content

dwgx/SmartCLI / skills/drive-tui

drive-tui

Verified

Drive interactive terminal (TUI) programs — CLIs, REPLs, installers, menu apps, agent CLIs, and editors like vim — through a PTY, reading semantic screen snapshots. A pattern library classifies a screen (REPL, menu, pager, fzf search, confirm dialog, form, spinner, wizard) and drives it with a ready recipe. Use when a program expects a live terminal (arrow-key menus, prompts, spinners, password fields, curses UIs), or when a piped command hangs or prints nothing.

references/LIMITATIONS.md

View on GitHub
# drive-tui — Known limitations & fix log

A living log the AI reads and appends to (see SKILL.md → "Known limitations &
self-improvement"). Read this FIRST when a program misbehaves — the cause may
already be here. When you fix something, add a dated entry: symptom, root cause,
fix, and **exactly how you verified it on the real run path**.

Rules for entries: measure ground truth (don't guess), verify by driving the
real program (not a mock), and if you touched `smartcli_core` note the
regression run (drive-probes + `_sandbox_posix_backend.py` on Linux).

---

## Fixed & verified

### 2026-08-09 · One connection could stall the daemon for every other caller
- **Shipped in v0.2.3** (2026-08-09).
- **Symptom:** any local process — with no credential at all — could connect to the
  session port, send bytes with no newline, and block every other caller. With
  `listen(8)`, nine such connections denied service for ~18s at a time, repeatable.
- **Root cause:** the accept loop was serial and the UNAUTHENTICATED transport read ran
  inline on it. An earlier fix split the read budget (2s pre-auth / 60s post-auth),
  cutting the worst case from 540s to 18s — a 30x mitigation that left the primitive
  intact, because the loop was still serial.
- **Fix:** three roles. The accept thread only accepts; a per-connection reader does the
  unauthenticated work (read, parse, check the token) so one silent peer burns only its
  own 2s; and a SINGLE worker thread is the only thread that ever touches the session.
  That last part is required, not caution — `visual_hash()` clears `screen.dirty` as a
  side effect, `pump()` is read-modify-write, and `resize()` mutates four fields in
  sequence, so two threads in the session corrupt perception silently.
- **Plus a second layer:** a long `wait*` occupied the worker, so a concurrent
  `snapshot` waited behind it (8s, measured) — the same block one layer in. The core's
  wait loops now take an optional poll hook, called in the idle gap ON THE WAITING
  THREAD, so fast verbs are answered without a second thread touching the session.
- **Verified:** `tests/test_daemon_concurrency.py` drives the real accept loop against a
  fake session (no PTY): 9 silent peers → authenticated request served in 0.00s;
  `snapshot` during a long wait → 0.04s; 200 KB request still succeeds; auth still
  enforced; session access provably single-threaded. Mutation-verified three ways
  (serial inline → 10.06s FAIL; no poll hook → 8.00s FAIL; handling on the reader
  thread → FAIL). On a live PTY: `snapshot` in 0.13s during a 20s `wait-regex`, zero
  leaked sessions. run_all 44/44; CI green on all 11 jobs including the three-OS
  real-PTY smoke.

### 2026-08-09 · A second `close` is idempotent only after the daemon has exited (the _mcp_probe flake)
- **Not a defect — the consequence of a deliberate fix.** `close` returns as soon as the
  daemon has SENT its reply; the daemon then exits through its own `finally`. A second
  `close` landing inside that window finds the request failing while the pid is still
  alive, and correctly REFUSES (see the 2026-08-07 entry above — deleting the entry there
  would destroy the token and pid that are the only handles left on a live child).
- **If you script a retry, poll.** Call `close` again on a bounded loop until it reports
  ok, rather than asserting the second call succeeds immediately. `tests/_mcp_probe.py`
  does exactly this; its assertion used to require `ok` unconditionally and was the
  load-dependent flake that failed 2 of 5 full-suite runs and one CI leg on macOS.
- **Resolved 2026-08-09.** Mutation-verified that the polled version still fails if
  `close` never becomes idempotent, and 5/5 under four CPU burners with zero leaks.

### 2026-08-07 · `close` after a timeout deleted a LIVE daemon's registry entry, and `list` then confirmed a lie
- **Symptom:** `close --id <SID>` printed `closed <sid>`, exited 0, and `list` reported
  zero sessions — while the daemon was still running and still owned a PTY child.
- **Root cause:** `_call` turns *any* transport failure, including a plain timeout,
  into a `SystemExit` whose message tells the operator to run `close` "to clean up the
  stale entry", and `cmd_close` then unlinked the registry file unconditionally. But
  the daemon's accept loop is **serial**, so a busy daemon looks exactly like a dead
  one at the socket. That file is the only store of both the capability token and the
  pid, so the deletion left a live daemon unreachable by protocol (token gone) and
  unfindable for a manual kill (pid gone). The red-line check every session is
  supposed to run — `list` → zero leaked sessions — would have confirmed a lie.
- **Fix:** death must be proven before deletion. `_pid_is_alive()` uses
  `os.kill(pid, 0)` on POSIX (`ProcessLookupError` = gone, `PermissionError` = exists
  but foreign) and `OpenProcess` + `GetExitCodeProcess` on Windows, where signal 0
  does not exist. On a failed request with a live pid, `close` **refuses**, exits 1,
  and prints the pid so you can act. `--force` overrides it.
- **What this means for you when it fires:** a refusing `close` is not a bug. Retry
  the verb (the daemon was probably just busy serving another connection), or kill the
  printed pid and re-run. Reach for `--force` only when you have confirmed the process
  is gone, because it restores the old data-loss behaviour by design.
- **Verified:** without spawning a PTY — a registry entry whose pid is the live test
  process and whose port nothing listens on. Pre-fix, `cmd_close` printed
  "closed livepid", returned 0, and unlinked the file while that pid was alive. Locked
  by `test_close_keeps_a_live_daemons_entry` in `tests/test_drive_security.py` (live
  pid → entry kept, rc!=0; absent pid → removed, rc==0; `--force` → removed anyway).
  Mutation-verified: restoring the unconditional unlink fails that check.

### 2026-08-07 · `--env` could re-inject the session token on Windows (case-sensitive guard)
- **Symptom:** on Windows, `--env smartcli_tui_token=stolen` was accepted and the
  driven child received `SMARTCLI_TUI_TOKEN` — the capability the daemon deliberately
  pops so a child cannot control its own session.
- **Root cause:** the guard was `key.startswith("SMARTCLI_TUI_")`, an exact-case
  check. Windows environment names are case-insensitive and CPython upcases keys on
  assignment (`os.py`: under `if name == 'nt'`, `encodekey` returns
  `encode(key).upper()`), so a lowercase spelling passed validation and
  `os.environ.update()` installed it under the reserved name anyway. Measured
  pre-fix: `smartcli_tui_token`, `SmartCli_Tui_Token`, `SMARTCLI_ROOT` and
  `SMARTCLI_MAX_SESSIONS` all ACCEPTED.
- **Fix:** compare uppercased unconditionally (free on POSIX, closes the bypass on the
  historical primary dev target), with the deny-list widened to the other variables
  this CLI reads: `SMARTCLI_ROOT`, `SMARTCLI_MAX_SESSIONS`, `SMARTCLI_AUTO_INSTALL`.
  Your own names are unaffected — a test asserts `SMARTCLI_USER_THING` still passes.
- **Verified:** `tests/test_drive_security.py`; mutation-verified — restoring the
  case-sensitive guard fails 7 checks. Same commit also rejects a non-dict JSON
  request before dispatch (`[1,2,3]` used to return an `AttributeError` string, with
  no `ok` field, to a peer that had not authenticated).

### 2026-08-06 · pyte had no alternate-screen buffer, so full-screen programs corrupted the primary screen
- **Symptom:** pyte implements no alternate-screen mode at all (1049/1047/47 just set an
  unknown bit), so `vim`/`less`/`htop` — every full-screen program this skill exists to
  drive — painted their alternate screen on top of the main one, and on exit the main
  screen was never restored. An agent read a merged, impossible screen with nothing
  reporting a problem.
- **Fix:** implemented per xterm and verified against tmux: 1049 saves the cursor and
  clears the alt buffer on entry, restores both on exit; 47/1047 switch without the
  cursor save; the cursor is deliberately NOT homed on entry. Private mode 1048
  (cursor save/restore only) is also supported, with a weaker evidence level: xterm
  defines it, but neither tmux nor GNU screen implements it, so there is no ground
  truth to check it against. The state now reaches every surface an agent reads —
  `ScreenModel.alt_screen`, `Snapshot.alt_screen`, the `to_text()` header (leads the
  flags, inserted before `selected`/`status`/`errors`), the JSON `hints`, and every
  drive-tui daemon reply — rather than only being reachable by poking the pyte object.
- **Verified:** six alt-screen cases diffed against a real terminal — four vs a tmux
  pane in `_diff_tmux_pyte.py`, plus two in the three-way `_diff_two_refs.py` against
  tmux AND GNU screen (with `altscreen on`, which GNU screen defaults off) — and a
  further 26 alt-screen assertions locked deterministically in
  `test_terminal_fidelity.py`. Keep those two numbers apart: only the six were
  compared against a real terminal, and the 26 are in-memory. On a live PTY: driving
  real `less` reports `alt_screen=True` in the header while inside it and `False`
  after quitting. (This entry originally said "sixteen ... diffed against a real tmux
  pane", copied from HANDOFF without counting; a review caught it. Count with
  `grep -c '"alt screen' tests/_diff_*.py`.)

### 2026-07-19 · Selection-only and cursor-only changes were invisible (formerly a "Still open" entry)
- **Symptom:** after an arrow key, a menu could move its selection using only
  reverse video/background attributes while its text stayed identical;
  text-only `wait_change` correctly remained stable but offered no alternative.
- **Fix:** added a separate `visual_hash` over cells, attributes, and cursor state,
  plus `wait_visual_change` in the core, daemon, CLI, one-shot steps, and MCP.
  Text readiness deliberately keeps its old content-only hash so blink/cosmetic
  churn cannot make output streams permanently unstable.
- **Verified:** `tests/test_visual_change.py` locks reverse-video, cursor-only,
  and unchanged-text behavior in memory; the macOS CLI and MCP real-PTY probes
  remain green with zero leaked sessions.

### 2026-07-13 · POSIX `terminate()` left a zombie child (formerly a "Still open" entry)
- **Symptom:** on Linux, after `close()`/`terminate()` the child stayed as a
  `<defunct>` (zombie) process — `SIGTERM` was sent but nothing reaped it.
- **Root cause:** `PosixPtyBackend.terminate()` called `os.kill(SIGTERM)` and
  `os.close(fd)` but never `os.waitpid()`, so the kernel kept the exit status.
- **Fix:** `terminate()` now polls `waitpid(WNOHANG)` up to ~1s, then falls back
  to `SIGKILL` + a blocking `waitpid`, reaping in all paths.
- **Verified:** real Debian 13 over SSH — `tests/_sandbox_posix_backend.py` went
  from `[KNOWN] zombie (state=Z)` to `[OK] no zombie … gone/reaped`. Windows
  drive-probe suite 1–6 + tui_cli still green (POSIX-only change).

### 2026-07-13 · Arrow keys ignored by curses/DECCKM apps (formerly a "Still open" entry)
- **Symptom:** sending `keys Up`/`Down` to a full-screen curses program moved
  nothing — the app never saw an arrow key.
- **Root cause:** we always emitted CSI arrows (`ESC [ A`). Apps that enable
  DECCKM (application cursor keys, `ESC[?1h` — what `curses.keypad(True)` does)
  expect SS3 (`ESC O A`); CSI is not recognised in that mode.
- **Fix:** `send_keys` now reads the live cursor-key mode via
  `ScreenModel.app_cursor` (pyte records DECCKM as mode value `32`) and emits SS3
  for cursor/nav keys when it's on, CSI otherwise. Fully automatic — callers
  still just send `keys Up`. `_resolve_key(token, app_cursor=…)` + `KEY_MAP_SS3`.
- **Verified:** real Debian 13 ncurses probe — `curses.keypad(True)` app read our
  adaptive `Up` as `KEY_UP` (`[PASS] #5 FIXED`); pyte reported `DECCKM=on` from
  the live screen. Windows default path unchanged (CSI when no DECCKM), Ctrl-C
  and all drive-probes unaffected.

---

## Still open (with reasons)

### A second LONG wait queues behind the first (inherent, not a defect)
- One session owns one PTY child, so it has one screen. Two concurrent `wait*` verbs on
  the same session have no meaningful independent answer, and the single session worker
  runs them in arrival order.
- **Fast verbs are NOT affected** — `snapshot`, `alive`, `list`, `send-text`,
  `send-line`, `keys` are answered *during* a long wait (measured on a live PTY: 0.13s
  while a 20s `wait-regex` was in flight). `resize` is deliberately excluded: it
  re-dimensions the pyte screen and therefore changes the content hash, so answering it
  mid-wait would make a caller blocked in `wait-change` read "the screen changed" and
  conclude its own keystroke had landed.
- If you need two independent waits, use two sessions.

### ConPTY (Windows) startup quiet-gap & Ctrl-C
- First prompt can land ~3s after spawn; use `wait-regex` with a 15s timeout for
  the FIRST prompt, never bare `wait`. Raw Ctrl-C is unreliable under ConPTY —
  recover with `close` + fresh `start`. (POSIX Ctrl-C works.)

### Environment notes
- POSIX backend verified on Debian 13 / Python 3.13 (2026-07-13) and macOS on
  Apple Silicon / Python 3.14 (2026-07-19). The BSD PTY EOF path, persistent
  CLI, MCP adapter, resize, REPL drive, and zombie-free close all passed.
- tmux launcher scripts (`skills/cmd-art/tmux/*.sh`) VERIFIED 2026-07-27 on real
  tmux 3.6b (macOS): `tests/_tmux_launcher_probe.py` drove both scripts through
  all five states — 18/18. It found a real bug (`fx-popup` leaked tmux's raw
  "no current client" with exit 1 when no client was attached); now guarded.
- The core detects pyte's capabilities at import time (`_PYTE_HAS_ALT`,
  `_PYTE_DCH_HANDLES_WIDE` in `smartcli_core/screen_model.py`) instead of pinning
  a version range. So behaviour can change from a `pip install -U pyte` alone,
  with no SmartCLI code change: once pyte ships its own alternate-screen support
  or wide-glyph-aware DCH, this core's override switches off automatically. If a
  screen or a delete-characters case looks wrong after a dependency upgrade,
  check the installed pyte version and these two flags before assuming a
  regression here.

## Service, budgets and close (A04, 0.3.0)

- **The daemon services the session on its own.** Every worker iteration spends one byte-bounded I/O
  turn, so a child keeps running (and its device queries keep being answered) even when no client
  asks anything. Measured: `ESC[6n` answered 0.169 s after the child sent it with no polling, and a
  256 KiB fixture completed unprompted; with the turn removed the same fixture stalls at 12 288 B.
- **A timeout is not an answer.** `io.local_cut` distinguishes `drained` from `budget_limited`,
  `unknown` and `error`, and `pending` carries `null` (not `0`) when a transport cannot count. A
  `STABLE` verdict requires `drained`; a caller that ignores `io` is back to guessing.
- **Windows is not wire-byte-exact.** ConPTY composes the stream it delivers (a 1 MiB burst arrived
  as ~12 KB), so `representation=conpty_reconstructed_utf8` and `source_wire_exact=false`. The
  payload cap bounds what this runtime has received -- it does not bound ConPTY, and it does not
  apply backpressure to the child.
- **Close is confirmed or admitted.** `close_unconfirmed` with `last_progress` is a real answer; a
  returned native call is never treated as an exit. The three liveness gaps that used to follow this
  sentence are closed (`tests/test_liveness_gaps.py`): an owed device reply is RESUMED on a later I/O
  turn from a ledger that re-sends only bytes the transport reported as NOT accepted, bounded by
  `SMARTCLI_REPLY_RETRY_BYTES` per spawn generation — **off by default**, so with it unset the pre-S6
  behaviour stands (one attempt per observation, the remainder reported and left unwritten, and bytes
  whose landing the transport cannot report never re-sent); the daemon's reader and job caps are now
  ADMISSION limits (`SMARTCLI_MAX_READERS` / `SMARTCLI_MAX_JOBS`, default 64 each) that refuse with
  `reason: too_many_readers` / `too_many_jobs` before a thread or a queue slot is spent, instead of
  filtering a list after the fact; and `_reply` sends non-blocking under its 60 s ceiling plus a
  no-progress stall window (`SMARTCLI_REPLY_STALL_SECONDS`, default 2 s), so a peer that stops reading
  releases the worker inside that window while a slow-but-reading peer keeps the patience it always
  had. **Still open:** `pump(max_bytes=...)` can hang once the reader pauses (see the OPEN section
  below; issue #15). Two residual bounds worth knowing: the caps bound ADMISSION, so a queue already
  admitted and re-queued by the interleave path can transiently exceed `MAX_JOBS` (refused work is
  never silently dropped, and admitted work still runs); and a peer that keeps making token progress
  can still hold the worker up to the 60 s ceiling, which is deliberate — that is the patience a
  healthy caller had before A06.

## Write-path scheduling edges (N2, 2026-09-16)

Three edges the 256 KiB real-pty test never reached were reachable on a real daemon, and all three
were defects in `PosixPtyBackend.write` / the reply path:

- **A trickling writer ran past the deadline.** The offset advances on every call, so a loop that
  checks its deadline only in the EAGAIN/EINTR/zero-progress branches never reached the check: 64
  one-byte calls completed past a deadline that expired after the third and still reported success.
  The deadline is now re-checked before EVERY write. A completed write is still never downgraded.
- **A `select.select` failure lost the receipt.** The exception propagated raw, so a caller could not
  tell whether 0 or 4095 bytes had landed; EINTR was not retried at all. Waiting is now bounded and
  converts: a signal retries the SAME suffix (never a re-send from zero, which would duplicate bytes),
  any other error reports the earned prefix as `IncompleteWrite(offset, total, reason)`.
- **A reply that could not be written was invisible.** A bare `except: pass` swallowed it, so a child
  blocked on a device query looked like a child that ignored the input. It is now reported as
  `io.pending.reply_error` with the known prefix subtracted from `reply_bytes`, attempted once (a
  blind whole-payload retry would duplicate a partial reply), and cleared by the next healthy write.

Verification: `tests/test_partial_write_edges.py` (7 cases, injected `os.write`/`select`/clock, no PTY)
failed 5/7 before the change and passes after; the real-pty test still delivers 262 144 B exactly once
(sha256 match, 32 writability waits). The wait count is scheduling-dependent -- 32/35/44 across
identical runs -- so never read it as a fixed property of the transport.

## CLOSED as a mis-diagnosis: `pump(max_bytes=)` does not hang (filed and closed 2026-09-16)

An earlier entry here claimed that a client asking for a byte budget could wait forever once the reader
paused at the payload high water. That was wrong, and the way it was wrong is worth keeping:

- the probe behind it wrote **262 144 bytes in one burst**. This transport does not deliver that: measured
  with raw `winpty.PtyProcess` and no SmartCLI code at all, a single 262 144-byte write arrives as
  **~12 326 bytes** while a plain pipe carries all 262 160 — so the probe could never pass, whatever the
  runtime did;
- re-measured with a transport-fair probe (child writes 4 KiB every 20 ms, high water lowered, client
  drains in slices) the runtime passes on unmodified HEAD: the reader really pauses
  (`max_accounted_payload` 106 654), every byte arrives (sha256 match), and the slowest `pump` turn is
  **7.7 ms**. No stall, no defect;
- what the episode DID produce: the S5 tests now drive a transport that can actually pause the reader
  (`tests/test_a04_winpty_backlog.py`), because the old fake returned instantly and made "the cap held"
  and "the reader paused" mutually exclusive states.

Repro: `verification\runs\v3\A04\s5-highwater\fair_highwater_probe.py`. Issue #15 is closed.

## Environment facts found while verifying (2026-09-16, Windows 11 + ConPTY)

- **A Git-for-Windows vim and this box's `less` never switch to the alternate screen** under ConPTY.
  `esc[?1049h` decoded out of a synthetic stream still sets `alt_screen=True`, so the model is fine:
  the programs simply never enter it here. Checks that demand `alt_screen is True` fail for the
  environment, so `examples/drive_vim.py` and `tests/_mcp_probe.py` now assert that the model AGREES
  with what the program did (entered it -> True, never entered it -> False). Where a program does
  enter the alternate screen, a False reading still fails.
- **`tests/_tmux_launcher_probe.py` SKIPs on Windows** -- there is no `termios`/`pty` at all, so the
  probe cannot run; it now says so instead of failing the aggregator. Run it on a POSIX host with tmux
  to get the 18/18 verification the script needs.
- **pywinpty `shlex.split`s a string command.** `session.start(r'"C:\Program Files\...\vim.exe" -u NONE')`
  never starts (the error is `The command was not found or was not executable: C:\Program`). Pass an
  ARGV LIST instead: `session.start([vim, "-u", "NONE", ...])`.
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall