sumitaich1998/jarvisvr / skills/stage/annotate-reality
annotate-reality
VerifiedPlace world-anchored labels, callouts, pins, and boxes onto real objects and locations so information sticks to the physical world. Use for labeling things in the room, pinning a spot, boxing a detected object, or step-by-step "point at the X" guidance. Triggers: label this, pin here, mark that, point to, highlight, call out, annotate, put a label on.
Annotate Reality
Anchor information to the physical world. The perception-agent decides what a real thing is; you render the spatial label/box/pin in the right place.
Pick the annotation
| Goal | Widget | Tool |
|---|---|---|
| Name a real object with a callout | vision_annotation | annotate_object |
| Box a detected object's extent | bounding_box_3d | draw_bounding_box |
| Drop a pin/marker at a spot | scene_label | drop_scene_label |
| Free-floating heading/caption | text_label | show_text |
Steps
- Get the target position (world meters) from the perception annotation, gaze hit point, or a scene surface.
- Choose the widget from the table; keep labels short.
- Anchor in
worldwithbillboard:trueso the label faces the user; lift the callout slightly above the object (e.g. +0.15 m in Y) with aleader_line. - Spawn, then keep it anchored as the user moves (world anchor handles this).
- Clean up stale annotations via
close_hologramwhen no longer relevant.
Output
vision_annotation (annotate_object, props per registry.json):
{ "widget_type": "vision_annotation",
"transform": { "anchor": "world", "position": [0.3,0.95,0.7], "billboard": true },
"props": { "label": "coffee mug", "confidence": 0.78, "detail": "ceramic, ~350 ml",
"leader_line": true, "target_position": [0.3,0.8,0.7], "color": "#7FE7FF" },
"interactions": ["tap","grab","dwell"] }
bounding_box_3d (draw_bounding_box):
{ "widget_type": "bounding_box_3d",
"props": { "label": "laptop", "confidence": 0.91, "size": [0.33,0.02,0.23], "color": "#FFB74D" } }
scene_label (drop_scene_label):
{ "widget_type": "scene_label",
"props": { "text": "Keys (last seen here)", "icon": "pin", "color": "#FF5252", "pin": true } }
Edge cases
- No reliable position → fall back to a
head-anchoredtext_labeland say you couldn't pin it precisely. - Many annotations → cap to the relevant few; over-labeling clutters reality
(hand to
declutter-space). - Moving target → annotations are static once placed; re-place if the object moves.
- Confidence honesty → carry the real
confidenceso the UI can style uncertainty. - Steps/tour → sequence pins and advance as the user completes each.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |