Skip to content

mxyhi/ok-skills / ai-elements

ai-elements

Verified

Build AI chat interfaces using ai-elements components — conversations, messages, tool displays, prompt inputs, and more. Use when the user wants to build a chatbot, AI assistant UI, or any AI-powered chat interface.

FileKindSize
SKILL.mdskill md7.1 KB
references/agent.mdmarkdown4.1 KB
references/artifact.mdmarkdown2.9 KB
references/attachments.mdmarkdown5.6 KB
references/audio-player.mdmarkdown5.2 KB
references/canvas.mdmarkdown1.1 KB
references/chain-of-thought.mdmarkdown3.3 KB
references/checkpoint.mdmarkdown5.5 KB
references/code-block.mdmarkdown5.4 KB
references/commit.mdmarkdown5.8 KB
references/confirmation.mdmarkdown8.6 KB
references/connection.mdmarkdown1.1 KB
references/context.mdmarkdown5.6 KB
references/controls.mdmarkdown971 B
references/conversation.mdmarkdown7.4 KB
references/edge.mdmarkdown2.1 KB
references/environment-variables.mdmarkdown3.7 KB
references/file-tree.mdmarkdown2.0 KB
references/image.mdmarkdown4.0 KB
references/inline-citation.mdmarkdown10.8 KB
references/jsx-preview.mdmarkdown3.2 KB
references/message.mdmarkdown9.4 KB
references/mic-selector.mdmarkdown7.7 KB
references/model-selector.mdmarkdown4.3 KB
references/node.mdmarkdown2.8 KB
references/open-in-chat.mdmarkdown2.7 KB
references/package-info.mdmarkdown3.2 KB
references/panel.mdmarkdown1.0 KB
references/persona.mdmarkdown5.4 KB
references/plan.mdmarkdown3.2 KB
references/prompt-input.mdmarkdown19.8 KB
references/queue.mdmarkdown5.4 KB
references/reasoning.mdmarkdown7.8 KB
references/sandbox.mdmarkdown4.3 KB
references/schema-display.mdmarkdown2.5 KB
references/shimmer.mdmarkdown1.6 KB
references/snippet.mdmarkdown2.2 KB
references/sources.mdmarkdown6.2 KB
references/speech-input.mdmarkdown6.5 KB
references/stack-trace.mdmarkdown7.9 KB
references/suggestion.mdmarkdown3.6 KB
references/task.mdmarkdown6.7 KB
references/terminal.mdmarkdown3.1 KB
references/test-results.mdmarkdown5.0 KB
references/tool.mdmarkdown8.5 KB
references/toolbar.mdmarkdown983 B
references/transcription.mdmarkdown4.9 KB
references/voice-selector.mdmarkdown10.3 KB
references/web-preview.mdmarkdown6.1 KB
scripts/agent.tsxother1.6 KB
scripts/artifact.tsxother2.9 KB
scripts/attachments-inline.tsxother3.5 KB
scripts/attachments-list.tsxother2.2 KB
scripts/attachments.tsxother2.0 KB
scripts/audio-player-remote.tsxother1.0 KB
scripts/audio-player.tsxother1.8 KB
scripts/chain-of-thought.tsxother138.6 KB
scripts/checkpoint.tsxother2.8 KB
scripts/code-block-dark.tsxother1.0 KB
scripts/code-block.tsxother2.9 KB
scripts/commit.tsxother2.1 KB
scripts/confirmation-accepted.tsxother1.2 KB
scripts/confirmation-rejected.tsxother1.1 KB
scripts/confirmation-request.tsxother1.7 KB
scripts/confirmation.tsxother1.7 KB
scripts/context.tsxother1005 B
scripts/conversation.tsxother4.1 KB
scripts/environment-variables.tsxother1.7 KB
scripts/file-tree-basic.tsxother380 B
scripts/file-tree-expanded.tsxother609 B
scripts/file-tree-selection.tsxother619 B
scripts/file-tree.tsxother1.5 KB
scripts/image.tsxother136.5 KB
scripts/inline-citation.tsxother3.2 KB
scripts/jsx-preview.tsxother3.2 KB
scripts/message.tsxother10.1 KB
scripts/mic-selector.tsxother1.2 KB
scripts/model-selector.tsxother8.5 KB
scripts/open-in-chat.tsxother704 B
scripts/package-info.tsxother1.3 KB
scripts/persona-command.tsxother2.5 KB
scripts/persona-glint.tsxother2.5 KB
scripts/persona-halo.tsxother2.5 KB
scripts/persona-mana.tsxother2.5 KB
scripts/persona-obsidian.tsxother2.5 KB
scripts/persona-opal.tsxother2.5 KB
scripts/plan.tsxother1.9 KB
scripts/prompt-input-cursor.tsxother15.1 KB
scripts/prompt-input-tooltip.tsxother1.1 KB
scripts/prompt-input.tsxother7.4 KB
scripts/queue-prompt-input.tsxother10.5 KB
scripts/queue.tsxother8.0 KB
scripts/reasoning.tsxother2.2 KB
scripts/sandbox.tsxother5.4 KB
scripts/schema-display-basic.tsxother221 B
scripts/schema-display-body.tsxother487 B
scripts/schema-display-nested.tsxother470 B
scripts/schema-display-params.tsxother374 B
scripts/schema-display.tsxother2.7 KB
scripts/shimmer-duration.tsxother934 B
scripts/shimmer-elements.tsxother1.1 KB
scripts/shimmer.tsxother443 B
scripts/snippet-plain.tsxother362 B
scripts/snippet.tsxother589 B
scripts/sources-custom.tsxother1.0 KB
scripts/sources.tsxother752 B
scripts/speech-input.tsxother2.3 KB
scripts/stack-trace-collapsed.tsxother1.1 KB
scripts/stack-trace-no-internal.tsxother1.2 KB
scripts/stack-trace.tsxother1.9 KB
scripts/suggestion-input.tsxother4.4 KB
scripts/suggestion.tsxother802 B
scripts/task.tsxother1.5 KB
scripts/terminal-basic.tsxother169 B
scripts/terminal-clear.tsxother523 B
scripts/terminal-streaming.tsxother1007 B
scripts/terminal.tsxother2.3 KB
scripts/test-results-basic.tsxother464 B
scripts/test-results-errors.tsxother1.3 KB
scripts/test-results-suites.tsxother901 B
scripts/test-results.tsxother2.7 KB
scripts/tool-input-available.tsxother738 B
scripts/tool-input-streaming.tsxother683 B
scripts/tool-output-available.tsxother1.7 KB
scripts/tool-output-error.tsxother1.1 KB
scripts/tool.tsxother6.7 KB
scripts/transcription.tsxother4.9 KB
scripts/voice-selector.tsxother7.1 KB
scripts/web-preview.tsxother2.8 KB

references/speech-input.md

View on GitHub
# Speech Input

A button component that captures voice input and converts it to text, with cross-browser support.

The `SpeechInput` component provides an easy-to-use interface for capturing voice input in your application. It uses the Web Speech API for real-time transcription in supported browsers (Chrome, Edge), and falls back to MediaRecorder with an external transcription service for browsers that don't support Web Speech API (Firefox, Safari).

See `scripts/speech-input.tsx` for this example.

## Installation

```bash
npx ai-elements@latest add speech-input
```

## Features

- Built on Web Speech API (SpeechRecognition) with MediaRecorder fallback
- Cross-browser support (Chrome, Edge, Firefox, Safari)
- Continuous speech recognition with interim results
- Visual feedback with pulse animation when listening
- Loading state during transcription processing
- Automatic browser compatibility detection
- Final transcript extraction and callbacks
- Error handling and automatic state management
- Extends shadcn/ui Button component
- Full TypeScript support

## Props

### `<SpeechInput />`

The component extends the shadcn/ui Button component, so all Button props are available.

| Prop | Type | Default | Description |
|------|------|---------|-------------|
| `onTranscriptionChange` | `(text: string) => void` | - | Callback fired when final transcription text is available. Only fires for completed phrases, not interim results. |
| `onAudioRecorded` | `(audioBlob: Blob) => Promise<string>` | - | Callback for MediaRecorder fallback. Required for Firefox/Safari support. Receives recorded audio blob and should return transcribed text from an external service (e.g., OpenAI Whisper). |
| `lang` | `string` | - | Language for speech recognition. |
| `...props` | `React.ComponentProps<typeof Button>` | - | Any other props are spread to the Button component, including variant, size, disabled, etc. |

## Behavior

### Speech Recognition Modes

The component automatically detects browser capabilities and uses the best available method:

| Browser         | Mode           | Behavior                                               |
| --------------- | -------------- | ------------------------------------------------------ |
| Chrome, Edge    | Web Speech API | Real-time transcription, no server required            |
| Firefox, Safari | MediaRecorder  | Records audio, sends to external transcription service |
| Unsupported     | Disabled       | Button is disabled                                     |

### Web Speech API Mode (Chrome, Edge)

Uses the Web Speech API with the following configuration:

- **Continuous**: Set to `true` to keep recognition active until manually stopped
- **Interim Results**: Set to `true` to receive partial results during speech
- **Language**: Configurable via `lang` prop, defaults to `"en-US"`

### MediaRecorder Mode (Firefox, Safari)

When the Web Speech API is unavailable, the component falls back to recording audio:

1. Records audio using `MediaRecorder` API
2. On stop, creates an audio blob (`audio/webm`)
3. Calls `onAudioRecorded` with the blob
4. Waits for transcription result
5. Passes result to `onTranscriptionChange`

**Note**: The `onAudioRecorded` prop is required for this mode to work. Without it, the button will be disabled in Firefox/Safari.

### Transcription Processing

The component only calls `onTranscriptionChange` with **final transcripts**. Interim results (Web Speech API) are ignored to prevent incomplete text from being processed.

### Visual States

- **Default State**: Standard button appearance with microphone icon
- **Listening State**: Pulsing animation with accent colors to indicate active listening
- **Processing State**: Loading spinner while waiting for transcription (MediaRecorder mode)
- **Disabled State**: Button is disabled when no API is available or required props are missing

### Lifecycle

1. **Mount**: Detects available APIs and initializes appropriate mode
2. **Click**: Toggles between listening/recording and stopped states
3. **Stop (MediaRecorder)**: Processes audio and waits for transcription
4. **Unmount**: Stops recognition/recording and releases microphone

## Browser Support

The component provides cross-browser support through a two-tier system:

| Browser | API Used       | Requirements           |
| ------- | -------------- | ---------------------- |
| Chrome  | Web Speech API | None                   |
| Edge    | Web Speech API | None                   |
| Firefox | MediaRecorder  | `onAudioRecorded` prop |
| Safari  | MediaRecorder  | `onAudioRecorded` prop |

For full cross-browser support, provide the `onAudioRecorded` callback that sends audio to a transcription service like OpenAI Whisper, Google Cloud Speech-to-Text, or AssemblyAI.

## Accessibility

- Uses semantic button element via shadcn/ui Button
- Visual feedback for listening state
- Keyboard accessible (can be triggered with Space/Enter)
- Screen reader friendly with proper button semantics

## Usage with MediaRecorder Fallback

To support Firefox and Safari, provide an `onAudioRecorded` callback that sends audio to a transcription service:

```tsx
const handleAudioRecorded = async (audioBlob: Blob): Promise<string> => {
  const formData = new FormData();
  formData.append("file", audioBlob, "audio.webm");
  formData.append("model", "whisper-1");

  const response = await fetch(
    "https://api.openai.com/v1/audio/transcriptions",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      },
      body: formData,
    }
  );

  const data = await response.json();
  return data.text;
};

<SpeechInput
  onTranscriptionChange={(text) => console.log(text)}
  onAudioRecorded={handleAudioRecorded}
/>;
```

## Notes

- Requires a secure context (HTTPS or localhost)
- Browser may prompt user for microphone permission
- Only final transcripts trigger the `onTranscriptionChange` callback
- Language is configurable via the `lang` prop
- Continuous recognition continues until button is clicked again
- Errors are logged to console and automatically stop recognition/recording
- MediaRecorder fallback requires the `onAudioRecorded` prop to be provided
- Audio is recorded in `audio/webm` format for the MediaRecorder fallback

## TypeScript

The component includes full TypeScript definitions for the Web Speech API:

- `SpeechRecognition`
- `SpeechRecognitionEvent`
- `SpeechRecognitionResult`
- `SpeechRecognitionAlternative`
- `SpeechRecognitionErrorEvent`

These types are properly declared for both standard and webkit-prefixed implementations.
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall