forcedotcom/sf-skills / plugins/builder/agentforce-adlc/skills/agentforce-test
agentforce-test
FlaggedWrite, run, and analyze structured test suites for Agentforce agents — functional AND security. TRIGGER when: user writes or modifies test spec YAML (AiEvaluationDefinition); runs sf agent test create, run, run-eval, or results commands; asks about test coverage strategy, metric selection, or custom evaluations; interprets test results or diagnoses test failures; asks about batch testing, regression suites, or CI/CD test integration; requests security testing, OWASP LLM Top 10, red-teaming, penetration testing, prompt-injection tests, a security grade, or a vulnerability assessment of an agent. DO NOT TRIGGER when: user creates, modifies, previews, or debugs .agent files (use agentforce-generate); deploys or publishes agents; writes Agent Script code; uses sf agent preview for development iteration; analyzes production session traces (use agentforce-observe); performs a static safety review of .agent file content (use agentforce-generate Section 15).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 39.3 KB |
| assets/basic-test-spec.yaml | config | 3.6 KB |
| assets/guardrail-test-spec.yaml | config | 5.8 KB |
| assets/payloads/excessive-agency.yaml | config | 4.9 KB |
| assets/payloads/misinformation.yaml | config | 6.3 KB |
| assets/payloads/output-handling.yaml | config | 4.3 KB |
| assets/payloads/prompt-injection.yaml | config | 5.5 KB |
| assets/payloads/sensitive-info-disclosure.yaml | config | 5.7 KB |
| assets/payloads/system-prompt-leakage.yaml | config | 6.3 KB |
| assets/payloads/unbounded-consumption.yaml | config | 4.9 KB |
| assets/standard-test-spec.yaml | config | 6.7 KB |
| references/action-execution.md | markdown | 5.7 KB |
| references/batch-testing.md | markdown | 12.0 KB |
| references/owasp-categories.md | markdown | 10.3 KB |
| references/preview-testing.md | markdown | 13.8 KB |
| references/remediation-guide.md | markdown | 7.6 KB |
| references/security-scoring-methodology.md | markdown | 5.1 KB |
| references/security-test-design.md | markdown | 26.4 KB |
| references/security-troubleshooting.md | markdown | 9.8 KB |
| references/test-report-format.md | markdown | 4.5 KB |
| references/troubleshooting.md | markdown | 3.7 KB |
assets/standard-test-spec.yaml
View on GitHub# Standard Agent Test Specification Template
# Compatible with: sf agent test create --spec <file> --api-name <name>
#
# Usage:
# 1. Replace <placeholders> with actual values
# 2. Create: sf agent test create --spec this-file.yaml --api-name <Test_Name> --target-org <alias>
# 3. Run: sf agent test run --api-name <Test_Name> --wait 10 --result-format json --target-org <alias>
#
# IMPORTANT: This YAML is parsed by @salesforce/agents — NOT a generic AiEvaluationDefinition format.
# Only use the fields documented below.
#
# NOTE: The Testing Center API uses "topic" terminology. In Agent Script, topics are called
# "subagents" (e.g., the `subagent` block). When writing tests, use "topic" to match the API,
# but understand that each expectedTopic value maps to a subagent in your .agent file.
# Required: Display name for the test (MasterLabel)
name: "<Agent_Name> Standard Tests"
# Required: Must be AGENT
subjectType: AGENT
# Required: Agent BotDefinition DeveloperName (API name)
subjectName: <Agent_Name>
testCases:
# ═══════════════════════════════════════════════════════════════════
# TOPIC ROUTING TESTS
# Verify utterances route to the correct topic
# ═══════════════════════════════════════════════════════════════════
- utterance: "User message that should trigger topic 1"
expectedTopic: <topic_name>
- utterance: "Alternative phrasing for topic 1"
expectedTopic: <topic_name>
- utterance: "User message that should trigger topic 2"
expectedTopic: <another_topic>
# ═══════════════════════════════════════════════════════════════════
# ACTION INVOCATION TESTS
# Verify actions are invoked (flat list of action name strings)
# ═══════════════════════════════════════════════════════════════════
- utterance: "Message that should trigger an action"
expectedTopic: <topic_name>
expectedActions:
- <action_name>
- utterance: "Message for a second action"
expectedTopic: <topic_name>
expectedActions:
- <action_name_2>
expectedOutcome: "Agent confirms the action and provides relevant details"
# ═══════════════════════════════════════════════════════════════════
# CONTEXT VARIABLE TESTS
# Pass runtime context to simulate authenticated sessions
# ═══════════════════════════════════════════════════════════════════
- utterance: "Show me my account details"
expectedTopic: <topic_name>
contextVariables:
- name: RoutableId
value: "<MessagingSession_ID>"
- name: CaseId
value: "<Case_ID>"
# ═══════════════════════════════════════════════════════════════════
# CONVERSATION HISTORY TESTS
# Simulate multi-turn conversations (roles: user and agent)
# ═══════════════════════════════════════════════════════════════════
- utterance: "Now process the return"
expectedTopic: <topic_name>
conversationHistory:
- role: user
message: "I need help with order #12345"
- role: agent
topic: <previous_topic>
message: "I found your order. It was delivered on March 1st. How can I help?"
expectedActions:
- <return_action_name>
# ═══════════════════════════════════════════════════════════════════
# ESCALATION TESTS
# ═══════════════════════════════════════════════════════════════════
- utterance: "I want to talk to a real person"
expectedTopic: Escalation
# ═══════════════════════════════════════════════════════════════════════
# NOTES — AGENT SCRIPT ACTION TYPES
#
# Agent Script agents (.agent files / AiAuthoringBundle) have TWO types
# of actions that appear in CLI test results:
#
# 1. TRANSITION ACTIONS (from start_agent reasoning.actions):
# - Named: go_<topic_name>
# - Target: @utils.transition to @subagent.<name>
# - Captured by single-utterance tests
#
# 2. BUSINESS ACTIONS (from subagent.actions + reasoning.actions):
# - Named: <action_definition_name> (Level 1 from subagent.actions block)
# - Target: apex://ClassName or flow://FlowName
# - May require conversationHistory to reach in multi-subagent agents
#
# Use expectedActions with the DEFINITION name (Level 1), not the
# invocation name (Level 2). E.g., use get_order_status, not check_status.
# ═══════════════════════════════════════════════════════════════════════
# ═══════════════════════════════════════════════════════════════════════
# NOTES — TOPIC NAME RESOLUTION
#
# The expectedTopic value depends on the topic type:
#
# Standard topics (Escalation, Off_Topic, etc.):
# Use localDeveloperName: "Escalation"
#
# Promoted topics (created in Setup UI, prefixed with p_16j...):
# MUST use the full runtime developerName with hash suffix
#
# To discover actual topic names:
# 1. Run one test with a guess
# 2. Check results JSON: .testCases[].generatedData.topic
# 3. Update expectedTopic with the actual value
# ═══════════════════════════════════════════════════════════════════════
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |