forcedotcom/sf-skills / skills/data360-code-extension-generate
data360-code-extension-generate
VerifiedCreate, develop, test, and deploy Data 360 (Data Cloud) Code Extensions with the sf CLI data-code-extension plugin and the salesforce-data-customcode Python SDK. Use this skill when creating or writing a code extension script or function (for example one that reads a DLO and writes a DLO), creating custom PySpark transformations for Data 360, editing an existing code extension payload/entrypoint.py, setting up a local code extension environment, scanning, running locally, deploying, or redeploying code extensions. Supports environment setup, init, scan, run, zip, and deploy for script and function packages.
data360-code-extension-generate
Overview
A Data 360 code extension is customer-owned Python (PySpark) that runs inside Data 360:
| Type | Purpose | Local test |
|---|---|---|
| script | Batch transformation deployed as a data transform. Reads and writes DLOs or DMOs. | script run against real org data |
| function | Custom node for Search Index Chunking (--use-in-feature SearchIndexChunking). | function run --test-with test.json |
Workflow: environment check → init → discover schemas → develop → scan → run locally → deploy → run the deployed transform. The skill needs only a shell, file editing, and the ability to ask the user questions. It works with a project venv or with a single system Python 3.11 (cloud workspaces, containers).
Use it to set up a code extension environment, create a script or function package, change payload/entrypoint.py, scan, run locally, deploy, redeploy, or clean up.
Agent Rules (apply to every phase)
- Pass
--target-org <ORG_ALIAS>explicitly torun,deployand everysf org/sf apicommand, even when it's the CLI default.init,scanandzipreject it (Nonexistent flag: --target-org). - Shell state doesn't persist. Start every block that runs
sf data-code-extension,python3orpipwith the<PY_ENV>line from Phase 0b; other commands don't need it. Write<ORG_ALIAS>,<PROJECT_DIR>,<API_VERSION>and<package_name>literally. Never use$(…), backticks oreval: some hosts (for example Code Builder) ask for approval on each one. CopyPY_ENVandAPI_VERSIONexactly ascheck_env.shprints them. - Check before you install, and skip anything already present.
- Confirm with the user before changing the org: deploy, delete, or running a deployed transform.
- Hand interactive or privileged steps to the user:
sf org login web, enabling Code Extension, installing or starting Docker Desktop,sudo, installing Homebrew. - Never print secrets (access tokens, session IDs, OAuth client IDs or consumer keys).
- Resolve, print, proceed. Don't ask for values the environment determines (Phase 0a). Ask only when a value is missing or ambiguous. If your shell can't prompt, ask in the conversation and write the files yourself.
- Schema first, then code (Phase 2). Never write code first and ask the user to check the schema afterwards.
- Data 360 MCP server first for org lookups (schemas, queries, runs, deletes). Do the MCP check (Phase 0b) once per session before the first lookup, even if Phase 0 was skipped. Fall back to
sf apior the UI only when it isn't connected or a call fails, and say which you used. OnSession not found(404), retry once before falling back. Details: references/mcp-integration.md. - stderr: when piping
sf … --jsonintojq, use2>/dev/null, never2>&1(warnings break the JSON). Never redirect stderr onscan,runordeploy; their errors and tracebacks go there.
Phase 0: Environment and Org Readiness
0a. Resolve parameters
Take what the user already gave and pass "" for the rest; check_env.sh (0b) resolves it:
| Parameter | Resolution order (first match wins) |
|---|---|
PROJECT_DIR | User value → $D360_PROJECT_DIR → /home/codebuilder/dx-project if it exists (Code Builder) → current folder |
ORG_ALIAS | User value → $D360_TARGET_ORG → CLI default org if Connected → the only Connected org. Alias, or username if there's no alias |
<package_name> | User value, else a snake_case name from the request (for example employee_upper) |
Tell the user the values in one line and continue without waiting ("Using org … and project …. Say so if you want different ones."). Stop and ask only if several orgs are Connected with no Connected default (show them), or if none is Connected (ask for the My Domain URL and hand the browser login to the user: environment-setup.md §2).
0b. Run the read-only check
The plugin needs exactly Python 3.11 and uses whatever python3 is on PATH. The check picks system mode (python3 is 3.11 and pip isn't blocked; <PY_ENV> puts the user-base bin folder on PATH) or venv mode (<PROJECT_DIR>/.venv; <PY_ENV> activates it). An existing .venv always means venv mode. Details: references/environment-setup.md §3.
bash <SKILL_DIR>/scripts/check_env.sh "<PROJECT_DIR or empty>" "<ORG_ALIAS or empty>" # optional 3rd argument: system | venv
It changes nothing. It prints ✅/❌ for sf and the plugin, Python 3.11 (after <PY_ENV>), pip or venv state, salesforce-data-customcode and pipreqs, Java, Docker and the org connection, then PROJECT_DIR=, ORG_ALIAS=, MODE=, PY_ENV= and API_VERSION=. Use the printed PY_ENV and API_VERSION from here on.
For each ❌, follow environment-setup.md in this order, then rerun the check:
sfcan't run in this shell: your host's sandbox blocks it. Rerun with permission to run outside the sandbox; until then, ignore the other ❌s.- Venv mode: create the venv with
python3.11 -m venv .venv(notuv venv), then runbash <SKILL_DIR>/scripts/venv_hook.sh "<PROJECT_DIR>". - Install the SDK and
pipreqsin one command withpython3 -m pip(never barepip3), as the check prints it. The first install downloads PySpark (about 300 MB); let it finish.
Don't start Phase 1 until every ❌ except Docker (deploy only) is fixed. If an install fails (for example, no network), stop and tell the user. Use only this project's Python; never borrow another project's venv. Java: JDK 17 is recommended (11 works); JDK 21 works except for toPandas() and pandas UDFs (environment-setup.md §Java).
MCP check (once, here). Look at your own tool list, including deferred tools listed by name only, for a server whose name contains d360 or data360 and whose tools end in search, payload_examples and execute. Load them if your host requires it, then print MCP: available (<server>) or MCP: not connected. Reuse this answer for the rest of the session.
0c. Enable Code Extension (manual, UI only)
There's no API or CLI for this. Before deploying, ask the user to turn on Data Cloud Setup → Feature Manager → Code Extension and confirm. The user also needs Data 360 admin permissions.
Phase 1: Initialize
Use script unless the user asks for a function or mentions Search Index Chunking. Print the choice and continue.
<PY_ENV>
cd <PROJECT_DIR>
sf data-code-extension script init --package-dir <package_name>
# function: sf data-code-extension function init --package-dir <package_name> --use-in-feature SearchIndexChunking
The package root <package_name>/ holds payload/entrypoint.py, payload/config.json (written by scan) and requirements.txt. Run scan, run and deploy from the package root (Command Reference). Full layout: references/quick-reference.md.
The stock entrypoint reads Account_std__dll and writes Account_std_copy__dll, which usually don't exist (table does not exist). Replace them with real objects (Phase 2).
Phase 2: Discover Source and Target Schemas (before writing code)
Do this every time before you write or edit entrypoint.py, including follow-ups such as "update entrypoint.py to read X and write Y". You can reuse schemas already fetched in this session.
- Identify the source and target objects. If one isn't named, ask.
- Get each object's fields and types. If MCP is available you must use it:
payload_examplesford360_dlo_get, thenexecute, one DLO per call. Otherwise use thesf apifallback. Say which you used. Commands and response shapes: references/schema-discovery.md. - If the logic filters or joins, look at about 20 rows of input data first (MCP
d360_query_sql, or the fallback in schema-discovery.md), especially for self-joins and root conditions (null vs0). - Print the field mapping, one line per target column:
<target column> ← <source field or expression>. Then write the code.
Mapping rules:
- Take column names from the target schema, not from the request wording. Match on the same or a similar name first, then on what the request asks for. Never invent a column.
- Lowercase only. The SDK lowercases the columns it reads, and writes accept only lowercase names. Leave out target fields with capitals, such as the standard ingestion fields
KQ_id__c,DataSource__c,DataSourceObject__c,InternalOrganization__candcdp_sys_SourceVersion__c, and say why. If the user needs one, they must add a lowercase field to the target DLO in Data Cloud. - Stop before writing code if a DLO doesn't exist, if no target column can hold a requested value, or if the user named a column the target doesn't have. If several columns fit, ask. Offer the options (add the field in Data Cloud, use another column or DLO, change the logic). Never create or change DLOs yourself.
Checklist before Phase 3:
- Mapping printed; every read and write object and every field exists
- The code will end with a
select()of lowercase target columns only - Join keys have the same type. A Number reads as
1.0, so cast it before comparing with Text:F.col("<key>").cast("long").cast("string") - No other data transform writes to the output DLO
Phase 3: Develop the Transformation
Write the code from the Phase 2 mapping, using only names it returned. read_dlo and read_dmo return PySpark DataFrames, not pandas.
from pyspark.sql.functions import col, upper
from datacustomcode.client import Client
from datacustomcode.io.writer.base import WriteMode
def main():
client = Client()
df = client.read_dlo("<SOURCE_DLO>")
# One expression per target column, in the order of the Phase 2 mapping.
result = df.select(
col("<source_field_a>").alias("<target_column_a>"),
upper(col("<source_field_b>")).alias("<target_column_b>"),
)
client.write_to_dlo("<TARGET_DLO>", result, WriteMode.OVERWRITE)
if __name__ == "__main__":
main()
SDK rules (scan enforces them): read DLOs or DMOs, never both; DLO→DLO or DMO→DMO only; at least one read, with the object name as a string literal; the output DLO must exist and match the columns you write.
Use WriteMode.OVERWRITE for a full recompute. APPEND, MERGE and MERGE_UPSERT_DELETE are also available; OVERWRITE_PARTITIONS raises an error. More patterns, write modes and the optional SDK helpers (LLM text generation, Einstein predictions, bundled files): references/quick-reference.md. For a function, implement function(request: SearchIndexChunkingV1Request, runtime: Runtime) -> SearchIndexChunkingV1Response from the generated template.
Phase 4: Scan
<PY_ENV>
cd <PROJECT_DIR>/<package_name>
sf data-code-extension script scan --entrypoint ./payload/entrypoint.py
scan rewrites payload/config.json (read and write permissions) and requirements.txt (with pipreqs). Rerun it after every code change, and check that permissions.read and permissions.write list exactly the Phase 2 objects. On PipreqsScanError, install pipreqs and rerun. Don't use --no-requirements unless the user asks: the deployed transform would be missing packages.
Phase 5: Run Locally
<PY_ENV>
cd <PROJECT_DIR>/<package_name>
sf data-code-extension script run --entrypoint ./payload/entrypoint.py --target-org <ORG_ALIAS>
# function: sf data-code-extension function run --entrypoint ./payload/entrypoint.py --test-with ./payload/tests/test.json
runreads real org data (1,000 rows per read) and prints the output instead of writing it, so the target DLO stays empty. That's expected.- An empty printed table usually means the filter or join matched nothing. Check the input data (Phase 2) before changing the code.
- On Java 21, the
Arrow optimization ... DirectByteBufferUserWarningis harmless.
Phase 6: Deploy
This changes the org. Confirm with the user first. Docker must be running (docker info), and Code Extension must be enabled (0c).
<PY_ENV>
cd <PROJECT_DIR>/<package_name>
sf data-code-extension script deploy \
--target-org <ORG_ALIAS> \
--name <name> \
--package-dir ./payload \
--package-version 0.0.1 \
--description "<description>" \
[--cpu-size CPU_L|CPU_XL|CPU_2XL|CPU_4XL]
-dis--description, not the directory.--package-diris always./payload, never the package root.--cpu-sizedefaults toCPU_2XL.function deploytakes the same flags.- Redeploy: a second
deploywith the same--namefails withDeploymentExists, even with a new--package-version. Delete the data transform, then the code extension (UI or MCP, confirming each delete), then deploy again. - An output DLO can belong to only one data transform (
Output objects ... are already in use).
Phase 7: Run the Deployed Transform and Verify
Writing to the DLO happens only when the deployed transform runs. Confirm with the user first. Start it from Data Cloud → Data Transforms → <name> → Run Now (runs take about 10 minutes), or with the MCP transform tools. Then check the output DLO in the UI or with an MCP query.
After every run, successful or not, read the run's messages and tracebacks in the system DLO DataCustomCodeLogs__dll: filter on DataCustomCodeName__c = '<name>' and take the latest ExecutionId__c. The query and how to read the rows: references/run-logs.md. To trace a deployed run, add print() or logging calls and redeploy.
Cleanup: delete the data transform, then the code extension, confirming each delete with the user.
Error Handling (most common)
| Error | Fix |
|---|---|
PythonVersionMismatch / PYTHON_VERSION_MISMATCH | Start the block with <PY_ENV>. Venv mode: rerun scripts/venv_hook.sh <PROJECT_DIR> |
PipreqsScanError | Install pipreqs with python3 -m pip, start the block with <PY_ENV>, and rerun scan |
table does not exist | Wrong DLO name (for example the stock Account_std__dll). See Phase 2 |
ValueError: The following columns are not present in the DLO | The listed columns have capital letters. Apply the lowercase rule (Phase 2) |
DeploymentExists | Delete the transform, then the code extension, then redeploy |
| Deployed run fails or writes nothing | Read DataCustomCodeLogs__dll (Phase 7) |
Everything else: references/troubleshooting.md.
Command Reference
| Command | Run from | Required flags | Optional flags |
|---|---|---|---|
script init / function init | Parent folder | -p/--package-dir <package_name> | function: -u/--use-in-feature SearchIndexChunking |
script scan / function scan | Package root | (-e/--entrypoint ./payload/entrypoint.py) | -d/--dry-run, -n/--no-requirements, --config-file |
script run | Package root | -e/--entrypoint, -o/--target-org | --config-file, --dependencies |
function run | Package root | -e/--entrypoint, -t/--test-with | --target-org, --config-file, --dependencies |
script zip / function zip | Package root | -p/--package-dir | -n/--network |
script deploy / function deploy | Package root | -n/--name, --package-version, -d/--description, -p/--package-dir ./payload, -o/--target-org | --cpu-size, --network (Jupyter packages only) |
Verified against plugin data-code-extension 1.4.1 and SDK salesforce-data-customcode 6.1.0. If the plugin version differs, check with sf data-code-extension <type> <command> --help.
Resources
- SF CLI plugin: https://github.com/salesforcecli/plugin-data-code-extension
- Python SDK: https://github.com/forcedotcom/datacloud-customcode-python-sdk (PyPI:
salesforce-data-customcode) - Hosted MCP servers (optional setup): https://developer.salesforce.com/docs/platform/hosted-mcp-servers/overview
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |