Skip to content

forcedotcom/sf-skills / skills/data360-code-extension-generate

data360-code-extension-generate

Verified

Create, develop, test, and deploy Data 360 (Data Cloud) Code Extensions with the sf CLI data-code-extension plugin and the salesforce-data-customcode Python SDK. Use this skill when creating or writing a code extension script or function (for example one that reads a DLO and writes a DLO), creating custom PySpark transformations for Data 360, editing an existing code extension payload/entrypoint.py, setting up a local code extension environment, scanning, running locally, deploying, or redeploying code extensions. Supports environment setup, init, scan, run, zip, and deploy for script and function packages.

FileKindSize
SKILL.mdskill md17.0 KB
references/README.mdmarkdown3.2 KB
references/environment-setup.mdmarkdown10.5 KB
references/mcp-integration.mdmarkdown3.6 KB
references/quick-reference.mdmarkdown7.1 KB
references/run-logs.mdmarkdown2.3 KB
references/schema-discovery.mdmarkdown2.7 KB
references/troubleshooting.mdmarkdown9.0 KB
scripts/check_env.shexecutablescript10.2 KB
scripts/venv_hook.shexecutablescript888 B

references/troubleshooting.md

View on GitHub
# Troubleshooting Data 360 Code Extensions

| Symptom | Cause | Fix |
|---|---|---|
| `EPERM: operation not permitted, open '~/.sf/sf-<date>.log'`, or the check prints "sf CLI is installed but can't run in this shell" | The agent host's sandbox blocks `sf` from writing its log and config folders. The plugin and org checks then show false ❌s | Rerun the command with the host's permission to run outside the sandbox |
| `PackageNotInstalled: salesforce-data-customcode` on `init` | Phase 1 started before the SDK was installed in this project's Python | Install the SDK and `pipreqs` into `<PROJECT_DIR>/.venv` (or `--user` in system mode), wait for it to finish, rerun `check_env.sh`, then `init`. Don't borrow another project's venv |
| `command data-code-extension not found` | Plugin not installed | `sf plugins install @salesforce/plugin-data-code-extension` |
| `PythonVersionMismatch: ... exactly version 3.11 is required` | `python3` on `PATH` isn't 3.11 | Venv mode: start the same block with `source <PROJECT_DIR>/.venv/bin/activate`. System mode: the default `python3` isn't 3.11, so switch to venv mode (environment-setup.md §3b) |
| `pip install` "worked" but `import datacustomcode` fails | The venv has no pip (created with `uv venv`), so the global pip was used | Recreate with `python3.11 -m venv .venv` and always use `python3 -m pip` |
| `uv pip install` hangs | uv network/index issue | Use `python3 -m pip` inside the venv |
| `PipreqsScanError` during `scan` | `pipreqs` not installed, or (system mode) installed with `--user` into a `bin` folder that isn't on `PATH` | `python3 -m pip install [--user] pipreqs`, and start the block with `<PY_ENV>` (system mode adds the user-base `bin` folder, for example `~/.local/bin`, to `PATH`). `check_env.sh` prints the exact install command. Then rerun the same `scan`; don't switch to `--no-requirements` to get past it |
| `PYTHON_VERSION_MISMATCH ... worker (3, x) ... driver 3.11` | Venv mode: Spark workers use the global `python3` because the PySpark hook is missing (for example, the venv was recreated) | `bash <SKILL_DIR>/scripts/venv_hook.sh <PROJECT_DIR>`, then activate the venv again |
| `error: externally-managed-environment` on `pip install` | System mode on a PEP 668 distro (Debian 12+, Ubuntu 23.04+) | Use venv mode. Don't use `--break-system-packages` |
| Plugin finds the SDK, but `run` fails with `ModuleNotFoundError: datacustomcode` | System mode: SDK installed with a `pip3` that belongs to another Python. The plugin accepts the package from any pip | Reinstall with `python3 -m pip install --user salesforce-data-customcode` |
| `python3 -m pip`: `No module named pip` | System mode: the 3.11 interpreter has no pip | `apt install python3-pip` (user action, sudo), or `python3 -m ensurepip --user` |
| `UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available` during local `run` | JDK 21 with PySpark 3.5: Arrow `toPandas()` (and likely pandas UDFs) isn't supported | Use JDK 17 (`JAVA_HOME`), or set `spark.sql.execution.arrow.pyspark.enabled` to `false` before `toPandas()`. JVM flags don't fix it. See environment-setup.md §Java |
| `Java gateway process exited` / `JAVA_HOME is not set` | No JDK found | Install JDK 17 (see environment-setup.md §Java). On macOS, check with `/usr/libexec/java_home -V` |
| `Cannot read from DLO and DMO in the same file` | Entry point reads both layers | Read only DLOs or only DMOs |
| `Cannot read from DLO and write to DMO in the same file` (or the reverse) | Mixed layers | DLO→DLO or DMO→DMO. Split the logic into separate code extensions |
| `Must read from at least one DLO or DMO` | No `read_dlo` / `read_dmo` call `scan` can detect | Call `client.read_dlo("<name>")` with a string literal |
| `AttributeError: 'DataFrame' object has no attribute 'str'` or similar | pandas syntax used on a PySpark DataFrame | Use PySpark (`withColumn`, `upper(col(...))`), or `.toPandas()` for small data only |
| `run` fails with `table does not exist` | DLO name doesn't exist in this org (for example the stock `Account_std__dll`) | Use real DLO API names (SKILL.md Phase 2) |
| `run` succeeds but the target DLO stays empty | Local run uses `PrintDataCloudWriter` | Expected. Deploy, then run the data transform |
| `run` exits 1 with no traceback | The command had `2>/dev/null`; the traceback goes to stderr | Rerun without redirecting stderr |
| `ValueError: The following columns are not present in the DLO '<output>'` | Code extensions support lowercase column names only. The SDK lowercases the columns it reads (`reader/utils.py`: `column.lower()`), and writes accept only lowercase names, so every listed column has a capital letter, for example `KQ_id__c`, `DataSource__c`, `DataSourceObject__c`, `InternalOrganization__c`, `cdp_sys_SourceVersion__c` | Leave out the standard ingestion fields. For a field the user needs, add a lowercase field to the target DLO in Data Cloud and write that (SKILL.md Phase 2) |
| `run` succeeds but the printed table has no rows | Filter or join matched nothing: root condition (`isNull()` vs `0`), or Number vs Text join keys (Number reads as `1.0`) | Query the input data with `jq` (schema-discovery.md); cast with `.cast("long").cast("string")` |
| Approval prompt for a query command that parses JSON with `python3 -c` | Agent hosts treat inline interpreters like `eval` | Parse with `jq` instead |
| `run` shows only ~1000 rows | Local reader default `default_row_limit: 1000` | Expected for local testing |
| Wrong org or expired-token error | Command used the CLI default org | Pass `--target-org <ORG_ALIAS>` |
| `Error (2): Nonexistent flag: --target-org` | `init`, `scan` and `zip` don't accept `--target-org` | Remove the flag; only `run` and `deploy` take it |
| The agent host asks for approval on every command ("command-substitution or eval pattern") | Commands contain `$(…)`, backticks or `eval` | Copy the literal `PY_ENV=` and `API_VERSION=` lines printed by `check_env.sh`; send SQL bodies with `--body -` and a quoted heredoc |
| `--json` parse errors from `sf` (`jq: parse error`) | stderr merged into stdout with `2>&1`, so warnings such as "update available" got into the JSON | Remove `2>&1`; use `2>/dev/null` if you want quiet output |
| `jq: error ... Cannot iterate over null` when reading DLO fields | The `ssot/data-lake-objects/<DLO_NAME>` response wraps the object in a `dataLakeObjects` array, so a top-level `.fields` is null | Use `jq '.dataLakeObjects[0] \| {name, fields: [.fields[] \| {name, dataType}]}'` (schema-discovery.md) |
| MCP `search` fails with `JSON_PARSER_ERROR: Unrecognized field "limit"` | The server rejects the `limit` parameter that the tool schema advertises | Call `search` with the query only |
| Agent uses `sf api` although the Data 360 MCP server is connected | MCP tools are listed as deferred (name only) and the agent treated them as unavailable, or the session skipped Phase 0 so the MCP check never ran | Agent Rule 9: check the tool list, including deferred tools, before the first org lookup in every session |
| MCP call fails with `Streamable HTTP error ... 404 ... Session not found` | The MCP server expired the client's session after it sat idle | Retry the call once (many clients reconnect on the next call). If it fails again, use the `sf api` fallback and reconnect or restart the Data 360 MCP server in the agent's MCP settings |
| `sf org login web` fails | Wrong instance URL | Use the org's My Domain URL, or `https://test.salesforce.com` for sandboxes |
| `deploy` fails with a feature/permission error | Code Extension not enabled, or missing Data 360 permissions | User enables **Data Cloud Setup → Feature Manager → Code Extension** (UI only), and checks the user's Data 360 permissions |
| `deploy` fails with a Docker error | Docker Desktop not running | Start Docker Desktop. Check with `docker info` |
| `deploy` fails, package not found / wrong layout | `--package-dir` points at the package root | `--package-dir ./payload` |
| `deploy` fails with `DeploymentExists` | A code extension with that `--name` already exists. A new `--package-version` doesn't help | Delete the data transform, then the code extension, then redeploy (confirm with the user first) |
| `deploy` fails with `Output objects ... are already in use` | Another data transform already writes to that DLO | Delete that transform, or write to a different DLO |
| Deployed transform runs but output is wrong or missing | Schema mismatch between written columns and the output DLO | Compare the DataFrame columns with the DLO fields (Phase 2). Check the run history in Data Transforms or with MCP `d360_code_extension_executions_list`, and read the run's messages in `DataCustomCodeLogs__dll` (run-logs.md) |
| Deployed run fails | An exception in the code | Query `DataCustomCodeLogs__dll` for the latest `ExecutionId__c` of the code extension and read the traceback in `Message__c` (run-logs.md) |
| `function run` asks for `--test-with` | Functions are tested with input JSON, not org data | `--test-with ./payload/tests/test.json` (see `example/*/tests/test.json`) |
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall