PranavNagrecha/AwesomeSalesforceSkills / skills/admin/data-import-and-management
data-import-and-management
VerifiedUse when planning, reviewing, or troubleshooting Salesforce data imports, migrations, and bulk updates. Triggers: 'Data Loader', 'Data Import Wizard', 'Bulk API', 'upsert', 'cutover', 'load failed', 'reconcile data'. NOT for sequencing a whole multi-object migration with automation bypass and rollback — use data/data-migration-planning. NOT for choosing the upsert key itself — use data/external-id-strategy. Also triggers on 'process-conf.xml', 'sdl mapping file', 'sfdc.loadBatchSize', 'insert null values', 'insertNulls', '#N/A', 'serial mode', 'hard delete', 'assignmentRuleId', 'lineEnding', 'unprocessedrecords', 'failedResults', 'load order'.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 19.3 KB |
| references/examples.md | markdown | 3.4 KB |
| references/gotchas.md | markdown | 21.5 KB |
| references/llm-anti-patterns.md | markdown | 10.1 KB |
| references/metadata-examples.md | markdown | 23.5 KB |
| references/well-architected.md | markdown | 4.6 KB |
| scripts/check_load_plan.py | script | 17.0 KB |
| templates/data-load-plan-template.md | markdown | 6.6 KB |
references/well-architected.md
View on GitHub# Well-Architected Mapping: Data Import and Management
## Pillars Addressed
### Reliability
Good load design prevents duplicates, broken relationships, and partial cutovers.
- External IDs and upsert design preserve identity across reruns.
- Reconciliation steps catch silent failures before users depend on bad data.
### Scalability
Bulk tooling and batch design determine whether a load works at real production volume.
- Bulk API and chunking patterns prevent admin tooling from becoming the bottleneck.
- Load-order design reduces lock contention and failed retries.
### Operational Excellence
Runbooks, rollback plans, and rehearsals separate controlled migrations from heroic recovery work.
- Clear cutover ownership and checkpointing make failures diagnosable.
- Explicit automation and duplicate-rule handling reduce surprise during release windows.
## Pillars Not Addressed
- **Security** - this skill touches permissions only as a prerequisite for running loads, not as a security-design pattern.
- **User Experience** - the concern here is cutover safety, not end-user page behavior.
## Official Sources Used
- **Data Loader Guide** (https://resources.docs.salesforce.com/262/latest/en-us/sfdc/pdf/salesforce_data_loader.pdf) — Settings reference for import batch size, Insert null values, and Assignment rule (the "Turning On Bulk API Silently Changes What Batch Size Means", "Blanks Do Not Clear Fields", and "Assignment Rule Overrides OwnerId" gotchas); the `process-conf.xml` sample bean, the process configuration parameter table, and the `.sdl` mapping syntax behind `references/metadata-examples.md` §2-3; `encrypt.bat` / `process.bat` usage; hard-delete prerequisites and the Bulk API Hard Delete permission; date formats, `sfdc.timezone`, and the GMT recommendation; Data Import Wizard vs Data Loader selection thresholds (under 50,000 records, fewer than 50 fields)
- **Bulk API 2.0 and Bulk API Developer Guide** (https://resources.docs.salesforce.com/262/latest/en-us/sfdc/pdf/api_asynch.pdf) — job state machine Open → UploadComplete → InProgress → JobComplete/Failed/Aborted; the create-job request body (`operation`, `externalIdFieldName`, `lineEnding`, `columnDelimiter`, `assignmentRuleId`, `contentType`); the required UploadComplete PATCH; 10,000-record automatic batching, the 5-minute batch budget and 20 automatic retries; the `successfulResults` / `failedResults` / `unprocessedrecords` resources and the failed-vs-unprocessed distinction; parent-lookup-by-External-ID CSV column headers and their indexing constraint; the 300 multiple-match upsert failure
- **Salesforce Developer Limits and Allocations Quick Reference** (App Limits Cheat Sheet) — the numeric allocations quoted in the Decision Matrix and gotchas: 15,000 batches per rolling 24 hours shared across Bulk API and Bulk API 2.0; 150,000,000 records per 24 hours; 10 MB per Bulk API batch and 150 MB (base64) per Bulk API 2.0 job with a 100 MB practical ceiling; 7-day results lifespan; 24-hour maximum open job; the 2,000-record threshold above which Bulk API 2.0 is the right tool; 131,072 characters per field and 400,000 per record
- **Apex Developer Guide** (https://resources.docs.salesforce.com/262/latest/en-us/sfdc/pdf/salesforce_apex_developer_guide.pdf) — API 21.0+ Bulk API trigger chunking: governor limits reset between the 200-record trigger invocations of one request but static variables do not, which is the grounding for the "Static Variables Are Not Reset" gotcha and for the Proactive Trigger on static recursion guards
- **Metadata API Developer Guide** (https://resources.docs.salesforce.com/262/latest/en-us/sfdc/pdf/api_meta.pdf) — the `CustomField` property table: `externalId` (returned only for AutoNumber, Email, Number, Text), `unique` and `caseSensitive` as independent booleans, and `length` for Text — the External ID field XML in `references/metadata-examples.md` §1 and the non-unique-upsert gotcha
- **Object Reference for the Salesforce Platform** (https://resources.docs.salesforce.com/262/latest/en-us/sfdc/pdf/object_reference.pdf) — `AssignmentRule` (read-only, `Active`, `SobjectType`, and the note that assignment rules can be specified when creating or upserting Cases and Leads via Bulk API) and `DuplicateRule` (`IsActive`, `sObjectType`, and the View Setup and Configuration requirement from Summer '20) behind the pre-load SOQL in `references/metadata-examples.md` §6
- **REST API Developer Guide** — sObject Collections (`/services/data/vXX.X/composite/sobjects`) as the synchronous alternative below the Bulk threshold: up to 200 records per upsert call against an external ID, `allOrNone` rollback semantics, and HTTP 300 when an external ID matches more than one record
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies. | Stars: 108k | Updated | Claude CodeCodex | Verified | Install |
| meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要". | Stars: 108k | Updated | Claude CodeCodex | Verified | Install |