datarobot-oss/datarobot-agent-skills / skills/datarobot-app-framework-cicd
datarobot-app-framework-cicd
FlaggedGuidance for setting up CI/CD pipelines for DataRobot application templates using GitLab, GitHub Actions, and Pulumi for infrastructure as code. Use when setting up CI/CD pipelines, configuring deployments, or managing infrastructure for DataRobot application templates.
| File | Kind | Size |
|---|---|---|
| QUICK_REFERENCE.md | markdown | 4.0 KB |
| README.md | markdown | 6.6 KB |
| SKILL.md | skill md | 19.6 KB |
| examples/github-cd-pulumi-cloud.yml | config | 1.1 KB |
| examples/github-deploy-pulumi-cloud.yml | config | 1.3 KB |
| examples/github-destroy-pulumi-cloud.yml | config | 1.2 KB |
| examples/workflows-README.md | markdown | 3.0 KB |
| references/example-taskfile.yaml | config | 895 B |
| references/workflow-examples.md | markdown | 2.5 KB |
| scripts/decrypt-secrets.shexecutable | script | 1.2 KB |
| scripts/encrypt-secrets.shexecutable | script | 1.9 KB |
| scripts/github-cd.yml | config | 2.0 KB |
| scripts/github-deploy.yml | config | 5.0 KB |
| scripts/github-destroy.yml | config | 1.9 KB |
| scripts/gitlab-ci.yml | config | 4.7 KB |
| scripts/infra-README.md | markdown | 6.4 KB |
| scripts/pulumi-setup.shexecutable | script | 10.8 KB |
| scripts/setup-github-secrets.shexecutable | script | 3.6 KB |
| scripts/setup-gitlab-variables.shexecutable | script | 4.4 KB |
| scripts/taskfile-snippets.yaml | config | 5.6 KB |
README.md
View on GitHub# DataRobot Application Templates CI/CD Skill
A skill that provides comprehensive guidance for setting up production-grade CI/CD pipelines for DataRobot application templates.
## Overview
Transform your DataRobot application templates from manual deployments to automated CI/CD workflows with:
- **Automated Testing**: Run linters and tests on every pull/merge request
- **Review Deployments**: Spin up full application stacks for PR validation
- **Continuous Delivery**: Automatically deploy changes when merged to main
- **Infrastructure as Code**: Use Pulumi for declarative infrastructure management
- **Secrets Management**: Secure handling of API keys and credentials
- **Multi-Platform Support**: Works with both GitLab and GitHub
## What's Included
### Documentation
- **SKILL.md**: Complete guidance for CI/CD setup
- Quick start guide
- Platform-specific configurations (GitLab, GitHub)
- Pulumi state management strategies
- Secrets management patterns
- Troubleshooting guide
### Example Configurations
The `scripts/` directory contains reference implementations that should be copied to the application template's `infra/` directory:
- **infra-README.md**: Documentation for the infra/ directory explaining structure and usage
- **gitlab-ci.yml**: Complete GitLab CI/CD pipeline
- Automated testing and linting
- Manual review app deployments
- Continuous delivery on merge
- Azure Blob Storage backend example
- **github-deploy.yml**: GitHub Actions deployment workflow
- Automated testing and linting
- PR-based review deployments
- GPG-encrypted secrets
- Pulumi Cloud backend example
- **github-destroy.yml**: GitHub Actions cleanup workflow
- Manual stack destruction
- Resource cleanup for review apps
- **setup-github-secrets.sh**: Automated GitHub secrets setup
- Uses GitHub CLI (`gh`)
- Interactive secret entry
- Configures all required secrets for Actions
- **setup-gitlab-variables.sh**: Automated GitLab variables setup
- Uses GitLab CLI (`glab`)
- Interactive variable entry
- Configures all required variables for CI/CD
- **encrypt-secrets.sh**: GPG encryption for .env files
- Encrypts root .env to .env.gpg
- Interactive encryption workflow
- GitHub Actions secrets preparation
- Step-by-step instructions
- **decrypt-secrets.sh**: GPG decryption for local development
- Decrypts root .env.gpg to .env
- Safe local secrets management
- Testing workflow simulation
- **taskfile-snippets.yaml**: CI/CD Task definitions
- Secrets management commands
- Pulumi deployment tasks
- CI/CD testing helpers
- Add an `includes` entry in the project's root `Taskfile.yml` pointing to `infra/Taskfile.yaml` (do NOT add tasks to root)
- **pulumi-setup.sh**: Interactive Pulumi setup script
- Backend configuration (Cloud, Azure, AWS)
- Initial stack creation
- Credential management
## Quick Start
When implementing CI/CD for an application template:
1. **Choose your platform**: GitLab or GitHub
2. **Create infra directory and copy scripts**:
```bash
# Create infra directory in project root and copy entire scripts folder
mkdir -p infra
cp -R <skill-path>/scripts infra/scripts
chmod +x infra/scripts/*.sh
# For GitLab - copy to root
cp infra/scripts/gitlab-ci.yml .gitlab-ci.yml
# For GitHub - copy to .github/workflows/
mkdir -p .github/workflows
cp infra/scripts/github-deploy.yml .github/workflows/deploy.yml
cp infra/scripts/github-destroy.yml .github/workflows/destroy.yml
```
3. **Create `infra/Taskfile.yaml`**:
```bash
# Copy taskfile-snippets.yaml to infra/Taskfile.yaml
cp infra/scripts/taskfile-snippets.yaml infra/Taskfile.yaml
# Then add a single includes entry to root Taskfile.yml:
# includes:
# infra:
# taskfile: ./infra/Taskfile.yaml
# dir: .
# ⚠️ Do NOT paste CI/CD tasks into the root Taskfile.yml
```
4. **Set up Pulumi**:
```bash
cd infra/scripts
./pulumi-setup.sh
cd ../..
# Install GitHub CLI: brew install gh
gh auth login
task setup-github-secrets
```
For GitLab (automated):
```bash
# Install GitLab CLI: brew install glab
glab auth login
task setup-gitlab-vars
```
6. **Encrypt your secrets** (for GitHub only):
```bash
# Creates .env.gpg in project root from .env
task encrypt-secrets
git add .env.gpg
git commit -m "Add encrypted secrets"
```
## Key Features
### GitLab CI/CD
- Parallel test execution for faster feedback
- DIY backend support (Azure Blob, S3, GCS)
- MR-specific stack names for isolation
- Automatic commenting with deployment info
- Manual cleanup jobs
### GitHub Actions
- GPG-encrypted secrets for better management
- PR comments with deployment URLs
- Pulumi Cloud integration
- Manual destroy workflows
- Matrix testing support
### Pulumi Integration
- Centralized state management
- Stack isolation per environment
- Idempotent deployments
- Cross-machine synchronization
- Codespace compatibility
## Use Cases
### Development Teams
- Test infrastructure changes in isolation
- Review applications before merging
- Automatically deploy to staging/production
- Track infrastructure state across team
### DevOps Engineers
- Implement IaC for AI applications
- Manage multiple environments
- Automate deployment workflows
- Monitor infrastructure changes
### Data Scientists
- Deploy models with applications
- Test changes in review environments
- Collaborate on application features
- Focus on ML, not infrastructure
## Example Repositories
See these live implementations:
- **GitLab**: [demo-data-agent](https://gitlab.com/datarobot-oss/demo-data-agent)
- **GitHub**: [demo-talk-to-my-data-agent](https://github.com/datarobot-forks/demo-talk-to-my-data-agent)
## Platform Support
- ✅ GitLab CI/CD
- ✅ GitHub Actions
- ✅ Pulumi Cloud
- ✅ Azure Blob Storage
- ✅ AWS S3
- ✅ Google Cloud Storage
## Resources
- [SKILL.md](SKILL.md) - Complete documentation
- [Task](https://taskfile.dev) - Workflow management
- [Pulumi](https://www.pulumi.com/docs/) - Infrastructure as Code
- [DataRobot Application Templates](https://docs.datarobot.com/en/docs/wb-apps/app-templates/index.html)
## Contributing
This is the **canonical upstream source** for this skill at [datarobot-agent-skills](https://github.com/datarobot-oss/datarobot-agent-skills/tree/main/skills/datarobot-app-framework-cicd). Downstream repositories (such as [af-component-base](https://github.com/datarobot-community/af-component-base)) bundle a local copy, when this skill is updated, those copies should be synced.
Contributions welcome via pull request.
## License
See the main repository LICENSE file.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |