Skip to content

opensite-ai/opensite-skills / aws-ai-ml

aws-ai-ml

Verified

Selects, deploys, and customizes AI models on Amazon SageMaker. Training or Processing jobs, fine-tuning (SFT/DPO/RLVR/RLAIF), model selection, dataset preparation, evaluation, SageMaker or Bedrock deployment, inference optimization and endpoint diagnostics. Covers the full lifecycle from planning through production. Use when fine-tuning models on SageMaker, choosing which base model to customize, fine-tune, or deploy from SageMaker JumpStart or Hub, SageMakerPublicHub, or the SageMaker public model catalog, transforming or validating training data, evaluating model quality, deploying or optimizing endpoints, configuring IAM/S3 for training, or managing SageMaker Managed MLflow. Use for endpoint health, failures, latency, logs, metrics, errors. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3. NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure.

FileKindSize
.aws-skill-metadataother18 B
SKILL.mdskill md8.1 KB
references/dataset-evaluation/overview.mdmarkdown5.4 KB
references/dataset-evaluation/references/custom-scorer-evaluation-dataset-formats.mdmarkdown3.6 KB
references/dataset-evaluation/references/strategy_data_requirements.mdmarkdown4.8 KB
references/dataset-evaluation/scripts/format_detector.pyscript26.7 KB
references/dataset-transformation/code_templates/transformation.pyscript1.5 KB
references/dataset-transformation/overview.mdmarkdown13.3 KB
references/dataset-transformation/references/code_output_guide.mdmarkdown2.8 KB
references/dataset-transformation/references/dataset_transformation_code.mdmarkdown5.2 KB
references/dataset-transformation/references/sagemaker_dataset_formats.mdmarkdown2.9 KB
references/dataset-transformation/scripts/transformation_tools.pyscript4.8 KB
references/directory-management/overview.mdmarkdown1.9 KB
references/endpoint-diagnostics/overview.mdmarkdown3.5 KB
references/endpoint-diagnostics/references/minimum_iam_policy.mdmarkdown1.2 KB
references/endpoint-diagnostics/scripts/collect_diagnostics.pyscript12.8 KB
references/finetuning-technique/overview.mdmarkdown2.1 KB
references/finetuning-technique/references/finetune_technique_selection_guide.mdmarkdown2.3 KB
references/finetuning-technique/scripts/get_recipes.pyscript817 B
references/finetuning/code_templates/dpo.pyscript6.4 KB
references/finetuning/code_templates/rlaif_builtin.pyscript5.8 KB
references/finetuning/code_templates/rlaif_custom_prompt.pyscript6.3 KB
references/finetuning/code_templates/rlvr.pyscript6.5 KB
references/finetuning/code_templates/sft.pyscript6.1 KB
references/finetuning/overview.mdmarkdown11.4 KB
references/finetuning/references/code_output_guide.mdmarkdown3.2 KB
references/finetuning/references/continuous_customization.mdmarkdown8.4 KB
references/finetuning/references/eula_links.mdmarkdown7.3 KB
references/finetuning/references/rlaif_guide.mdmarkdown3.8 KB
references/finetuning/references/rlvr_reward_function.mdmarkdown10.0 KB
references/finetuning/scripts/mlflow_reference.pyscript841 B
references/finetuning/templates/nova_rlvr_reward_function_source_template.pyscript12.8 KB
references/finetuning/templates/rlvr_reward_function_source_template.pyscript9.1 KB
references/manage-mlflow/overview.mdmarkdown4.8 KB
references/manage-mlflow/references/app-creation-workflow.mdmarkdown3.2 KB
references/manage-mlflow/references/app-deletion-workflow.mdmarkdown2.3 KB
references/manage-mlflow/references/app-discovery-workflow.mdmarkdown1.5 KB
references/manage-mlflow/references/app-update-workflow.mdmarkdown1.7 KB
references/manage-mlflow/references/sm-mlflow-guide.mdmarkdown1.3 KB
references/model-deployment/code_templates/deploy-jumpstart-sagemaker.pyscript7.5 KB
references/model-deployment/code_templates/deploy-nova-bedrock.pyscript2.0 KB
references/model-deployment/code_templates/deploy-nova-sagemaker.pyscript1.9 KB
references/model-deployment/code_templates/deploy-oss-bedrock.pyscript4.0 KB
references/model-deployment/code_templates/deploy-oss-sagemaker.pyscript2.6 KB
references/model-deployment/code_templates/optimize-benchmark.pyscript4.6 KB
references/model-deployment/code_templates/optimize-huggingface-to-s3.pyscript2.0 KB
references/model-deployment/code_templates/optimize-recommendation.pyscript6.4 KB
references/model-deployment/overview.mdmarkdown16.6 KB
references/model-deployment/references/code_output_guide.mdmarkdown3.2 KB
references/model-deployment/references/deploy-jumpstart-sagemaker.mdmarkdown14.3 KB
references/model-deployment/references/deploy-nova-bedrock.mdmarkdown4.4 KB
references/model-deployment/references/deploy-nova-sagemaker.mdmarkdown5.8 KB
references/model-deployment/references/deploy-oss-bedrock.mdmarkdown5.2 KB
references/model-deployment/references/deploy-oss-sagemaker.mdmarkdown12.9 KB
references/model-deployment/references/model-licenses.mdmarkdown11.9 KB
references/model-deployment/references/optimization/benchmark-workflow.mdmarkdown12.7 KB
references/model-deployment/references/optimization/benchmarking-guidance.mdmarkdown8.9 KB
references/model-deployment/references/optimization/huggingface-to-s3.mdmarkdown3.1 KB
references/model-deployment/references/optimization/interpreting-results.mdmarkdown8.5 KB
references/model-deployment/references/optimization/recommendation-deploy.mdmarkdown1.5 KB
references/model-deployment/references/optimization/recommendation-options.mdmarkdown3.1 KB
references/model-deployment/references/optimization/recommendation-workflow.mdmarkdown10.3 KB
references/model-evaluation/code_templates/custom_scorer_evaluator.pyscript2.5 KB
references/model-evaluation/code_templates/llmaaj_evaluator.pyscript2.8 KB
references/model-evaluation/overview.mdmarkdown5.2 KB
references/model-evaluation/references/code_output_guide.mdmarkdown3.3 KB
references/model-evaluation/references/create-reward-function.mdmarkdown3.0 KB
references/model-evaluation/references/custom-lambda-scorer.mdmarkdown4.4 KB
references/model-evaluation/references/custom-scorer-evaluation.mdmarkdown10.5 KB
references/model-evaluation/references/evaluation-type-guide.mdmarkdown10.1 KB
references/model-evaluation/references/llmaaj-builtin-evaluation.mdmarkdown5.0 KB
references/model-evaluation/references/llmaaj-custom-evaluation.mdmarkdown2.5 KB
references/model-evaluation/references/llmaaj-evaluation.mdmarkdown14.6 KB
references/model-evaluation/references/supported-judge-models.mdmarkdown2.9 KB
references/model-evaluation/scripts/nova_reward_function_source_template.pyscript13.1 KB
references/model-evaluation/scripts/reward_function_source_template.pyscript9.2 KB
references/model-evaluation/scripts/validate_custom_metrics.pyscript3.5 KB
references/model-selection/overview.mdmarkdown10.3 KB
references/model-selection/references/benchmarks/agentic-index.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/coding-index.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/gpqa.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/hle.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/ifbench.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/intelligence-index.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/mmmu-pro.mdmarkdown3.0 KB
references/model-selection/references/benchmarks/tau2.mdmarkdown2.8 KB
references/model-selection/references/model-licenses.mdmarkdown7.3 KB
references/model-selection/references/model-selection.mdmarkdown8.2 KB
references/model-selection/references/select-for-deployment.mdmarkdown19.8 KB
references/model-selection/scripts/filter_deployable_models.pyscript10.1 KB
references/model-selection/scripts/get_deployable_models.pyscript21.7 KB
references/model-selection/scripts/get_model_names.pyscript1.4 KB
references/model-selection/scripts/search_deployable_models.pyscript34.9 KB
references/planning/overview.mdmarkdown10.3 KB
references/planning/references/deploy-base-model-plan.mdmarkdown699 B
references/planning/references/evaluate-first-plan.mdmarkdown1.8 KB
references/planning/references/input-output-contracts.mdmarkdown3.1 KB
references/planning/references/model-customization-plan.mdmarkdown1.5 KB
references/planning/references/region-availability.mdmarkdown1.2 KB
references/planning/references/skill-routing-constraints.mdmarkdown2.7 KB
references/processing-jobs/code_templates/dataset_definition_processing.pyscript4.1 KB
references/processing-jobs/code_templates/framework_processing.pyscript3.0 KB
references/processing-jobs/code_templates/script_processing.pyscript2.9 KB
references/processing-jobs/code_templates/spark_processing.pyscript3.7 KB
references/processing-jobs/overview.mdmarkdown6.7 KB
references/processing-jobs/references/configuration.mdmarkdown4.6 KB
references/processing-jobs/references/troubleshooting.mdmarkdown2.4 KB
references/sdk-getting-started/overview.mdmarkdown643 B
references/sdk-getting-started/references/execution-role-setup.mdmarkdown2.4 KB
references/sdk-getting-started/references/sagemaker-python-sdk-setup.mdmarkdown2.1 KB
references/training-jobs/code_templates/custom_training.pyscript6.2 KB
references/training-jobs/code_templates/jumpstart_training.pyscript5.6 KB
references/training-jobs/code_templates/recipe_training.pyscript5.7 KB
references/training-jobs/overview.mdmarkdown9.1 KB
references/training-jobs/references/configuration.mdmarkdown4.0 KB
references/training-jobs/references/troubleshooting.mdmarkdown4.4 KB
references/use-case-specification/overview.mdmarkdown2.0 KB
references/use-case-specification/references/spec-for-deployment.mdmarkdown3.9 KB
references/use-case-specification/references/spec-for-finetuning.mdmarkdown2.0 KB

references/model-deployment/references/deploy-nova-sagemaker.md

View on GitHub
# Deploy Nova LoRA to SageMaker

## Scenario

- **Model Type**: Nova
- **Fine-tuning Method**: LoRA
- **Deployment Target**: SageMaker Single Model Endpoint
- **Approach**: SageMaker ModelBuilder

## Overview

Deploys a Nova fine-tuned model to a SageMaker endpoint using `ModelBuilder`.

Nova deploys as a model-on-variant (no inference components), so you invoke the endpoint directly without specifying an `InferenceComponentName`.

**Required inputs** (collected in the steps below):

- Training job name
- Instance type
- IAM execution role ARN
- AWS region
- Endpoint name

## Prerequisites

Requires SageMaker Python SDK >= 3.7.0 (installed by Cell 1).

## Workflow

### Important Instructions

- Make sure to use dedicated tools instead of bash commands whenever possible

### Step 1: Gather Training Job Name

The training job name was identified in Step 1 of the main workflow. Confirm you have it.

### Step 2: Determine Instance Type

For this step, you need: **the instance type.**

First, determine the Nova variant from the training job's model package. Use your AWS tool to run `sagemaker describe-training-job` for the training job name and extract the `OutputModelPackageArn` from the response. Then inspect the model package to find the `hub_content_name` (e.g., `nova-textgeneration-micro`).

The following are supported instances by Nova variant (smallest to largest); larger instances support longer context lengths. Treat this as a starting point that may be outdated — instance support changes over time and there is no API that enumerates the supported instance types for a model package. Pick from the list below; if the deploy call rejects the instance type, first try the next larger size within the same family (e.g. ml.g5.12xlarge → ml.g5.24xlarge), and if no larger size is available, escalate to the next family (g5 → g6 → p5).

Nova Micro (`nova-textgeneration-micro`): ml.g5.12xlarge, ml.g5.24xlarge, ml.g6.12xlarge, ml.g6.24xlarge, ml.g6.48xlarge, ml.p5.48xlarge

Nova Lite (`nova-textgeneration-lite`): ml.g6.48xlarge, ml.p5.48xlarge

Nova Lite v2 (`nova-textgeneration-lite-v2`): ml.p5.48xlarge

Nova Pro (`nova-textgeneration-pro`): ml.g6.48xlarge, ml.p5.48xlarge

Present the supported instance types and ask which one the user would like to use. The larger instances will be more expensive, but have larger context windows.

⏸ Wait for user to confirm before moving on.

### Step 3: Verify IAM Role

Use the IAM role from the training job (extracted in Step 1 of the main workflow via `describe-training-job`). This role should already have the necessary SageMaker and S3 permissions. Confirm with the user.

### Step 4: Confirm Region

The region was identified in Step 1 of the main workflow. Nova deployment is supported in: us-east-1, us-west-2, eu-west-2, ap-northeast-1. There is no API that returns feature-level region availability for Nova, so this list may be outdated — treat it as guidance. If the user's region is not listed, warn them it is likely unsupported; if a deploy is attempted in an unlisted region and fails with a region/availability or access error, tell the user Nova SageMaker deployment is not available in that region.

### Step 5: Choose Endpoint Name

Suggest a name based on the model, e.g., `nova-micro-deploy-<timestamp>`. Ask the user to confirm or provide their own.

⏸ Wait for user before moving on.

### Step 6: Confirm Configuration

> "Here's the deployment setup:
>
> - Model: [base-model-name] fine-tuned with LoRA (e.g., "Nova Micro fine-tuned with LoRA")
> - Deployment target: SageMaker Endpoint
> - Training Job: [name]
> - Instance Type: [type]
> - IAM Role: [arn]
> - Region: [region]
> - Endpoint Name: [name]
>
> Does this look right?"

⏸ Wait for user approval.

### Step 7: Generate Code

Read `../references/code_output_guide.md` for output format rules.

If a project directory already exists (from earlier in the workflow), use it. Otherwise, load the **directory-management** reference to set one up.

⏸ Wait for user.

## Code Structure

### Markdown Header

```json
{
  "cell_type": "markdown",
  "metadata": {},
  "source": [
    "# Deploy Nova Fine-Tuned Model to SageMaker"
  ]
}

```

### Cells

Each cell's content comes from `../code_templates/deploy-nova-sagemaker.py`, split on the `# Cell N:` comments. Each marker starts a new notebook cell — everything between one marker and the next becomes that cell's content.

- **Cell 1**: Setup (pip install)
- **Cell 2**: Configuration
- **Cell 3**: Build Model
- **Cell 4**: Deploy Endpoint
- **Cell 5**: Test Inference
- **Cell 6**: Save Manifest

### Placeholders

Cell 2:

- `[REGION]` → AWS region
- `[TRAINING_JOB_NAME]` → Training job name
- `[ROLE_ARN]` → IAM execution role ARN
- `[INSTANCE_TYPE]` → SageMaker instance type (e.g., `ml.g5.12xlarge`)
- `[ENDPOINT_NAME]` → Endpoint name

## Step 8: Provide Run Instructions

```
To run:
1. Cell 1 — install SDK packages, then restart the kernel before continuing
2. Cell 2 — set configuration values
3. Cell 3 — build model via ModelBuilder (~30s, creates SageMaker Model resource)
4. Cell 4 — deploy endpoint (waits for InService, ~10-15 min)
5. Cell 5 — test inference with a sample prompt
6. Cell 6 — saves the deployment manifest to `manifests/deploy-<endpoint-name>.json`

```

## Common Issues

- **"No module named 'sagemaker.core'" or "No module named 'sagemaker.train'"**: Re-run Cell 1 to install the required packages, then restart the kernel.
- **"Must setup local AWS configuration with a region"**: Set `AWS_DEFAULT_REGION` env var or configure `~/.aws/config`
- **"Cannot create already existing endpoint configuration"**: An endpoint with that name already exists. Use a different name or delete the existing one first.
- **Endpoint fails to reach InService**: Check CloudWatch logs for the endpoint. Common causes: wrong instance type for the model size, or IAM role missing permissions.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall