Skip to content

opensite-ai/opensite-skills / aws-ai-ml

aws-ai-ml

Verified

Selects, deploys, and customizes AI models on Amazon SageMaker. Training or Processing jobs, fine-tuning (SFT/DPO/RLVR/RLAIF), model selection, dataset preparation, evaluation, SageMaker or Bedrock deployment, inference optimization and endpoint diagnostics. Covers the full lifecycle from planning through production. Use when fine-tuning models on SageMaker, choosing which base model to customize, fine-tune, or deploy from SageMaker JumpStart or Hub, SageMakerPublicHub, or the SageMaker public model catalog, transforming or validating training data, evaluating model quality, deploying or optimizing endpoints, configuring IAM/S3 for training, or managing SageMaker Managed MLflow. Use for endpoint health, failures, latency, logs, metrics, errors. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3. NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure.

FileKindSize
.aws-skill-metadataother18 B
SKILL.mdskill md8.1 KB
references/dataset-evaluation/overview.mdmarkdown5.4 KB
references/dataset-evaluation/references/custom-scorer-evaluation-dataset-formats.mdmarkdown3.6 KB
references/dataset-evaluation/references/strategy_data_requirements.mdmarkdown4.8 KB
references/dataset-evaluation/scripts/format_detector.pyscript26.7 KB
references/dataset-transformation/code_templates/transformation.pyscript1.5 KB
references/dataset-transformation/overview.mdmarkdown13.3 KB
references/dataset-transformation/references/code_output_guide.mdmarkdown2.8 KB
references/dataset-transformation/references/dataset_transformation_code.mdmarkdown5.2 KB
references/dataset-transformation/references/sagemaker_dataset_formats.mdmarkdown2.9 KB
references/dataset-transformation/scripts/transformation_tools.pyscript4.8 KB
references/directory-management/overview.mdmarkdown1.9 KB
references/endpoint-diagnostics/overview.mdmarkdown3.5 KB
references/endpoint-diagnostics/references/minimum_iam_policy.mdmarkdown1.2 KB
references/endpoint-diagnostics/scripts/collect_diagnostics.pyscript12.8 KB
references/finetuning-technique/overview.mdmarkdown2.1 KB
references/finetuning-technique/references/finetune_technique_selection_guide.mdmarkdown2.3 KB
references/finetuning-technique/scripts/get_recipes.pyscript817 B
references/finetuning/code_templates/dpo.pyscript6.4 KB
references/finetuning/code_templates/rlaif_builtin.pyscript5.8 KB
references/finetuning/code_templates/rlaif_custom_prompt.pyscript6.3 KB
references/finetuning/code_templates/rlvr.pyscript6.5 KB
references/finetuning/code_templates/sft.pyscript6.1 KB
references/finetuning/overview.mdmarkdown11.4 KB
references/finetuning/references/code_output_guide.mdmarkdown3.2 KB
references/finetuning/references/continuous_customization.mdmarkdown8.4 KB
references/finetuning/references/eula_links.mdmarkdown7.3 KB
references/finetuning/references/rlaif_guide.mdmarkdown3.8 KB
references/finetuning/references/rlvr_reward_function.mdmarkdown10.0 KB
references/finetuning/scripts/mlflow_reference.pyscript841 B
references/finetuning/templates/nova_rlvr_reward_function_source_template.pyscript12.8 KB
references/finetuning/templates/rlvr_reward_function_source_template.pyscript9.1 KB
references/manage-mlflow/overview.mdmarkdown4.8 KB
references/manage-mlflow/references/app-creation-workflow.mdmarkdown3.2 KB
references/manage-mlflow/references/app-deletion-workflow.mdmarkdown2.3 KB
references/manage-mlflow/references/app-discovery-workflow.mdmarkdown1.5 KB
references/manage-mlflow/references/app-update-workflow.mdmarkdown1.7 KB
references/manage-mlflow/references/sm-mlflow-guide.mdmarkdown1.3 KB
references/model-deployment/code_templates/deploy-jumpstart-sagemaker.pyscript7.5 KB
references/model-deployment/code_templates/deploy-nova-bedrock.pyscript2.0 KB
references/model-deployment/code_templates/deploy-nova-sagemaker.pyscript1.9 KB
references/model-deployment/code_templates/deploy-oss-bedrock.pyscript4.0 KB
references/model-deployment/code_templates/deploy-oss-sagemaker.pyscript2.6 KB
references/model-deployment/code_templates/optimize-benchmark.pyscript4.6 KB
references/model-deployment/code_templates/optimize-huggingface-to-s3.pyscript2.0 KB
references/model-deployment/code_templates/optimize-recommendation.pyscript6.4 KB
references/model-deployment/overview.mdmarkdown16.6 KB
references/model-deployment/references/code_output_guide.mdmarkdown3.2 KB
references/model-deployment/references/deploy-jumpstart-sagemaker.mdmarkdown14.3 KB
references/model-deployment/references/deploy-nova-bedrock.mdmarkdown4.4 KB
references/model-deployment/references/deploy-nova-sagemaker.mdmarkdown5.8 KB
references/model-deployment/references/deploy-oss-bedrock.mdmarkdown5.2 KB
references/model-deployment/references/deploy-oss-sagemaker.mdmarkdown12.9 KB
references/model-deployment/references/model-licenses.mdmarkdown11.9 KB
references/model-deployment/references/optimization/benchmark-workflow.mdmarkdown12.7 KB
references/model-deployment/references/optimization/benchmarking-guidance.mdmarkdown8.9 KB
references/model-deployment/references/optimization/huggingface-to-s3.mdmarkdown3.1 KB
references/model-deployment/references/optimization/interpreting-results.mdmarkdown8.5 KB
references/model-deployment/references/optimization/recommendation-deploy.mdmarkdown1.5 KB
references/model-deployment/references/optimization/recommendation-options.mdmarkdown3.1 KB
references/model-deployment/references/optimization/recommendation-workflow.mdmarkdown10.3 KB
references/model-evaluation/code_templates/custom_scorer_evaluator.pyscript2.5 KB
references/model-evaluation/code_templates/llmaaj_evaluator.pyscript2.8 KB
references/model-evaluation/overview.mdmarkdown5.2 KB
references/model-evaluation/references/code_output_guide.mdmarkdown3.3 KB
references/model-evaluation/references/create-reward-function.mdmarkdown3.0 KB
references/model-evaluation/references/custom-lambda-scorer.mdmarkdown4.4 KB
references/model-evaluation/references/custom-scorer-evaluation.mdmarkdown10.5 KB
references/model-evaluation/references/evaluation-type-guide.mdmarkdown10.1 KB
references/model-evaluation/references/llmaaj-builtin-evaluation.mdmarkdown5.0 KB
references/model-evaluation/references/llmaaj-custom-evaluation.mdmarkdown2.5 KB
references/model-evaluation/references/llmaaj-evaluation.mdmarkdown14.6 KB
references/model-evaluation/references/supported-judge-models.mdmarkdown2.9 KB
references/model-evaluation/scripts/nova_reward_function_source_template.pyscript13.1 KB
references/model-evaluation/scripts/reward_function_source_template.pyscript9.2 KB
references/model-evaluation/scripts/validate_custom_metrics.pyscript3.5 KB
references/model-selection/overview.mdmarkdown10.3 KB
references/model-selection/references/benchmarks/agentic-index.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/coding-index.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/gpqa.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/hle.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/ifbench.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/intelligence-index.mdmarkdown2.8 KB
references/model-selection/references/benchmarks/mmmu-pro.mdmarkdown3.0 KB
references/model-selection/references/benchmarks/tau2.mdmarkdown2.8 KB
references/model-selection/references/model-licenses.mdmarkdown7.3 KB
references/model-selection/references/model-selection.mdmarkdown8.2 KB
references/model-selection/references/select-for-deployment.mdmarkdown19.8 KB
references/model-selection/scripts/filter_deployable_models.pyscript10.1 KB
references/model-selection/scripts/get_deployable_models.pyscript21.7 KB
references/model-selection/scripts/get_model_names.pyscript1.4 KB
references/model-selection/scripts/search_deployable_models.pyscript34.9 KB
references/planning/overview.mdmarkdown10.3 KB
references/planning/references/deploy-base-model-plan.mdmarkdown699 B
references/planning/references/evaluate-first-plan.mdmarkdown1.8 KB
references/planning/references/input-output-contracts.mdmarkdown3.1 KB
references/planning/references/model-customization-plan.mdmarkdown1.5 KB
references/planning/references/region-availability.mdmarkdown1.2 KB
references/planning/references/skill-routing-constraints.mdmarkdown2.7 KB
references/processing-jobs/code_templates/dataset_definition_processing.pyscript4.1 KB
references/processing-jobs/code_templates/framework_processing.pyscript3.0 KB
references/processing-jobs/code_templates/script_processing.pyscript2.9 KB
references/processing-jobs/code_templates/spark_processing.pyscript3.7 KB
references/processing-jobs/overview.mdmarkdown6.7 KB
references/processing-jobs/references/configuration.mdmarkdown4.6 KB
references/processing-jobs/references/troubleshooting.mdmarkdown2.4 KB
references/sdk-getting-started/overview.mdmarkdown643 B
references/sdk-getting-started/references/execution-role-setup.mdmarkdown2.4 KB
references/sdk-getting-started/references/sagemaker-python-sdk-setup.mdmarkdown2.1 KB
references/training-jobs/code_templates/custom_training.pyscript6.2 KB
references/training-jobs/code_templates/jumpstart_training.pyscript5.6 KB
references/training-jobs/code_templates/recipe_training.pyscript5.7 KB
references/training-jobs/overview.mdmarkdown9.1 KB
references/training-jobs/references/configuration.mdmarkdown4.0 KB
references/training-jobs/references/troubleshooting.mdmarkdown4.4 KB
references/use-case-specification/overview.mdmarkdown2.0 KB
references/use-case-specification/references/spec-for-deployment.mdmarkdown3.9 KB
references/use-case-specification/references/spec-for-finetuning.mdmarkdown2.0 KB

references/model-deployment/code_templates/optimize-recommendation.py

View on GitHub
# Cell 0 [markdown]: Inference Optimization — Deployment Recommendations

# Cell 1: Setup

# sagemaker>=3.20.0 ships the GenAI deployment-recommendation interface
# (ModelBuilder.generate_deployment_recommendations / model_builder.recommendations).
# %pip install --upgrade "sagemaker>=3.20.0,<4.0" --quiet  # NOTEBOOK_ONLY

# Cell 2: Configuration

import json
import os

os.environ["AWS_DEFAULT_REGION"] = "[REGION]"

from sagemaker.core import Attribution, set_attribution
from sagemaker.serve import ModelBuilder
from sagemaker.serve.ai_inference_recommender import Workload

set_attribution(Attribution.SAGEMAKER_AGENT_PLUGIN)

MODEL_S3_URI = "[MODEL_S3_URI]"
ROLE_ARN = "[ROLE_ARN]"
S3_OUTPUT_LOCATION = "[S3_OUTPUT_LOCATION]"
RECOMMENDATION_JOB_NAME = "[RECOMMENDATION_JOB_NAME]"
WORKLOAD_CONFIG_NAME = "[WORKLOAD_CONFIG_NAME]"
PERFORMANCE_TARGET = "[PERFORMANCE_METRIC]"  # "cost", "ttft-ms", or "throughput"
TOKENIZER = "[TOKENIZER]"

# Cell 3: Define the Workload

workload = Workload.synthetic(
    tokenizer=TOKENIZER,
    prompt_input_tokens_mean=[INPUT_TOKENS_MEAN],
    prompt_input_tokens_stddev=[INPUT_TOKENS_STDDEV],
    output_tokens_mean=[OUTPUT_TOKENS_MEAN],
    output_tokens_stddev=[OUTPUT_TOKENS_STDDEV],
    concurrency=[CONCURRENCY],
    request_count=[REQUEST_COUNT],
)
# For throughput + optimization, a dataset is required. Build the workload from
# a dataset instead of synthetic prompts. Inspect the data to detect its schema;
# custom_dataset_type is an OPTIONAL hint naming that schema ("openai-chat",
# "openai-completions", "sharegpt", "sagemaker-datacapture") — omit it to let
# AIPerf auto-detect. If the format is unsupported, convert it first with the
# dataset-transformation skill.
#   workload = Workload.from_dataset("[DATASET_S3_URI]", tokenizer=TOKENIZER,
#       custom_dataset_type="openai-chat",  # optional; from the schema you detected
#       prompt_input_tokens_mean=[INPUT_TOKENS_MEAN],
#       concurrency=[CONCURRENCY], request_count=[REQUEST_COUNT])

# Cell 4: Run the Recommendation Job

# Model source A — your own HuggingFace-format weights in S3 (the default path):
model_builder = ModelBuilder(model_path=MODEL_S3_URI, role_arn=ROLE_ARN)

# Model source B — a SageMaker JumpStart base model (no S3 copy of your own).
# build() resolves the JumpStart artifacts to their cache S3 URI, which the
# recommendation job reads. Works for UNGATED JumpStart models; a GATED model
# resolves to a private cache your role can't read and the job fails with an
# "Access denied" / "Invalid ModelSource.S3.S3Uri" error — in that case stage
# the weights to your own S3 (see references/optimization/huggingface-to-s3.md)
# and use source A instead. Use this INSTEAD of the line above:
#   from sagemaker.core.jumpstart.configs import JumpStartConfig
#   from sagemaker.core.training.configs import Compute
#   model_builder = ModelBuilder.from_jumpstart_config(
#       jumpstart_config=JumpStartConfig(model_id="[JUMPSTART_MODEL_ID]", model_version="*"),
#       compute=Compute(instance_type="[INSTANCE_TYPE]", instance_count=1),
#       role_arn=ROLE_ARN,
#   )
#   model_builder.build()  # required: resolves the JumpStart model's S3 artifacts

job = model_builder.generate_deployment_recommendations(
    workload=workload,
    performance_target=PERFORMANCE_TARGET,
    output_path=S3_OUTPUT_LOCATION,
    job_name=RECOMMENDATION_JOB_NAME,
    workload_config_name=WORKLOAD_CONFIG_NAME,
    # instance_types=["ml.g6.12xlarge"],  # optional: up to 3 candidates (latency/throughput only)
    # advanced_optimization=False,        # optional: default True (kernel tuning, speculative decoding)
    # framework="VLLM",                   # optional: "VLLM" or "LMI"; default auto-selected
    wait=True,
)
print(f"Recommendation job {job.get_name()} finished: {job.ai_recommendation_job_status}")

# Cell 5: Review Recommendations

recommendations = model_builder.recommendations
print(recommendations)

# Cell 6: Deploy Top Recommendation

# Only include this cell if the user wants to deploy after reviewing results.
# Deploys the top-ranked recommendation (index 0). Pass recommendation_index=N
# to deploy a different row, or recommendation_spec_name="..." to pick by name.
#
# A recommendation's ModelPackage is created unapproved, so deploy() would raise
# unless the package is approved first. auto_approve=True approves it in place as
# part of this deploy. This bypasses manual-approval governance on the package's
# model package group — appropriate here because the user is deploying their own
# recommendation. Remove auto_approve and approve the ModelPackage through your
# normal process if that governance must be preserved.
ENDPOINT_NAME = "[ENDPOINT_NAME]"
endpoint = model_builder.deploy(
    endpoint_name=ENDPOINT_NAME,
    recommendation_index=0,
    role=ROLE_ARN,
    auto_approve=True,
    wait=True,
)
print(f"Endpoint {endpoint.endpoint_name} status: {endpoint.endpoint_status}")

# Deploy from a recommendation job run in a different session (no in-memory
# model_builder — reconstruct it from the job name) — use INSTEAD of the block
# above when the recommendation job was run elsewhere (another session or a
# teammate):
#   model_builder = ModelBuilder.from_recommendation_job("[RECOMMENDATION_JOB_NAME]")
#   endpoint = model_builder.deploy(
#       endpoint_name="[ENDPOINT_NAME]",
#       recommendation_index=0,
#       role=ROLE_ARN,
#       auto_approve=True,
#       wait=True,
#   )
#   print(f"Endpoint {endpoint.endpoint_name} status: {endpoint.endpoint_status}")

# Cell 7: Save Manifest
# Save manifest - record output of workflow step for future reference
from pathlib import Path
manifest_dir = Path("[PROJECT_DIR]") / "manifests"
manifest_dir.mkdir(parents=True, exist_ok=True)
manifest_path = manifest_dir / f"recommendation-{RECOMMENDATION_JOB_NAME}.json"
# Record a model-source-appropriate identifier: MODEL_S3_URI for source A (S3 weights),
# or the JumpStart model id for source B (uncomment the appropriate line below).
manifest_path.write_text(json.dumps({
    "recommendation_job_name": RECOMMENDATION_JOB_NAME,
    "model_source": {"type": "s3", "model_s3_uri": MODEL_S3_URI},
    # For a JumpStart-sourced run, use this instead of the line above:
    #   "model_source": {"type": "jumpstart", "model_id": "[JUMPSTART_MODEL_ID]"},
    "workload_config_name": WORKLOAD_CONFIG_NAME,
    "performance_target": PERFORMANCE_TARGET,
    "status": job.ai_recommendation_job_status,
}, indent=2))
print(f"Manifest saved: {manifest_path}")
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall