Skip to content

DexForce/EmbodiChain / .agents/skills/add-task-env

add-task-env

Verified

Route and scaffold a new EmbodiChain task by solution type: handwritten expert trajectory, Task Program expert trajectory, simulator RL, lightweight RL, or an environment-only baseline. Use when creating a task, environment, expert demo, or RL task.

FileKindSize
SKILL.mdskill md7.4 KB
agents/openai.yamlconfig95 B
references/handwritten-expert.mdmarkdown3.6 KB
references/rl.mdmarkdown4.8 KB
references/task-program-expert.mdmarkdown3.2 KB

references/rl.md

View on GitHub
# Reinforcement Learning Task

Use this route when the task should be trainable by PPO, GRPO, APG, or another
registered learning algorithm. EmbodiChain already has an RL system; the first
decision is which environment boundary the prompt requires.

## Route RL environment type

| Requirement | Environment path | Training selector |
|---|---|---|
| Physics simulator, robot, sensors, manager functors | Simulator Gym | `trainer.gym_config` |
| Tensor-only/vectorized dynamics | Lightweight learning env | `trainer.learning_env` |
| Differentiable dynamics or APG | Differentiable lightweight env | `trainer.learning_env` |

Do not use a simulator Gym config for APG. Differentiable algorithms require a
`DifferentiableEnvProtocol` and are rejected on the simulator path.

## Simulator Gym RL

Use the normal task-first simulator layout. A manipulation deployment with a
root `rl_task` mapping dynamically registers `EmbodiedEnv` and enables its
optional `TerminationManager`; no task Python module is needed. Reuse existing
action/observation/reward/event managers; declare
measured success, failure and optional truncation condition functors. The ID
must be free or match an identical configuration-owned declaration.

Virtual target poses are reset-owned and recorded for single-environment dynamic
replay. Combine `rl_task` and `task_program` only in explicit `task_program.execution: policy` mode with complete policy resource ownership; expert execution remains exclusive. See the policy-stage subset and reward/termination bindings in the configuration contract. See the owning
[configuration contract](../../../../agent_context/topics/env-framework/configuration.md#configuration-owned-manipulation-rl)
and [user guide](../../../../docs/source/guides/configured_manipulation_rl.md).

For tasks requiring custom environment behavior, keep a flat task module with
import-owned `@register_env` registration instead.

Register a Python-owned simulator task with explicit RL capability:

```python
@register_env(
    "MyTaskRL",
    max_episode_steps=300,
    supports_rl=True,
)
class MyTaskEnv(EmbodiedEnv):
    ...
```

The runnable Gym deployment owns or selects the physical environment,
embodiment, observations, rewards, actions, events, and randomization. Put
training files under:

```text
<task config>/agents/<algorithm>.yaml
```

Select it from the training config with:

```yaml
trainer:
  gym_config: embodichain/extensions/tasks/configs/tasks/<category_path>/<task_name>/<deployment>.yaml
```

Simulator environments use standard rollouts. PPO and GRPO are current
standard examples.

## Lightweight learning RL

Keep the task entry point flat:

```text
embodichain/extensions/tasks/<category_path>/<task_name>.py
```

Register the factory with `@register_learning_env` and implement the
`LearningEnvProtocol` structural contract:

- `num_envs`, `device`, `single_observation_space`, and
  `single_action_space`;
- batched `reset()` and `step()`;
- terminal reward/done for completed rows while auto-resetting those rows to
  their next initial observation;
- `close()`.

For APG or another differentiable algorithm, implement
`DifferentiableEnvProtocol.detach_state()` as the truncated-backpropagation
boundary. It must detach dynamic state without resetting or resampling the
task.

Select it with:

```yaml
trainer:
  learning_env:
    name: MyTaskRL
    cfg: {}
```

## Training configuration

Every agent config has exact top-level blocks:

- `trainer`: environment selector and runtime/evaluation/checkpoint settings;
- `policy`: registered policy and network configuration;
- `algorithm`: registered algorithm name and algorithm config.

Read values from the current config classes and the closest official example;
do not copy example hyperparameters as global defaults. Keep all algorithms for
the same task in its single `agents/` directory.

Current routing:

- PPO and GRPO: `RolloutKind.STANDARD`;
- APG: `RolloutKind.DIFFERENTIABLE`.

Reference configurations:

- simulator: `classic_control/cart_pole/agents/` and
  `manipulation/push_cube/agents/`;
- lightweight standard/differentiable:
  `classic_control/point_mass/agents/`.

## Validation

Cover the selected boundary:

### Simulator

1. parse the runnable Gym config;
2. prove `@register_env(..., supports_rl=True)` discovery when Python-owned;
3. construct/reset a minimal environment;
4. test observation/action dimensions and reward/termination terms;
5. run trainer routing with the chosen config.

### Lightweight

1. prove `@register_learning_env` discovery;
2. test vector reset/step shapes, devices, async row completion, and auto-reset;
3. test terminal metrics;
4. for differentiable environments, prove reward gradients reach actions and
   `detach_state()` creates the intended TBPTT boundary;
5. run the focused trainer/collector routing tests.

Use `tests/gym/envs/tasks/` for simulator task tests and `tests/learning/`
for lightweight/training tests.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall