DexForce/EmbodiChain / .agents/skills/add-task-env
add-task-env
VerifiedRoute and scaffold a new EmbodiChain task by solution type: handwritten expert trajectory, Task Program expert trajectory, simulator RL, lightweight RL, or an environment-only baseline. Use when creating a task, environment, expert demo, or RL task.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.4 KB |
| agents/openai.yaml | config | 95 B |
| references/handwritten-expert.md | markdown | 3.6 KB |
| references/rl.md | markdown | 4.8 KB |
| references/task-program-expert.md | markdown | 3.2 KB |
references/rl.md
View on GitHub# Reinforcement Learning Task
Use this route when the task should be trainable by PPO, GRPO, APG, or another
registered learning algorithm. EmbodiChain already has an RL system; the first
decision is which environment boundary the prompt requires.
## Route RL environment type
| Requirement | Environment path | Training selector |
|---|---|---|
| Physics simulator, robot, sensors, manager functors | Simulator Gym | `trainer.gym_config` |
| Tensor-only/vectorized dynamics | Lightweight learning env | `trainer.learning_env` |
| Differentiable dynamics or APG | Differentiable lightweight env | `trainer.learning_env` |
Do not use a simulator Gym config for APG. Differentiable algorithms require a
`DifferentiableEnvProtocol` and are rejected on the simulator path.
## Simulator Gym RL
Use the normal task-first simulator layout. A manipulation deployment with a
root `rl_task` mapping dynamically registers `EmbodiedEnv` and enables its
optional `TerminationManager`; no task Python module is needed. Reuse existing
action/observation/reward/event managers; declare
measured success, failure and optional truncation condition functors. The ID
must be free or match an identical configuration-owned declaration.
Virtual target poses are reset-owned and recorded for single-environment dynamic
replay. Combine `rl_task` and `task_program` only in explicit `task_program.execution: policy` mode with complete policy resource ownership; expert execution remains exclusive. See the policy-stage subset and reward/termination bindings in the configuration contract. See the owning
[configuration contract](../../../../agent_context/topics/env-framework/configuration.md#configuration-owned-manipulation-rl)
and [user guide](../../../../docs/source/guides/configured_manipulation_rl.md).
For tasks requiring custom environment behavior, keep a flat task module with
import-owned `@register_env` registration instead.
Register a Python-owned simulator task with explicit RL capability:
```python
@register_env(
"MyTaskRL",
max_episode_steps=300,
supports_rl=True,
)
class MyTaskEnv(EmbodiedEnv):
...
```
The runnable Gym deployment owns or selects the physical environment,
embodiment, observations, rewards, actions, events, and randomization. Put
training files under:
```text
<task config>/agents/<algorithm>.yaml
```
Select it from the training config with:
```yaml
trainer:
gym_config: embodichain/extensions/tasks/configs/tasks/<category_path>/<task_name>/<deployment>.yaml
```
Simulator environments use standard rollouts. PPO and GRPO are current
standard examples.
## Lightweight learning RL
Keep the task entry point flat:
```text
embodichain/extensions/tasks/<category_path>/<task_name>.py
```
Register the factory with `@register_learning_env` and implement the
`LearningEnvProtocol` structural contract:
- `num_envs`, `device`, `single_observation_space`, and
`single_action_space`;
- batched `reset()` and `step()`;
- terminal reward/done for completed rows while auto-resetting those rows to
their next initial observation;
- `close()`.
For APG or another differentiable algorithm, implement
`DifferentiableEnvProtocol.detach_state()` as the truncated-backpropagation
boundary. It must detach dynamic state without resetting or resampling the
task.
Select it with:
```yaml
trainer:
learning_env:
name: MyTaskRL
cfg: {}
```
## Training configuration
Every agent config has exact top-level blocks:
- `trainer`: environment selector and runtime/evaluation/checkpoint settings;
- `policy`: registered policy and network configuration;
- `algorithm`: registered algorithm name and algorithm config.
Read values from the current config classes and the closest official example;
do not copy example hyperparameters as global defaults. Keep all algorithms for
the same task in its single `agents/` directory.
Current routing:
- PPO and GRPO: `RolloutKind.STANDARD`;
- APG: `RolloutKind.DIFFERENTIABLE`.
Reference configurations:
- simulator: `classic_control/cart_pole/agents/` and
`manipulation/push_cube/agents/`;
- lightweight standard/differentiable:
`classic_control/point_mass/agents/`.
## Validation
Cover the selected boundary:
### Simulator
1. parse the runnable Gym config;
2. prove `@register_env(..., supports_rl=True)` discovery when Python-owned;
3. construct/reset a minimal environment;
4. test observation/action dimensions and reward/termination terms;
5. run trainer routing with the chosen config.
### Lightweight
1. prove `@register_learning_env` discovery;
2. test vector reset/step shapes, devices, async row completion, and auto-reset;
3. test terminal metrics;
4. for differentiable environments, prove reward gradients reach actions and
`detach_state()` creates the intended TBPTT boundary;
5. run the focused trainer/collector routing tests.
Use `tests/gym/envs/tasks/` for simulator task tests and `tests/learning/`
for lightweight/training tests.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |