lasp/adamant-skills / adamant-generator-dev
adamant-generator-dev
VerifiedAdamant Generator Development
Patterns and workflows for developing custom generators for Adamant framework components. Generators produce Ada source, YAML types, HTML docs, and ground system artifacts from YAML model files.
Architecture Overview
Adamant's code generation pipeline:
YAML Model File -> Schema Validation -> Python Model Object -> Jinja2 Template -> Generated Output
Discovery chain:
env/set_python_path.shadds every directory containing__init__.pyto PYTHONPATHredo/util/meta.pyscans PYTHONPATH for directories namedgenerators/orgenerator/- All
.pyfiles in those directories are imported meta.get_all_subclasses_of(generator_base)finds all generator classes- Each generator's
input_file_regex()determines which YAML files it processes - The generator database maps (input_file, generator) -> output_file for redo
Directory Structure
Component-local generators live inside the component directory:
src/components/<component_name>/
gen/
__init__.py # REQUIRED: empty file, makes gen/ a Python package
generators/ # NO __init__.py here!
<generator_name>.py # Generator classes
models/ # NO __init__.py here!
<model_name>.py # Model classes (parse YAML into Python objects)
schemas/ # NO __init__.py here!
<schema_name>.yaml # Pykwalify schemas for YAML validation
templates/ # NO __init__.py here!
<template_dir>/
name.ads # Jinja2 templates ("name" is replaced with model name)
doc/ # Optional: LaTeX documentation for the generator
<generator_name>.tex
CRITICAL: __init__.py goes ONLY in gen/, NEVER in subdirectories. Adding __init__.py to gen/models/ or gen/generators/ makes them Python packages that shadow the framework's models and generators packages (which contain models.base, models.exceptions, generators.basic, etc.), causing ModuleNotFoundError on any redo build. The framework's 15 existing generators confirm this: none have __init__.py in subdirectories.
All 15 framework components with custom generators follow this exact structure.
YAML File Naming Convention
CRITICAL: YAML model files follow a strict naming pattern that determines how they are discovered and processed:
[specific_name.]model_name.model_type.yaml
model_type= the type string used for regex matching (e.g.,ccsds_router_table,parameter_table)model_name= typically the assembly name (for assembly submodels) or component namespecific_name= optional qualifier when multiple instances of the same model type exist
Examples:
linux_example.ccsds_router_table.yaml # model_name=linux_example, model_type=ccsds_router_table
test_parameter_table.test_assembly.parameter_table.yaml # specific_name=test_parameter_table, model_name=test_assembly
linux_example.product_packets.yaml # model_name=linux_example, model_type=product_packets
The model_type in the filename MUST match model_class.__name__ in the generator (which becomes input_file_regex = r".*\.<model_type>\.yaml$").
Generator Class Patterns
Pattern 1: Basic Generator (most common)
For generators that produce a single output file from a single YAML input. Inherit from both basic_generator and generator_base:
import os.path
from models import my_model
from base_classes.generator_base import generator_base
from generators.basic import basic_generator
class my_output_ads(basic_generator, generator_base):
def __init__(self):
this_file_dir = os.path.dirname(os.path.realpath(__file__))
template_dir = os.path.join(this_file_dir, ".." + os.sep + "templates")
basic_generator.__init__(
self,
model_class=my_model.my_model,
template_filename="name.ads",
template_dir=template_dir,
)
def output_filename(self, input_filename):
dirname, specific_name, model_name, *ignore = self._split_input_filename(
input_filename
)
if not specific_name:
build_dir = self._get_default_build_dir()
a = self.template_basename.rsplit("name", maxsplit=1)
output_filename = (model_name + "_my_suffix").join(a)
return dirname + os.sep + build_dir + os.sep + output_filename
return basic_generator.output_filename(self, input_filename)
def generate(self, input_filename):
m = my_model.my_model(input_filename)
m.load_assembly() # if assembly submodel
print(m.render(self.template, template_path=self.template_dir))
Pattern 2: Assembly-Aware Generator (needs assembly context)
When the generator needs to resolve references against the assembly (component names, connector indices, IDs):
def generate(self, input_filename):
# Load the assembly model
dirname, view_name, assembly_name, *ignore = self._split_input_filename(
input_filename
)
assembly_model_name = model_loader.get_model_file_path(
assembly_name, model_types="assembly"
)
if not assembly_model_name:
error.error_print(
"Could not find a model file for assembly '" + assembly_name + "'."
)
error.abort()
a = assembly.assembly(assembly_model_name)
# Load the submodel and resolve against assembly
r = my_model.my_model(input_filename)
r.resolve_against_assembly(a)
print(r.render(self.template, template_path=self.template_dir))
Pattern 3: Assembly Generator Extension (extending assembly templates)
For generators that produce assembly-level outputs with component-local templates (e.g., event_to_text):
from generators.assembly import assembly_generator
class my_assembly_output_ads(assembly_generator, generator_base):
def __init__(self):
this_file_dir = os.path.dirname(os.path.realpath(__file__))
template_dir = os.path.join(this_file_dir, ".." + os.sep + "templates")
assembly_generator.__init__(
self, "name_my_output.ads", template_dir=template_dir
)
Pattern 4: Ided Suite Override (custom commands/events/data_products/faults)
When a component needs to generate its own commands, events, data products, or faults that depend on the assembly-level configuration YAML:
from generators.basic import add_basic_generators_to_module
from generators.ided_suite import fault_templates, command_templates, data_product_templates
# Override the standard ided_suite generators with custom model classes
add_basic_generators_to_module(
my_custom_faults.my_custom_faults, fault_templates, module=globals()
)
add_basic_generators_to_module(
my_custom_commands.my_custom_commands, command_templates, module=globals()
)
This reuses standard templates (events/name_events.ads, etc.) but with a custom model class that computes different content.
Model Class Patterns
Component Submodel (for component-scoped generators)
Use component_submodel when the generator's YAML is scoped to a single component and needs to verify against that component's connectors, parameters, or other features. The YAML filename uses the component name: component_name.model_type.yaml.
from models.component import component_submodel
from models.exceptions import ModelException
class my_component_config(component_submodel):
def __init__(self, filename):
this_file_dir = os.path.dirname(os.path.realpath(__file__))
schema_dir = os.path.join(this_file_dir, ".." + os.sep + "schemas")
super(my_component_config, self).__init__(
filename, schema_dir + "/my_schema.yaml"
)
def load(self):
"""Parse YAML data."""
super(my_component_config, self).load() # sets self.component = None
# Parse self.data here...
def set_component(self, component):
"""Called when the component model is loaded. Validate against component here."""
super(my_component_config, self).set_component(component)
# self.component is now set -- verify connectors, features, etc.
for connector in self.component.connectors:
# connector.name, connector.kind, connector.type, connector.count
pass
# Raise ModelException if validation fails
The generator calls load_component() to trigger the lifecycle:
def generate(self, input_filename):
m = my_component_config(input_filename)
m.load_component() # triggers set_component() callback
print(m.render(self.template, template_path=self.template_dir))
Do NOT use base with manual model_loader calls for component-scoped generators -- use component_submodel which handles caching, dependency tracking, and the set_component() lifecycle automatically.
Assembly Submodel (most component generators use this)
Assembly submodels are loaded by the assembly model during its load process. The assembly calls set_assembly() on each submodel, giving it access to the full assembly context.
from models.assembly import assembly_submodel
from models.exceptions import ModelException
class my_model(assembly_submodel):
def __init__(self, filename):
this_file_dir = os.path.dirname(os.path.realpath(__file__))
schema_dir = os.path.join(this_file_dir, ".." + os.sep + "schemas")
super(my_model, self).__init__(
filename, schema_dir + "/my_schema.yaml"
)
def load(self):
"""Called during __init__. Parse YAML data into Python objects."""
super(my_model, self).load() # sets self.assembly = None
self.name = None
self.description = None
# Access raw YAML via self.data (dict from pykwalify-validated YAML)
if self.specific_name:
self.name = ada.formatType(self.specific_name)
else:
self.name = ada.formatType(self.model_name) + "_My_Suffix"
if "description" in self.data:
self.description = self.data["description"]
# Parse YAML entries into model objects
for entry_data in self.data["entries"]:
# ... build Python objects from YAML data
def set_assembly(self, assembly):
"""Called by assembly model during load. Resolve assembly references here."""
super(my_model, self).set_assembly(assembly)
# Now self.assembly is set -- resolve component names, connector indices, IDs
self._resolve_references()
Key Model Base Class Members
Available in all models inheriting from base:
self.data-- parsed YAML content (dict)self.model_name-- extracted from filename (e.g., "linux_example")self.model_type-- extracted from filename (e.g., "parameter_table")self.specific_name-- optional prefix from filename (or None)self.full_filename-- absolute path to YAML fileself.dependencies-- list of file paths this model depends onself.render(template, template_path)-- render Jinja2 template with self.dict
Available in assembly submodels:
self.assembly-- the parent assembly model (set viaset_assembly())self.load_assembly()-- explicitly load the assembly (called by generators, not during normal assembly load)
Assembly Model API (for resolving references)
# Get a component by its instance name
comp = self.assembly.get_component_with_name("My_Component_Instance")
# comp.instance_name, comp.name (type name), comp.full_filename
# comp.connectors, comp.commands, comp.parameters, comp.events, comp.faults, comp.data_products
# Get a connector by name
connector = comp.connectors.of_name("Ccsds_Space_Packet_T_Send")
# Get connections on a connector
connections = connector.get_connections()
# Each connection: c.to_component, c.to_connector, or None/"ignore"
Schema Patterns (Pykwalify)
Schemas validate YAML input before model loading:
---
type: map
mapping:
# Required string field
instance_name:
type: str
required: True
# Optional description
description:
type: str
required: False
# Required list with minimum 1 entry
entries:
seq:
- type: map
mapping:
name:
type: str
required: True
value:
type: int
required: True
mode:
type: str
enum: ['option_a', 'option_b', 'option_c']
required: False
range:
min: 1
required: True
Template Patterns (Jinja2)
Templates use Jinja2 with access to all model object members via self.__dict__:
-- Generated file. DO NOT EDIT.
{% if description %}
{{ printMultiLine(description, '-- ') }}
{% endif %}
package {{ name }} is
{% for entry in entries %}
-- Entry for {{ entry.name }}
{{ entry.name }}_Value : constant := {{ entry.value }};
{% endfor %}
end {{ name }};
Template naming: name in the template filename is replaced with the model name in the output filename. Example: template name.ads with model name linux_example_my_table produces linux_example_my_table.ads.
Ada float literals: YAML float values that are integer-valued (e.g., 85.0 in YAML) may lose their decimal point when formatted in Python (repr(85.0) -> '85.0' but "%.6g" % 85 -> '85'). Ada requires float literals to always have a decimal point (85.0 not 85). Use a helper:
def _to_ada_float(value):
s = repr(float(value))
if "." not in s and "e" not in s and "E" not in s:
s = s + ".0"
return s
Store the Ada-formatted string in the model object (e.g., self.value_ada = _to_ada_float(self.value)) and reference it in the template.
Template location: Templates go in gen/templates/<template_subdir>/ where <template_subdir> matches the model type or a descriptive name.
Output file placement rules (from basic_generator._get_default_build_dir()):
.ads,.adbfiles ->build/src/(unless name ends in-implementation,-implementation-tester,main,test->build/template/).texfiles ->doc/build/tex/.htmlfiles ->build/html/.yamlfiles ->build/yaml/- Other extensions ->
build/<extension>/
Utility Functions
from util import ada
ada.formatVariable("my_component") # -> "My_Component" (Ada variable naming)
ada.formatType("my_type") # -> "My_Type" (Ada type naming)
ada.isTypePrimitive("Natural") # -> True
from util import model_loader
model = model_loader.try_load_model_by_name("assembly_name", model_types="assembly")
path = model_loader.get_model_file_path("component_name", model_types="component")
from util import error
error.error_print("message")
error.error_abort("fatal message")
Framework Components with Custom Generators (Reference Catalog)
15 of 59 framework components have custom generators:
| Component | Model Type | Assembly-Aware | Generates |
|---|---|---|---|
| ccsds_router | ccsds_router_table | Yes (resolves connector indices) | Ada routing table spec |
| parameters | parameter_table | Yes (resolves param IDs, connector indices) | Ada param table spec, YAML record type |
| fault_correction | fault_responses | Yes (resolves fault/command IDs) | Ada fault response table, YAML record/enum |
| task_watchdog | task_watchdog_list | Yes (resolves pet connector indices) | Ada watchdog list spec, YAML record types |
| product_packetizer | product_packets | Yes (resolves data product IDs) | Ada packet definitions, HTML |
| ccsds_downsampler | ccsds_downsampler_filters | Model-only | Ada filter config |
| ccsds_command_depacketizer | (assembly-level) | Yes | Ada command routing |
| ccsds_packetizer | (assembly-level) | Yes | Hydra XML config |
| ccsds_product_extractor | extracted_products | Yes | Ada extraction table |
| command_sequencer | (assembly-level) | Yes | Ada sequence config |
| cpu_monitor | (assembly-level) | Yes | Custom packets |
| queue_monitor | (assembly-level) | Yes | Custom packets |
| stack_monitor | (assembly-level) | Yes | Custom packets |
| event_text_logger | (assembly-level) | Yes | Ada event-to-text mapping |
| sequence_store | sequence_store | Yes | Ada sequence storage config |
Creating a New Generator: Checklist
-
Define the YAML schema in
gen/schemas/<model_type>.yaml- Use pykwalify format
- Required fields, types, enums, ranges
-
Create the model class in
gen/models/<model_name>.py- Inherit from
assembly_submodelif assembly context is needed - Implement
load()to parseself.data - Implement
set_assembly()if resolving assembly references - Add resolved data to
self.dependenciesfor redo tracking
- Inherit from
-
Create Jinja2 templates in
gen/templates/<template_dir>/namein filename is replaced with model name- Access all model members directly in template
-
Create the generator class in
gen/generators/<generator_name>.py- Inherit from
basic_generatorandgenerator_base - Point to model class and template
- Override
output_filename()for custom naming - Override
generate()if assembly loading is needed
- Inherit from
-
Create empty
__init__.pyingen/- REQUIRED for PYTHONPATH discovery
set_python_path.shusesgit ls-filesto find__init__.py-- files must be committed (not just staged)- After committing new
gen/__init__.py, run this EXACT sequence to make the generator discoverable:# Step 1: Commit the gen/ files on host git add src/components/<name>/gen/ && git commit -m "Add generator" # Step 2: Sync container git index + rebuild PYTHONPATH (SINGLE COMMAND) bash docker/adamant_env.sh exec "cd /home/user/<project> && git read-tree HEAD && rm -f /tmp/.*_env_snapshot" # Step 3: Next exec will do full activation with updated PYTHONPATH - Do NOT skip
git read-tree HEAD-- the container's git index does not auto-sync from host commits due to the bind mount. Without this,git ls-fileswon't see new files even after commit. - Verify discovery:
adamant_env.sh exec "echo \$PYTHONPATH | tr ':' '\n' | grep <component>"
-
Create the YAML model file in the assembly directory
- Named:
[specific.]assembly_name.<model_type>.yaml - Must be in a directory with
.all_path
- Named:
-
Refresh the environment: Run
adamant_env.sh refreshto rebuild the cached environment snapshot (picks up the new__init__.py), thenredo clear_cacheto invalidate stale generator registry -
Test: Run
redoand verify the generated output appears inbuild/src/
Common Patterns in Existing Generators
Resolving Component Connector Indices
Many generators need to translate component names into connector output indices (1-based). Pattern from ccsds_router_table:
# Get the component's output connector
connector = component_model.connectors.of_name("Output_T_Send")
connections = connector.get_connections()
connected_components = {}
for idx, c in enumerate(connections):
if c is not None and c != "ignore":
connected_components[c.to_component.instance_name] = idx + 1
Generating YAML Type Definitions (Cascading Dependency Chain)
Some generators produce YAML type definitions (records, enums) that feed back into the type system. These go to build/yaml/ and are picked up by the standard type generators.
When a downstream generator (e.g., Ada spec) needs to USE a type produced by an upstream generator (e.g., YAML enum), the downstream generator must:
- Declare the upstream output in
depends_on()so redo builds it first - The downstream Ada TEMPLATE must
withandusethe generated package
Example: enum generator produces build/yaml/<name>_phases.enums.yaml which becomes <Name>_Phases Ada package with Phase_Name.E enum type. The Ada spec template must reference it:
with {{ name }}_Phases; use {{ name }}_Phases;
-- ...
Phase : Phase_Name.E; -- field using the generated enum
The depends_on() in the Ada spec generator ensures redo builds the enum YAML first, which triggers the standard enum generator to produce the .ads, which is then available for the Ada spec to with:
class my_record_yaml(basic_generator, generator_base):
def __init__(self):
# ...
basic_generator.__init__(
self,
model_class=my_model.my_model,
template_filename="name_record.record.yaml",
template_dir=template_dir,
)
Overriding Standard Ided Suite Generators
When a component needs assembly-aware commands/events/faults/data_products, create a custom model that inherits from the standard model but adds assembly resolution:
# In gen/models/my_custom_commands.py
from models.commands import commands
class my_custom_commands(commands):
def __init__(self, filename):
# Custom loading that computes commands from assembly config
...
Then in the generator, use add_basic_generators_to_module to register the custom model with standard templates.
Error Handling
Use ModelException for user-facing errors in models:
from models.exceptions import ModelException, throw_exception_with_filename
@throw_exception_with_filename
def resolve(self):
if not self.assembly.get_component_with_name(name):
raise ModelException(
'Component "' + name + '" does not exist in assembly "' + self.assembly.name + '".'
)
The @throw_exception_with_filename decorator adds the YAML filename to error messages. Apply it to methods, NOT classes -- applying it to a class replaces the class with a wrapper function, breaking model_type detection and generator discovery.
Anti-Patterns
- Do NOT use
rm -rfon build directories -- useredo cleanorredo clean_all - Do NOT generate files outside
build/-- redo owns the build directory lifecycle - Do NOT hardcode paths -- use
os.pathand the model'sfull_file_dir - Do NOT forget
__init__.pyingen/-- without it, generators won't be discovered - Do NOT put
__init__.pyingen/generators/,gen/models/, or other subdirs -- they shadow the framework's packages and break all imports - Do NOT forget
adamant_env.sh refresh-- after adding__init__.py, the environment snapshot is stale;adamant_env.sh refreshrebuilds it without restarting the container. Follow withredo clear_cache. - Do NOT use same
model_typestring as an existing model -- causes regex collisions - Do NOT forget to add dependencies -- redo needs
self.dependenciesfor incremental builds - Do NOT use
print()for debugging -- generators capture stdout as output; usesys.stderr.write() - Model .py changes are NOT auto-tracked by redo -- redo tracks the generator .py file and the input YAML, but NOT the model .py file (loaded dynamically). After modifying model code, run
redo cleanin the component dir or touch the YAML file to force regeneration - Do NOT forget
git commitbefore refresh --set_python_path.shusesgit ls-fileswhich only sees committed (or staged) files. Untracked__init__.pyfiles are invisible to the Python path builder
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |