Systemorph/MeshWeaver / .claude/skills/new-deployment
new-deployment
VerifiedBring up a NEW MeshWeaver instance the ONLY sanctioned way: a Hosting/Deployment record in the estate's private deployments repository, provisioned by the control instance's Hosting plugin (a Provision InstanceAction on the control instance) — never kubectl, never deploy.sh, never the ACR image: images come from the fleet's own registry cr.meshweaver.cloud. Use when standing up a customer/internal portal, when a Provision refuses or fails, or when auditing whether an instance is wired the way its record says. Covers the human-done prerequisites (Entra app, registry instance key, db-connection secret, a tag present in cr), what the operator's Provision plan does in what order, the refusals that are ANSWERS not errors, and the traps that read as success.
/new-deployment — a new instance is a record, provisioned by the control instance
A deployment is one Hosting/Deployment record — mesh/Deployments/<id>.json in the estate's PRIVATE
deployments repository — that the control instance (e.g. https://control.example.com, its own record carrying
operator.enabled: true) turns into a namespace, a database, DNS, a Helm release and a certificate
when a global admin files a Provision action against it. Everything else — cluster, ingress,
Postgres server, Key Vault, the registry, observability — already exists and is shared.
🔒 The inventory is private. Hosts, namespaces and database names live in the estate's private deployments repository (
mesh/Deployments/*.json, its inventory, the Deployments tab) — see "Where an estate's instances are declared" in Instances.md. The public MeshWeaver repo carries the mechanism (chart, operator scripts, the Hosting plugin's sources) and must never carry who runs what.
🚨 The maintained runbook is docs/new-deployment.md in the estate's private deployments repository. It is authoritative
wherever it and this page disagree; this page is the map, not a copy of the commands.
Where the work happens — and where it does not
| who | how | |
|---|---|---|
| the record | a human, by PR | mesh/Deployments/<id>.json (GitSynced to the portals; a mesh-side edit is written back as a PR by DeploymentGitSync) |
| secrets an instance cannot mint for itself | a human, once | Key Vault <vault>, names <keyVaultSecretPrefix><Section>-<Key> |
| namespace, database, identity, DNS, pull secret, release, TLS | the operator, from the Provision plan | a Hosting/InstanceAction node on the control instance |
| the cluster | nobody by hand | kubectl is not part of this procedure; a step that needs it is a defect in the lane, file it |
1. What a human does first
Every one of these is a prerequisite the Provision REFUSES without, so do them before filing it.
<prefix> is the record's keyVaultSecretPrefix.
- The sign-in app (multi-tenant; invitation-only is the gate, not the audience):
az ad app create --display-name "<Name> Portal (<host>)" --sign-in-audience AzureADMultipleOrgs --web-redirect-uris https://<host>/signin-microsoft, thenaz ad sp create --id <appId>, then the client secret into the vault as<prefix>Authentication-Microsoft-ClientSecret— through the record page's Set Key Vault secrets… dialog on the control instance (write-only; the governed writer identity stores it, policysecrets-write-only-entry,Doc/Architecture/SecretsWriteOnlyEntry), never a vault command of your own. The appId goes on the record assignIn.microsoftClientId. 🚨az … credential resetPRINTS the secret on stderr:--query password -o tsv > "$S/x" 2>/dev/null, paste it from that file into the dialog, thenrm. Nevercata stderr capture from a minting verb. - The registry instance key — ONE key, two jobs (plugin catalog AND image pull):
POST https://<plugin-registry-instance>/api/instances/registerwith{"bootstrapKey":"","instanceId":"<id>","displayName":"…","homeUrl":"https://<host>"}(open registration = thefreeplan; an admin raises it under Admin ▸ Instance grants). Paste.instanceKeyinto the same Set Key Vault secrets… dialog as<prefix>PluginCatalog-RegistryToken.hosting-kv-ensurelists this object as REQUIRED and refuses the whole Provision without it — "minting a random value here would produce an instance that boots, registers nothing, and shows an empty Store with no error". - The database connection string —
<prefix>db-connection(InstanceSpec.DatabaseSecretName), composed from the shared admin password in vault objectmemex-postgres-password:Host=<pg-server>.postgres.database.azure.com;Port=5432;Username=<pg-admin>;Password=…;Database=<db>;SslMode=Require;Trust Server Certificate=true. FQDN, never an IP — Azure moves the backing instance and DNS follows it. Mapped on the record'skeyVaultSecrets.secretsasConnectionStrings__memex. - An image tag that EXISTS in
cr.meshweaver.cloud.imageRepository: cr.meshweaver.cloud/memex-portal-ai,pinnedImageTag: <tag>— a first install with no pin is REFUSED byhosting-deploy("Pin an image tag on the Deployment record for the first install"). If the tag is not in cr yet (CD pushes to ACR), copy it there as the registry'spublisheraccount:crane copy meshweaver.azurecr.io/memex-portal-ai:<tag> cr.meshweaver.cloud/memex-portal-ai:<tag>and the same formemex-migration(the migration image is DERIVED by replacingmemex-portal-ai→memex-migration; both must be present). - The record itself, by PR, with (a sibling record is the worked example):
imagePullSecret: registry-pull;volumes[]fordata/content/attachments/userswithclaimName,mountPath,size,storageClass: azurefile-memex,accessMode: ReadWriteManyandcreate: true— the chart creates the claims; withoutcreatea new instance gets emptyDir and every restart wipes it;keyVaultSecretsnaming the three vault objects above plus<prefix>Ai-KeyProtection-MasterKey(GENERATED byhosting-kv-ensure, never regenerated — it seals every storedenc:provider key);pluginRepos[0] = { name: "Plugins", url: "https://<plugin-registry-instance>" }(the name is the REGISTRY's name, never the instance's — a barepreInstallid is qualified against it and the catalog FAILS CLOSED on a mismatch);preInstall: ["Essentials"];updatePolicy: Stablefor a customer;extraPortalConfigwithHosting__Deployment: <id>andHosting__ReportTo: https://control.example.com(the control instance). Gates that read it:check-record-renders-overlay.py(record ↔values.<release>.public.yaml,extraPortalConfigone way,keyVaultSecretsboth ways),check-image-pins.py(resolves every pin —cr.meshweaver.cloudpins needMW_REGISTRY_KEY),config-key-coverage.
2. What the operator does — the Provision plan, in order
Composed by InstanceActionPlan.ProvisionSteps (MeshWeaver.Plugins, in-mesh — it compiles in the
portal, no CI type-checks it), run by MeshWeaver.Plugins/deploy/hosting-operator/bin/run.sh in a Job in memex-ops,
fail-fast, the failing STEP named on the node:
az postgres flexible-server db create— control plane, workload identity. (Memex#132 —PGPASSWORDhas no secret channel — blocks Backup/Suspend/Teardown/Restore, not this.)hosting-kv-ensure— generates the master key, REQUIRES the registry token.hosting-federate— federated credential, subject exactlysystem:serviceaccount:<ns>:memex-portal-sa; a mismatch does not error, it silently stops self-update tag discovery.hosting-dns upsert— the A record at the shared ingress IP. DNS BEFORE TLS: HTTP-01 needs it.hosting-pull-secret— ensures the namespace, turns the vault's registry token into thekubernetes.io/dockerconfigjsonSecret named byimagePullSecret, reads it back, never prints the key.hosting-deploy --image <imageRepository>:<pinnedImageTag>with values rendered from the record (HelmValues) — the chart is BAKED INTO THE OPERATOR IMAGE, so the control record'soperator.image(meshweaver.azurecr.io/hosting-operator:<short sha>, published on every push to MeshWeaver main) decides which chart deploys. A stale operator image deploys a stale chart.- rollout wait → 8.
hosting-tls(waits for the Secret to carry a key pair) → 9.hosting-verify→ 10.hosting-verify-catalog(the expected mounts and packages, so an empty Store cannot read as healthy).
3. Filing it
On the control instance (through its own MCP server), create under Deployments/:
{ "id": "<id>-provision", "namespace": "Deployments", "nodeType": "Hosting/InstanceAction",
"name": "Provision <id>", "content": { "$type": "InstanceActionContent",
"deployment": "Deployments/<id>", "requestedAction": "Provision",
"confirmation": "<id>", "dryRun": true, "reason": "…" } }
Dry run first — Authorize → Read record → Plan → "Dry run — nothing changed" prints the composed
step list on the node's log; a refusal here (state: Refused, error naming the field) is the
ANSWER, not an error to retry. Then the real one (dryRun: false). Read state/phase/message
and the log on the node; a failed step names itself and the plan is idempotent from the top, so
fix the cause and file again.
4. After it answers
- A cold instance recompiles every dynamic NodeType. The startup budget on the record
(
startupProbe.budgetSeconds, 3 h on the siblings) exists for this; a pod0/1during the bake is the gate doing its job. Do not cycle pods while it warms. - Register the GitHub Environment and a deployments entry in the estate's private deployments
repository, add the inventory row (see "Where an estate's instances are declared" in
Instances.md), set
Admin/UpdatePolicy(Stable for a customer). A deployment nobody recorded is a deployment nobody will remember to patch.
🚨 Traps that read as success
- HTTP 200 proves nothing. The Blazor shell answers 200 for an error page or a paywall redirect;
hosting-verifychecks rendered content — so should you. - Nobody can sign in is a perfectly healthy instance: every
Authentication__*__ClientIdempty and dev login off means it boots, passes probes, serves, and has nothing to authenticate against. Step 1 is not optional. - The chart ConfigMap emits only the keys it templates; a record
extraPortalConfigkey the chart has no line for reaches NO container, with no error.config-key-coverageis the guard. kubectl set envis not a deployment. Helm's three-way merge never removes an inline entry it did not own; the record'sinlineEnvexists to make such drift visible, not to bless it.- Never emit
""for an int/bool key — it fails binding, DI throws, the landing page dies. - A refused Provision with
hosting-kv-ensurenaming a missing vault object wants the object ISSUED (step 1.2), never a random value. azprints minted secrets on stderr — see step 1.1; a leaked value is rotated in the same turn.
Related
/plugins— the registry the instance consumes packages from/release— how images get built; a tag missing from cr is copied there by hand (step 1.4)/deployment·/delete-user·/storm·/debug— operating a live instance- Design: ContainerRegistryInMemex.md (the registry as a separate service) · Memex#132 (the operator's PGPASSWORD gap)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 192k | Updated | Claude CodeCodex | Verified | Install |