Support multi-environment deployment to a single APIM instance

Summary

Support deploying multiple environments (e.g., dev, staging, prod) to the same Azure API Management instance, with clear separation between environments via naming conventions or prefixes.

Problem

Currently, apiops-cli assumes a 1:1 mapping between environments and APIM instances. Many organizations want to consolidate multiple environments into a single APIM instance for cost or operational reasons, but need clear separation between environment-specific APIs.

Proposed Solution

Extract

Support extracting APIs from a single APIM instance while tagging/grouping them by environment (e.g., via API name prefix, path prefix, or tags).
Allow filter configuration to scope extraction to a specific environment subset.

Publish

When publishing to a target environment (e.g., "Dev"), automatically apply an environment-specific prefix or suffix to API names/paths (e.g., dev-my-api or /dev/my-api).
Support configuration to define the naming/prefixing strategy per environment.
Ensure environment-specific named values, backends, and other resources are correctly scoped.

Configuration

Extend the configuration system to support an environment concept with per-environment prefixing rules.
Allow users to define environment mappings in their configuration files (e.g., configuration.dev.yaml targets the same APIM instance but with a dev- prefix).

Acceptance Criteria

Extract supports filtering by environment within a single APIM instance
Publish supports applying environment-specific prefixes/suffixes to APIs
Configuration schema supports environment-to-prefix mappings
Documentation covers multi-environment setup on a single APIM instance
Tests cover extract and publish with environment prefixing

Hope we could have this feature implemented soon, our organization is waiting for this feature for long....

jxyiliu1

Thanks jxyiliu1 for the heads-up! I also saw htwashere was interested in this feature too.

I'm working on a design now, but there are some unexpected complexities. Could you review the rough design & plans below and let me know if you have any concerns so we can refine the design?

The current design is that there would either be a 'source of truth' APIM instance, or artifacts in code, which would be the 'base' for all the environments. That way we can do a clean 'extract' without any environment-specific naming. Is that reasonable?
There will probably be some 'global' resources (like policy fragments or loggers) that are specific to the instance and not associated with a particular environment (like dev, qa, stage, prod). Would it be reasonable to expect users to filter those out when publishing? There could be a separate pipeline for publishing those if changes are needed.
The way we expect it to work is that there's a new property in the environment overrides file that has a 'NamePrefix' for that environment. Then when we publish, we would prefix all the names that we publish with that name prefix. (like "dev_" or "prod_").
We would have to do deep re-writes of names as we publish, for example, if a Policy XML uses a named value, we'd need to deeply rewrite the names to point to the environment specific names. Does that sound reasonable?

Thanks!

Peter Hauge (@petehauge)

Thanks @jxyiliu1 for the heads-up! I also saw @htwashere was interested in this feature too.

I'm working on a design now, but there are some unexpected complexities. Could you review the rough design & plans below and let me know if you have any concerns so we can refine the design?

The current design is that there would either be a 'source of truth' APIM instance, or artifacts in code, which would be the 'base' for all the environments. That way we can do a clean 'extract' without any environment-specific naming. Is that reasonable?

There will probably be some 'global' resources (like policy fragments or loggers) that are specific to the instance and not associated with a particular environment (like dev, qa, stage, prod). Would it be reasonable to expect users to filter those out when publishing? There could be a separate pipeline for publishing those if changes are needed.

The way we expect it to work is that there's a new property in the environment overrides file that has a 'NamePrefix' for that environment. Then when we publish, we would prefix all the names that we publish with that name prefix. (like "dev_" or "prod_").

We would have to do deep re-writes of names as we publish, for example, if a Policy XML uses a named value, we'd need to deeply rewrite the names to point to the environment specific names. Does that sound reasonable?

Thanks!

Hi Peter Hauge (@petehauge), thanks for your attention to this feature. Here are my thoughts to your design notes above:
Point 1: I think it is reasonable. Your ApiOps Extract currently asks for an APIM instance as a source of truth. APIM enforces the API-name to be unique anyway. If we use one APIM instance to serve multiple environments for the same API, we need to differentiate them via the API-name anyway so technically, APIM will consider them as two separate APIs (as you are pointing it out in point 3)
Point 2: In our use case after examining your ApiOps Extract, I think we will be using your filter feature to limit the extracted resources. So I think when we publish to the next APIM, those global resources will be excluded?
Point 3: Could the NamePrefix be a token where we can place anywhere in the API-Name string? I am just thinking for organizations like ours who already have certain naming convention. Or, perhaps the environment be a "suffix" better?
Point 4: I think you are correct regarding deep-rewrite. I am not too familiar with the overall structure of the files yet.
BTW, is there an explanation on all the files that are extracted for the resources? At this moment, I am trying to understand the purpose of file apiInformation.json in the context of the extract and publish. Where does APIM use it?
I will be away next week so will be slow in responding to you in subsequent dialogues.
Thank you!
Henry

htwashere

Hi htwashere -

For 2 - you're correct, that if you filter on extract, anything filtered out don't have associated files on disk and aren't published

For 3 - could you share your naming scheme? We could do either a prefix or suffix - just wanted to confirm that we wouldn't need to replace tokens in the middle of the names?

We don't have any docs on explanation of the specific files but we could probably write something. If you need it - could you log a new github issue requesting the docs? In terms of apiInformation.json, these are typically the properties associated with an individual API. I included a screenshot of an example I have - things like the revision, path, display name, protocols, service URL, etc. It's the key properties we get back when we do a 'get' against the API in API Management. You'll see a lot of these things on the "Settings" tab for an API (2nd screenshot below)

Peter Hauge (@petehauge)

Hi Peter Hauge (@petehauge) , our naming convention is still being standardized but I see many of our teams are settling to: "dept env apiname" or "dept apiname env". I think we can settle for a suffix but if you could do replace tokens for the name, I can see that being the best? While we are on this name topic, I suppose with the displayName, we will continue to use the "override" feature to change the Display Name so no issue there?

Thanks for the apiInformation.json explanation. Are you using it for any logic processing in apiops publish? I am thinking if that file needs to be considered (overrides) for the target APIM?

htwashere

Hi htwashere - on the replacing tokens, this is hard because either:

There's some tag you put in all the right places like the word "env" in the live instance, and never have another name that uses the same text accidently (like envelope, envoy, etc), or
You manually fix all the artifacts on disk to add the token like "{env}" that we would replace

Via the cli, we could do the replacement, but I'm just concerned about the user experience for it - that it might be error prone and difficult for users. If we support both prefix & suffix, it sounds like that would be enough (and is straightforward from a user experience perspective)?

On the apiInformation.json file - there are some things we currently do during publish:

After reading in the json file:
Normalize MCP tool operation references
Apply overrides (this is a deep merge)
If there's a spec file next to apiInformation.json, we strip null-valued properties
Fixup the properties.apiType field
Sanitize the spec (add missing openAPI path parameters for example)
Put the resulting payloads to APIM
If we were replaying revisions, then run again to re-align the active-revision metadata

(and then if this issue is fixed, it will add another bullet to apply name prefix/suffix)

Multi-Environment Publish — Design Proposal

Scoping assumptions:

Extract stays as-is. Artifacts on disk hold canonical (unprefixed) names, produced from a dedicated "source-of-truth" APIM.
Publish applies an environment namePrefix and/or nameSuffix declared in the override file.

1. Config surface

Extend the override YAML with a new top-level environment block:

# configuration.dev.yaml
environment:
  namePrefix: "dev-"      # optional
  nameSuffix: ""          # optional
  # Applies to which resource types (see §2).
  # Omit = apply to the default set. Explicit list = apply only to those.
  appliesTo:              # optional
    - apis
    - products
    - namedValues
    - backends
    - loggers
    - policyFragments
    - versionSets
    - tags
    - groups
    - subscriptions
    - workspaces
  # Independent from name affix — API URL path also needs uniqueness on shared APIM.
  apiPathPrefix: "dev/"   # optional; prepended to properties.path on APIs

apis:
  - name: petstore-api    # <-- canonical name, no prefix
    properties:
      serviceUrl: https://petstore-dev.contoso.com

Key rules:

The override file continues to reference resources by canonical name (matches artifacts on disk). This preserves the "extract once, override per env" model.
environment is optional. If absent, behaviour is identical to today (100% back-compat).
Bump schema v1 → v2 per the repo's update-config-schema-version skill. v1 files continue to load (no environment block = no-op).

Why in the override file, not CLI flags: Env metadata belongs with env-specific values, is version-controlled, reviewable, and testable. CLI flags could be added later as an escape hatch but shouldn't be the primary UX.

2. Which resource types get affixed

Not everything can be renamed safely. Proposed default set:

Type	Affix?	Why
Api	✅	Primary case
Product	✅	Primary case
NamedValue	✅	Displayname is APIM's lookup key for `{{ref}}` — see §4
Backend	✅	Referenced by name in policies
PolicyFragment	✅	Referenced by name in policies
Logger	✅	Referenced by name in diagnostics
VersionSet, Tag, Group, Subscription, Workspace	✅	Global namespace on APIM
Gateway	⚠️ opt-in	Often shared infra; default off
ServicePolicy (singleton)	❌	Fixed name "policy"; can't be renamed
Child resources (ApiOperation, ApiDiagnostic, ApiPolicy, ApiRelease, ProductGroup, ProductApi, GatewayApi, ApiTag, etc.)	❌ directly, ✅ via parent	Names are scoped inside the parent; only the parent's segment is affixed
Documentation, GlobalSchema, PolicyRestriction	✅	Global namespace

Users override the default via appliesTo when needed.

3. Where the transform lives — a single `env-mapper` module

New file: src/services/env-mapper.ts. Exports:

export interface EnvMapping {
  prefix: string;
  suffix: string;
  appliesTo: Set<ResourceType>;
  apiPathPrefix?: string;
}

export function buildEnvMapping(overrides: OverrideConfig | undefined): EnvMapping | undefined;

// canonical → deployed name (used at publish time)
export function toDeployedName(name: string, type: ResourceType, m: EnvMapping): string;

// deployed → canonical (used by delete-unmatched to scope diff)
export function toCanonicalName(deployedName: string, type: ResourceType, m: EnvMapping): string | undefined;

// belongs to this env's namespace?
export function isInEnvNamespace(deployedName: string, type: ResourceType, m: EnvMapping): boolean;

// rewrite a descriptor's nameParts (parent segment + optionally child segments)
export function mapDescriptor(d: ResourceDescriptor, m: EnvMapping): ResourceDescriptor;

Design choice — map at the publish edge, not throughout the code. Everywhere that reads artifacts, applies overrides, does dependency graph work: keep using canonical names. Rewrite to deployed names in exactly two places:

Right before client.putResource(...) in resource-publisher.ts
Right before client.deleteResource(...) in the delete path

Advantages: filter, override-merger, artifact-store, dependency-graph, transitive-resolver all stay untouched. Testability stays high. applyOverrides() still matches by canonical name, which is what the user wrote in their YAML.

4. Cross-reference rewrites (the hard part)

At publish time the following JSON/XML payloads carry embedded references that must be rewritten from canonical → deployed:

Location	Ref shape	Fix
`ProductApi`, `ProductGroup`, `GatewayApi` association entries	`entry.name`	Map through `toDeployedName` for the child type (API/Group). Already loops per-entry in `resource-publisher.ts:307-322` — inject the mapping there.
`Subscription.properties.scope`	`/apis/<apiName>` or `/products/<productName>`	Extend the existing `normalizeSubscriptionScope` path to also affix the trailing segment.
`ApiRelease.properties.apiId`	ARM path ending `/apis/<apiName>[;rev=N]`	Extend existing `normalizeApiReleaseApiId` to affix.
API revisions `sourceApiId`	ARM path	Same treatment.
Logger `credentials` `{{ref}}` values	Placeholder tokens resolved by NamedValue displayName	Existing `rewriteNamedValueReferences` already looks up the canonical named value. If NamedValue is in the affix set, its `displayName` override (if any) is already applied there — but if the user didn't set displayName, we must also affix it. Extend that function to affix displayName when NamedValue ∈ `appliesTo` and no explicit override was provided.
Policy XML content	`{{namedValueRef}}`, `<include-fragment fragment-id="..."/>`, `<send-request>` back-refs, etc.	New policy-xml rewriter (§5).
API `properties.path`	URL suffix	Prepend `apiPathPrefix` when present. Small, isolated change in the Api branch of `resource-publisher.ts`.
Workspace-scoped child descriptors	`descriptor.workspace` and child `nameParts[0]` (parent id)	`mapDescriptor` handles both segments.

5. Policy XML rewriting

New util src/services/policy-ref-rewriter.ts:

export function rewritePolicyRefs(xml: string, m: EnvMapping, ctx: {
  knownNamedValues: Set<string>;      // canonical names present in artifacts
  knownFragments: Set<string>;
  knownBackends: Set<string>;
}): string;

Approach:

Do not parse XML. Use targeted regex passes for {{name}} tokens and known element/attribute patterns (<include-fragment fragment-id="...">, <set-backend-service backend-id="...">). This matches the existing "opaque payload" philosophy (FR-009) — we only touch tokens we can confidently identify.
Only rewrite refs where the canonical name is known to be part of the artifact set for that type and that type is in appliesTo. Unknown tokens pass through unchanged (they may be runtime context refs like {{context.request.headers}}).
Called from the policy branch of resource-publisher.ts before PUT.

Risk: false-positives if a customer's XML text incidentally matches. Mitigated by (a) restricting to tokens whose base name is in the known artifact set and (b) opt-out via appliesTo not listing that type.

6. `delete-unmatched` — the safety-critical piece

Today computeDeleteActions will happily delete a Prod resource when publishing Dev. Change:

client.listResources(context, type) returns ARM (deployed) names.
For each resource, if EnvMapping is present and this type ∈ appliesTo: skip anything not in this env's namespace (isInEnvNamespace(deployedName, type, m) === false).
For resources in-namespace: convert deployed name → canonical name via toCanonicalName, then do the existing "is this in local artifacts?" check.
Delete uses the deployed descriptor (rewritten via mapDescriptor).

This makes shared-APIM publish safe: Dev can never delete Prod because Prod resources are outside the dev-* namespace.

Edge case: if a user drops the prefix mid-life (namePrefix: "" after previously publishing with dev-), delete-unmatched can't distinguish the two envs anymore. Document as a known constraint.

7. Collision & validation checks (fail fast at publish start)

Add a pre-flight validator in publish-service.ts:

If environment present, warn if prefix/suffix contain characters APIM disallows in resource names (must match ^[A-Za-z0-9-]+$ for most types).
Error if appliesTo includes a type that can't be renamed (ServicePolicy, association children).
Warn if an override entry references a canonical name that doesn't exist in artifacts.
Error if apiPathPrefix is set but no API in the artifact set — likely a typo.

8. Extract stays canonical — but add one convenience

Per the scoping assumption, extract needs no code change. Worth adding one docs section: users should extract from a dedicated non-shared APIM (or an APIM whose resources happen to have no prefix). If someone points extract at a shared APIM, they'll get all envs' resources mixed together — call this out explicitly.

Optional (Phase 3, not blocking): apiops extract --strip-prefix dev- for teams that only have a shared APIM. Purely a name-strip pass at write time. Not required for MVP.

9. `init` changes

Minimal:

The scaffolded configuration.<env>.yaml gets a commented-out environment: block with prefix/suffix examples.
The Copilot prompt (configure-overrides-prompt.md) gains a section explaining when to enable it.
No change to workflow templates — the same apiops publish --overrides configuration.<env>.yaml command handles both single-APIM-per-env and shared-APIM setups.

10. Testing

Unit tests for env-mapper.ts (round-trip: canonical → deployed → canonical for every type; unaffected types unchanged).
Unit tests for policy-ref-rewriter (known refs rewritten, unknown left alone, no XML corruption).
Integration tests for resource-publisher.ts covering: association entries, subscription scope, apiRelease apiId, api path prefix, logger credentials.
Delete-unmatched test using a mock APIM listing that returns both dev-foo and prod-foo — publish with namePrefix: dev- must skip prod-foo.
Back-compat: existing test suite must pass unchanged with no environment block.

⚠️ Stale issue notice — This issue has had no activity for 30 day(s).

It will be automatically closed after 60 days of inactivity.

To keep this issue open:

Leave a comment describing the current status, or
Remove the stale label.

This notice was posted automatically by the stale issue workflow.

feat: multi-environment publish to shared APIM instance

Navigation Menu

Repository navigation

Support multi-environment deployment to a single APIM instance #132

Summary

Problem

Proposed Solution

Extract

Publish

Configuration

Acceptance Criteria

Multi-Environment Publish — Design Proposal

1. Config surface

2. Which resource types get affixed

3. Where the transform lives — a single `env-mapper` module

4. Cross-reference rewrites (the hard part)

5. Policy XML rewriting

6. `delete-unmatched` — the safety-critical piece

7. Collision & validation checks (fail fast at publish start)

8. Extract stays canonical — but add one convenience

9. `init` changes

10. Testing

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support multi-environment deployment to a single APIM instance #132

Description

Summary

Problem

Proposed Solution

Extract

Publish

Configuration

Acceptance Criteria

Activity

jxyiliu1 commented on Jul 3, 2026

petehauge commented on Jul 3, 2026

htwashere commented on Jul 3, 2026

petehauge commented on Jul 3, 2026

htwashere commented on Jul 3, 2026

petehauge commented on Jul 4, 2026

petehauge commented on Jul 4, 2026

Multi-Environment Publish — Design Proposal

1. Config surface

2. Which resource types get affixed

3. Where the transform lives — a single env-mapper module

4. Cross-reference rewrites (the hard part)

5. Policy XML rewriting

6. delete-unmatched — the safety-critical piece

7. Collision & validation checks (fail fast at publish start)

8. Extract stays canonical — but add one convenience

9. init changes

10. Testing

github-actions commented on Aug 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

3. Where the transform lives — a single `env-mapper` module

6. `delete-unmatched` — the safety-critical piece

9. `init` changes