Testing Bicep Before It Breaks Production: What-If, PSRule and CI Validation

The deployment that taught me this lesson took down an API for forty minutes. The change was a one-line edit to a Bicep template, adding a subnet to an existing virtual network. What-if reported the subnet being added and nothing else. What actually happened was that submitting the subnets array without the existing subnet in it removed the existing subnet, along with the private endpoints that lived in it.

What-if told the truth about the template. It did not tell the truth about the outcome, and those are different things. That distinction is what this post is about.

Bicep has no test framework in the way application code does, and people take that to mean Bicep cannot be tested. It can. You just have to layer three things that each catch a different class of problem: the compiler catches syntax, PSRule catches bad configuration, and what-if catches unintended change. Any one of them alone leaves a gap big enough to walk an outage through.

I cover what-if and conditional deployments on video, which is useful background for the third layer below:

YouTubeYouTubeAzure Bicep From Scratch: One Template, Every Environment (Ep. 03)youtube.comWatch the video

The Three Layers, and What Each Actually Catches

Layer	Catches	Misses
`az bicep build` and linter	Syntax, unused params, bad references	Anything semantically valid but wrong
PSRule for Azure	Insecure or non-compliant configuration	Whether the change is what you intended
`what-if`	Change to existing resources	Property-level nuance on some types

Run all three, in that order, and stop the pipeline at the first failure. The order matters because each is more expensive than the last, and there is no point running what-if against a subscription for a template that does not compile.

Layer One: Build and Lint

Cheapest and most often skipped, because people assume the editor already did it. The editor did it for files you had open.

# Compiles and surfaces linter warnings
az bicep build --file main.bicep --stdout > /dev/null

The linter is configured in bicepconfig.json, and the defaults are too permissive for a shared codebase. This is the set I use:

{
  "analyzers": {
    "core": {
      "enabled": true,
      "rules": {
        "no-hardcoded-env-urls": { "level": "error" },
        "no-unused-params": { "level": "error" },
        "no-unused-vars": { "level": "error" },
        "prefer-interpolation": { "level": "error" },
        "secure-parameter-default": { "level": "error" },
        "outputs-should-not-contain-secrets": { "level": "error" },
        "no-hardcoded-location": { "level": "error" },
        "use-recent-api-versions": { "level": "warning" }
      }
    }
  }
}

outputs-should-not-contain-secrets and secure-parameter-default are the two that earn their place immediately. Both prevent a secret ending up somewhere it can be read later, and both are set to warning by default, which means most people never see them.

Pro tip: Set no-unused-params to error rather than warning. An unused parameter usually means someone refactored and left a caller passing something that no longer does anything, which is the kind of drift that makes a template lie about its own interface.

Layer Two: PSRule for Azure

PSRule is where the actual quality bar lives. It expands your Bicep to ARM, then evaluates it against several hundred rules derived from the Well-Architected Framework and Azure security baselines. It catches things like storage accounts allowing public blob access, key vaults without soft delete, and network security groups permitting inbound from any.

Configure it with ps-rule.yaml at the repository root:

# ps-rule.yaml
include:
  module:
    - PSRule.Rules.Azure

input:
  pathIgnore:
    - '**/*.md'
    - 'docs/'
    - '**/*.test.bicep'

configuration:
  # Point PSRule at the Bicep CLI so it can expand templates
  AZURE_BICEP_FILE_EXPANSION: true
  AZURE_BICEP_PARAMETER_FILE_EXPANSION: true
  AZURE_BICEP_FILE_EXPANSION_TIMEOUT: 30

rule:
  exclude:
    # We use a central egress firewall, so per-subnet UDR checks do not apply
    - Azure.Subnet.UseNSG

output:
  culture:
    - 'en-CA'

AZURE_BICEP_FILE_EXPANSION is the setting that makes this work on Bicep source rather than compiled JSON. Without it PSRule has nothing to evaluate.

Run it locally before you push:

Install-Module -Name PSRule.Rules.Azure -Scope CurrentUser -Force
Assert-PSRule -InputPath './infra/' -Module PSRule.Rules.Azure -Format File

The first run against an existing codebase is humbling. Expect a few hundred failures on a mature repository, most of them legitimate.

Suppress Deliberately, Not Reflexively

You will need to suppress rules. The question is whether the suppression records a decision or hides a problem.

Do it in ps-rule.yaml with a comment explaining why, rather than by deleting the rule from the module:

suppression:
  Azure.Storage.UseReplication:
    # Diagnostics storage is regenerable. LRS is a deliberate cost decision.
    - 'stdiagnosticsprod'

  Azure.KeyVault.PurgeProtect:
    # Ephemeral vault, torn down with the environment nightly.
    - 'kv-ephemeral-test'

Suppression is scoped to the named resource, so a new storage account does not silently inherit the exception. That is the whole point, and it is why suppressing by resource beats excluding the rule globally.

Warning: Never suppress a rule to make a pipeline green under deadline pressure without writing down why. Six months later nobody can tell the difference between a considered exception and someone who wanted to go home, and the safe assumption becomes "it was probably fine", which is how standards erode.

What-if compares your template against the live environment and reports the delta. It is the only layer that knows about what already exists.

az deployment group what-if \
  --resource-group rg-app-prod \
  --template-file main.bicep \
  --parameters @prod.bicepparam \
  --result-format FullResourcePayloads

Use FullResourcePayloads in CI rather than the default. The default summarises, and the summary is where nuance goes to die.

Now the blind spots, because trusting this thing blindly is how I lost forty minutes of API availability.

Arrays are replaced, not merged. This is the one that got me. Submitting a subnets array replaces the whole array. What-if shows the resource being modified without always making clear that omitted array members disappear. The same applies to NSG rules, access policies and firewall rules.

Some resource types report noisy false positives. Certain properties are reported as changing on every run because the API returns a normalised value different from what you submitted. Teams learn to skim past these, which is the dangerous habit.

Nested and child resources are inconsistently reported. A change to a child resource declared inside a parent sometimes surfaces as a change to the parent with no detail.

It cannot see what your deployment does not describe. Deleting a resource block from your template produces no what-if output at all, because incremental mode is not going to touch it. This is the Bicep deletion gap, and no amount of what-if will surface it.

The mitigation for the array problem specifically is to read the existing resource and merge rather than assume:

// Read what is already there
resource existingVnet 'Microsoft.Network/virtualNetworks@2023-11-01' existing = {
  name: vnetName
}

// Declare subnets as child resources, so each is managed independently
resource newSubnet 'Microsoft.Network/virtualNetworks/subnets@2023-11-01' = {
  parent: existingVnet
  name: 'snet-api'
  properties: {
    addressPrefix: '10.20.3.0/24'
  }
}

Declaring subnets as child resources rather than as an array on the parent means adding one does not touch the others. Do this for every array property that represents independently managed things.

Wiring It All Into CI

Here is the pipeline that runs all three layers, with what-if posted onto the pull request so a human reviews the change rather than the code that produced it.

name: Validate Infrastructure

on:
  pull_request:
    paths: [ 'infra/**' ]

permissions:
  id-token: write
  contents: read
  pull-requests: write

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build and lint
        run: |
          for f in $(find infra -name '*.bicep' -not -name '*.test.bicep'); do
            echo "Building $f"
            az bicep build --file "$f" --stdout > /dev/null
          done

      - name: PSRule analysis
        uses: microsoft/ps-rule@v2
        with:
          modules: PSRule.Rules.Azure
          inputPath: 'infra/'
          outputFormat: Sarif
          outputPath: psrule-results.sarif

      - name: Upload PSRule findings
        if: always()
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: psrule-results.sarif

      - uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: What-if against production
        id: whatif
        run: |
          {
            echo 'RESULT<<EOF'
            az deployment group what-if \
              --resource-group rg-app-prod \
              --template-file infra/main.bicep \
              --parameters @infra/prod.bicepparam \
              --no-pretty-print
            echo EOF
          } >> $GITHUB_OUTPUT

      - name: Comment what-if on the PR
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: `### What-if against prod\n\n\`\`\`\n${{ steps.whatif.outputs.RESULT }}\n\`\`\``
            })

Two details worth copying. PSRule output goes to SARIF and gets uploaded to code scanning, so findings appear inline on the diff rather than in a log nobody opens. And what-if runs against production on the pull request, using a read-only identity, because a what-if against a dev subscription tells you nothing about what will happen in prod.

Gotcha: The identity running what-if needs write permissions on paper even though it changes nothing, because ARM validates the deployment as if it would run. Give it a scoped role on the target resource group and accept that, or the what-if step fails with an authorization error that looks like a bug.

What I Would Add Next

If you have all three layers running and want to go further, the next thing worth building is a deployment test: deploy the template to an ephemeral resource group, assert the resources came out right, tear it down.

RG="rg-test-$(date +%s)"
az group create --name $RG --location canadacentral --tags purpose=ci-test

az deployment group create \
  --resource-group $RG \
  --template-file main.bicep \
  --parameters @test.bicepparam

# Assert what matters, not everything
TLS=$(az storage account show --resource-group $RG \
  --name $(az storage account list -g $RG --query "[0].name" -o tsv) \
  --query minimumTlsVersion -o tsv)
[ "$TLS" = "TLS1_2" ] || { echo "FAIL: TLS version is $TLS"; exit 1; }

az group delete --name $RG --yes --no-wait

Keep this for shared modules rather than every application template. It costs real deployment time, and the value is highest exactly where a mistake propagates to many consumers.

Conclusions

Bicep is testable, just not with one tool. The compiler catches what is malformed, PSRule catches what is misconfigured, and what-if catches what is about to change. Skip any of the three and you have a category of failure with nothing watching it.

The one thing I would fix first, if you do nothing else here, is the array replacement trap. Declare independently managed things as child resources rather than as arrays on a parent. That single change removes the most dangerous gap between what what-if says and what actually happens.

What's Next

Bicep modules and the private registry, where a validation pipeline matters most because a bad publish reaches every consumer
Azure Verified Modules, which arrive with their own tests and remove a large share of what you would otherwise be validating
Bicep vs Terraform on Azure for the deletion gap that no amount of testing closes