The parts a pipeline keeps to itself

A pipeline that works is a temptation. The first one takes a week to get right; the second is a copy of the first with two words changed; by the thirty-first, an organisation has thirty-one YAML files that all build, test, and deploy in almost exactly the same way — and the day someone decides every deploy must now run a security scan, that is thirty-one files to edit, thirty-one reviews to chase, and one file quietly missed that becomes the gap an auditor finds a year later. Copy-paste is not a time-saver here; it is a debt that compounds silently, paid in the one pipeline nobody remembered to change. This part is about not incurring it — about writing the shared thing once and pointing many pipelines at it.

Azure Pipelines gives you three tools for staying out of that trap, and they are the whole of this part. Variables pull the values that differ — a resource-group name, a build configuration, a secret — out of the pipeline body so the same steps run against different inputs. Parameters let a reusable pipeline be shaped by its caller before it runs, with types the system checks. And templates let steps, jobs, or whole pipeline shapes be written once and reused across many files, so the security scan is added in one place and lands in all of them. Keep the three straight and a pipeline estate stays DRY, configurable, and — as the templates section will argue — safe. Confuse them, and the most common way to do so is a trap hiding in a brace.

A variable is a name-value pair you define once and reference many times — a build configuration, a resource-group name, a flag that turns a step on or off. That much is ordinary. What trips up nearly everyone, on the exam and painfully on the job, is that Azure Pipelines reads variables at three different moments, and the moment is chosen by the punctuation you wrap the name in. Get the punctuation wrong and the pipeline does not error; it quietly reads the wrong thing — usually an empty string — and ships it. This is the single most consequential fact in this part, so it is worth slowing down for.

Macro syntax, $(var), is the everyday form. It is expanded at runtime, just before a step runs, and the agent substitutes the value in place. If the variable has no value, the agent leaves the text $(var) untouched rather than blanking it, on the reasoning that an empty substitution might silently break the script it lands in. Template expression syntax, ${{ variables.var }}, is different in kind: it resolves at compile time, before the run starts at all — so it can only see values that already exist when the YAML is assembled, and anything set later, during the run, reads as empty. Runtime expression syntax, $[ variables.var ], is evaluated as jobs are dispatched, which is why it is the form you use in conditions and in passing output variables between jobs. Three braces, three clocks.

Pick the wrong brace, ship an empty string.

The rule that saves you is to ask, of every variable, does the value exist yet when this line is read? A value another step produces during the run does not exist at compile time, so reaching for it with ${{ }} is asking a question before the answer was written — and the answer comes back blank. That one confusion is the spot-the-error drill at the end of this part, because it is the mistake you will actually make.

Some values must not sit in the YAML at all. A database password committed to a repository is a password published to everyone who can read the repository — and in a pipeline that means every contributor, every fork, and the whole history that git never forgets. Azure Pipelines lets you mark a variable as secret, which keeps its value out of logs and off the definition, and it masks the value in output. But a secret typed into a pipeline is still a secret you are managing by hand, per pipeline, and that does not scale past a few.

The answer to values shared across many pipelines is a variable group, which lives in the project's Library rather than in any one pipeline. Define the group once — the connection strings, the region, the non-secret configuration a family of pipelines shares — and each pipeline references it. Change a value in the Library and every pipeline that references the group gets the new value, with no file edited. And the group can do the one thing that matters most for secrets: it can link to Azure Key Vault, so that the secret values are never stored in Azure DevOps at all. The group holds only the names of the secrets; the pipeline fetches the current values from Key Vault at run time, under an access policy you control, and the secret lives its whole life in the vault built to hold it.¹

Variable group

A named set of variables stored in the project's Library and shared across pipelines, so a shared value is defined and changed in one place. A group can be linked to an Azure Key Vault, in which case it stores only the secret names and the pipeline reads the values from the vault at run time — the secret never rests inside Azure DevOps.

That link is the shape you want to reach for by reflex. It puts secrets where audit, rotation, and access control already live, and it means a person with permission to edit a pipeline is not thereby a person who can read production's database password. The secret and the pipeline that uses it are kept deliberately apart — the same instinct, in miniature, that keeps a backup off the disk it protects.

Variables carry values through a pipeline while it runs. Parameters do something earlier and different: they let the caller of a reusable pipeline decide its shape before it runs at all. A parameter is declared with a type — string, number, boolean, object, or the special step, job, and stage list types — and it is resolved at compile time, alongside the ${{ }} template expressions of §2. Because the type is checked when the YAML is assembled, a caller who passes the string "maybe" where a boolean was declared is stopped before a single agent starts, not three minutes into a deploy.

The distinction from a variable is worth stating cleanly, because it is a common interview question. A parameter is compile-time and typed, and it can change what steps exist — you can wrap a whole test stage in ${{ if parameters.runTests }} so that, depending on the value passed, the stage is present in the pipeline or is never generated. A variable is a value the running pipeline reads; it cannot add or remove steps, because by the time a variable is read at runtime the set of steps has already been decided. Put plainly: parameters decide what the pipeline is; variables decide what it works with. That is exactly why templates, next, take parameters and not merely variables — a template's whole job is to let its caller shape it.

A template is a piece of pipeline — a set of steps, a job, a stage, or a whole pipeline shape — written in its own file and reused by others. It is the direct answer to the thirty-one-copies problem: write the build-and-deploy sequence once, add the security scan to it once, and every pipeline that uses the template gets the scan. There are two ways a pipeline pulls a template in, and the difference between them is not cosmetic — it is the difference between a convenience and a control.

With include (referencing a template as a step, job, or stage), the template's content is inserted at the point you name, and your pipeline stays in charge of everything around it. Handy, DRY, and entirely voluntary — a pipeline can include a template and then do whatever else it likes. With extends, the relationship inverts: the template owns the whole pipeline shape, and your file may only supply the parameters the template allows. It cannot add a rogue stage, skip the scan, or slip in a step the template did not sanction. That inversion is why extends is a security boundary, not merely reuse: an organisation can require, through a check on a protected resource, that every pipeline touching production extends a specific approved template — and thereby guarantee that every deploy, no matter who wrote the pipeline, passes the gates the template encodes.²

Step / job / stage template

A fragment reused by include. The calling pipeline stays in control and can do anything around the fragment.

extends template

Owns the entire pipeline shape. The caller supplies only allowed parameters and can add nothing the template did not permit — which is what makes it enforceable.

Required check

A rule on an environment or service connection that a pipeline must extend a named template to use it. This is how an org turns a template into a gate no one can route around.

Here is a step template shaped by a parameter, and a pipeline that extends it — the caller changes one value and can change nothing else:

# build-deploy.yml — the approved template, written once
parameters:
- name: project
  type: string
- name: runScan
  type: boolean
  default: true

stages:
- stage: Build
  jobs:
  - job: build
    steps:
    - script: dotnet build $(project)
    - ${{ if eq(parameters.runScan, true) }}:
      - script: ./security-scan.sh   # the gate the org requires

# azure-pipelines.yml — one of thirty-one, now four lines
extends:
  template: build-deploy.yml
  parameters:
    project: warehouse.csproj

Every pipeline in the estate collapses to those last four lines, and the scan can never be dropped from any of them, because the pipeline is not permitted to remove it. One file to change, one gate no one can skip — DRY and safe in the same move.