Get best of the week

Using variables in Azure DevOps pipelines

We continue to review the wonderful tool for development for Windows and not only, Azure DevOps. This time, tormented with environment variables, I decided to put all the experience in one article.

Starting from the fact that for each runtime environment they have different syntax, ending with the lack of a standard ability to transfer variables from one stage of the pipeline to another.

I’ll make a reservation that the main examples will be at Release Pipelines, because YAML has not yet reached it, and I need the functionality of many stages and many artifacts. This, it seems, became available in regular Pipelines, which practically leveled them in functionality. In Pipelines, YAML added and added to the text view a small graphic tooltip with parameters that can be set. Very convenient, no need to go into the documentation for each module. But I will describe this in the next article, but for now, here is a picture of the innovation itself.



Storage and use


To begin with, we have default variables in the system. They begin, depending on the origin, with the words Release, System, etc. A complete list (as it turned out, no) is available in the documentation . All schizophrenia with syntax is illustrated by an example from the documentation below. The same variable has three representations, depending on where we call it.

steps:
 - bash: echo This script could use $SYSTEM_ACCESSTOKEN
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)
  - powershell: Write-Host "This is a script that could use $env:SYSTEM_ACCESSTOKEN"
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)

If you set a variable on the agent that is executing the task, this is $ (System.AccessToken). If you want to use it inside a powershell script on the same agent, it will already be $ env: SYSTEM_ACCESSTOKEN. If, God forbid, you want to use this variable on some remote host using the PowerShell on target machines task, you need to pass this through an argument to the script using param . With bash simpler, you can just throw it in using the argument and syntax $ SYSTEM_ACCESSTOKEN.

The same laws do not apply to your own variables, here you are already responsible for the syntax. You can set variables locally in each task.



Or globally in the repository of variables, and then link them from the repository. Very comfortably.



As a bonus to this, if the variables are very secret, they can be stored in the Azure cloud in a storage called Azure Vault, Vault can be linked to the project in the Library.



In general, everything is clear with variables, in pipelines you can still set them manually for each launch, in release there is no such functionality. You can see what you transfer to the pipeline again in the agent initialization logs, but note that they are already in the converted form there.



Dynamic variables


The most interesting part starts when we want to get some value in one stage and pass it to the next.



Such functionality was not delivered to us. But our hands are not for boredom and with the help of Google a solution was found. Thank God, Azure DevOps has an API that allows us to do a little more than we drew in the interface.

So, we need a call to update global variables, which we will do right from the inside of the pipeline. The address is taken from environment variables, the same of which there is not a word in the documentation, as mentioned earlier. You can ask them yourself or, what’s there, hardcode if they close the shop.

$releaseurl = ('{0}{1}/_apis/release/releases/{2}?api-version=5.0' -f $($env:SYSTEM_TEAMFOUNDATIONSERVERURI), $($env:SYSTEM_TEAMPROJECTID), $($env:RELEASE_RELEASEID)  )

Set the empty value of the variable that we want to pass, set Scope - Release.



For example, we make some random generator of values. Pay attention to the syntax of the variable declaration inside this stage, such functionality was introduced.



In the next step, we pass the variable to the script, yes yes, it’s impossible directly, it is necessary through the argument.



The script under the spoiler

Powerhell
#Script requires stageVar variable in release variables set to Release scope

param ( [string] $expVar )
#region variables
$ReleaseVariableName = 'StageVar'
$releaseurl = ('{0}{1}/_apis/release/releases/{2}?api-version=5.0' -f $($env:SYSTEM_TEAMFOUNDATIONSERVERURI), $($env:SYSTEM_TEAMPROJECTID), $($env:RELEASE_RELEASEID)  )
#endregion


#region Get Release Definition
Write-Host "URL: $releaseurl"
$Release = Invoke-RestMethod -Uri $releaseurl -Headers @{
    Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
#endregion

#region Output current Release Pipeline
Write-Output ('Release Pipeline variables output: {0}' -f $($Release.variables | ConvertTo-Json -Depth 10))
#endregion


#region Update StageVar with new value
$release.variables.($ReleaseVariableName).value = "$expVar"
#endregion

#region update release pipeline
Write-Output ('Updating Release Definition')
$json = @($release) | ConvertTo-Json -Depth 99
Invoke-RestMethod -Uri $releaseurl -Method Put -Body $json -ContentType "application/json" -Headers @{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
#endregion

#region Get updated Release Definition
Write-Output ('Get updated Release Definition')
Write-Host "URL: $releaseurl"
$Release = Invoke-RestMethod -Uri $releaseurl -Headers @{
    Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
#endregion

#region Output Updated Release Pipeline
Write-Output ('Updated Release Pipeline variables output: {0}' -f $($Release.variables | ConvertTo-Json -Depth 10))
#endregion


Or

Bash
INPUT_VAR=$1
RELEASE_VAR=$2

echo Test ID: ${INPUT_VAR}

RELEASE_URL="${SYSTEM_TEAMFOUNDATIONSERVERURI}${SYSTEM_TEAMPROJECTID}/_apis/release/releases/${RELEASE_RELEASEID}?api-version=5.0"

echo release url: $RELEASE_URL

RELEASE_JSON=$(curl -H "Authorization: Bearer $SYSTEM_ACCESSTOKEN" $RELEASE_URL)

OUTPUT=`jq ''.variables.${RELEASE_VAR}.value' = '\"${INPUT_VAR}\"'' <<< $RELEASE_JSON`

curl -H "Authorization: Bearer $SYSTEM_ACCESSTOKEN" -H "Content-Type: application/json" -X PUT -d "$OUTPUT" $RELEASE_URL


In a nutshell, our script takes the input variable myVar and using the API puts the value of this variable in stageVar. In the next step, using the syntax of system variables, we can look at it.



The example is quite simple, but the functionality offers us good opportunities, in total with my last article , when we can create a virtual machine at the first stage of testing, do some further manipulations with it, and at the same time several. And the final step is to destroy it. Now we are chasing product self-tests every time on fresh virtual machines. Given that they live for 10 minutes, it’s worth a penny.

In the next article, if necessary, I will talk about YAML pipelines, there are quite a lot of interesting innovations lately.

More articles:


All Articles