How to create pipeline variables from terraform output

I show you how to transform terraform output into pipeline variables from script

Intro

This is how to set pipeline variable from script. You can read more here

echo "##vso[task.setvariable variable=varname]value"

Terraform output can be outputted as json content.

terraform output -json > sample.json

Suppose the terraform output looks like below:

{
    "app_service": 
    {
        "name": "myapp",
        "fqdn": "myapp.azurewebsites.net"
    },
    "database": {
        "name": "example",
        "server_fqdn": "myserver.database.windows.net",
        "connection_string": "server=myserver.database.windows.net, initial catalog=example"
    },
    "application_insights_key": "12345"
}

You have to think how to transform json representation into pipeline variables. The solution has to work with any json structure.

Solution

First problem to solve is to transform tree representation into flat map of entries like below

{
	"app_service.name": "myapp"
	"app_service.fqdn": "myapp.azurewebsites.net"
	...
}

This is the technique of joining internal nodes by using special character. To do that you can use jq tool.

cat sample.json | jq '[leaf_paths as $path | { key: $path | join("_"), value: getpath($path)}] | from_entries' > flat.json

{
  "app_service_name": "myapp",
  "app_service_fqdn": "myapp.azurewebsites.net",
  "database_name": "example",
  "database_server_fqdn": "myserver.database.windows.net",
  "database_connection_string": "server=myserver.database.windows.net, initial catalog=example",
  "application_insights_key": "12345"
}

Next step is to apply transformation over each entry and produce list of entries separated by new line character, similar to below

##vso[task.setvariable variable=var1]value
##vso[task.setvariable variable=var2]value
##vso[task.setvariable variable=varX]value

Again you use can use jq tool and execute the below command

cat flat.json | jq -r 'to_entries | . as $in | keys[] | $in[.].key as $key | $in[.].value as $value | "##vso[task.setvariable variable=" + $key + "]" + $value' > variables.txt

##vso[task.setvariable variable=app_service_name]myapp
##vso[task.setvariable variable=app_service_fqdn]myapp.azurewebsites.net
##vso[task.setvariable variable=database_name]example
##vso[task.setvariable variable=database_server_fqdn]myserver.database.windows.net
##vso[task.setvariable variable=database_connection_string]server=myserver.database.windows.net, initial catalog=example
##vso[task.setvariable variable=application_insights_key]12345

Finally you have to iterate over the list and output variables to the stream using your system interpreter.

Bash sample

cat variables.txt | xargs -n 1 -d '\n' echo

Powershell sample

Get-Content -Path variables.txt | ForEach-Object {Write-Host $_}

Troubleshoot

If you use powershell to play with jq please remember to escape quotes in filter query to avoid errors. For instance

cat sample.json | jq '[leaf_paths as $path | { key: $path | join(\"_\"), value: getpath($path)}] | from_entries' > flat.json