Every now and again you stumble across a command that solves a problem you didn’t realise you had. Recently, I discovered envsubst.
“Substitutes the values of environment variables.”
I’ve found myself substituting environment variables in configuration files increasingly often (see: Kubernetes). On more than one occasion I’ve tied myself in knots trying to do this with scripts. It turns out that envsubst does most of what I need.
Take the following file, example.yaml:
name: $MY_NAME
weather: $MY_WEATHER
I can use envsubst to swap these environment variables with the following.
export MY_NAME=bill
export MY_WEATHER=sunny
cat example.yaml | envsubst
This will output the following.
name: bill
weather: sunny
This is delightfully simple, but there is a catch. Beware the behaviour when these environment variables are not set.
unset MY_NAME
Then re-run the command above.
cat example.yaml | envsubst
Your output should now look like the below.
name:
weather: sunny
By default envsubst replaces all environment variables in the file regardless of whether they have actually been set. If they aren’t set, the placeholder will be removed from the output. This can make errors difficult to spot. A far more useful behaviour is to replace only those variables that have been set.
We can control the behaviour of envsubst by using the SHELL-FORMAT parameter. I found the name to be particularly confusing and still don’t understand its relationship to the shell, but here is an example of it in use.
cat example.yaml | envsubst '$MY_WEATHER'
This only replaces the $MY_WEATHER variable and leaves the $MY_NAME untouched.
name: $MY_NAME
weather: sunny
We can build on this to only replace environment variables that have been set. The following command passes in a list of existing environment variables that have been set. It does this by reformating the output of the printenv command.
cat example.yaml | envsubst "`printenv | sed 's/\(.*\)=.*/$\1/' | tr '\n' ' '`"
There is no chance I’ll remember that and I prefer this as the default behaviour. To do this, I alias envsubst to the full command above.
alias envsubst="envsubst \"`printenv | sed 's/\(.*\)=.*/$\1/' | tr '\n' ' '`\""