Building Docker images and configuring your dockerized apps doesn’t have to bea try-fail-repeat Google extravaganza.This article will help you work with Docker ARG, ENV, env_file and .env files with confidence.
The only prerequisite: make sure that you’re comfortable with the basics of Docker.
Read on and you will understand how to configure your Docker images and dockerized apps with ease - with the power of Docker build-time variables, environment variables and docker-compose templating.
Select Advanced system settings. In the Advanced tab, select Environment Variables. You will now be able to enter the environmental variable. Most environmental variables are given to you in the form 'Variablename=Variablevalue'. For details on creating an environment from this environment.yml file, see Creating an environment from an environment.yml file. Restoring an environment ¶ Conda keeps a history of all the changes made to your environment, so you can easily 'roll back' to a previous version.
Frequent Misconceptions
Create .env File Linux
This is a long, in-depth read. Let’s start with something you can use right now, without having to read the whole thing!
Here’s a list of easy takeaways:
- The .env file, is only used during a pre-processing step when working with docker-compose.yml files. Dollar-notation variables like $HI are substituted for values contained in an “.env” named file in the same directory.
- ARG is only available during the build of a Docker image (RUN etc), not after the image is created and containers are started from it (ENTRYPOINT, CMD). You can use ARG values to set ENV values to work around that.
- ENV values are available to containers, but also RUN-style commands during the Docker build starting with the line where they are introduced.
- If you set an environment variable in an intermediate container using bash (RUN export VARI=5 && …) it will not persist in the next command. There’s a way to work around that.
- An env_file, is a convenient way to pass many environment variables to a single command in one batch. This should not be confused with a .env file.
- Setting ARG and ENV values leaves traces in the Docker image. Don’t use them for secrets which are not meant to stick around (well, you kinda can with multi-stage builds).
An Overview
Alright, let’s get started with the details.The guide is split up into the following topics:
Feel free to jump right to the one you need right now. However, you’ll getthe best result if you read through them all thoroughly.
The Dot-Env File (.env)
This one is quite simple, and only confusing because of bad examples andsimilar concepts using the same format, sounding a lot like it. What’s importantis the dot in front of env - .env, not an “env_file”.
If you have a file named .env in your project, it’s only used to put valuesinto the docker-compose.yml file which is in the same folder. Those are usedwith Docker Compose and Docker Stack. It has nothing to do with ENV, ARG, or anything Docker-specific explained above. It’s exclusively a docker-compose.yml thing.
The values in the .env file are written in the following notation:
Those key-value pairs, are used to substitute dollar-notation variables inthe docker-compose.yml file. It’s kind of a pre-processing step, and theresulting temporary file is used. This is a nice way to avoid hard-codingvalues. You can also use this to set the values for environment variables,by substituting the string, but that does not happen automatically.
Here is an example docker-compose.yml file, relying on values provided from a.env file:
Hint:When working with an .env file, you can debug your docker-compose.yml files quite easily.Just type docker-compose config. This way you’ll see how the docker-compose.yml file content looks after the substitution step has been performed without running anything else.
Here’s a gotcha you should know: environment variables on your host can override the values in your .env file. Read more here.
ARG and ENV Availability
When using Docker, we distinguish between two different types of variables -ARG and ENV.
Env File Converter
ARG are also known as build-time variables. They are only availablefrom the moment they are ‘announced’ in the Dockerfile with an ARG instructionup to the moment when the image is built. Running containers can’t accessvalues of ARG variables. This also applies to CMD and ENTRYPOINT instructionswhich just tell what the container should run by default. If you tell aDockerfile to expect various ARG variables (without a default value) but noneare provided when running the build command, there will be an error message.
However, ARG values can be easily inspected after an image is built, by viewing the docker history of an image. Thus they are a poor choice for sensitivedata.
ENV variables are also available during the build, as soon as you introducethem with an ENV instruction. However, unlike ARG, they are also accessibleby containers started from the final image. ENV values can be overridden whenstarting a container, more on that below.
Here is a simplified overview of ARG and ENV availabilities around theprocess around building a Docker image from a Dockerfile, and running a container. They overlap, but ARG is not usable from inside the containers.
Setting ARG Values
So, you have your Dockerfile, which defines ARG and ENV values. How to setthem, and where? You can leave them blank in the Dockerfile, or set defaultvalues. If you don’t provide a value to expected ARG variables which don’thave a default, you’ll get an error message.
Here is a Dockerfile example, both for default values and without them:
When building a Docker image from the commandline, you can set ARG values using –build-arg:
How To Create An Env File Python
Running that command, with the above Dockerfile, will result in the following line being printed (among others):
So, how does this translate to using docker-compose.yml files?When using docker-compose, you can specify values to pass on for ARG, in an args block:
(docker-compose.yml file)
When you try to set a variable which is not ARG mentioned in the Dockerfile, Docker will complain.
Setting ENV Values
So, how to set ENV values? You can do it when starting your containers (and we’ll look at this a bit below),but you can also provide default ENV values directly in your Dockerfile by hard-coding them. Also,you can set dynamic default values for environment variables!
When building an image, the only thing you can provide are ARG values, as described above. Youcan’t provide values for ENV variables directly. However, both ARG and ENVcan work together. You can use ARG to setthe default values of ENV vars. Here is a basic Dockerfile, using hard-coded default values:
And here is a snippet for a Dockerfile, using dynamic on-build env values:
Once the image is built, you can launch containers and provide values for ENV variables in threedifferent ways, either from the command line or using a docker-compose.yml file. All of those willoverride any default ENV values in the Dockerfile.Unlike ARG, you can pass all kinds of environment variables to the container. Even ones not explicitlydefined in the Dockerfile. It depends on your application whether that’ll do anything however.
1. Provide values one by one
From the commandline, use the -e flag:
From a docker-compose.yml file:
2. Pass environment variable values from your host
It’s the same as the above method. The only difference is, you don’t provide a value, but just namethe variable. This will make Docker access the current value in the host environment and pass it onto the container.
For the docker-compose.yml file, leave out the equation sign and everything after it for the same effect.
3. Take values from a file (env_file)
Instead of writing the variables out or hard-coding them (not in good taste according to the 12-factor folks),we can specify a file to read values from. The contents of such a file look something like this:
The file above is called env_file_name (name arbitrary) and it’s located in the current directory.You can reference the filename, which is parsed to extract the environment variables to set:
With docker-compose.yml files, we just reference a env_file, and Docker parses it for the variables to set.
Here is a small cheat sheet, combining an overview of ARG and ENV availability with common waysto set them from the command line.
Before we move on: a frequent gotcha, if you’re new to Docker and not used to think about images and containers:if you try to set the value of an environment variable from inside a RUN statementlike RUN export VARI=5 && ..., you won’thave access to it in any of the next RUN statements. The reason for this, is that for each RUNstatement, a new container is launched from an intermediate image. An image is saved by the end ofthe command, but environment variables do not persist that way.
If you’re curious about an image, and would like to know if it provides default ENV variable values beforethe container is started, you can inspect images, and see which ENV entries are set by default:
Phew, that was quite a bit. The only thing left is - if you have so many different ways of settingthe values of ENV variables, which ones override others?
Overriding ENV Values
Assuming, you have an image built from a Dockerfile, which provides default ENV values.Containers started from it, have access to ENV variables defined in the Dockerfile.However, those values can be overridden by providing single environment variables, or env_files, from which environment variables are parsed and passed into the container.
Create Env File Nodejs
Once a process runs inside the container, or when a command is evaluated, they can change the environment values for themselves. Stuff like:
will completely override any SOME_VAR you might have set otherwise for the app.py script, even if there were some value with a -e flag before the final command.
The precedence is, from stronger to less-strong: stuff the containerized application sets, values from single environment entries, values from the env_file(s) and finally Dockerfile defaults.
In Conclusion
This was a really thorough look at all the ways you can set ARG and ENV variables when building Docker images and starting containers. By now, you should have a really good overview build-time arguments, environment variables, env_files and docker-compose templating with .env files. I hope you got a lot of value out of it, and canuse the knowledge to save yourself lots of bugs in the future.
NOTE: If you want to learn more about building good Docker images, make sure to check out my free email course: 5 Days to Better Docker Images.
To really master these concepts, just reading about them is not enough. You have to see them in actionand apply them to your own work to truly make them part of your tool belt. The best way to make sure youwill be able to make use of this information, is to learn by doing – go ahead and try some of thosetechniques in your current projects!