Configuring your Python application using environment variables

This blog post is a short explanation how to give and consume environment variables in your (Python) code. It’s a nice little trick, but if you are not well-versed in UNIX systems you might not know it.

Please see the original blog post for syntax color examples.

1. Configuring your application

An application can be configured usually in three ways. Through

  • command-line arguments (I prefer simplistic plac library with Python)
  • configuration files (I prefer YAML or Python standard library ConfigParser for INI style files)
  • environment variables

The two first ones are the most well-known methods. However, the last option is the easiest for quick and dirty hacks.

2. Using environment variables in UNIX

Note: Environment variables work alike in UNIX and Windows. However, I have no longer valid experience about Windows and how to set environment variables in the latest cmd.exe incarnations. (I used to write AUTOEXEC.bats for DOS, but that’s like long time ago). So if there are Windows gurus around please leave a comment.

Environment variables are

  • Easy way to make deployment specific changes in your application: run application differently on different computers by starting the app with a different command
  • Pass parameters for your application when you cannot parse command line yourself. E.g. your application is using Django or Zope launcher script and poking the launch script arguments is not easy.
  • Can be consumed everywhere inside your application, whether it is module level code (run on import), inside function, inside class and so on…
  • Needs only two lines to set up (import os ; os.environ.get())
  • Environment variables are especially useful if you want to have, in your scripts, some secret variables (username, password) which must not be committed on a public code repositories like Github (example). Note: the command lines are public for the all users of the UNIX system, so don’t do this on a shared server if you cannot expose the information to other server users.

Whether and how using environment variables is a good practice is debatable. But it definitely saves your butt when you need to make something quick and dirty.

You can give environment variables to your application on UNIX command line by simply prefixing your command with them (bash, sh, etc. shells):

DEMO_MODE=true python myapp.py

Or if you want to make the effect persistent for the current shell session use export (bash style)

export DEMO_MODE=true
python myapp.py
# Again
python myapp.py

Then you can read out for this environment variable easily using os.environ dictionary (pseudo code)

import os

# Just to check for the existing of DEMO_MODE environment variable,
# but you could also compare its value, pass it forward and so on
DEMO_MODE = os.environ.get("DEMO_MODE", None)

# We make some of the class members conditional 
# by given environment vaiables
class MyForm(object):

    name = StringField()

    if not DEMO_MODE:
       secret = PasswordField()

If you are using Python as the configuration language of your application (Django’s settings.py, Pyramid’s Configurator) you can also use this trick there to make some settings conditional which you normally hardcode in Python. Example settings.py:

import os

# Set Django debug mode based on environment variable
DEBUG = "DEBUG_MODE" in os.environ

Then, on the staging server, you would launch your Django application as

DEBUG_MODE=xxx python manage.py runserver

You can also give several environment variables once:

DEBUG_MODE=xxx API_SECRET=yyy python manage.py runserver

3. Another trick: socket.gethostname()

This is a way to bind certain settings to certain computers in your Python code by checking the name of the computer as returned by gethostname() call. This is useful if you don’t want to forget giving a specific launcher command on a specific server.

import socket

# Temporary hack to run hidden fields on a demo server
if socket.gethostname() in ['mikko-laptop', 'demoserver.example.com']:
     import special_config as config 
else:
     import normal_config as config

The only downside is that sometimes the hostname is not stable: I have noticed this behavior on OSX when connecting different networks on my laptop and apparently the name is given by the network.

 Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

2 thoughts on “Configuring your Python application using environment variables

  1. Just a quick note about Python 3 compatibility. In 3.x on Unix, environment variables are decoded using the file system encoding and the surrogateescape error handler. It also adds a bytes mapping named os.environb in case you need the raw values.

    Here’s an invalid UTF-8 string handled by surrogateescape: b’start \xf4\x90\x80\x80 stop’.decode(‘utf-8′, ‘surrogateescape’) == ‘start \udcf4\udc90\udc80\udc80 stop’. This round trips intact if encode() also uses surrogateescape.

    http://docs.python.org/3/library/os.html#os.environ
    http://docs.python.org/3/library/os.html#os.environb

  2. Pingback: Sharpen the Python Saw or Die Trying

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>