Archive for April, 2015

Puppet feature: external environment locator (EEL)

Thursday, April 16th, 2015

In this post I will briefly glance over historic and current Puppet environment configuration options, and introduce a new feature called External Environment Locator, which facilitates usage of external program to map environment name to a corresponding filesystem path.

Legacy environments configuration options

Config-file environments – LEGACY

In the early days of puppet, if you wanted to use multiple environments, you had to set up each an every one of them in you puppet.conf. Each environment was configured as it’s own section with corresponding manifest and module paths.

Configuration example for legacy config-file environments:

Link to documentation: https://docs.puppetlabs.com/puppet/latest/reference/environments_classic.html

Dynamic environments – LEGACY

This static configuration was quickly recognized as cumbersome when more than a few environments were required. Therefore a feature appeared which dynamically constructed path to environment based on it’s name.

Configuration example for legacy dynamic environments:

The problem with this approach was that there was “no good way to set config_version”. Better, more in-depth explanation of this anomaly is available .

Directory environments

Puppet 3.5 introduced a new feature called “directory environments”. It obsoleted all previously-implemented environment configurations, in much nicer and consistent manner. It goes like this: environment name == directory name.

Sample configuration of directory environments:

External environment locator (EEL)

Directory environments have their use, but what if you would want to implement more policy regarding environment locations? There are a couple of use cases where this feature might come in handy:
– you need to have your environments organized in multiple directories (I do), or
– you need to check if environment name prefix matches the directory name that contains it, or
– you would like to do a git clone/pull dynamically, or
– git pull automatically for certain environments (i.e. production), but not for others (development), or
– anything else I currently can not foresee.

Imagine you could do all this with a simple puppet master configuration, like this:

What the configuration above would do is:
1. whenever environment is requested (and is not cached), it would call external program to retrieve its location, and
2. it would cache this location for specified period of time (this is actually a puppet feature).
External program can do whatever it wishes to either provide requested environment (i.e. do a git clone before returning data), or decide that requested environment does not exist (i.e. after checking if git branch exist and seeing it does not) and return found=false.

External environment locator input and output specification

EEL input

The program that is used as EEL is passed a single argument: name of the environment to locate. For example, if we are searching for environment “my_prod”, the following command would be run:

EEL exit status

EEL should return zero exit status regardless whether environment is found or not. Non-zero exit status is reserved for error conditions that are not predicted and therefore result (whether environment exists or not) can not be trusted. See EEL output below for additional details.

EEL output

If environment exist, EEL should return the following YAML output:

If environment is not found, then the following YAML output should be presented:

Where to get the code?

Puppet patch

The code is currently implemented as a patch for puppet. It is available for the following versions:
– for v3.7.5: puppet-3.7.5-externalEnvironments-0.1.diff (works with puppet 3.7.4 too)
– for v4.0.0: puppet-4.0.0-externalEnvironments-0.1.diff

Patch development can be followed (and contributed) here:
3.x branch
master/4.x branch
– 3.x diff: see 3.x diff on github
– master/4.x diff: see master/4.x diff on github

Sample EEL program

Here is a simple (PoC) environment locator which works identically to directory environments:

Conclusion

Suggestions welcome. Especially in the form of patch improvement, configuration variables naming porposal, etc. Also, if you can think of a use case for you which is not yet mentioned here, write it into a comment below.

Have fun!