This website is based on the upcoming changes of the elabs Rails engine. Everything's not fine, but you can see the changes coming in the next releases.

Legal stuff about this site and your private life:

  • We use a cookie to keep your preferences (language, NSFW status, login status) during navigation.
  • We use your local storage to save the announces you closed (like this one).
  • We don't save informations you don't give
  • We don't share your email address if you comment or register
  • There is no tracker of any kind.

If you're not OK with this, move your way.

This is a compilations of notes that leads to the setup of a control repo managed with r10k and the use of Hiera.

This article follows:

This is a compilation of notes that leads to an installation of a Puppet server on an Ubuntu server 18.04.

These notes were heavily inspired by various articles and documentation pages, notably:

Why Ubuntu 18.04 and not some generic article? Because I tested on an Ubuntu 18.04 and don't know what would change for other distros (paths, packages, ...)

Creating a control repo managed with Git: r10k

Links:

The goal these notes targets is simple: we want to use a version control system to manage our Puppet manifest.

"r10k" is a powerful tool to manage your Puppet manifests. It bases its comportment on the branches of a git repository; each branch corresponding to an environment. This makes testing new things really easy.

Setup

Links:

Install r10k on Puppet server:

# Well, installing gems with sudo is discouraged, but here, we need a system wide install
sudo gem install r10k

The control-repo

Links:

Clone the control repo template anywhere on your host:

git clone https://github.com/puppetlabs/control-repo
cd control-repo

Configure r10k configuration in /etc/puppetlabs/r10k/r10k.yamlto specify the different paths used:

# The location to use for storing cached Git repos
:cachedir: '/var/puppet/r10k/cache'

# A list of git repositories to create
:sources:
  # This will clone the git repository and instantiate an environment per
  # branch in /usr/local/etc/puppet/environments
  :code:
    remote: '/path/to/the/control-repo/.git'
    basedir: '/etc/puppetlabs/code/environments'

Copy your current manifest:

cp /etc/puppetlabs/code/environments/production/manifests/site.pp manifests/site.pp

Add the modules you previously used and the modules they need in the Puppetfile, present in the repository:

mod 'zleslie-ssh', '1.2.0'

# These modules should be included too, as zleslie-ssh needs them.
mod 'puppetlabs-stdlib', '6.0.0'
mod 'puppetlabs-concat', '6.0.0'

My first thought was "Erk! we have to manually specify the modules dependencies", and then I was like "Ah, ok, maybe that way I won't use obscure things to manage my infrastructure".

Note for a fresh repo:

Maybe you don't want to keep the template's history, so you can remove the .git directory and re-create a new repo.

rm -rf .git
git init

# There's no "master" environment in our control repo
git checkout -b production

In any case, commit your work.

Deploy

To deploy the changes in the control repo, use r10k from the control repo:

sudo r10k deploy production --verbose --puppetfile

The changes will be applied in the next 30 mins.

Automate "r10k deploy"

In order to automate the deployment of the changes done in the branches, we will configure a post-commit hook:

#!/usr/bin/env bash
branch=$(git rev-parse --abbrev-ref HEAD)

echo "Deploying environments ${branch} with r10k…"
sudo r10k deploy environment ${branch} --verbose --puppetfile

echo "Caching types for environments ${branch}…"
sudo /opt/puppetlabs/bin/puppet generate types --environment ${branch}

Note: If you use an online VCS for this repo, you may want to create a post-merge hook to, so it deploys on pull.

Each time a commit is done in a branch, the corresponding Puppet environment will be updated.

Each time a branch is pulled with changes, the corresponding Puppet environment will be updated.

You can make a change in a manifest and commit, to see if everything is still fine.

The role/profile organisation

Links:

When we're speaking of file organisation, everyone has it's idea on what is good, and what is not. I propose to follow the Puppet's recommendations: the Role/Profile model:

  • Profiles are manifests for a given resource (packages, ssh configuration, etc...).
  • Roles are composed of different profiles.
  • A node has one role only.

If we look at the repo template we cloned earlier, we can see this structure:

control-repo
|-- LICENSE
|-- Puppetfile        <-- Where you put modules
|-- README.md
|-- data              <-- Hiera stuff, for later
|   `-- [...]
|-- environment.conf
|-- hiera.yaml
|-- manifests
|   `-- site.pp       <-- All your manifest, for now
|-- scripts
|   `-- [...]
`-- site-modules      <-- Where you should split your site.pp
    |-- profile
    |   `-- manifests
    |       |-- base.pp
    |       `-- example.pp
    `-- role
        `-- manifests
            |-- database_server.pp
            |-- example.pp
            `-- webserver.pp

You should edit a few files from the role and profile manifests to understand what's going on.

Split site.pp

A profile manifest looks like:

We're going to split site.pp to have something more roleprofilesque:

  • Remove all files in profile/manifests
  • Create a base_packages.pp file with all the packages common to all your nodes:
    # Notice the class name, it should match the file name.
    class profile::base_packages {
      package { 'cloud-init':
        ensure => purged
      }
      # ...
    }
    
  • Create a motd.pp file with the MOTD configuration
  • Create a openssh.pp file with the openssh configuration
  • Create some other <something>.pp files for the rest of the common stuff, leaving only the node-related entries in the site.pp

Now, let's create a base profile. Puppet classes can inherit from other classes. It's discouraged to do so, except for the roles, so we're going to create a base_node role, and a puppet_server role, extending the base role.

  • Remove all files in roles/manifests
  • Create a base_node.pp, which will include all the common stuff:
    class role::base_node {
      include profile::base_packages
      include profile::motd
      include profile::openssh
      #...
    }
    
  • Create a puppet_server.pp, which will extend the base node and add the server-specific stuff:
    class role::puppet_server inherits role::base_node {
      # include profile::<server-specific profiles, if any>
    }
    
  • Move the code related to the server from site.pp to the role.
  • Repeat this for the "puppet-node" host, with a puppet_node role and the puppet-node's specific configuration.

Let's update site.pp to use the different roles:

# Puppet server
node puppet {
  include role::puppet_server
}

# Some testing node
node puppet-node {
  include role::puppet_node
}

Cleaner, huh ?

Scratching the Hiera's surface

Links:

Hiera is a powerful tool that allows to define some values, per-node.

Because maybe we wanted to use different SSH keys or different usernames for the node and server, we should use Hiera to do so.

Hiera data is stored in data, in the control-repo

control-repo/data/
|-- common.yaml            <-- Values for each nodes
`-- nodes
    `-- example-node.yaml  <-- Values for a specific node

As this is only an intro, we're not doing anything fancy here, but we'll specify an user and a public key for all the nodes. This will remove the hardcoded username and ssh key from our manifests.

First, we'll have to change the user profile to accept variables:

class profile::user (
  String[1] $username,
  String[1] $ssh_key,
) {
  user { $username:
    ensure         => present,
    groups         => [
      'adm',
      'sudo',
    ],
    shell          => '/bin/bash',
    home           => "/home/${username}",
    managehome     => true,
    password       => '*',
    purge_ssh_keys => true,
  }

  $ssh_elements = $ssh_key.split(' ')

  ssh_authorized_key { $ssh_elements[2]:
    ensure => present,
    user   => $username,
    type   => $ssh_elements[0],
    key    => $ssh_elements[1],
  }
}

And then, we tell Hiera which values to assign to the variables:

# data/common.yaml
---
profile::user::username: johndoe
profile::user::ssh_key: ssh-rsa SOME_LONG_KEY john_doe@somewhere

Commit and apply.

Conclusion

We now have a versioned control repo with a role/profile organisation. The deployment of the control repo is almost automated via git hooks and r10k.

What's next?

  • Create linters and checkers to test the manifests before they are deployed.
  • Have a Git server on the puppet server to deploy on push.
  • Write your own modules.

Leave a comment

You want to react to this content or ask something to the author? Just leave a comment here!

Note that the comments are not publicly visible, so don't worry if you don't see yours.

All the information you give will only be visible to the author. We don't share anything with anyone.

Don't fill this field if you want this form to be saved.