Setting up puppet master and agent with simple scripts on EC2

Puppet

In this tutorial, we'll setup puppet master and agent with two scripts on EC2 with Ubuntu 14.

First, we need to have two instances of EC2: hostnames are puppet and agent.

Then, we'll run two scripts: one for master and the other for agent.

Also, in later section of this article, we'll see how we can install puppet from our desktop (via ssh) without logging into the instances.

Script for master, master.sh:

#!/bin/bash
if [ $# -eq 0 ]
  then
    echo "No hostname supplied"
    exit
fi
env=$1
echo $env
hostname=`hostname`
apt-get update -y
apt-get upgrade -y
wget http://apt.puppetlabs.com/puppetlabs-release-trusty.deb
dpkg -i puppetlabs-release-trusty.deb
apt-get update -y
apt-get -y install puppetmaster
sed -i "s/$hostname/$env/g" /etc/hostname
sed -i "s/no/yes/g"  /etc/default/puppetmaster
echo "127.0.0.1  puppet.example.net    puppet" >>  /etc/hosts
echo "172.31.44.238  puppet.example.net    puppet" >>  /etc/hosts
reboot

Run the script on master. The script takes one arg which is the hostname:

ubuntu@ip-172-31-44-238:~$ chmod u+x master.sh
ubuntu@ip-172-31-44-238:~$ sudo ./master.sh puppet

Script for master, agent.sh:

#!/bin/bash
if [ $# -eq 0 ]
  then
    echo "No hostname supplied"
    exit
fi
env=$1
echo $env
hostname=`hostname`
apt-get update -y
apt-get upgrade -y
wget http://apt.puppetlabs.com/puppetlabs-trusty.deb
dpkg -i puppetlabs-release-trusty.deb
apt-get update -y
apt-get -y install puppet
sed -i "s/$hostname/$env/g" /etc/hostname
sed -i "s/no/yes/g"  /etc/default/puppet
echo "172.31.44.238  puppet.example.net    puppet" >>  /etc/hosts
echo "172.31.44.238  puppet" >>  /etc/hosts
puppet agent --waitforcert 60
reboot

Run the script on master:

ubuntu@ip-172-31-44-239:~$ chmod u+x agent.sh
ubuntu@ip-172-31-44-239:~$ sudo ./agent.sh agent

On master side:

ubuntu@puppet:~$ sudo puppet cert list
  "agent.us-west-1.compute.internal"            (SHA256) 98:1E:B6:03:AA:7A:D0:...
  "ip-172-31-44-239.us-west-1.compute.internal" (SHA256) 17:CC:98:92:EE:6D:CD:...

ubuntu@puppet:~$ sudo puppet cert sign agent.us-west-1.compute.internal
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1139:in `issue_deprecation_warning')
Notice: Signed certificate request for agent.us-west-1.compute.internal
Notice: Removing file Puppet::SSL::CertificateRequest agent.us-west-1.compute.internal at '/var/lib/puppet/ssl/ca/requests/agent.us-west-1.compute.internal.pem'

Puppet always starts compiling with either a single manifest file or a directory of manifests that get treated like a single file. This main starting point is called the main manifest or site manifest.

The puppet agent periodically checks in with the puppet master (typically every 30 minutes). During this time, it will send facts about itself to the master, and pull a current catalog--a compiled list of resources and their desired states that are relevant to the agent, determined by the main manifest. The agent node will then attempt to make the appropriate changes to achieve its desired state. This cycle will continue as long as the Puppet master is running and communicating with the agent nodes.

On the master:

ubuntu@agent:~$ sudo vi /etc/puppet/manifests/site.pp

Now add the following lines to describe a file resource:

file {'/tmp/example-ip':                                     # resource type file and filename
  ensure  => present,                                        # make sure it exists 
  mode    => 0644,                                           # file permissions
  content => "Here is my IP Address: ${ipaddress_eth0}.\n",  # note the ipaddress_eth0 fact
}

This will make ensure that all agent nodes will have a file at /tmp/example-ip, with -rw-r--r-- permissions, and text that contains the node's public IP address.

The default runinterval is 1800. Here, we want to reduce it to 300 which is 5 minutes:

[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
prerun_command=/etc/puppet/etckeeper-commit-pre
postrun_command=/etc/puppet/etckeeper-commit-post
runinterval=300

[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

To check if it is set:

ubuntu@agent:~$ sudo puppet agent --configprint runinterval
300

We can either wait until the agent checks in with the master automatically, or we can run the following on the agent:

ubuntu@agent:~$ cat /tmp/example-ip
cat: /tmp/example-ip: No such file or directory

ubuntu@agent:~$ sudo puppet agent -t
Notice: Skipping run of Puppet configuration client; administratively disabled (Reason: 'Disabled by default on new installations');
Use 'puppet agent --enable' to re-enable.

ubuntu@agent:~$ sudo puppet agent --enable
ubuntu@agent:~$ cat /tmp/example-ip
cat: /tmp/example-ip: No such file or directory

ubuntu@agent:~$ sudo puppet agent -t
Info: Caching certificate_revocation_list for ca
Info: Retrieving plugin
Info: Caching catalog for agent.us-west-1.compute.internal
Info: Applying configuration version '1424403628'
Notice: /Stage[main]/Main/File[/tmp/example-ip]/ensure: created
Info: Creating state file /var/lib/puppet/state/state.yaml
Notice: Finished catalog run in 0.02 seconds

Then run the 'cat' command again to print the file:

ubuntu@agent:~$ cat /tmp/example-ip
Here is my IP Address: 172.31.44.239.

We're now able to install puppet master/agent using a shell script. However, wouldn't it be more convenient if we can do it from our desktop instead of going into the instances?

Here is the shell script (myscp.sh) we want to run on our desktop:

#!/bin/bash
# 
# 1. deploy local setup-puppet.sh to EC2 instance
# 2. run the setup-puppet.sh with alias and EC2_hostname (the alias is defined in .ssh/config)
# (ex) ./myscp.sh simple ec2_simple
target_server=$1
host_name=$2
scp setup-puppet.sh $1:/home/ubuntu
ssh -t $target_server "
sudo chmod u+x setup-puppet.sh;
sudo ./setup-puppet.sh $2;
"

It does the following:

1. It does scp to put our script for setting up puppet (setup-puppet.sh) on an instance.
2. It uses "ssh -t" for remote run.
3. It sets the permission using chmod.
4. Finally, it runs the puppet setup script.

Note that we uses ./ssh/conf file to get the target_server, and the conf file looks like this:

Host myagent
  Hostname      54.175.50.125
  User          ubuntu
  IdentityFile  ~/.ssh/bogo.pem

Actually, if we want to simply login to the instance, we can just use:

$ ssh myagent

Also, though we've already seen the puppet setup script, it looks like this:

# setup-puppet.sh
#!/bin/bash
if [ $# -eq 0 ]
  then
    echo "No hostname supplied"
    exit
fi
env=$1
echo $env
hostname=`hostname`
apt-get update -y
apt-get upgrade -y
wget http://apt.puppetlabs.com/puppetlabs-trusty.deb
dpkg -i puppetlabs-release-trusty.deb
apt-get update -y
apt-get -y install puppet
sed -i "s/$hostname/$env/g" /etc/hostname
sed -i "s/no/yes/g"  /etc/default/puppet
echo "172.31.44.238  puppet.example.net    puppet" >>  /etc/hosts
echo "172.31.44.238  puppet" >>  /etc/hosts
puppet agent --waitforcert 60
reboot

Puppet