Setting Up A Cross-Platform SaltStack Development Environment on AWS

This walkthrough is to help you get a dev environment up and running with SaltStack on AWS. If you are interested in a quick SaltStack with minimal testing, you can use a SaltStack Docker that I built.

https://git.corp.adobe.com/dcurtis/SaltStack-Docker-CentOS

We will configure the following AWS servers:

• One Master on Ubuntu 18.04
• One Minion running locally on the Master
• One Minion running independently from master on AWS Linux

In order to get started, you will need an AWS account and a basic understanding of EC2 servers.

Login to AWS and surf over to EC2 management.

Click "Launch Instance" and pick the cheapest server with minimal resources. Resources can be increased later, if needed.

For my master I am going to use Ubunutu 18.04

"Next" through all configuration screens, using the default specs provided by Amazon.

After the Server has finished provisioning, we can poke holes in the firewall that Salt will need for communication.

By highlighting our new server in the EC2 dash, we can view the current firewall settings by clicking the Security Group that was automatically applied to the server.

Mine is called 'launch-wizard-8'. After clicking the Security Group, we are redirected to the EC2 -> Security Groups section. Click on the appropriate Security Group, finally arriving at our settings.

Click Edit rules to begin adding entries.

Since minions connect to masters, the only firewall configuration that must be done is on the master. By default, ports 4505 and 4506 must be able to accept incoming connections on the master. 

After adding the ports, we are ready to connect to our EC2 instance. Click Instances in the left column to be taken back to our servers.

Once you are back on the EC2 Dashboard viewing the instances, you can right click on our new Salt master and connect. If you haven't already, you will need to setup an SSH key in order to ssh in to the box. You cannot connect directly via Public IP, so I always use this 'connect' option to get the connection info.

After setting up your SSH key, connect via terminal, sudo up and update the box.

Once the server has been updated, it's time to install the SaltStack Master and local Minion.

The dependencies for running Salt at the time of writing are as follows:

• Python 2 – Version 2.6 or greater (Salt is not Python 3-compatible)
• Msgpack – python
• YAML
• Jinja2
• MarkupSafe
• ZeroMQ – Version 3.2.0 or greater
• PyZMQ – Version 2.2.0 or greater
• Tornado
• PyCrypto
• M2Crypto

The easiest way to ensure that the dependencies for Salt are met is to use the official Salt Bootstrap script to handle all of the system-specific commands for you. Salt Bootstrap is an open source project with the goal of creating a Bourne shell-compatible script that will install Salt on any compatible server. The project is managed and hosted by the SaltStack team. You can find more information at https://github.com/saltstack/salt-bootstrap.

From your terminal, run the following commands to grab the bootstrap script and run it

# curl -L https://bootstrap.saltstack.com -o install_salt.sh
# sudo sh install_salt.sh -M -P

By default, Salt Bootstrap will only install the Salt minion. We want both the Salt minion and the Salt master, which is accomplished above by passing in the -M flag. We also want to pass in the -P flag to allow bootstrap to install Tornado using pip.

After the bootstrap is complete, you can check the Salt Master installation using the versions report command:

# sudo salt --versions-report

Salt Version:
           Salt: 3000.3


Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.8
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.17 (default, Apr 15 2020, 17:20:14)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5


System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-1065-aws
         system: Linux

        version: Ubuntu 18.04 bionic

After the SaltMaster is confirmed installed, we can update our server to download needed dependencies.

# apt update

We should also confirm that the ports we opened in AWS EC2 Dashboard are open

# netstat -aon | grep 450
tcp        0      0 0.0.0.0:4505            0.0.0.0:*               LISTEN      off (0.00/0/0)
tcp        0      0 0.0.0.0:4506            0.0.0.0:*               LISTEN      off (0.00/0/0)
tcp        0      0 127.0.0.1:40730         127.0.0.1:4505          ESTABLISHED keepalive (105.38/0/0)
tcp        0      0 127.0.0.1:4505          127.0.0.1:40730        
 ESTABLISHED keepalive (105.38/0/0)

tcp        0      0 172.31.34.57:4505       3.22.101.249:39874      ESTABLISHED keepalive (105.38/0/0)

As you can see from the output, ports 4505-4506 are open.

Minion Configuration

Next, we will need to point our Minion at our Master by editing the minion configuration file configuration files are located in the /etc/salt/minion

You should also manually configure the minion ID so that you can more easily communicate with the server.

master: localhost
id: myminion

Save and exit the text editor.

Now we need to start (or restart) our Salt master and Salt minion:

# sudo service salt-minion restart
# sudo service salt-master restart

Next we will have to add the Salt Minion key to the trusted keys on the master.

# sudo salt-key
Accepted Keys:
Denied Keys:
Unaccepted Keys:
myminion

Rejected Keys:

Note that our minion "myminion" is listed in the Unaccepted Keys section. This means that the minion has contacted the master and the master has cached that minion's public key, and is waiting for further instructions as to whether to accept the minion or not.

We can inspect the key's fingerprint to ensure that it matches our minion's key:

# sudo salt-key -f myminion
Unaccepted Keys:
myminion:  a8:1f:b0:c2:ab:9d:27:13:60:c9:81:b1:11:a3:68:e1

We can use the salt-call command to run a command on the minion to obtain the minion's key:

# sudo salt-call --local key.finger
local:    a8:1f:b0:c2:ab:9d:27:13:60:c9:81:b1:11:a3:68:e1

Since the fingerprints match, we can accept the key on the master:

# sudo salt-key -a myminion
The following keys are going to be accepted:
Unaccepted Keys:
myminion
Proceed? [n/Y] Y
Key for minion myminion accepted.

We can also check that the minion key was accepted:

# sudo salt-key
Accepted Keys:
myminion
Denied Keys:
Unaccepted Keys:
Rejected Keys:

At this point, we should have a working Salt Master with a local Salt Minion. Let's test the communication.

$ salt '*' test.ping

myminion:

    True

You should get a reply from "myminion" as seen above.

Next, let's ensure we have the salt directory needed to run salt states and save modules.

$ mkdir /srv/salt/

Adding a New Minion

Let's add a secondary minion on it's own box. This time we will use AWS Linux for a cross platform SaltStack.

Launch a new instance, using the following AMI

After you have completed launching the instance, don't forget to add ports 4505-4506 in the Security Groups dash, as noted above.

SSH into your new AWS Linux instance and sudo up. After you have root, you can run the following:

$ curl -L https://bootstrap.saltstack.com -o install_salt.sh
$ sudo sh install_salt.sh

After installing the same version running on your master, you can configure your minion to connect to your master's IP with the name "awsminion". You can add the following to the top of your /etc/salt/minion file on your new minion:

master: 69.164.192.51
id: centminion

The Public IP for your Salt Master can be grabbed from the EC2 dashboard

Next restart your minion:

# sudo service salt-minion restart

Now, we can return to our Salt Master to accept the new minion's key so that we can communicate with it:

$ salt-key
Accepted Keys:
myminion
Unaccepted Keys:
awsminion
Rejected Keys:

$ sudo salt-key -a awsminion
The following keys are going to be accepted:
Unaccepted Keys:
awsminion
Proceed? [n/Y] y
Key for minion awsminion accepted.

We know that we're successful if we can ping the new minion, as follows:

# sudo salt '*' test.ping
awsminion:
    True
myminion:
    True

We can double-check that our minion is running a RedHat distribution instead of a Debian distribution by checking the os_family grain, as follows:

# sudo salt '*' grains.item os_family
myminion:
    ----------
    os_family:
        Debian
awsminion:
    ----------
    os_family:
        RedHat

Happy Testing!