Take chef and ansible as examples to get started with server configuration

This article discusses how to install and configure software in our environment , This task is often called The server To configure （Server Provisioning）.

The server To configure

Before introducing modern tools , Let's take a look at the most basic and practical The server Configuration tool ：shell Script . stay Chef、Ansible or Puppet Before appearance , Many operation teams use Bash To configure the server （ stay Windows I use PowerShell Script ）.

for example , If you want to run Ubuntu Of Amazon EC2 Install... On the instance Nginx, You can use the following script （install-nginx.sh）：

#!/bin/sh
ssh -t [email protected]$1 sudo apt-get upgrade
ssh -t [email protected]$1 sudo apt-get -y install nginx

Copy

We can use shell Script to configure everything on the server . as far as I am concerned , All mainstream configuration tools use the secure transport layer （ Such as SSH） Of shell Order or PowerShell（Chef May be an exception ）. Even if you use the configuration tool , Scripts are also needed in some cases . therefore , When you start using the configuration tool （ Such as Chef or Ansible） when , Learn how to use basic shell Scripts can also bring you many benefits .

You may ask yourself , Why is it shell The script can do all the work, but also learn the configuration tool ？ Many environments have been used shell Script for server configuration , So why use configuration tools instead of them ？

First ,shell Scripts usually use declarative syntax .shell The script installs the software by running a sequence of commands , The configuration tool only needs to specify which software should be installed on the server , In this way, you can use the same code on different operating systems 、 Use different package managers and specify different versions to install and configure the same software .

secondly , Configuration tools often provide a way to organize infrastructure . Although the use of shell Scripts can also do this , But configuration tools usually provide a more concise solution . Because it's an industry standard , Developers can find out more easily QA Which servers in the environment are running RabbitMQ.

Third , Every major configuration tool has a thriving community , They build reusable modules to install most open source software . You can specify the memory limit directly in the module configuration , Without having to remember Postgres Where is the configuration file , This can save a lot of time .

Of course , There are many more reasons , Here are not all examples . Although the learning curve is a little steep , But learning to configure tools is still worthwhile . And shell Scripts are compared to , Configuration tools are easier to use , Easy to think , It's also easier to maintain .

About the name

Learn to use Chef（ Server configuration tool ） The last few weeks of left a deep impression on me . The getting started guide shows how to create a “recipe”, It contains instructions for installing or configuring the software , I can understand the meaning behind this metaphor .recipe Must exist in “cookbook” in , It makes sense . Then you are there. “kitchen” Li test cookbook, But I'm beginning to doubt .

This metaphor is a little confusing , So I decided to take a look at other tools , Such as Ansible.Ansible The first page of the document introduces “playbook” The concept of , and playbook Contains a series of “play”.

that , Are these questions important ？ Of course, it's very important , because Before learning the configuration tool , You should know , They are likely to introduce a lot of confusing terms . Even for basic tasks , You also have to relearn many terms . If you are just beginning to learn configuration tools , I strongly recommend that you write down the definitions of these terms at any time , You still have a lot to learn .

Every software developer will create different meanings for existing words , They even invented some words , such as “uninitialize” and “unregister”. This has become part of software development .

I will try to explain these tools in familiar terms .

Configuration Management

You decide to use fancy configuration tools to install on the remote server Nginx. Before setting up the database backup node , Everything went well . You have written MySQL Configuration file of the master server , But you're not sure how to configure MySQL From inside the server DNS Address . At this time, configuration management comes in handy .

When setting up the server , It's best to think of an application as consisting of two parts ： Immutable part （ Usually code or compiled binaries ） And variable parts （ Usually configuration files or environment variables ）. Most modules created by the community will install binaries by default , And provide as reasonable a configuration as possible , And it will expose some properties for us , It is convenient to cover it .

These attributes usually contain values specific to the user environment . Most configuration tools provide a mechanism for users , Insert environment specific values into the configuration file through the template , Or insert it directly into the environment variable .

You can use the configuration management provided by the configuration tool to configure MySQL Configuration file of the master server , Then configure the slave server in it .

Secret management

In this way, the above problems can be solved , But then it turned out , You must upload AWS Vouchers can make MySQL Access from the server S3. You know you can't submit these credentials directly to the code base , Therefore, these vouchers can only exist in your machine and NSA Server .

What you need at this time is Secret management .

Like everything in the field of Automation , You also have many options for managing secret keys . Google offers a project called KMS Service for ,AWS An item called Secret Manager Service for ,Chef Provides encrypted packets ,Hashicorp There is a model called Vault Products ,Ansible There is also a product called Vault Products . except KMS Will encrypt the string , All these tools provide the same functionality ： Protect access to encryption keys （ These keys are used in configuration management ）.

Several times , I accidentally submitted the secret key to the code base . Such things have been happening , And very dangerous .

Do not store in clear text API Key or credential .

have access to Secret Management solutions to store this data , Then bind it to the configuration tool .

A simple example ：Chef

Installation is required first Chef Development Kit（ChefDK）.

As mentioned earlier , We need a recipe To install Nginx. For the purpose of teaching , We will create it from scratch , Not from the community cookbook Take out one of them .

We need to create one cookbook.cookbook It usually exists in `cookbooks` Directory , Run the following command at the root of the project ：

mkdir cookbooks

Copy

Now let's create one cookbook, Used to place our new recipe：

chef generate cookbook cookbooks/application

Copy

This order is in `cookbooks/application` Many files are created in the directory , What we care about is `cookbooks/application/recipes/default.rb` This file . This file contains the default recipe, We will install Nginx Put the command of into this file .

apt_update

package 'nginx'

cookbook_file '/var/www/html/index.html' do
  source 'index.html'
  owner 'www-data'
  group 'www-data'
  mode '0755'
  action :create
end

Copy

The first two commands in this file will perform what you expect ：

• `apt_update` Update your aptitude package .
• `package ‘nginx’` Use the operating system default package manager to install `nginx` package （ In this example , It uses aptitude）.

The last command will `cookbooks/application/files/index.html` Copy to... On the remote server `/var/www/html/index.html`, And set the permissions of the file , Give Way Nginx The server can access it .

This file doesn't exist yet , So you need to create it . The first thing to do is to create ` file ` Catalog ：

mkdir cookbooks/application/files

Copy

Then create the file `cookbooks/application/files/index.html`, It includes the following ：

<html lang="en-us">
  <head>
    <title>Hello, World!</title>
  </head>
  <body>
    Chef has landed.
  </body>
</html>

Copy

to update `packer.json`, Join in Chef Related configuration ：

{
  "builders": [{
    "type": "amazon-ebs",
    "region": "us-east-1",
    "source_ami": "ami-04169656fea786776",
    "instance_type": "t2.small",
    "ssh_username": "ubuntu",
    "ami_name": "Ubuntu 16.04 Nginx - {{timestamp}}",
    "tags": {
      "Image": "application"
    }
  }],
  "provisioners": [{
    "type": "chef-solo",
    "cookbook_paths": ["cookbooks"],
    "run_list": ["recipe[application]"]
  }]
}

Copy

We're right `packer.json` Two changes have been made .

First , We are AMI Added a `Image` label . We started from Packer Copy in the output of AMI ID, And paste it into Terraform In the code . This is not a maintainable solution , because AMI ID It's going to change all the time , And we should not push the changes to the repository every time they happen . contrary , We use Terraform Of `data` Resources to dynamically read AMI ID（ Use `Image=application` Check the latest AMI）.

secondly , We use `chef-solo` To replace the `shell`. We tell it where to find cookbooks Catalog , And which one to run recipe. By default ,`run_list` Medium `recipe[COOKBOOK]` The entry will execute `recipes/default.rb`. We can also explicitly specify explicity:`recipe [COOKBOOK::RECIPE]` To override the default behavior . Because of our recipe Save in `recipes/default.rb` in , So the default behavior will be used .

Now start building our AMI：

packer build packer.json

Copy

Our new AMI There is one `Image` label , Now modify `terraform.tf` Hard coded AMI, Let it find by tag AMI.

Add the following to `terraform.tf` in ：

data "aws_ami" "web" {
  most_recent = true
  owners = ["self"]
  filter {                       
    name = "tag:Image"     
    values = ["application"]
  }                              
}

Copy

Now use `aws_ami.web resource` Output ID Replace `aws_instance.web1` and `aws_instance.web2 `resource Medium AMI ID：

resource "aws_instance" "web1" {
  ami                    = "${data.aws_ami.web.id}"
  availability_zone      = "us-east-1a"
  instance_type          = "t2.small"
  vpc_security_group_ids = ["${aws_security_group.application.id}"]
  subnet_id              = "${aws_subnet.private1.id}"
}

resource "aws_instance" "web2" {
  ami                    = "${data.aws_ami.web.id}"
  availability_zone      = "us-east-1b"
  instance_type          = "t2.small"
  vpc_security_group_ids = ["${aws_security_group.application.id}"]
  subnet_id              = "${aws_subnet.private2.id}"
}

Copy

Run the following command to create Chef To configure Server for , Then launch the browser , Open the domain name whose address is the load balancer ：

terraform plan -out terraform.plan
terraform apply "terraform.plan"
open "http://$(terraform output dns)"

Copy

You should be able to see ：Chef has landed!

A simple example ：Ansible

Let's use Ansible To build this same example . Installation is required first Ansible.

Ansible Organize installation and configuration instructions into `tasks` in , And then `tasks` Organize to `playbook` in . Let's make playbook Create a directory structure .

mkdir playbook
mkdir playbook/files

Copy

This is not an organization Ansible playbook Best practices . Because our use case is very simple , So a simplified version is used . If you are right about Ansible Interested in , It should be built according to the official advice playbook.

stay `playbook/application.yml` Created in playbook, The contents are as follows ：

---
- hosts: all
  gather_facts: False
  become: yes
  pre_tasks:
  - name: Install Python 2.7
    raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)
- hosts: applications
  become: yes
  tasks:
  - name: Install Nginx
    apt:
      name: nginx
      state: present
      update_cache: yes
  - name: Update contents of index.html
    copy:
      src: index.html
      dest: /var/www/html/index.html
      owner: www-data
      group: www-data
      mode: 0755

Copy

This playbook The file contains all the information needed to configure our server . Now let's discuss its structure .

Every playbook Contains a “play” list , Every play Contains a “tasks” list ,task For installing and configuring software . our playbook Contains two play. first play stay Ubuntu Installation on Python 2.7（ Used to run the Ansible）. the second play install and configure Nginx.

We are in each play The root node of is configured with two parameters ：`hosts` and `become`.`hosts` Parameters tell Ansible Which machine should it run on playbook（“all” Means running on all machines ）.`become:yes` Express Ansible Will pass through sudo Run all commands , Otherwise, many permission errors will occur .

play One of the first task Responsible for installation and configuration Nginx, It will be updated aptitude cache , And ensure `nginx` The package exists . If already installed `nginx` package , This command will do nothing .

the second task take `files/index.html` Copy to the remote server , And assign them the right permissions .

This file doesn't exist yet , So let's create it . Add the following to `playbook/files/index.html` in ：

<html lang="en-us">
  <head>
    <title>Hello, World!</title>
  </head>
  <body>
    Ansible has landed.
  </body>
</html>

Copy

This is our configuration Ansible Everything you need . Now let Packer Use this configuration . Update... With the following `packer.json`：

{
  "builders": [{
    "type": "amazon-ebs",
    "region": "us-east-1",
    "source_ami": "ami-04169656fea786776",
    "instance_type": "t2.small",
    "ssh_username": "ubuntu",
    "ami_name": "Ubuntu 16.04 Nginx - {{timestamp}}",
    "tags": {
      "Image": "application"
    }
  }],
  "provisioners": [{
    "type": "ansible",
    "playbook_file": "./playbook/application.yml",
    "host_alias": "applications"
  }]
}

Copy

We only modified the use Ansible As a configurator , Need to provide a point to playbook Path to file , We set it to `./playbook/application.yml`. We can see the installation Nginx Of play There is a line at the top ：`hosts: applications`. This is what we use to tell Ansible You need to install the host alias of the application . We need to tell Packer We are building an image for one of the hosts , So we're going to `host_alias` Property is set to `applications`.

Run the following command to create Ansible To configure Server for , Then launch the browser , Open the domain name whose address is the load balancer ：

packer build packer.json
terraform plan -out terraform.plan
terraform apply "terraform.plan"
open "http://$(terraform output dns)"

Copy

You should be able to see on the open browser page ：Ansible has landed!

The original English text ：http://stephenmann.io/post/a-brief-introduction-to-provisioning/