Learning Chef

5 minute read

I’m currently in the process of learning Chef – the infrastructure automation framework by Opscode – and wanted to share some of the things I’ve picked up along the way. To give you a little bit of background, we’ve chosen Chef at work to gradually replace our deployment and configuration system, which is, unfortunately, entirely based on custom Debian packages (and plenty of magic). There’s still lots of work ahead of us and I’m far away from being a Chef expert, but I do feel comfortable enough to give you some tips on how to get started with Chef.

Learn Ruby (First)

The first thing people will usually tell you about Chef is that it allows you to write “cookbooks” with “recipes” defining how to configure a server to do X, with the standard example being an “Apache” cookbook which sets up the Apache web server.

Then you’ll probably hear that the whole system is based on Ruby. While Opscode claims that you don’t need to have experience with Ruby to write recipes (thanks to the straightforward recipe DSL), I still decided to learn Ruby first. In fact, I had wanted to learn this wonderful language for a while and Chef gave me another good reason. In hindsight, that was the right thing to do, in particular in regard to more advanced topics like automated cookbook testing.

Sooner or later, you need to dive into Ruby.

Vagrant and Chef Solo

Fast forward and several Ruby experiments later, I came back. By then, I had not only become a passable Ruby programmer, I also knew the best way to get started with Chef: Vagrant. (If you’ve been following this blog, you already know I have a thing for it.) Vagrant comes with provisioners for two flavors of Chef: Chef Solo and Chef Server.

The Chef Server can be used to distribute cookbooks, manage and authenticate nodes, and to query infrastructure information. It is, however, completely optional and I will blissfully ignore it in this post. (The same goes for its counterpart: Chef Client)
With Chef Solo, you can run cookbooks in the absence of a Chef Server. All you need are the cookbook files somewhere on your disk. Sounds simple enough? As you will see in a minute, it really is.

The Guts of a Cookbook

Before continuing with Vagrant and Chef Solo, I want to tell you a bit more about Cookbooks.

Cookbooks are the fundamental units of distribution in Chef. They encapsulate all the resources you need to automate your infrastructure and are easily sharable with other Chef users. -- Opscode

To put it simple, a cookbook is a bunch of files and directories. For someone new to Chef, the most important components to know are:

Attributes define default values to be used by the cookbook, e.g. installation path or service port. Those values may be overridden by role-specific or node-specific values.
Recipes specify (mostly) platform-independent Resources which are executed in the order defined to configure the node. For example, the package resource installs software packages, remote_file downloads a file, and bash executes shell code.
Files and Templates are transferred to specified paths on the node. While the former are static, the latter are powered by ERB – ideal to generate configuration files.
Metadata describes the recipes, dependencies, supported platforms, etc.

I’ve recently written a Chef cookbook that installs Git from source and optionally configures it. Its directory structure is shown below. You can dig into the source code at GitHub.

.
├── attributes/
│   ├── config.rb
│   └── source.rb
├── metadata.rb
├── README.md
├── recipes/
│   ├── config.rb
│   └── source.rb
└── templates/
    └── default/
        └── gitconfig.erb

Let’s Play!

Now enough with the theory. Let’s provision a virtual machine with Vagrant, Chef Solo, and some cookbooks! The result will be a great learning resource for Chef – a playground that you can use for further research and testing. (If you need an introduction to provisioning with Vagrant, I recommend reading my last blog post first. I won’t repeat the basics here.)

As a practical example, let’s use the Git cookbook I mentioned earlier. The cookbook’s git::source recipe will compile and install Git on the target node. It depends on two other cookbooks:

apt – updates the Debian package index for us (a sane thing to do for Debian/Ubuntu VMs)
build-essential – installs everything required to compile C programs like Git (gcc, make, etc.)

To make a long story short, here is how to set things up:

# Create a new project folder
$ mkdir cookbook-dev && cd cookbook-dev/

# Clone required cookbooks into "cookbooks" folder
$ mkdir cookbooks
$ git clone git://github.com/opscode-cookbooks/apt.git cookbooks/apt
$ git clone git://github.com/opscode-cookbooks/build-essential.git cookbooks/build-essential
$ git clone git://github.com/mlafeldt/git-cookbook.git cookbooks/git

# Create Vagrantfile as seen below
$ vim Vagrantfile

The Vagrantfile tells Vagrant what to do:

Vagrant::Config.run do |config|
  # VM will be based on Ubuntu 12.04 (64 bit)
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"

  # Configure the Chef Solo provisioner
  config.vm.provision :chef_solo do |chef|
    # Tell Vagrant where the cookbooks are located
    chef.cookbooks_path = "cookbooks"
    # Add recipes to be executed in given order
    chef.add_recipe "apt"          # Shortcut for apt::default
    chef.add_recipe "git::source"  # Will include build-essential::default
  end
end

Now that we have everything in place, run vagrant up to boot the VM and let Chef Solo do the work for us. As you will see in the log output, the cookbooks are executed in the order defined.

$ vagrant up
...
[default] Booting VM...
...
[default] Running provisioner: Vagrant::Provisioners::ChefSolo...
[default] Generating chef JSON and uploading...
[default] Running chef-solo...
[Mon, 10 Sep 2012 14:51:25 +0000] INFO: *** Chef 0.10.10 ***
...
[Mon, 10 Sep 2012 14:51:25 +0000] INFO: Run List expands to [apt, git::source]
[Mon, 10 Sep 2012 14:51:25 +0000] INFO: Starting Chef Run for precise64
...
[Mon, 10 Sep 2012 14:53:49 +0000] INFO: bash[build and install git] ran successfully
[Mon, 10 Sep 2012 14:53:49 +0000] INFO: Chef Run complete in 144.378525 seconds

The result speaks for itself:

$ vagrant ssh
vagrant@precise64:~$ which git
/usr/local/bin/git
vagrant@precise64:~$ git --version
git version 1.7.12

Mathias Lafeldt

Learning Chef

Learn Ruby (First)

Vagrant and Chef Solo

The Guts of a Cookbook

Let’s Play!

Further Reading