Learning Chef

I’m currently in the process of learning Chef – the infrastructure automation framework by Opscode – and wanted to share some of the things I’ve picked up along the way. To give you a little bit of background, we’ve chosen Chef at work to gradually replace our deployment and configuration system, which is, unfortunately, entirely based on custom Debian packages (and plenty of magic). There’s still lots of work ahead of us and I’m far away from being a Chef expert, but I do feel comfortable enough to give you some tips on how to get started with Chef.

Learn Ruby (First)

The first thing people will usually tell you about Chef is that it allows you to write “cookbooks” with “recipes” defining how to configure a server to do X, with the standard example being an “Apache” cookbook which sets up the Apache web server.

Then you’ll probably hear that the whole system is based on Ruby. While Opscode claims that you don’t need to have experience with Ruby to write recipes (thanks to the straightforward recipe DSL), I still decided to learn Ruby first. In fact, I had wanted to learn this wonderful language for a while and Chef gave me another good reason. In hindsight, that was the right thing to do, in particular in regard to more advanced topics like automated cookbook testing.

Sooner or later, you need to dive into Ruby.

Vagrant and Chef Solo

Fast forward and several Ruby experiments later, I came back. By then, I had not only become a passable Ruby programmer, I also knew the best way to get started with Chef: Vagrant. (If you’ve been following this blog, you already know I have a thing for it.) Vagrant comes with provisioners for two flavors of Chef: Chef Solo and Chef Server.

The Guts of a Cookbook

Before continuing with Vagrant and Chef Solo, I want to tell you a bit more about Cookbooks.

Cookbooks are the fundamental units of distribution in Chef. They encapsulate all the resources you need to automate your infrastructure and are easily sharable with other Chef users. -- Opscode

To put it simple, a cookbook is a bunch of files and directories. For someone new to Chef, the most important components to know are:

I’ve recently written a Chef cookbook that installs Git from source and optionally configures it. Its directory structure is shown below. You can dig into the source code at GitHub.

├── attributes/
│   ├── config.rb
│   └── source.rb
├── metadata.rb
├── README.md
├── recipes/
│   ├── config.rb
│   └── source.rb
└── templates/
    └── default/
        └── gitconfig.erb

Let’s Play!

Now enough with the theory. Let’s provision a virtual machine with Vagrant, Chef Solo, and some cookbooks! The result will be a great learning resource for Chef – a playground that you can use for further research and testing. (If you need an introduction to provisioning with Vagrant, I recommend reading my last blog post first. I won’t repeat the basics here.)

As a practical example, let’s use the Git cookbook I mentioned earlier. The cookbook’s git::source recipe will compile and install Git on the target node. It depends on two other cookbooks:

To make a long story short, here is how to set things up:

# Create a new project folder
$ mkdir cookbook-dev && cd cookbook-dev/

# Clone required cookbooks into "cookbooks" folder
$ mkdir cookbooks
$ git clone git://github.com/opscode-cookbooks/apt.git cookbooks/apt
$ git clone git://github.com/opscode-cookbooks/build-essential.git cookbooks/build-essential
$ git clone git://github.com/mlafeldt/git-cookbook.git cookbooks/git

# Create Vagrantfile as seen below
$ vim Vagrantfile

The Vagrantfile tells Vagrant what to do:

Vagrant::Config.run do |config|
  # VM will be based on Ubuntu 12.04 (64 bit)
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"

  # Configure the Chef Solo provisioner
  config.vm.provision :chef_solo do |chef|
    # Tell Vagrant where the cookbooks are located
    chef.cookbooks_path = "cookbooks"
    # Add recipes to be executed in given order
    chef.add_recipe "apt"          # Shortcut for apt::default
    chef.add_recipe "git::source"  # Will include build-essential::default

Now that we have everything in place, run vagrant up to boot the VM and let Chef Solo do the work for us. As you will see in the log output, the cookbooks are executed in the order defined.

$ vagrant up
[default] Booting VM...
[default] Running provisioner: Vagrant::Provisioners::ChefSolo...
[default] Generating chef JSON and uploading...
[default] Running chef-solo...
[Mon, 10 Sep 2012 14:51:25 +0000] INFO: *** Chef 0.10.10 ***
[Mon, 10 Sep 2012 14:51:25 +0000] INFO: Run List expands to [apt, git::source]
[Mon, 10 Sep 2012 14:51:25 +0000] INFO: Starting Chef Run for precise64
[Mon, 10 Sep 2012 14:53:49 +0000] INFO: bash[build and install git] ran successfully
[Mon, 10 Sep 2012 14:53:49 +0000] INFO: Chef Run complete in 144.378525 seconds

The result speaks for itself:

$ vagrant ssh
vagrant@precise64:~$ which git
vagrant@precise64:~$ git --version
git version 1.7.12

Further Reading

I’m aware that I really just scratched the surface of what you can do with Chef. Fortunately, there is a vital community around the project that has produced tons of information. Here are the sites I found most helpful while learning Chef: