Behaviour-Driven Development

• Dan North (instigator of BDD) says: "BDD is a second-generation, outside-in, pull-based, multiple-stakeholder, multiple-scale, high-automation, agile methodology. It describes a cycle of interactions with well-defined outputs, resulting in the delivery of working, tested software that matters" (Wikipedia)

Cucumber

http://cukes.info/

• Allows us to express requirements in something very close to plain English (using Gherkin)
• Executable Specification

Write a feature

Feature: Sign in to the member directory
  As a member, I need to sign in to the system to modify my account details

  Scenario: Successful signin
    Given that I have a membership number and password
    When I visit the sign in page
    And I enter my membership number and password
    And the password is correct
    When I click sign in
    Then I should have signed in successfully

Watch it fail

cucumber features/signin.feature -f progress
UUUUUU

1 scenario (1 undefined)
6 steps (6 undefined)
0m0.004s

You can implement step definitions for undefined steps with these snippets:

When(/^I enter my membership number and password$/) do
  pending # express the regexp above with the code you wish you had
end

Then(/^I should have signed in successfully$/) do
  pending # express the regexp above with the code you wish you had
end

Define the steps

Drive out the code required to make the tests pass

Given /^that I have a membership number and password$/ do
  member = Member.create(
    :email => 'sam@foobar.com',
  )
  member.confirm!
  @membership_number = member.membership_number
  @password = 'p4ssw0rd'
end

When /^I enter my membership number and password$/ do
  fill_in('member_membership_number', :with => @membership_number)
end

Then /^I should have signed in successfully$/ do
  page.should have_content "Signed in successfully"
end

Live documentation

Cucumber extensions

Aruba

Pre-baked step definitions (and some other stuff)

Scenario: one without postcodes
  When I successfully run `noodile sample.csv`
  Then a file named "outputs/complete.no.postcodes.csv" should exist

Then /^a file named "([^"]*)" should exist$/ do |file|
  check_file_presence([file], true)
end

def check_file_presence(paths, expect_presence)
  prep_for_fs_check do
    paths.each do |path|
      if expect_presence
        File.should be_file(path)
      else
        File.should_not be_file(path)
      end
    end
  end
end

Robot-powered Infrastructure

All watched over by machines of loving grace

Things used to suck

• Physical iron, in a datacentre, with everything hand-installed
• Crappy documentation (if you were lucky)
• Maybe some hacky bash scripts
• SNOWFLAKE!

Treat your servers as cattle, not as pets

• Identical
• Repeatable
• Disposable

Chef

• Infrastructure as code
• Describe your (desired) infrastructure with a Ruby DSL
• Key concepts
  • Nodes
  • Recipes which are contained inside cookbooks
  • Roles
  • Environments
  • Data bags
  • Knife, the command-line tool which drives it all

Simple abstractions

For example, this:

package 'nginx' do
  action :install
end

installs stock nginx. Under the hood, Chef works out from the host OS whether it needs apt or yum or whatever, but we don't need to care about that

Elegant idempotency

• Wikipedia says "In computer science, the term idempotent is used... to describe an operation that will produce the same results if executed once or multiple times"
• OpsCode's resources (e.g. package) are guaranteed to be idempotent
• But we can also cast aside the safety net and dive right in...

Raw scripts

script 'Bundling the gems' do
  interpreter 'bash'
  cwd current_release_directory
  user running_deploy_user
  code <<-EOF
  bundle install --without=development --quiet --path #{bundler_depot}
  EOF
end

• This is valid Chef
• No guarantees here - you're on your own
• There's nothing to stop you putting

  :(){ :|:& };:

  in there (DON'T DO THIS!)
• This might be a good way in if you're wanting to try out Chef

Test-driven Infrastructure

TDD allows me to demonstrate my incompetence in the tests *as well* as in the code. Awesome.

Cucumber-chef

• Test lab
  • Chef server
  • LXC test instances
• Set of cucumber step definitions

The Chef server

LXC

• Wikipedia says: "LXC (LinuX Containers) is an operating system-level virtualization method for running multiple isolated Linux systems (containers) on a single control host."
• This is how Heroku works
• Docker is also based on LXC
• Dokku, a Heroku of your own

Step definitions

Things like

And /^I run "([^\"]*)"$/ do |command|
  @result    = @connection.exec(command, :silence => true)
  @output    = @result.output
  @exit_code = @result.exit_code
end

Then /^I should( not)? see "([^\"]*)" in the output$/ do |boolean, string|
  if (!boolean)
    @output.should =~ /#{string}/
  else
    @output.should_not =~ /#{string}/
  end
end

A cucumber-chef example

Ecosystems

The Labfile

ecosystem "odc" do
  container "web-certificate-01" do
    distro "ubuntu"
    release "precise"
    persist true
    ip "192.168.98.30"
    mac "00:00:5e:16:89:b5"
    chef_client (
      {
        :environment => "odc-production",
        :run_list => [
          "role[certificate]"
        ]
      }
    )
  end
end

Features

Feature: webserver
  Background:
    * I ssh to "web-certificate-01"

  Scenario: Core dependencies are installed
    * package "git" should be installed

  Scenario: Ruby 1.9.3 is installed
    When I run "su - certificate -c 'ruby -v'"
    Then I should see "1.9.3" in the output

  Scenario: configuration stuff is correct
    * file "current/config/database.yml" should exist
    When I run "cat current/config/database.yml"
    Then I should see "host: 192.168.98.20" in the output

Step definitions

Then /^package "([^\"]*)" should be installed$/ do |package|
  command = ""
  if (dpkg = @connection.exec("which dpkg 2> /dev/null",
            silence: true).output).length > 0
    command = "#{dpkg.chomp} --get-selections"
  elsif (yum = @connection.exec("which yum 2> /dev/null",
            silence: true).output).length > 0
    command = "#{yum.chomp} -q list installed"
  end

  @result = @connection.exec(command, :silence => true)
  @result.output.should =~ /#{package}/
end

Making it pass

Add the recipe to the role

name 'certificate'
  default_attributes 'user'              => 'certificate',
                     'group'             => 'certificate',
                     'migration_command' => 'bundle exec rake db:migrate'

  run_list "role[base]",
           "recipe[chef-client::cron]

name "base"
  run_list "recipe[git]"

And now this test passes

Librarian

• Tool for managing cookbooks
• A bit like bundler
• Driven by the Cheffile

site 'http://community.opscode.com/api/v1'

cookbook 'chef-client'
cookbook 'apt', '= 1.9.0'
cookbook 'git'
cookbook 'mongodb', :github => 'edelight/chef-mongodb'
cookbook 'mysql'
cookbook 'fail2ban', :github => 'opscode-cookbooks/fail2ban'

Putting it into production

Vagrant

• Is amazing
• Started as a wrapper around VirtualBox
• Since 1.1, can drive cloud servers too
• http://www.vagrantup.com/

The Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.define :certificate_theodi_org_01 do |config|

  config.vm.provider :rackspace do |rs|
    rs.flavor   = /512MB/
    rs.image    = /Precise/
    rs.auth_url = "https://lon.identity.api.rackspacecloud.com/v2.0"
  end

  config.vm.provision :chef_client do |chef|
    chef.environment     = "odc-production"
    chef.chef_server_url = "https://chef.theodi.org"
      chef.run_list      = [
      "role[certificate]"
    ]
  end
end

Continuous deployment

• Pull-request-based development (Gitflow)
• Continuous integration (http://jenkins.theodi.org/)
• Every Chef run checks for new code and deploys
• If it's ready, it goes live