Enrique Amodeo

(who is this guy?)

• Programming since 1984
• Currently Software Engineer at SoundCloud
• Has loved JS since 2005
• Test infected
• Enthusiast of the Agile/Lean way
• Follow me at @eamodeorubio

This talk

1. To cucumber or not to cucumber
2. No need to test the UI?
3. I want my UI tested
4. Testing xBrowser issues
5. Maintainable tests
6. Conclusions

Specification by example

(trendy)

• Write your features in natural language
• Don't specify business rules...
• ...but key business examples

• Customer can read them
• Customer can give you early feedback
• Customer can even write some of them

Gherkin

(a "pseudonatural" language)

Feature: get cash from an ATM
  Background:
    Given the ATM has 1000
    And the user John is authenticated
    And the user's account has 5000
  Scenario: success
    When the user asks the ATM for 500
    Then the ATM will have 500
    And the user's account will have 4500
    And the ATM will provide 500 in cash
  Scenario: not enough money in the ATM
    When the user asks the ATM for 1500
    Then the ATM will have 1000
    And the user's account will have 5000
    And the ATM will notify the user it does not have enough cash

Cucumber

(in a nutshell)

Will cucumber test the features?

No, not really

1. Parse the gherkin
2. Match gherkin with your test functions
3. Execute the matched test functions
4. Generate a report

Cucumber

(everywhere)

• Ruby
• JVM
• ... and JS too

CucumberJS: Steps

(to setup, execute, assert)

module.exports = function() {
  this.World = require('./support/World');
  this.Given("the ATM has $cash", function(cash,done){
    this.ATM().cashAvailable(Number(cash), done);
  });

  this.When("the user asks the ATM for $cash", function(cash,done){
    this.ATM().requestCash(Number(cash), done);
  });

  this.Then("the user's account will have $cash",
   function(cash, done) {
    this.DB().accountFor(this.userId).then(function(acc) {
      expect(cash).to.be.eql(acc.cash);
    }).then(done, done);
  });
};

CucumberJS: World

(utilities & context)

    // A new World will be created by scenario
    module.exports = function(done) {
      var db, atm;
      this.DB = function() {
        return db;
      };
      this.ATM = function() {
        return atm;
      };
      openDB(function(DB) {
        db = DB;
        openUI(function(ui) {
          atm = new ATM(db, ui);
          done();
        });
      });
    };

CucumberJS: Hooks

(refactor!)

    // Before & after each scenario
    module.exports = function() {
      this.Before(function(done) {
        this.DB()
            .eraseAll()
            .createDefaultData()
            .then(done, done);
      });
      this.After(function(done) {
        var db = this.DB();
        this.ATM()
            .shutdown()
            .fin(db.shutdown.bind(db))
            .fin(done);
      });
    };
  

CucumberJS: how do I run it?

(nodeJS)

  // package.json
  {
    // ....
    "scripts": {
      "cucumber": "cucumber-js features/ -r features/step_defs/"
    },
    "devDependencies": {
      "cucumber": "~0.3.0"
    }
  }
  

  $> NODE_ENV=test npm run-script cucumber
  

YAGNI

(early customer feedback?)

• My customer won't read the gherkin
• My customer won't understand the gherkin
• My customer won't be available
• The technical team is the customer
• I won't get value from gherkin

Mocha

(early customer feedback?)

• A JS testing framework
• Not only for unit testing
• It has a BDD like API available

Just write your functional tests!

Mocha BDD interface

(describe/context/test)

describe('Feature: get cash from an ATM:', function() {
  context('Scenario: success', function() {
    describe('When the user asks the ATM for 500', function() {

      it('Then the ATM will have 500', function() {
        expect(world().getATM().remainingCash()).to.be.eql(500);
      });

      it("Then the user's account will have 4500", function(done) {
        world().getDB().accountFor(userId).then(function(acc) {
          expect(cash).to.be.eql(acc.cash);
        }).then(done, done);
      });

    });
  });
});

describe and context only for reports?

Set up & action

(a bit of refactor)

var world = require('./support/world');
describe('Feature: get cash from an ATM:', function() {
  beforeEach(function(done) {
    world().getATM().cashAvailable(Number(cash), done);
  });
  // Any other background set up ....
  context('Scenario: success', function() {
    beforeEach(function() {
      // Any scenario specific setup, none in this case
    });
    describe('When the user asks the ATM for 500', function() {
      beforeEach(function(done) {
        world().getATM().requestCash(500, done);
      });
    });
  });
});

"Global" hooks

(start and stop your world)

var world;
before(function(done) {
  world = new World(done);
});

beforeEach(function(done) {
  world.getDB().eraseAll().createDefaultData().then(done, done);
});

afterEach(function(done) {
  world.getATM().shutdown().fin(db.shutdown.bind(db)).fin(done);
});

module.exports = function() {
  return world;
};

Running Mocha

(with NPM)

  // package.json
  {
    // ....
    "scripts": {
      "test": "mocha -u bdd -R dot --recursive test/unit/",
      "bdd": "mocha -u bdd -R dot --recursive test/bdd/"
    },
    "devDependencies": {
      "mocha": "~1.8.1"
    }
  }
  

  $> NODE_ENV=test npm run-script bdd
  

Running Mocha

(with Grunt & simplemocha)

  grunt.initConfig({
    // ....
    simplemocha: {
      options: {
        timeout: 3000,
        ui: 'bdd',
        reporter: 'dot'
      },
      unit: {
        files: { src: ['test/unit/**/*.js'] }
      },
      bdd: {
        files: { src: ['test/bdd/**/*.js'] }
      }
    }
  });

  $> NODE_ENV=test grunt simplemocha:bdd

Mocha VS. CucumberJS

(it is your decision)

• CucumberJS is more customer-friendly
• Is mocha more developer friendly? Maybe
• Mocha is more mature, CucumberJS still green!
• Mocha is more integrated with other tools
• Mocha for BDD is a bit more awkward

Should my BDD attack the UI?

(it depends on...)

• How much it will cost you (tech)
• Do you really have that much logic in the UI?
• Have you already unit tested your UI?
• Is the UI generating many bugs?

A typical web app

(maybe yours looks similar)

Why not?

(fast & cheap)

Or even this?

(faster, cheaper)

BDD only your domain

(simply use NodeJS)

Works for either Mocha or CucumberJS

100s of tests per second

Classic BDD

(drive the UI)

Faking the browser

(a good trade off)

• No need to test for xBrowser issues
• Only UI & DOM logic worries me
• Very fast!
• ZombieJS!

Going Zombie

Headless browser

(you must be sure)

• Real DOM!
• Real browser quirks!
• Real HTML5 (or lack of) features!
• Not so fast as you think
• More complex (start/stop, retrieve results)

Enter PhantomJS

PhantomJS

(not a silver bullet)

• I want my NodeJS back!
• Complex coordination: reporter/tests/SUT
• Low level API (CasperJS gives a better API)
• Only WebKit
• What's the advantage over FF with XVFB?

Multi browser test

(the only way)

• Each browser is a different world
• Complex coordination: reporter/tests/SUT
• Do it for each browser!

Enter Karma

(previously testacular)

• A NodeJS tool
• Multiple browser support (Chrome, IE, FF, Phantom)
• Multiple testing framework support (MOCHA, JASMINE)
• Extensible (but not well documented)
• CucumberJS is not yet supported

Good karma (1)

Web Page icon from http://commons.wikimedia.org/wiki/File:1328101978_Web-page.png

Good karma (2)

Configuring Karma (1)

    // karma.conf.js

    reporters = ['dots', 'junit'];
    port = 9876;
    runnerPort = 9100;
    logLevel = LOG_DEBUG;
    browsers = ['Chrome', 'Firefox', 'PhantomJS'];
    singleRun = false;

    //....
  

Configuring Karma (2)

// karma.conf.js (continued)

var BASE_DIR = 'src/test/bdd/';

files = [
  MOCHA,
  MOCHA_ADAPTER, 
  {pattern: BASE_DIR + "vendor/**/*.js", watched: false},
  {pattern: 'node_modules/chai/chai.js', watched: false},
  {pattern: BASE_DIR + '/features/**/*.js', watched: false},
  {pattern: 'www/**', watched: false, included: false}
];
  

Avoid UI Coupling: Gherkin

(bad gherkin)

  Scenario: success
    When the user enters 500 into the "amount" field
    And the user press the submit button
    Then the ATM will have 500
    And the user's account will have 4500
    And the ATM will provide 500 in cash

  

Trivial change in UI -> Multiple changes in Gherkin and Steps

Avoid UI Coupling: Steps

(bad step)

  // ...

  this.When("the user asks the ATM for $cash",function(cash, done){
    browser.fill('amount', Number(cash))
           .pressButton('submit')
           .then(done, done);
  });

  // ...
  

Trivial change in UI -> Multiple Steps affected

Avoid UI Coupling: Mocha

(bad specs)

  // ...

  describe('When the user asks the ATM for 500', function() {
    beforeEach(function(done) {
      browser.fill('amount', 500)
             .pressButton('submit')
             .then(done, done);
    });

    // ...
  });

  // ...
  

The page object

(encapsulate UI)

• Changes in UI?
• Change only your page object
• A new scenario?
• Reuse page object to drive the UI
• Complex page?
• Use composite page object

Page object with Zombie

var Browser = require('zombie');
module.exports = function() {
  var browser = new Browser({ site:baseURL });
  this.userIsLoggedIn = function(userId) {
    browser.cookies('.atm.com', '/')
           .set(SESSION_COOKIE, newSessionToken(userId));
  };
  this.visitPage = function(pageName) {
    return browser.visit('/' + pageName + '.html'); 
  }; 
  this.requestCash = function(money) {
    return browser.fill('amount', money).pressButton('submit');
  };
  this.displayedCurrentAmount = function() {
    return Number(browser.text('.current-amount'));
  };
};

Page object with a real browser

(the iframe trick)

Isolates the SUT

Page object with a real browser

(visiting a page)

var bdd = bdd || { count: 0 };
bdd.UI = function () {
  var childDOC, self = this, id='fr' + (bdd.count++);
  this.visitPage = function(pageName, done) {
    destroyIframeIfExists();

    $('body').append('<iframe id="'+frameId+'"></iframe>');
    
    $('#' + frameId).load(function () {
      childDOC = this.contentDocument;
      setTimeout(done, 500); // Wait for app to initialize
    });

    $('#fr1').attr('src', '/' + pageName + '.html');
  };
  // ...
};

Page object with a real browser

(filling a form)

  // ...

  this.requestCash = function(money, done) {
    $('input[name="amount"]', childDOC).val(text)
                                       .trigger('keyup')
                                       .trigger('change');

    $('button[type="submit"]', childDOC).focus().click();

    done();
  };

  // ...
  

Page object with a real browser

(simulating events)

  this.requestCash = function(money, done) {
    $('input[name="amount"]', childDOC).simulate("key-sequence", {
      sequence: String(money),
      callback: function () {
        $('button[type="submit"]', childDOC).simulate('click');

        done();
      }
    });
  };
  

Using jQuery simulate https://github.com/eduardolundgren/jquery-simulate

and jQuery simulate extensions https://github.com/j-ulrich/jquery-simulate-ext

Page object with a real browser

(inspecting the page)

  // ...

  this.displayedCurrentAmount = function() {
    return Number($('.current-amount', childDOC).first().text());
  };

  // ...
  

Page object factory

(declarative page objects)

  bdd.definePageObject('ui', {
    requestCash: {
      selector: 'form.request-cash',
      fields: { amount: 'input[name="amount"]' },
      submit: 'button[type="submit"]'
    },
    displayedCurrentAmount: {
      selector: '.current-amount',
      conversor: Number
    }
  });

  bdd.newPageObject('ui').visitPage('atm')
                         .requestCash({amount: 500})
                         .then(done, done);

It could be nice to get rid of all the boilerplate!

Decouple set up too!

(a "page object" for test data)

module.exports = function() {
  this.World = require('./support/World');
  this.Given("the ATM has $cash", function(cash,done){
    this.ATM().cashAvailable(Number(cash), done);
  });

  this.When("the user asks the ATM for $cash", function(cash,done){
    this.ATM().requestCash(Number(cash), done);
  });

  this.Then("the user's account will have $cash",
   function(cash, done) {
    this.DB().accountFor(this.userId).then(function(acc) {
      expect(cash).to.be.eql(acc.cash);
    }).then(done, done);
  });
};

What JS gives us?

(compared with other languages)

• Simple page objects
• Direct access to DOM
• No need to add an extra layer (Selenium, WebDriver, Capybara...)
• Same language/platform as the UI
• Same skills as the UI
• The tools are now ok!

Making hard decisions

(usually early in the project)

• Should I use BDD at all?
• Should I use gherkin?
• Should I test the UI or only the core?
• JS tool ecosystem changes very fast!
• I'd like to change my mind in the future

Deferring our technical decisions

(invest in good testing architecture)

You can start cheap & simple

Changed your mind?

You can evolve to a more complex testing platform

Some code

For CucumberJS with page object using ZombieJS: http://bit.ly/ZKApZH

For Mocha, Karma & the iframe page object: http://bit.ly/ysoZHx

Warning: There is still some code to clean up there!