• Rationale
• Test Runner UI
  • Log
  • Controls
  • Settings
  • Defining a Test Suite
  • Defining Tests
• Installation
• Getting Started
• Transition.js API

Rationale

Transition.js was created to address specific issues I was having with trying to test my websites on Mobile and Tablet devices.

The websites I needed to test were invariably asynchronous. This lead to writing tests that poll and wait for asynchronous events to complete. Detecting when events completed was error-prone. The tests were brittle and would fail inconsistently due to timing issues.

I wanted a test suite that drove the user interface of the website as close to how a user would. Not just a framework that would test the underlying JavaScript code at the unit level.

I wanted a solution that would execute a test suite on mobile devices (phones and tablets).

At the time I started Transition.js there was no tool or framework that I knew of that fit this criteria.

Inverting the need to Poll

Since waiting for a condition to become true before continuing was a core part of the tests I was writing, I made the idea of waiting core to the testing framework. The most natural way of expressing this for me was to implement tests as a State Machine.

Each test has a name, an optional `initialize` function and a list of states. Each state has a name, an initializer, and defines the set of transitions to subsequent states. A transition consists of the name of the target state and a predicate function.

When a test is run by Transition.js, the framework will call the test’s initializer, then:

• transition to the first state of the test
• call the initializer for the first state
• try each of the current state’s transitions

If a predicate returns true, then the target state will be transitioned to. If no predicates return true, the framework will set a timeout (default is 250ms) before trying again. If the test takes more than a pre-set maximum (default 30s), then the test is considered failed. If it reaches a success state before the timeout, it is considered passed.

The Runner UI

Test Suite

The ‘Suite’ menu allows you to select a test to run, optionally by first filtering the list. You may also reload your test suite. Reloading does not effect the state of the Runner UI and is an effective tool when developing and debugging your tests.

The Test Log

Transition.js has a logging system that logs to the Runner UI instead of the browser’s debug console. The browser’s debug console is often either not visible on some devices, difficult to enable or not fully featured. The Transition.js logger is visible, supports searching (filtering) and setting different levels of output.

Test Controls

There are several control buttons in the runner UI:

• Run Suite: this starts a full run of the suite from the first test.
• Start: starts the currently selected test, or the first test in the suite if none is selected.
• Stop: this will halt the currently running test or full suite.
• Step: stepping allows you to walk through your test one poll at a time so you can observe what it is doing.
• Back: if you are single-stepping a test, the Back button will move back a state (executing its trigger function)
• Continue: if you have single-stepped a test, Continue will start the polling process so the test continues to run automatically.

Settings

Some settings can be changed via the test runner’s UI. Settings that are not exposed through the UI can be set from your test-suite.js:

// in test-suite.js
Transition.models.settings.set('pollTimeout', 250);
Transition.models.settings.set('logLevel',    Transition.Log.Levels.DEBUG);

// load your scripts here
// Transition.loadScript('/test/test-name.js');

int: frame-divider-upper-pct default=50

int: frame-divider-lower-pct default=50

These control the location of the divider between the main and test frames in the runner UI.

boolean: sortByLastModified default=true

If set to true, when tests are loaded via loadScript, the test suite will be ordered in reverse by time. The most recently modified test will be first, while The least recently modified will be last.

miliseconds: perStateTimeout default=10000

If a test stays in a single state for longer than this amount of time, the test will be marked as failed.

miliseconds: testTimeout default=30000

If a test takes longer than this amount of time, the test will be marked as failed.

miliseconds: suiteTimeout default=60000

If the test suite takes longer than this amount of time, the suite will fail.

miliseconds: pollTimeout default=500

This is the amount of time the framework will wait before checking the test’s current state to see if it should proceed to a subsequent state.

enum: logLevel default=Transition.Log.Levels.TRACE

This controls the verbosity of the log display. This value can also be adjusted via the Log drop-down menu. The following log levels are supported:

1. TRACE
2. DEBUG
3. INFO
4.. WARN
5. ERROR
6. FATAL

Defining a Test Suite

After creating the symbolic link to the Transition.js public directory, your next step is to define your test suite. You do this in the file test-suite.js in the same place you created the symbolic link:

public/
  +-- transition -> ~/projects/transition.js/public
  +-- test-suite.js
  +-- tests
      +-- add-a-list.js
      +-- add-todo-item.js
      +-- index-page.js
      +-- mark-item-as-done.js

Tests can be defined within the test-suite.js file directly, or in separate files. Best practice is to place utility code for testing your application in the test-suite.js and then each test in its own file:

(function () {
  var root        = this,
      TodoTestLib = {};

  this.TodoTestLib = TodoTestLib;

  TodoTestLib.deleteTestList = function () {
    ...
  };

  TodoTestLib.createTestList = function () {
    ...
  };

}.call(this));

Transition.loadScript('/tests/index-page.js');
Transition.loadScript('/tests/add-a-list.js');
Transition.loadScript('/tests/add-todo-item.js');
Transition.loadScript('/tests/mark-item-as-done.js');

Optinally Defining a test-suite.html for Static Content

If your tests themselves require helper libraries or other static content (like their own UI controls, html, etc.), you can create a test-suite.html file next to test-suite.js. The content of this file will be fetched and appended to the <body> before the test-suite.js file itself is loaded.

Defining Tests

Tests are added by calling Transition.addTest, passing an object of properties.

The test must have 2 properties:

• name: the name of the test
• states: the set of states that comprise the test.

Additional properties will become available as attributes on the Test model (available within your test with this.get('attr') and this.set('attr', 'value')).

Properties that are functions will be lifted to become methods on the Test model itself. Be careful not to re-define any existing Backbone Model methods.

States are constructed with Transition.newState. Each state must have:

• name: the name of the state
• on-enter callback, which is called once, when the state is entered.
• one or more exit transitions, created with to

Here is an example that loads the root page of the example TODO application and asserts that the form for creating lists is present:

(function () {

  this.addTest({
    name: 'Test Index Page',

    states: [
      this.newState('init', this.navigateTo_('about:blank'))
        .to('mainPage', this.constantly_(true)),
      this.newState('mainPage', this.navigateTo_('/'))
        .to('success').when('form[action="/lists"]')
    ]
  });

}.call(Transition));