JavaScript Tests

JavaScript tests in Windmill consist of a directory tree full of .js files containing your tests. On the server-side, Windmill just recursively parses the directory tree and serves up the files inside -- so the definitions for test ordering and tweaking the framework are all handled in JavaScript.

The JavaScript test framework can call the same set of UI commands as the JSON or Python tests, and also allows you to use all the same asserts available in both of the major JSUnit testing frameworks (see "Asserts," below).

Test Setup and Parsing

Windmill will search recursively through the tested applications's window object and register any functions (or arrays, see Test Formats below) that have a name beginning with "test_" ('test' plus underscore), "setup", or "teardown." These tests will be run hierarchically in whatever order that the parser finds them. (For test objects the same level of a hierarchy, setups happens first, and teardowns happens last, as described below.)

Test Phases

The "setup"/"test_"/"teardown" naming determines what phase the tests belong to: within the same level, tests named "setup" are run first, followed by any named "test_", followed by any named "teardown." This allows you to create a complex hierarchy of tests, and make sure that whatever setup is needed for a specific test happens exactly the same way every time.

(There is also an option available when running tests to run only the tests in a particular phase (see "Test-Writing Workflow," below). This saves a lot of time when you're developing tests by cutting out redundant steps, and can help in debugging by allowing you to home in on a single phase.)

Order of Loading and "require"

Like with any other JavaScript-heavy application, working with the Windmill JavaScript test framework requires you to think a little bit about the order in which your code loads and executes. By default, the test parser will simply find your tests in the order they are added to the window object during the eval process of the code in your test files.

The require command solves this problem, allowing you to set up a hierarchy of tests that run in the order you want. It also allows you to modularize and share code throughout your tests. More about that below, in "Organizing Your Tests."

Organizing Your Tests

When you run your tests, Windmill evals the contents of your test .js files in the window scope of the application being tested. This just means that if you have someNamespace in your app, you can access it directly in your JavaScript test code without doing anything special.

If you just drop a bunch of .js files in a test directory, and run your tests, there is NO GUARANTEE OF THE ORDER in which your files will be eval'd and parsed.

For the simplest cases, this may be okay, but as you do more testing, you'll soon find you want your tests to happen in a specific order. The Windmill JavaScript test framework provide the handy "require" method to make that happen for you.

windmill.jsTest.require forces the framework to load and eval a file immediately, like so:

 windmill.jsTest.require('shared/test_login.js');

The path for the require is always relative to the location of the top-level directory of JavaScript tests you're running -- NOT to the file where you're doing the require.

The require method works recursively -- you can require files that require other files. The framework will only eval the code in a particular file once during a test run, no matter how many times you require it.

Test Environment Initialization

The Windmill JavaScript test framework gives you one other little tool for the order-of-loading problem. When loading, eval'ing, and parsing test files, it first looks for a file named initialize.js, and runs the code in it first, before doing anything with any other files in your test directory.

This can be useful for setting up shortcuts to use in your tests, or creating namespaces for holding your test objects temporarily until you're ready to hang them on the dependency hierarchy.

An initialize.js file might have stuff in it like this:

var BASE_PATH = '/some_path'
var ACCOUNT_PREFS = myApp.base.getPrefs();
var USERNAME = fleegix.cookie.get('username');
var LOGOUT_REDIRECT = 'http://subdomain.mydomain.org/thanks';

var fooNamespace = {};
var barNamespace = {};

Note that you can get essentially the same effect as using an initialize.js with a strategically placed require, but using initialize.js can be a little more straightforward. Just remember that any code in initialize.js will run before anything else in your test code.

Test Formats

JavaScript tests can be written in two formats:

• Single functions (just like JSUnit tests)
• JavaScript arrays which contain either UI commands objects or executable functions.

Single functions

Single-function tests look just like JSUnit tests:

function test_fooThing () {
  var foo = 'asdf';
  jum.assertEquals(foo, 'asdf');
  var bar = 'qwer';
  jum.assertNotEquals(bar, 'asdf');
}

Test Arrays

Array-style tests use a JavaScript array which can contain two types of items:

• UI command objects
• Functions

UI Command Objects

These are plain JavaScript objects in the format { method: 'someMethod', params: { // Params to pass to the method } }. They are passed to the Windmill controller and executed, one by one.

Advanced usage tip: keep in mind that note that code containing these kinds of UI control objects is eval'd inline when the code for that .js test file is loaded. You can't reference objects in your command objects that don't exist on the page when the test code is eval'd.

For dealing with dynamically created UI elements, you're probably better off driving the UI using function-style tests that call the equivalent method available on the windmill.jsTest.actions object (see below).

Functions

These are anonymous functions that are simply executed in the window scope of the application you're testing.

var test_barThing = [
  // Click a button
  { method: "click", params: { id: "fooButton" } },
  // Wait for an Ajaxy update
  { method: "waits.sleep", params: { milliseconds: 4000 } },
  // Verify the Ajaxy update to the page happened
  function () {
    var someText = 
      document.getElementById('fooDiv').innerHTML;
    jum.assertEquals(someText, 'Howdy');
  }
];

Advanced usage tip: Note that the UI command-object methods are also available as JavaScript method calls on the windmill.jsTest.actions object. This can be useful if you have to do some kind of dynamic lookup of a UI element, or need to drive pieces of UI that aren't on the page on initial load.

Here's an example:

var test_bazThing = function () {
  var nodeId = getSomeDomNodeDynamically();
  windmill.jsTest.actions.click({ id: nodeId });
}

Asserts

Windmill's JavaScript test framework has the full range of JSUnit-compatible asserts, accessible inside your JavaScript test functions from the "jum" object. Here are the currently available asserts:

assertTrue
assertFalse
assertEquals
assertNotEquals
assertNull
assertNotNull
assertUndefined
assertNotUndefined
assertNaN
assertNotNaN
assertEvaluatesToTrue
assertEvaluatesToFalse
assertContains

The syntax is precisely the same as JSUnit, allowing an optional comment as a first parameter:

jum.assertNotNaN('Totally not a number!', 'Geddy Lee');
=> (Totally not a number!) assertNotNaN -- <Geddy Lee> (String) expected to be a number but was not a number (NaN).
jum.assertNotNaN('Neil Peart');
=> assertNotNaN -- <Neil Peart> (String) expected to be a number but was not a number (NaN).

Running JavaScript Tests

To run your JavaScript tests you can use one of the following methods.

From the shell:

run_js_tests('dir/to/js/tests') 

From the command line: windmill firefox jstests=./js/test/dir

Developing Tests -- Running Specific Tests or Phases

When you're working on a test, you don't want to have to run the setup prerequisites for your new test over and over while you're trying to debug it. The Windmill JavaScript test framework lets you run only specific tests, or only specific phases of a particular test.

To limit to a specific test, in the Windmill shell, add the desired test as a second parameter, like so:

run_js_tests("./fleegix_js/tests", 'test_fleegixEvent.test_foo')

Add the ns: prefix to limit your run to all the tests in a specific namespace:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixEvent')

To limit your test run to a specific phase, pass the desired phase (or phases in a comma-delimited string), as a third parameter to the run_js_tests command. This comes in handy when you're working on a specific test, and don't want to go through a full run of setup, test, and teardown every time.

First, get the test ready to run, by running just the setup:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'setup')

Then, run your the test only, as many times as you need:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'test')

When you want to run the whole thing, run the teardown once, and re-run the whole shebang:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'teardown')
run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'setup, test, teardown')