1. loomis
  2. givwenzen-flex

Wiki

Clone wiki

givwenzen-flex / UITesting

Automated acceptance criteria validation in Flex

This short tutorial will teach you how to use various testing tools to run acceptance criteria validation. First of all I want to explain what acceptance criteria validation is.

In short this is some kind of automated test that verifies if given criteria is met. I avoid the name functional testing but in fact this is very similar. The main difference is that the criteria should be short and more concise than the fully flavored functional test.

I'm going to use some of the latest cool tools for testing Flex:

givwenzen - an acceptance criteria validation framework in BDD style
u-input - a UI automation toolkit
morefluent - a fluent interface for verification of the asynchronous nature of flex and events in general
flexunit4 - a unit testing framework

I'm using IntelliJ in my daily work and this example but you should be able to do pretty much the same using any IDE. In the next tutorial I'll show you how to run the automation from continuous integration. In the meantime take a look at this thread.

All right, let's get the ball rolling.

First of all we'll start with some acceptance criteria:

I want to be able to enter credentials and log in
After login in I want to see welcome screen
If the name or password is not provided I should not be able to log in

Above does not look like a good BDD style AC. Let's rephrase it a bit:

Story: Logging in the system

Given
    I entered user name 'kris' and password 'secret'
When
    I submit credentials
Then
    The welcome screen is shown

Story: Prevent logging with credentials missing

Given
    The login screen is displayed
When
    I cleared user name
    I cleared password
Then
    The submit button is disabled

Now it looks a bit better and more BDD.

It's about a time to start creating a project. From the File menu I choose: New Project and Create new project from scratch. I also created a flex module.

creating project

Since I don’t care for the communication in this tutorial I'm going to skip over the wizard and just finish out with all the default wizard options. The only thing I want to change is the name of the main mxml file which I gave SampleApp name. But no worries it can be anything or you can rename it later.

I'll need some of the libraries. Here is a zip file with all I'm going to need. After unzipping you should have a lib folder with some swc files in it next to the src folder.

Once the libs are in place, I'm going to create a test folder next to the libs and src folders.

Now I'm going to update project settings to include libraries and tests from the test folder. To do it open up the File -> Project Structure. Choose the module and mark tests folder as a test folder.

adding test folder

Then switch to the Dependencies tab, create new module library and add lib folder to the set.

setting up dependencies

Well done. You should have a hello world project set up for later work.

Since I want to have my development test driven, I'll start off with preparing test scenarios before any other coding. To do that I'm going to create separate files for each acceptance criteria and put them to the test/stories folder.

The file login_in_the_system.txt

Given
    I entered user name 'kris' and password 'secret'
When
    I submit credentials
Then
    The welcome screen is shown

The file prevent_login_with_credentials_missing.txt

Given
    The login screen is displayed
When
    I cleared user name
    I cleared password
Then
    The submit button is disabled

Now I have my scenarios ready for GivWenZen. Let's make a next step and add these scenarios to the GivWenZen suite. GivWenZen has a special test suite that you can extend and easily add scenarios to it. In order to load scenarios into the GivWenZen, I created two static fields and added Embed metadata to both pointing to the corresponding files. Please note the mimeType I used.

Later on I added scripts to execution with addTestScriptFromResource() function.

package stories
{
import org.givwenzen.integration.flexunit3.GivWenZenSuite;

public class Stories extends GivWenZenSuite
{
    [Embed(source="/stories/login_in_the_system.txt", mimeType="application/octet-stream")]
    public static var login_in_the_system:Class;

    [Embed(source="/stories/prevent_login_with_credentials_missing.txt", mimeType="application/octet-stream")]
    public static var prevent_login_when_credentials_missing:Class;

    public function Stories()
    {
        super();
        addTestScriptFromResource(prevent_login_when_credentials_missing);
        addTestScriptFromResource(login_in_the_system);
    }
}
}

Now when I right click on the Stories suite I can run validation.

running tests the first time

All the acceptance criteria failed complaining about missing domain steps. Domain steps are meant to drive the validation process. Each step from the acceptance criteria should have a corresponding function in one of the domain steps. The domain steps work best when organized around certain functionality. I'm going to create a LoginDomainStep that will drive behaviors related to the login.

package steps
{
public class LoginDomainStep
{
    public function LoginDomainStep()
    {
    }
}
}

The domain step does not have to extend anything specific to be recognized, instead it needs to be registered in GivWenZen. To do that I changed the Scenario constructor a bit:

    public function Stories()
    {
        super();
        givWenZen().addSteps(LoginDomainStep);
        super.addTestScriptFromResource(prevent_login_when_credentials_missing);
        super.addTestScriptFromResource(login_in_the_system);
    }

Now I should start adding some functions to the LoginDomainStep class. The mapping from the script to the domain step class is realized using [DomainStep] metadata parameterized with the regexp that matches corresponding step. Any argument that the function may have will be given a regexp group. If you are not familiar with regexp, a group is whatever you put within the parenthesis in the pattern. You can read more about the Flex regexp in the Flex documentation.

The easiest way to start filling a domain step with mappings is to copy the actual step from the script:

package steps
{
import org.uinput.*;

public class LoginDomainStep
{
    public function LoginDomainStep()
    {
    }

    [DomainStep("I entered user name 'kris' and password 'secret'")]
    public function enterUserNameAndPassword(userName:String, password:String):void
    {
    }
}
}

As you can see I want to pass some arguments to the enterUserNameAndPassword() function. So I have create groups in the regexp. The final form is going to be:

    [DomainStep("I entered user name '(.*)' and password '(.*)'")]
    public function enterUserNameAndPassword(userName:String, password:String):void

As you can see I used the most basic pattern to match the arguments within groups: .* which stands for match any number of any characters. In reality that's what you will mostly use.

Now it's time to do something from the step. But before we get there a short excursion.

Typically the preferred layer for the functional or the acceptance testing should be controller layer but since in Flex you often times need to check the integration of the controller layer with the UI or simply want to have E2E testing. This is what I want to achieve.

I'm going to use a u-input library to simulate user interaction on the UI. I added import:

import org.uinput.*;

Then I simulated entering user name and password:

    [DomainStep("I entered user name '(.*)' and password '(.*)'")]
    public function enterUserNameAndPassword(userName:String, password:String):void
    {
        textInput("userName").typeText(userName);
        textInput("password").typeText(password);
    }

U-input provides several query functions that can direct you to specific components and then series of functions that simulate user interactions. In the step I used query for the two hypothetical text inputs named "userName" and "password". I also used typeText() to simulate entering text to these text fields.

When you try validating the criteria you should notice that the step failed for different reason than before: now the step is being recognized but there is something missing about the setup of the u-input.

Let's leave it for now and continue filling in LoginDomainStep:

    [DomainStep("I submit credentials")]
    public function submitLogin():void
    {
        button("submitLogin").click();
    }

As you can see I simulate a mouse click on a submit button to trigger the login.

The last step requires some verification. Due to the asynchronous nature of the login (most likely application will go some external service) I need to wait for the change in the system and don't let the test carry on. GivWenZen uses special version of the flex unit test case to execute acceptance criteria, since that I can use morefluent library to control how and when the test proceeds. In short - you can delay execution of the upcoming steps by using the morefluent facilities.

That's how I'm going to verify the change in the application after successful login. My assumption is that there will be a state defined on the main app that I will switch to display a welcome screen.

Firstly I added the import for the morefluent api:

import org.morefluent.integrations.flexunit3.*;

Then I coded the last step for this acceptance criteria:

    [DomainStep("The welcome screen is shown")]
    public function verifyWelcomeScreenIsShown():void
    {
        var app:SampleApp = container(":SampleApp").component;
        poll().assert(app, "currentState").equals("welcomeScreen");
    }

The container() query by type allowed me to get access to the application object. Then I'm using polling assertion to determine the current state of the application.

I want to stop for a while and give you some idea of what the polling is. In most cases you can wait for an event and then proceed with verification. However from time to time it's much simpler to poll from time to time and check if the asserted property has changed to the expected value.

I now try running the tests.

All of the steps now complain about wrong setup of the u-input library. It is high time to perform right setup. GivWenZen offers an entry point that allows taking extra steps before actual step execution. We need to make sure three things happen:

  1. proper initialization of the application
  2. proper initialization of u-input library
  3. proper initialization of morefluent library

You may ask why you need to worry about initializing the application. But please note that when you run tests it's not the main application that is executed. Instead there is a separate application compiled and executed that allows running all of the tests. So for the purpose of the tests I have to create and add the actual application to the stage.

I'm going to implement a BeforeTest interface from GivWenZen and add all three steps to it:

package stories
{
import flash.events.Event;

import flexunit.framework.TestCase;

import mx.core.Application;
import mx.events.FlexEvent;
import mx.managers.ISystemManager;

import org.givwenzen.integration.flexunit3.BeforeTest;
import org.morefluent.integrations.flexunit3.integrate;
import org.uinput.install;

public class ApplicationStartup implements BeforeTest
{
    public function ApplicationStartup()
    {
    }

    public function run(testCase:TestCase, testExecutionFunction:Function):void
    {
        // needed by morefluent
        integrate(testCase);

        if (!applicationInitialized())
        {
            var sampleApp:SampleApp = new SampleApp();
            // make sure the test is not executed before the components are ready
            sampleApp.addEventListener(FlexEvent.CREATION_COMPLETE, function (e:Event):void
            {
                testExecutionFunction();
            });
            // adds the application
            installComponent(sampleApp, testExecutionFunction);
            // needed by u-input
            install(sampleApp);
        }
        else
        {
            testExecutionFunction();
        }
    }

    private function applicationInitialized():Boolean
    {
        var systemManager:ISystemManager = Application(Application.application).systemManager;
        return (systemManager.getChildAt(0) is SampleApp);
    }

    private function installComponent(sampleApp:SampleApp, testExecutionFunction:Function):void
    {
        var application:Application = Application(Application.application);
        var systemManager:ISystemManager = application.systemManager;
        systemManager.removeChild(application);
        systemManager.addChild(sampleApp);
    }
}
}

I'm not going to spend too much time explaining it - in most cases you should be able to reuse above class for your purpose w/o understanding. In short I call the integration functions of the u-input and morefluent and ensure that the tested application is put on stage instead of the flex unit runner UI. Also note that the application is created only once - it means that subsequent executed scenarios may interfere with each other and you have to take care of it.

The last thing to do is to change the Stories suite and make sure ApplicationStartup is used.

package stories
{
import org.givwenzen.givWenZen;
import org.givwenzen.integration.flexunit3.BeforeTest;
import org.givwenzen.integration.flexunit3.GivWenZenSuite;

import steps.LoginDomainStep;

public class Stories extends GivWenZenSuite
{
    [Embed(source="/stories/login_in_the_system.txt", mimeType="application/octet-stream")]
    public static var login_in_the_system:Class;

    [Embed(source="/stories/prevent_login_with_credentials_missing.txt", mimeType="application/octet-stream")]
    public static var prevent_login_when_credentials_missing:Class;

    public function Stories()
    {
        super();
        givWenZen().addSteps(LoginDomainStep);
        super.addTestScriptFromResource(prevent_login_when_credentials_missing);
        super.addTestScriptFromResource(login_in_the_system);
    }

    override protected function provideBeforeTest():BeforeTest
    {
        return new ApplicationStartup();
    }
}
}

Ready. I run the tests and got different errors this time. Now the messages indicate there are no components of the specified name.

Now when you have a properly failing test it's high time to implement some real code.

I'm going to use a model presenter for the login.

package org.acme.model
{
public class LoginPM
{
    private var _username:String;
    private var _password:String;

    public function LoginPM()
    {
    }

    public function doLogin():void
    {
        
    }

    [Bindable]
    public function get username():String
    {
        return _username;
    }

    public function set username(value:String):void
    {
        _username = value;
    }

    [Bindable]
    public function get password():String
    {
        return _password;
    }

    public function set password(value:String):void
    {
        _password = value;
    }
}
}

Then some UI:

<?xml version="1.0"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Script><![CDATA[
        import org.acme.model.LoginPM;

        [Bindable]
        public var model:LoginPM = new LoginPM();
        ]]></mx:Script>

	
    <mx:Form id="loginForm">
        <mx:FormItem label="Login">
            <mx:TextInput id="userName" />
        </mx:FormItem>
        <mx:FormItem label="Password">
            <mx:TextInput id="password" displayAsPassword="true" />
        </mx:FormItem>
        <mx:Button label="Login" name="submitLogin" />
    </mx:Form>
</mx:Application>

Finally I got some green tests. Now the time to implement some of the login logic. I'm not going to call any real service instead I'll code something really simple in the LoginPM.

    public function doLogin():void
    {
        if (username != "kris" || password != "secret")
        {
            applicationState = null;
            throw new Error("Login error");
        }
        applicationState = "welcomeScreen";
    }

    [Bindable]
    public function get applicationState():String
    {
        return _applicationState;
    }

    public function set applicationState(value:String):void
    {
        _applicationState = value;
    }

    private var _applicationState:String;

Also I need to update view with some data binding, event handling and states:

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" currentState="{model.applicationState}">

    <mx:Binding source="userName.text" destination="model.username" />
    <mx:Binding source="password.text" destination="model.password" />

Next I need to call the doLogin() when the button gets clicked:

<mx:Button label="Login" name="submitLogin" click="model.doLogin()"/>

And the last thing is to define states:

    <mx:states>
        <mx:State name="welcomeScreen">
            <mx:RemoveChild target="{loginForm}" />
            <mx:AddChild>
                <mx:Panel title="Welcome!" />
            </mx:AddChild>
        </mx:State>
    </mx:states>

Done! The login_in_the_system criteria passed.

I'll leave implementation of the remaining criteria as an exercise. In case you have issues with it you can check out the zip file with the complete project.

Updated