Advanced Testing Techniques with CATJS

In my previous post I introduced CATJS, an open source automation framework for mobile web testing. This time I’d like to get deeper into an advanced testing technique of using this tool.

The Developer’s Testing Dilemma

As all of us (developers) probably know testing your application is an exhausting and never ending task. We all know how important it is, however we all try to avoid it. There are many reasons for this, and believe me, laziness is not at the top of the list. We all do unit testing but prefer not to do GUI testing. As a developer, the main reason to avoid testing for me is the test code maintainability. GUI and functional testing tools observe the application at the UI level. That means they identify the object by its properties as reflected in the HTML DOM. When I write a code usually I use frameworks.  For example, I create an object by calling

var mybutton = new Button("OK");

Everywhere in my code I identify this object as mybutton, however the testing tools will identify the same button as “OK”, probably by using its text and if this is not enough then by its other properties (e.g. width height, type etc.). I’m not saying this is wrong, it is probably the best way to identify the GUI object externally, but for me as a developer doing the matching all the time and after every change I make in my code is a hassle. So I always have the same dilemma: to test, or not to test. Luckily, I have found another way. A testing tool that allows me to write my GUI tests in same way I write my unit tests, from my code and by using objects’ names that I know. This tool called CATJS.

Using Data Driven Test in CATJS

In my previous post I showed how to create a scrap (a test) that logs into my example app, iReminder. To cover all the variations of username and password I created 4 different tests. Going back to the maintainability issue, for every change that I make in my login page I need to go through all the 4 tests and apply the change there too, but this time all 4 tests share the same functionality and the only difference is in the data that these tests are filling in. Luckily for me I can simplify this task by using CATJS‘s data driven capabilities.

Firstly we need to tell CATJS to run the same test multiple times. In the test configuration file catproject.json located inside the cat-project folder the test object can get a repeat property  (line 12 in the code below) in this case I set it to 5.

{
  "_comment": {"description":"This is the cat test project"},
  "name": "reminder",
  "ip": null,
  "port": null,
  "tests": [{
    "name": "general"
  }],
  "scenarios": {
    "general": {
      "tests": [{
        "name": "test_one" , "repeat": 5
      }, {
        "name": "test_two"
      }, {
        "name": "test_three"
      }, {
        "name": "test_four"
      }]
    }
  },
  "ui": true,
  "report": {
    "format": ["console", "junit"],
    "disable": false
  },
  "assert": {
    "errors": true
  },
  "run-mode": "tests",
  "test-failure-timeout": 30
}

Now we need a place for our data, No, we don’t need to implement the retrieval functionality, CATJS makes this task easier by supplying an internal functionality for reading an external datafile. Inside the folder /cat-project/src/config we can find a JSON file called testdata.json. Any object in this file is accessible within CATJS scrap. As you can see below, I created an array of objects called “users”, each user object contains 4 fields: id, username, password and message. I populated the users array with 5 different login values.

{
    "_comment": "Test one data to verify all variations of invalid credentials",
    "users" : [
      {"id":1 , "username":"","password":"" , "message":"Missing username."},
      {"id":2 , "username":"jodoe","password":"","message":"invalid username or wrong password."},
      {"id":3 , "username":"jodoe","password":"mypass","message":"invalid username or wrong password."}
      {"id":4 , "username":"admin","password":"","message":"invalid username or wrong password."}
      {"id":5 , "username":"","password":"iamadmin","message":"Missing username."}
    ]
}

Finally, Let’s look at the scrap that reads this data. CATJS has an integration with JSPath, this means that we can use the JSPath syntax to query our datafile. CATJS‘s scrap language supports two basic query functions: @d.find(JSPath query string) and @d.random(JSPath query string). You can read more about it in the CATJS git site. In our case we need something else, we need a way to retrieve different data per each run. This time we will use CATJS’s hidden capability. Within the scrap‘s function @code we are able to access a variable that was previously defined in my javascript code. In our case, as you can see below, I define two variables, testuser and t_indx (lines 3-4). Later in my code, I assign testuser value with user data that is retrieved by calling to the  _cat.utils.TestsDB.find(“.users”)[t_indx++] function (line 18). Every time the scrap runs, the t_indx value increases by one and testuser gets the next object in the users array.

 
<script>
var testuser= "";
var t_indx=0;
</script>
	<!--
		@[scrap
		@@name catImport
		@@import[
		   /cat/lib/cat.js
		]
		]@
-->
<!--
@[scrap
	@@name  test_one
	@@embed true
	@@code	testuser= _cat.utils.TestsDB.find(".users")[t_indx++]
	@@jqm   setText("#username", testuser.username);
	@@jqm   setText("#password", testuser.password);
	@@jqm   clickButton("#loginbtn")
	@@assert ok($('#message').text()==testuser.message,"test one failed")
]@
-->

Now let’s run our test (catcli -cbs). As you can see, test_one runs 5 times. Each time the test uses different login values.

Conclusion

CATJS‘s data driven capability helps you keep your test short and simple, and eases your test code maintainability.

To summarize I can say: by using CATJS I have no dilemma. I choose to TEST.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s