How I’m NOT going to delete my Windows profile folder again?

So I deleted the complete profile folder under C:\Documents and Settings\<my user id> on my work machine. Everything under that was gone, except for a few read-only folders that survived. My Documents, Application Data, Desktop, Local Settings, Favorites…gone. My Outlook archives, my OneNote notes and a few years worth of accumulated documents…all done for. Plus all my Java projects, since I configured git under that folder and all my git projects were there.
How did I do that? Well…I was creating a new Java project using Eclipse to demo the use of excel-testng. But since I didn’t want to use the default workspace location, I specified the location where I keep the git projects, which happens to be the profile folder. Usually when I use the default location, eclipse creates a sub-folder under the default location with the same name as the project as below:

But when I didn’t use the default location, it didn’t create sub-folder with the project name. It created the project right under that location!

And having not realized that and since I wanted to remove the project (can’t remember why), I proceeded to delete the project.

Now, to be fair to myself and going back to that moment just before I clicked OK, I have never ever before needed to not delete the project contents from disk. The whole idea of deleting it from eclipse workspace has always been to get rid of the project, completely. And that’s what I did. At that time, I found it a little surprising that it was taking a little longer that usual. But it was 5pm on Friday evening and things usually get a little strange around that time, so I was ready for anything. Just not ready for what happened next.
It probably took me less than a few seconds to realize what had happened since I had an explorer window open on that folder, and I saw blank white screen where I had my profile folder contents previously. And I’m not going to mention the words that came out of my mouth next, or the next few hours I spent trying to restore the contents unsuccessfully and recreate the git projects.
There’s a happy ending to this: I had got my laptop replaced just a week ago and had a backup of most of that folder. I’ll most probably never find out if there was anything I was NOT able to recover, and that’s a good thing!

excel-testng: driving TestNG tests through MS Excel

Over last few days, I have been working on a small project in my spare time. It's called excel-testng and it provides a way to drive TestNG tests through MS Excel. The code repository is here: http://code.google.com/p/excel-testng/ and the jars can be downloaded from here. I also put together a small automation project using Selenium WebDriver to demonstrate its use here: https://github.com/randomsync/excel-testng-demo.

Introduction

During functional testing projects, after we create tests for a certain application, we need to review them with the rest of the team (developers, analysts, project managers etc). We may use a test management tool to document the test cases and then export them into an easily distributable format like MS Excel or PDF. Sometimes, we may also create the tests in Excel directly specifying the test name, description, parameters and other data in the spreadsheets. And then finally, we may automate a part (or all) of the tests.
So after the tests are automated and when executing the tests, we need to specify which tests to run and the test data (parameters). In Quality Center, it involves creating a test set (kind of like a test suite) and then adding tests to it. If they are automated in QTP which integrates with QC, they can be executed by running the test set. But we have started using Selenium WebDriver for its cross browser capabilities and that means we need to specify the tests in a format that can drive the Selenium tests. We use TestNG as the framework for test execution, assertions and reporting.

Extending TestNG

TestNG is a great framework for test execution and its input can be in form of an XML file that specifies which tests to run, where to find the test classes/methods, test parameters and a whole lot of other features that gives you a fine grained control over each test execution. However for our UI functional tests, we wanted to be able to specify the tests and test executions in an easily distributable format (like MS Excel) as mentioned above. The good thing is that TestNG provides the capability to extend it so that its input can be created and executed programmatically. So this way, the test specification in Excel files can be parsed and driven through TestNG. And this is exactly what excel-testng does. It externalizes all Excel parsing and TestNG XmlSuite creation so that the focus can be in creating the test classes and methods. Once that is done and Excel specifications created, all that needs to be done is to provide a main method that does this:

   1: ExcelTestNGRunner runner = new ExcelTestNGRunner("input.xls"); // this can be a single file

2: // or a directory, in which case all spreadsheets in that directory are parsed

3: runner.run(); // run the tests

With ExcelTestNGRunner class, you just specify the input location of the Excel file(s) that constitute the test specifications and then call the run() method, which parses the excel file using default parser (an instance of ExcelSuiteParser) into XmlSuites, creates a TestNG object if not already created and then runs them. After that TestNG takes care of test execution and reporting.

Excel Test Specification

ExcelTestNGRunner parses each worksheet in the Excel file(s) into a separate suite using the included parser (ExcelSuiteParser). Each suite can have suite level parameters specified in the worksheet and the test specifications that specify which tests will be run, their name, description, parameters and the test classes. Here's what a test suite in Excel looks like (a demo spreadsheet can also be downloaded from here and can be used as a starting point):

Specifying TestNG tests in Excel File

The top few rows in the worksheet provide the suite information. ExcelSuiteParser looks for the string "Suite Name" and retrieves the name of the suite from next cell in the same row. Similarly, it looks for string "Suite Parameters" and retrieves suite parameters from the next cell. "Suite Configuration" is not currently used. You can customize the location of these values by providing your own map to the parser. See "Customizing Input" for more details.

To retrieve the tests that will be executed, it looks for the 1st row containing "Id" in the 1st column. This will be the header row, below which each row is a separate test. The header row and tests must have columns specifying Id, Test Name, Test Parameters and Test Configuration (which specifies the classes containing the Test methods). If "Id" is left blank, the test will not be added to the suite. Finally, it parses each row under the header row into a TestNG XmlTest and adds it to the suite. The test specifications are provided as:

• Test Name: generated by concatenating Id & Test Name
• Test Parameters: retrieved from "Test Parameters" column and need to be provided in valid properties (<key>=<value> etc.) format. You can also specify functions as parameter values and then add the logic to parse and evaluate the functions in your test classes (maybe a Base Test class)
• Test Classes: specified under "Test Configuration" column as classes property. Currently, you can only specify a single test class of which, all @Test annotated methods will be executed as a part of this test execution. I'm going to add the ability to select the test methods in later releases.

Customizing Input

If your test cases are specified in Excel but in different format, there are 2 levels of customizations you can do with ExcelTestNGRunner on how to parse the input spreadsheet(s):

1. Custom Parser Map (currently not implemented): You can use the in-built parser but specify your own parser map, which tells the parser where it can find the suite and test data
2. Custom Parser: You can create your own parser by implementing IExcelFileParser interface. You need to parse the spreadsheet file and return a list of TestNG XmlSuites.

ExcelTestNGRunner also provides helper methods to customize the TestNG object it uses to execute tests. For example, you can specify any custom listeners using addTestNGListener() method. If you need to have more control, you can create your own TestNG object and then pass it to ExcelTestNGRunner. Please see javadocs for more details.

Putting it together

You can see the project at https://github.com/randomsync/excel-testng-demo for a complete working demo of excel-testng to parse the input test specifications in Excel. It uses Selenium WebDriver to automate the testing of basic Google search functionality.

Update (3/12/2012): I'm now using google code to host this project because of the provision to host the downloadable jars, javadocs and wiki easily. I'll keep it synced with github but you can find the project documentation and downloads there.