Random Sync

Puzzler Solution: Prisoners and Hats and a Jungle

2011-11-23T13:56:00.001-08:00

Can’t believe it’s been 2 months since I posted the puzzle. I’ve been involved in test automation using Selenium WebDriver over last couple of months and it has been a great experience designing and coding the test suite. The 1st step is complete - smoke tests for 2 of our applications have been automated and can be run on different browsers concurrently. Now we’re moving to the next steps (which are still TBD). I have lots of ideas in my head but I have to evaluate their feasibility and to break them into different phases so we can realize the added value as we continue to improve the automation suite.

Back to the puzzler solution. For reference, here is the puzzle statement and the code. My implementation of the guessStrategy method is below.

The strategy is that the 1st person to guess speaks the color of the odd hats ahead. So if the (number of black hats ahead) % 2 == 0, the guess is “Black” (lines 4-8). How does it help that person? It doesn’t, that prisoner is unlucky enough to be the 1st one to guess (end of the line) and has a 50% chance. But the rest of the prisoners can guess the color of their hat correctly now that they know the initial guess, all guesses since then and the color of hats ahead. So for example, if the 1st prisoner guesses black, there must be odd black hats ahead. Now if the 2nd prisoner sees odd black hats ahead, s/he must be wearing a white hat because the previous prisoner also saw odd black hats. And the logic goes on like that…if the nth prisoner sees odd number of black hats, and previously, odd number of people have guessed black, then s/he must be wearing a white hat (lines 13-23).

public static char guessStrategy(String prevGuesses, String remArr) {   2:     int numB = getNumOfChars(remArr, 'B');
    int numPrevB;
    if (prevGuesses.length() == 0) { // initial guess
        if (numB % 2 == 0)
            return 'W'; // if B is even, return W
        else
            return 'B'; // else return B
    } else {
        // strategy: if previously even number of people have said black,
        // and I see even black hats, I have a white hat, else black
        numPrevB = getNumOfChars(prevGuesses, 'B');
        if (numPrevB % 2 == 0) {
            if (numB % 2 == 0)
                return 'W';
            else
                return 'B';
        } else {
            if (numB % 2 == 0)
                return 'B';
            else
                return 'W';
        }
    }
}
  
/**
 * @param arr
 * @param ch
 * @return number of times the specified char appears in the array
 */
public static int getNumOfChars(String arr, char ch) {
    int num = 0;
    for (int i = 0; i < arr.length(); i++) {
        if (arr.charAt(i) == ch)
            num++;
    }
    return num;
}

Dennis Ritchie

2011-10-13T23:25:00.000-07:00

http://www.nytimes.com/2011/10/14/technology/dennis-ritchie-programming-trailblazer-dies-at-70.html

No doubt, comparisons will be made with Steve Jobs and how much media buzz was created.

iPhone has had a huge impact in my life, but not as much as C. C has defined my thinking. During college, most of the programming I did was in C and K&R was THE reference book I had with me most of the time (I still have it). The precise and succinct way the book is written helped me appreciate the beauty of coding. Every function was defined in clear and concise terms. That not only guided me in my programming adventures, but somehow it also helped me put a different perspective around life and accumulate a no-nonsense attitude of holding myself to much higher standards than anyone else.

From the article linked above:

Colleagues who worked with Mr. Ritchie were struck by his code — meticulous, clean and concise. His writing, according to Mr. Kernighan, was similar. “There was a remarkable precision to his writing,” Mr. Kernighan said, “no extra words, elegant and spare, much like his code.”

In essence, this guy created something wonderful and powerful in the world of computing. And I hope his legacy lives on.

-Gaurav Gupta

Puzzler: Prisoners and Hats and a Jungle, Oh My!

2011-09-29T15:38:00.000-07:00

This puzzler was mentioned in Car Talk sometime ago and I really liked it. In brief, it goes like this:

A prison has 30 prisoners sentenced to be executed and the warden, who has the authority to pardon, decides to give them a chance to escape the punishment. He will stand all the prisoners in a straight line with each prisoner able to see the heads of all prisoners in front of him but not of those behind him. Next, he will put either a white or black hat on each prisoner’s head and ask them to guess the color of their hat one by one (starting with the 1st prisoner in the back of the line who can see all 29 other prisoners’ heads). If he guesses correctly, he’s pardoned. What is the strategy that the prisoners can use to maximize their chances of being pardoned?

The answer I came up with was grossly wrong so when these guys gave the answer (link to which I’m not posting here but can be found easily), I was intrigued. When listening to the answer, it sounded very simple but when I actually thought about it some more, I had to listen to it again to understand the strategy. For example, does the current prisoner need to know all the previous guesses or only the most previous guess?

I decided to write a simple program for this. The line of prisoners here is a StringBuffer of predefined size (in this case, 30) that is randomly filled with ‘B’ or ‘W’ chars. The objective is to write a method that will be called for each character in the StringBuffer with all the previous guesses (as a String) and remaining series (as a String). The method has to return the current character and should be implemented in such a way as to maximize the correct answers. Here’s the code:

import java.util.Random;   2: public class Test {
    static int SIZE = 30;
   
    public static void main(String[] args){
        int correctGuesses = 0;
        StringBuffer pRow = new StringBuffer(SIZE);
        StringBuffer prevGuesses = new StringBuffer(SIZE-1);
        char currGuess;
        
        Random rnd = new Random(System.currentTimeMillis());
        //---print the series
        System.out.print("Series:\t");
        for (int i=0;i<SIZE;i++){
            if (rnd.nextInt(2) == 0)    //0=black, 1=white
                pRow.append('B');
            else pRow.append('W');
            System.out.print(pRow.charAt(i) + " ");    
        }
        //---strategy
        System.out.print("\nStrat:\t");
        
        for (int i=0;i<SIZE;i++){
            currGuess = guessStrategy(prevGuesses.toString(),pRow.substring(i+1));
            System.out.print(currGuess + " ");
            if(currGuess == pRow.charAt(i)) correctGuesses++;
            prevGuesses.append(currGuess);
        }
        System.out.print("\nCorrect Guesses=" + correctGuesses);
    }
  
    public static char guessStrategy(String prevGuesses, String remArr){
        //random
        Random rnd = new Random(System.currentTimeMillis());
        if (rnd.nextInt(2) == 0)    //0=black, 1=white
            return 'B';
        else return 'W';
    }
}

First, I generate the series and print it (lines 13-19). And then I call the guessStrategy method for each character in the series with all the previous guesses and the remaining series. But currently, there is no strategy and each time, it randomly returns ‘B’ or ‘W’. And obviously, result is that correct guesses average around 15. Your job, should you decide to accept this assignment is to provide a better implementation of the guessStrategy method which maximizes the chances of guessing correctly. If you’re stuck, you can look at the Car Talk website or search on internet to find the answer and then try to implement it.

I’ll post my implementation of the method in a few days. Hopefully it provides enough of a challenge to some people to work on this besides their otherwise busy life.

Friday Lunch: Content-Type, Content-Disposition HTTP Headers and MIME type handling in IE

2011-07-01T20:20:00.001-07:00

We have a scripting tool that is used to create web-based scripts that simulate application scenarios. These scripts are run through a monitoring tool on a continuous basis to assess the health of these applications. The scripts & scripting tool are also used to validate the scenarios if the monitoring tool issues an alert. The scripting tool uses IE to run the HTTP request/response scenarios. When recording the script for a recent new application, it was noticed that after submitting the request which was an HTTP POST with XML data, the tool was bringing up a dialog box to save the response file, something you’d get when clicking on a HTML link in the browser to a file (maybe a word .doc etc) and if you don’t have an application to open that kind of file. We checked the response and it contained a standard XML that should usually be displayed within the browser (like if you click on Amazon’s AWS service). The fact that it was bringing up a download prompt and not displaying the XML in the browser was a problem because the non-technical team that uses it to validate the application will not be able to determine whether the script ran successfully or not.

The application is a servlet that takes the initial POST request and returns an XML response. After further investigation with the tool and looking at the response headers, it was found that the application was returning “application/x-www-form-urlencoded” in “Content-Type” header erroneously. I haven’t found something specific that would say that content type is not correct for HTTP response, but it is more common to use this content type when submitting form-submission data that is URL encoded (see here).

Investigating with WebScarab

My first conjecture was that the file download prompt was being displayed because of this Content-type header’s value. If we could intercept the response and replace its value with something like “text/xml”, the tool or browser will be able to display the content within the application instead of bringing up the file download prompt. And to validate that, I used WebScarab to intercept the request/responses. But for some reason (maybe because of invalid Content-Type header), it didn’t intercept the response. It would intercept the request and after accepting, it would just complete the request and we’ll end up seeing the file download prompt in the tool. Clearing the “Only MIME-Types matching:” didn’t help.

So the workaround was to write a BeanShell script within WebScarab to replace the “Content-Type” header with a proper value instead of manually intercepting the response. That turned out to be quite straightforward. All we needed to do was put:

response.setHeader("Content-Type","text/xml");

in WebScarab’s Bean Shell interface under the fetchResponse method. As is obvious, this replaces the value of Content-Type header in the response to “text/xml”.

Once we tried it out, it worked and the browser displayed the XML response without bringing up the file download prompt. This proved that the Content-Type header was the problem. But what exactly was the problem? What makes the browser display a prompt to the user to download a file instead of just displaying the content? Further searching revealed this article: http://www.jtricks.com/bits/content_disposition.html

So basically the Content-Disposition header along with Content-Type specifies the MIME type of the file returned and whether the browser should display it or prompt the user to download it. The RFC has all the details but the gist is that the Content-Disposition header can be set to “attachment” for the browser to display the file download prompt. But the response we were getting didn’t have a Content-Disposition header. And IE was still displaying the file download prompt. Further reading warranted.

How does IE handle MIME types in a response, anyways?

I kept on searching and reading on how and when a browser brings up the file dialog prompt. The Content-Type header still bothered me. Then I came across this MSDN article: MIME Type Detection in IE. It provides details on what steps IE takes to detect the content-type of the content before deciding what to do with the file:

The purpose of MIME type detection, or data sniffing, is to determine the MIME type (also known as content type or media type) of downloaded content using information from the following four sources:
The server-supplied MIME type, if available
An examination of the actual contents associated with a downloaded URL
The file name associated with the downloaded content (assumed to be derived from the associated URL)
Registry settings (file name extension/MIME type associations or registered applications) in effect during the download

If you look at the known MIME types for FindMimeFromData, “application/x-www-form-urlencoded” is not one of them. It is also not an ambiguous MIME type ("text/plain," "application/octet-stream," an empty string, or null) either. So with the MIME type as unknown,

If the "suggested" (server-provided) MIME type is unknown (not known and not ambiguous), FindMimeFromData immediately returns this MIME type as the final determination.

And what happens after that is in this MSDN article: Handling MIME Types in IE. It doesn’t mention exactly what happens if the MIME type is unknown, there’s no application associated with it in the registry and there’s no Content-Disposition header. But I’m guessing that the only thing for it left to do is to let the user decide by showing the file download prompt as was happening in our case. And we were able to prove that. I looked up the registry key for “text/xml” under HKEY_CLASSES_ROOT\MIME\Database\Content Type and added it to a new key “application/x-www-form-urlencoded”. The new registry key looked like this:

[HKEY_CLASSES_ROOT\MIME\Database\Content Type\application/x-www-form-urlencoded]
"CLSID"="{48123BC4-99D9-11D1-A6B3-00C04FD91555}"
"Extension"=".xml"
"Encoding"=hex:08,00,00,00

This tells IE that for responses received with Content-Type equal to “application/x-www-form-urlencoded”, use extension “.xml” and open it however XML files are opened (which is to display them inline). Once added, we went back to the tool and ran the script. And this time, it displayed the response xml without bringing up the download prompt.

So through this exercise, I learned about how IE displays different MIME types and about Content-Type and Content-Disposition headers.

GLaDOS is back…

2011-06-06T14:29:00.001-07:00

So it's ok if she tries to incinerate me alive for sake of “science” but when I try to escape and torch her in the process, she gets all psyched up and starts singing sarcasm laden song?

C++ turns 25

2010-10-14T15:44:00.001-07:00

Just wanted to share a quick link - Bjarne Stroustrup’s reflections on the 25th anniversary of C++’s first release: http://www.wired.com/thisdayintech/2010/10/1014cplusplus-released/all/1

A Generic Function to enhance BPT components

2010-10-08T08:36:00.001-07:00

About 2 years ago, we created a set of QTP function libraries to drive the automation effort using Quality Center's Business Process Testing (BPT) functionality. What those libraries provide are a set of generic functions from clicking on a UI object to verifying an object's property, which then can be used to create both generic BPT components that can be used on any application and specific components that target a particular application or a screen within an application. We have used those libraries and components to successfully automate our application testing without much modification and maintenance.

Recently, we were using the same set of components to create/update the test cases for a new application and I took this opportunity to enhance the libraries by adding a few functions. One of them is a generic Eval function that provides some more flexibility in creating components. As we know, VBScript has its own Eval function that evaluates the provided expression and returns the result. So to be able to use that functionality from within the components, I created a new function that returns the result of that expression or the string itself if it is not an expression. Here's the function listing:

' Function EvalFunction
' ------------------
' Evals the function call specified in the parameter and returns the result
' Parameter: expr - Any VBScript expression
'@Description Evals the function call specified in the parameter and returns the result
'@Documentation Evals the function call specified in the parameter and returns the result
Public Function EvalFunction(ByVal expr)
  Dim res
  On Error Resume Next
  If (expr <> "") Then
    res = Eval(expr)
    If (Err <> 0)Then
      res = -1
      Err.Clear
    Elseif res = "" Then
      res = expr
    End If
  End If
  EvalFunction = res
  On Error Goto 0
End Function

The function itself is quite simple but it provides a lot of added functionality. I can use this function in my BPT components to evaluate any expression during run-time. And in addition, I also create a generic component that calls this function and returns the result. When I need to create a test case that needs a run-time value, I can use this component within my test cases to have that functionality.

Using the EvalFunction function in components:

For a part of our application's functionality, we had a component ("AddIP") that adds an IP to a user. Since there were 4 text fields for each of the octets, we had 4 component input parameters for each octet. In the picture below of AddIP component, each octet is a different text field (“IP1”…”IP4”) requiring a different input parameter (BrowserWebEditSet is another generic function to enter value in a WebEdit object)

But while revisiting the testing, I felt that its much easier to provide the IP parameter as a whole when you're creating a large test case where multiple IPs need to be added. So instead of having to create a new function, I just used the EvalFunction to create another component ("AddIPv2") that takes the full IP address and splits it into each octet and then enters the values as required in different text boxes. I use the VBScript “split” function to split the input parameter (the IP address) into each octet like this: "Split(""" & Parameter("IP") & """,""."",-1,vbTextCompare)(0)" which returns the 1st octet. As in the picture of the component below, I call the function 4 times for each octet which returns the result in a local parameter which I can then use to set for each text field.

So using this enhanced component, the user has to provide only the whole IP address “xxx.xxx.xxx.xxx” instead of each octet in a separate parameter.

Using the EvalFunction component in BPT tests:

The EvalFunction component is a one operation component that calls the EvalFunction and returns the result in a component output parameter. This component can be used in any test to evaluate any expression and use the result in a subsequent component.

For example, a lot of our test cases require creating a new user. Instead of specifying a fixed value for the user ID or having the tester enter a value in a run-time parameter before every run of the test, the user ID can be generated using a timestamp which guarantees its uniqueness. This is done by using the EvalFunction component. In the BPT test case below, the EvalFunction component is used with input parameter a string prefix concatenated with the current timestamp (DateFormatter is another function that returns the current date/time in appropriate format). It returns the result in a component parameter “result” which is then later used in AddUser component to create a user with that user ID.

This way, this test can be run any number of times without requiring any modification.

Epilogue:

The function that I showed in this post is a part of the automation libraries that we created. Over time, as we use them for testing multiple applications, we have enhanced the libraries even more. These libraries, along with the BPT functionality provided by Quality Center, have helped us create reusable components that require very less maintenance. And these components are being used by testers to create and run automated test cases for different applications and different functionalities.

HP Quality Center (ALM 11.0) and REST

2010-09-29T10:09:00.001-07:00

We are currently using Quality Center 9.2 and one of my goals for this year is to upgrade it to the latest version. While browsing through the What’s New documentation for version 11.0 (seems like it has been rebranded to Application Lifecycle Management, or ALM), I saw this line:

There are now ALM REST resources available. For details, see the HP ALM REST API Reference.

That to me is something that stood out from all the other features. Of course, from 9.2 there have been other great features added as well, like version control and flows in version 10 and Sprinter and others in 11. But being able to write clients using the REST API is a great feature and something that I definitely will be exploring once we have this version installed. I went through their REST API reference and as expected, it exposes all the entities in a RESTful way that would make it easier to write the clients in Java or any other language.

Related: a previous REST-related post

Intercepting SSL traffic using WebScarab

2010-07-02T22:05:00.000-07:00

The last time I wrote about intercepting web requests using WebScarab, I was successful in intercepting SSL traffic generated through a custom Java client. Even though the process to do that was quite tedious – involving exporting the WebScarab server certificates into .cer format, importing the certificate into a Java keystore and then running WebScarab as a reverse proxy – I was able to intercept and view the SSL traffic that was being generated. But there was an inherent issue with that process that I overlooked.

When a proxy is setup to intercept SSL traffic, the security issue is that the SSL certificate that is presented by the proxy is not signed by a trusted authority. Web browsers detect this and give the user an option to accept or not accept this risk. So there is no problem in using the proxy to intercept web traffic to secure sites and we can just point the browser to the proxy and accept when warned about certificate error. But in case of Java clients using JSSE, there is no assumption of an interactive user session and so by default it throws an exception if there are any certificate related issues – be it an unknown certificate in which case it throws the exception:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

or a hostname mismatch:

javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching … found

The latter is thrown because when a new HTTPS connection is created using HttpsURLConnection class, it implements a default HostnameVerifier interface which checks if the host we’re trying to connect to matches the name in the certificate in its certificate store (specifically the cn within the certificate). If it doesn’t, it throws the above exception. The client I was using earlier overrode the default HostnameVerifier with a custom one, which ignored the hostname mismatch. But this time with a new client for a different application, it didn’t and I had to go one extra step to intercept the requests, which is detailed below. So first:

Start WebScarab and run it as a reverse proxy on port 443. This is so that WebScarab behaves as a secure server to the client, even if with a self-signed certificate instead of one signed by a trusted authority. (If running WebScarab from the same machine that is generating the requests, we should also select “Intercept requests” check box. This is important because in that case, the proxy is an infinite loop to its own interface and so we want to be able to break the flow and Abort after the first intercept)
Modify the hosts file to point WebScarab hostname to the IP of the machine where it is running. In case of local, it should be:
127.0.0.1 WebScarab
This is specifically so we can get around the issue of hostname mismatch because we’ll try to connect to host “WebScarab” instead of actual target server. If the client overrides the default HostnameVerifier to ignore those errors, it can be setup so that the client points to the actual host:
<ip where WebScarab is running> <target hostname>
Use the java program available here to create a keystore with the WebScarab certificate
>>java InstallCert WebScarab
Since WebScarab hostname is pointing to the WebScarab proxy, this program will connect to it and retrieve its certificate. It will create a keystore file called jssecacerts with WebScarab’s certificate (keystore password is blank by default).
Configure the client to use WebScarab as the host within the URL. So instead of https://<hostname>/<path>, it should be https://WebScarab/<path>.
Run the java client with the truststore and password properties: -Djavax.net.ssl.trustStore=<location to jssecacerts file> -Djavax.net.ssl.trustStorePassword=<password, default blank>

At this point, WebScarab proxy should intercept the request. I can review it, and abort it so it doesn’t repeat. Obviously, the request can’t be sent to the actual destination server. As I’ve noted above and as far as I know, there’s no way to get around the hostname mismatch error unless the default HostnameVerifier is overridden. But in my case, I was fine with just intercepting the request and creating my LoadRunner scripts using the raw HTTP request.

Goodbye multiple putty windows!

2010-05-21T22:27:00.001-07:00

If you use PuTTY for your remote telnet or SSH needs, chances are you have had a situation where you had to open and manage PuTTY windows. I used to do that and it was sometimes confusing. But recently, just out of luck (my laptop had crashed and I was looking for ways to export/import PuTTY connection profiles), I found this: PuTTY Connection Manager. Its a tabbed version of PuTTY client and provides a solution for managing multiple PuTTY instances. I’ve tried and it works great!

Since I’m endorsing some good tools that I use everyday, here’s others:

- Agent Ransack: A file searching utility for Windows, it has great features for searching by regular expression, within files etc. I haven’t used Windows built-in Search since I found this.

- SketchPath: is an XPath tool that you can use to view XMLs and run XPath queries.

- Sysinternals: a collection of utilities to perform various troubleshooting tasks in Windows. It provides a lot of helpful utilities like Process Explorer (an advanced version of Task Manager), tcpview (shows all open connections with their processes) etc.

- WinMD5: an MD5 utility for Windows.

That’s it for now. I’ll update this post if I can remember more tools to recommend.

LoadRunner Scripting Challenge – KodakGallery.com (AJAX, JSON, REST & XML)

2010-05-15T22:26:00.001-07:00

If you are not familiar with KodakGallery.com, it is an online photo publishing and sharing site offered by Kodak. It offers features to store and share your photos online and to print them or order certain photo products, just as competing sites like flickr, Snapfish, Picasa etc. What they don’t offer is an API to interact with your account online (uploading/downloading pictures etc) without having to go through their website. Flickr (the site I use) has a published API that can be used.

Background:

Last year, KodakGallery changed their storage policy so that users would have to make a minimum purchase from the site based on their storage size in order to continue storing pictures. Even though the cost is minimal for the storage they offer, it was still considered worthwhile to explore other options. And the direct impact to me was that I was tasked with downloading all the pictures that have been stored there for last few years. Manual download was out of the question because of the number of pictures that had accumulated over that duration. And scripting it with Perl or LoadRunner (my weapon of choice) was very viable because of my experience with finding myself in these kind of situations and the thrill of being faced with a challenge and learning something new out of it.

Note: If you navigated here looking for an automated way to download all your pictures from KodakGallery.com, I’m planning to create a Perl script to automate that. You can safely ignore the rest of the post and write me a comment, which will provide me with the motivation to stay up extra hours in the night.

Challenge:

So here’s what we needed to do: Build a script that logs in to KodakGallery.com website and downloads all stored pictures to the local disk categorized by their album folders.

If you’re a LoadRunner user who has to use it in your daily life (atleast while at work) and want to give it a shot, please do so before going through the post. You’ll need a KodakGallery account and a few albums with pictures in them. I promise that it’ll be fun and challenging. I also have to state that scripting/automating interactions with a website has to be done with some caution. You may be using the website’s resources in a way they were not intended to be used. Sean Burke, author of Perl & LWP puts it in a very succinct and precise manner here:

…the data on the Web was put there with the assumption (sometimes implicit, sometimes explicit) that it would be looked at directly in a browser. When you write an LWP program that downloads that data, you are working against that assumption. The trick is to do this in as considerate a way as possible.

Scripting:

With that in mind, let’s get to it. Scripting a live website comes with its own unknowns. The fact that you have no idea about the underlying technologies used to transfer the data and present it to the user provides a great opportunity to learn not only about some new technologies but also about the tool that you use because you may have to use the tool (or some of its features) in ways that you’ve never done before.

The first step in LoadRunner scripting of course is to record the user interaction with the website. For the record, I only have availability to LoadRunner 8.0 that I’m using for scripting here. I’ve heard that newer versions have better support for new web technologies that have come up in recent years. But in last few years at my current role, there’s never been a time that I wasn’t able to deliver a script because of using an older version and I have never felt that I’m missing something.

I created a script using my preferred recording options:

Recording Mode: HTML-based script containing explicit URLs only
Not using correlation
Not recording headers
Not excluding content types
Do not record this content types as a resource:
text/html
text/xml
Record non-HTML elements in current HTML function

I put a comment in the script before every action so that when editing the script later on, I know clearly where each action starts and ends. But after recording, the script in this case was not very intuitive. Some of the actions didn’t correspond to my script comments. For example, in the script where I put the comment for login, I didn’t see any web request that would match a login request or any form-based submission with login parameters. Instead, there was a web_url request to “storagestatus.jsp”. When I put a comment for clicking on an album, there were no steps.

So after scanning through the recorded script and the recording log, I realized that the login and other actions were being submitted through JavaScript and the content-type for these requests was non-HTML. My current recording settings specified that the requests that do not have a content-type of “text/html” or “text/xml” are not to be recorded as a resource, and so those were considered a part of current step and included after the EXTRARES parameter or the original request. Here’s the initial request. The login request is included in the original step (I later found that it has the Content-Type=text/javascript) and the authentication itself is handled through submitting the username and password in an HTTP cookie called “ssoCookies”:

web_add_cookie("ssoCookies=%7B%22email%22%3A%22abc%40example.com%22%2C%22password%22%3A%22test%22%7D; DOMAIN=www.kodakgallery.com");
web_url("welcome.jsp",
 "URL=http://www.kodakgallery.com/gallery/welcome.jsp",
 "TargetFrame=",
 "Resource=0",
 "RecContentType=text/html",
 "Referer=",
 "Snapshot=t1.inf",
 "Mode=HTML",
 EXTRARES,
 ... //lots of images
 "Url=https://www.kodakgallery.com/gallery/account/login.jsp?&uid=791701608",
 ENDITEM,
LAST);

So with that information, I saved the current script and recorded another script with different recording options. I used URL-based script that records all the content (including non-HTML, like css, js, gif etc) in a separate web_url function. That meant the script was longer and a little harder to navigate through but atleast I could scan through the script and figure out what requests are being submitted by their corresponding actions.

Having recorded the actions (login, navigate to an album, download a full-resolution image etc), scanning through the script multiple times, using WebScarab (related post) to scan through the HTTP traffic, I found a lot of interesting things:

Login is handled through JavaScript. A lot of JavaScript. They use MooTools JavaScript libraries for a lot of functionality but for login, it creates the cookie “ssoCookies” and sends a request to “login.jsp:” which (after successful authentication) returns some user information (ssId: probably some sort of unique user identifier, firstname: firstname of the user) in a script (Content-Type: text/javascript) which executes another JavaScript function (“callSignInComplete”), which sends the user to “/gallery/storagestatus.jsp”, which then ultimately redirects (HTTP 302) to “/gallery/creativeapps/photoPicker/albums.jsp”.
>>Here’s some interesting information on how this lazy JavaScript loading works: http://ajaxpatterns.org/On-Demand_Javascript
The site uses JavaScript and AJAX extensively to request resources and to present the response. For example, the album list (the images it uses to show the links to individual albums) and the pictures within the albums (once you click on an album) are retrieved asynchronously using XMLHttpRequest object.

Complimentary to AJAX, it uses JSON to exchange the data. For example, the request for the list of albums returns a JSON response with the album list and the details. Here’s the HTTP request headers for the album list and as you can see (x-request and x-requested-with are custom HTTP headers used by the app):

"GET /site/rest/v1.0/albumList HTTP/1.1\r\n"
"Accept-Language:
en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"Connection:
Keep-Alive\r\n"
"Accept: application/json\r\n"
"x-request:
JSON\r\n"
"x-requested-with: XMLHttpRequest\r\n"

And here’s a part of json response:

{"AlbumList":{"TOS":{"storageSize":"11111111","storageStartDate":
"2010-05-05T08:54:54.676-07:00","tosTotalTransactionsAmt":"0",
"tosStatus":"4","tosComplianceDate":"2010-08-27T00:00:00.000-07:00",
"warningZone":"false"},"Album":[{"id":"12355"...

It implements the services using REST architecture. Representational State Transfer (REST) is an architectural style to expose services on the web and you can read more about it online…but to a LoadRunner scripter, XML services exposed RESTfully are no different than any other XML based service over HTTP. Some more information about REST:
It includes some tracing cookies and requests to third-party sites that can be safely ignored and commented out.

So with all that information, it was fairly easy to visualize the script’s high-level steps:

Navigate to home page & login.
Send a REST-style request to get the list of Albums (name, URI etc)
For each Album in the list
1. Get the name of the Album and create a corresponding folder on the local disk
2. Send a request to get the list of all photos in the album
3. For each photo
  1. Send a request to download the image file
  2. save the image file in the album folder
Logout

Step 1:

The 1st step is to navigate to the “welcome.jsp” page which returns the session cookies to be used throughout the session. I deleted all the extra requests in the script for images, css files etc. Next step is to login and to do that, we need to send a request to login.jsp with a random 9-digit number and also the username and password in the “ssoCookies” cookie. All other session cookies are automatically handled by LR.

web_url("welcome.jsp",
 "URL=http://www.kodakgallery.com/gallery/welcome.jsp",
 ...
 LAST);

web_add_cookie("ssoCookies=%7B%22email%22%3A%22abc%40example.com%22%2C%22password%22%3A%22test%22%7D; DOMAIN=www.kodakgallery.com");

web_url("login.jsp",
 "URL=https://www.kodakgallery.com/gallery/account/login.jsp?&uid={randnum}",
 ...
 LAST);

Step 2:

Once login is successful, we need to get a list of albums that the user has. This is done by sending a GET request to this REST-style URL: “http://www.kodakgallery.com/site/rest/v1.0/albumList” directly. We’ll also need to save the response body in a parameter that we’ll have to parse to get the album details.

web_reg_save_param("albumList", "LB=", "RB=", "Search=Body", LAST);
web_url("albumList",
 "URL=http://www.kodakgallery.com/site/rest/v1.0/albumList",
 ...
 LAST);

Now here’s the beauty of how the service has been implemented. When you visit the website through a browser, the actual response to this request is returned in JSON formatted string. But we can actually send the request in such a way that it returns the album list in XML rather than JSON. All we have to do is to not include the “Accept: application/json” header, instead just send “Accept: */*”. And since it’s easier to use LR’s built-in XML functions to parse XML strings, we do exactly that. LR’s web_url() function sends “Accept: */*” by default so we get an XML response with the album list.

Step 3:

So once we have the Album List in XML, I use lr_xml_get_values() to get the id of all the albums in the list.

numAlbums = lr_xml_get_values("Xml={albumList}", "Query=/AlbumList/Album/id","SelectAll=yes", "ValueParam=albumId", LAST);

It returns the number of matches that match the XPath Query, which is the number of albums the user has. This parameter “albumId” holds all these ids and will be used to get the list of all the photos in an album.

Step 3.1:

Now for each of these albums, we get the id from the parameter “albumId” and then get the name of the album using lr_xml_get_values again and using the id in the XPath. Then we go ahead and create the directory as specified.

    for(j=1;j<=numAlbums;j++){
     sprintf(sfx,"{albumId_%d}", j);
     lr_save_string(lr_eval_string(sfx), "aid");

     lr_xml_get_values("Xml={albumList}", "Query=/AlbumList/Album[id='{aid}']/name","ValueParam=albumName","NotFound=Continue",LAST);

     sprintf(dname,"%s\\%s",baseDir,lr_eval_string("{albumName}"));
     if (mkdir(dname)) {    //works, but need better error handling
         lr_output_message("Create directory %s failed", dname);

         return -1;        }

Step 3.2:

With the album Id, we send another GET request to the URL: “http://www.kodakgallery.com/site/rest/v1.0/album/{aid}”. Just like we did above for the album list, we save the response body which is a list of all the photos in this album.

        //---get album details
     web_reg_save_param("albumDetails", "LB=", "RB=", "Search=Body", LAST);
     web_url("albumDetails",
         "URL=http://www.kodakgallery.com/site/rest/v1.0/album/{aid}",
         ...
         LAST);

     numPics = lr_xml_get_values("Xml={albumDetails}", "Query=/Album/pictures/photoUriFullResJpeg","SelectAll=yes", "ValueParam=fullResURI", LAST);

And again, just like we did above, we user lr_xml_get_values() to get the URIs to the full resolution picture. It returns the number of pictures and the URIs in a parameter.

Step 3.3:

Now for each of the pictures, we get the URI to full-resolution image from the parameter (lines 3-4 below). We need to get the filename which is returned in the “Content-Disposition” HTTP header (line 7) and also, we have to save the whole body (binary image data) in a parameter (line 9) that we can later use to store on the local disk.

   1:         //get all photos in the album   2:         for (i=1;i<=numPics;i++){   3:             sprintf(sfx,"{fullResURI_%d}", i);   4:             lr_save_string(lr_eval_string(sfx), "uri");   5:     6:             //save the file name that's part of Content-Disposition header   7:             web_reg_save_param("filename", "LB=Content-Disposition: attachment;filename=", "RB=\r\n", "Search=Headers", LAST);   8:             //save the whole HTTP body of request   9:             web_reg_save_param("body", "LB=", "RB=", "Search=Body", LAST);  10:    11:             web_url("FS",  12:                 "URL={uri}",  13:                 "TargetFrame=",  14:                 "Resource=1",  15:                 "RecContentType=image/jpeg",  16:                 "Referer=",  17:             LAST);  18:    19:             lr_eval_string_ext("{body}",strlen("{body}"), &buf, &prmLen, 0, 0, -1);

Then we send a GET request to the URI which returns us the full-resolution image. But since this is a dynamically generated response (“Transfer-Encoding: chunked”), the server doesn’t return the size of the file in the Content-Length HTTP header which we could have used to write the contents in a file. Instead, we have to use lr_eval_string_ext() (line 19) to save the value in a buffer and to get the length of the buffer.

Now we have everything to save the final file: name, size and its contents. We use standard C file handling functions for that and finally free the memory by using lr_eval_string_ext_free().

            sprintf(fname, "%s\\%s", dname, lr_eval_string("{filename}"));
         if ((file = fopen(fname, "wb" )) == NULL) {
             lr_output_message("Unable to create %s", fname);
             return -1;
          }
         fwrite(buf,prmLen, 1, file);
         fclose(file);
         lr_eval_string_ext_free(&buf);

The code then loops to save each image in each album locally.

Step 4:

The last step is to logout which is just another request to logout.jsp with a random number. It goes through similar steps as the login and finally redirects back to the home page.

Epilogue:

This was a great challenge and if you actually made it this far, I hope you enjoyed reading it and trying it yourself. I got to learn a lot from this and I hope you do too. Specifically, I learned about REST, JSON and about JavaScript lazy loading as well as HTTP chunked transfer. I also learned a little bit more about looking into LoadRunner recording logs, saving the HTTP response in a file and the XML functions in LR.

As I noted earlier, if you went through all this just looking for an automated way to download your pictures from KodakGallery, please leave me a comment and I’ll work on a Perl script to automate that.

LoadRunner, Memory Violations, lr_eval_string_ext and Pointers (ANSI C Style)

2010-03-03T10:15:00.001-08:00

I think it’ll be quite accurate to say that an average programmer like me is daunted when first faced with the concept of pointers and memory management in C. During the initial programming years (much of which inevitably had to be in C), I tried my best to avoid using pointers in my code. Whether it be using character arrays with pre-defined size (who cares if it takes much more memory than is needed) or some other “nifty” trick… I thought I could get away as long as I can compile and run just “this” program. But I had to face it during a networking class in school when I worked on a peer-to-peer file sharing project and one of my classmates convinced me to “do it right” and pay heed to the requirement of the program being able to work with other students’ code.

So after much head-scratching and soul-searching, I begrudgingly revisited the concepts and began (or so I thought) to grasp the idea of address spaces. l-values vs. r-values and pointers. It all seemed to make sense and fit in place like a jigsaw puzzle finally coming together. And not before long, with some encouragement and confidence building, I rewrote the program using pointers instead of pre-allocated char arrays and making it as compatible as possible. But if it was supposed to make me happy and content, it didn’t last long and whatever feeling of competence I felt was shattered to bits when I compiled the program for the first time and got compile time errors that didn’t make any sense at all. It was even worse and torturous when after a few cycles of modifying and debugging, it finally compiled beautifully (oh the elation…) and then the first time I ran it, it threw a slew of memory violation errors and crashed as beautifully as it compiled.

Years have passed since then…some of which were spent in writing code in other languages (it doesn’t have pointers? I’ll take 10). And some others in trying to come to terms with how even after understanding the concepts if I now have to write a C program and I decide to (or have to) use pointers, the error messages still baffle the heck out of me. Brief moments of competence have existed…when after writing, re-writing, debugging, debugging again…debugging a few more times, I was finally able to turn out a decent piece of code that could accomplish what it was supposed to in a relatively efficient manner.

Background:

So how does all this relate to the subject of this post? A few days ago, I was writing a LoadRunner script for a web application which had an inquiry page that submitted a request. Depending on the data submitted, the request returned either a response page with the final result or a set of intermediate questions that needed to be answered. After submitting the answers, it again returned either another set of questions or the final result. Once the answers were submitted the second time, it returned the final result page. The number of questions returned was not constant, however 8 questions was the maximum. So a part of scenario logic was this:

a. Submit the initial inquiry
b. If Question Set A is returned, determine the number of questions, construct an answer string and submit
c. If Question Set B is returned, determine the number of questions, construct an answer string and submit
d. Final response

Problem:

Since the questions returned were in a select box, it was easy to find the left and right boundaries and use web_reg_save_param with "Ord=All". In this case, since the questions were in the form:

<SELECT NAME="Answer2" SIZE="5">
  <OPTION value="0"> ABC</OPTION><br>
  <OPTION value="1"> DEF</OPTION><br>
  <OPTION value="2"> MTG</OPTION><br>
  <OPTION value="3"> SVG</OPTION><br>
  <OPTION value="4">NONE</OPTION><br>
</SELECT>

it will be:

web_reg_save_param ("suffix", "LB=<SELECT NAME=\"Answer", "RB=\" SIZE", 
"Ord=All","NOTFOUND=Warning", LAST);

However, it wasn't as easy as determining the number of matches from {suffix_count} and looping to create a custom answer string. Each question had a relative order from 1 – 8 like the one above (the numeral after “Answer”) and the answer string had to be constructed based on that. The challenge was that the number of questions was variable and the order didn’t always start with 1 and go 1, 2, 3…and so on. So if 4 questions were returned, they could be labeled Answer2, Answer3, Answer5, Answer6 and based on this, the answer string would be answer2=2&answer3=2&answer5=2&answer6=2 (it didn’t matter if the questions were answered correctly, so a constant 2 would do). If I submitted answer1=2… here, it would return an error saying “Invalid Response” or something like that.

Solution:

So what I had to do was to save that suffix number in a temp variable and construct the answer string by concatenating it together. So, something like:

c = atoi(lr_eval_string("{suffix_count}"));
if(c>0){
  for(i=1;i<=c;i++){
    strcat(answerString, "Answer");
    sprintf(sfx, "{suffix_%d}", i);
    strcat(answerString, lr_eval_string(sfx));
    strcat(answerString, "=2&");
  }
  lr_save_string(answerString, "aString");
  ...

And then I would create a web_custom_request and submit it:

web_custom_request("AnswerSetA",
  …
  …
  …
  "Body={aString}submit1.x=61&submit1.y=27",
"LAST");

My first instinct, as I’ve mentioned earlier was to use character arrays for both answerString and sfx.

char answerString[256], sfx[10];

I figured that the chances of the string being longer than 256 chars was remote so I was safe. And it worked fine when I ran it in VuGen. But when I ran the load scenario, all the users belonging to this script’s group failed exactly after 4th or 5^th iteration. The error was something I hadn’t seen before:

Action.c(162): Error (-17991): Failed to add item to mfifo data structure.

on the line with lr_eval_string. I searched online and came across this link (http://www.sqaforums.com/showflat.php?Number=420958) which suggested using lr_eval_string_ext instead of lr_eval_string to free memory earlier. The help on lr_eval_string also mentions:

Note: lr_eval_string allocates memory internally. The memory is freed at the end of each iteration. If you evaluate a parameter or parameters in a loop, conserve memory by not using lr_eval_string . Instead, use lr_eval_string_ext and free the memory in each loop iteration with lr_eval_string_ext_free.

I changed the code to:

c = atoi(lr_eval_string("{suffix_count}"));
if(c>0){
  for(i=1;i<=c;i++){
    strcat(answerString, "Answer");
    sprintf(sfx, "{suffix_%d}", i);
    lr_eval_string_ext(sfx,strlen(sfx), &sfx1, &prmLen, 0, 0, -1);
    strcat(answerString, sfx1);
    strcat(answerString, "=2&");
    lr_eval_string_ext_free(&sfx1);
  }
  lr_save_string(answerString, "aString");
  ...

but to my disappointment, it still threw an error when executing in Controller after 4^th iteration. The good part was that the error message was familiar, the bad part was that it was a memory violation exception:

Action.c(168): Error: C interpreter run time error: Action.c (168): Error -- memory violation : Exception ACCESS_VIOLATION

For some reason, I felt that all the years of avoiding or trying to avoid using pointers obligated me to get to the root of the issue this time and fix it instead of coming up with a workaround. The actual issue however turned out to be something else and I’ll come back to it later. First order of the day…use char pointers instead of static arrays to manage the strings.

To start:

char *answerString; //instead of char answerString[256]
char sfx[10]; //this still can be an array

Next step was to figure out how much memory will I need to allocate based on the number of questions returned and then allocate it. This was done using malloc:

//we need AnswerX=2& times the number of questions, + 1 for '\0' 
if((answerString = (char *)malloc(c * 10 * sizeof(char) + 1)) == NULL){
  lr_output_message("Insufficient Memory!!");
  return -1;
}

Also, we have to initialize it because there may be some garbage in the allocated space that may hinder the proper functioning of strcat:

*answerString = '\0';

Now we have something very similar to a brand new char array, but only of the exact size that we need. Next, we create the string exactly as above, and I used lr_eval_string instead of lr_eval_string_ext because I honestly didn’t think that was the issue. After creating the string, I null-terminated it.

for(i=1;i<=c;i++){ 
  strcat(answerString, "Answer"); 
  sprintf(sfx, "{suffix_%d}", i); 
  strcat(answerString, lr_eval_string(sfx)); 
  strcat(answerString, "=2&"); 
} 
answerString[c * 10 * sizeof(char)] = '\0'; 
lr_save_string(answerString, "aString"); 
free(answerString);

And the best part, after saving the string in a parameter, I free the associated memory and relish my guilt-free existence (at least in terms of this script). The script worked like a charm, not only through VuGen but multiple iterations through the scenario in Controller.

So what was the issue with using character array: the issue was not that LR agents were running out of memory because I had used 256 bytes when I actually only needed less than that. The issue was that I was not emptying the array before using it. I had declared it within the Action itself:

Action()
{
  int i,c;
  char answerString[256], sfx[10];

and I wrongly assumed that LoadRunner throws away variables from previous iteration and initializes brand new variables in every new iteration. Instead, what happened as in this case when I used strcat was that it was concatenating the new answer string from this iteration to whatever was left from the previous iteration. So after a few iterations, it ran out of the pre-allocated 256 bytes of space and threw the memory violation exception. I could’ve continued to use char array (and I’m glad I didn’t) by just re-initializing it in every iteration.

So lesson learnt. Hopefully all this helps somebody not make the same mistakes I made. I certainly won’t and I will also be less hesitant in using pointers. Even though I’m pretty sure this is not the last I’ve seen of memory violation exceptions, I can say that I’ll be ready to learn something new next time that happens.

By the way…that peer-to-peer file sharing application, I finally was able to compile it and make it work using a mix of pointers and character arrays. It worked great and I felt satisfied when I completed it. But of course when I was demonstrating it to the TA, it didn’t function as expected and I later found out that it was because I had forgotten to null-terminate a string.

Parrot AR.Drone – iPhone controlled flying experience

2010-01-14T15:58:00.001-08:00

A colleague of mine forwarded me this and I was immediately impressed: AR.Drone is a iPhone/iPod Touch controlled (via wi-fi) helicopter that you can not only just fly around but also play augmented reality games with.

Check out the video below and others on YouTube.

2 new blogs to follow

2010-01-06T16:32:00.001-08:00

There are 2 new technology/technical blogs that I’m following.

- The Daily WTF (http://thedailywtf.com/): contains some really funny real-life situations, some of them too ridiculous to not make you go wtf?

- Digital Inspiration (http://www.labnol.org/): informative posts about new and interesting technologies/applications etc.

QTP Automation: A Generic Function to perform mouse click

2009-09-25T18:07:00.001-07:00

For all the automation testing I’ve been doing in recent months, I’ve built a set of function libraries in QTP that contain functions doing a lot of different things from clicking on a button, validating properties of different web objects to parsing data in excel files. One of these libraries contains all the WebBrowser related functions and one of those is BrowserButtonClick. It takes the title of the browser and name/index of the button as input and does the obvious – clicks on the button within the specified browser. It does all that through descriptive programming, not needing to have any objects in the GUI repository and gives me the advantage of reusing this library framework in different projects/applications. This has allowed me to automate functional tests of different applications very quickly.

But this post is not about that function or about the framework. I’ll write about that later - how it all fits together to provide an automation framework that can speed up functional automation. In this post, I want to share another QTP VBScript function that I wrote to perform a mouse click on any web object. This function forms the basis of descriptive programming that I’ve used in my libraries and provides greater flexibility instead of tying it down to a specific object type.

Like I described, the BrowserButtonClick takes the name of the button as parameter. This works fine as long as the button object has the name property defined (which most of them do) and I know it when creating the tests. For one of the application page however, there were 2 buttons with no name specified but with “html id” property. This led me to come up with another function that will take the property names (“html id” or “name” etc) and property values as input parameters, search for a button object with specified properties and perform the action (left button click) on that. But thinking about it a little more, since the class of the object (“micclass”) is also another property of the object, it doesn’t have to be restricted to a “WebButton”. So the end result was this generic function that takes the list of appropriately delimited properties and values, finds the object and performs a left-click on the object. The input parameter is in the format “propName1=propValue1;propName2=propValue2…”. For example, “micclass=WebButton;html id=Search”. This way, it can be used to perform a click on any object and similar functions can be used to perform any action on any GUI object.

Here’s the function.

Part 1: Function Definition

' Function BrowserObjectClick
' ------------------
' Perform a mouse click on an object within the specified Browser
' Parameter: browserTitle - Title of the browser
' Parameter: props - a list of name-value pairs of properties and their values, semi-colon delimted. ("propName1=propValue1;propName2=propValue2…”)
' Parameter: objIndex - the index (0-based) of the object that needs to be clicked (if multiple objects match)
'@Description Perform a mouse click on the object that matches specified properties within the specified Browser. Returns 0 is successful, -1 if no object is found or the object is disabled
'@Documentation Perform a mouse click on object that matches <props> properties within <browserTitle> browser
Public Function BrowserObjectClick(ByVal browserTitle, ByVal props, ByVal objIndex)

As I already explained above, the function performs a mouse click on any object that matches the specified properties within the specified browser. The browser title needs to be provided as input parameter and so do the properties, which are name-value pairs delimited by semi-colon. For example, to find a WebButton object with name “Search”, the input will be “micclass=WebButton;name=Search”. For a link with outertext property “Log Off, the input will be “micclass=Link;outertext=Log Off”. If multiple objects match the identification properties, objIndex (0-based) will specify which one of those objects is clicked.

Part 2: Getting the browser objects

    On Error Resume Next
    If (props <> "") Then
        indx = CInt(objIndex)
        If (Err <> 0 Or indx < 0)Then
            indx = 0
            Err.Clear
        End If

        'Get the browser objects
        Set desc = Description.Create()
        desc("micclass").Value = "Browser"
        Set bObjs = Desktop.ChildObjects(desc)
    
        For i = 0 To bObjs.Count -1
            If Instr(1, bObjs(i).GetROProperty("title"), browserTitle, 1) > 0 Then

First we make sure that the properties are not blank in the input parameters. If they are blank, there’s not much we can do so we just exit out of the function. Next thing to do is to make sure the objIndex is an integer >=0. In case it isn’t, we take the default value of 0, which means that the first object matching the properties will be clicked. Then we get all existing browser objects and iterate through them until we find the one with matching title. If none are found, we’ll exit out with appropriate return values. If multiple browsers are found, we’ll use the 1st one that matches the title. In the applications I’ve automated so far, I haven’t come across a case where multiple browsers match the title…but if needed (in case the application pops up multiple windows, for example), the function can be easily modified to handle that.

Part 3: Getting the Object and performing the Action

                desc("micclass").Value = "Page"
                Set pObjs = bObjs(i).ChildObjects(desc)
                'Make sure you got one
                If (pObjs.Count > 0) Then
                    'load the properties in an array
                    Dim propArray : propArray = Split(props,";",-1,vbTextCompare)
                    Dim prop
                    Dim propsDict    'As Scripting.Dictionary
                    Set propsDict = CreateObject("Scripting.Dictionary")
                    For Each prop in propArray
                        propsDict.Add Split(prop,"=",-1,vbTextCompare)(0), Split(prop,"=",-1,vbTextCompare)(1)
                    Next
                    
                    'Get the Object
                    Dim j, pKeys : pKeys = propsDict.Keys
                    For j = 0 to propsDict.Count-1
                        desc(pKeys(j)).Value = propsDict.Item(pKeys(j))
                    Next
                    Set propsDict = Nothing
                    Set objs = pObjs(0).ChildObjects(desc)

So once we find the browser with specified title, we get its “Page” child objects since all other objects are child objects of the page object. We split the properties that we got as input by the semi-colon “;” and load it in an array. At this point, the array is populated with the name value pairs of the properties delimted by an equals sign “=”. I’m not doing much error handling here because I trust the input provided will be appropriately delimited.

Next, we split each of the array values by an equals sign “=” and load it in a “Scripting.Dictionary” object as key-value pairs, where the key is the name of the property. We create a Description object, load all the properties in it and find all matching child objects.

Part 4: Performing action on the object

                    If (objs.Count > indx) Then
                        'Execute the event
                        If (objs(indx).GetROProperty("disabled")) Then
                            BrowserObjectClick = -1
                        Else
                            objs(indx).Click
                            BrowserObjectClick = 0
                        End If
                        'Free up the objects
                        Set desc = Nothing
                        Set bObjs = Nothing
                        Set pObjs = Nothing
                        Set objs = Nothing
                        Exit Function
                    End If

Now that we have a collection of all objects that match the properties, we’ll make sure that the index is not more than the number of objects returned, that it is not disabled and then perform the click action. If it is disabled, we’ll return a –1 so that caller can handle it appropriately. If not, we perform the mouse click by calling the click method of the object and return a 0. Finally, we do some cleanup and exit function.

PostScript

This function provides a generic method to implement a mouse-click on any object using descriptive programming and is particularly useful where the standard properties of an object (name) are not available. It can be further extended to implement any action, not just clicks and provide other functionality as well.

If you have any questions or suggestions to improve or simplify the function, let me know.

Real Life Testing Scenario – Bank ATMs

2009-08-14T10:41:00.001-07:00

I think a lot of real life defects you come across would have to do with Bank ATMs. Maybe because of complexity of coding the human-machine interface logic, the plethora of usage scenarios and/or real-time nature of the transactions that it is really hard to make a defect free (or almost) ATM machine.

Yesterday I faced one such scenario when trying to deposit some cash in an ATM machine. This was one of those BoFA ATMs that don’t need an envelope to deposit. You just put the cash in a slot in the ATM and it does the rest. It scans and counts the bill and tells you the total amount deposited.

So as I put all the cash in the slot, it closed the slot door and scanned and counted all the bills. But after it was done, It gave me a message on the screen that some of the bills couldn’t be accepted and opened the slot door for me to take the bills out. There was only 1 bill there and all others were accepted. I took out the one bill it couldn’t accept and it closed the slot door and showed the amount that it had accepted correctly. Pretty neat…so far.

On the same screen, it asked whether I wanted to add more money to the deposit. I pressed “Cash” and it opened the slot door again. After pressing and straightening the bill it had rejected previously. I put that in the slot. It didn’t accept the bill this time either. It gave me the error message and opened the slot door. I took the bill out and it closed the door. But…the error screen didn’t go away. It continued beeping and complaining that it couldn’t accept the bill even after it had closed the slot door. There were no options available…cancel the transaction, return the card…nothing. Just the error message and no buttons to select. At this point, my card was inside the ATM and so was the cash and I had no proof of the deposit! I didn’t have the customer service number because it is printed on the back of the card.

Well, after fruitlessly pressing random buttons including the cancel button and including keeping the cancel button pressed for sometime, I gave up. I approached the lady at next ATM slowly lest she thought I’m going to mug her. But she probably saw the ordeal I was going through and gave me the customer service number from the back of her card. And after spending about 40 minutes on the phone during which I faced numerous call transfers and 1 disconnect and had to talk to the amazingly annoying automated voice response system, I got to talk to a real person who seemed to understand what I was going through and gave me a temporary credit pending research. The previous card was rejected and a new card will be sent.

So…here are the steps:
1. Deposit Card in the ATM slot and go through entering the PIN etc.
2. Select “Deposit”…and “Cash”
3. Once the cash slot door opens, enter several bills at least 1 of which will definitely be rejected.
4. After it complains about the bills, take the bill(s) out.
5. Once the slot door closes and it counts and shows the amount, press the option to add more money to the current deposit.
6. After the slot door opens, add the rejected bill(s)
7. After it shows the error screen again and opens the slot door, take out the rejected bill.

The expected result at this point is for the machine to detect that I have taken the rejected bill out, close the slot door and continue with the amount that was previously determined. It should show me the options again that were shown at Step 5 and let me choose whether I want to try adding more cash again or just go ahead with whatever amount that has been added already. No matter how many times I add previously rejected bills, it should either accept the bills or reject them. This seems to be a normal usage scenario to me, which should’ve been tested thoroughly. This kind of scenario is very likely to happen and it’s not something a user would have to try real hard to go through. I honestly doubt that this issue can be replicated again by following the exact same steps. There has to be some other variable which caused the machine to malfunction.

If I had to test the scenario, there are several things I would want to know:
Firstly, how does the bill scanning logic works. How is it decided whether a bill should be accepted or rejected. Once I know that, I can test with several combinations of different amounts, with different combinations of good and bad bills and different degree of bad quality bills. Maybe also with junk (coins? fake bills? other paper? leaves? lint?)instead of bills!
Secondly, how does it detect whether the bills have been added or removed from the slot to close the slot door. Based on this, I can test with different number of bills to validate if it detects the change.
There are a lot of other exception scenarios also that come to mind…for example, what if somebody props up the slot door open by jamming something through the door? What if somebody continues to add rejected bills 5 times? 10 times? 100 times? (of course, not all the scenarios are realistic so the likelihood of the scenario would have to be weighed against its criticality).

It seems that in this case, it did recognize that I had taken the rejected bills out because it closed the slot door. This is assuming that it closes the slot door only after it detects that the bills have been removed. If it closes the door after a fixed time (I hope not), then this statement is invalid. Anyways, assuming it detected the change, it seems to have failed to continue to the next step which is to carry on with the transaction and provide the options to the user to choose what to do next. This is exactly where I would look to identify the problem by running various scenarios or a combination of scenarios.

12 Balls Solution

2009-06-22T10:45:00.001-07:00

Here’s the solution to the 12 Balls puzzle that I mentioned a few days (gosh…a few weeks now) ago . To state it again:

You have 12 identical-looking balls. One of these balls has a different weight from all the others. You also have a two-pan balance for comparing weights. Using the balance in the smallest number of times possible, determine which ball has the unique weight, and also determine whether it is heavier or lighter than the others.

The third branch following the 1st decision is left blank because it is very similar to the 1st one. The larger image file is available here.

Closed for Cleaning!

2009-06-19T15:24:00.001-07:00

I wish somebody would pay me to create an application that would detect that the work restrooms are closed for cleaning and update all employees automatically. We probably have 2 times during the day when they are being cleaned and somehow my calls are synchronized with that time.

Here’s how that application would work at a distant high-level:

1. Have a camera pointed at the restroom door that monitors the door for appearance of a yellow sign put between the door edges
-or-
(slightly cheaper) Install an infra-red monitor that is flipped whenever the yellow sign is put between the door edges blocking the monitor
-or-
(even cheaper and less automated) have the janitor flip a switch whenever they start the process.

2. The trigger chosen above should send a message to Microsoft Office Communicator server (or any other instant messaging application) where a resource for each of the restrooms is setup (just like the meeting rooms). That should turn the status of the bathroom to “Busy” or “Unavailable”.

This way, each interested person (with synchronized bladders) can add the restrooms to their contact list and find out if they are available without getting up.

12 Balls

2009-05-28T11:07:00.001-07:00

Wednesday morning, out of sheer impulse I navigated to Willy Wu’s riddles site consisting of the best puzzle compilation on internet (I added the site to the links section as well). The puzzle that was staring at me was the 12 Balls one:

You have 12 identical-looking balls. One of these balls has a different weight from all the others. You also have a two-pan balance for comparing weights. Using the balance in the smallest number of times possible, determine which ball has the unique weight, and also determine whether it is heavier or lighter than the others.

I figured the solution once I was able to find time to put some random thoughts to this puzzle. I’m currently working on creating a visio diagram to display the solution but if you’re not familiar with the puzzle and feel challenged enough to attempt a stab at it without searching online for a solution, feel free.

If after putting some time into it, you’re feeling frustrated and just want the solution, go ahead and google. If you just want some hint(s) and not the complete solution, let me know and I’ll post it before I post the solution.

Canon EOS 5D Mark II

2009-05-27T15:49:00.001-07:00

Link to a video shot entirely with 5D, a DSLR that supports capturing full HD movies: http://www.usa.canon.com/dlc/controller?act=GetArticleAct&articleID=2326

Really impressive.

QuickTest Pro: Handling XMLs using XMLDOM object

2009-01-30T16:17:00.001-08:00

This is how diverse my writings are going to be. Testing tools, that’s it. I could write about my adventures in photography but that wouldn’t be very interesting until I put some more time and money into it. Even though I’ve been using my 50mm f/1.8 lens with some interesting results, I still feel that there’s lots more I need to learn before I can write about it …maybe someday soon. But this one is about a different, though as interesting topic - handling XML objects in QTP.

Background:

I’m currently involved in test automation for IDM application. If you are familiar with Sun’s IDM product, you know that all objects are stored in database as XML and these objects can be viewed/checked out/edited through the debug interface. Now the way we have customized it is that managers (users with Manager role) or end users are allowed to submit requests for creating users or modifying users’ attributes. The requests are submitted through the user interface and the workflows take care of taking appropriate actions in the background. There is very little information displayed on the web page once the request is submitted that can be used to validate if the action actually succeeded or not. The only way to validate that the action was successful is to either login through admin interface and check the task results or through debug pages, pull up the user’s XML object and verify the changes have gone through.

As I was creating the BPT components and test cases to handle different test scenarios, it was increasingly obvious to me that the most foolproof way to validate the results of the test would be to create a component that validates the user’s XML. I wasn’t much familiar with handling XML objects in VBScript so I left it for later. And later is now (or was yesterday). So I spent yesterday going through the XML Document Object Model (DOM) and Microsoft’s implementation of it and creating a VBScript function to validate an XML’s attribute value. It worked out fine with a little initial struggle.

Please keep in mind that what I’m providing below is in no way a complete solution of using XMLDOM to do whatever you want to do. All I needed to do was to retrieve an attribute value from an XML using the XPath that is provided as input parameter and to compare it with the expected value. What you want do with XMLDOM may be different and based on your application’s and automation needs. The reason I’m posting this is that if you’re using XMLDOM with QTP/VBScript for the first time, you can avoid the initial struggle that I went through and get the job done faster.

For Microsoft DOM reference (this is where I got most of the information I needed), visit this: http://msdn.microsoft.com/en-us/library/ms764730(VS.85).aspx. The XPath language is described at: http://www.w3.org/TR/xpath

Solution:

1. The first step is to create a parser object and load the XML. The XML can be from a file or from a string. Here I’m using the loadXML() method to load it from the innertext property of a WebElement object that I retrieved earlier. To use a file, use the load() method instead.

Set xmlDoc = CreateObject("Microsoft.XMLDOM")
xmlDoc.async = false
xmlDoc.loadXML(objs(0).GetROProperty("innertext"))

You set the async property to false so that it doesn’t move on before the document is completely loaded. The reason I’m using CreateObject(“Microsoft.XMLDOM”) instead of XMLUtil.CreateXML() which is provided in QTP OM Reference is that when I used that, it gave me an error specifying that it couldn’t find the DTD file referenced in the XML. It seems that it needs the DTD if it’s mentioned in XML to be able to load it. I had the DTD and when I uploaded it to my local component folder, it worked fine. But since I could be running the test on a remote host, I didn’t want to have to upload that DTD to all my host machines. And I didn’t look into any other way. But if you decided to use the XMLData object provided by QTP, the objects and method names are different even though the overall steps will be the same. For example, to load the document, you’ll use:

Set xmlDoc = XMLUtil.CreateXML()
xmlDoc.load(objs(0).GetROProperty("innertext"))

2. Once you have the XML object, you need to get to the desired element. I kept it simple and asked for the XPath to the desired element as an input parameter. Once I have that, I used the selectNodes() method to get all the nodes in the desired XPath.

Set objNodes = xmlDoc.selectNodes(nodeXPath)

This gives me a collection of all the nodes (specifically, the IXMLDOMNodeList object) that match the specified parameter ‘nodeXPath’. With QTP XMLData, you can use:

Set objNodes = xmlDoc.ChildElementsByPath(nodeXPath)

If nodeXPath is invalid or doesn’t match any elements, the length of the collection will be 0.

3. So now, I need to check the length of the collection and get to the actual element that I want. Since I didn’t need to care if multiple elements match the XPath, I just took the first element in the collection. The item property returns a single node (IXMLDOMNode) from the collection specified by the index, in this case 0.

If objNodes.length > 0 Then
Set objNode = objNodes.item(0)

4. Now that I have the actual element, I need to get the value of specified attribute. The getAttribute(name) method returns the value of the ‘name’ attribute. The attribute name and its expected value are passed as input parameters:

If StrComp(attrValue, objNode.getAttribute(attrName), vbTextCompare) = 0 Then
'expected value matches actual value

attrValue is the input parameter containing the expected value and attrName is the attribute name. If it matches, I write a Pass in the results and if not, it’s a fail. Once done, I clean up the objects.

Set xmlDoc = Nothing
Set objNodes = Nothing
Set objNode = Nothing

That’s pretty much it. Once we start using the component actively, I may come up with some more ideas to improve it. I’ll write about it if they provide me enough of a challenge.

----------------------------------------------------------------------------------

Sample XML:

<User name="abcdefg" creator="Configurator" email="a.a@x.com" disabled="false" locked="false">
<Roles>
<ObjectRef name="Employee" isWeak="false"/>
</Roles>
<Attribute name="employmenttype" type="string" value="Employee" syntax="string"/>
<Attribute name="firstname" type="string" value="a" syntax="string"/>
<Attribute name="fullname" type="string" value="a, a" syntax="string"/>
<Attribute name="ismanager" type="string" value="true" syntax="string"/>
<Attribute name="lastname" type="string" value="a" syntax="string"/>
<Attribute name="middlename" type="string" value="M" syntax="string"/>
<Attribute name="phone" type="string" value="9999999999" syntax="string"/>
<AdminRoles>
<ObjectRef type="AdminRole" name="Manager" isWeak="false"/>
</AdminRoles>
</User>

For example, the XPath to get to <Attribute> with name=employmenttype is: User/Attribute[@name='employmenttype’]

LoadRunner: Creating manual web_custom_request for HTTP file uploads

2008-10-08T19:21:00.001-07:00

There was a recent forum post in Advanced LoadRunner yahoo group asking how LoadRunner captures the file upload requests and if there was a way to customize that request. At that time, I replied that it may be possible to generate a manual web_custom_request by using the underlying raw HTTP request. I hadn't tried it then but it sounded interesting enough to be marked as to-do. I recently got some time to try it and it was fun.

Problem:

As mentioned in the forum post, LoadRunner captures any form based file uploads as web_submit_data with the file name (and location) within the list of data within the request. An example request:

web_submit_data("FileUpload",
        "Action={URL}",
        "Method=POST",
        "EncType=multipart/form-data",
        "TargetFrame=",
        "RecContentType=text/html",
        "Mode=HTML",
        ITEMDATA,
        "Name=File", "Value=C:\\testdata\\readme1.txt", "File=yes", ENDITEM,
        LAST);

There are 2 obvious drawbacks to this:

1. For this kind of script, the files to be uploaded have to be transferred to each of the Load Generator machines being used in the scenario. Or they have to be transferred to a shared network location so that both the machine used for creating the script and the load generator machines can reference it.

2. As mentioned in the forum post, any parameterization in the file contents is not possible. If the scenario requires files to be uploaded iteratively with unique names, that many files have to be created manually.

Solution:

There are 2 solutions to this and one is more universally acceptable (by web servers) than the other. I'll describe both and let you decide which one you want to use. I also highly recommend reading RFC1867 which provides excellent background on how multipart file uploads are implemented in HTML form based submissions.

1. web_custom_request POST: To arrive at the solution, I did exactly what I usually do when I'm stuck with some HTML script or want to see the raw HTTP request being sent to any web server. I opened Webscarab to capture the HTTP request that was being sent for file uploads. (Webscarab related post here).

Here's a screenshot of the HTTP post request as captured by Webscarab:

And here's the relevant portions of the raw request:

Content-Type: multipart/form-data; boundary=---------------------------327572003712859
Content-length: 228
-----------------------------327572003712859
Content-Disposition: form-data; name="File"; filename="readme1.txt"
Content-Type: text/plain
testdata
readme
readme
123456789
-----------------------------327572003712859--

The Content-Type HTTP header is what specifies the submission to be a multipart data submission. The "boundary" is a string that doesn't occur in the file data and is used to mark the boundaries of the data being sent. Content-length, as the name implies is the length of the data that is being sent.

The body of the request starts with the boundary string and is followed by the 2 headers to specify the content-disposition which includes the name of the file to be uploaded. Content-type specifies the encoding of the data being submitted. Following that within the body is a blank line and the actual contents of the file that will be uploaded. After the contents is the boundary once again to signify end of the submission data.

So this kinda makes it easier to create a web_custom_request. The Content-Type HTTP header is specified by "EncType" attribute of the function. Content-length header can be ignored since web_custom_request itself generates it by calculating the length of the data at runtime. The rest goes in the "Body" attribute starting with the boundary. Content-Disposition specifies among other things the name of the file to be uploaded and this can be parameterized. Content-Type is the type of file being uploaded and can be text/plain, image/gif etc. And in the end is the boundary once again. The final request looks like this:

web_custom_request("{rndString}.txt",
        "URL={testURL}",
        "Method=POST",
        "EncType=multipart/form-data; boundary=---------------------------17773322701763",
        "TargetFrame=",
        "Resource=1",
        "RecContentType=application/octet-stream",
        "Body=-----------------------------17773322701763\r\n"
        "Content-Disposition: form-data; name=\"File\"; filename=\"{rndString}.txt\"\r\n"
        "Content-Type: text/plain\r\n"
        "\r\n"
        "testdata\r\n"
        "readme\r\n"
        "readme\r\n"
        "{rndString}\r\n"
        "-----------------------------17773322701763--\r\n",
        LAST);

I have parameterized the file name to be a random string so I can run this script iteratively without having to worry about server rejecting duplicate files. I can also parameterize the contents of the file within the body to whatever I like so that each file is unique or based on some other logic.

2. web_custom_request PUT: The second option is more straightforward but less universally accepted. I've read that not all web servers implement this. It involves using the HTTP PUT to create a web_custom_request. For more on different HTTP methods, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html. It mentions that:

The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource.

I've also read that PUT is more efficient so if your test site implements the uploads using POST, better do it that way instead. Nevertheless, here is the web_custom_request with PUT:

web_custom_request("{rndString}.txt",
        "URL={URL}/{rndString}.txt",
        "Method=PUT",
        "Resource=1",
        "RecContentType=application/octet-stream",
        "Body=testdata\r\n"
        "readme\r\n"
        "readme\r\n"
        "{rndString}\r\n",
        LAST);

The resulting function is straightforward, URL attribute is the URI identifying the file to be uploaded and Body is the exact contents of the file.

Of Code Reviewers and Food Critics

2008-08-08T10:41:00.000-07:00

For last 2-3 months, I was involved in testing our in-house customizations of Sun's IdM product. My responsibilities included not only leading the QA effort for the customizations but also to support the development effort by reviewing and suggesting improvements to the code/design.

So engrossed in my newfound glory as a unit tester/code reviewer, I realized that it may be easy for me to look at the code and find out a problem with it. But if I had to write something like that myself, I'd end up spending much more of my mind and time than it usually would take an average programmer. That realization reminded me of something I had recently heard in a movie.

The movie was "Ratatouille" and the scene was when Anton Ego starts writing his critique after visiting Gustaeu's Restaurant and having a meal cooked by the new but unknown chef Remy:

In many ways, the work of a critic is easy. We risk very little yet enjoy a position over those who offer up their work and their selves to our judgment. We thrive on negative criticism, which is fun to write and to read. But the bitter truth we critics must face, is that in the grand scheme of things, the average piece of junk is more meaningful than our criticism designating it so.

The parallels between a food critic as described above and what I was doing seemed surrealistically close, so much that I had to finish this post (after realizing it had been lying in my drafts for a few weeks). In some ways, that was exactly what I was doing. I enjoyed a position over those who wrote the code because finding out a problem with a piece of code drew much more attention than the original act of writing it. I thrived on negative criticism because it would get immediate credit, instead of the work of a developer which was the reason my role existed in the first place.

There are some fundamental differences as well. One being that Anton Ego, as a food critic does not have to care about the success of the Restaurant he is reviewing. This affords him the luxury of being overtly aggressive and have a dismissive attitude. My role on the other hand (both as a tester and code reviewer), is as responsible for the success of the product as any other stakeholder. That is the reason I'm as much pained when I find an issue in some code that I previously have reviewed and may have overlooked as the developer with whom I have to revisit the information once again. But the basic responsibility of both is still the same: to challenge the creator of an artifact to produce a better product which aligns with or exceeds the expectations of the users/consumers.

----------------------------------------------------------------------------------

The project is nearing its completion now and in hindsight, this is what I have to say:

This was a dream project for me. The kind of project that comes like once in a few years and offers amazing challenges with tremendous learning opportunities. Right kind of people, right kind of responsibilities and right kind of control over what I wanted to do. As much as I enjoyed working every moment in this project, I have to realize that in my enthusiasm, I may have stepped over some boundaries. I may have offered some unwarranted advices that were probably irrelevant or I may have criticized something to hide my lack of competence. I will take full responsibility for all the misgivings I caused, in however form I may deserve. But I do hope it will be realized that most of whatever I did was aimed at the common cause of project's success, and the fact that I enjoyed every bit of it and tried to make it enjoyable for everyone else should make it all the more worthwhile.

Calculating Virtual User Ramp up and Pacing for Load Tests

2008-04-09T08:27:00.000-07:00

I was working on performance testing for a project not long ago that had different VUser types each with different load requirements. This meant that in one load test scenario, there were several user groups having unique transactional load characteristics (like desired TPM), not based on the whole scenario but by individual groups. At that time I created a simple spreadsheet to calculate the ramp-ups, pacing and actual load for each of the group and totals given the expected load and the number of virtual users. I recently got some time to improve it a little and am sharing it now.

So here's that table with some sample values:

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	30	100	1	30	2.00	2	200	30.00
2	34.2	14	1	34.2	1.75	2	28	30.00
3	24.6	14	1	24.6	2.44	2	28	30.00
4	19.8	10	1	19.8	3.03	3	30	20.00
5	1.5	1	1	1.5	40.00	40	40	1.50
6	9.3	5	1	9.3	6.45	6	30	10.00
7	6.9	3	1	6.9	8.70	9	27	6.67
8	34.2	14	1	34.2	1.75	2	28	30.00
9	24.6	14	1	24.6	2.44	2	28	30.00
Total	185.1	175		185.1				188.17

A cleaner Google Spreadsheet read-only version is available here;
and an excel spreadsheet for your own use can be downloaded from here

How to use this table:

Basically, this table represents one load test scenario. So each of the row entry is a separate VUser group that is included in this scenario. Under VUser Group Column, the name of user group can be entered. If less or more VUser groups are needed to include in test scenario, rows can be removed or added accordingly.

There are 3 values that need to be entered for each user group: the base load that is desired for that user group, the number of virtual users that are to be used, and a multiplier (which in most cases can be 1). These are explained below:

Base Load: This is the load (in transactions per minute) at which that particular user group will be run. This is a user provided value based on actual requirements of the load test. The Total in last row gives the total load (in TPM) for this whole scenario. The total may or may not mean much but it does provide the total TPM at a glance.

Number of Virtual Users: This is the number of virtual users that will be used to run that user group. This is also a user provided value. Determining this will be a little trickier and there will have to be some estimation involved if you have a small number of VU licenses available (as I do). You will need the estimated time taken by a complete iteration of the transaction represented by this VUser group. Once you have that, you'll have to use enough virtual users to keep VU Pacing of this group above that number. Actually, you should use more Virtual users than that because in a load test, transaction time will vary depending on how stressed the test servers are and may be higher than the times you see when running a single user.

Let me use an example:
For my user group 1, I need to generate a load of 30TPM and I know by running the script with a single user that it takes around 10 seconds to complete one iteration. I want to keep the pacing much above 10 seconds, say at least 60 seconds. Its not difficult to see that I'll need to use 30 users to achieve a pacing of 60 seconds. I can just put 30 in the Number of VU Column and find out what the ramp up and pacing values will be.

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	30	30	1	30	2.00	2	60	30.00

Also, if more virtual users are available, they can be used based on the requirements and/or a desire to increase the VU pacing even more. So if I want to use 100 users for this user group, the numbers will be:

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	30	100	1	30	2.00	2	200	30.00

On the other hand, if I have only 15 users available for this group, the values will be:

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	30	15	1	30	2.00	2	30	30.00

Now to achieve 30TPM with 15 users, I will have to use a pacing of 30 seconds which is much closer to 10 seconds. But in real life with limited number of virtual users available, I may have to go with these values and hope that the iterations don't take more than 30 seconds during the test.

There is another thing to keep in mind: With this table, the values will be accurate if the expected load for each user group is less than 60. If it is more, the expected ramp up will be less than a second and it will be rounded off to 0 or 1. If it is zero, you'll get a division by zero error in Actual Load Column and if it is 1, you'll get the maximum Actual Load of 60.

So the thing to do if you want to calculate the values for loads above 60TPM is to break the group into 2 different groups. For example, to generate the values for 90TPM load, you can break it up into 2 groups of 60TPM and 30TPM. That way, you can generate 90TPM accurately. Other thing that can be done is to ramp up more than 1 user per interval. So with 90TPM, the expected ramp up will be 0.67 per user which means 3 users in 2 second ramp up scheme. This calculation will have to be done manually as the spreadsheet doesn't take care of fractional ramp up values and rounds them to nearest integer value. (see more of this under Actual Ramp Up)

Multiplier (Load Factor): This column solely exists because I had to run different tests at different loads (for example, a stress test that was run at 5 times the load of general load test) and I wanted to quickly find out how many users will be needed at different load factors and what will be their ramp up and pacing values. For example, if I had a user group with 1.5TPM load and 1 user with a pacing of 40 seconds and wanted to find out how many users will be needed to have the same pacing but with 5 times the load, I can use a Load Factor of 5 to find out that I'll need 5 users to maintain a pacing of 40 seconds, as shown in table below:

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	1.5	5	5	7.5	8.00	8	40	7.50

So based on these 3 values entered for each user group, the remaining values, namely Ramp up time, Pacing and Actual TPM achieved are calculated.

Expected Load (TPM): This is calculated by simply multiplying base load with the load factor.

Expected Load (TPM): This is the theoretical ramp up time that is required to achieve the expected load and is calculated by dividing 60 by expected load. For example, if expected load is 30TPM, the ramp of course will be 60/30 = 2 users per second.

Actual Ramp up: Since the theoretical ramp up calculated in previous column can be a fractional value (indicating millisecond ramp up times) and we can have actual ramp up times only at a 1-second granularity, the actual ramp up is calculated by rounding off the expected ramp up to nearest integer. For example, if the expected load is 18TPM, the expected ramp up will be 3.33 seconds. This has to be rounded off to 3 seconds. This has the obvious effect of making the Actual Load different than the desired load.

Another case to watch out for, as I mentioned earlier is when the expected load is above 60TPM. In this case, the expected ramp up time is always going to be less than 1 and the actual ramp up will get rounded up to 0 or 1. The maximum Actual Load will be 60 as I'm not accounting for ramp ups greater than 1 user per second.

However, for both the above cases, there are 2 ways you can achieve the actual load that is closer to the desired load:

1. Break the group into multiple groups: For example, a 90TPM group can be broken into 2 groups - one with desired load of 60TPM and other with 30TPM. This way, an actual total TPM of 90 can be achieved.

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	60	60	1	60	1.00	1	60	60.00
2	30	30	1	30	2.00	2	60	30.00

In case of 18TPM, it can be broken into 2 groups of 15 and 3TPM. The values will be:

VUser Group	Base Load (TPM)	Number of VU	Multiplier (Load Factor)	Expected Load (TPM)	Expected Ramp Up (seconds)	Actual Ramp Up (seconds)	VU Pacing (seconds)	Actual Load (TPM)
1	15	30	1	15	4.00	4	120	15.00
2	3	6	1	3	20.00	20	120	3.00

2. More users per second ramp-up: Another way is to have multiple users per ramp up interval. For example, 90TPM with a single user group can be achieved by having a ramp up of 3 users every 2 seconds. Similarly, 18TPM can be achieved by having a ramp up of 3 users every 10 seconds.

I usually prefer the 1st method and choose the start time so that the ramp ups are even between the multiple groups of same user type. However it does make it more tedious as I have more user groups to keep account of. The 2nd method has the drawback of introducing a relative burst of users every ramp up interval.

VU Pacing: This is the interval between iterations and is calculated by multiplying the ramp up value with the number of users. Again, you have to make sure that the pacing value is comfortably above the single user transaction time; otherwise if the transactions take longer than the pacing value and next iteration is started immediately after the previous iteration, it may effect the load pattern.

Also, I'm assuming that you are pretty much content with having a stable pacing. If you want to add certain randomness to the the pacing times, that can also be done using this value. See my previous post for more on this here.

Actual Load: Finally, the Actual load is calculated by dividing 60 by the actual ramp up value. This is the actual load that you can hope to generate during your load test for this user group. What is actually generated during the load test depends on a lot of other factors like how stressed your system is etc. This can be found out by analyzing the results of the load test.

So that's that. It turned out to be a longer post than I expected, but hopefully it helps. If you have any comments or think of any suggestions to improve it, let me know.

Rubik Timer

2008-03-21T16:25:00.000-07:00

I'm currently working on writing a Java based Rubik Timer. A Rubik timer is a specialized timer that can be used to track the time it takes to solve a Rubik's Cube. There are some implementations available currently but the ones that I tried lacked some much needed features. And since the current projects I'm working on are not proving challenging enough, I needed something else to keep my life exciting.

Even though I've written some code already and am faced with some design decisions, it came to my mind that it'll be a good opportunity to put myself in different shoes - as a requirements/business analyst and document the requirements, as a programmer/designer and design/develop the solution, as a tester and test the implemented solution and finally as a user and use the application. Inextricably entangled with all these roles will be a project management role that I will have throughout the project. So I plan to follow and document the project as it goes through its own lifecycle, in an attempt to learn more about different phases that a project goes through. In the process, I also want to understand the roles of all these different actors and the challenges they face. I know it is easier said than done and at times, it may seem like an undiscerning adventure. But I want to at least give it a try and even though the project is smaller in terms of effort required as compared to real life ones, I believe that it'll prove to be an enjoyable and learning experience. And the fact that I'll be playing all those roles will make it as challenging as (or even more than) those I face in real-life.

In upcoming related posts, I plan to detail the progress as I find time to work on this project. As it seems, resource availability will be the toughest challenge this project faces since my current projects, even though minimally challenging can be substantially time consuming and resource (in this case, my mind!) draining. And every now and then, smaller challenges have a tendency to pop-up and provide exciting distractions.

Business Case for Testing Tools & Automation

2007-12-19T14:16:00.001-08:00

This post had been lying in my drafts for a long time. For some reason, I decided not to publish it. But it's still relevant...except that I have submitted the business case since then and it's lying at somebody at Finance's desk or email Inbox for review/approvals.

----------------------------------------------------------------------------------

I'm currently working on a business case to invest in testing tools and functional test automation. The case will be sent to upper management and finance for approval and once they do, we're going to start working on building an automation framework. I foresee a lot of hurdles and am not keeping my hopes high based on the tightening of budget and how the business case is coming out to be.

One of the things that I have to add in the business case is financial analysis which includes an estimate on return of investment. And it has been a very "enlightening" process. Based on the current estimates on the capital and labor costs, it'll take us 2.9 years to get a return on investment. And of course, it's based on a lot of assumptions and I'm kind of skeptic when claiming that number for ROI.

You see, one of things that are never in short supply at my workplace is the list outstanding tasks. And with that comes extreme work pressure and very little time to invest in exploring new technologies and coming up with new ideas. I'm quite sure that designing the automation framework will be a time consuming and learning process and I will have to be devoted at least half-time (if not more) in this effort. And I'm quite sure that when we actually start working on that effort, the estimates that I put in for labor in the business case will start looking like best case scenario.

The brighter side is that I have now under my belt the experience of creating a business case. And once/if approved, we'll be undertaking the effort of functionally automating the test cases for our applications. We're also planning to expand our QA service by offering the testing tools and/or automation to other teams.

My colleague and I had been talking about the design of the framework since we first conceptualized the idea of automation. We had this whole idea in our minds on how to make it reusable, to maximize customizability with least rework. He has experience in building these kind of frameworks and I have had a brief encounter at my last job. I recently looked up online and found that it is now one of the established practices, one of the buzzwords surrounding test automation process - Keyword Driven Test Automation Frameworks. It must have been a long time since both of us were in automation business because what I read matched what we had been planning to do. Seems like a lot of people have written their experiences with building automation frameworks with this idea. Seems like we'll be able to use some of the knowledge to our benefit.

Coming up...

2007-12-19T14:08:00.001-08:00

A few weeks ago, I spent some time going over JMeter and building some advanced web test plans. But then I got distracted and buried under some intense projects, including a 1-week training on a product that I didn't know much about and that I have to be an advanced user of within a few weeks. (BTW...if you want to know why attending a 1-week advanced training on a product without having a solid background is a bad idea, let me know).

Now I have some time again but it's right before my vacation and I just don't feel like getting immersed into it again. But when I come back, I plan on posting some of my notes on how to build an advanced web test plan in JMeter. I plan on using a more advanced scenario than the one with basic Get requests, probably something similar to my previous post about using an MD5 library to create a LoadRunner script. If I can't find something else, I'll use that very example. I also plan on exploring some webservices and pure XML over HTTP kind of scripts. Stay tuned...

Oh... and I should also mention my reasons for looking into JMeter. It has been increasingly frustrating to borrow time on LoadRunner Controller from an external team, partially because of having to justify to the project team the additional charge back incurred to the project. I can't believe how many emails I have to write to justify a few 100 dollars. But anyway, turning misery into a learning experience, I wanted to explore if JMeter provides enough capabilities and is robust enough to be used for production performance testing.

LoadRunner Scripting Challenge - LiveJournal

2007-10-29T16:51:00.001-07:00

My last few posts have unabashedly been about LoadRunner even though I vowed to diversify my writings into more distinct areas. But this one again is about LoadRunner and is too interesting to pass...

Background:

I had an old journal in LiveJournal back from the times when 'blogging' wasn't a very familiar word or activity and most of the popular blogging sites like Blogger and WordPress (these are the only free ones that I've liked so far) didn't exist. I came across LiveJournal and was immediately hooked. I wrote prolifically at that time and I should also mention that my creative writing abilities then were much superior to my current showcase. But over time, I've outgrown LiveJournal because their free version doesn't provide the most needed features and the one that does is ad-supported. To get rid of the ads, you have to pay a nominal amount. I think that wasn't the only reason and I was a paid member for sometime supporting their efforts. But I thought I needed a more traditional 'blog' rather than a journal. I also haven't been writing much lately and have no communities that I would like to keep track of in LiveJournal.

So the result was some of my most cherished writings were stored in LiveJournal's history of posts and I wanted to import all of them to a WordPress blog. WordPress allows importing an XML file of LiveJournal formatted entries. The problem however is that LiveJournal allows exporting the posts only by month. So if I wanted to export my posts from back in 2002, I would have to export it month by month for over 6 years worth of posts. That was certainly not very exciting. But soon it became pretty exciting when I decided to use LoadRunner to export all my posts month by month in XML format and I learned something very interesting in the process - which is of course the whole idea of this post.

Challenge:

When I recorded the script by going to the export page and logging in, I noticed in the script that my password wasn't hard coded anywhere. I expected to find it with the request when I submitted the form containing my username and password. Instead, there were the "chal" field and the "response" field and the password field was blank.

web_submit_data("login.bml",
"Action=http://www.livejournal.com/login.bml?ret=1",
"Method=POST",
"RecContentType=text/html",
"Referer=http://www.livejournal.com/?returnto=/export.bml",
"Snapshot=t5.inf",
"Mode=HTML",
ITEMDATA,
"Name=returnto", "Value=/export.bml", ENDITEM,
"Name=mode", "Value=login", ENDITEM,
"Name=chal", "Value=c0:1193421600:2223:300:DsqGiIDCk0A4hs9sNsmH:c7f09b46d0ba2d82bb68945ea84532fc", ENDITEM,
"Name=response", "Value=b84434e3c2d193c4a1c345d7874a89a3", ENDITEM,
"Name=user", "Value=myusername", ENDITEM,
"Name=password", "Value=", ENDITEM,

Honestly, that was surprising to me. At that time, I couldn't come up with any explanation of how my user ID can be authenticated and successfully logged in without sending the password in form submission. I could smell the hint of another challenge. If you want to try it before reading on to get a better handle, try recording a script on LiveJournal (http://www.livejournal.com) by logging in and making it work.

Solution:

On thinking a little more and looking at the script again, I realized that they could be using JavaScript to generate the response which was some kind of hashed form of the password. On viewing the source of the initial page, I found a js file with this code:

var pass = pass_field.value;
var chal = chal_field.value;
var res = MD5(chal + MD5(pass));
resp_field.value = res;
pass_field.value = ""; // dont send clear-text password!

So on further investigation, I figured what was happening:

1. On initial navigation to any page with the login form, a challenge string is sent to the client. I guess it has to have some kind of expiration time because if you use it sometime later with the correct response, it'll return that the challenge has expired.

2. An event handler is registered with the submit button of the login form.

3. When the submit button is pressed, the event handler takes the field values including the hidden challenge string. It takes care of creating an MD5 digest of the challenge string concatenated with the MD5 digest of the password and clearing the actual password field value. This is the value that you see in the "response" form parameter in the code above. The values are then submitted and the server takes care of validating the response value.

This page explains this in high-level: http://blog.paranoidferret.com/index.php/2007/07/22/secure-authentication-without-ssl-using-javascript/

So to make the LR script work, I had to do 2 things:

1. Get the value of the challenge by correlating appropriately

2. Once I have the challenge string, create the 'response' value by getting the MD5 digest of the challenge string concatenated with digest of the password string.

The MD5 digest calculation is based on the MD5 Message Digest Algorithm by Ron Rivest of MIT. There was no way I was going to write an MD5 function myself. A quick search revealed some implementations of this algorithm. The one I used was from http://www.fourmilab.ch/md5/ because this was in C and saved me from including a lot of legalese. All I had to do was include the header and C file and use the MD5 functions. So this is the outline of final script. I have left out the details which should serve as an interesting exercise.

1. Firstly, get the MD5 digest of the password. This will later be used to concatenate to the challenge string.

MD5Init(&md5c);
MD5Update(&md5c,(unsigned char *)mesg,strlen(mesg));
MD5Final(signature,&md5c);

where mesg is the password string.

2. Correlate the script to save the challenge string at the appropriate place. This is the string that is sent by the server as a hidden form parameter of the login form.

3. Once you have the challenge string, concatenate the MD5 digest of the password (calculated in Step 1) to it. Remember that this digest has to be concatenated as a lower-case hexadecimal number. Get the MD5 digest of this concatenated string.

4. Pass this MD5 digest calculated in Step 3 as the value to 'response' parameter in the login step:

"Name=chal", "Value={chal}", ENDITEM,
"Name=response", "Value={response}", ENDITEM,

That should do it. Now I can parameterize the year and month values in export step and export all my journal entries by month in XML files. It was a simple task to concatenate all the files into 1 big XML file and importing it through WordPress blogging features. I'm a happy WordPress blog user now...we'll just have to see how long that lasts before I start looking for other options.

I guess I should also add that the exercise gave me a great opportunity to learn how web site authentication can be handled without using SSL and how I can script that in LoadRunner.

Note 1: Since creating this script, I explored LiveJournal's server protocol documentation and found that they have documented the authentication mechanism pretty well. Great work in encouraging other users to build custom clients.

Note 2: I felt I should mention that LoadRunner licensing agreement specifically prohibits using the software to run tests on public domain sites. It basically means that you cannot load up your LR Controller with the script you created above and run a load test with any number of virtual users.

Random Virtual User Pacing in LoadRunner

2007-10-09T16:26:00.001-07:00

I guess there's always a first time. I had never used LoadRunner's random virtual user (VU) pacing earlier, but am working on a project that will need to use this feature for the first time. And as I thought about it a little more, I may start using it more frequently now. Here's how it happened:

This is one of the rare projects that provided me with excellent documentation - not only system documentation like the system specs, API Guides etc but also performance requirements like actual production volume reports and capacity models that estimated the projected volumes.

The capacity models estimated the maximum transaction load by hour as well as by minute (max TPM). What I needed to do was take maximum hourly load, divide it by 60 to get a per minute transactional load and use this as the average TPM. The idea was to vary the VU pacing so that over the whole duration of test, average load stays at this Average TPM but it also reaches the Max TPM randomly.

For example, if the maximum hourly transaction rate is 720 requests and maximum TPM is 20, the average TPM will be 720/60 = 12 and I will need to vary the pacing so that the load varies between 4TPM and 20TPM and averages to around 12TPM.

The Calculation:

To vary the transactional load, I knew I had to vary the VU Pacing randomly. Taking above example, I had to achieve 12TPM and I knew the transactions were taking around 1-2 seconds to complete. So I could have the pacing of around 120 seconds if I needed to generate a fixed load of 12TPM with a 5 second Ramp-up and 24 users.

Script	TPM	Number of VUs	Pacing (sec)	Ramp Up
Script 1	12	24	120	1 VU/5sec

So now to vary the TPM to x with the same 24 virtual users, I will need to have a pacing of 24*60/x. I got this from an old-fashioned logic which goes in my head this way:

24 users with a pacing of 60 seconds generate a TPM of 24
24 users with a pacing of 120 seconds generate a TPM of 24 * 60/120
24 users with a pacing of x seconds generate a TPM of 24 * 60/x

So using above formula, to vary the load from 20 to 4TPM I will need to vary the VU pacing from 72 to 360. So now we have:

Script	TPM	Number of VUs	Pacing (sec)	Ramp Up
Script 1	4 to 20	24	Random (72 to 360)	1 VU/5sec

Of course, there's a caveat. The range of 72 to 360 seconds has an arithmetic mean of 216. 120 is actually the harmonic mean of the 2 numbers. So the actual variation in TPM will depend on the distribution of random numbers that LoadRunner generates within the given range. If it generates the numbers with a uniform distribution around the arithmetic mean of the range, then we have a problem.

I ran a quick test to find this out. I created an LR script and used the rand() function to generate 1000 numbers between the range with the assumption that LR uses a similar function to generate the random pacing values.

int i;
srand(time(NULL));
for (i=0;i<1000;i++){
lr_output_message("%d\n", rand() % 289 + 72);
}

And of course, the average came out close to the arithmetic mean of 72 and 360, which is 216.

So with the assumption that the function used by LoadRunner for generating random pacing values generates numbers that are uniformly distributed around the arithmetic mean of the range, we'll need to modify the range of pacing values so that the arithmetic mean of the range gives us the arithmetic mean of the TPM that we want...phew. What it means is that the above pacing values need to be modified from 72 to 360 (arithmetic mean = 216) to 72 to 168 (arithmetic mean = 120). However, this gives us the TPM range of 20 to 8.6 TPM with a harmonic mean of 12TPM.

But I'll live with it. I would rather have the average load stay around 12TPM. So here are the new values. Note the asterisk on TPM. I need to mention in the test plan that the actual TPM will vary from 8.6 to 20TPM with an average of 12TPM.

Script	TPM*	Number of VUs	Pacing (sec)	Ramp Up
Script 1	4 to 20	24	Random (72 to 168)	1 VU/5sec

TextPad Regular Expression

2007-09-26T11:08:00.001-07:00

\x20+,\|,\x20+

If you use LoadRunner for creating load test scripts and TextPad to view/modify its parameters file AND you get a file with 3000 test records to be used for parameterizing the script with blank spaces before and after the comma delimiter that need to be trimmed, this is the regular expression to use in TextPad to search for all such occurrences and replacing them with a comma instead.

Why I'll continue to recommend and use LoadRunner instead of eLoad

2007-07-24T16:56:00.000-07:00

I've raised this concern in meetings with Empirix and haven't heard this as being a priority. However, this is a major reason why I'll continue to recommend and use LoadRunner instead of eLoad:

Lack of ability to generate specified transactional load in eLoad and why it's very important

Background:
Web based (3 and n-tier) applications are different from client-server types of applications in terms of how they deal with user load (specifically how the system performs when users use the system) as explained below:

Load in web-based applications (3 or n-tier) is generally represented in terms of number of transactions expected per unit time (seconds/minutes etc) and not in terms of users per unit time. This is because in web/application systems, a user using the system can generate varying amount of load on the system depending on how active the user is. For example, a single user who submits 10 transactions every second will use much more system resources than 100 users who submit 1 transaction every minute (given the 'transaction' as defined is equal in both cases, and let's rule out caching as well). Even though those 100 users in latter case are using some fixed amount of resources on the system (for maintaining session information, other objects etc) that is 100 times more than that being use by the single user, significant resources are only consumed when the users are actively interacting with the system or waiting for response from the system.

So if while planning performance tests for an application, I get business requirements stating that X users are expected to use the system in a day, I have to work with them to further refine these requirements to specify what is the user activity profile (or scenario profile) and what is the frequency of their actions. For example, how many users will logon every hour/minute, how many will navigate to certain web pages or consume a service every hour/minute and how many will logoff every hour/minute. For simple scenarios, a transaction can be 1 to 3 steps - logon, stepA and logoff. Based on answers from previous questions, I will then refine the requirements to state how many transactions are expected to be executed every minute (or TPM). For complex scenarios, multiple transaction types may have to be defined - usertype1(stepA, stepB...stepX), usertype2(stepB...stepY) etc. In this case as well, I will have to use same questions to refine the requirements for each transaction type, i.e., for each transaction type, what is the expected transactional load in TPM or TPS (transactions per second). Then I'll go ahead and create the load test scenarios based on these transactional loads.

This kind of transaction based load specification is even more important in XML over HTTP/web services applications because there is no concept of user. Rather the system deals with requests (or transactions) that can come from a third-party web application, a custom client etc. that are generally referred to as service consumers. There is rarely a need to maintain session information and each transaction is idempotent (at least from the web/application server perspective). Note that the whole transaction may be non-idempotent but still be made up of a series of idempotent transactions. For example, a credit check service may be non-idempotent because it writes the number of inquiries to the user's credit profile and gives it a weight in calculating the user's credit score. But from the web and application server perspective and especially in test environments where limited test data is available, each transaction can be considered idempotent since we don't care about test user's credit score and need to generate a production load using limited data.

How the 2 tools handle (or do not handle) this:
Coming back to the point of why I prefer to use LoadRunner over eLoad in these scenarios (there are other reasons too but let's just focus on this one for now)...
Both eLoad and LoadRunner let me specify the iteration delay (or VU pacing in LoadRunner terminology, don't confuse it with VU pacing in eLoad which actually means think time!) that controls how long a Virtual User (VU) waits before starting the next iteration. However there is only one way to specify this in eLoad - from the time previous iteration ends (see Figure 1). This creates a problem because the time when next iteration will be started depends on how long the previous transaction took. For example, if you specify this delay to be 30 seconds and the previous transaction took 30 seconds, next iteration will be started after the end of 30 + 30 = 60th second. However, if the previous transaction took 5 seconds to complete, next transaction will start after 5 + 30 = 35th second. Now suppose you want to generate a transactional load of 10 TPM. You use 10 virtual users and specify the delay to be 55 seconds expecting each transaction to take around 5 seconds. This way you can have each of the 10 VUs submitting 1 transaction every minute, thus generating a load of 10 TPM. But when you run the load, server (or the application under test) gets busy and takes 30 seconds or more to return a response. eLoad's virtual users are still going to wait 30 + 55 seconds before starting subsequent iterations, thus reducing the overall transactional load by almost 30% (or 10 * (1/85) * 60 = 7 TPM as compared to required 10 * (1/60) * 60 = 10 TPM). But you really need to find out how the system behaves at production load of 10 TPM even at busy periods...do the requests keep queuing up and ultimately cause the system to become unresponsive or the system returns to stable state soon after! Well...hard luck, because eLoad is going to decrease the load if the system starts taking longer to return the responses. You can possibly add more virtual users to the scenario to increase the load when this happens but I don't want to have to sit and watch the load test for this to happen when I'm running a 12 hour test and am only allowed to run the tests in non-business hours. And I'm not even sure if I'll be able to calculate that fast how many users to increase/decrease everytime this happens.

Figure 1: eLoad VU Settings

LoadRunner gives 3 options in setting the iteration delay/pacing (see Figure 2):
a) As soon as the previous iteration ends
b) After the previous iteration ends: with a fixed/random delay of XXX seconds (In case of random delay, it lets you specify a range)
c) At fixed/random intervals, every XXX seconds.

Figure 2: LoadRunner VU Settings

So the above situation is handled very easily by selecting the 3rd option and choosing a delay of 60 seconds. In the above example, if the previous iteration took 5 seconds, it'll wait for 55 seconds and if it took 30 second, it'll wait another 30 seconds before starting next iteration. No matter how long the previous iterations take (as long as they are less than 60 seconds), it will always start subsequent transactions at specified intervals of 60 seconds from the start of previous transactions. thus keeping the load stable at 10 TPM. If I expect the transactions to take longer than 60 seconds, I can start with more VUs and increase the delay. For example, I can use 20 VUs and set the delay to 120 seconds. This will still generate a load of 10 TPM if I specify the ramp-up time correctly.

Conclusion:
So my conclusion is that I will use LoadRunner as much as I'm able to. In case you're wondering why my company has 2 load testing tools when buying 1 is costly enough, my team is under a different business unit that owns eLoad licenses because LoadRunner was considered too expensive. However, there is another business unit that has LoadRunner licenses and even though I had to go through a lengthy procedure, I got them to agree on letting us use LoadRunner and charge us for the usage.

---------------------------------------

Note 1: You can argue that load in real-life production scenario is never stable. But when it comes to defining the system's performance, I prefer to use multiple scenarios with increasingly different loads. For example, if the business expects about 10,000 transactions per day, considering a 10 hour business day this comes to 16.67 TPM. I will run load tests at 16 TPM (1x), around 40 TPM (2.5x) and around 80 TPM (5x) to give them numbers on how the system can be expected to perform if the transactional load varies from 16 to 80 TPM. I will probably also run some stress tests by running a background load of stable 16 TPM and then submitting a batch of multiple requests (100/200 etc.) to see how the system recovers in this case. Again, this will depend on business requirements and expectations. Also, If I really need to vary the load, I would rather use the random option in LoadRunner and vary the load but still keep the overall load stable.

Note 2: I am not implying that defining load in terms of number of concurrent users is not important. For some applications (e.g., citrix or remote desktop applications) it is the most important load defining criteria. Even for web based applications, you may want to find out how many users you can concurrently support before the server runs out of memory. This will help you determine when you'll need to buy extra hardware. But any commercial load testing tool has to support the ability to generate transactional load as well now that XML and webservices are becoming more and more common.

Note 3: Current eLoad version that I'm using is 8.10 and LoadRunner is 8.1.4

WebScarab

2007-07-13T10:53:00.000-07:00

There have been times during LoadRunner scripting that I needed to see the low-level HTTP request that is being sent from my client (which I am using to record, e.g., a browser, or a custom client) to the server. Earlier, I used Ethereal (http://www.ethereal.com/) successfully but the problem is that it doesn't support SSL directly. So if the communication is over SSL, all I would see is encrypted data and there was no way to see the headers/data being transferred. This made me look for alternatives. I recently came across WebScarab and I wouldn't say it's free of bugs but I'm sticking with this tool for as far as I can see in future. Here's how I was able to solve some of the problems in LoadRunner scripting using this tool.

Solution 1: (SSL Intercept)
First things first, WebScarab proxy

is able to observe both HTTP and encrypted HTTPS traffic, by negotiating an SSL connection between WebScarab and the browser instead of simply connecting the browser to the server and allowing an encrypted stream to pass through it.

This is a major advantage. So I no longer have to hope that one of the test environments will not have SSL implemented and will let me observe the non-SSL HTTP traffic. Watching browser traffic was just as easy as starting the WebScrarab proxy and pointing the browser to the local proxy. It gives the options to intercept request and/or responses and lets you modify the requests in any way before passing them on the server. For a custom client, I can capture the exact headers being sent and add them to my web_custom_request:

    web_add_header("Cache-Control", "no-cache");
    web_add_header("SOAPAction", "\"\"");
    web_add_header("Accept-Encoding", "gzip, deflate");

And I can also copy the body if it's an HTTP XML post for example, to get the exact XML data being sent and put that in the body of the request:

    web_custom_request("SampleService",
        "URL={URL}",
        "Method=POST",
        "EncType=text/xml; charset=utf-8",
        "TargetFrame=",
        "Resource=0",
        "RecContentType=text/xml",
        "Mode=HTTP",
        "Body="…

and so on. See screenshot.

Solution 2: (Reverse Proxy/Act as a web server)
As it happened, the client I was using (for more details on this, see my previous post) had pre-configured options of selecting the URL. So I didn't have any way to point the client to the local WebScarab proxy. I looked through the help contents and found this:

WebScarab allows you to specify a "base address" for a Listener. The base address instructs the Listener to operate as a reverse proxy, and should be formatted as a HTTP or HTTPS URL. In this mode, it will act as a web server, rather than as a proxy server, and will construct the URL by concatenating the base URL and the path that appears in the request line. If the base URL is an HTTPS URL, it will immediately negotiate an SSL tunnel prior to trying to read the request from the browser. This is useful for the situation where you are using a custom HTTP-based client that does not support configuring an upstream proxy. Simply change the hosts file on the computer on which the custom client is running to point the site in question to the computer on which WebScarab is running on, and WebScarab will receive requests for the targeted website.

This meant that I could use the hosts file to point to the proxy and specify the base address in the proxy listener to intercept the requests. In a few tries, I was able to intercept the SSL requests over a non-local base address. Again, I could get the headers and body and use it in the web_custom_request. See screenshot.

Solution 3: (SSL Server Certificate)
Another client that I was recording my script against had a similar issue. The client didn't support pointing to an upstream proxy so I configured the hosts file to point to the listener proxy. However when running the client, it threw this exception:

{http://xml.apache.org/axis/}stackTrace:javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

I tried adding the WebScarab certificate to the keystore through Java Control Panel but no luck. After googling a little more, I came across this forum thread: http://forum.java.sun.com/thread.jspa?threadID=220329&tstart=165
Apparently, Java uses its default keystore and to use another keystore, it has to be created and provided in one of the arguments. So after importing the WebScarab certificates into IE trusted certificates and then exporting it into .cer, creating a keystore with the certificates and modifying the batch run file to add
-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword=

I had my fingers crossed when running the client again. But fortunately, this time it worked as expected and intercepted the HTTPS requests without any errors. Once again, I used the custom headers and the XML body to create LR web_custom_header request.

LoadRunner function: web_convert_param

2007-06-29T10:57:00.000-07:00

After spending about 6 hours trying to figure out a solution to the problem of parameterizing URL encoded strings in my LoadRunner script, I finally found the solution: a built-in function provided by LR.

The problem:
I recently recorded a script against an Java application that ultimately sends a HTTP POST request to the server being tested with an XML in the Body of the request. The script came out fine, except that the body (XML content) was URL encoded. It would have been fine if I didn't need to parameterize the data in the body but since I had to, I realized in a few seconds that I would have to do some extra work to convert the data from parameter file to URL encoded format. The data in parameter file is of the form:

Fname,Mname,Lname,Gen,SSN,A1,A2,A3,A4,A5,City,State,Zip
John,,Smith,,123456789,123,Main,St,#,88,TestCity,TestState,987654321

However, in the script, address is one field that is a concatenated string of A1 - A5 parameters with a space between each of them.

Further Research:
A1 - A5 are the components that make up the street address that is supposed to be concatenated to one field in the script. No problem.
sprintf(as,"%s %s %s %s %s", lr_eval_string("{a1}"), lr_eval_string("{a2}"), lr_eval_string("{a3}"), lr_eval_string("{a4}"), lr_eval_string("{a5}"));
lr_save_string(as, "addressStreet");

…but the problem is there are spaces in between this components, which we all know get converted to a '+' character in URL encoding. Ok, so I could've just used sprintf(as,"%s+%s+%s+%s+%s",…
But some of the addresses also have '#' signs that get converted into '%23'. Now I had 2 options:
1. remove all the addresses that have a '#' sign
2. write a URLEncode(char *) function that does the obvious.

1st option wasn't very tempting because however unlikely it may be, the addresses can have other characters that need to get URL encoded as well. And if a sizeable chunk of the data given to me has these characters, I could lose a lot of records. Also I didn't want to have to modify the data every time I get a new 10,000 record file.
2nd option seemed the way to go. But it was late Friday and I had to start the tests in a short time so I was lacking the much needed patience. Anyways, I ended up running my tests by removing the records with '#' character. Luckily, there were few.

Solution:
On Monday after running all the tests, I got back to writing the URLEncode function. After researching online on what the characters should get converted to and trying to find an already published function in C that I could tweak to my purpose, and stumbling on PHP, Java, JavaScript functions, I started getting a sense that LR may have some kind of built-in function that does that or something similar. I don't know why I hadn't thought of using Mercury Knowledgebase before that. So a simple search of "url encoding" brought up 3 articles - none of which seemed helpful. But reading through the 2nd one (KB Problem ID: 18880), I couldn't believe what I saw:
Pass the 'XMLSource' parameter as an input to the web_convert_param function, and store the result as 'TargetXML'
web_convert_param("TargetXML", "SourceString={XMLSource}", "SourceEncoding=HTML", "TargetEncoding=URL", LAST);

So this web_convert_param function seemed to do exactly what I needed. I tried it out with a couple of different strings and it did as promised. So my new script had:
sprintf(as,"%s %s %s %s %s", lr_eval_string("{a1}"), lr_eval_string("{a2}"), lr_eval_string("{a3}"), lr_eval_string("{a4}"), lr_eval_string("{a5}"));
lr_save_string(as, "addressStreet");

web_convert_param("encAddressStreet", "SourceString={addressStreet}", "SourceEncoding=PLAIN", "TargetEncoding=URL", LAST);
lr_output_message("%s", lr_eval_string("{encAddressStreet}"));

It took me only a few minutes to make that change. And I didn't need to worry about the '#' characters or any other unsafe characters in the address string. Phew…

Morale:
Use the Mercury Knowledgebase more often. If you think there should be any other, let me know.

Random Sync

Puzzler Solution: Prisoners and Hats and a Jungle

Dennis Ritchie

Puzzler: Prisoners and Hats and a Jungle, Oh My!

Friday Lunch: Content-Type, Content-Disposition HTTP Headers and MIME type handling in IE

Investigating with WebScarab

How does IE handle MIME types in a response, anyways?

GLaDOS is back…

C++ turns 25

A Generic Function to enhance BPT components

Using the EvalFunction function in components:

Using the EvalFunction component in BPT tests:

Epilogue:

HP Quality Center (ALM 11.0) and REST

Intercepting SSL traffic using WebScarab

Goodbye multiple putty windows!

LoadRunner Scripting Challenge – KodakGallery.com (AJAX, JSON, REST & XML)

Challenge:

Scripting:

Step 1:

Step 2:

Step 3:

Step 3.1:

Step 3.2:

Step 3.3:

LoadRunner, Memory Violations, lr_eval_string_ext and Pointers (ANSI C Style)

Background:

Problem:

Solution:

Parrot AR.Drone – iPhone controlled flying experience

2 new blogs to follow

QTP Automation: A Generic Function to perform mouse click

Part 1: Function Definition

Part 2: Getting the browser objects

Part 3: Getting the Object and performing the Action

Part 4: Performing action on the object

PostScript

Real Life Testing Scenario – Bank ATMs

12 Balls Solution

Closed for Cleaning!

12 Balls

Canon EOS 5D Mark II

QuickTest Pro: Handling XMLs using XMLDOM object

Background:

Solution:

LoadRunner: Creating manual web_custom_request for HTTP file uploads

Problem:

Solution:

Of Code Reviewers and Food Critics

Calculating Virtual User Ramp up and Pacing for Load Tests

VUser Group

Base Load (TPM)

Number of VU

Multiplier (Load Factor)

Expected Load (TPM)

Expected Ramp Up (seconds)

Actual Ramp Up (seconds)

VU Pacing (seconds)

Actual Load (TPM)

Total

185.1

175

185.1

188.17

How to use this table:

VUser Group

Base Load (TPM)

Number of VU

Multiplier (Load Factor)

Expected Load (TPM)

Expected Ramp Up (seconds)

Actual Ramp Up (seconds)

VU Pacing (seconds)

Actual Load (TPM)

VUser Group

Base Load (TPM)

Number of VU

Multiplier (Load Factor)

Expected Load (TPM)

Expected Ramp Up (seconds)